Rendering Shrek@Home?

JimCricket writes "There's an interesting piece at Download Aborted about using distributed computing (a la SETI@Home, Grid.org, etc.) in the film industry. With the recent release of Shrek 2, which required a massive amount of CPU time to complete, one must wonder why the film industry doesn't solicit help from their fans. I'd gladly trade some spare CPU time in exchange for the coolness of seeing a few frames of Shrek 3 rendered on my screensaver!"

Doubt it'll happen...

[Zweistein_42]

Security issues would be a concern I'm sure. There's plenty of hackers who'd see no harm in, for example, extracting a number of images from around the world and sticthing a trailer, etc. And of course, rendering is a "trial-and-error" process - would they want people to have access to broken scenes? Or deleted scenes? Speculation would seriously dampen their ability to control marketing and release info. On the technical side, farms are reliable and predictable. Who can figure out how many fans will keep their computers up tonight for the critical preview tomorrow? What about the decline of interest after first little while? Distributed computing of this sort isn't well suited for commercial projects with fixed schedules. Not that I don't think it'd be COOL... I just don't think it'll happen :-/

Difficult to police...

[Anonymous Coward]

Surely this would be a bit difficult to co-ordinate. You'd need someone checking every single frame of information that was returned, just to make sure there were no 'Tyler Durden' style additions to the movie!

copyright

[ciscoeng]

How would the legal aspect work out? Seems like you'd have to sign a fairly strict license saying the movie studio still owns what your computer rendered, copyright, etc.

Very cool idea nonetheless.

Would it be worth it????

[millahtime]

The film industry can afford it so...

Why would they want to do the distributed??? They are using 10Gbs etho and blow your mind away servers to render at amazingly high rates. Probubally several times faster than something like the SETI network could imagine.

And hell, those sysadmins have the most owerful systems in the world. Who would give that up? They even get whole new systems every couple years.

Re:I had this idea a long time ago :)

[DetrimentalFiend]

I beg to differ. I suspect that the reason why no one's ever bothered suggesting this is that the amount of bandwidth required to download the frame data and upload the rendered frame are prohibitively large. Besides that, the licensing costs for the rendering technology would be enormous, and what film company would want to freely distribute all of the models, textures, and animation that they spent dozens of man-years working on?

Slashdot BLOG advertising...

[Anonymous Coward]

Nice to see you can advertise your NEW BLOG on slashdot...

how much did it cost?

unschedulable resource

[marcsiry]

Films and other large productions are tightly scheduled, with costs against these schedules mapped out months in advance. I can't think of a producer who would count on an essentially unschedulable resource as a vital part of their production pipeline, regardless of its economy.

That said, I could totally see a use for a 'render pool' catering to independent filmmakers, students, and nonprofits for whom cheap is more important than timely.

reality used to be a friend of mine.

[Anonymous Coward]

In a world without lawyers and assholes, this would be wonderful. A whole movie could be rendered in a few short hours (double or triple checked, of course) if the planet was your farm.

But alas, reality sets in and one must realize how this will never ever ever ever work.

Ever.

do you really want this?

[jfroebe]

Do you really want the MPAA to run programs on your computer?

The reason why.....

[reality-bytes]

The main reason they don't employ this technique is that their own 'render-farms' are a known quantity; they can, with reasonable accuracy, calculate how long a given scene will take to render, whereas with public distributed computing this calculation is not possible.

There are many variables in distributed public computing such as:

*Different CPU capabilities.
*Different OS capabilities
*High/Low use Systems
*People's 'uptime'
*Users leaving the project before its completion etc.

Another risk is that another movie-house could start a production which everyone sees as 'cooler' and your entire userbase decides to up-sticks and render for them instead.

How cool would the other way be?

[jmpresto_78]

How cool would it be to see them allocate THEIR distributed system to projects like SETI, etc. Even though I'm sure there are other projects being worked on, one would imagine the system is pretty dormant after a release.

Do you realize just how many gigabytes of...

[exp(pi*sqrt(163))]

...data, nay terabytes of data, can go into a single frame in a movie? You might be able to farm out stuff like some fragments of procedurally rendered smoke that rely on computing noise functions repeatedly, rather than accessing a scene database, but in general this is completely impractical. If visual effects houses wish to share data the easiest thing to do is FedEx a bunch of hard drives. So unless Shrek@Home includes some kind of hard drive exchange program it ain't gonna work!

There's an I/O problem

[Bruce Perens]

At Pixar, distributed rendering, even within the same building, was sometimes I/O bound rather than compute

Why bother?

[hal2814]

It's hard enough to solve issues regarding parallel processing of images in a clustered environment they can control. Why put that process in an environment they can't control? It's not like movie studios can't afford a computer cluster. That's a small cost compared to the cost of hiring someone to write the distributed software they use.

From what I've read, Seti@Home works well because users heavily process a small amount of data and return a small solution. If we were processing frames, it would require the user to take in large amounts of data and return even larger results.

Re:I doubt it...

[millahtime]

Why go through the trouble?

The simple answer is to see if you can. It's /. so of course someone would try.

Duh

[gerardrj]

Because the production of a blockbuster movie tends to be kept a secret up until near the premier. distributed computing provides little to no security.

There's no way a studio could send a scene's model to a compute node encrypted, process it encrypted, store the interim image encrypted, then send the whole mess back encrypted. At some point in processing the information must be in plain computer processable formats.

What that boils down to is that a competing studio could sign up hundreds of compute nodes and get a preview of the story line and animation. Anyone who could gather enough images could piece together clips from the film and release them in full digital format. Imagine a nefarious group of nodes all collecting the images they generate and later piecing them all together in to perfect digital non-DRMed copy of the movie; before release and before the DVD is available.

Hollywood can't stand the idea of people copying DVDs to the internet, could you imagine what they'd think of full film resolution copies of their films floating around? The heads bits: on the walls.

No... this is just a stupid suggestion from the point of view of the studios. At least until there's and OS is produced where a user it prohibited access to certain portions of RAM, and can't intercept the network traffic to/from the box.

Re:Slashdot BLOG advertising...

[happyfrogcow]

that, and the question being obviously not a good idea makes me wonder why this is even an acceptable story.

if you want to spend your time rending frames of animations, check out the Internet Movie Project [imp.org]

Who said it would have to be frames?

[clifgriffin]

I think this is a neat idea and has potential, but it would not work or even be attempted in the way the poster suggested. (ie, screensaver / frames)

Rendering is essentially a gazillion calculations. Divide those calculations up among a few thousand Joe Dialups and you potentially have a nice render farm.

But don't fool yourself with this notion that any individual machine should do any individual frame. Not only would it take the gigabytes of models, textures, etc, it would take the average machine years to render just one frame from one of these movies. Am I the only one who watches the documentary portions of these films??

Clif

Re:Doubt it'll happen...

[joib]

the film studios I'm sure have crazy fiber/multi-gigabit interconnects within their rendering farms.


While the amount of data to move around probably is too much for dialup, gigabit ethernet is certainly fast enough, and dirt cheap as it's integrated on motherboards. If you look at the top500 list, you see that weta digital (the company which did the CG for lord of the rings IIRC) has a couple of clusters on the list, and they have gig ethernet.

Basically, while rendering is cpu-intensive it's not latency sensitive, so there's no point in blowing a huge amount of cash on a high end cluster interconnect.

Why it's appropriate for SETI and not for film

[bobhagopian]

Despite the coolness of the SETI project, the major reason I support SETI and other scientific projects (e.g., protein folding@home) is that they are notoriously underfunded. SETI and the organization which operates folding@home (Stanford?) do not make profit, and at each step have to literally beg the government (usually the NSF) for more grants. This is especially true of SETI, which has become a pretty out-of-fashion program in funding circles. In short, the whole point of donating CPU cycles is to allow somebody access to computing power that it would not otherwise have. While I enjoy the Shrek movies just as much as the other guy, I'm not so philanthropic when it comes to a company that's capable of making $128 million in one weekend. Here's an analogy: you might donate clothing to the Salvation Army, but would you donate to Sak's Fifth Avenue? I think not -- I suspect many of you, like me, would rather support the little guy with no alternative.

Umm...No

[retro128]

I could see this for an indie project, but no way for a feature film. The reason why, I think, is because that in order for a computer to start rendering a CGI frame, it must have several things: Geometry, textures, lighting algorithms, and any procedurals needed to make things like fur, hair, realistic water effects, etc. Now, if I were PDI, or any company that has spent millions in R&D in creating these things, do you think I would want this info on Joe Schmoe's computer just waiting to be opened up and reverse engineered? I don't think so.

ILM's render farm: The Death Star

[steveha]

There is a great article about how ILM does their rendering. It was a cover story in Linux Journal magazine.

http://www.linuxjournal.com/article.php?sid=6783 [linuxjournal.com]

People have been saying that even if the studio didn't care about the security issues, there are bandwidth issues that would keep this from really working. There are a few quotes in the article that confirm this: all the rendering machines make a sort of denial-of-service attack on their NFS servers, for example. And the article talks about their VPN, which they call the ILM Conduit; it sends everything double-encrypted with Blowfish. They really are worried about security.

The coolest thing, to me, is that ILM has rolled out Linux all the way across their organization; people run Linux on their desktop computers. When people go home at night, their computers get added to the render farm!

steveha

Render Times

[TexTex]

Awhile ago, John Lasseter of Pixar was in some promotional documentary for one of their films. He claimed that when they originally created their short film with the desklamp, render times were around 7 hours per frame.

He said that for Finding Nemo today, render times were about...7 hours per frame.

More machines and faster processors let you cram much more detail and technology into the same package. Working in commercial advertising, digital editing and graphic workstations are fantastic and powerful...but their advantage isn't speed. We spend the same amount of time making a commercial as 10 years ago...but now we make 7 versions and change it 30-some times along the way. Power gives you the ability to change your mind....and that's a creative force which people gladly pay for.

Re:I had this idea a long time ago :)

[YoJ]

I agree with most of the comments so far about why the idea wouldn't work directly, but I'm more optimistic about the general idea. For example, there is a technique called "partial abstract interpretation". The idea is that given code and the input data, one can see what the code would do on the input data and then change the code to not accept any input and do the correct thing on that particular given input. If the company distributed code in this way, it would just be code and no data (so their artwork doesn't leak out), and the code would only work to generate one scene; it would be hard or impossible to uninterpret the code (so they doen't leak their proprietary rendering technology).

Bandwidth.

[stephenisu]

The sole reason this will not work using current internet infrastructure is bandwidth.

In the making of Final Fantasy, it took longer to send the information to the nodes than it took the nodes to process it. That is with dedicated gigabit networking.

too many unknowns for money

[jonathanduty]

Remember, Dreamworks and Pixar are companies trying to produce a product that they will sell. Every day it takes to build that product is money down the drain. They do not want their development time-line to depend on how many of their "fans" currently have their machines on or what their Internet connection speed is or even how crappy of a computer they have and what other tasks it is running. Seems to me trying to plan an expensive project like a movie with that type of model would be a nightmare.

Re:Making things worse

[American AC in Paris]

Not exactly hard?! At 24 fps, there are 24*60*60=86400 frames per hour of film. That's not what i would refer to as "not exactly hard".

...and if you have a pool of junior editors/interns at your disposal, it's a simple matter of giving them each 15 minutes of film and having them step through it at 2 fps. That's only 3 hours of viewing. Given five or six people, it'd take about a week for them to review a movie in such a fashion several times over.

It's not that hard--especially when you consider that old-school cartoons had people drawing every freakin' frame of a feature-length movie by hand...

Not exactly "no real actors"

[xswl0931]

Depends on how you define "no real actors". "Real actors" were used for the voice work. If you ever played a video game where the voice acting was horrendous (about 80% of the time), then you know that good voice acting isn't that easy to come by. You also need talented animators to turn the 0's and 1's into emotion. In any case, Hollywood has always been more than just the actors, there's a whole production crew behind the picture.

Bandwidth/Administration Hell

[tinrobot]

As someone who works in a digital studio, it's painful enough getting things rendered with every computer in the same room. Frames get dropped, mangled, lost. In addition, every machine needs to be at the same software revision, and you can't have conflicting apps running. Scattering the render boxes across the planet and having boxes that contain unknown software will only amplify the pain to the Nth degree.

Added to that are huge bandwith problems. In order to render a 2K image, you may need dozens of texture maps, some of which may be even larger than 2K because you zoom in or something -- meaning to get a 2K frame back, you're sending the render box probably 10-20 times that amount of data. With a nice gigabit internal network, that's not a huge problem, but shipping them down a DSL line is just not gonna happen.

Re:I suspect you're wrong...

[NanoGator]

"What is much more likely is that the grass, skin, hair, etc. is described by some relatively simple input parameters from which millions of polygons are generated."

That's a very good point. Procedural elements of rendering could be distributed quite efficiently. Shrek 2 had some awesome smoke looking effects that I bet was very CPU intensive. That's exactly the type of thing that could be distributed.

Re:Hold it there for a second

[CGP314]

Giving away CPU cycles so that a multi-million dollar company can improve its product is a wholly different thing.

People pay to wear shirt that advertise mult-million dollar companies. : (


-Colin [colingregorypalmer.net]

I'd rather...

[sharkdba]

help finding a small pox [ud.com] vaccine than helping the already way-too-rich entertainment industry.

Re:I suspect you're wrong...

[Bingo Foo]

He's right, this time. Using more procedural elements is not likely to result in smaller datasets for rendering. What's more likely is that other parts of the scene will get more complicated and always exploit the full resources available. Procedural grass, hair, etc. means that the other non-procedural textures can now be more numerous and complex.

Re:...a whole new world

[Syberghost]

You do still need voice actors. With an animated feature, a really good voice actor can really add to the experience.

Yes, but your pool is WAY more open.

In the days before TV, ugly people with great voices were stars. Today, it's a lot harder for that to happen. (it does happen, but they aren't playing romantic leads.)

An independant filmmaker can find an actor with a great voice, and it doesn't matter what he looks like, what his physical capabilities are, etc.

A quadrapeligic could play James Bond.

Re:Would it be worth it????

[henryhbk]

Since hardware is becoming a commodity, and the budgets for these types of movies is huge (finding nemo production budget ~$94M, and shrek2 was ~$70M according to www.the-numbers.com) and at the price of a simple blade server (figure $2000-3000 for a 2xXeon-2 ghz/1gb ram) you can buy a substantial render farm if you are the contract render house for a film like this.

The security and copyright issues are too big, compared to the low cost (for them) of a render farm. The other costs of a movie outweigh the headaches of distributed rendering with "the public".

You can't have the release date of your movie slip because the latest internet worm is loose, and took out 50% of your "farm" users.

Re:Doubt it'll happen...

[Rothron the Wise]

I've been unable to dig up the reference, but I read in an article about Pixar's "Monster's Inc." that for some frames it took longer to load the geometry than actually rendering the frame.

SETI and Folding@Home work because of the massive asymmetry between the amount of data and the CPU power required, and although you _perhaps_ could find subtasks that could easilly be "offsourced" so to speak, that made sense performance wise, I very much doubt that it would interface very nicely with the way the artists work, or make any sort of economic sense.

Re:...a whole new world

[Anonymous Coward]

The other side of that is that it's only seven minutes long, has no real voices, and took him three years to make it. Imagine making a full length film? It would take you so long that the quality of what came out of the major studios by the time you were finished would be far and away beyond what you were producing when you started.

And that doesn't even touch distribution issues...

Is this really likely?

[bossesjoe]

The movie industry, give you something like that for free? I doubt it, maybe if you paid them so they could render on your computer....

right...

[sad_]

first we would render parts of the movie on our own pc's and if we would like to go to see the movie in the theatres we'd have to pay 6.5 euro for something i helped create.
next i won't be able to play the dvd legaly (which i had to pay for again) on my linux box.
can't wait to start...

Re:Sounds like imp.org

[mbbac]

I'd rather donate the cycles to something other than entertainment. But, if I had to choose entertainment, I'd donate to the needy (read: not Pixar or Dreamworks).