Rendering Shrek@Home? 345
JimCricket writes "There's an interesting piece at Download Aborted about using distributed computing (a la SETI@Home, Grid.org, etc.) in the film industry. With the recent release of Shrek 2, which required a massive amount of CPU time to complete, one must wonder why the film industry doesn't solicit help from their fans. I'd gladly trade some spare CPU time in exchange for the coolness of seeing a few frames of Shrek 3 rendered on my screensaver!"
This made the front page? (Score:5, Informative)
Lobotomizing it to the point where this wouldn't be useful would probably make it useless for distributing the workload as well.
NOT proprietary rendering technology (Score:5, Informative)
Both studios are using Renderman compliant renderers, so that's not the issue.
And there's no reason that any one machine has to render an entire image file. You could have any node build N number of scanlines and send the packet back home.
The risk would be someone running a port monitor on the return address, and re-assembling digital image files.
Re:Doubt it'll happen... (Score:5, Informative)
There'd be no sound.
I'm sure people would sit through it anyway, though.
You don't have the machine for it... (Score:5, Informative)
Even on high end machines they often do not render a full frame, but a layer of a frame which is then composited with other layers into the full frame. Why? Many reasons but one of them is that even the high end machines don't have enough RAM and the render would take too long (the machine would need to swap).
So aside from the issues of fans returning bogus data, or extracting highly proprietary information out of the client as other threads have mentioned, this would be a real show stopper. Breaking the problem into small enough pieces to be handled by joe-blow's computer would be prohibitive and require tons of calculations to figure out which pieces of textures are actually required for a given piece of rendering etc. It would probably require a compute farm just to manage it!
Rendering is also a lot more complex than you might think, there are render wranglers who manage the rendering queues and look at the outputs... many renders may require specific versions of the rendering software, so a frame that rendered with 2.7.2.1 won't render anymore without errors with 2.7.2.2... so many copies of the software are managed in parallel with the wranglers helping to clean up the errors. How would you manage this in a distributed client environment?
Furthermore most of the proprietary rendering apps are certified against VERY specific platforms, eg. one specific kernel version and build level, specific versions of shared libraries etc.
Long and short is there's a reason why movies cost millions.
It wouldn't be that bad... (Score:2, Informative)
Distributed rendering can be compute-bound (Score:5, Informative)
This is not just tracing all of the rays in the scene.
Bruce
Re:Would it be worth it???? (Score:1, Informative)
http://setiathome2.ssl.berkeley.edu/totals.html
SETI@Home averaged 72.27 TeraFLOPs / sec in the last 24 hours, I check often, and seeing a 55-70 range is normal.
And these are the top 5 'supercomputers' in the world:
http://www.top500.org/
At:
1) 35.87 TeraFlops
2) 13.88
3) 10.28
4) 9.819
5) 8.633
Now SETI = 72 TFlops/sec
For raw power, there isnt a server farm out there than can rival 5M+ users.
Re:Would it be worth it???? (Score:5, Informative)
The network, too, isn't going to be anything as exotic as 10Gb/s. In fact the only single component that's really high-end is the storage -- a lot of data, and hundreds of clients accessing it simulataneously.
I work at an effects shop not a million miles from Pixar, and our rendering is done on a few hundred Athlons, some dedicated and some user workstations. Pixar is much bigger, and they have much more horsepower, but it's not orders of magnitude stuff.
I think SETI@Home is probably a long way ahead in raw aggregate CPU performance. Probably less far ahead in memory surface (but still ahead). But you couldn't use SETI@Home for a reason mentioned by another poster in this thread: bandwidth to storage. The render pipeline has a lot of I/O in it, and your distributed clients would be forever waiting to read or write from network-distant storage. Efficiency would suck, and reliability, too.
Even if you could do it, you wouldn't for issues of information security (which someone else mentioned here, too.)
Re:Doubt it'll happen... (Score:5, Informative)
I can give you a little data here. Take a look at this image [reflectionsoldiers.com] I made. The scene is roughly 1.5 million polygons, and virtually everything is textured. The folder containing the bare bones version of this scene is roughly 600 megabytes. I could probably cut that size in half via JPEG etc, but we're still looking at a massive amount of data to send to one person to render a frame. I know this because I seriously discussed sharing the rendering with a friend of mine on the east coast. We both felt it'd take longer to ship than it would to render.
I doubt this scene is anything close to what they were doing in Shrek 2, let alone whatever will happen with 3.
Rendering time (Score:5, Informative)
WETA BY THE NUMBERS
HUMANPOWER
IT staff: 35
Visual f/x staff: 420
HARDWARE
Equipment rooms: 5
Desktop computers: 600
Servers in renderwall: 1,600
Processors (total): 3,200
Processors added 10 weeks before movie wrapped: 1,000
Time it took to get additional processors up and running: 2 weeks
Network switches: 10
Speed of network: 10 gigabits (100 times faster than most)
Temperature of equipment rooms: 76 degrees
Fahrenheit Weight of air conditioners needed to maintain that temperature: 1/2 ton
STORAGE
Disk: 60 terabytes
Near online: 72 terabytes
Digital backup tape: 0.5 petabyte (equal to 50,000 DVDs)
OUTPUT
Number of f/x shots: 1,400
Minimum number of frames per shot: 240
Average time to render one frame: 2 hours
Longest time: 2 days
Total screen time of f/x shots: 2 hours
Total length of film: Rumored to be 3.5 hours
Production time: 9 months
I suspect you're wrong... (Score:5, Informative)
What is much more likely is that the grass, skin, hair, etc. is described by some relatively simple input parameters from which millions of polygons are generated. The "rendering" process almost certainly includes generating the polygons from raw input data and seeded random number generators through perlin noise distribution routines through fractal instantiation through spline generation through polys to a rendered frame as the final product.
However, much of that work would only have to be done once, then shots taken from different angles on the resulting textured polygon structure, whereas on a distributed architecture any info that isn't sent to your machine would have to be regenerated for your machine. Not to mention that memory requirements are likely to be pretty darn high.
Frame rendering? (Score:2, Informative)
Re:Doubt it'll happen... (Score:3, Informative)
By the way, heres the trailer for appleseed. Quite a beautiful CG animation to behold. apple.co.jp trailer [apple.co.jp]
Sounds like imp.org (Score:5, Informative)
My big question is why would you rather donate to a large commercial organization well funded from it's previous Shreck flick -- rather than donate the cycles to a project like the IMP works themselves?
Re:NOT proprietary rendering technology (Score:5, Informative)
Pixar's renderer is actually PRMan.
From Renderman.org [renderman.org]:
There are a lot of people when you hear them talking about RenderMan and how great the images are from it, etc. They are most likely really talking about Pixar's PhotoRealistic RenderMan® (PRMan).
RenderMan is actually a technical specification for interfacing between modeling and rendering programs. From 1998 until 2000 the published RenderMan Interface Specification was known as Version 3.1. In 2000 Pixar published a new specification, Version 3.2. Coming soon Version 3.3
Seeing that its my farm... (Score:5, Informative)
We used thousands of processors to render. We had terabytes of storage. It is a large undertaking. Every single frame and element of the frame had to be tracked. It had to be qualified. If something didn't work, we had to diagnose the system and get it back up and running. This is something that is too large of budget for a home brew system to work.
With other distributed systems, there are some checks and balances on the data ran, a way to know if you are sending back somewhat good data. The only way you can tell with this is to visually inspect the end result. If a person has a system that returns a bad slice of a frame, you now have to recreate that slice and track it, because its possible the problem is in the code, in the data files or it was a one time glitch with the system. Not a fun thing to do for hundreds of remote systems that aren't similar.
Render time also varies. It can be 5 minutes to 12+ hours. If a job gets halted, you lose that data, and have to recreate it. This isn't like generating millions of keys. There isn't a second init time before turning out data. At a previous studio, we had scene load times of over 30 minutes before it even started rendering. That needs to be accounted for in how you split up frames. If you have 30 minutes to load (after 45 minutes to download the data) and only render for an hours worth, you are getting a heavy hit on over head.
There are just too many issues with this working in a current setup. Stick to crunching numbers.
-Tim
Too much speculation (Score:2, Informative)
Further elaboration on the impossibleness of this (Score:5, Informative)
A
And then the textures... They do use lots of procedurals, but they also use lots of 16 bit per channel textures of 4000x4000 for face textures, or even higher. Some people are using tiles if 16 bit tiffs for displacement maps now that equate to like a 100,000x100,000 image for displacement maps, because the accuracy requirements for close up renders are so bloody high. That can be many many gigs of data there.
And, if you're raytracing like in Shrek 2, then you need to have as much of that data in RAM at once, or else render time spirals out of sensibility, unlike scanline renderman where swapping is easier, because the rays bouncing throughout the scene make scene divisions more difficult (but still possible).
I work with 4 gigs of RAM and we can just barely render 6 million polygons + a few 4k displacement maps all raytraced at once (in windows unfortunately). And, when we render sequences and stuff, we often almost kill our network because distributing all this data to just 20-30 rendernodes is pretty tough (and how would that scale to a big renderfarm with thousands of rendernodes...)
So, yeah, like everyone else is saying, bandwidth limitations and that people running the screen saver probably don't have the hardware and OS to really run 4+ gigs of RAM, this Shrek@home idea seems rather unlikely. It would be cool though, if it worked...
Hooray for my totally unoriginal post!
Too Huge A Job (Score:4, Informative)
First off, what is rendered by the computer is not what you see on screen. There are perhaps a dozen object layers that are rendered individually and composited in the postproduction phase. So, for example, Shrek might exist on one layer, the donkey on another, the ground on a third, some foreground objects on a fourth, several layers of background objects on the fifth through tenth, et cetera.
Now, each object layer will also be split into several render layers, for color, shadows, specularity, reflectivity, transparency, and probably several others that I can't think of right now. It is not an exaggeration to say that a single frame of a completely CGI scene can be made up of upwards of fifty individual frames, all composited together in post.
Why is this done? First off because it's easier to edit and change one of these layers and re-render it, than to change and re-render the entire scene. If Shrek is too gruesomely gleaming, but Donkey is just fine, you just have to edit Shrek's specular layer. This is easilly done in any professional postproduction software package. Alternatively, if it's completely wrong, you just have to re-render that specific layer -- saves a LOT of time! Some post tools are extremely powerful, which makes rendering to object/render layers very appealing.
Now, while you could conceivably do Shrek@Home, you would need a fairly large render program -- and you're already distributing a very powerful program, which the people who wrote it would be very uncomfortable doing. Secondly, the processing power in even high-end PCs is going to be jack compared to what they have in render farms, and they have a lot more of those computers besides. Rendering is very processor-intensive, too. It's a complex mathematical process that can take hours. Many computers will chug along at 99% load on the processor because they HAVE to.
Add to the fact the stake in the heart of this idea: that the producers want reliability first and formost. An in-house render farm, or even renting time at a farm (an idea I've sometimes played with) that signs and seals and delivers is going to be reliable and dependable or they will know exactly whose head needs to roll. If you start having half the internet rendering the million or so frames of your blockbuster, who do you hold accountable when the deadline comes and you're short 1000 various random frames?
Need rapidly fading (Score:4, Informative)