CGTalk > Main Forums > CG General Discussion > Render farms vs cloud computing?

View Full Version : Render farms vs cloud computing? MrBook 03-23-2010, 07:08 PM PDA

Out of pure idle curiosity, I was wondering if anyone uses so called "cloud services" like amazon's ec2 or rackspace's therackspacecloud in lieu of more traditional paid renderfarms (renderfriend, etc). Is this possible? Legal? Efficient? Cost-effective? Discuss.

noouch

03-24-2010, 10:44 AM

Why use non-specialized services when you can pay for experienced people to wrangle your jobs on their (possibly highly optimized) farm, where they (hopefully) own their own rendering licenses etc.? Using a cloud service is pretty much equivalent of taking your job down to the local university's supercomputer facility: you could do it, but it's just not worth the hassle.

MDuffy

03-24-2010, 05:59 PM

I think there are several problems you'd have to solve for rendering on cloud computers: *) Bandwidth to high resolution textures, vertex data, baked sims, other render passes, etc. *) Amount of memory available to the proc you have on the target machine... I've seen render jobs require multiple gigs of memory to render. *) software licenses for your renderer. *) processor/architecture differences between clients which can lead to slight differences in calculations (usually shows up as slightly flashing frames... you don't notice it in still renders but do in animations). *) Software to kill, reprioritize, requeue, assign, and overall track jobs and dependencies. *) The ability to guarantee/schedule availability of processors. Not much of a problem if you are looking for a dozen jobs, but definitely an issue if you are looking to schedule 100,000 jobs at once. Cheers, Michael

RobertoOrtiz My biggest issues about the cloud are simple. The big three in my book are: Cost, Security and Reliability. Cost: With the cloud, you would be moving towards a Rendering as a service economy. Think about it for a second. You would have to pay, either by instance or subscription, to be able to render a scene. I would only use this for finals you say? Well when was the last time you rendered a project that you did not have to tweak? Security: First you would have to send your assets, to a third party to render. Again think about that. How well do you trust that vendor in handling your data? Are you sure you want to trust them with all the intellectual property of your company (rigs, shaders, etc) you may have on your scenes? And how about the liability issues with your clients over their Ip being share with another third party?

03-24-2010, 06:11 PM

Reliabilitiy: Even Google has dropped the ball in terms of dependability of the cloud. What happens when you have a close dateline and the cloud is not there? Can't happen? Tell that to Microsoft :http://techcrunch.com/2009/10/10/t-mobile-sidekick-disaster-microsofts-servers-crashed-and-they-dont-have-a-backup/ IBM http://blogs.siliconvalley.com/gmsv/2009/10/air-new-zealand-boss-lands-hard-on-ibm.html Google http://www.wired.com/epicenter/2009/05/when-google-goes-down-it-goes-down-hard/

Why do this at all, and to be frank GPU rendering promises to erase any advantage in speed cloud computing might offer? And btw way, call me an old fart, but I heard of this tune before. So why not call the Cloud by its true name. MAINFRAME 2.0 In the old days we used to hate this kind of dependence on a SINGLE POINT OF FALIURE. There is a reason why we moved away from it. Like a crime one has ask a core question when the push for the cloud is brought up... Who benefits?

neuromancer1978 I have to agree that cloud rendering is not the best option when dealing with high end rendering. 1. Rendering app

03-24-2010, 07:10 PM

Not all render engines are the same, some such a Renderman use a lot of custom shaders, shadow and baked passes etc... The only way to ensure that there is consistent environment is to have the farm on a network, otherwise you run the risk loosing things, say a single shader that did not make it to the cloud for some reason will RUIN your render, and just imagine 1000 bad frames. Grant you it would be foolish to NOT ensure that all your libraries, code, assets and so on are not on the cloud first but we are human and we do make mistakes. Something so simple as a single missing shader can cost a fortune. 2. Bandwidth Most production rendering consists of large files, such as texture maps and objects, so unless you have a high bandwidth connection these will take time to transfer. If you are transfering data to massive cluster like Google or Amazon it should not be a problem on their end but there will be a a LOT of data being shuffled around from your system, or server or wherever. 3. BOINC This would be a possible solution to cloud computing but it is not always the best option. One reason is that you are not in control over other people's systems nor their network connection. Another reason is as mentioned above, the rendering software NEEDS to be exactly the same as your system and there is no sure way to make sure that it does. This is not so much an issue when your render engine is internal to an app like Maya, Max, Blender or LW however when working with Renderman or MentalRay all your assets are external to the app so again that factor of ensuring a duplicate environment is nearly impossible when dealing with a cloud. 4. Architecture Not everyone uses the same CPU type and this becomes a problem when working with procedural texturing, the shaders are compiled for that system and things like fractals and noise render slightly different. So when you render two different frames from two different architectures there can be a noticble flicker when viewed in full frame rate. While the still frames would be barely different to the naked eye, in motion you will notice right away. So using a service like BOINC would be terrible for something like Renderman for instance, there is no way to ensure everyone is using the same system, for all you know it could be a mix of 32 and 64 bit CPU's from Intel, AMD, SUN etc.. even though they all do the same thing (compute) - the differences are there and this is reflected in the software you use. So... Most rendering services only cater to the apps that have internal renderers, have a high price and if your render is not the exact way you want then you need to pay again to fix it and submit the job. In all reality it may be worth more to the average 3D artist to build a small 6-8 node renderfarm where you have complete control and if you are a studio the same applies there but in a larger scale. My 2 pennies

mr Bob

03-24-2010, 09:19 PM

Houdini can already use the EC2 Amazon cloud . I've had no problems with it at all and its certainly cheaper than running a small render farm. B

Medinilla There are several issues that make the cloud are not viable as rendersfarms.

04-18-2010, 10:34 PM

1. One fundamental is that the cost of using one of the solutions as EC2 reaching 80% of the cost of using a specialized renderfarm. This makes the risk is not offset by the reduction of costs. 2. The infrastructure for receiving and processing jobs require licenses installed on the machines to travez sometimes hardware such as USB vray not possible to connect in EC2. 3. To make a one CS2 software should be developed renderfarm own render queue management. The reason is that when you throw a

job from 3ds max, for example, communication takes place without compression or encryption or verification check shipments. This makes it very possible that information is lost in heavy scenes. 4. Win vs Linux vs 3d applications

thev

04-19-2010, 06:23 AM

2. The infrastructure for receiving and processing jobs require licenses installed on the machines to travez sometimes hardware such as USB vray not possible to connect in EC2.As far as V-Ray is concerned, this is solved by using floating licenses and making the cloud machines take their license from the internet. Best regards, Vlado

adarsh

04-27-2010, 03:43 AM

Interested to know how studios/artists/free lancers are 1) managing their 3D app licensing to used cloud as a rendering platform 2) using the in house license servers or rental licenses 3) Using/Subscribing the AMI's provided by the Houdini, mental ray and other standalone rendering softwares == 4) Would like to know whether anybody have benchmarked the performances of the AMI's (virtual machines) and the physical bare metal machines 5) how is the data managed ? scene upload, output downloaded. 6) how often the scene has to be modified ? :) 7) Have any startup studios started their venture on the cloud before renting the physical machines. Any answers or insight on this would be help. Thank you, Adarsh ---------------------http://www.adarshpatil.com

mr Bob 1) managing their 3D app licensing to used cloud as a rendering platform In the case of Sidefx houdini , its all been taken care of. 2) using the in house license servers or rental licenses You can fire a job off using any of the sidefx licence types 3) Using/Subscribing the AMI's provided by the Houdini, mental ray and other standalone rendering softwares You can only use mantra. I would suggest you try it ....... B

04-27-2010, 08:36 AM

RobertoOrtiz you guys are going to LOVE this.

05-05-2010, 02:27 PM

US Treasury Department shuts down 4 cloud-hosted Web sites after infection (http://gcn.com/articles/2010/05/04/treasury-hackupdate-050410.aspx?sc_lang=en) The Treasury Department has taken offline four public Web sites for the Bureau of Engraving and Printing after the discovery Monday of malicious code on a parent site.

The bureau began using a third-party cloud service provider to host the sites last year, it said Tuesday in a statement about the incident. The hosting company used by BEP had an intrusion and as a result of that intrusion, numerous websites (BEP and non-BEP) were affected, the statement said. The Treasury Government Security Operations Center was alerted to the problem and notified the bureau, which responded by taking the sites offline. http://gcn.com/articles/2010/05/04/treasury-hack-update-050410.aspx?sc_lang=en

Titus The amazon cloud isn't reliable as they say, they have problems with the uptime.

05-05-2010, 03:14 PM

mr Bob The amazon cloud isn't reliable as they say, they have problems with the uptime .

05-05-2010, 10:01 PM

Where is it stated that there's a problem ? Do you have an article on this ? . So far I have had no problems what so ever. b

Titus .

05-05-2010, 11:58 PM

Where is it stated that there's a problem ? Do you have an article on this ? . So far I have had no problems what so ever.

b I don't have an article, I've had the problems.

adarsh Google mail : mails are scanned for content based advertising. Public Clouds : might also be scanned for location/content based advertising. Websites on shared/dedicated hosting and websites hosted on the cloud. SLA with respect to security has to be check when hosting our services, data, source on the public infrastructure.

05-06-2010, 01:15 AM

Security is the main reason for keeping the customers/industries to stay away from the Public cloud and used the Private Cloud (cluster sitting behind the firewall). Data security is the main concern. Machines behind the firewall are compromised due to Operating System loop holes why not the OS running on the hypervisor. Cheers, Adarsh -adarshpatil.com

trancerobot I know this has been mentioned before, but I will repeat it because it's pretty much a show-stopper.

05-07-2010, 05:09 AM

It could work - but the bottleneck will be bandwidth, and that will make it pointless for any large film project or even short film projects. You're going to want gigabit connections, not what passes for 'broadband' on the internet. Also, do you really want to download hundreds of gigabytes worth of individual frames? Even if they were all compressed into a zip file (OR quickly shuffled into a zerocompression tar file - tars are useful for quickly combining thousands of files into one file, one bigger file is faster to transfer than thousands of smaller files. Seriously, try it) I work at a university as a workstudy student, and I have access to 10 computers. I've used these computers on occasion to create a small render farm. That experience is small potatoes sure, but I know how important bandwidth is. It won't matter how many computers are in the cloud, your point of diminishing returns will happen sooner on a low-bandwidth internet connection. So if you don't have the bandwidth to support it, you might as well not bother.

sundialsvc4

05-07-2010, 01:45 PM

My gut-feeling on this is that you are dealing both with "immense volumes of data" and (probably) "very long 100%-CPU processing times." Data transmission speeds, storage space and so on will be a very big deal. And I'm not quite sure that a "cloud" provider is necessarily going to be shooting for that kind of workload; nor might they be happy to receive it. I would think that dedicated specialists, who've got the multi-terabyte SAN farms and the optical infrastructure and the racks of mega-CPUs, would be more to your liking. You're going to be working on a tight deadline (no matter what the deadline is), and you're going to need to negotiate a definite service-level agreement with a company that actually has the wherewithal to do it. Wouldn't surprise me if you have to wait in line for the services of a company like that; and, needless to say, "you'll pay for it but you'll get it." As far as I can see, the "cloud people" are shooting to serve a high-volume-quantity workload consisting of comparatively lightweight requests. Not much CPU time, maybe a lot of storage, "transactions per second" is king. The workload is very orthagonal and can be farmed-out to a lot of parallel machines without overheating any one of them. That sort of thing -- and there's a lot of "that sort of thing" out there ... just not here.

mjhm

05-31-2010, 09:56 PM

I've done some first hand investigation into this. As mentioned there are several obstacles. 1. Bandwidth. My experience was that transferring data into EC2 was at best 200Mbs, and the cross sectional bandwidth among EC2 servers was no better than 500Mbs. Compared to the 10Gbs networks at most major facilities, this is a major obstacle. Transferring files with consumer cable @ 20Mbs is of course a deal killer. 2. Synchronization of assets and software. Managing synchronization of assets and software versions on internal infrastructure is tough enough. But managing assets and software across a low bandwidth pipe and various firewalls would also be a deal killer. 3. Software Licensing. Hourly rental of licenses for most cg software is not available, with exceptions being HQueue http://www.sidefx.com/index.php?option=com_content&task=view&id=1630&Itemid=66, and Mental Ray http://cloud.mental.com/direct/. I suspect that these are not being used too much, since they haven't come out of beta. 4. Security. Security concerns for production assets are no more and no less than other businesses. So while security is a concern, I'm pretty confident that by the time problems #1 and #2 are solved, production will be able to coattail security solutions from other industries. 5. Tools. Anyone trying to do cloud based production, at a scale large enough to make sense, is going to have to roll a lot of their own tools to manage the unique workflows, problems, and opportunities of this paradigm. For example with potentially infinite processors available, the concept of a render queue changes completely. Incidentally I believe RenderRocket uses EC2 to supplement their internal render capabilities. And Axceleon offers their EnFuzion solution for EC2 as well. I have no doubt that cloud based production can be done, and probably will be done by someone in the next couple years. But it's not going to be done by an existing facility casually porting parts of their internal infrastructure to the cloud, but rather by someone with unique requirements that are a good fit for the paradigm. Someone also mentioned the competition of desktop GPU's vs. cloud CPU's. The next year will see the introduction of cloud based GPUs, and that could create a whole new ball game for this discussion. To get a hint of what may be capable, check out the Project Twitch Maya demo at http://labs.autodesk.com/technologies/trials/overview/ John McLaughlin

CGTalk Moderation

05-31-2010, 09:56 PM

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.

vBulletin v3.0.5, Copyright 2000-2012, Jelsoft Enterprises Ltd.