Getting workers to share the load.

[LEVEL]

Hello,

I have a 4 worker render farm running Qube 6.0 running 4 Boxx render nodes with 2.27GHz 16 cores each and 16 GB of ram that renders jobs from Maya and After Effects.

My question is how do I configure the nodes to share the load?
It seems like regardless of the Host order settings I use it will all load up one node leaving the others just sitting there doing nothing. Other times it seems to automatically use all nodes and then not use them all the next time I resubmit the same exact job. Am I missing some setting?

[jburk]

The hostorder field is probably not the value you want to be modifying here. It's for tweaking the host selection order when the supervisor is selecting candidates to dispatch subjobs to, and almost never needs to be modified from the default setting, which is blank or null.

The supervisor sees all job slots in the farm as equal.  There is also some dispatch optimizations that occur that can load up one worker before others receive any subjobs.

We've found that overall job throughput is usually increased by running fewer subjobs at the same time on a particular worker, and having those those subjobs run multi-threaded renders.

The simplest way to achieve this is to your Maya jobs 'renderThreads' option to 16, and
set the job's "reservations" to "host.processor=1+".  ("1+" means "reserve all cores")

You can experiment with this by configuring your job's application to run multi-threaded but less than the total number of cores on the machine, and then matching the number processors reserved by the job to the number of threads that your render is running.

For example, on a 16-core host, set the renderThreads=8 and job.reservations=host.processors=8 to allow 2 8-threaded renders to run on the worker at the same time.

Many sites have found that it's simplest to configure the workers to only advertise 1 job slot (set worker.cpus=1), and then run everything on the worker as multi-threaded with as many threads as there are cores.

[LEVEL]

This worked perfectly!

Thanks for the help

[ptanna]

I am having the same problem with C4d.  There is no 'renderThreads' option, but I do see a 'threads' option.  I set that to 16.  I also set the host.processor=1+.  However, the supervisor is still not spreading the work among the available workers.  Is there something additional that needs to be specified for C4d renders?

[jburk]

When you set C4D's 'threads' option, you've instructed the application to consume all processors on a single running instance of C4D.

Now you need to tell Qube how many separate instances of C4D you want to run on your farm.  You control this by setting the number of "subjobs" the job is comprised of. 

In Qube 5, this setting was named job.cpus (it's a holdover from the when Qube was first written in the mid-to-late 90's, when almost all applications were single-threaded).  It's displayed in the QubeGUI jobLayout tab as 'cpus'.

In Qube 6, this setting has been renamed "subjob processes".  It's displayed in the QubeGUI jobLayout tab as 'processes'.

Setting this value to a number greater than 1 will allow more than 1 single instance of a multi-threaded C4D render to start on your farm.  Since you've also set the job's reservations to "host.processors=1+", this will ensure that only 1 multi-threaded render starts on each worker host on your farm.

[ptanna]

Even when I set the subjob processes to 8, only two instances of C4d runs at a time.  I am outputting in the TIFF format.  The job completes just fine, but only two frames get rendered at a time.  So that is probably why the second worker node never gets used.  Our license at the top (above the menu) says 2 licenses... I assumed thats was the number of worker nodes we could have.  Can the license also limit the number of subjob processes allowed?  Or is it something else?

[jburk]

It would appear that your qube licenses are not installed correctly; the fallback if the qube supervisor is unable to detect any valid licenses is to only dispatch 2 subjobs. 

This is to allow for evaluation customers to try Qube without requiring a license.

If your qubeGUI is still reporting only 2 licenses, you should probably open a support case.