Tip: How to have only 1 process running on each Worker

[Scot Brew]

Question:
Each of my Workers is an 8-core system and by default has 8 Qube job slots open.  When I run a job, each Worker will run up to 8 processes on it at a time.  I only want 1 process to run at a time (though still use all available cores).  How can I set this up?

Solution:
The easiest way is to lock the Workers have them only have 1 process slot available.  That way each subjob will run on a different Worker.  The processes themselves will use all available cores by default if the process is multi-threaded (like with most renderers).

Implementation Steps:

• Launch the QubeGUI
• Click on the Host Layout tab
• Select all the Workers, rt-click, and select the "Lock/Unlock" popup menu item
• Specify "1" and then click "Yes"

[njushchyshyn]

Is there a way to do this dynamically, based on job load on the farm.

Here's what I would like ....
Let's say we have a little 10 worker farm, each worker should be able to handle 4 rendering tasks simultaneously ... so the CPU value for each is set to 4.
Farm has no work to start ... then a job comes in to render 1000 frames.

I'd like that job to automatically partition down to 10, 100 frame sub jobs, with each of the workers taking on a single subjob.

While that's running, a second 1000 frame job comes along...
I'd like that to also, automatically partition to 10, 100 frame sub jobs, with each worker getting one of the 10 sub jobs.

So now I would have 20 sub jobs running, where each worker is working on 2 subjobs.

If I apply your solution above, the second job waits for the first to complete, because the job slots left unused by the first job are locked.
But if I leave all the sub-job slots UNlocked (4 per worker), I can end up with just 5 workers loaded down with 4 subjobs each instead of having the load distributed over the whole farm.

Does this make sense?

[michael.graf]

Using the lock/unlock was not acceptable for the way we needed our jobs to run.  I ended up using reservations.  This way I can leave the host with 8 cpus open and then set $job->{reservations} host.processors=4 and have two 4 cpu jobs run on the host with no additional configuration.  In qube gui, it reflects that it is a 1 cpu job, but in reality with the reservations, it is a 4 cpu job, or 8, or 2, etc.

But I am not sure if this will work in your case with subjobs, as we only have 1 subjob.  Our default job type is not rendering but running long simulations.