custom runner and threading

[mhenrie]

Hello--

We are using our own python nuke runner but I am having issues using licenses effectively.  The license registration is not the issue but rather, when a nuke process runs I want to make sure it is using as many threads/cores as possible.

I can use reservations: 'host.processors=1+' to tell qube to use as many threads as possible, but on the runner side, I am creating the 'nuke -m <threads>' command and need to know how many threads are available to use.  What is the best was to determine how many cores are available?  I basically need to know what '1+' evaluated to so I can run the 'nuke' command with the correct number of threads.  Any ideas?

[Scot Brew]

Yes, we have heard from another studio on the interest of seeing through an environment variable how many "process slots" the "subjob process" is taking up.  If it is of need, we can see about writing up a feature request for it.

You may, however, be able to meet your needs with the current functionality.  If you explicitly set the # of "host.processors" (more accurately termed "process slots"), and you have setup the Worker so that the # of "process slots" equals the # of cpu cores, then you can set host.processors=8 on an 8 core machine and explicitly set the nuke render threads (nuke -m as well.

This may or may not work in your setup, though we wanted to offer it as a possible direct solution.

NOTE: We are adjusting our terminology is this posting to avoid confusion between Qube "process slots" and machine cpu cores. 

[mhenrie]

Thanks for the work-around.  It does seem like I can hard-wire things to use all cores.  I am specifically trying to deal with the case of 'use as many cores as possible'.

I think the variable would be a useful thing to have.  The bigger problem I am trying to solve is license management.  It does not seem like qube has a good system for host-based licensing (like nuke/houdini).  If I am trying to run a number of 1 core nuke jobs, I was hoping to have a mechanism to group the jobs on hosts that are already pulling a nuke license.  It seems like my options are to always use all cores or create a cluster that will only run nuke jobs.  Does that sound correct?