Job Priority

[jesse]

I'm having some issues understanding the default handling of job priority.  Currently I have Job A running with a priority of 2500.  I then submit Job B with a priority of 5000.  As Job A completes frames, I would expect that workers would pick up Job B frames until there are not more high priority frames to process.  Then Job A would be allowed to continue processing frames.

As it is working for me now, if a frame for Job A completes another frame for Job A starts keeping Job B in a blocked state until all of Job A's frames have begun processing.

Have I misconfigured Qube? Is this the designed behavior?

Thanks,
Jesse

[eric]

This is normal behavior. Job priority is attached to the subjobs, not the frames. If you submit a high priority job, it will consume subjob slots, if necessary preempting those of lower priority jobs, until all the requested subjobs have been dispatched.
If you then submit a lower priority job and if there are no available slots, it will pend until the higher priority subjob completes, which may not be until all its frames have rendered.

[jesse]

Ok, I don't know if I am understanding this correctly yet.  From the documentation I see that the --cpus flag is what determines the number of subjobs.  I've tested this with this set of jobs:

job name	cpus	frames	priority	submitted	completed
Job A	100	100	2500	10:01	10:15
Job B	200	200	4000	10:05	10:21
Job C	30	30	5000	10:09	10:22

Job A has the lowest priority but was submitted first.
Job B has a higher priority and was submitted second.
Job C has the highest priority and was stubmitted last.

As Job A completes subjobs, more subjobs from Job A start to process rather than subjobs from Job B or Job C starting.  If Job B began to process before Job C was submitted, Job B must start all of it's subjobs before Job C can start to work.

It seems that an Active job always has priority over a Pending job.   I would like subjobs from the highest priority pending job to be processed first.  Is it possible to do this with qube?

Thanks,
Jesse

[eric]

You have the priority reversed. 1 is the highest priority, and 9999 is the lowest.

[jesse]

That's fabulously spectacular.
Thank you.