Round-Robin instead of FIFO?

[pboucher]

Is it possible for workers to round-robin on jobs of the same priority instead of the usual FIFO.

Will this imply my writing my own queuing code?

Cheers,
Pat

[anthony]

Hey Pat,

    Actually the way Qube! is implemented makes round robin impossible to implement even with custom queuing code.  At least in the truest sense.  However, we're working on 5.3 and are always open to queuing algorithm suggestions.  This is the first someone has suggested something beyond priority or user balancing. 

    On the other hand, is there a motivation behind using a round-robin approach?  are you trying to balance each job's access to the farm? or is this an attempt to balance how much of the farm each users gets?

     A.

[pboucher]

    On the other hand, is there a motivation behind using a round-robin approach?  are you trying to balance each job's access to the farm? or is this an attempt to balance how much of the farm each users gets?

I've had to think about this one quite a bit. Had to find what my real underlying reasons were...

What I'm really looking for is a way for users to get, as quick as possible, at least one or a few frames of their job rendered so they can validate output. Render settings are often edited at submission time and if the submission is caught third in line the artist may not be able to validate in an appropriate time frame.

I see this as being balanced job access to the farm. If it were balanced user access, a single user submitting many jobs might have a single job go before his other jobs.

There's also a point about quick access to a shot by the comp team... If many render jobs need to be comped together, it could (arguably) be preferable to have the beginning of all your layers instead of just one full layer. For example three layers of 100 frames for a single comp could render out the 50 first frames of all layers as RR instead of 100 frames of layer 1, 50 frames of layer 2 and 0 frames of layer 3 as FIFO. In the round robin case, you could start working on the first 50 frames of the comp while the rest renders. In the FIFO case you might be slightly handicapped depending on the shot.

[anthony]

Ah ok so essentially a job's importance is based on how many frames it currently has running. 

You'll be able to implement such a queuing algorithm with the current qube! api. 

This isn't one of the standard queuing models, so you'll need to create a perl module and tell the supervisor to use the module when dispatching jobs.

The simplest form if what you're writing about, is the comparison of job status tally data.

jobB.todotally.running <=> jobA.todotally.running

Essentially you're looking to compare the number of running frames for each job.  If one job has more than the other then the job with less running frames is the preferred job to start.

A.


I'm attaching a queueing algorithm example to show you how to do it.








[attachment deleted by admin]