Best practice for when to do a custom JobType or just wrap it as a cmdrange job?

[instinct-vfx]

Hey everyone,

is there a general rule when it is a good idea to create an actual jobtype vs. just wrapping up a cmdline/cmdrange job. The question probably boils down to what do i get in addition to a wrapped cmdline job i guess.

Any pointers would be greatly appreciated!

Cheers,
Thorsten

[jburk]

The cmdrange jobs are useful when you simply need to iterate over a list of frame numbers, and can merely replace the frame numbers in the command line as you iterate.

Custom jobtypes are pretty much unlimited in what they can do.  For instance, we have an api-submission only jobtype that allows you to submit a job where each agenda item is a different command; several customers are using this in a fashion similar to the cmdrange jobtype, iterating over a list of 3-letter codes for the various languages for a localization process, where they mux a particular language track over the same video.

Another custom jobtype allows for mutli-dimensional variable substitution; a cmdrange job is 1-dimensional, in that it's like a spreadsheet with only 1 column, and there's a different frame number for each row.  But if you're rendering tiles and you're specifying the X and y coord for each tile, and each frame is broken down into 4 tiles, you have a 3-dimensional variable:

frame 1, x1, y1
frame 1, x2, y2
frame 1, x3, y3
farme 1, x4, y4
frame 2, x1, y1
.
.
.

So I've written a multi-dimensional jobtype that allows for any number of dimensions, and the job command-line contains tokens that look like QB_Dn, where

So a job that looks like:

Code: [Select]

xRange = 2
yRange = 3
zRange = 4

xyzJob = {
    'prototype': 'pyCmdrangeND',
    'priority': 5000,
    'name': '3-dimension demo',
    'label': 'XYZ',
    'package' : {
        'padding': [4,0,2],       
        'cmdline': 'printf "\\n%8s %8s %8s\\n\\n" x:QB_D1 y:QB_D2 z:QB_D3; sleep 1',
        },
    'cpus': 4
}

agenda = []
for x in range(xRange):
    for y in range(yRange):
        for z in range(zRange):
            work = { 'name': '%s/%s/%s' % (x,y,z) }
            agenda.append(work)

xyzJob['agenda'] = agenda


Builds commands that look like:

COMMAND: /bin/sh -c printf "\n%8s %8s %8s\n\n" x:0000 y:0 z:00; sleep 1
COMMAND: /bin/sh -c printf "\n%8s %8s %8s\n\n" x:0000 y:0 z:01; sleep 1
COMMAND: /bin/sh -c printf "\n%8s %8s %8s\n\n" x:0000 y:0 z:02; sleep 1
COMMAND: /bin/sh -c printf "\n%8s %8s %8s\n\n" x:0000 y:0 z:03; sleep 1
COMMAND: /bin/sh -c printf "\n%8s %8s %8s\n\n" x:0000 y:1 z:00; sleep 1
COMMAND: /bin/sh -c printf "\n%8s %8s %8s\n\n" x:0000 y:1 z:01; sleep 1
COMMAND: /bin/sh -c printf "\n%8s %8s %8s\n\n" x:0000 y:1 z:02; sleep 1
COMMAND: /bin/sh -c printf "\n%8s %8s %8s\n\n" x:0000 y:1 z:03; sleep 1
COMMAND: /bin/sh -c printf "\n%8s %8s %8s\n\n" x:0000 y:2 z:00; sleep 1

[instinct-vfx]

Thanks a LOT for that really in-depth answer John! So if i just need to wrap a call to an external post processing exe, i am fine with a cmdrange job? Or are there still advantages (like reporting, statistics, stdout/stderr  or whatever).

Thanks again!
Thorsten