creating complex dependant jobs

[cam]

Hi support,

How are complex (multi step) jobs supported in qube?

I have no experience with qube but have used other proprietary and commercial solutions, such as Alfred.

The end goal is to chain together a complex job tree, for say a renderman render.

For example:
1. mkdirs
2. build maya file from assets, save scene
3. generate ribs from maya file (say 4 shadows, 1 beauty pass)
4. render shadows
5. render beauty pass
6. run a frame checker script
7. run a 2d process on the frames, say warp for comp
8. run rib clean up script
8. send a mail to say it's all done.

This raises a couple of questions:

1. From the video it seems that when two jobs are on the farm (let's say job 100 and job 101) you can right click and make job 101 wait for job 100 -good!

Can you explain how this is handled on submission (can you know a job id prior to submission?).

2. From the video you can write custom submit scripts/guis in python --> great!

Can you write complex jobs with dependencies in python?

3. Is there skip, stop, requeue functionality within the dependency tree?

requeue --> In the example above if the beauty pass is running and you notice 1 of the shadow maps is corrupt - can you requeue the 1 of 3 shadows? This requeue would then stop the beauty pass, requeue it and block it until the shadow map is complete?

skip --> if your beauty pass is done and your frame checker script is getting stuck on known false errors. Can you skip this task and let the job carry on to the next step?

4. Can you filter a job tree?

- Can you select one task/job and hit a 'show connected' button?
- Is there a job tree view a la Alfred?

Lots of questions - thanks in advance for any info.

cl

[shinya]

Hi cam,

#1, #2)
Complex dependencies will need to be submitted by scripts (Python or Perl).

#3)
While there aren't "skip, stop, requeue" features per se, you may define arbitrary
complex dependencies with Qube, so it's possible to implement, for example, a
dependency that will stop a job downstream when a job upstream changes its
status.  Needless to say, it's also easy to manually block (stop) a job.

#4)
The Qube GUI has a feature where you can apply a filter to only display jobs in the same "process group" or "pgrp"-- usually, all jobs in a dependency tree is put in a single pgrp.


Have a look at our Development related documentation, which is available on our support site http://support.pipelinefx.com/ for more info on creating dependencies.

Hope that helps!

-shinya.

[Dangertaz]

Hi there,


I'm trying and trying to get the Maya 2009 render passes to work with our Qube but only my Master Beauty pass comes back correctly, any others (reflection, diffuse, etc) come back black!  They contain data (maybe 1.5 megs each) but they are 100% black.

I can render the passes natively in Maya with a batch render and they come back correctly.

What are the settings in Maya and within Qube that I should use to make this work??

Thanks in advance!