Author Topic: Custom backend: how to get 1 entry for every processed frame in qube supervisor?  (Read 2594 times)

juanjux

  • Jr. Member
  • **
  • Posts: 2
Hi,

I'm implementing a custom backend, similar to mayaprompt. It works, but I've the problem that in the Qube Supervisor "Agenda/Frames" tab there isn't a 1 to 1 correspondence between the entries in the GUI and the processed frames output, that is, if I've a work with 10 frames and a single worker (to simplify the example), for example the frame "1" in the tab shows as complete but without output (it just shows "requesting work for: XXXX.X"), same for the frame "2", then the output entry for the frame "3" shows the output of the first 3 frames, etc. It repeats the same pattern every three frames, that is, the entries for the frames 4 and 5 would again miss any output and the entry for the frame 6 would have the output of the frames 4, 5 and 6.

I'm using the way documented here and what I've seen mayaprompt uses (I'm using Python):

Code: [Select]
job = qb.jobobj()

jobstate = 'complete'

while True:

  agendaItem = qb.requestwork()
  if agendaItem['status'] == 'complete':
    break
  elif agendaItem['status'] == 'blocked':
    jobstate = 'blocked'
    break
  elif agendaItem['status'] == 'pending':
    jobstate = 'pending'
    break

  else:
    try:
      print 'BEGIN ITEM %s' % str(agendaItem['name'])
      # ...omitted, work with the job and job['package'] to generate the frame...
    except Exception, e:
      print 'WORKER: Exception: %s\n' % str(e)
      print_exc()
      agendaItem['status'] = 'failed'
      jobstate = 'failed'
   else:
      agendaItem['status'] = 'complete'
   qb.reportwork(agendaItem)
qb.reportjob(state)
    

Thanks,
Juanjo Alvarez
(Ilion Animation Studios Developer)
« Last Edit: March 28, 2012, 07:48:15 AM by juanjux »

jburk

  • Administrator
  • *****
  • Posts: 493
This has to do with the job's "agenda", or list of work to be performed at the time the job is built at the submitting host.

If you're working over a frame range, your agenda currently probably contains a single item with the frame range as a string: '1-5'  You have a list with a single item in it.

What you want to do is to generate a list of work, one per frame:

Code: [Select]
job = {}
fStart = 1
fEnd = 5
fStep = 1

job['agenda'] = []

# you can do it manually
for i in range(fStart, fEnd+1):
    agendaItem = {'name': i}
    job['agenda'].append( agendaItem )

# job['agenda'] is now
# [{'name': 1}, {'name': 2}, {'name': 3}, {'name': 4}, {'name': 5}]

# or you can use one of qb's gen* convenience functions:
import qb

# the same thing as the above iteration loop
job['agenda'] = qb.genframes( '%s-%s' % (fStart, fEnd))

# job['agenda'] is also now
# [{'name': 1}, {'name': 2}, {'name': 3}, {'name': 4}, {'name': 5}]

# divide the agenda into 2-frame "chunks"
job['agenda'] = qb.genchunks(2, '%s-%s' % (fStart, fEnd))

# job['agenda'] is now
# [Work({'name': '1-2'}), Work({'name': '3-4'}), Work({'name': '5'})]

# Or if you don't care about the chunksize, but want to split it up across a known number of items:
# split 73 frames evenly as possible across 4 items:
job['agenda'] = qb.genpartitions(4, '1-73')

# job['agenda'] is now
# [Work({'name': '1-19'}), Work({'name': '20-38'}), Work({'name': '39-57'}), Work({'name': '58-73'})]
   

juanjux

  • Jr. Member
  • **
  • Posts: 2
No, I'm already generating the agenda (using genframes). I've printed job['agenda'] and all seems to be Ok, but it still only process 3 frames in every third entry. This is a screencap of the Qube Supervisor for this work:



Of the frames on this image, the distribution of works really processed are (using a single worker):

1: None
2: None
3: 1,2,3
4: None
5: None
6: 4, 5, 6
7: None
8: None
9: 7, 8, 9

...etc...