PipelineFX Forum
Qube! => Developer Customization => Topic started by: lcatd on January 26, 2014, 06:30:25 AM
-
Hi all,
I'm new to Qube and to this forum as well. I spent some time on the dev/python doc and google search in general, and please forgive me if my question has come up and been answered before.
In the GUI, under "Jobs", there is one column "Time To Finish", and that's very useful.
Where can I retrieve that piece of information via any API call (seems not present in qb.jobinfo() )? And can anyone tell me the logic behind the estimate?
Much appreciated,
Kevin
-
Welcome to the
borg board.
The ETA columns (either ETA or Time to finish) are calculated by the GUIs, themselves -- they are not directly available via the API. Calculating the ETA requires retrieving the agenda and/or subjob list, which is an expensive request. For this reason, as you may have noticed, neither GUI displays ETA until you click on the job.
The jist of calculating the ETA is fairly simple: For all frames that have finished, you average their running time, then you project that time to the frames that have not finished and do a little date math to get the time to finish.
Here's [mostly] how the ArtistView does the calculation. Note that we're in a job object, so "self" is a job.
def calculateAvgFrameTime(self,force_update=False):
# Use either Tasks or Subjobs if no tasks
if self['todo'] > 0: # Use Tasks
itemField = 'agenda'
tallyField = 'todotally'
else: # Use Subjobs: No agenda items, so must just have subjobs
itemField = 'subjobs'
tallyField = 'cpustally'
try:
agenda = self[itemField]
except KeyError:
return None
elapsedSeconds_avg = 0
numCompleteTasks = self[tallyField]['complete']
if numCompleteTasks > 0:
for a in agenda:
if a['status'] == 'complete':
elapsedSeconds = a['timecomplete'] - a['timestart']
elapsedSeconds_avg += elapsedSeconds
elapsedSeconds_avg /= numCompleteTasks
return datetime.timedelta(seconds=elapsedSeconds_avg)
else:
return None
def calculateETA(self, now=None, force_update=False, remaining=False):
# if not running, no need to calc an ETA
if self['status'] != 'running':
return "--"
# if we can't figure an average frame time, no need to calc ETA
avgFrame = self.calculateAvgFrameTime()
if not avgFrame:
return "--"
# Use either Tasks or Subjobs if no tasks
if self['todo'] > 0: # Use Tasks
itemField = 'agenda'
tallyField = self['todotally']
else: # Use Subjobs: No agenda items, so must just have subjobs
itemField = 'subjobs'
tallyField = self['cpustally']
if tallyField['complete'] > 0 and tallyField['running'] > 0: # has both complete items and is running
if not now:
now = datetime.datetime.now()
# Multiply pending by frame average
eta = avgFrame * tallyField['pending']
eta += avgFrame * tallyField['running']
# Reduce by running frames elapsed time (cap at avg frame time)
agenda = self.get(itemField, [])
for a in agenda:
if a['status'] == 'running':
runningDuration = now - datetime.datetime.fromtimestamp(a['timestart'])
eta -= min(runningDuration,avgFrame)
eta /= tallyField['running']
now -= datetime.timedelta(microseconds=now.microsecond)
eta -= datetime.timedelta(microseconds=eta.microseconds)
if remaining:
return eta
eta_datetime = now+eta
if eta_datetime.date() == now.date():
return eta_datetime.time()
return eta_datetime
else:
return "--"
Important Note: As I mentioned, this is an expensive operation. You DO NOT want to do this for every job on your farm - doing so could quite possibly stop the farm while it is working (remember, you're getting numberJobs * numberFrames records - each of which needs to be retrieved, serialized, and shipped by the supervisor, which is also doing things like job prioritization and dispatch).
-
Thank you very much. I'll keep the note in mind :)