Author Topic: python API for "Time to finish"  (Read 7372 times)

lcatd

  • Jr. Member
  • **
  • Posts: 2
python API for "Time to finish"
« on: January 26, 2014, 06:30:25 AM »
Hi all,

I'm new to Qube and to this forum as well. I spent some time on the dev/python doc and google search in general, and please forgive me if my question has come up and been answered before.

In the GUI, under "Jobs", there is one column "Time To Finish", and that's very useful.
Where can I retrieve that piece of information via any API call (seems not present in qb.jobinfo() )? And can anyone tell me the logic behind the estimate?

Much appreciated,
Kevin

BrianK

  • Hero Member
  • *****
  • Posts: 107
Re: python API for "Time to finish"
« Reply #1 on: January 27, 2014, 05:41:16 PM »
Welcome to the borg board.

The ETA columns (either ETA or Time to finish) are calculated by the GUIs, themselves --  they are not directly available via the API.  Calculating the ETA requires retrieving the agenda and/or subjob list, which is an expensive request.  For this reason, as you may have noticed, neither GUI displays ETA until you click on the job.

The jist of calculating the ETA is fairly simple:  For all frames that have finished, you average their running time, then you project that time to the frames that have not finished and do a little date math to get the time to finish.

Here's [mostly] how the ArtistView does the calculation.  Note that we're in a job object, so "self" is a job.

Code: [Select]

    def calculateAvgFrameTime(self,force_update=False):
        # Use either Tasks or Subjobs if no tasks
        if self['todo'] > 0:   # Use Tasks
            itemField = 'agenda'
            tallyField = 'todotally'
        else: # Use Subjobs: No agenda items, so must just have subjobs
            itemField = 'subjobs'
            tallyField = 'cpustally'

        try:
            agenda = self[itemField]
        except KeyError:
            return None
        elapsedSeconds_avg = 0
        numCompleteTasks = self[tallyField]['complete']
        if numCompleteTasks > 0:
            for a in agenda:
                if a['status'] == 'complete':
                    elapsedSeconds = a['timecomplete'] - a['timestart']
                    elapsedSeconds_avg += elapsedSeconds
            elapsedSeconds_avg /= numCompleteTasks
            return datetime.timedelta(seconds=elapsedSeconds_avg)
        else:
            return None


    def calculateETA(self, now=None, force_update=False, remaining=False):
        # if not running, no need to calc an ETA
        if self['status'] != 'running':
            return "--"

        # if we can't figure an average frame time, no need to calc ETA
        avgFrame = self.calculateAvgFrameTime()
        if not avgFrame:
            return "--"

        # Use either Tasks or Subjobs if no tasks
        if self['todo'] > 0:   # Use Tasks
            itemField = 'agenda'
            tallyField = self['todotally']
        else: # Use Subjobs: No agenda items, so must just have subjobs
            itemField = 'subjobs'
            tallyField = self['cpustally']
    
        if tallyField['complete'] > 0 and tallyField['running'] > 0: # has both complete items and is running
            if not now:
                now = datetime.datetime.now()
            # Multiply pending by frame average
            eta  = avgFrame * tallyField['pending']
            eta += avgFrame * tallyField['running']
            # Reduce by running frames elapsed time (cap at avg frame time)
            agenda = self.get(itemField, [])
            for a in agenda:
                if a['status'] == 'running':
                    runningDuration = now - datetime.datetime.fromtimestamp(a['timestart'])
                    eta -= min(runningDuration,avgFrame)
            eta /= tallyField['running']
            now -= datetime.timedelta(microseconds=now.microsecond)
            eta -= datetime.timedelta(microseconds=eta.microseconds)

            if remaining:
                return eta

            eta_datetime = now+eta
            if eta_datetime.date() == now.date():
                return eta_datetime.time()
            return eta_datetime
        else:
            return "--"


Important Note:  As I mentioned, this is an expensive operation.  You DO NOT want to do this for every job on your farm - doing so could quite possibly stop the farm while it is working (remember, you're getting numberJobs * numberFrames records - each of which needs to be retrieved, serialized, and shipped by the supervisor, which is also doing things like job prioritization and dispatch).

« Last Edit: January 27, 2014, 05:43:23 PM by BrianK »

lcatd

  • Jr. Member
  • **
  • Posts: 2
Re: python API for "Time to finish"
« Reply #2 on: February 08, 2014, 05:22:15 AM »
Thank you very much. I'll keep the note in mind  :)