Email the image as an attachment on job completion

[bowentw]

Hi all,

This is probably a loaded question, but is it at all possible to attach the job output to an email dependency before sending?

Thanks in advance.

[BrianK]

Hi all,

This is probably a loaded question, but is it at all possible to attach the job output to an email dependency before sending?

Thanks in advance.

You can, but you'll need to create your own callback rather than relying on the built-in email callback.

If you create your own callback, you'll need to submit through the Python API (at least that is true as of Qube 6.5.  It may change in the future).  A Python language callback example can be found here: http://docs.pipelinefx.com/x/VIp

You would, then, need to create and send the email with the callback. There are plenty mail libraries for Python that help manage attachments.

In that example, you will see how to get the job id.  Using the job id in the python script that does the actual mail (that is invoked from the python callback, typically through subprocess.Popen), you can find the image output from the job's frames and either attach them directly or include URLs.

[bowentw]

Thanks for the reply, I'm trying that technique but I can't get that external script to fire. In this instance for testing the external script just writes to a text file. Does anything stand out in this script?

Code: [Select]

#!/bin/env python
 
import sys,os
 
sys.path.append('%s/api/python' % os.environ['QBDIR'])
import qb
 
scriptPath = r'path to script'
 
job = {
    'prototype': 'cmdrange',
    'name': 'supervisor-side script execution callback test',
    'package': {'cmdline': 'hostname'},
    'agenda': [],
    'callbacks': []
}
 
cb = {
'language': 'python',
'triggers': 'done-job-self',
'code': '''
import os

print 'hello!'
script = '%s'
if os.path.exists(script):
pid = os.spawnl(os.P_NOWAIT, script, os.path.basename(script))
'''
% (scriptPath)
}
 
# append the frame's callback to the job's callback list
job['callbacks'].append(cb)

submitted = qb.submit(job)
for job in submitted:
    print 'submitted %(id)s: %(name)s' % job

[BrianK]

That was a bit of a stumper.

The primary problem was that your callback code block ends on an indent.  It should end on line 1 or at the end of a significant line.  Leaving a tab at the end confuses the parser.

Of course, you'll need to provide something in "path to script", though I'm guessing you just left that out when you posted to the web.

The script your callback is calling needs to be executable.  if it is not, you'll need to provide "python" as the executable and your script as an argument.  You might want to use os.spawnlp so you don't have to provide the full path to python.

I've updated the example in the docs (http://docs.pipelinefx.com/x/VIp) to use subprocess.Popen rather than os.spawnl[p].  The example used os.spawn because it was made back when we still supported older Redhat systems that came with python 2.3 (before the subprocess module existed).  

All that said, a working version of your script is here:

Code: [Select]

#!/bin/env python
 
import sys,os
 
sys.path.append('%s/api/python' % os.environ['QBDIR'])
import qb
 
scriptPath = r'/Users/briank/test/testCbScript.py'
 
job = {
    'prototype': 'cmdrange',
    'name': 'supervisor-side script execution callback test',
    'package': {'cmdline': 'hostname'},
    'agenda': [],
    'callbacks': []
}
 
cb = {
'language': 'python',
'triggers': 'done-job-self',
'code': '''
import os,subprocess

script = '%s'
if os.path.exists(script):
    pid = subprocess.Popen(['python',script]).pid
'''
% (scriptPath)
}
 
# append the frame's callback to the job's callback list
job['callbacks'].append(cb)

submitted = qb.submit(job)
for job in submitted:
    print 'submitted %(id)s: %(name)s' % job

edited to add:  While debugging, take note of the .cb file in the job log directory.  This is where thrown exceptions in your code are stored.
Also note that printing anything out your cb['code'] section doesn't print anywhere you can see it.  It's best to write to a file for debug prints.

[bowentw]

Thanks for that, I've been running some more tests to no avail. Even tried writing to a text file for purposes of debugging but nothing is writing. It's almost as if the callback isn't functioning at all?

Initially it appears we didn't have python callbacks enabled on the supervisor. Now the .cb file only states...

executing callback: done-job-self - begin
executing callback: done-job-self - end

Code: [Select]

#!/bin/env python
 
import sys,os
 
sys.path.append('%s/api/python' % os.environ['QBDIR'])
import qb
 
scriptPath = r'C:\Test\dependencyTest.py'
 
job = {
    'prototype': 'cmdrange',
    'name': 'supervisor-side script execution callback test',
'groups' : 'workstations',
    'package': {'cmdline': 'hostname'},
    'agenda': [],
    'callbacks': []
}
 
cb = {
'language': 'python',
'triggers': 'done-job-self',
'code': '''
import os,subprocess,sys

file = open("C:/Test/logfile.txt", "w")
file.write("script fired\n")

script = '%s'

if os.path.exists(script):

file.write("dependency path located\n")
file.close()

    pid = subprocess.Popen(['python',script]).pid
'''
% (scriptPath)
}
 
# append the frame's callback to the job's callback list
job['callbacks'].append(cb)

submitted = qb.submit(job)
for job in submitted:
    print 'submitted %(id)s: %(name)s' % job


[bowentw]

Ok, here's some info. I've been testing with the code below and the error I'm getting is...

Error from job id 26473
No module named subprocess

Is it maybe an issue of the version of python associated with Qube?

Code: [Select]

#!/usr/bin/python
 
import sys,os
import subprocess
sys.path.append('%s/api/python' % os.environ['QBDIR'])
import qb
 
scriptPath = r'C:\Test\dependencyTest.py'
 
job = {
    'prototype': 'cmdline',
    'name': 'python callback test',
    'package': {
        'cmdline': 'hostname',
    },
    'callbacks': [
        {
            'language': 'python',
            'triggers': 'done-job-self',
            'code': '''
try:
import sys
import subprocess
fh = open('c:/temp/err.txt', 'w')
fh.close()
fh = open('c:/temp/foo.txt', 'w')
fh.write('Hello from job id %s\\n' % qb.jobid())
fh.write('sys.version : %s\\n' % sys.version)
fh.write('sys.version info : %s\\n' % '.'.join([str(x) for x in sys.version_info]))
fh.write('sys.executable : %s\\n' % sys.executable)
fh.close()

except Exception, e:
    fh = open('c:/temp/err.txt', 'w')
    fh.write('Error from job id %s\\n' % qb.jobid())
    fh.write('%s\\n' % e)
    fh.close()
'''
        }
    ]
}

 


submitted = qb.submit(job)
for job in submitted:
    print 'submitted %(id)s: %(name)s' % job

[bowentw]

More info, I also get...

No module named os

...when importing os WITHIN the callback. I can import these modules fine running python scripts on the supervisor, but not when they're part of callbacks.

[BrianK]

More info, I also get...

No module named os

...when importing os WITHIN the callback. I can import these modules fine running python scripts on the supervisor, but not when they're part of callbacks.

Which version of Qube are you running?

Also remember you always need to fh.close().  In one of the examples above, you're only fh.close()ing if os.path.exists.

As another form of debug, write out cb['code'] to a file, then try to execute that file.

[bowentw]

We're running 6.5, I'm getting a feeling maybe the Python API didn't get updated?

If I run a print(sys.version) in the callback it says....

sys.version : 2.5 (r25:51908, Sep 19 2006, 10:05:36) [MSC v.1400 64 bit (AMD64)]
sys.version info : 2.5.0.final.0


In answer to your other question, writing out the callback code and running that file works fine.

[BrianK]

We did some more testing on the Windows side.  There is a solution, but it's a bit of a run-around, I'm afraid.

You'll need to install python 2.5 on your supervisor.  Python's word size should match that of the supervisor (so if you're running a 64 bit supervisor, you should install 64 bit python).

Then you'll need set PYTHONPATH for that version of python (or modify sys.path in your callback code, e.g. sys.path.insert(0, "C:/Python25/Lib")).

At that point, you should be able to "import os" and/or "import subprocess".

This issue will be addressed at some point in the future.  Until then, the above is the only work-around.

[bowentw]

Great! sys.path.insert did the trick! Thanks!

I knew it was a loaded question.....