Author Topic: prman netrender in qube  (Read 10098 times)

kevinc

  • Jr. Member
  • **
  • Posts: 3
prman netrender in qube
« on: July 06, 2010, 12:02:25 PM »
Hi,
i'm currently evaluating qube and would like to be able to submit
RenderManProserver netrender  jobs to qube.
The difficulty i am having is that i need the supervisor
to pass a list of hosts to a command that runs on the submission host.

So the command i want to run on the submission host is something like this:
netrender -h worker1 -h worker2 -h worker3 file.rib

The workers would then be reserved for this render until the job on the
submission host has finished.

can you point me towards any similar sorts of examples?

kevin

jburk

  • Administrator
  • *****
  • Posts: 493
Re: prman netrender in qube
« Reply #1 on: July 06, 2010, 05:04:35 PM »
There are several non-trivial hurdles involved in getting netrender to work with anything other than Alfred.

It's possible to integrate netrender into qube, but it involves a fair amount of development.  I had a written a framework that would do this at a previous employer, but the code doesn't belong to Pipelinefx, so I can't share it with you.

What you need is a jobtype that gets started on the execution hosts that doesn't do anything other than "park" on the job slot, and that communicates back to the Netrender app on the submission host.  These jobtypes will run until the netrender app either explicitly releases the workers or exits (exit is detected via a heartbeat between netrender and the qube job).  At this time, the worker detects that the job is finished and surrenders the job slot back to the available pool.

The Qube supervisor assigns workers to the job.  The jobs start on the workers, and send their hostname and socket address back to the netrender app.  Then they simply "park" on the slot.

Netrender keeps track of the hosts that have been assigned to it (much like the way it works with Alfred), and builds the "netrender" cmdline accordingly.

One of the first things you'll have to write is a socket-based client/server remote messaging framework.   The server will have to be multi-threaded to handle the multiple and possibly simultaneous incoming connections from the worker in a non-blocking fashion.  Both the client and server should use dynamic ports assigned by the OS, to avoid any possible conflicts with existing apps.  Troll though the python "logging" module documentation for a Socket handler, http://docs.python.org/release/2.5.1/lib/network-logging.html.

One server starts up on the submission host and listens for hosts that have been assigned to the job.  The workers start up a client to communicate back to the submission host server, and also start up a server to listen for communication originating from the submission host.  As each new worker is detected by the server on the submission host, it spawns a new client specifically to communicate with the server running on that worker.

So the submission host runs 1 server, and as many clients as machines that have been assigned.  Each worker runs 1 client (to send data back to the server on the submission host), and 1 server (to listen for heartbeats and/or shutdown commands from the submission host).

You can configure the submisssion host to wait until at least some number of workers have been assigned to the job before starting.

"I'm asking for 50 workers.  I'll wait for up to 5 minutes before I start, or I'll start when I have at least 35 workers".  Ask for 50, start as soon as you get 35, or I'll settle for as many as I can get in 5 minutes.

The advantage to this scheme over Alfred is that in Alfred, once the Netrender app starts on the worklist, the hostlist is fixed and can't grow if more hosts become available during the processing of the worklist.  With this scheme and Qube, the netrender app can re-evaluate how many hosts have been assigned after each node in the worklist is processed, allowing for the hostlist to grow in the middle of the worklist.

The jobtype itself is simple.  All it really does is instantiate 1 instance each of the remote messaging client and server.  The job package contains the ip address and port of the submission host's remote messaging server, so the remote messaging client knows where to send it's identifying data.  The hard part is writing the remote messaging client/server framework.

Now all you have to do is tweak the Netrender app the end-users run to work with all this.

« Last Edit: July 06, 2010, 05:10:13 PM by jburk »

kevinc

  • Jr. Member
  • **
  • Posts: 3
Re: prman netrender in qube
« Reply #2 on: July 06, 2010, 05:40:45 PM »
Thanks for the detailed reply.
Could you give a rough estimate of how long it would take to develop such a framework?
Would a month sound reasonable?

kevin


jburk

  • Administrator
  • *****
  • Posts: 493
Re: prman netrender in qube
« Reply #3 on: July 06, 2010, 10:12:53 PM »
It all depends on how fast you are, but a month would be on the fast side.  Plan on 2 months to be conservative to get everything hooked up.

NetRender (the UI, not the cmd-line utility) is rather an opaque beast; it may prove to be the hardest part and the biggest time suck.

kevinc

  • Jr. Member
  • **
  • Posts: 3
Re: prman netrender in qube
« Reply #4 on: July 07, 2010, 08:52:49 AM »
thanks a lot.
kevin