Author Topic: Is it possible to allow more than one machine to process a single job ??  (Read 8178 times)

upperstorey

  • Full Member
  • ***
  • Posts: 13
My company did some initial testing on the Qube! Software but we encountered some problems. More specifically, when trying to process a job, we were unable to process it using 3 of the machines at once. What I means is that only the local subjob will start, the remote subjob will not. I am using one Qube! Supervisor and 3 Qube! workers. These Qube! Supervisor are group in a group called 'Orion3d'. Did a pbping, license still valid (5 licenses), qbhosts shows all host active. To my understanding, this is possible right ?/










Any help will be appreciated.

anthony

  • Senior Software Engineer
  • Hero Member
  • *****
  • Posts: 183
Re: Is it possible to allow more than one machine to process a single job ??
« Reply #1 on: February 01, 2006, 11:04:16 AM »
Hey upperstorey,

      Before I can help you, I'll need to know a little more information on your environment.  Studying the screen shots, I'll assume that the workers you're using are all windows based workers. 

     How is user authentication handled between the workers?  do you have a Primary Domain Controller? or do you use local accounts on each host?

      How are you planning to share file data between the workers?  are you using a windows file server? If so, are you planning on using UNC paths? or do you prefer using mounted drive letters?

      If you have any questions for us, please don't hesitate to call or toll free number: 1-866-856-qube (7823) or you can email support@pipelinefx.com.  Also please feel free to continue posting support questions here on the forum if that's what you prefer.

      Thanks,
           Anthony

upperstorey

  • Full Member
  • ***
  • Posts: 13
Re: Is it possible to allow more than one machine to process a single job ??
« Reply #2 on: February 01, 2006, 11:17:20 AM »
yup.........i am using a primary Domain controller for user accounts.......

Using NTFS,

Shared files by using declaring folders as share, Both UNC and mount drives are find by me.

To my understand, if i started a job in one of the workers, say Orion1 , the job will be shared by all the workers in the group (orion1, orion2, orion3), where each worker wil be assigned a subjob of the process, am i right ???


anthony

  • Senior Software Engineer
  • Hero Member
  • *****
  • Posts: 183
Re: Is it possible to allow more than one machine to process a single job ??
« Reply #3 on: February 01, 2006, 12:47:48 PM »
Hey upperstorey,

      Yes that's correct.  The workers will each be assigned a subjob.  However this depends on a few factors.  Such as subjob failure as well as host availability.  Assuming each worker host is functioning properly, qube! will distribute a subjob to each host.

      Thanks for the information, I'll describe how you'll want to setup your farm:

      Because you're using a PDC (which is the best windows studio environment) you'll probably want output files owned by the job to be under that job user's permissions. In this environment you'll want to make 2 simple changes.  1 on the worker and 1 on the client.  You'll also be required to register your account with the supervisor.

      First, goto each worker and open up the "Configuration" tool under Start->Programs->Pipeline FX->qube! 4.0

      You'll want to modify the worker's configuration to run all jobs using the real user.  To do this, select the "worker" tab and then select the "proxy" settings tab.  In this dialog, switch the proxy execution mode from "proxy" to "user" then click on "OK"  allow the tool to restart the worker. 

      Second, on the host you're submitting your jobs from, you'll want qube! to detect the mounted directories and duplicate them on the workers.  Again, open up the Configuration Tool on the client host this time.  Select Client Settings and look for the "Auto-Mount" feature.  This should be turned on.

      Now that your client and workers are all setup, the last step is to register you're user's account with the supervisor.  To do this, login as the user you plan to submit your jobs from.  Run the login tool:  Start->Programs->Pipeline FX->qube! 4.0 

      This tool will prompt you for your domain password.  (This is only required every time you change your password.) Once you've finished inputting it, submit a new job into the queue.  By either submitting the job from maya, or by using qubic!

      If you're still having trouble, it helps to get the following information:

            The worker's log file:  C:\Program Files\pfx\qube\logs\workerlog.*
            The job's detailed information:   run command line:   qbjobs -l <jobid>

      The worker's log will contain information on login attempts and also will list issues with mounting/unmounting drives.

      Thanks,
           Anthony

upperstorey

  • Full Member
  • ***
  • Posts: 13
Re: Is it possible to allow more than one machine to process a single job ??
« Reply #4 on: February 02, 2006, 04:33:30 AM »
Hi i tried a test job again and this is the error i got



Just did all the configurations u told me to, still the same.

upperstorey

  • Full Member
  • ***
  • Posts: 13
Re: Is it possible to allow more than one machine to process a single job ??
« Reply #5 on: February 02, 2006, 04:51:09 AM »
More pictures of my configuration and errors.


anthony

  • Senior Software Engineer
  • Hero Member
  • *****
  • Posts: 183
Re: Is it possible to allow more than one machine to process a single job ??
« Reply #6 on: February 02, 2006, 05:17:28 AM »
Hey upperstorey,

     Taking a look at the errors you're getting from the printenv job, I don't think printenv is a valid command in windows.  You should use:

       qbsub --flags host_list set

      The job above will submit and guaranteed run once per host on your farm.  The subjobs which fail indicate which hosts are having configuration problems. 

      Thanks,
          Anthony

     

upperstorey

  • Full Member
  • ***
  • Posts: 13
Re: Is it possible to allow more than one machine to process a single job ??
« Reply #7 on: February 02, 2006, 08:37:37 AM »
Hi, i tried your method and it was able to run it successfully with all 3 workers processing and complete it. However, i did another test using Qubic, i tried submitting a sample Maya job and for some reason, only my local Worker is able to process it.. The other 2 remote workers can't process the subjob. This is the screens...



And the error message i got is this

'Prototype mismatch: sub Socket::TCP_NODELAY () vs none at C:/Perl/lib/Socket.pm line 395'

Is there a way for me to process my job using cmd line for my file stored in D:\Work\Maya\Nokia\MNK019_MAGNUM\Animation\MAG_ANI_02_SCs\scenes\test.ma?? I read the .pdf manuals for cmd line usage but was confused by it. Thanks very much for ur help.
« Last Edit: February 02, 2006, 08:48:47 AM by upperstorey »

anthony

  • Senior Software Engineer
  • Hero Member
  • *****
  • Posts: 183
Re: Is it possible to allow more than one machine to process a single job ??
« Reply #8 on: February 02, 2006, 10:52:51 AM »
Hey upperstorey,

      At this point, I'm going to need to study a few settings files as well as get more information.  Could you send me these files?  It will probably be easier if you email them to support@pipelinefx.com.

     On the host Orion3d2 I need these files:

            C:\Program Files\pfx\qube\logs\workerlog.<somenumber>
            C:\windows\qb.conf

     On your client host, please send me this file:
 
            C:\windows\qb.conf


     It would also be helpful to get the output from this command:

             qbout 1068.1
             qbout 1068.2

    As well as
         
             qbjobs -x 1068

     Thanks,
             Anthony