Author Topic: Setting up a Scoped Resource  (Read 3681 times)

ja

  • Jr. Member
  • **
  • Posts: 2
Setting up a Scoped Resource
« on: February 24, 2011, 12:33:20 AM »
Hi all,

I am trying to limit the number of jobs of a type that can run at once per node.

In qb.conf I have set:
supervisor_global_resources = scoped.rvio=25

In qbwrk.conf I have set:
worker_resources = scoped.rvio=5

On each job submission I set:
reservations = scoped.rvio=1

If I'm understanding correctly, this should mean a total of 25 rvio jobs can run at once, only 5 rvio jobs can run on each node at once, and each job consumes 1 rvio resource.  Is this correct?

The problem I am having is when I submit a single job, the state stays pending with a status of:
status: pending (scoped resource scoped.rvio unavailable)

Any idea why this is happening?  Again there is only 1 rvio job submitted so there should be a free resource.

Thanks.

BrianK

  • Hero Member
  • *****
  • Posts: 107
Re: Setting up a Scoped Resource
« Reply #1 on: February 24, 2011, 03:49:30 AM »
Hi Ja,

  You've got it just about right & your logic is sound.  I believe this problem is coming from the qbwrk.conf file.  If the one line is all that's in qbwrk.conf, then you're missing the necessary section name (a label between square brackets "[" and "]").

The short answer is this:
Use one of the special platform section names for the header: "[winnt]" "[osx]" or "[linux]".  Insert the correct one above your worker_resources line and the push via "qbadmin worker --reconfigure".  In your example, the file would look something like this for a mixed OSX/Linux farm:


[osx]
worker_resources=scoped.rvio=5

[linux]
worker_resources=scoped.rvio=5



A little more info:
Beyond the special section names, you can specify resources based on hostname (or hostname ranges) to get very granular with configs.  It's a little beyond the scope of this post, so I'll point you to the Administration Guide, section 4.2.4 for a detailed explanation of how section names, inheritance and variables work. (The admin guide can be found most easily via the Qube GUI under the Help pulldown menu)

Once your qbwrk.conf file is complete to your liking, you can check it with the command line utility "qbconfigfile" which is located in Qube's sbin dir, next to qbusers, qbadmin, etc. (not to be confused with Qube's bin dir which has qbjobs, qbhosts, etc).  You use the utility like so:


qbconfigfile /path/to/qbwrk.conf


In your example, w/o a section name, you will see reported back something like


[localhost]
worker_resources = scoped.rvio=5


which means that resource directive is not going to any machine.  You can confirm that no workers picked up the resource by either going to the Farm Layout tab in the GUI, clicking a node, and checking the "Resources" section under the "Properties" tab, or by running "qbhosts -l <hostname>" and looking at the Resources section of the return.

If you include a section name, say "OSX" in my case, as I'm on a Mac at the moment, and run the qbconfigfile util as before, you would see:


[osx]
worker_resources = scoped.rvio=5


which looks correct & means that this resource rule is going to be pushed to all machines running OSX.  At that point, you would distribute the settings via "qbadmin worker --reconfigure".  Checking the properties and/or qbhosts for the host in question will show an entry under Resources that looks something like:


   Resources:
      host.cores=0/2
      host.processors=0/4
      host.memory=1308/2048
      scoped.rvio=0/5
      host.swap=0/0


At that point, your scoped resource should be working as you would expect.
« Last Edit: February 24, 2011, 03:53:02 AM by BrianK »

ja

  • Jr. Member
  • **
  • Posts: 2
Re: Setting up a Scoped Resource
« Reply #2 on: February 24, 2011, 07:07:20 PM »
Thanks for the indepth reply.  I'm glad to know I'm at least going down the correct path.

Right now the qbwrk.conf file is setup with section specifying the options for each node.
For example

[node1]
worker_resources = scoped.rvio=5

Does it matter specifying the node name instead of the OS?

Thanks again.