Author Topic: Hostorder by custom CPU-ranking  (Read 1890 times)

Alexander

  • Jr. Member
  • **
  • Posts: 4
Hostorder by custom CPU-ranking
« on: May 29, 2016, 04:27:40 PM »
Maybe this can help someone.
Scenario: You want to have the hostorder based on cpu. Ex Arnold-render etc

The job properties
Host Order: +host.comprank

qbwrk.conf on supe:
add this to each worker
worker_properties = "host.comprank=<INSERT NUMBER HERE>"

To find the comprank number I made a little python script. Not the prettiest code... :-)
Code: [Select]
import multiprocessing, cpuinfo, decimal, socket
host = socket.gethostname()
info = cpuinfo.get_cpu_info()

speed = info['hz_actual']
speed = speed[:-4]
speed = float(speed)

amount = multiprocessing.cpu_count()
rank = amount * speed
rank = decimal.Decimal(str(rank))
rank = int(round(rank,0))
line = host + ' ' + str(rank)

f = open('\\\\SERVER\\\\qube\\ranking\\ranking.log','a')
f.write(line + '\n')
f.close()

edit \\\\SERVER\\\\qube\\ranking\\ranking.log to where you want it saved.

module cpuinfo can be found here:  https://pypi.python.org/pypi/py-cpuinfo

Run that script on each machine, one at the time.

This script could probably be done so that it writes direct to the qbwrk.conf. ;-)

Enjoy
« Last Edit: May 29, 2016, 04:30:29 PM by Alexander »
"Ill put a dropbox installer on my dropbox, so I have a dropbox installer when I need to install dropbox"

BrianK

  • Hero Member
  • *****
  • Posts: 107
Re: Hostorder by custom CPU-ranking
« Reply #1 on: June 01, 2016, 09:30:05 PM »
This is an excellent tip.

For what it's worth, there also exists +host.processor_speed and +host.processors.avail.  You can combine them in a host_order like so:

+host.processors.avail,+host.processor_speed

... which would mean, "Choose the machines with the fastest processors AND the most number of currently available processors (e.g. those processors not reserved by other jobs)".

Having said that, Alexander's suggestion is also perfectly valid and some may prefer it, as it is somewhat more concrete than the multi-key host order I've suggested, which is a bit too "black-box" for some.

Thanks for the tip, Alexander!


Alexander

  • Jr. Member
  • **
  • Posts: 4
Re: Hostorder by custom CPU-ranking
« Reply #2 on: June 02, 2016, 08:09:32 AM »
This is an excellent tip.

For what it's worth, there also exists +host.processor_speed and +host.processors.avail.  You can combine them in a host_order like so:

+host.processors.avail,+host.processor_speed

... which would mean, "Choose the machines with the fastest processors AND the most number of currently available processors (e.g. those processors not reserved by other jobs)".

Having said that, Alexander's suggestion is also perfectly valid and some may prefer it, as it is somewhat more concrete than the multi-key host order I've suggested, which is a bit too "black-box" for some.

Thanks for the tip, Alexander!

Thank you for the kind words!

Question about "+host.processors.avail", if the user has designer-license, will that one work or will it just think that the worker has 1 cpu?
"Ill put a dropbox installer on my dropbox, so I have a dropbox installer when I need to install dropbox"