Maya MR License issue in Qube

[thegooch49]

Hi, we are having some issues with some qube jobs getting a license error. The error we see in stderr is:

Error: (Mayatomr) : could not get a license

It seems pretty cut and dry that we exceeded our license count. But I don't really understand how. We are using the MR batch licenses in Maya 2010 that gives you 5 batch per unlimited seat. So we have 95 batch licenses, and only 93 render nodes. I can't see how 93 nodes could cause a problem against 95 seats unless a node is pulling more than 1 license.

Has anyone else seen this or had this problem? I would love some ideas or thoughts if so.

Thanks!

[jburk]

Check the license usage when there are no jobs running for licenses that are still checked out.

Occasionally license fail to get released; license managers usually have a mechanism to force the return of a license back to the unused pool.  Or you can restart the license server service.

[thegooch49]

Thanks for the reply jburk. With no jobs on the farm, there are 0 licenses checked out. Here is another clue. When a big job was running a bit ago, I queried the license server. I caught this problem in the act. The job ran on about 40 hosts. When I queried, I saw that 5 (of 40 hosts) had 2 licenses checked out. Interesting that they are all 1 minute apart from eachother. Same user, same job (there was only 1 running when I queried). Does this help?

msmith   rnode-15.example2.example.com   Thu 4/8 9:59
msmith   rnode-15.example2.example.com   Thu 4/8 9:58
msmith   rnode-03.example2.example.com   Thu 4/8 10:01
msmith   rnode-03.example2.example.com   Thu 4/8 10:00
msmith   rnode-05.example2.example.com   Thu 4/8 9:58
msmith   rnode-05.example2.example.com   Thu 4/8 9:57
msmith   rnode-27.example2.example.com   Thu 4/8 9:58
msmith   rnode-27.example2.example.com   Thu 4/8 9:57
msmith   rnode-65.example2.example.com   Thu 4/8 9:59
msmith   rnode-65.example2.example.com   Thu 4/8 9:58

[thegooch49]

Hi jburk (and others). I have a bit more info here. This 'double license' checkout only seems to happen on renders that are submitted with render layers. So an agenda in qube, would have say 4 render layers. Looking at the license server, it looks like there is a license checkin/checkout at the end and beginning of each render layer. So a single agenda, being rendered on a single host, would have 4 license checkin/checkouts (or one for each layer).

Anyone seen this issue, or have suggestions? I think my next step is to run the render command via CLI to see if it does the same problem outside of qube, to rule it out.

[jburk]

A few of our customers have reported cases where the host are doing double checkouts when the licenses are checked back in and out in rapid succession, as can happen while rendering layers with the Maya jobtype.

A workaround has been to submit your jobs with the "Maya BatchRender (mr)" jobtype instead of the Maya jobtype.

[thegooch49]

Thanks jburk, where can I find this jobtype? I'm on the FTP repo, and I don't see this JT in the Maya or MR directory. Also, we are running 5.4.1. Can I use this JT on this version?

[jburk]

it's built into the GUI, and requires no additional installation on the workers; it submits "cdmrange" type jobs, which are part of the core qube installation.

QubeGui->Submit->MayaBatchRender->MayaBatchRender (mr) Job

and in Maya:

Qube->Submit BatchRender->MayaBatchRender (mr)