export_environment exported functions and .bashrc overriding exports

[MartinCollins]

Hi,

Setup:
  qube 6.2.0 on dispatch/worker
  qube 5.5 on qmanager

First Problem:

Our environment uses exported functions (environment modules in particular but its more a generic thing) - when submitting a job with the export_environment flag these are not carried forward producing syntax errors to stderr

with a clean shell:

# qbsub --flags export_environment env
32769
# cat 32769_0.err
#


# foo() { echo "bar"; }
# export -f foo
# qbsub --flags export_environment env
32770
# cat 32770_0.err
-sh: foo: line 1: syntax error: unexpected end of file
-sh: error importing function definition for `foo'

If your jobs shell is /bin/bash it sources /etc/profile.d/modules.sh and configures the environment for it everything will continue to work as it should for us at least but still get the stderr messages as it tries to set the function - however doing this spawns (an existing problem)...

The Second Problem:

Environment variables set in ones .bashrc override the variables set in the job. This one causes us a few more problems.

~/.bashrc contains:

export NUKE_VER=6.2.4

# ./qbsub --hosts lon3dwks12 --flags export_environment,elite --shell /bin/bash env

Job's Properties shows env: {...,'NUKE_VER': '6.2.4',....}

stdout includes NUKE_VER=6.2.4

correct - continuing

# export NUKE_VER=5.2.3
# ./qbsub --hosts lon3dwks12 --flags export_environment,elite --shell /bin/bash env

Job's Properties shows env: {...,'NUKE_VER': '5.2.3',....}

correct

stdout includes NUKE_VER=6.2.4

incorrect, it has been overridden by .bashrc.

using  --shell /bin/sh naturally avoids this problem by not sourcing bashrc, but without that, the module functions don't work (first problem).

The only work around i can think of currently is if the first problem is fixed, then we can continue to use /bin/sh instead (which is actually still bash) and then we wont encounter the 2nd problem.

Another is asking all our wrappers around software (written in bash) to source the module functions so they can be used - somewhat less ideal but a workable solution!

If anyone can think of any work arounds/fixes, that would be rather useful.

Without more digging (will be tomorrow morning) I am currently assuming it only sets environment variables that aren't already set after spawning the shell (so its more a case of dispatch environment variables not overriding existing environment variables after the shell)

so:

Dispatch: MAYA_VER=2012
Destination: no MAYA_VER
Result: MAYA_VER=2012

Dispatch: MAYA_VER=2012
Destination: MAYA_VER=2011
Result: MAYA_VER=2011

This isn't really ideal, but makes sense in the case of HOSTNAME (which also follows the above behaviour)..

Thoughts ?

Thanks,
Martin

[MartinCollins]

After thinking about it for a moment, I have decided to just use a custom shell that exports the module() function we need and then pass though the arguments to bash nicely.

I still get the stderr messages about exported functions which would be nice to see go away but at least everything seems to be working as it should.

Thanks,
Martin

[MartinCollins]

Hi,

Sorry for another post, just noticed, it does error quite a bit over the environment in the GUI causing a rather long error output stalling the gui with the environment from the SQL for each job dispatched with an exported function.

Malformed env var in parsing: [['}']]
JobEnv from SQL:
......
module=() {  eval `/usr/bin/modulecmd bash $*`
}
.......

Regards,
Martin

[shinya]

Hi Martin,

It seems that our system has issues when an environment variable spans more than one line, although the issue at the job's run-time vs the GUI may be unrelated.

I was also able to reproduce #2, but with a .profile instead of .bashrc.  The latter has no effect in my environment-- what's your platform/OS?

In any case, I'll file these as formal bugs, but it's hard to give a clear ETA for the fixes at this point.  Am I correct in assuming that you have a workaround with the "module()" being in its own script/file? 

Thanks,
-shinya.