Author Topic: PipelineFX releases QubeMobileServer 0.9.1 beta  (Read 7926 times)

BrianK

  • Hero Member
  • *****
  • Posts: 107
PipelineFX releases QubeMobileServer 0.9.1 beta
« on: October 15, 2012, 10:19:40 PM »
A new beta release of QubeMobileServer has been made available.  The new version, 0.9.1, comes with the following fixes and enhancements:

==== CL 10156 ====
@FIX:Better documentation in README.txt file
@TWEAK:Better startup messages
@FIX:Less communication with supervisor for improved performance
@FIX:Non-agenda based jobs now properly report percent complete
@FIX:Fixed bug that would cause self-signed certificates to be generated at every launch
@FIX:Allow Windows to run in ssl mode if a certificate has already been generated

==== CL 10138 ====
@NEW:QubeMobileServer is now threaded for better performance.
@FIX:Use OS-specific temporary storage if the given directory for certificates and/or sessions does not exist or is not writable.
@TWEAK:Added a favicon to reduce errors when connecting from non-mobile devices
@FIX:Fixed bug in QubeMobileServerUserAdmin that prevented it from starting in some instances

==== CL 10053 ====
@NEW:Shipping as both source and compiled binary, both of which have customizable templates.  Linux rpms now available as well.
@NEW:Configuration moved into a qb.conf-like configuration file.  All config settings can be overridden on the command line.


You can find this version of QubeMobileServer on our ftp site under /pub/beta/qubeMobile/0.9.1/

------------------------------------------------------------------------

                         Qube! Mobile Server


I.   Overview
II.  Quick Start
III. Requirements
IV.  Startup / Configuration options
V.   Operation Overview / Examples
VI.  User Management
VII. Modification
VIII.Support


I. Overview

QubeMobileServer is a simple, cross-platform python webserver that is
purpose-built to display Qube! jobs over http.  It can run over SSL,
require authentication and allow users to control their job through
the web interface (though each of those features can be toggled off if
the admin chooses to do so).

To note: Performance on Windows machines has proven to be quite slow.
Linux and OS X machines run qubeMobileServer *much* better than
Windows.


II. Quick Start

To run the server, simply run qubeMobileServer[.py] with your python
interpreter.  The [internal] URL to connect will be displayed when the
server starts.  Simply point a web browser at that URL to see your
jobs.  Start the server with a '-h' or '--help' command line argument
to see a list of startup options.


III. Requirements

Requirements for QubeMobileServer as a python script (in interpreted
mode):
1. Python 2.6 or newer (http://python.org) to run with ssl. Python
2.4 or newer is required if not running over ssl. The server is
100% python.

2. [optional] PyOpenSSL (https://launchpad.net/pyopenssl).
QubeMobileServer can be run over SSL for added security.  If the user
does not already have a signed SSL certificate, QubeMobileServer will
create a self-signed certficate for its own use.  Doing so requires
PyOpenSSL.  PyOpenSSL is compiled into the pre-built binary for Linux
and OS X, but not Windows.  Regardless of the OS, you can generate your
own certficate outside of qubeMobileServer with the following command:

openssl req -new -x509 -days 365 -nodes -out cert.pem -keyout key.pem

Doing so will be necessary for those who are running the server on
Windows, choose to run the server over ssl, and do not already have a
signed certificate.

3. [supplied] Mako Templates (http://www.makotemplates.org/). The
actual web pages are rendered with Mako templates.



Requirements for QubeMobileServer binary:
None.  All required packages are included.  PyOpenSSL is not included
with the Windows binary, so Windows machines cannot generate their own,
self-signed certificates.  Windows machines may generate their own
certficates outside of qubeMobileServer if they choose.  See III.2 above.



IV. Startup / Configuration options.

For this section, most files are relative to the qubeMobileServer
executable (be it the python script or compiled binary).  It follows,
then, that "./" is the location of qubeMobileServer.

The following options control how QubeMobileServer runs.  They can be
set either as command line options (run the server with '-h' or
'--help' to see this list and how to set the options) or they can be
set in the configuration file: ./config/qbms.conf.

- settings-file: The location of the qbms.conf settings file.  By
                 default, it is in the conf directory next to the
                 qubeMobileServer executable.

- ssl: Tells the server to use SSL, SSL requires public and private
       key/certificate files.  If the user has these files for
       another purpose (or has purchased one for this purpose),
       they are welcome to use those files by telling
       QubeMobileServer the location of said files in startup
       options or in the config file.  If certificate and key files are
       not available, QubeMobileServer will generate self-signed keys so
       SSL can be used.  Note that because self-signed keys are not
       verified by a known certificate authority, they will be
       considered suspect by all browsers & ask the user if the
       certificate is trusted before the site can be viewed.
       Inform your users that the certificate can be trusted & by
       doing so, all communication between the device and this
       server will be encrypted.  Note that when running in ssl
       mode, the suer must access the server via https://. 
       Attempting to use http:// will result in failure.
       Default is to use SSL.

- no-ssl: Tells the server to NOT use SSL.  Note that when the
             server is not running over ssl, the user must access
             the server via http://, *not* https://
     Default is to use SSL.

- port: The port on which the server will run.  
     Default: 8080

- jobs_in_cache: The number of jobs held in memory by
           QubeMobileServer. The server will never get more than
           the last n jobs from the supervisor.  When filtering
           results, filtering happens *after* the jobs are
           retrieved from the supervisor. A smaller number here
           will improve performance on the device but may mean
           that some of the device users' jobs will be truncated
           from the list of viewable jobs on very busy farms.
       Default is to hold the 200 jobs in memory cache.

- refresh_interval: The number of seconds of operation after which the
              data in memory is considered stale, causing the
              data in memory to be discarded and renewed from
              the supervisor.
          Default is 3600 seconds (1 hour)

- certificates_dir: Directory that will hold self-signed
            certificates. This will be used if the server runs
            over SSL and the user has not specified a
            certificate file.
          Default: "./certs"
          Note: If the running user cannot create or write to
            ./certs, OS-specific temporary storage will be used.

- certificate_file: Full path to a signed certificate file.  This is
            optional.  If not given, the server will create
            its own, self-signed file in certificates_dir.
          Default: None

- key_file: Full path to a signed key file.  This is optional.  If not
         given, the server will create its own, self-signed file in
         certificates_dir.
       Default: None

- sessions_dir: Directory that will hold session information so
        authentication and device-made options can persist
        across restarts.
      Default: "./sessions"

- authentication: Tells the server to require password
        authentication. Password authentication requires the ssl
        module.  If you do not have python >=2.6, you cannot
        import ssl & therefore cannot use passwords.  With
        passwords turned off, a device user will still be able
        to filter out any single user's jobs, but will not be
        allowed to control any user's jobs.  Password
        information lives in a single file database - the
        location of which is configurable, with the default
        location being in the current directory.  The password
        database is obfuscated and does not contain any
        clear-text passwords, so it should be no less secure
        than, e.g., /etc/shadow.  Users administration is
        performed through the utility
        qubeMobileServerUserAdmin[.py].  See that program's help
        screen for more info.
      Default is to use authentication.

- no-authentication: Tells the server to NOT require password
        authentication.
      Default is to use authentication.

- password-file: The location of the local user database.  This file
                 will only be used if authentication is used.  The
                 password file can be edited while the server is running
                 by use of the qubeMobileServerUserAdmin program.  See
                 that program's help screen for more info.
       Default: "./.passwd"

- template-dir: The path to the directory that holds the html
                tempaltes.
      Default: "./qubeMobile/view"

- allow-control: Gives device users control over their jobs through
         the device interface.  This option has two
         requirements:
       1. Passwords (and therefore SSL) must be used so we
         ensure we're giving control to the correct user.
       2. QubeMobileServer is started by a Qube!
         administrator.
       If those two requirements are met, then for jobs
         owned by the logged in user (when viewing the queue
         on a device), "Kill" and "Retry Failed" buttons will
         be displayed on the job details page.  If the
         usernames do not match, the buttons will not be
         displayed.
       Default is to NOT allow control.





V. Operation Overview / Examples

Typically, one would start the server by simply running:

./qubeMobileServer[.py]  (or /path/to/python qubeMobileServer.py)

Doing so assumes you have ssl, PyOpenSSL (or a signed certificate) and
Mako templates available. Mako templates ship with QubeMobileServer &
are located in the root directory of the distribution (next to
qubeMobileServer[.py]).  ssl ships with python, starting with version
2.6.

If you want to give users control over their jobs when viewing jobs on
the device, launch with the following option:

./qubeMobileServer[.py] --allow-control

When you start the server, it print out a message that gives a URL
that can be used to reach the server.  This is not necessarily the
only URL that will work.  All requests that go to this machine and the
specified port should result in success, the only caveat being that
servers started with SSL *must* use https:// whereas servers started
without SSL *must* use http://.

To run a server on a Windows machine in a facility that does not already
have a signed, ssl certificate file:

./qubeMobileServer[.py] --no-ssl


VI. User Management

QubeMobileServer uses its own user management.  While it may be
possible to tie into your existing authentication system, doing so is
unsupported and will require modifying the source code.

Next to qubeMobileServer[.py], you will find
qubeMobileServerUserAdmin[.py].  This utility allows you to add, remove
and list current users.  To change a user's password, simply re-add
them to the database.  Once users are added, a username and hashed
password will be added the to password file.  By default, this file is
.passwd in the root of the distribution package.  The file can be
moved either by changing the config file or giving a command line
argument.  If the latter, the same argument must be used for starting
the server or running the user admin utility.  See the help screen for
qubeMobileServerUserAdmin by giving the '-h' or '--help' command line
argument.


VII. Modification

All html pages come from qubeMobile/view.  In that directory, you will
find html templates that are rendered by Mako.  Changing the templates
will change the look and content of the data that is displayed on the
device that connects to QubeMobileServer.  

Data that changes the farm is sent via POST and handled in the
server's do_POST method.  If you choose to add more functionality by
way of an additional button in a template, you must also change the
do_POST method to accommodate the new functionality.


VIII. Support

You can find support either through the PipelineFX forum or via our
support system by writing in to support@pipelinefx.com.  If you write
into the support system, be sure to include the company with which you
work so that we may better track the support request.



#######################################################################
#
# Licensed Materials - Property of Pipelinefx L.L.C.
#
#
# (C) COPYRIGHT Pipelinefx Limited Liability Corporation.
#  All Rights Reserved.
#                              
#  US Government Users Restricted Rights - Use, duplication or
#  disclosure restricted by GSA ADP Schedule Contract with
#  Pipelinefx L.L.C.
#
# QubeMobileServer
#
# Written by: Brian Knudson / PFX
#             briank@pipelinefx.com
#             2012-06-04
#
# Description:
#  Python-based web server, designed specifically for qubeMobileView
#
# Notes:
#
#######################################################################


Mako templates are distributed under MIT license.
http://www.opensource.org/licenses/mit-license.php

Copyright (C) 2006-2012 the Mako authors and contributors <see AUTHORS file>.
Mako is a trademark of Michael Bayer.

Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the "Software"), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or
substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.

« Last Edit: October 16, 2012, 09:30:41 PM by BrianK »

BrianK

  • Hero Member
  • *****
  • Posts: 107
Re: PipelineFX releases QubeMobileServer 0.9.1 beta
« Reply #1 on: October 17, 2012, 05:23:55 PM »
Quick fix to the 0.9.1 is 0.9.1a.  This fix was made to address performance issues on Windows machines.  Along with the Windows fix, comes the following:

==== CL 10175 ====
@FIX:Windows servers will now perform as fast as other OSs (when running without ssl).  This issue was related to DNS, so those with proper FQDN results in DNS may see no difference in performance.
@FIX:Windows defaults to not use ssl.  In fact, it ignores the --ssl option, requiring, instead, --force-ssl if one wants to run the server over secure sockets.
@FIX:Better handling of command line arguments --add and --drop in qubeMobileServerUserAdmin.
@FIX:Better formatting of user list when using the --list option for qubeMobileServerUserAdmin.