Module deployables_lib
[show private | hide private]
[frames | no frames]

Module deployables_lib

Deployables_lib provides functions that simplifies implementation of a Power Tool. Most common tasks in managing a DSP for an Ensim server are available through this API.

To simplify understanding of the API arguments to the functions follow same name convention as given below - Unless specified the arguments have the meaning as given above.
Classes
Msgs  
TranslationWrapper  

Function Summary
  add_slash(url)
Adds trailing slash ('/') to a URL / path if it is not already there.
  apply_instance_patches(site, site_conf, old_site_conf, instance_conf, old_instance_conf, patches)
  archive_manifest(instance_conf, dir_moves, patches, file_mods, file_manifest)
  archive_unpack(msgs, st, site, site_conf, instance_conf, dir_moves, file_mods, patches, fileinfo, context)
Unpacks the archive in directory given by 'instance_conf["install_loc"]'.
  check_accept_lic(msgs, st, conf)
Verifies that the license acceptance ('accept_lic' option in site configuration ) is True.
  check_index(msgs, st, site, dir)
Checks if the index file (index.htm/index.html/index.shtml) exists in given directory inside specified site's filesystem.
  check_mysql_enabled(msgs, st, conf)
Verifies that MySQL is installed and enabled for the site.
  check_output(output, success_re, failure_re)
check_output applies provided success and failure patterns to output.
  chmodfile(path, uid, gid, perms)
Chmods file with specified uid gid and the permissions.
  create_htdigest(msg, st, site, instance_conf, appuser, realm, directory, password)
Creates htdigest file within the specified directory.
  create_htpasswd(msgs, st, site, instance_conf, appuser, realm, directory, password)
Creates htdigest password for the instance.
  crypt_DES_password(password)
Generate a 2-character salt.
  crypt_MD5_password(password)
Generate an 8-character salt.
  db_create(msgs, st, site, dbadmin, dbname)
Creates a database (if it doesn't exist) with name and admin as 'dbname' and 'dbadmin'.
  db_drop(site, site_conf, instance_conf)
Drops MySQL databse used by DSP.
  db_exec_sql(site, site_conf, instance_conf, sql)
Executes given sql query on DSP database.
  db_exec_sql_file(site, site_conf, instance_conf, sql_file)
Opens the SQL file from install location and executes the file.
  directory_merge(srcdir, dstdir, fileinfo)
  disk_quota_exhausted(site, user)
Return status of Disk Quota.
  do_backup(msgs, st, site, site_conf, instance_conf)
Backup files given by instance_conf['backup_files'] and db of the given instance to a backup location given by instance_conf['backup_loc'] .
  do_install_actions(site, site_conf, instance_conf, msgs, st, actions, args_dict)
do_install_actions executes the list of actions required to install a DSP.
  do_reconfigure_actions(context, site, site_conf, old_site_conf, instance_conf, old_instance_conf, msgs, st, actions, args_dict)
do_reconfigure_actions executes the actions required in order to upgrade a DSPI, This function takes the actions as one of the arguments and executes them in order.
  do_upgrade_actions(site, site_conf, instance_conf, old_instance_conf, msgs, st, actions, args_dict)
do_upgrade_actions executes the actions required in order to upgrade a DSPI, This function takes the actions as one of the arguments and executes them in order.
  dsp_create_doc(instance)
  dsp_create_instance(dsp, version)
Returns the instance of DSP i.e a dictionary representing DSP elements (keyed by element names) A DSP description is a dictionary with the following elements:
  dsp_fill_instance(context, instance, site, site_conf, new_conf, old_conf, common_options)
This function takes an instance description (dictionary representing dsp.xml) , a site configuration, a new instance configuration, and optional old instance configuration and list of well-known options, and fills in the various properties of the passed in instance.
  dsp_get_localization(instance_conf, locale)
  dsp_get_theme(instance_conf, theme)
  dsp_install_directory(dsp, version)
Returns Install directory for requested DSP (name and version)
  dsp_lib_directory(dsp, version)
Returns lib directory for requested DSP (name and version).
  dsp_libuser_admin(site)
Helper function for User administration.
  dsp_load_localizations(dsp_desc)
  dsp_load_themes(dsp_desc)
  dsp_locale_directory(dsp, version)
Returns the themes directory -> /usr/lib/ensim/deployables/<DSP>-<version>/lib/locale/
  dsp_source_directory(dsp, version)
Returns source directory for requested DSP (name and version) The source directory is $Install_Dir/lib -> /usr/lib/ensim/deployables/<DSP>-<version>/source
  dsp_theme_directory(dsp, version)
Returns the themes directory -> /usr/lib/ensim/deployables/<DSP>-<version>/lib/themes/
  dsp_versioned_name(dsp, version)
Returns string - <DSP>-<version> for given DSP-name and version
  dump_command(cmd, out)
  edit_conf_file(site, site_conf, instance_conf, file, lang, pat, sub, vars)
Edits a conf file for the provided key, value and comment(optional) triplet tuples in vars argument.
  enable_patches(instance, instance_patches)
Enables patches for the instance.
  exec_cmd(cmd)
Executes the command
  exec_curl_cmd(url, opts)
Executes curl command with requested opts.
  exec_mysql(msgs, st, site, site_conf, instance_conf, sql)
Executes SQL statements given by 'sql'.
  exec_php_script(site, site_conf, instance_conf, exec_dir, script, args, success_re, failure_re, post)
Executes php scripts (given by argument 'script').
  find_free_dbname(prefix, dbname)
Returns a (non-existing) MySQL databse name for DSPI installation.
  find_free_path(site, path)
Find a (non-existing) directory for DSPI installation.
  finish_url(msgs, st, site, site_conf, instance_conf, file)
Constructs the DSPI URL from provided parameters and appends this URL to msgs.
  get_domain_path_pairs(site, site_conf)
Finds subdomains and paths to their publishing(html) directory.
  get_module(dsp, version)
Returns dictionary for a DSP with specified version
  get_mysql_host(site)
Returns mysql host for the site
  get_site_user_ids(site, user, context)
Returns (uid, gid) pair for the user.
  get_site_user_name(site, uid, gid, context)
Find the username and group for specified uid,gid.
  get_site_users(site, site_conf, context)
Get site users for requested site.
  handle_db_changes(context, site, new_siteconf, old_siteconf, new_conf, old_conf)
Rename a DSP MySQL database if the DB prefix is changed.
  handle_security_mode(site, site_conf, instance_conf, files)
Sets file and directory permissions for files based on weather 'jail' is 'True' or 'False'.
  handle_url_loc(context, site_conf, old_site_conf, instance_conf, old_instance_conf)
Computes the new value of instance_conf['url_loc'] if there is a change in site domain (during a site edit).
  handle_username_change(site, old_conf, new_conf)
Corrects the file ownerships of DSP files if the site user is changed.
  install_loc_cleanup(site, site_conf, instance_conf, dir_moves, patches, file_mods)
Recursively deletes all files/directory ( within install location ) which were -
  load_dAPI_module(dsp, version)
Loads and returns ref to the dAPI module of requested DSP
  load_dsp_description_from_file(xml_path)
Reads and converts requested dsp.xml to dictionary format
  load_dsp_description_from_string(xml_str)
Reads and converts requested DSP xml string to dictionary format
  load_msgs_module(dsp, version, symtab)
Load 'msgs' module deployed in the 'lib' directory of requested DSP into the 'systab' system table
  modify_string(site, site_conf, str_or_tuple)
Prefix site fst path to given string(path) if the site is a high security site.
  mysql_installed(conf, old_conf)
Return 1 if mysql service is enabled for the site else returns 0.
  readfile(path, uid, gid)
Slightly paranoid file reader.
  recursive_chown(dir, orig_uid, orig_gid, uid, gid)
This function is intended to provide a safe recursive chown.
  refresh_dsps(dsp, version)
This function is automatically called when the module is loaded.
  regression_check_index(action, index, test_data, string)
regression helper method
  remove_files(site, instance_conf, files)
Recursively removes all files/directories (within install location ) given in 'files'.
  require_url_loc(msgs, st, instance_conf)
Checks if instance_conf has the key - 'url_loc' Adds error 'URL_LOCATION_REQUIRED' to msgs on not finding the key.
  resolve_location(msgs, st, context, site, site_conf, instance_conf)
Given url_net and url_path (in Instance Conf), resolves these two variables to an installation location in the filesystem and a URL location.
  set_htdigest(msgs, st, site, instance_conf, realm, directory)
Edits ".htdigest" to set htdigest for the instance.
  set_htpasswd(msgs, st, site, instance_conf, realm, directory)
Sets htdigest password for the instance.
  shell_escape_arg(arg)
Escapes any 's in the given string so that the given string may be surrounded in 's prior to being used in a command line context.
  site_fs(site)
Returns site file system path from given site.
  site_fs_jail(site, site_conf)
Returns site file system path from given site.
  url_net_sub(url, new_domain, old_domain)
This function replaces the (trailing) portion of the domain name segment of a URL.
  url_path_sub(str, new_url, old_url)
This function replaces the old domainname to new(changed) domainname from given str.
  url_sub(url, new_url, old_url)
This function replaces the entire domain and the starting portion of the path of URLs.
  writefile(path, uid, gid, data)
File writer method that writes given data to specified file path and assigns uid, gid.
  __chmod(perms, dirname, names)
  __chown(id, dirname, names)
  __conf_quote(value, lang)
Add quotes requested string
  __conf_unquote(value, lang)
Unquotes requested string
  __find_conflicts(info, dirname, names)
  __load_dsp_description(doc)
  __rename(info, src_dir, names)

Variable Summary
int CONTEXT_AUTO_UPGRADE = 6                                                                     
dict context_dict = {1: 'install', 2: 'upgrade', 3: 'remove',...
int CONTEXT_INSTALL = 1                                                                     
int CONTEXT_RECONFIGURE = 4                                                                     
int CONTEXT_REMOVE = 3                                                                     
int CONTEXT_SITE_EDIT = 5                                                                     
int CONTEXT_UPGRADE = 2                                                                     
dict dsp_modules = {}
NullTranslations dsp_null_translation = <gettext.NullTranslations instanc...
dict dsp_themes = {'default': {'doc': <xmlDoc (/usr/lib/ensim...
lock libuser_lock = <thread.lock object at 0xb7f0a0f0>
int MODSTR_JAIL = 1                                                                     
int REGRESSION_DO_TEST = 2                                                                     
int REGRESSION_GET_TEST_STRING = 1                                                                     
tuple __supported_types = ('Boolean', 'ExclusiveSelect', 'Mult...
str _cryptsalt = './abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMN...

Function Details

add_slash(url)

Adds trailing slash ('/') to a URL / path if it is not already there.
Parameters:
url - Path / URL eg. /home/virtual
           (type=string)
Returns:
url string with trailing slash.

archive_unpack(msgs, st, site, site_conf, instance_conf, dir_moves, file_mods, patches, fileinfo=None, context=1)

Unpacks the archive in directory given by 'instance_conf["install_loc"]'. It creates the install path (with chmod 755) if it doesn't exists. The supported formats are:
  • tar.gz
  • zip

After unpacking all the 'source' and 'dest' pairs identified from 'dir_moves' and directories are moved accordingly. File or directory specific actions are Directories and files (with some default content) can also be created/deleted with specific ownership and permissions using 'file_mods'. Default permission for directories and files is 0755 and 0644 respectively.

Args:
  • dir_moves - list of tuples. Each tuple contains a source and destination pair for files/directories. All the 'source' files/directories mentioned in 'dir_moves' are moved to 'destination'. And those which are not specified in 'dir_moves' are erased.
  • file_mods - list of dictionaries. Each dictionary contains file/directories creation details. Keys in the dictionaries can be one or more among the following -
    • 'path' : gives file/directory name. Directory is distinguished by a trailing '/'
    • 'create' : Existence of this key initiates creation of the file/directory given by 'path' with permissions and ownership given by 'perms' , 'uid' and 'gid' respectively. The value of this key is taken as default text for the file.
    • 'perms' : Gives file permissions.
    • 'uid' : Gives file owners UID.
    • 'gid' : Gives file owners GID.
If any of the 3 - 'perms' , 'uid' and 'gid' are present and 'create' is not present, the function tries to set permission/ownership of an existing file (if any). But doesn't give any error/warning if the file doesn't exist.
  • patch - list of dictionaries. Each dictionary describes a patch and has following keys :
    • 'patch' : Gives path of the patch to be applied.
    • 'args' : Options to be passed to the patch command. Default options are taken to be '-f -s -p1'
  • fileinfo - dictionary with the following keys:
    • 'overwrite' : indicates if we should overwrite existing files (defaults to 0)
    • 'fail_on_existing_files': indicates if we should check for any matching files that already exist in the destination directory, and fail if we find any (defaults to 1)
    • 'excluded_files' : files that should not be overwritten
    • 'conflicting_files' : out parameter that lists the files found that conflict with the 'fail_on_existing_files' flag
    • 'dst' : internal parameter; don't set
    • 'rootlen' : internal parameter; don't set
  • context - The context in which this function has been called. Default value for context is CONTEXT_INSTALL.

NOTE: If these arguments are not present in args_dict empty objects are passed as defaults.

Errors : Throws Critical errors if any of the following -
  1. Unrecognized Archive
  2. Unable to unpack Archive
  3. Unable to apply patch
  4. Unable to remove unused files.
  5. Unable to create Installation Path
  6. Conflicting files.

check_accept_lic(msgs, st, conf)

Verifies that the license acceptance ('accept_lic' option in site configuration ) is True.
Returns:
Adds ERROR to msgs

check_index(msgs, st, site, dir)

Checks if the index file (index.htm/index.html/index.shtml) exists in given directory inside specified site's filesystem.
Parameters:
dir - Directory to check.
           (type=String)
Returns:
If it exists it appends a warning to msgs. Doesn't stop the execution either ways.

check_mysql_enabled(msgs, st, conf)

Verifies that MySQL is installed and enabled for the site.
Returns:
Adds ERROR to msgs if mysql is not found /enabled / installed.

check_output(output, success_re='', failure_re='')

check_output applies provided success and failure patterns to output. Most useful when you want to process a commands output for certain condition.
Parameters:
output -
           (type=string)
success_re - re pattern
           (type=re pattern)
failure_re - re pattern
           (type=re pattern)
Returns:

1 if pattern does not match with output string

else 0

chmodfile(path, uid, gid, perms)

Chmods file with specified uid gid and the permissions. Permissions should be in in octal format (eg. 07000). Does not return success/failure.

create_htdigest(msg, st, site, instance_conf, appuser, realm, directory, password)

Creates htdigest file within the specified directory.
Parameters:
appuser - Username of the application user. Default value is taken as instance_conf['instance_admin']
           (type=string)
realm - Realm of the instance.
           (type=string)
directory - The directory for creating ".htdigest" file. Default is taken as instance_conf['install_loc']
           (type=string)
password - The password for installation. Default is taken as instance_conf['instance_pass1'] i.e the password supplied for installation (if any).
           (type=String)

create_htpasswd(msgs, st, site, instance_conf, appuser, realm, directory, password)

Creates htdigest password for the instance.
Parameters:
appuser - String representing the username of the application user. Default value is taken as instance_conf['instance_admin']
realm - String representing realm of the instance.
directory - String representing the directory for creating ".htdigest" file. Default is taken as instance_conf['install_loc']
password - String representing the password for installation. Default is taken as instance_conf['instance_pass1'] i.e the password supplied for installation (if any).

crypt_DES_password(password)

Generate a 2-character salt.
Parameters:
password - password
           (type=string)
Returns:
Tuple of password and hashed password

crypt_MD5_password(password)

Generate an 8-character salt.
Parameters:
password - password
           (type=string)
Returns:
Tuple of password and hashed password

db_create(msgs, st, site, dbadmin, dbname)

Creates a database (if it doesn't exist) with name and admin as 'dbname' and 'dbadmin'.
Parameters:
dbadmin - Database admin name.
           (type=String)
dbname - Database name to create.
           (type=String)
Returns:
Appends error messages like MYSQL_DATABASE_LIMIT, DATABASE_CREATION_FAILED or DATABASE_EXISTS on failure of database creation,

db_drop(site, site_conf, instance_conf)

Drops MySQL databse used by DSP. Name of MySQL DB is decided by instance_conf dict['dbname'].

db_exec_sql(site, site_conf, instance_conf, sql)

Executes given sql query on DSP database.
Returns:
Query result

db_exec_sql_file(site, site_conf, instance_conf, sql_file)

Opens the SQL file from install location and executes the file. The file is broken into lines separated by "; " and each line is executed individually.
Parameters:
sql_file - Path of sql file in install location. The path shouldn't start/end with '/'.
           (type=string)
Returns:
Returns 'Critical' Errors (if any)

disk_quota_exhausted(site, user)

Parameters:
user - The username for whom the disk quota needs to be checked.
           (type=string)
Returns:

status of Disk Quota. Return codes are: 0: all clear 1: user quota exhausted 2: group quota exhausted 3: error retrieving quota

The return codes are manipulated by do_install_actions and appended to 'msgs' as appropriate Errors/Warnings.

do_backup(msgs, st, site, site_conf, instance_conf)

Backup files given by instance_conf['backup_files'] and db of the given instance to a backup location given by instance_conf['backup_loc'] .

Errors stop execution of queued actions.

do_install_actions(site, site_conf, instance_conf, msgs, st, actions, args_dict)

do_install_actions executes the list of actions required to install a DSP. It supports a range of actions commonly required in installing a DSP.

Most actions expect some data to perform given task. This data is provided in the form of python dictionary (args_dict) to do_install_actions method.

An args_dict structure looks like:
args_dict = {'action1': 
              {'action1Arg1': action1Arg1Data, 
               'action1Arg2': action1Arg2Data, .. }, 
             'action2': ... }
List of Permitted actions -
  • check_accept_lic
  • check_mysql_enabled
  • resolve_location
  • check_index
  • db_create
  • archive_unpack
  • run_sql_file
  • edit_conf_file
  • handle_security_mode
  • exec_php_script
  • finish_url
  • access_url
  • check_pass
  • check_install_loc_owner
  • db_probe
  • exec_mysql
  • set_php_cgi
  • create_htdigest
  • set_htdigest
  • create_htpasswd
  • set_htpasswd
  • remove_files
  • require_url_loc
  • check_disk_quota
  • set_trace
  • archive_unpack
  • db_create

check_accept_lic

Verifies that the license aceptence ('accept_lic' option in site configuration ) is True. Adds error message to msgs if SA refuses accept the DSP license.

check_mysql_enabled

Verifies that MySQL is installed and enabled for this site.

resolve_location

Given url_net and url_path (in Instance Conf), resolves these two variables to an installation location in the filesystem and a URL location. Adds these variables as install_loc and url_loc to instance_conf.

check_index

Checks if the index file (index.htm/index.html/index.shtml) exists in given directory inside specified site's filesystem. If it exists it appends a warning to msgs. Doesn't stop the execution either ways.

args

  • dir : install_loc directory will be chacked for index files this arg is not specified in args_dict.

db_create

Creates a database table (if it doesn't exist) with name and admin as 'dbname' and 'dbadmin'. The args are not required to be specified for the most of the DSPs.

args

  • dbadmin
  • dbname

archive unpack

Unpacks the DSP source archive.

args

  • dir_moves: List of (src,dst) tuples. src should be the directory/file packed in the archive. archive_unpack renames all src directories/files to dst after unpacking the archive. If dst is an empty string (as in most cases) the unpacked sources would be renamed to install location as specified by SA during install.

run_sql_file

Execules sql statements from given file.

args

  • sql_file : File containing sql statements to be executed for MySQL database.

edit_conf_file

Edits a conf file for the provided key, value and comment(optional) tripplet tuples in vars argument.

args

edit_conf_file actions need following args. Refer edit_conf_file for details.
  • file:
  • lang:
  • pat:
  • sub:
    • vars:

handle_security_mode

Sets the permissions on file as specified in args. We need to pass two sets of (octal)permissions. One is for low and medium security mode while the other is for high security mode sites.

args

  • files: list of tuples. Each tuple has 3 objects (in order) as given below
    • string representing file/directory path
    • Permissions (in octal format) if for high security.
    • Permissions (in octal format e.g. 07000 ) for low/medium security.
  • eg. [('lists/config/config.php', 0600, 0600),('tmp', 0700, 0700)]

exec_php_script

args

  • exec_dir
  • success_re
  • failure_re
  • script
  • args

finish_url

Constructs the DSPI URL from provided parameters and appends this URL to msgs.

access_url

Constructs the DSP URL for SA to access after DSP install. The same URL will be deplayed after successful DSP install. args ----
  • file: Filename to be appended to URL. eg. start.php
  • user: String representing User that owns the DSP instance.

check_pass

Compares instance_pass1 and instance_pass2 in instance_conf. Adds 'PASSWORD_MISMATCH' error to 'st' object if comparison fails.

check_install_loc_owner

Checks if user specified during DSP install is the owner and group owner of the install directory. Adds 'DIR_OWNER_MISMATCH' or 'DIR_GROUP_MISMATCH' error on mismatch.

db_probe

db_probe action checks if the database is accessible with given mysql connection details(in site_conf and instance_conf). If database is not accessible it appends ERROR message to msgs object.

exec_mysql

Executes sql query as in sql arg. Results of sql query is stored in args_dict['exec_mysql']['results'].

args

  • sql : String representing sql statement to be executed.

set_php_cgi

Adds necessary directives to ".htaccess" file placed in dstdir to turn off php cgi mode. .htaccess is created if it is not there already. Default value of 'install_loc' is taken for dstdir unless specified. Useful in cases where a tool requires to override global php configuration.

args_dict { 'dir': path_to_write_htaccess }

By default this writes into install_loc directory which can be changed by providing 'dir' in args_dict

create_htdigest

Creates htdigest file within the specified directory.

Refer create_htdigest for details

args

  • appuser
  • realm
  • directory
  • install_loc
  • password

set_htdigest

Edits ".htdigest" to set htdigest for the instance. It appends following content to .htaccess in specified directory.:
AuthType Digest
AuthName <realm>
AuthDigestFile <directory>/.htdigest
Require valid-user

args

  • realm: Authentication realm
  • directory: Directory for creating .htaccess

create_htpasswd

Creates .htpasswd file in given directory. File configuration is based on the args provided.

args

  • username: String representing the username of the application user. Default value is taken as instance_conf['instance_admin']
  • password: Password
  • realm: Authentication realm
  • directory: Directory for creating .htaccess

set_htpasswd

Creates .htpasswd file in given directory. It points AuthUserFile directive to i.htaccess file given directory path. File contents looks as given below.

AuthType Basic AuthName "<realm>" AuthUserFile <directory>/.htpasswd Require valid-user

args

  • realm: Authentication realm
  • directory: Directory for creating .htaccess
set_trace Executes pdb.set_trace to start debugging from specified point.

remove_files

Recursively removes all files/directories (within install location ) given in 'files' arg.

args:

  • files: List containing paths (relative to Install location) to all files/directories to be removed recursively.

require_url_loc

Checks instance_conf for key url_loc and adds error 'URL_LOCATION_REQUIRED' if key is not present.

check_disk_quota

Checks the diskquota of given user and adds the error messages if quota available is not sufficient. args ----
  • user: String representing User that owns the DSP instance.
Parameters:
actions -

List of actions. Each string starts with the name of permitted function/action ( of Deployables_lib API ) to be called. E.g. actions = ['archive_unpack', 'check_index', 'exec_php_script']

E.g. actions = ['archive_unp', 'archive_unp', 'check_in', 'exec_php_scr'] -> This has same meaning as above except the fact that the "archive_unpack" would be called twice.
           (type=List of stings)
args_dict -

A dictionary keyed by action name and valed by dictionary representing its arguments.

E.g. args_dict = {'archive_unpack' : 'dir_moves':dir_moves, 'file_mods':file_mods, 'exec_php_script' : 'script' : 'setup/index.php', 'args' : }

If required arguments are not found in args_dict then default values are taken from 'instance_conf' or 'site_conf' depending upon the function.
           (type=dictionary)
Returns:
All the actions append the errors (if any) to 'msgs'. It reads the actions list sequentially and calls appropriate functions. In case of any errors returned by the functions it checks the error severity level. If the error is not critical it appends the action it to 'successful actions' list else after iterating through all the actions it undos all the (undoable) actions.

do_reconfigure_actions(context, site, site_conf, old_site_conf, instance_conf, old_instance_conf, msgs, st, actions, args_dict)

do_reconfigure_actions executes the actions required in order to upgrade a DSPI, This function takes the actions as one of the arguments and executes them in order. Below is the list of permitted actions. See do_install_actions for more details on every action.

List of Permitted actions:
  • handle_url_loc
  • resolve_location
  • archive_unpack
  • handle_username_change
  • handle_db_changes
  • edit_conf_file
  • handle_security_mode
  • require_url_loc
  • db_probe
  • exec_mysql
  • apply_patch:
    
    

apply_patch

Applies patches specified in patches list. args ----
  • patches: List of patch files to be applied
Parameters:
context - Context in which the function has been called. Context can take values "CONTEXT_SITE_EDIT" and "CONTEXT_RECONFIGURE"
           (type=string)
old_site_conf - The site configuration before the changes.
           (type=dictionary)
old_instance_conf - The instance configuration before the changes.
           (type=dictionary))
actions -

List of strings. Each string starts with the name of permitted function/action ( of Deployables_lib API ) to be called.(See the permitted actions list for list of functions. )

E.g. actions = ['archive_unpack', 'check_index', 'exec_php_script']

E.g. actions = ['archive_unpack', 'archive_unpack2', 'check_index', 'exec_php_script'] -> This has same meaning as above except the fact that the "archive_unpack" would be called twice.
           (type=list)
args_dict -

A dictionary keyed by function/action name and valued by dictionary representing its arguments.

E.g. args_dict = { 'archive_unpack' : 'dir_moves':dir_moves, 'file_mods':file_mods, 'exec_php_script' : 'script' : 'setup/index.php', 'args' : }

If required arguments are not found in args_dict then default values are taken from 'instance_conf' or 'site_conf' depending upon the function.
           (type=dictionary)
Returns:
All the actions append the errors (if any) to 'msgs'. It reads the actions list sequentially and calls appropiate functions. In case of any errors returned by the functions it checks the error severity level. If the error is not critical it appends the action it to 'successful actions' list else after iterating through all the actions it undos all the (undoable) actions.

do_upgrade_actions(site, site_conf, instance_conf, old_instance_conf, msgs, st, actions, args_dict)

do_upgrade_actions executes the actions required in order to upgrade a DSPI, This function takes the actions as one of the arguments and executes them in order. Below is the list of permitted actions. See do_install_actions for more details on every action.

List of Permitted actions:
  • check_accept_lic
  • check_mysql_enabled
  • resolve_location
  • check_index
  • db_create
  • archive_unpack
  • run_sql_file
  • edit_conf_file
  • handle_security_mode
  • exec_php_script
  • finish_url
  • access_url
  • check_pass
  • check_install_loc_owner
  • db_probe
  • exec_mysql
  • set_php_cgi
  • create_htdigest
  • set_htdigest
  • create_htpasswd
  • set_htpasswd
  • remove_files
  • require_url_loc
  • check_disk_quota
  • set_trace
  • archive_unpack
  • db_create

dsp_create_instance(dsp, version=None)

Returns the instance of DSP i.e a dictionary representing DSP elements (keyed by element names) A DSP description is a dictionary with the following elements:
- 'shortname'   : short name of DSP
- 'longname'    : long name of DSP
- 'version'     : version of DSP (this is the script package's version)
- 'dapiversion' : dAPI version of DSP (this is Ensim dAPI version)
- 'dapirelease' : dAPI release of DSP (this is dAPI's release number)
- 'upgrades'    : list of dictionaries, where each dictionary contains
   a shortname, version, dapiversion, and dapirelease of
   a DSP that this DSP can upgrade
- 'options'     : a list of option dictionaries, where each dictionary
                contains at least the following:
 - 'name'       : name of option
 - 'title'      : displayed title for option
 - 'type'       : type of option (see supported types)
 - 'hidden'     : whether this option should be displayed in the form
 - 'inschema'   : whether this option is part of the database schema for
   the DSP
 - 'insummary'  : whether this option should be part of an instance
   summary display
 - 'summaryTitle' : what title should be displayed for a given summary
   option; if not present, it will default to the title content

 - Following parameters can also be used with option
  - 'parent'     : the option that this option is a suboption of
  - 'helpmsg'    : a help message for this option

This dictionary is simply a copy of dsp_modules['dsp']['desc'].

supported types = ('Boolean', 'ExclusiveSelect', 'MultiInput', 'SingleInput', 'MultiSelect', 'Label')

dsp_fill_instance(context, instance, site, site_conf, new_conf, old_conf=None, common_options=None)

This function takes an instance description (dictionary representing dsp.xml) , a site configuration, a new instance configuration, and optional old instance configuration and list of well-known options, and fills in the various properties of the passed in instance. This function is not invoked when an instance is edited due to a site edit.
Parameters:
instance - The current instance configuration (dictionary representation of DSP elements as being returned by dsp_create_instance.)
           (type=dictionary)
new_conf - The current instance configuration. ( same as instance_conf argument for other functions )
           (type=dictionary)
old_conf - The old (previously populated) instance conf. It is mainly used for upgrades or reconfiguration.
           (type=dictionary)
common_options - Options in the 'instance' which needs to checked an populated from 'new_conf' and 'old_conf' depending upon the context. It sets 'default' ,currentValue and (if needed) internalValue for the options.

dsp_install_directory(dsp=None, version=None)

Returns Install directory for requested DSP (name and version)
Parameters:
dsp - Name of the DSP
version - DSP version
Returns:
String representing DSP install directory.

dsp_lib_directory(dsp, version=None)

Returns lib directory for requested DSP (name and version). The lib directory is $Install_Dir/lib -> /usr/lib/ensim/deployables/<DSP>-<version>/lib/
Parameters:
dsp - Name of the DSP
version - DSP version
Returns:
String representing lib directory.

dsp_libuser_admin(site)

Helper function for User administration.
Returns:
libuser admin object for given site environment

dsp_locale_directory(dsp=None, version=None)

Returns the themes directory -> /usr/lib/ensim/deployables/<DSP>-<version>/lib/locale/

dsp_source_directory(dsp, version=None)

Returns source directory for requested DSP (name and version) The source directory is $Install_Dir/lib -> /usr/lib/ensim/deployables/<DSP>-<version>/source
Parameters:
dsp - Name of the DSP
version - DSP version
Returns:
String representing source directory.

dsp_theme_directory(dsp=None, version=None)

Returns the themes directory -> /usr/lib/ensim/deployables/<DSP>-<version>/lib/themes/

dsp_versioned_name(dsp, version=None)

Returns string - <DSP>-<version> for given DSP-name and version

dump_command(cmd, out='')

Parameters:
cmd -
           (type=string)
out - default: ''
           (type=string)

edit_conf_file(site, site_conf, instance_conf, file, lang, pat, sub, vars)

Edits a conf file for the provided key, value and comment(optional) triplet tuples in vars argument.

eg. conversion

vars("somekey","somevalue","somecomment") => "somekey" = "somevalue"; #somecomment

Parameters:
site - site-id
           (type=string)
site_conf -
           (type=dictionary)
instance_conf -
           (type=dictionary)
file -
           (type=string)
lang - The scripting language of the specified file. eg php
           (type=string)
pat - Regular expression pattern to match the lines in the config file
           (type=string)
sub - A string modifier pattern for writing the lines to config file
           (type=string)
vars -

Vars argument contains the tuples with key value pairs of the configuration items that has to be added to config file.

eg.

[('siteurl', instance_conf['url_loc']), ('admin_email', instance_conf['instance_admin_email']), ('dbname', instance_conf['dbname']), ('dbhost', dl.get_mysql_host(site)), ('dbusername', site_conf['mysql']['dbaseadmin']), ('dbpassword', instance_conf['dbpass']), ('fileupload_realpath', (instance_conf['install_loc'] + '/data', dl.MODSTR_JAIL))]
           (type=List)
Returns:
This function does not return anything.

enable_patches(instance, instance_patches)

Enables patches for the instance.

exec_cmd(cmd)

Executes the command
Parameters:
cmd - Command to be executed.
           (type=string)
Returns:
Tuple of exit status(int) and command output(str).

exec_curl_cmd(url, opts=[])

Executes curl command with requested opts.
Parameters:
url - URL to be passed to curl command.
           (type=string)
opts - options to ba passed to curl(optional)
           (type=list)
Returns:
Tuple of exit status(int) and command output(str).

exec_mysql(msgs, st, site, site_conf, instance_conf, sql)

Executes SQL statements given by 'sql'. The result of execution is returned and eventually stored in args_dict['exec_mysql']['results'].

exec_php_script(site, site_conf, instance_conf, exec_dir, script, args, success_re, failure_re, post=1)

Executes php scripts (given by argument 'script'). Most of the times these are the web-installer/wizard scripts that configures and installs a DSP.
Parameters:
exec_dir - The path of execution directory. If not present in args_dict, default value of instance_conf[install_loc] is taken.
           (type=string)
script - Path of the script relative to install location.
           (type=string)
args - Dictionary of arguments and their values to be passed to the script.
           (type=dictionary)
success_re - Regular expression which represents the pattern of success message returned by the script.
           (type=string)
failure_re - Regular expression which represents the pattern of failure message returned by the script.
           (type=string)
post - Boolean. If set to true sets request method to HTTP_POST.

find_free_dbname(prefix, dbname)

Returns a (non-existing) MySQL databse name for DSPI installation.
Parameters:
prefix - database prefix (eg. some_site_com_-_)
           (type=string)
dbname - suggested MySQL databse name for DSPI install
           (type=string)
Returns:
Returns the MySQL database name(string) if there is no existing db by the same name. Else tries by adding digit/s to the db name

find_free_path(site, path)

Find a (non-existing) directory for DSPI installation.
Parameters:
site - site-id (eg. site12)
           (type=string)
path - Directory name for DSPI install
           (type=string)
Returns:
Returns the complete install path(string) if there is no existing directory by the same name. Else tries by adding digit/s to the directory name

finish_url(msgs, st, site, site_conf, instance_conf, file)

Constructs the DSPI URL from provided parameters and appends this URL to msgs.

get_domain_path_pairs(site, site_conf)

Finds subdomains and paths to their publishing(html) directory.
Parameters:
site - site-id
           (type=string)
site_conf - site configuration
           (type=dictionary)
Returns:
List of tuples containing subdomains and paths to their publishing directory in following format. [('subdomain1'),(path_to_html1),('subdomain2'),(path_to_html2)]

get_module(dsp, version=None)

Returns dictionary for a DSP with specified version

eg.
>>> dl.get_module('gallery','1.4.1').keys()
['themes', 'desc_last_modified', 'dAPI_last_modified', 'msgs', 'locale',
'msgs_last_modified', 'module', 'desc']
>>> print dl.get_module('gallery','1.4.1')['themes']
{'default': {'doc': <xmlDoc
(/usr/lib/ensim/deployables/gallery-1.4.1/lib/themes/default.xsl) object at
0x885dcb4>, 'last_modified': 1077180404, 'xsl': <libxslt.stylesheet
instance at 0x88b1854>}}

get_mysql_host(site)

Returns mysql host for the site

get_site_user_ids(site, user, context=None)

Returns (uid, gid) pair for the user.
Parameters:
user - Username
           (type=string)
Returns:
Tuple (uid, gid)

get_site_user_name(site, uid=None, gid=None, context=None)

Find the username and group for specified uid,gid.
Returns:
Tuple (username, groupname)

get_site_users(site, site_conf, context=None)

Get site users for requested site.
Returns:
List of site users(with uid above 500)

handle_db_changes(context, site, new_siteconf, old_siteconf, new_conf, old_conf)

Rename a DSP MySQL database if the DB prefix is changed.
Parameters:
context -
Returns:
0 on Success

handle_security_mode(site, site_conf, instance_conf, files)

Sets file and directory permissions for files based on weather 'jail' is 'True' or 'False'.
Parameters:
files - list of tuples. Each tuple has 3 objects (in order) as given below
  • string representing file/directory path
  • Permissions (in octal format i.e 07000 ) for if Jail is True.
  • Permissions (in octal ... ) if Jail is False.

handle_url_loc(context, site_conf, old_site_conf, instance_conf, old_instance_conf)

Computes the new value of instance_conf['url_loc'] if there is a change in site domain (during a site edit).

handle_username_change(site, old_conf, new_conf)

Corrects the file ownerships of DSP files if the site user is changed.
Parameters:
site - site-id
           (type=string)
old_conf - Old configuration
           (type=dictionary)
new_conf - New configuration
           (type=dictionary)
Returns:
-1 on failure. 0 indicates success.

install_loc_cleanup(site, site_conf, instance_conf, dir_moves, patches=None, file_mods=None)

Recursively deletes all files/directory ( within install location ) which were -
  • unpacked from the archive
  • provided as patches
  • mentioned in file_mods.
Parameters:
dir_moves - list of tuples. Each tuple contains a source and destination pair for files/directories. All the 'source' files/directories mentioned in 'dir_moves' are moved to 'destination' during the archive unpack.
           (type=list)
patches - list of dictionaries. Each dictionary has a key that describes a patch and path of the patch to be applied as the value.:
eg. [{'file' : 'Search.php.patch'}]

           (type=List)
file_mods - list of dictionaries. Each dictionary contains file/directories deletion details. Keys of the dictionaries can be one or more among the following -
  • 'path' : gives file/directory name. Directory is distinguished by a trailing '/'
  • 'delete' : delete the file given by the path.

load_dAPI_module(dsp, version=None)

Loads and returns ref to the dAPI module of requested DSP
Parameters:
dsp - Name of the DSP
Returns:
Module object

load_dsp_description_from_file(xml_path)

Reads and converts requested dsp.xml to dictionary format

load_dsp_description_from_string(xml_str)

Reads and converts requested DSP xml string to dictionary format

load_msgs_module(dsp, version=None, symtab=None)

Load 'msgs' module deployed in the 'lib' directory of requested DSP into the 'systab' system table

modify_string(site, site_conf, str_or_tuple)

Prefix site fst path to given string(path) if the site is a high security site.
Parameters:
site - site name
           (type=string)
site_conf - site config
           (type=dictionary)
str_or_tuple - string to be modified if it's tuple str_or_tuple(0) get mofied provided str_or_tuple(1) is true
           (type=string/tuple)
Returns:
Modified string

mysql_installed(conf, old_conf=None)

Parameters:
conf - site_conf dictionary
old_conf - old site_conf dictionary (Optional: default: None)
Returns:
1 if mysql service is enabled for the site else returns 0.

readfile(path, uid, gid)

Slightly paranoid file reader.
Parameters:
path - Path of file to read
           (type=string)
uid - uid set on file
gid - gid set on file
Returns:
String of file content read

recursive_chown(dir, orig_uid, orig_gid, uid, gid)

This function is intended to provide a safe recursive chown. It refuses to follow symlinks, uses fchown, and will not chown anything unless it matches the expected uid/gid.
Parameters:
dir - Directory to be recursively chowned
orig_uid - Existing uid value set on dir
orig_gid - Existing gid value set on dir
uid - New uid
gid - New gid

refresh_dsps(dsp=None, version=None)

This function is automatically called when the module is loaded.

Locates the deployable with name and version as 'dsp' and 'version' in deployables install dir. i.e - "/usr/lib/ensim/deployables". If no name('dsp') or version info is passed it considers all deployables in the install directory.

(Re-)Builds a dictionary 'dsp_modules' keyed by 'dsp' containing objects and last-modified-date of -
  1. dAPI modules as dsp_modules['dsp']['module'] and dsp_modules['dsp'][dAPI_last_modified']
  2. dsp.xml modules as dsp_modules['dsp']['desc'] and dsp_modules['dsp']['desc_last_modified']
  3. msgs.py modules as dsp_modules['dsp']['msgs'] and dsp_modules['dsp']['msgs_last_modified']
The dAPI and msg.py are simply loaded as python modules, but the DSP description is a dictionary (extracted out of itz DOM model) with the following elements:
  • 'shortname' : short name of DSP
  • 'longname' : long name of DSP
  • 'version' : version of DSP (this is the script package's version)
  • 'dapiversion' : dAPI version of DSP (this is Ensim's version)
  • 'dapirelease' : dAPI release of DSP (this is Ensim's release)
  • 'upgrades' : list of dictionaries, where each dictionary contains a shortname, version, dapiversion, and dapirelease of a DSP that this DSP can upgrade
  • 'options' : a list of option dictionaries, where each dictionary contains at least the following:
  • 'name' : name of option
    • 'title' : displayed title for option
    • 'type' : type of option (see __supported_types)
    • 'hidden' : whether this option should be displayed in the form
    • 'inschema' : whether this option is part of the database schema for the DSP
    • 'insummary' : whether this option should be part of an instance summary display
    • 'summaryTitle' : what title should be displayed for a given summary option; if not present, it will default to the title content
An option may also contain the following parameters
  • 'parent' : the option that this option is a suboption of
  • 'helpmsg' : a help message for this option

regression_check_index(action, index, test_data, string)

regression helper method

remove_files(site, instance_conf, files)

Recursively removes all files/directories (within install location ) given in 'files'.
Parameters:
files - List containing paths (relative to Install location) to all files/directories to be removed recursively.
           (type=List)

require_url_loc(msgs, st, instance_conf)

Checks if instance_conf has the key - 'url_loc' Adds error 'URL_LOCATION_REQUIRED' to msgs on not finding the key.

resolve_location(msgs, st, context, site, site_conf, instance_conf)

Given url_net and url_path (in Instance Conf), resolves these two variables to an installation location in the filesystem and a URL location. Adds these variables as install_loc and url_loc to instance_conf.
Returns:
Errors are appended to msgs; 0 is returned if any errors that occurred shouldn't stop the processing immediately, and 1 is returned if they should.

set_htdigest(msgs, st, site, instance_conf, realm, directory)

Edits ".htdigest" to set htdigest for the instance.

Args:
Parameters:
realm - Authentication realm of the instance.
           (type=String)
directory - The directory for creating ".htdigest" file. Default is taken as instance_conf['install_loc']
           (type=String)

set_htpasswd(msgs, st, site, instance_conf, realm, directory)

Sets htdigest password for the instance.
Parameters:
realm - String representing realm of the instance.
directory - String representing the directory for creating ".htdigest" file. Default is taken as instance_conf['install_loc']
Returns:
'non-Critical' Errors (if any)

shell_escape_arg(arg)

Escapes any 's in the given string so that the given string may be surrounded in 's prior to being used in a command line context. This is typically needed when we want to execute a command that is going to have user input as part of its parameters, and we want to prevent the user from trying to get us to execute something we shouldn't.

site_fs(site)

Returns site file system path from given site.

NOTE: YOU RARELY WILL NEED TO USE THIS FUNCTION! IN ALMOST ALL CASES, YOU WANT SITE_FS_JAIL INSTEAD! ONLY USE THIS FUNCTION WHEN YOU ARE ATTEMPTING TO ACTUALLY MANIPULATE A FILE WITHIN A SITE!!! (rationale: site_fs_jail makes allowances for export/import that this function can't)
Parameters:
site -
Returns:
String: /home/virtual/<site>/fst

site_fs_jail(site, site_conf)

Returns site file system path from given site.

url_net_sub(url, new_domain, old_domain)

This function replaces the (trailing) portion of the domain name segment of a URL. It is meant to be used in CONTEXT_SITE_EDIT mode.

url_path_sub(str, new_url, old_url)

This function replaces the old domainname to new(changed) domainname from given str.

eg.
>>> dl.url_path_sub('My Domain: foo.com','foo-bar.com','foo.com')
>>> 'My Domain: foo-bar.com'
It is meant to be used in CONTEXT_RECONFIGURE mode.
Parameters:
str - string to be modified
           (type=string)
new_url - New URL
           (type=string)
old_url - Old URL
           (type=string)

url_sub(url, new_url, old_url)

This function replaces the entire domain and the starting portion of the path of URLs. It is meant to be used in CONTEXT_RECONFIGURE mode.

writefile(path, uid, gid, data)

File writer method that writes given data to specified file path and assigns uid, gid. Does not return success/failure.

__conf_quote(value, lang)

Add quotes requested string

eg.
>>> import deployables_lib as dl
>>> dl.__conf_quote("XX",'php')
"'XX'"
Supported languages
  • php
  • html
  • apache-config
Parameters:
value - string to unquote
           (type=string)
lang - Language @type lang
Returns:
Quoted string

__conf_unquote(value, lang)

Unquotes requested string

eg.
>>> import deployables_lib as dl
dl.__conf_unquote("'XX'",'php')
'XX'
Supported languages
  • php
  • html
  • apache-config
Parameters:
value - string to unquote
           (type=string)
lang - Language @type lang
Returns:
Unquoted string

Variable Details

CONTEXT_AUTO_UPGRADE

Type:
int
Value:
6                                                                     

context_dict

Type:
dict
Value:
{1: 'install',
 2: 'upgrade',
 3: 'remove',
 4: 'reconfigure',
 5: 'site_edit',
 6: 'auto_upgrade'}                                                    

CONTEXT_INSTALL

Type:
int
Value:
1                                                                     

CONTEXT_RECONFIGURE

Type:
int
Value:
4                                                                     

CONTEXT_REMOVE

Type:
int
Value:
3                                                                     

CONTEXT_SITE_EDIT

Type:
int
Value:
5                                                                     

CONTEXT_UPGRADE

Type:
int
Value:
2                                                                     

dsp_modules

Type:
dict
Value:
{}                                                                     

dsp_null_translation

Type:
NullTranslations
Value:
<gettext.NullTranslations instance at 0xb7abdb8c>                      

dsp_themes

Type:
dict
Value:
{'default': {'doc': <xmlDoc (/usr/lib/ensim/deployables/lib/themes/def\
ault.xsl) object at 0xb7a66e2c>,
             'last_modified': 1109189500,
             'xsl': <libxslt.stylesheet instance at 0xb7a66e8c>}}      

libuser_lock

Type:
lock
Value:
<thread.lock object at 0xb7f0a0f0>                                     

MODSTR_JAIL

Type:
int
Value:
1                                                                     

REGRESSION_DO_TEST

Type:
int
Value:
2                                                                     

REGRESSION_GET_TEST_STRING

Type:
int
Value:
1                                                                     

__supported_types

Type:
tuple
Value:
('Boolean',
 'ExclusiveSelect',
 'MultiInput',
 'SingleInput',
 'MultiSelect',
 'Label')                                                              

_cryptsalt

Type:
str
Value:
'./abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'     

Generated by Epydoc 2.1 on Wed Mar 23 00:13:48 2005 http://epydoc.sf.net