You are here: Commander Installed Tools

Commander Installed Tools

 

This reference topic can acquaint you with some Commander tools you may not know about or provide quick access to information for tools you use already. Select from the links in the table below to go directly to information for that tool.

Tool Name Description
eccert A command-line tool used to manage the Electric Commander Certificate Authority (CA) and the certificates configured in Commander Server and Commander Agent installations.
ecconfigure A command-line tool that can change configuration values for any locally installed Commander server, web, agent, or repository service. ecconfigure is a more user-friendly mechanism for configuring aspects of Commander that would otherwise require manual configuration file updates. ecconfigure actually manipulates relevant service configuration files on your behalf.
ecdaemon A "wrapper" program that can be used to start another program from a Commander job step—the "started" program will run as a daemon process. The Commander agent uses the facilities of the underlying operating system to make sure the process runs in a separate process group on a UNIX-based system, or outside of the normal "Windows Job" grouping in a Windows system. In either case, the Commander agent does not treat the process as one it should wait for or one it should try to "kill" if Commander needs to abort the step.
ecproxy A driver script with built-in support for SSH. Every major operation can be overridden by defining a Perl function in the Proxy Customization field on the New Proxy Resource panel, available from the Resources page,
ecremotefilecopy Remote file copy: When Commander agents (on platforms other than Linux or Windows) run steps that create log files in a workspace the Commander web server cannot access (through Linux or Windows agents), use ecremotefilecopy to recreate job logs so they are visible on those Commander agents, which then enables the web server to retrieve and render those log files.
ClusterTool A command-line tool that imports your ElectricCommander database configuration information into your ZooKeeper server.

 

eccert

A command-line tool used to manage the Electric Commander Certificate Authority (CA) and the certificates configured in Commander Server and Commander Agent installations.

Usage

eccert [ options ] command [ arg ... ]

Commands

 
addTrustedServer crtAdd a server CA certificate to the agent's keystore.
getCRLRetrieve the contents of the current certificate revocation list.
initAgent [ --local | --remote ] [ options ]
 Initialize the agent keystore with a new public/private key pair.
Generates the agent certificate signing request.
If run on the server host, the certificate will automatically be signed by the server CA, and the CA certificate and the signed agent certificate are installed in the agent's keystore.
If run on a non-server host, the signing request is left in the agent directory. If CA Cert is provided, the CA certificate is installed in the agent's keystore.
 --local

Use the local server CA to sign the agent certificate.

 --remote

Connect to a remote Commander server to sign the agent certificate.

 --force

Replace any existing keystore.

 --cname name

Use the specified name as the common name (CN) in the agent certificate subject. This is normally the fully qualified domain name used by clients to connect to the agent.

 --altNames entries

Use the specified list of entries (comma or space separated) as the subjectAlternateNames list in the agent certificate. Simple names are interpreted as dns entries. Entries may begin with "dns:" or "ip:" to indicate the type (for example, "ip:192.168.0.1" or "dns:myHost"). If no entries are specified, then reverse DNS is used to look up the registered name(s) of the host's IP addresses.

initCAInitialize the server CA. Creates a new CA key and certificate.
initServer [ options ]Initialize the server keystore. Creates and signs the server certificate. Installs the CA certificate and the signed server certificate into the server's keystore.
 --force

Replace any existing keystore.

 --cname name

Use the specified name as the common name (CN) in the server certificate subject. This is normally the fully qualified domain name used by clients to connect to the server.

 --altNames entries

Use the specified list of entries (comma or space separated) as the subjectAlternateNames list in the server certificate. Simple names are interpreted as dns entries. Entries may begin with "dns:" or "ip:" to indicate the type (for example, "ip:192.168.0.1" or "dns:myHost"). If no entries are specified, then reverse DNS is used to look up the registered name(s) of the host's IP addresses.

list [ --agent | --server | --index [ --verbose ]
 Display certificate information for agent and/or server keystores or the CA certificate index. If no options are specified, both the agent and server keystores are listed.
 --agent

List the contents of the agent keystore.

 --server

List the contents of the server keystore.

 --index

List the contents of the CA issued certificates index.

 --verbose

Display additional details.

refreshCRLRefresh the certificate revocation list from the Commander server.
revoke indexRevoke a previously issued certificate by index.
signCertificate csr crtSign the certificate signing request provided in file csr and write the signed result to the file crt. The request is rejected by the CA if there is a matching certificate already in the CA database.
updateAgentCertificate crt
 Install a previously signed certificate crt into the agent's keystore.
       

Server Communication Options

--server host
Address of the ElectricCommander server. Defaults to the value of the COMMANDER_SERVER environment variable. If that does not exist,it defaults to localhost.
--securePort port
HTTPS listener port on the server. Defaults to 8443.
       

Global Options

--help
Print the help message.
--version
Print the version message.


Examples

Example 1: Configure an agent to talk to any server (untrusted mode)

This example generates a new self-signed certificate for the agent and recreates the keystore with no trusted authorities.

    $ eccert initAgent -force

    Generating keys
    Generating certificate request
        cname=myAgent.company.com
        san=dns:myAgent.company.com

Example 2: Configure an agent to accept connections only from a single remote Commander server

This example generates a new certificate for the agent that is signed by the remove server's certificate authority and installs the signed certificate and its associated trust chain in the agent's keystore. After this point, the agent will only accept requests from the specified server and will be used as a trusted resource by the server.

    $ ectool --server myserver login admin pw
    $ eccert --server myserver initAgent -remote

    Generating certificate request
        cname=myAgent.company.com
        san=dns:myAgent.company.com
    Asking server 'myserver' to sign certificate
    Importing 'CA:myserver.company.com' certificate
    Importing 'jetty' certificate

Example 3: Configure a Commander server with additional host names in the certificate

This example regenerates the Commander Server Certificate, the specified common name, and alternate subject names to allow trusted connections with multiple external dns names.

    $ eccert initServer --force --cname "myServer.company.com" --altNames "myServer,server2.company.com"

    Generating keys
    Generating certificate request
        cname=myserver.company.com
        san=dns:myserver,dns:server2.company.com
    Signing server certificate
    Importing 'CA:myserver.company.com' certificate
    Importing 'jetty' certificate

 

ecconfigure

A command-line tool that can change configuration values for any locally installed Commander server, web, agent, or repository service. ecconfigure is a more user-friendly mechanism for configuring aspects of Commander that would otherwise require manual configuration file updates. ecconfigure actually manipulates relevant service configuration files on your behalf.


Usage

ecconfigure [options]

Commander agent configuration options

--agentInitMemory=percentInitial java heap size as a percentage of the total system memory.
--agentKeystorePassword=passwordPassword used to access the agent's keystore..
--agentMaxMemory=percent Maximum java heap size as a percentage of the total system memory.
--agentInitMemoryMB=sizeInitial java heap size in MB.
--agentMaxMemoryMB=sizeMaximum java heap size in MB.
--agentLogFile=pathPath where the agent log file should be written.
--agentLogMaxFiles=maxMaximum number of log files to accrue.
--agentLogMaxSize=max Maximum size of each log file. The value can be suffixed with a unit (MB, KB, B). Without a unit,the value is interpreted as bytes.
--agentPluginsDirectory=pathThe path used by the agent to get to the plugins directory on the Commander server where its resource definition lies.
--agentLocalPort=port Port used by the Commander agent for HTTP communication on the localhost network interface.
--agentPort=port Port used by the Commander agent for HTTPS communication on any network interface.
--agentProtocolProtocol used by the agent.
--agentProxyHost=hostThe IP address of the proxy server.
--agentProxyPort=portThe port of the proxy server.
--agentNoProxyHosts=hostsComma delimited list of hosts that should be reached directly, bypassing the proxy server.
--agentEnableProxySettings=<0|1>Enable or disable the proxy server configuration.
If enabling for the first time, --agentProxyHost and --agentProxyPort must be specified.
       

Apache web server configuration options

--webHostName=host The host name of the current machine in the form that users will typically use in their browser to access the web server.
--webHttpPort=portThe HTTP port of the web server.
--webHttpsPort=portThe HTTPS port of the web server.
--webTargetHostName=host The host name of the Commander server to which the web server points.
--webTargetHttpPort=port The HTTP port of the Commander server to which the web server points.
--webTargetHttpsPort=port The HTTPS port of the Commander server to which the web server points.
--webTimeZone=timezone The Olson TimeZone format (example: America/Los Angeles)for the php web server.
--webPluginsDirectory=pathThe path used by the web server to get to the plugins directory on the Commander server to which it points.
--webProxyUrl=url The IP address and port of the proxy server in the following format:
http://<IP_ADDRESS_PROXY>:<PROXY_PORT>
--webNoProxyHosts=hosts Comma-delimited host list that should be reached directly, bypassing the proxy server.
--webEnableProxySettings=<0|1>Enable or disable the proxy server configuration.
If enabling for the first time, --webProxyUrl must be specified.
--webDLC=urlThe URL to use for downloadable content requests.
       


Commander Server configuration options

--serverFileTransferPort=portThe file transfer port of the server.
--serverHttpPort=portThe HTTP port of the server.
--serverHttpsPort=portThe HTTPS port of the server.
--serverInitMemory=percentInitial java heap size as a percentage of the total system memory.
--serverMaxMemory=percent Maximum java heap size as a percentage of the total system memory.
--serverInitMemoryMB=sizeInitial java heap size in MB.
--serverMaxMemoryMB=sizeMaximum java heap size in MB.
--serverPasskeyFile=pathPath to the server's passkey file.
--serverProxyHost=hostThe IP address for the proxy server.
--serverProxyPort=portThe port for the proxy server.
--serverNoProxyHosts=hostsComma-delimited host list that should be reached directly, bypassing the proxy server.
--serverEnableProxySettings=<0|1>Enable or disable the proxy server configuration.If enabling for the first time,
--serverProxyHost and --serverProxyPort must be specified.
       

Repository Server configuration options

--repositoryPortThe repository server port.
--repositoryInitMemory=percentInitial java heap size as a percentage of the tot al system memory.
--repositoryMaxMemory=percentMaximum java heap size as a percentage of the tot al system memory.
--repositoryInitMemoryMB=sizeInitial java heap size in MB.
--repositoryMaxMemoryMB=sizeMaximum java heap size in MB.
--repositoryStorageDirectory=pathPath to the repository backing store.The artifact repository will use this directory to store artifacts.
--repositoryTargetHostName=hostThe host name of the Commander server to which th e repository server points.
--repositoryTargetHttpPort=portThe HTTP port of the Commander server to which th e repository server points.
--repositoryTargetHttpsPort=portThe HTTPS port of the Commander server to which t he repository server points.
--repositoryTargetProtocol=<http|https>
 The protocol the repository server uses to talk to the Commander server.
--repositoryProtocol The protocol the repository server uses to talk to client applications.
--repositoryProxyHost=hostThe IP address for the proxy server.
--repositoryProxyPort=portThe port for the proxy server.
--repositoryNoProxyHosts=hostsComma-delimited host list that should be reached directly, bypassing the proxy server.
--repositoryEnableProxySettings=<0|1>
 Enable or disable the proxy server configuration.
If enabling for the first time,--repositoryProxyHost and
--repositoryProxyPort must be specified.
--repositoryValidateFromDisk=<0|1>Enable or disable disk validation.
       

General options

-v,--versionDisplay version information.
-h,--helpDisplay this information.

 

Examples

Setting initial and maximum memory settings

For example, to set the Commander Server initial memory percentage to 21% and the maximum memory percentage to 31%, specify:

ecconfigure --serverInitMemory 21 --serverMaxMemory 31

__________

If your Commander server, web server, or repository server is deployed behind a proxy server that inhibits certain Internet access, you can use ecconfigure to set proxy settings for each server in your installation.

To use the following perl scripts, remove the brackets ("< >"), and replace the bracketed example text with the values you need.

ecconfigure --serverProxyHost <IP_ADDRESS_PROXY> --serverProxyPort <PORT>
--serverNoProxyHosts "<HOST1,HOST2>"

ecconfigure --repositoryProxyHost <IP_ADDRESS_PROXY> --repositoryProxyPort <PORT>
--repositoryNoProxyHosts "<HOST1,HOST2>"

ecconfigure --webProxyUrl <http://IP_ADDRESS:PORT> --webNoProxyHosts <HOST1,HOST2,HOST3>

__________

Changing the Apache web server port

Run ecconfigure with --webHttpPort and --webHttpsPort.

You web server port setting will be changed appropriately in httpd.conf, ssl.conf, and config.php.

__________

Configuring the backingstore location for the Artifact Repository

ecconfigure --repositoryStorageDirectory <new-path>

 

ecdaemon

ecdaemon is a "wrapper" program that can be used to start another program from a Commander job step—the "started" program will run as a daemon process. The Commander agent uses the facilities of the underlying operating system to make sure the process runs in a separate process group on a UNIX-based system, or outside of the normal "Windows Job" grouping in a Windows system. In either case, the Commander agent does not treat the process as one it should wait for or one it should try to "kill" if Commander needs to abort the step.

Use Cases

ecdaemon launches the command and exits. Optionally, it sets a property in Commander with the pid of the program it spawned to make it possible for a later step to "kill" the daemonized program if desired.

For example:
ecdaemon c:/install.exe a b c

Using ecdaemon

$ ecdaemon
Usage: ecdaemon [options] cmd cmdArg1 cmdArg2 ...
ecdaemon --version
Required:
command The command to daemonize, followed by its args, if any.
Options:
--pidProperty=prop-path A property to set with the pid of the daemonized process.
--version Print the version of this program and exit.

 

Command-line parsing

ecdaemon supports the standard UNIX-style '--' flag to indicate there are no more ecdaemon options and all subsequent options should be treated as simple arguments to the command.

This is particularly important for commands that themselves take '--' arguments.

For example:

ecdaemon /usr/bin/myserver --config /etc/myserver.conf

will not run properly because ecdaemon will attempt to parse the --config option instead of passing it to the myserver program.

The correct way to invoke ecdaemon in this case is:

ecdaemon -- /usr/bin/myserver --config /etc/myserver.conf


If you want to store the daemonized process's pid in a property, do so as follows:

ecdaemon --pidProperty /myJob/serverPid -- /usr/bin/myserver --config /etc/myserver.conf


As a daemon process, any output goes to /dev/null, therefore no output file is generated.

ec-perl considerations

Use the perl system() call to start ecdaemon.

System() returns an exit status, "backticks" capture and return output that waits for the daemonized command to complete on Windows, and exec never returns at all if it is successful.

 

ecproxy

A driver script with built-in support for SSH. Every major operation can be overridden by defining a Perl function in the Proxy Customization field on the New Proxy Resource panel, available from the Resources page (by specifying which operation this function re-implements. These operations must have certain "signatures" for the driver to invoke them properly—the operations are listed and described below. For more detail, see the SSH implementation in ecproxy.pl.

ecproxy Algorithm

ecproxy invokes the operations detailed below to perform the following actions:

  1. Uploads the command-file to a "workingDirectory" on the proxy target, using the protocol specified in the proxy config. Currently, only SSH is supported.
  2. Creates a wrapper sh shell script that CD's to workingDirectory, sets "COMMANDER_ environment variables" that exist in the proxy agent's environment, and runs the command-file previously uploaded.
  3. Uploads the wrapper script to workingDirectory on the proxy target.
  4. Runs the wrapper script on the proxy target and streams its output to the proxy agent's stdout.
  5. Deletes the local wrapper shell script, the remote wrapper, and remote command-file.
  6. Exits using the exit code of running the wrapper script.

ecproxy Operations

getDefaultWorkingDirectory

DescriptionComputes the default working directory where a command needs to run on the proxy target, if the step is not defined with a working directory.
ArgumentsNone
ReturnsThe path to a directory as it would be accessed on the proxy target.
SSH implementation functionNot SSH-specific, so the function is 'getDefaultWorkingDirectory'
Existing implementationReturn $ENV{COMMANDER_WORKSPACE_UNIX};
Reason to override If the "Working Directory" field is empty on a step that is going to run on a proxy target, the working directory for the step should be the workspace, just as it would be if the step were running on a non-proxy Commander agent.
ecproxy is guaranteed to run in the workspace directory on the proxy agent, but it is not guaranteed that the proxy target has the same path to the workspace.
For example, the workspace on a Windows proxy agent is not the same as the path a Unix proxy target uses to access the workspace—so the existing implementation of this operation simply returns the UNIX path to the workspace. However, if the proxy target has a different path for accessing the workspace, the existing implementation will give the wrong answer. Thus, a user can provide a different implementation that gives the right answer.
Note: This operation is applicable only if "Working Directory" is empty, and is used as the working directory on the proxy target in that case for running the command.


getDefaultTargetPort

DescriptionComputes the default port where the proxy target is listening for this protocol.
ArgumentsNone
ReturnsThe default port.
SSH implementation functionssh_getDefaultTargetPort
Note: This operation is applicable only if the resource definition specifies no port value.


connect

DescriptionOpens a connection to the proxy target using the desired protocol.
Argumentshost, port (optional)
Returns"connection-context hash-ref" on successful connection. This context can contain anything other functions can use to perform their tasks (for example, a connection handle).
Example 1my $context = connect('myhost', 22)
Example 2my $context = connect('myhost')
SSH implementation functionssh_connect
Note: Because the port is optional, the implementation of connect can default to whatever is reasonable for the protocol. This means the 'Proxy Target Port' need not be specified in the Commander web UI for proxied agents reachable on the default port.


uploadFile

DescriptionUploads the given srcFile to the proxy target as tgtFile.
Argumentscontext, srcFile (typically simple file-name), tgtFile (typically workingDirectory/file-name)
ReturnsNothing; on failure it does a 'die' with an appropriate error message.
ExampleuploadFile($context, 'agent123.tmp', '/opt/work/joe/agent123.tmp')
SSH implementation functionssh_uploadFile


generateWrapperScript

DescriptionGenerates the script body that will run the command-file on the proxy-target in the workingDirectory.
ArgumentsworkingDirectory, cmd, cmdArg1, ..., cmdFileName
(just the base-name, no directory)
ReturnsA string containing the script to execute on the proxy target.
ExamplegenerateWrapperScript('/opt/work/joe', 'perl', 'agent123.tmp'
SSH implementation functionNot SSH-specific, so the function is 'generateWrapperScript'
Existing implementationcd workingDirectory; set COMMANDER_ environment variables; run command-file, properly quoting the command and args.
Reason to override If the proxy target does not have sh, the wrapper script needs to be written in a language available on the target.


uploadWrapperScript

DescriptionUploads the wrapper-script code to the proxy target.
Argumentscontext workingDirectory. wrapperScriptBody
ReturnsThe path to the wrapper-script on the proxy target.
Examplemy $wrapperFile = uploadWrapperScript($context, '/opt/work/joe', 'cd /opt/work/joe; perl agent123.tmp')
SSH implementation function3.1, 3.1.1: ssh_uploadWrapperScript
3.1.2 and laterNot SSH-specific anymore, so the function is 'uploadWrapperScript'
Notes:
  1. This function must generate a uniquely named file that will not conflict with other ecproxy invocations that might be occurring in parallel steps. The recommended approach is to generate a file name containing the job-step-id.
  2. Depending on the protocol and facilities available in the Perl implementation, you may or may not need to create a local tmp file to upload to the proxy target. If you do, record that fact in the context and clean up the local file in the cleanup operation. Setting the local wrapper file path in $context->{wrapperFile} is recommended because the default cleanup operation implementation looks for that string.


generateWrapperInvocationCommand

DescriptionGenerates the command-line for running the wrapper script on the proxy target.
ArgumentsremoteWrapperFile (path on proxy target)
ReturnsA string containing the command-line for running the wrapper script file on the proxy target.
Examplemy $wrapperCmdLine = generateWrapperInvocationCommand($wrapperFile)
SSH implementation functionNot SSH-specific, so the function is 'generateWrapperInvocationCommand'
Existing implementationReturn "sh $remoteWrapperFile";
Reason to overrideThe default implementation of this function returns something like 'sh $wrapperFile'. If the wrapper script is not an sh script, or if you want to pass different arguments to the shell, you must override this function.

runCommand

DescriptionRuns the given command-line on the proxy target.
Arguments:context, cmdLine
Returnsexit-code from running the command on the proxy target, undef if the command could not be run for some reason.
ExamplerunCommand($context, $wrapperCmdLine)
SSH implementation functionssh_runCommand

cleanup

DescriptionPerforms any cleanup task after the command has completed on the proxy target. Typically, it deletes any locally created temp files and uploaded files on the proxy target.
Argumentscontext, cmdFile, wrapperFile (both are of the form workingDirectory/file-name)
Returns1 on success, undef on failure.
ExamplecleanupTarget($context, '/opt/work/joe/agent123.tmp', '/opt/work/joe/cmdwrapper.123.tmp')
SSH implementation function3.1, 3.1.1: ssh_cleanup
3.1.2 and laterNot SSH-specific anymore, so the function is cleanup
Note: The default implementation deletes the locally created wrapper script file whose path is stored in $context->{wrapperFile}, if it exists. Thus, if the uploadWrapperScript operation is overridden, it is recommended the overriding function set this attribute—that way, cleanup need not be overridden.


ping

DescriptionA test to see if the proxy target is usable.
Argumentshost, port (optional)
Returns1 on success, undef on failure. 
Exampleping('myhost', 22)
SSH implementation functionNot SSH-specific, so the function is ping.
Existing implementation Opens a socket connection to the proxy target on the desired port.
Reason to overrideThe existing implementation may be deemed too simple for doing a ping; overriding ping to open a connection and do some protocol-specific handshaking might be more appropriate for some protocols / use cases.

Available Helper Functions

To make proxy customization easier, ecproxy provides the following helper functions.

mesg

DescriptionDebug logging function. Writes to the file referenced in the ECPROXY_DEBUGFILE environment variable (if it exists). No-op otherwise.
Argumentsmessage
Examplemesg("myCleanup: about to delete $cmdFile on proxy target");
Note: This function automatically adds a newline to whatever it emits, so the caller does not have to incorporate a newline in message.


readFile

DescriptionReads a file.
ArgumentsfileName
ReturnsContents of the file. If there is an error, it returns an empty string.
Examplemy $data = readFile("foo.txt");


writeFile

DescriptionCreates a local file containing data.
ArgumentsfileName, data
Returns1 on success, undef on failure.
ExamplewriteFile("myWrapper.$ENV{COMMANDER_JOBSTEPID}.cmd", "perl foo.pl")


initDispatcher

DescriptionInitialize the operation dispatcher map to point to functions for the given protocol. For each operation, initDispatcher checks if a function named protocol_operation exists, and if so, assigns that function as the implementation for that operation.
Argumentsprotocol
ExampleinitDispatcher("ssh") sets the "connect" operation to run "ssh_connect", "uploadFile" => "ssh_uploadFile", etc.

 

setOperation

DescriptionSets the implementation of an operation to be a particular function.
Argumentsoperation, function. The 'function' argument may be the name of a function or a reference to a function.
ExamplesetOperation("ping", "my_ping"); sets the "ping" operation to run the "my_ping" function
ExamplesetOperation("ping", \&my_ping); same as above, but using a function ref
Note: This function manipulates the gDispatcher hash, but provides a safe interface to it.


loadFile

DescriptionLoad proxy customizations from a file.
Arguments fileName
ExampleloadFile("custom.pl")

setSSHKeyFiles

DescriptionSet the paths to the public and private key files that ssh will use to authenticate with the proxy target.
ArgumentspublicKeyFile, privateKeyFile
ExamplesetSSHKeyFiles('c:\foo\pub.key', 'c:\foo\priv.key')
Note: This is very useful on Windows proxies, where there is no reasonable default for ssh to use.

setSSHUser

DescriptionSet the name of the user to authenticate with the proxy target.
ArgumentsuserName
ExamplesetSSHUser('user1')
 Note: By default, the user name the agent is "running as" is used to log into the proxy target. If key-based authentication is configured on the target system such that 'agentUser' can log into the 'user1' account on the proxy target, this function leverages that configuration.

useMultipleSSHSessions

DescriptionNormally, ecproxy uses one ssh session with a number of "channels" to perform tasks like uploading files, running the command, and running a cleanup command on the proxy target. Some SSH servers don't allow this. This method configures ecproxy to use a separate SSH session for each operation; this requires authenticating with the SSH daemon on the proxy target several times, and thus it may perform worse than the single-session-multi-channel mode.
ArgumentsNone
ExampleuseMultipleSSHSessions()

Examples

Specify public/private key files for SSH

  1. Set the proxyCustomization property on the resource like this: ?setSSHKeyFiles('c:\foo\pub.key', 'c:\foo\priv.key');
  2. Set the ECPROXY_SSH_PRIVKEYFILE and ECPROXY_SSH_PUBKEYFILE environment variables on the proxy agent as system environment variables.

Override one of the operations (for example, to enable SSH connection with username/password)

Set the proxyCustomization property on the resource like this: ?sub myConnect($$) {...} setOperation("connect", \&myConnect);

Load proxy customizations from a file rather than having all the logic in the proxyCustomization property on the resource

Set the proxyCustomization property on the resource like this: ?loadFile('c:\foo\custom.pl');

Implement a whole new protocol

Specify protocol as 'myproto' and have a proxy customization block like this:

sub myproto_getDefaultTargetPort() {
...
} 
sub myproto_connect($;$) {
...
} 
sub myproto_uploadFile($$$) {
...
} 
sub myproto_uploadWrapperScript($$$) {
# Note: As of 3.1.2, the default implementation is likely good enough, so 
it may not be necessary to define this override
....
} sub myproto_runCommand($$) {
...
} sub myproto_cleanup($$$) {
# Note: As of 3.1.2, the default implementation is likely good enough, so 
it may not be necessary to define this override
....
} 
# Initialize the dispatcher to run these functionsinitDispatcher("myproto");

Override ping to do a connect operation (which does a full protocol handshake, authentication, and so on)

Write a specialized ping function for the proxy customization like this:

sub heavy_ping($$) {   
   my ($host, $port) = @_;
   return ssh_connect($host, $port);
} setOperation("ping", \&heavy_ping);

 

"Real World" Examples

ClusterExec

A basic integration for using clusterupload and clusterexec to reach a proxy target is here. It has been tested on a Windows target with a Cygwin installation. It will not work "out-of-the-box" because it makes the following assumptions:

To make this proxy customization work on a Windows machine that does not have Cygwin, the generateWrapperScript operation would need to be overridden with a function that generates a cmd batch script, and the generateWrapperInvocationCommand operation would have to be overridden to generate a "cmd /c ..." command rather than "sh ...".

MySQL

The idea here is that the proxy target need not be a host for running arbitrary commands. It could be a special entity like a db. This integration uses the mysql clt to run the step command (which should be SQL) on the db referenced by the proxy target host and port.

A bare-bones integration with MySQL:

# Set the path to the mysql binary; if the directory is in the proxy agent's
# PATH, this variable can simply contain the name of the executable.
my $gMySQL = "c:/cygwin/usr/local/tools/i686_win32/bin/mysql.exe";
sub mysql_getDefaultTargetPort() {
return 3306;
}
sub mysql_connect($;$) {
# This "protocol" implementation is just going to use the mysql
# command-line tool, so just save off host/port.
my $host = $_[0];
my $port = $_[1] || mysql_getDefaultTargetPort();
return {host => $host,
port => $port};
}
sub mysql_uploadFile($$$) {
my ($context, $cmdFile, $rmtCmdFile) = @_;
# We do not need to upload the command-file to the proxy target.
# We are going to run the mysql clt on the proxy agent to run
# the query (contained in the local command-file),
# so just save off the name of the command-file.
$context->{cmdFile} = $cmdFile;
}
sub mysql_uploadWrapperScript($$$) {
my ($context, $workingDir, $wrapperScript) = @_;
# This has no meaning for this integration. No-op.
}
sub mysql_runCommand($$) {
my ($context, $cmdLine) = @_;
# cmdLine is a command-line for running the wrapper script, which
# has no meaning for this integration. We just want to run
# 'mysql' for the desired host/port with the command-file.
system("$gMySQL -D commander -h $context->{host} -P $context->{port} " .
"-u commander -pcommander -e \"source $context->{cmdFile}\"");
}
sub mysql_cleanup($$$) {
# We didn't create any temp files. No-op.
}
# Initialize the dispatcher to run these functions
initDispatcher("mysql");


Android

This example uses the adb tool to upload files to the device and run commands on it. Initial testing has been only against the android emulator, but it is implemented in such a way that it should work against a real android device attached using USB to the proxy agent, or a device on the network.

A first attempt at proxying to android devices:

# Set the path to the adb binary; if the directory is in the proxy agent's
# PATH, this variable can simply contain the name of the executable.

my $gADB = "c:/android-sdk-windows-1.6_r1/tools/adb.exe";

sub android_getDefaultTargetPort() {
    # Not sure what a good meaningful value is here.
    return 0;
}
sub android_connect($;$) {
    # This "protocol" implementation uses the adb
    # command-line tool.  Depending on the value of
    # host, construct the appropriate adb command-line
    # argument.

    my $host = $_[0];
    my $context = {};
 #   if ($host eq "emulator") {
    if ($host eq "localhost") {
        # We want to talk to the emulator running on this host.
        $context->{targetArg} = "-e";
    } elsif ($host eq "usb") {
        # We want to talk to the single android device connected
        # to the computer via a USB.
        $context->{targetArg} = "-d";
    } else {
        # This must be the serial number of some device somewhere.
        $context->{targetArg} = "-s $host";
    }
    return $context;
}
sub android_uploadFile($$$) {
    my ($context, $srcFile, $tgtFile) = @_;
    my($filename, $directories) = fileparse($tgtFile);

    my $result = `$gADB $context->{targetArg} push $srcFile 
    "/data/tmp/$filename" 2>&1`;
    if ($? != 0) {
        die ("android_uploadFile: Error uploading file $srcFile to 
        /data/tmp/$filename: $result\n");
    }
}
sub android_runCommand($$) {
    my ($context, $cmdLine) = @_;

    # cmdLine is a command-line for running the wrapper script, which
    # has no meaning for this integration.  We just want to run
    # 'adb' for the desired device with the command-file.

    system("$gADB $context->{targetArg} shell $cmdLine");
}
sub android_cleanup($$$) {
    my ($context, $remoteCmdFile, $remoteWrapperFile) = @_;

    # This was copied from ssh_cleanup except that we do "rm",
    # not "rf -f".

    mesg("cleaning up");

    # Delete the locally generate wrapper file.
    unlink($context->{"wrapperFile"});

    # Delete the cmd-file and wrapper script file on the proxy target.
    $gDispatcher{"runCommand"}($context,
        "rm $remoteWrapperFile $remoteCmdFile");
}
sub android_ping($;$) {
    my ($host, $port) = @_;
    $port = $gDispatcher{"getDefaultTargetPort"}() unless isPortValid($port);

    my $socket = IO::Socket::INET->new(PeerAddr => $host,    
                                       PeerPort => $port,
                                       Proto    => "tcp",                   
                                       Type     => SOCK_STREAM)
                           or die "Couldn't connect to $host:$port : $@\n";
}

# Initialize the dispatcher to run these functions
initDispatcher("android");
1;
ecremotefilecopy

When Commander agents (on platforms other than Linux or Windows) run steps that create log files in a workspace the Commander web server cannot access (through Linux or Windows agents), use ecremotefilecopy to recreate job logs so they are visible on those Commander agents, which then enables the web server to retrieve and render those log files.

Using postp and ecremotefilecopy, the log file is populated and recreated in a workspace accessible to the Commander web server, allowing the Job Details page to display the log file. Although this functionality is supported, it is not a recommended method of operation. This method should be used only as a last resort when a shared file system (between alternate agents and primary platform agents [Linux and Windows]) is not possible.

The reasons ecremotefilecopy is not recommended are:

Setting up the process

Creating a Setup Step

In your procedure, create a step called "Setup". This step needs to be in your procedure before any step running on a remote workspace.

Note: This is your top-level procedure, not a subprocedure.

Step Name: Setup

Command(s): ecremotefilecopy setup

Resource: local

Workspace: you have two choices:

Update the Postprocessor field for steps in your procedure

This step defines a postprocessor that will run at the end of the steps you specify. Add the following information to every step running on a remote agent if you want to see its results in the web interface.

Add a Final Step to your procedure

This step adds a new step at the end of your existing procedure. This step finds all properties created by the postprocessor, then reads the properties and creates local log files based on the properties, then deletes the properties.

Step Name: Recreate the Log Files

Always run step: (Check the box)

Command(s): ecremotefilecopy recreateFiles

Resource: local

Workspace: default

After the final step runs, you should see links (icons) displayed in the Log column on the Jobs Details page.

Click the icon to display the log file.

Copying Other Files from the Workspace

By default, ecremotefilecopy copies only postp log and diag files, and step logs. You can also copy other files from the workspace using a function named postpEndHook2.

You must do the following in your step:

  1. Make sure that the file you want to copy is in the step workspace. (It can be copied there, created there, etc.)
  2. For your procedure, create a property (named postpEndHook2, for example).
  3. Define a function named postpEndHook2 inside the property. For example:
sub postpEndHook2() {

    # Missing param does not cause an error
    $::gCommander->abortOnError(0);

    # Add filename to a "special" property such that it will be picked up by ecremotefilecopy
    my $fileName = 'paul.txt';
    copyFileToProperty ($fileName);

    # Restore default error handling
    $::gCommander->abortOnError(1);
}
  1. Add the following line in the Postprocessor field of this step:
postp --check none --loadProperty /myJob/jobSteps[Setup]/postpExtensions --loadProperty /myProcedure/postpEndHook2

 

ZKConfigTool

You can ZooKeeper server with configuration information that all ElectricCommander server nodes will use in a clustered configuration. This command-line tool that imports your ElectricCommander database configuration information into your ZooKeeper server. The following minimum set of files is imported:

Prerequisites

Command

$ java -jar zk-config-tool-jar-with-dependencies.jar -<options>

Usage: ZKConfigTool

OptionDescription
-commanderPropertiesFile <path_to_file> Import the ElectricCommander commander.properties file.
-databasePropertiesFile <path_to_file> Import the ElectricCommander database.properties file.
-helpShow the command help.
-keystoreFile <path_to_file>Import the ElectricCommander keystore file.
-passkeyFile <path_to_file> Import the ElectricCommander passkey file.
-wrapperConfFile <path_to_file>Import thewrapper.conf file.
-writeFile <path_on_ zookeeper> <path_to_file> Write the specified file to the ZooKeeper server.

Import Files

Run the ClusterTool command to completely populate the ZooKeeper server with configuration information. The system must have the Tools install and can be able to communicate with ZooKeeper.

Example command:

COMMANDER_ZK_CONNECTION=10.168.33.10:2181 java -jar zk-config-tool-jar-with-dependencies.jar --databasePropertiesFile database.properties --keystoreFile keystore --passkeyFile passkey --wrapperConfFile wrapper.conf --commanderPropertiesFile commander.properties

Example command:

$ COMMANDER_ZK_CONNECTION=10.168.33.10:2181 java -jar zk-config-tool-jar-with-dependencies.jar

of