Description Greg Watson 2011-08-30 15:26:13 EDT

Currently, monitoring support for a job scheduler is provided by the DA driver plugin. However, there is no way for support for new job schedulers to be added other than adding a module to the DA driver. This is problematic as it prevents users (or developers) from adding support without rebuilding PTP.

The RM XML configuration should be able to specify the necessary information to provide support for a particular job scheduler. It is proposed that this be achieved by allowing the specification of a new workflow as part of the XML schema. A workflow is currently created for each supported job scheduler using the add_exec_step_to_workflow() function in LML_da_driver.pl. It is proposed that this be replaced with an XML description of the workflow with the following syntax:

		<workflow>
			<vardefs>
				... variable definitions ...
			</vardefs>
			<managed-files>
				... managed file definitions...
			</managed-files>
			<step id="... >
				... step definition ...
			</step>
			... additonal steps ...
		</workflow>
		
The DA driver would convert this description to the internal representation of the workflow, then execute the workflow as normal.

The <vardefs> section would be used to define variables that could be referred to in the workflow using the $varname syntax.

The <managed-files> section would allow the specification to provide additional functionality using perl (or other) scripts. As an example, a new script could be specified using:

		<managed-files>
			<file name="$tmpdir/da_nodes_info_LML.pl" mode="0755">
				<contents>
<![CDATA[
... source of perl script ...
]]>
				</contents>
			</file>
		</managed-files>
		
Files specified in the <managed-files> section are uploaded to the remote system and stored in the specified file name. They can then be executed during one or more workflow steps.

The <step> sections would be used to specify a series of commands that would be run during the step. For example:

			<step id="getdata" active="0" exec_after="" type="execute">
				<cmd exec="$tmpdir/da_system_info_LML.pl $tmpdir/sysinfo_LML.xml"/>
				<cmd exec="$tmpdir/da_jobs_info_LML.pl $tmpdir/jobs_LML.xml"/>
				<cmd exec="$tmpdir/da_nodes_info_LML.pl $tmpdir/nodes_LML.xml"/>
			</step>

This step would execute three commands in sequence. The exec_after attribute is used to define the ordering of steps (default is the first step). The assumption is that if a command fails, the whole script will fail. It may be necessary to provide additional attributes to deal with command failures.

The full XSD for the specification is as follows. Note that managed-files-type is a pre-existing definition in the JAXB schema.

	<xs:complexType name="workflow-type">
		<xs:sequence>
			<xs:element name="vardefs" minOccurs="0" maxOccurs="unbounded" type="rm:vardefs-type"/>
			<xs:element name="managed-files" minOccurs="0" maxOccurs="unbounded" type="rm:managed-files-type"/>
			<xs:element name="step" minOccurs="0" maxOccurs="unbounded" type="rm:step-type"/>
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="vardefs-type">
		<xs:choice>
			<xs:element name="var" minOccurs="0" maxOccurs="unbounded" type="rm:var-type"/>
		</xs:choice>
	</xs:complexType>
	<xs:complexType name="var-type">
		<xs:attribute name="key" type="xs:string"/>
		<xs:attribute name="value" type="xs:string"/>
	</xs:complexType>
	<xs:complexType name="step-type">
		<xs:choice>
			<xs:element name="cmd" minOccurs="0" maxOccurs="unbounded" type="rm:monitor-command-type"/>
		</xs:choice>
		<xs:attribute name="id" type="xs:string"/>
		<xs:attribute name="active" type="xs:string"/>
		<xs:attribute name="exec_after" type="xs:string"/>
		<xs:attribute name="type" type="xs:string"/>
	</xs:complexType>
	<xs:complexType name="monitor-command-type">
		<xs:attribute name="directory" type="xs:string"/>
		<xs:attribute name="redirectStderr" type="xs:boolean" default="false"/>
		<xs:attribute name="ignoreExitStatus" type="xs:boolean" default="false"/>
		<xs:attribute name="exec" type="xs:string"/>
		<xs:attribute name="name" type="xs:string"/>
	</xs:complexType>