Inheritance diagram for Bio::EnsEMBL::Hive::RunnableDB::SystemCmd:

Public Member Functions
public	param_defaults ()

public	run ()

public	write_output ()

public Void	complete_early_if_branch_connected ()

Public Member Functions inherited from Bio::EnsEMBL::Hive::Process
public	new ()

public	life_cycle ()

public	say_with_header ()

public	enter_status ()

public	warning ()

public	param_defaults ()

public	fetch_input ()

public	run ()

public	write_output ()

public Bio::EnsEMBL::Hive::Worker	worker ()

public	execute_writes ()

public Bio::EnsEMBL::Hive::DBSQL::DBAdaptor	db ()

public Bio::EnsEMBL::Hive::DBSQL::DBConnection	dbc ()

public Bio::EnsEMBL::Hive::DBSQL::DBConnection	data_dbc ()

public Returns	run_system_command ()

public Bio::EnsEMBL::Hive::AnalysisJob	input_job ()

public	input_id ()

public	param ()

public	param_required ()

public	param_exists ()

public	param_is_defined ()

public	param_substitute ()

public	dataflow_output_id ()

public	dataflow_output_ids_from_json ()

public	throw ()

public This	complete_early ()

public Int	debug ()

public	worker_temp_directory ()

public	cleanup_worker_temp_directory ()

Detailed Description

Synopsis

standaloneJob.pl Bio::EnsEMBL::Hive::RunnableDB::SystemCmd --cmd 'ls -1 ${ENSEMBL_CVS_ROOT_DIR}/ensembl-hive/modules/Bio/EnsEMBL/Hive/RunnableDB/*.pm >building_blocks.list'

Description

    This RunnableDB module acts as a wrapper for shell-level command lines. If you behave you may also use parameter substitution.

    The command line must be stored in the parameters() as the value corresponding to the 'cmd' key.
    It allows to pass in other parameters and use the parameter substitution mechanism in its full glory.

    This Runnable also allows the creation of dataflow using JSON stored in an external file.
    Each line of this file contains an optional branch number, followed by a complete JSON serialisation of the parameters (output_id)
    appearing on the same single line. For example, a line to direct dataflow on branch 2 might look like:

          2 {"parameter_name" : "parameter_value"}

    If no branch number is provided, then dataflow of those parameters will occour on the branch number
    passed to SystemCmd in the 'dataflow_branch' parameter, if given. Otherwise, it will default to
    branch 1 (autoflow).

    A sample file is provided at ${EHIVE_ROOT_DIR}/modules/Bio/EnsEMBL/Hive/Examples/SystemCmd/PipeConfig/sample_files/Inject_JSON_Dataflow_example.json

Definition at line 31 of file SystemCmd.pm.

Member Function Documentation

◆ complete_early_if_branch_connected()

public Void Bio::EnsEMBL::Hive::RunnableDB::SystemCmd::complete_early_if_branch_connected ( )

  Arg[1]      : (string) message
  Arg[2]      : (integer) branch number
  Description : Wrapper around complete_early that first checks that the
                branch is connected to something.
  Returntype  : void if the branch is not connected. Otherwise doesn't return

Code:

click to view

◆ param_defaults()

public Bio::EnsEMBL::Hive::RunnableDB::SystemCmd::param_defaults ( )

Undocumented method

Code:

click to view

◆ run()

public Bio::EnsEMBL::Hive::RunnableDB::SystemCmd::run ( )

    Description : Implements run() interface method of Bio::EnsEMBL::Hive::Process that is used to perform the main bulk of the job (minus input and output).
                  Here it actually runs the command line.

    param('cmd'): The recommended way of passing in the command line. It can be either a string, or an array-ref of strings. The later is safer if some of the
                  arguments contain white-spaces.

    param('*'):   Any other parameters can be freely used for parameter substitution.

Code:

click to view

◆ write_output()

public Bio::EnsEMBL::Hive::RunnableDB::SystemCmd::write_output ( )

    Description : Implements write_output() interface method of Bio::EnsEMBL::Hive::Process that is used to deal with job's output after the execution.
                  Here we take actions based on the command's exit status.

Code:

click to view

sub write_output {
    my $self = shift;
 
    my $return_value = $self->param('return_value');
 
    ## Success
    unless ($return_value) {
        # FIXME branch number
        $self->dataflow_output_ids_from_json($self->param('dataflow_file'), $self->param('dataflow_branch')) if $self->param('dataflow_file');
        return;
    }
 
    ## Error processing
    my $stderr = $self->param('stderr');
    my $flat_cmd = $self->param('flat_cmd');
 
    if ($return_value == -1) {
        # system() could not start, or wait() failed
        die sprintf( "Could not start '%s': %s\n", $flat_cmd, $stderr);
 
    } elsif ($return_value == -2) {
        $self->complete_early_if_branch_connected("The command was aborted because it exceeded the allowed runtime. Flowing to the -2 branch.\n", -2);
        die "The command was aborted because it exceeded the allowed runtime, but there are no dataflow-rules on branch -2.\n";
 
    # Lower 8 bits indicate the process has been killed and did not complete.
    } elsif ($return_value & 255) {
        # It can happen because of a MEMLIMIT / RUNLIMIT, which we
        # know are not atomic. The best is to wait a bit that LSF kills
        # the worker too
        sleep 30;
        # If we reach this point, it was killed for another reason.
        die sprintf( "'%s' was killed with code=%d\nstderr is: %s\n", $flat_cmd, $return_value, $stderr);
 
    } else {
        # "Normal" process exit with a non-zero code (in the upper 8 bits)
        $return_value >>= 8;
 
        # We create a dataflow event depending on the exit code of the process.
        if (ref($self->param('return_codes_2_branches')) and exists $self->param('return_codes_2_branches')->{$return_value}) {
            my $branch_number = $self->param('return_codes_2_branches')->{$return_value};
            $self->complete_early(sprintf("The command exited with code %d, which is mapped to a dataflow on branch #%d.\n", $return_value, $branch_number), $branch_number);
        }
 
        if ($stderr =~ /Exception in thread ".*" java.lang.OutOfMemoryError: Java heap space at/) {
            $self->complete_early_if_branch_connected("Java heap space is out of memory. A job has been dataflown to the -1 branch.\n", -1);
            die $stderr;
        }
 
        die sprintf( "'%s' resulted in an error code=%d\nstderr is: %s\n", $flat_cmd, $return_value, $stderr);
    }
}

The documentation for this class was generated from the following file:

modules/Bio/EnsEMBL/Hive/RunnableDB/SystemCmd.pm

Public Member Functions

Detailed Description

Synopsis

Description

Member Function Documentation

◆ complete_early_if_branch_connected()

◆ param_defaults()

◆ run()

◆ write_output()