ensembl-hive
2.5
|
Upon initialisation, GuestProcess forks, and the child process executes the wrapper that will allow running Runnables of the other language. The communication is ensured by two pipes and is schematically similar to running "| wrapper |", except that GuestProcess uses non-standard file descriptors, thus allowing the Runnable to still use std{in,out,err}. The wrapper receives the two file-numbers that it is meant to use (one for reading data from GuestProcess, and one to send data to GuestProcess). All the messages are passed around in single-line JSON structures. The protocol is described below using the convention: ---> represents a message sent to the child process, <--- represents a message sent by the child process The initialisation (in the constructor) consists in checking that both sides spek the same version of the protocol: <--- { "version": "XXX" } ---> "OK" GuestProcess will bail out if the response is not "OK" Then, the child process (i.e. the runnable) will send its default parameters to GuestProcess. This fills the usual param_defaults() section of the Runnable: <--- { ... param_defaults ... } ---> "OK" The child process then goes to sleep, waiting for jobs to be seeded. Meanwhile, GuestProcess enters a number of life_cycle() executions (as triggered by Worker). Each one first sends a JSON object to the child process to initialize the job parameters ---> { "input_job": { "parameters": { ... the unsubstituted job parameters as compiled by Worker ... }, // followed by several attributes of the job "input_id": { ... }, "dbID": XXX, "retry_count": XXX }, "execute_writes": [1|0], "debug": XXX } <--- "OK" From this point, GuestProcess acts as a server, listening to events sent by the child. Events are JSON objects composed of an "event" field (the name of the event) and a "content" field (the payload). Events can be of the following kinds (with the expected response from GuestProcess): <--- JOB_STATUS_UPDATE // The content is one of "PRE_CLEANUP", "FETCH_INPUT", "RUN", "WRITE_OUTPUT", "POST_HEALTHCHECK", "POST_CLEANUP" ---> "OK" <--- WARNING // The content is a JSON object: { "message": "XXX", "is_error": [true|false], } ---> "OK" <--- DATAFLOW // The content is a JSON object: { "branch_name_or_code": XXX, "output_ids": an array or a hash, "params": { "substituted": { ... the parameters that are currently substituted ... } "unsubstituted": { ... the parameters that have not yet been substituted ... } } } ---> dbIDs of the jobs that have been created <--- WORKER_TEMP_DIRECTORY // The content is the "worker_temp_directory_name" as defined in the runnable (or null otherwise) ---> returns the temporary directory of the worker <--- JOB_END // The content is a JSON object describing the final state of the job { "complete": [true|false], "job": { "autoflow": [true|false], "lethal_for_worker": [true|false], "transient_error": [true|false], }, "params": { "substituted": { ... the parameters that are currently substituted ... } "unsubstituted": { ... the parameters that have not yet been substituted ... } } } ---> "OK"
Definition at line 105 of file GuestProcess.pm.
protected Hashref Bio::EnsEMBL::Hive::GuestProcess::_get_all_registered_wrappers | ( | ) |
Example :
Description : Lists all the languages and wrappers that are registered (either under via a EHIVE_WRAPPER environment variable, or via a "wrapper" file under $EHIVE_ROOT_DIR/wrappers/). Returntype : Hashref { String => String } Exceptions : None
protected String Bio::EnsEMBL::Hive::GuestProcess::_get_wrapper_for_language | ( | ) |
Example :
Description : Finds the wrapper that understands the given language Returntype : String Exceptions : Can die if the wrapper doesn't exist
public Bio::EnsEMBL::Hive::GuestProcess::check_version_compatibility | ( | ) |
Undocumented method
public IO::Handle Bio::EnsEMBL::Hive::GuestProcess::child_in | ( | ) |
Example :
Example :
Description : Getter/Setter for the file handle that allows talking to the child process. Returntype : IO::Handle Exceptions : none
public IO::Handle Bio::EnsEMBL::Hive::GuestProcess::child_out | ( | ) |
Example :
Example :
Description : Getter/Setter for the file handle that allows receiving data from the child process. Returntype : IO::Handle Exceptions : none
public Int Bio::EnsEMBL::Hive::GuestProcess::child_pid | ( | ) |
Example :
Example :
Description : Getter/Setter for the process ID of the child Returntype : integer Exceptions : none
public void Bio::EnsEMBL::Hive::GuestProcess::DESTROY | ( | ) |
Description : Destructor: tells the child to exit by sending an empty JSON object Returntype : none
public String Bio::EnsEMBL::Hive::GuestProcess::get_protocol_version | ( | ) |
Example :
Description : Returns the version number of the communication protocol Returntype : String
public Instance Bio::EnsEMBL::Hive::GuestProcess::json_formatter | ( | ) |
Example :
Example :
Description : Getter/Setter for the JSON formatter. Returntype : instance of JSON Exceptions : none
public Hashref Bio::EnsEMBL::Hive::GuestProcess::life_cycle | ( | ) |
Example :
Description : Runs the life-cycle of the input job and returns the timings of each Runnable method (fetch_input, run, etc). See the description of this module for details about the protocol Returntype : Hashref Exceptions : none
public Bio::EnsEMBL::Hive::GuestProcess Bio::EnsEMBL::Hive::GuestProcess::new | ( | ) |
Arg[1] : $language: the programming language the external runnable is in Arg[2] : $module: the name of the runnable (usually a package name) Example :
Description : Constructor Returntype : Bio::EnsEMBL::Hive::GuestProcess Exceptions : if $language or $module is not defined properly or if the pipes / child process could not be created
public Hashref Bio::EnsEMBL::Hive::GuestProcess::param_defaults | ( | ) |
Example :
Example :
Description : Getter/Setter for the default parameters of this runnable. Hive only uses it as a getter, but here, we need a setter to define the parameters at the Perl layer once they've been retrieved from the child process. Returntype : Hashref Exceptions : none
public void Bio::EnsEMBL::Hive::GuestProcess::print_debug | ( | ) |
Example :
Description : Prints a message if $self->{'_protocol_debug'} is set Returntype : none
public Perl Bio::EnsEMBL::Hive::GuestProcess::read_message | ( | ) |
Example :
Description : Wait for and read the next message coming from the child. Again, the message itself is serialized and transmitted via the pipe Returntype : Perl structure Exceptions : raised by JSON / IO::Handle
public void Bio::EnsEMBL::Hive::GuestProcess::send_message | ( | ) |
Example :
Description : Send the Perl structure to the child process via the pipe (and serialized in JSON). Returntype : none Exceptions : raised by JSON / IO::Handle
public void Bio::EnsEMBL::Hive::GuestProcess::send_response | ( | ) |
Example :
Description : Wrapper around send_message to send a response to the child. Returntype : none Exceptions : raised by JSON / IO::Handle
public void Bio::EnsEMBL::Hive::GuestProcess::wait_for_OK | ( | ) |
Example :
Description : Wait for the child process to send the OK signal Returntype : none Exceptions : dies if the response is not OK, or anything raised by read_message()
public String Bio::EnsEMBL::Hive::GuestProcess::worker_temp_directory_name | ( | ) |
Example :
Description : Returns the name of the temp directory for this module The value in $self is initialized at the WORKER_TEMP_DIRECTORY event above and returned to the caller if defined. This allows runnables to redefine the name Returntype : string Exceptions : none