Public Member Functions
protected	_compare_job_warnings ()

protected	_compare_job_dataflows ()

public void	standaloneJob ()

public void	init_pipeline ()

protected void	_test_ehive_script ()

public void	runWorker ()

public void	seed_pipeline ()

public void	beekeeper ()

public void	tweak_pipeline ()

public void	generate_graph ()

public void	visualize_jobs ()

public void	peekJob ()

public void	db_cmd ()

public void	run_sql_on_db ()

public void	load_sql_in_db ()

public Bio::EnsEMBL::Hive::DBSQL::DBConnection	make_new_db_from_sqls ()

public void	make_hive_db ()

public Listref	get_test_urls ()

public Db	get_test_url_or_die ()

public void	safe_drop_database ()

public Array	all_source_files ()

Detailed Description

Definition at line 17 of file Test.pm.

Member Function Documentation

◆ _compare_job_dataflows()

protected Bio::EnsEMBL::Hive::Utils::Test::_compare_job_dataflows ( )

Undocumented method

Code:

click to view

◆ _compare_job_warnings()

protected Bio::EnsEMBL::Hive::Utils::Test::_compare_job_warnings ( )

Undocumented method

Code:

click to view

◆ _test_ehive_script()

protected void Bio::EnsEMBL::Hive::Utils::Test::_test_ehive_script ( )

  Arg[1]      : String $script_name. The name of the script (assumed to be found in
                ensembl-hive/scripts/ once the .pl suffix added)
  Arg[2]      : String $url. The location of the database
  Arg[3]      : Arrayref $args. Extra arguments given to the script
  Arg[4]      : String $test_name (optional). The name of the test
  Description : Generic method that can run any eHive script and check its return status
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : other methods in Utils::Test
  Status      : Stable

Code:

click to view

◆ all_source_files()

public Array Bio::EnsEMBL::Hive::Utils::Test::all_source_files ( )

  Arg [n]    : Directories to scan.
  Example    :

my @files = all_source_files('modules');

  Description: Scans the given directories and returns all found instances of
               source code. This includes Perl (pl,pm,t), C(c,h) and
               SQL (sql) suffixed files.
  Returntype : Array of all found files

Code:

click to view

◆ beekeeper()

public void Bio::EnsEMBL::Hive::Utils::Test::beekeeper ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : Arrayref $args. Extra arguments given to beekeeper.pl
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

beekeeper($url, [$arg1, $arg2], 'Run beekeeper with two arguments');

  Description : Very generic function to run beekeeper on the given database with the given arguments
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ db_cmd()

public void Bio::EnsEMBL::Hive::Utils::Test::db_cmd ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : Arrayref $args. Extra arguments given to db_cmd.pl
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

db_cmd($url, [-sql => 'DROP DATABASE'], 'Drop the database');

  Description : Very generic function to run db_cmd.pl on the given database with the given arguments
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ generate_graph()

public void Bio::EnsEMBL::Hive::Utils::Test::generate_graph ( )

  Arg[1]      : String $url or undef. The location of the database
  Arg[2]      : Arrayref $args. Extra arguments given to generate_graph.pl
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

generate_graph($url, [-output => 'lm_analyses.png'], 'Generate a PNG A-diagram');

  Description : Very generic function to run generate_graph.pl on the given database with the given arguments
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ get_test_url_or_die()

public Db Bio::EnsEMBL::Hive::Utils::Test::get_test_url_or_die ( )

  Arg [1]     : see get_test_urls()
  Example     :

my $url = get_test_url_or_die(-driver => 'mysql', -tag => 'longmult')

  Example     :

my $url = get_test_url_or_die(-tag => 'gcpct')

  Example     :

my $url = get_test_url_or_die(-driver => 'sqlite')

  Example     :

my $url = get_test_url_or_die()

  Description : Wrapper around get_test_urls() that returns one of the test URLs, or
                die if no databases are available
  Returntype  : db connection URL as a string

Code:

click to view

◆ get_test_urls()

public Listref Bio::EnsEMBL::Hive::Utils::Test::get_test_urls ( )

  Arg [1]     : -driver => driver, -tag => tag, -no_user_prefix => 1
  Example     :

my @urls = get_test_urls(-driver => 'mysql', -tag => 'longmult')

  Example     :

my @urls = get_test_urls(-tag => 'gcpct')

  Example     :

my @urls = get_test_urls(-driver => 'sqlite')

  Example     :

my @urls = get_test_urls()

  Description : Creates a listref containing db urls based on the drivers specified in
              : the environment variable EHIVE_TEST_PIPELINE_URLS.
              : The URLs will be standard eHive URLs, looking like driver://connection/database
              : A database name consisting of [username]_ehive_test will be created
              : and placed in the URL
              : For example - mysql://me@127.0.0.1/ghopper_ehive_test
              :
              : If -tag is specified, then the list will have db names appended with '_tag' 
              : For example - (-tag => 'longmult') giving mysql://me@127.0.0.1/ghopper_ehive_test_longmult
              :
              : If -driver is specified, then the list will be restricted to urls for the
              : particular driver or comma-separated list of drivers specified (e.g. 'mysql,pgsql')
              :
              : If -no_user_prefix is specified, then the automatically-generated database names
              : won't be prefixed with the name of the current user
              :
              : If no drivers are specified in EHIVE_TEST_PIPELINE_URLS, it will check
              : to see if sqlite is available in the current path, and return a sqlite url
              : in the listref. Otherwise it will return an empty listref.
              :
 Returntype   : listref of db connection URLs as strings

Code:

click to view

sub get_test_urls {
  croak "wrong number of arguments for get_test_urls(); has to be even" if (scalar(@_) % 2);
  my %args = @_;
  my %argcheck = %args;
  delete(@argcheck{qw(-driver -tag -no_user_prefix)});
  croak "get_test_urls only accepts -driver and -tag as arguments" if (scalar(keys(%argcheck)) > 0);
 
 
  my %url_parses_by_driver;
  if (defined($ENV{EHIVE_TEST_PIPELINE_URLS})) {
    my @urls = split( /[\s,]+/, $ENV{EHIVE_TEST_PIPELINE_URLS} );
    foreach my $url (@urls) {
      if (my $parse = Bio::EnsEMBL::Hive::Utils::URL::parse($url) ) {
    push(@{$url_parses_by_driver{$parse->{driver}}}, $parse);
      } else {
    croak "badly formed url \"$url\" in EHIVE_TEST_PIPELINE_URLS";
      }
    }
  } else {
    my ($fh, $filename) = tempfile(UNLINK => 1);
    $url_parses_by_driver{'sqlite'} = [Bio::EnsEMBL::Hive::Utils::URL::parse('sqlite:///' . $filename)];
  }
 
  my $constructed_db_name = ($args{-no_user_prefix} ? '' : whoami().'_') . 'ehive_test';
 
  my @driver_parses;
  if (defined($args{-driver})) {
    my @requested_drivers = split(/,/, $args{-driver});
    
    foreach my $requested_driver (@requested_drivers) {
      $requested_driver =~ s/^\s+|\s+$//g;  #trim whitespace
      if (defined($url_parses_by_driver{$requested_driver})) {
    push(@driver_parses, @{$url_parses_by_driver{$requested_driver}});
      }
    }
  } else {
    foreach my $parses_for_driver (values(%url_parses_by_driver)) {
      push (@driver_parses, @{$parses_for_driver});
    }
  }
 
  my @list_of_urls;
  foreach my $parsed_url (@driver_parses) {
 
    ## Use the default database name if needed, and append the tag (if given)
    $parsed_url->{'dbname'} ||= $constructed_db_name;
    $parsed_url->{'dbname'} .= $constructed_db_name if ($parsed_url->{'driver'} eq 'sqlite') && ($parsed_url->{'dbname'} =~ /\/$/);
    $parsed_url->{'dbname'} .= '_'.$args{-tag} if defined $args{-tag};
 
    my $final_url = Bio::EnsEMBL::Hive::Utils::URL::hash_to_url($parsed_url);
 
    push (@list_of_urls, $final_url); 
  }
 
  return \@list_of_urls;
}

◆ init_pipeline()

public void Bio::EnsEMBL::Hive::Utils::Test::init_pipeline ( )

  Arg[1]      : String $file_or_module. The location of the PipeConfig file
  Arg[2]      : String $url. The location of the database to be created
  Arg[3]      : (optional) Arrayref $args. Extra parameters of the pipeline (as on the command-line)
  Arg[4]      : (optional) Arrayref $tweaks. Tweaks to be applied to the database (as with the -tweak command-line option)
  Example     :

init_pipeline(
                   'Bio::EnsEMBL::Hive::Examples::LongMult::PipeConfig::LongMultServer_conf',
                   $server_url,
                   [],
                   ['pipeline.param[take_time]=0']
               );

  Description : Initialize a new pipeline database for the given PipeConfig module name on that URL.
                $options simply represents the command-line options one would give on the command-line.
                Additionally, tweaks can be defined. Note that -hive_force_init is automatically added.
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ load_sql_in_db()

public void Bio::EnsEMBL::Hive::Utils::Test::load_sql_in_db ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : String $sql_file. The location of a file to load in the database
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

load_sql_in_db($url, $file_with_sql_commands);

  Description : Execute an SQL script on the given database and test its execution.
                This expects the command-line client to return a non-zero code in
                case of a failure.
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ make_hive_db()

public void Bio::EnsEMBL::Hive::Utils::Test::make_hive_db ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : Boolean $use_triggers (optional, default 0). Whether we want to load the SQL triggers
  Example     :

make_hive_db($url);

  Description : Create a new (empty) eHive database using the two above functions.
                This function follows the same step as init_pipeline
                Note that it first issues a DROP DATABASE statement in case the database already exists
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ make_new_db_from_sqls()

public Bio::EnsEMBL::Hive::DBSQL::DBConnection Bio::EnsEMBL::Hive::Utils::Test::make_new_db_from_sqls ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : Arrayref of string $sqls. Each element can be a SQL command or file to load
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

make_new_db_from_sqls($url, 'CREATE TABLE sweets (name VARCHAR(40) NOT NULL, quantity INT UNSIGNED NOT NULL)');

  Description : Create a new database and apply a list of SQL commands using the two above functions.
                When an SQL command is a valid filename, the file is loaded rather than the command executed.
                Note that it first issues a DROP DATABASE statement in case the database already exists
  Returntype  : Bio::EnsEMBL::Hive::DBSQL::DBConnection $dbc
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ peekJob()

public void Bio::EnsEMBL::Hive::Utils::Test::peekJob ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : Arrayref $args. Extra arguments given to peekJob.pl
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

peekJob($url, [-job_id => 1], 'Check params for job 1');

  Description : Very generic function to run peekJob.pl on the given database with the given arguments
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ run_sql_on_db()

public void Bio::EnsEMBL::Hive::Utils::Test::run_sql_on_db ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : String $sql. The SQL to run on the database
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

run_sql_on_db($url, 'INSERT INTO sweets (name, quantity) VALUES (3, 'Snickers')');

  Description : Execute an SQL command on the given database and test its execution. This expects the
                command-line client to return a non-zero code in case of a failure.
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ runWorker()

public void Bio::EnsEMBL::Hive::Utils::Test::runWorker ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : Arrayref $args. Extra arguments given to runWorker
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

runWorker($url, [ -can_respecialize => 1 ]);

  Description : Run a worker on the given pipeline in the current process.
                The worker options have been divided in three groups: the ones affecting its specialization,
                the ones affecting its "life" (how long it lasts), and the ones controlling its execution mode.
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ safe_drop_database()

public void Bio::EnsEMBL::Hive::Utils::Test::safe_drop_database ( )

  Arg[1]      : DBAdaptor $hive_dba
  Example     :

safe_drop_database( $hive_dba );

  Description : Wait for all workers to complete, disconnect from the database and drop it.
  Returntype  : None
  Caller      : test scripts

Code:

click to view

◆ seed_pipeline()

public void Bio::EnsEMBL::Hive::Utils::Test::seed_pipeline ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : Arrayref $args. Extra arguments given to seed_pipeline
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

$seed_pipeline($url, [$arg1, $arg2], 'Run seed_pipeline with two arguments');

  Description : Very generic function to run seed_pipeline on the given database with the given arguments
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ standaloneJob()

public void Bio::EnsEMBL::Hive::Utils::Test::standaloneJob ( )

  Example     :

standaloneJob('Bio::EnsEMBL::Hive::RunnableDB::JobFactory',
                             { 'inputlist' => [ [1,2], [3,4] ], 'column_names' => ['a', 'b'] },
                             [
                               [ 'DATAFLOW',
                                 [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 }, ],
                                 2
                               ]
                             ]
               );

  Description : Run a given Runnable in "standalone job" mode, i.e. with parameters but no connection to the database.
                One can also give a list of events that the job is expected to raise. Currently, dataflows and warnings
                are supported. Examples can be found under t/05.runnabledb/
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

sub standaloneJob {
    my ($module_or_file, $param_hash, $expected_events, $flags) = @_;
 
    my $events_to_test = $expected_events ? [@$expected_events] : undef;
 
    my $input_id = stringify($param_hash);
 
    local $Data::Dumper::Terse = 1;
    local $Data::Dumper::Sortkeys = 1;
 
    # When a list of events is given, it must match exactly what the
    # Runnable does (no missing / extra events, etc)
    my $_test_event = sub {
        my ($triggered_type, @got) = @_;
        if (@$events_to_test) {
            my $expects = shift @$events_to_test;
            my $expected_type = shift @$expects;
            if ($triggered_type ne $expected_type) {
                fail("Got a $triggered_type event but was expecting $expected_type");
                diag "Got: " . Dumper([@_]);
            } elsif ($triggered_type eq 'WARNING') {
                _compare_job_warnings(\@got, $expects) or diag "Got: " . Dumper([@_]);
            } else {
                _compare_job_dataflows(\@got, $expects) or diag "Got: " . Dumper([@_]);
            }
        } else {
            fail("event-stack is empty but the job emitted an event");
            diag "Got: " . Dumper([@_]);
        }
    };
 
    # Local redefinition to hijack the events
    local *Bio::EnsEMBL::Hive::Process::dataflow_output_id = sub {
        shift;
        &$_test_event('DATAFLOW', @_);
        return [1];
    } if $expected_events;
 
    # Local redefinition to hijack the events
    local *Bio::EnsEMBL::Hive::Process::warning = sub {
        shift;
        &$_test_event('WARNING', @_);
    } if $expected_events;
 
    subtest "standalone run of $module_or_file" => sub {
        plan tests => 2 + ($expected_events ? 1+scalar(@$expected_events) : 0);
        lives_ok(sub {
            my $is_success = Bio::EnsEMBL::Hive::Scripts::StandaloneJob::standaloneJob($module_or_file, $input_id, $flags, $flags->{flow_into}, $flags->{language});
            if ($flags->{expect_failure}) {
                ok(!$is_success, 'job failed as expected');
            } else {
                ok($is_success, 'job completed');
            }
        }, sprintf('standaloneJob("%s", %s, (...), %s)', $module_or_file, stringify($param_hash), stringify($flags)));
 
        if ($expected_events) {
            ok(!scalar(@$events_to_test), 'no untriggered events');
            diag("Did not receive: " . Dumper($events_to_test)) if scalar(@$events_to_test);
        }
    }
}

◆ tweak_pipeline()

public void Bio::EnsEMBL::Hive::Utils::Test::tweak_pipeline ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : Arrayref $args. Extra arguments given to beekeeper.pl
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

tweak_pipeline($url, [$arg1, $arg2], 'Run tweak_pipeline with two arguments');

  Description : Very generic function to run tweak_pipeline on the given database with the given arguments
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

◆ visualize_jobs()

public void Bio::EnsEMBL::Hive::Utils::Test::visualize_jobs ( )

  Arg[1]      : String $url. The location of the database
  Arg[2]      : Arrayref $args. Extra arguments given to visualize_jobs.pl
  Arg[3]      : String $test_name (optional). The name of the test
  Example     :

visualize_jobs($url, [-output => 'lm_jobs.png', -accu_values], 'Generate a PNG J-diagram with accu values');

  Description : Very generic function to run visualize_jobs.pl on the given database with the given arguments
  Returntype  : None
  Exceptions  : TAP-style
  Caller      : general
  Status      : Stable

Code:

click to view

The documentation for this class was generated from the following file:

modules/Bio/EnsEMBL/Hive/Utils/Test.pm

Public Member Functions

Detailed Description

Member Function Documentation

◆ _compare_job_dataflows()

◆ _compare_job_warnings()

◆ _test_ehive_script()

◆ all_source_files()

◆ beekeeper()

◆ db_cmd()

◆ generate_graph()

◆ get_test_url_or_die()

◆ get_test_urls()

◆ init_pipeline()

◆ load_sql_in_db()

◆ make_hive_db()

◆ make_new_db_from_sqls()

◆ peekJob()

◆ run_sql_on_db()

◆ runWorker()

◆ safe_drop_database()

◆ seed_pipeline()

◆ standaloneJob()

◆ tweak_pipeline()

◆ visualize_jobs()