13 When we attempt to create a
new semaphored group of jobs and the creation of the funnel job fails
14 because there is already a job with the same primary key, we reuse
this funnel job and simply top up
15 the semaphore group with extra fan jobs. This is what we call
"leaching".
17 This is a small pipeline to test leaching via
using templates.
19 Note 1:
this is not the recommended way to create a single backbone from multiple independent streams!
21 Note 2: the
'start' analysis is only needed because of the recently adopted rule of automatically param-stacking
22 the very first job that does not have a local parent (used mainly
for maintaining cross-database
return capability,
23 but could prove itself useful in other contexts as well). If/when we decide against
this feature,
24 we can also drop the
'start' analysis, leaving two independent seeding jobs in
'factory' analysis
25 and changing the leeching.t test accordingly (removing one execution of runWorker).
29 See the NOTICE file distributed with
this work
for additional information
30 regarding copyright ownership.
32 Licensed under the Apache License, Version 2.0 (the
"License"); you may not use
this file except in compliance with the License.
33 You may obtain a copy of the License at
37 Unless required by applicable law or agreed to in writing, software distributed under the License
38 is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
39 See the License
for the specific language governing permissions and limitations under the License.
43 Please subscribe to the Hive mailing list: http:
48 package Bio::EnsEMBL::Hive::Examples::Factories::PipeConfig::LeachingTest_conf;
53 use base (
'Bio::EnsEMBL::Hive::PipeConfig::HiveGeneric_conf'); # All Hive databases configuration files should inherit from HiveGeneric, directly or indirectly
56 sub pipeline_analyses {
59 { -logic_name =>
'start',
60 -module =>
'Bio::EnsEMBL::Hive::RunnableDB::Dummy',
66 'factory' => [ {
'inputlist' => [ 11, 33, 55, 66, 77 ]}, {
'inputlist' => [22, 44, 55, 66 ]} ],
72 { -logic_name =>
'factory',
73 -module =>
'Bio::EnsEMBL::Hive::RunnableDB::JobFactory',
75 'column_names' => [
'alpha' ],
77 -flow_into => { # There are two input jobs...
79 'A->1' => {
'funnel' => {
'funnel_id' => 7 }}, # ...however we want all of the children to report to one common funnel
83 { -logic_name =>
'fan',
84 -module =>
'Bio::EnsEMBL::Hive::RunnableDB::Dummy',
86 1 =>
'?accu_name=alphas&accu_input_variable=alpha&accu_address=[]',
90 { -logic_name =>
'funnel',
91 -module =>
'Bio::EnsEMBL::Hive::RunnableDB::SystemCmd',
93 'alpha_csv' =>
'#expr( join(",", @{#alphas#}) )expr#',
94 'cmd' =>
'echo "#alpha_csv#"',
98 'A->1' => {
'aggregator' => {}, },
102 { -logic_name =>
'aggregator',
103 -module =>
'Bio::EnsEMBL::Hive::RunnableDB::Dummy',