Announce: CafeBatch -- a batch file whose scope is network wide-- assessing interest

We wrote a generic perl system for provisioning our services and
thought others may have a use for it. We call it CafeBatch. Basically
it is an OO, network wide batch creation and execution system. When
run, one batch file on a computer can make an arbitrary number of
commands or scripts be run on arbitrary systems on the network and have
the results stored in itself -- A Batch file whose scope is the entire
network or even WWW. YAMl is used to write the objects to files and
SCP is used to send requests and results across the network. The system
consists of 3 main objects -- Batch, Request, and Command. A Batch has
many Requests that get sent to one or more servers . There a Request
essentially is mapped to a series of Commands which are executed. The
results are all sent back and stored in the Batch object which is
YAMLized to a file for evaluation of the results. Its a long
explanation but bear with me as it could be useful to you.

SYNOPSIS: How we use it to activate a dialup account.

Many servers need to take part in this so we have a "Request" named
'activate_dialup'. Which servers take part in this request is
specified in the delivery_map ahead of time.

localhost:/usr/local/CafeBatch/config/delivery_map.pl :

%Request::DELIVERY = (
'activate_dialup' => [ ['jlore', 'monk1', cluster1'] ],
);


In the billing database script that created the new dialup, we use
CafeBatch to provision it like so:

# Make a request
my $r = CafeBatch::Request->new(
{name => 'activate_dialup' , obj_data {username => 'cranky', pw
=> 'plane'}
);

# Put it in a batch
my $b = CafeBatch::Batch->new( { name => 'WhateverIwant', requests =>
[$r
]} );

# Deliver the batch to the run queue and forget about it.
$b->Deliver;

Thats it. The BatchQue daemon will pick it up and run it in a child
process. If we did not want to worry about queing we could have forked
and done $b->Run; instead of Deliver. Initial setup is some work but
once that is done everything is a breeze.

So what happens when the Batch is Run? The request is sent to each
server in the delivery map. There the RequestQue picks them up and
runs them. A RequestQue is pretty necessary here.

So what happens when a request is run? The server's pre made command
map is consulted to see what commands to execute. Each server has its
own command map because each one can do a little different job
regarding activating a dialup. For this we have

jlore:/usr/local/CafeBatch/config/command_map.pl --
%Request::COMMAND = (
'activate_dialup' => ['add_unix_user.pl' , '/usr/bin/blahh'],
)

monk1:/usr/local/CafeBatch/config/command_map.pl --
%Request::COMMAND = (
'activate_dialup' => ['add_mail_box.pl', 'add_radius.pl'],
)

cluster1:/usr/local/CafeBatch/config/command_map.pl --
%Request::COMMAND = (
'activate_dialup' => ['setup_spam_filter.pl'],
)

So a Request request acquires a list of Command objects when it is Run
and when those Commands are run, the results end up in the objects in
the Request Object which ends up back in the Batch object on the
originating server. There in the 'done' or 'error' directory is a
YAMLized Batch object known as a executed Batch file where you can see
the results of every command run on every server in the Batch
execution.

Thats the basics and there is [MORE].
There is still lots of room for improvement.
Is there anyone who may find this useful? Any suggestions on
Namespace, category, etc? In naming Cafe means a group of computers
and Batch is a batch file. Our company name is Computer Cafe so that
also influenced it in developement. We do not want to brand this and
feel Cafe works generally and has a nice ring too it and we cant think
of anything better so we kept it. I was thinking of Cafe::Batch
Cafe::Request, Cafe::Command rather than CafeBatch::* . Does anyone
like that better?

thanks,

Peter

[MORE]
You can avoid queues and maps and create and execute things yourself
too. Also an original Batch is saved before it is run in a separate
file in case it is something you want to run again. Setup is a little
complex but our actual is condensed into one or two lines mostly.
Requests can be run serially or all at the same time or both. The
array of array format of the delivery map specifies this. Each array
in the array is delivered to serially. In the example there as only one
group (array) of servers so they all got the request at the same time.
Had there been another group, they would have gotten the request only
after all of the first groups had run and returned successfully.

Also , all the different modules can be used independently of eachother
(unless there is a bug in there preventing this :) ). That is you do
not need a batch to run a request and get the results stored in a file.
You do not need a command map . You can add Commands to the request
before you run it. You do not need a Request to run a command and have
results stored.