distributing software built on mod

distributing software built on mod_perl

am 10.09.2009 15:55:04 von Mike Barborak

Hello,

We have built some software that depends on mod_perl and are working
on how to distribute it to customers and have them install it. It
seems the task we're asking of our customer is not for the faint of
heart as it involves a lot of command line operations, an
understanding of their web server installation, and the ability to
install mod_perl and some number of Perl modules on which our software
depends. Standing at the foot of this documentation and installation
script mountain, we're looking for trails blazed by those who have
come before.

So, can anyone point me at other software with these requirements and
their solution to this problem? Particularly ones that have handled
theses issues well?

Alternately, do you think our approach is fundamentally unsound for
our intentions? Is there a better way to write and distribute Apache
modules?

Thanks,
Mike

Mike Barborak
Technical Director
barborak@basikgroup.com

------------------------------------

BASIK GROUP
1201 Broadway, #704
New York, NY 10001

office: 646 201 9347
cell: 646 263 7029
www.basikgroup.com

Re: distributing software built on mod_perl

am 10.09.2009 16:42:28 von ELINTPimp

--001485f20b849c225304733a3684
Content-Type: text/plain; charset=ISO-8859-1

Hi Mike,

The information you provided indicates that you have determined your
customer base for this application might not always have a dedicated IT
staff to conduct these installs. Your company has a couple options in
achieving this "last mile", which can be observed across the industry,
mod_perl or otherwise. These suggestions are at a very "high level" because
I really don't know anything about your product, your business model, and
what your business outlook is for the product or as a whole. Unfortunalty,
this question has a mix of technical and business implications since it
deals with not only this project but potentially the software configuration
management of your business.

Some generic options:

- One option is that your company can do the install for the customer as
part of the purchase price. This may include consulting fees but would also
give you the ability to upsale any additional modules or features that
particular customer may need. This is something you'll find from companies
like BMC and several other enterprise-level software providers.
- Another option is to create a "certified consultant" program where
external companies can learn to install and maintain your system, you
certify them, and provide a yellow-book type recommendation to those
certified consultants. This model is similar to what you may find from
PayPal and many open source projects that have taken off.
- You could use a packaging and installation utility. While this is
usually the most attractive option, both astetically for the customer as
well as the outward apperance of professionalism in your product...it is
also the most costly both upfront and long term. These build utilities are
typically platform dependent and you must be concerned about different
adapter layers (ie the choice of web server...apache/iis/etc). This is a
great option, and there are MANY utilities spanning from make, RPM, CSW to
Install Anywhere that may suite your needs. However, you should be sure you
know what you are getting into, and have a good understanding why you are,
before doing so.

I was suprised to not find many books on this specific subject - but that
may be a result on my search parameters. If your development team hasn't
done this before, and you do not have a business strategy to support product
delivery, you might look into hiring a consultant in your area to analyze
your situation and recommend some courses of action.

Regards,

Steve

On Thu, Sep 10, 2009 at 9:55 AM, Mike Barborak wrote:

> Hello,
>
> We have built some software that depends on mod_perl and are working
> on how to distribute it to customers and have them install it. It
> seems the task we're asking of our customer is not for the faint of
> heart as it involves a lot of command line operations, an
> understanding of their web server installation, and the ability to
> install mod_perl and some number of Perl modules on which our software
> depends. Standing at the foot of this documentation and installation
> script mountain, we're looking for trails blazed by those who have
> come before.
>
> So, can anyone point me at other software with these requirements and
> their solution to this problem? Particularly ones that have handled
> theses issues well?
>
> Alternately, do you think our approach is fundamentally unsound for
> our intentions? Is there a better way to write and distribute Apache
> modules?
>
> Thanks,
> Mike
>
>
> Mike Barborak
> Technical Director
> barborak@basikgroup.com
>
> ------------------------------------
>
> BASIK GROUP
> 1201 Broadway, #704
> New York, NY 10001
>
> office: 646 201 9347
> cell: 646 263 7029
> www.basikgroup.com
>

--001485f20b849c225304733a3684
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Mike,

The information you provided indicates that you have determ=
ined your customer base for this application might not always have a dedica=
ted IT staff to conduct these installs.=A0 Your company has a couple option=
s in achieving this "last mile", which can be observed across the=
industry, mod_perl or otherwise.=A0 These suggestions are at a very "=
high level" because I really don't know anything about your produc=
t, your business model, and what your business outlook is for the product o=
r as a whole.=A0 Unfortunalty, this question has a mix of technical and bus=
iness implications since it deals with not only this project but potentiall=
y the software configuration management of your business.

Some generic options:

=A0- One option is that your company can d=
o the install for the customer as part of the purchase price.=A0 This may i=
nclude consulting fees but would also give you the ability to upsale any ad=
ditional modules or features that particular customer may need.=A0 This is =
something you'll find from companies like BMC and several other enterpr=
ise-level software providers.

=A0- Another option is to create a "certified consultant" program=
where external companies can learn to install and maintain your system, yo=
u certify them, and provide a yellow-book type recommendation to those cert=
ified consultants.=A0 This model is similar to what you may find from PayPa=
l and many open source projects that have taken off.

=A0- You could use a packaging and installation utility.=A0 While this is u=
sually the most attractive option, both astetically for the customer as wel=
l as the outward apperance of professionalism in your product...it is also =
the most costly both upfront and long term.=A0 These build utilities are ty=
pically platform dependent and you must be concerned about different adapte=
r layers (ie the choice of web server...apache/iis/etc).=A0 This is a great=
option, and there are MANY utilities spanning from make, RPM, CSW to Insta=
ll Anywhere that may suite your needs.=A0 However, you should be sure you k=
now what you are getting into, and have a good understanding why you are, b=
efore doing so.

I was suprised to not find many books on this specific subject - but th=
at may be a result on my search parameters.=A0 If your development team has=
n't done this before, and you do not have a business strategy to suppor=
t product delivery, you might look into hiring a consultant in your area to=
analyze your situation and recommend some courses of action.

Regards,

Steve

On Thu, Sep 10,=
2009 at 9:55 AM, Mike Barborak < borak@basikgroup.com">barborak@basikgroup.com> wrote:
ckquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, 204,=
204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">

Hello,

We have built some software that depends on mod_perl and are working

on how to distribute it to customers and have them install it. It

seems the task we're asking of our customer is not for the faint of

heart as it involves a lot of command line operations, an

understanding of their web server installation, and the ability to

install mod_perl and some number of Perl modules on which our software

depends. Standing at the foot of this documentation and installation

script mountain, we're looking for trails blazed by those who have

come before.

So, can anyone point me at other software with these requirements and

their solution to this problem? Particularly ones that have handled

theses issues well?

Alternately, do you think our approach is fundamentally unsound for

our intentions? Is there a better way to write and distribute Apache

modules?

Thanks,

Mike

Mike Barborak

Technical Director

------------------------------------

BASIK GROUP

1201 Broadway, #704

New York, NY 10001

office: 646 201 9347

cell: 646 263 7029

www.basikgroup.com<=
/a>

--001485f20b849c225304733a3684--

Re: distributing software built on mod_perl

am 10.09.2009 16:46:27 von John D Groenveld

In message , Mike
Barborak writes:
>So, can anyone point me at other software with these requirements and
>their solution to this problem? Particularly ones that have handled
>theses issues well?

Best Practical's Request Tracker might be a useful case study.

It makes use of mod_perl and Mason and a load of other CPAN
dependencies.

Installation from source is pretty straight-forward, though
I provide it its own Perl and Apache.

Happy hacking,
John
groenveld@acm.or

Re: distributing software built on mod_perl

am 10.09.2009 16:50:00 von aw

Steven Siebert wrote:
....
Great answer, Steve.
I have the same issue as the OP, and unfortunately, as you point out,
there is no free lunch once the application is other than trivial.
At least perl has the advantage to run "as is" on almost any platform.

Mike, if you ever find a good solution, I am interested also.
Good luck.

André

RE: distributing software built on mod_perl

am 10.09.2009 16:56:54 von morten.bjornsvik

Hi

We ship commercial software based on mod_perl2 and mason.

We build the entire stack from perl,database,apache,openssl + own =
software
under /opt to separate it completely from RedHat Enterprise/SLES =
patching breaking
the system.

Currently we have stable rpm spec files for base perl and apache + ssl

Then we have a script that downloads and creates a cpan repository based =
on a config
file to get the correct versions. Those modules we store locally and use
as a distribution repository.

Then we have an install script that install all those cpan modules based =
on
/opt/perl/bin/perl ./Makefile.PL + options in correct order and solves =
all dependencies.
It also holds path to MQ,DB2,Oracle etc software if needed.

When you deal with cpan you must keep a local repository, because cpan =
moves,
and suddenly a developer releases a new version, and deletes the old one =
because
it has an error. We also have some modules that we have bugfixed =
ourselves.

We can then choose to create a rpm of entire perl + libraries for =
apache, mod_perl
etc, but mostly this is only needed when customers do not have compilers =
and dev libraries
installed.

Then we set up our own software and configure the system.

This works fairly ok, I use around 30 minutes to set up a rather complex =
lamp-stack.
(if there are no other issues). We also support 386 and x64 seemlessly =
and can
easily test all varities of perl like 5.8.8, 5.8.9 5.10.0 and 5.10.1 and =
apache 2.0 and
2.2 to see what is most stable and fastest.

We also create a group, use umask 0002, chmod g+rws(dir) and =
g+rwx(files)
on all our software, so we do not need to be root to restart anything on =
the stack.
We have our own perl based /etc/init.d/startup scripts.

We have had lots of issues in the past when we used redhats lamp-stack.
Sysadmins tend to update every time redhat issues a critical patch.

--
Morten Bjoernsvik, Experian Decision Analytics, Oslo Norway

-----Original Message-----
From: mab2001@gmail.com [mailto:mab2001@gmail.com] On Behalf Of Mike =
Barborak
Sent: 10. september 2009 15:55
To: modperl@perl.apache.org
Subject: distributing software built on mod_perl

Hello,

We have built some software that depends on mod_perl and are working
on how to distribute it to customers and have them install it. It
seems the task we're asking of our customer is not for the faint of
heart as it involves a lot of command line operations, an
understanding of their web server installation, and the ability to
install mod_perl and some number of Perl modules on which our software
depends. Standing at the foot of this documentation and installation
script mountain, we're looking for trails blazed by those who have
come before.

So, can anyone point me at other software with these requirements and
their solution to this problem? Particularly ones that have handled
theses issues well?

Alternately, do you think our approach is fundamentally unsound for
our intentions? Is there a better way to write and distribute Apache
modules?

Thanks,
Mike

Mike Barborak
Technical Director
barborak@basikgroup.com

------------------------------------

BASIK GROUP
1201 Broadway, #704
New York, NY 10001

office: 646 201 9347
cell: 646 263 7029
www.basikgroup.com

Re: distributing software built on mod_perl

am 10.09.2009 17:10:50 von aw

Morten Bjørnsvik wrote:
....

>
> Then we have a script that downloads and creates a cpan repository based on a config
> file to get the correct versions. Those modules we store locally and use
> as a distribution repository.
>
> Then we have an install script that install all those cpan modules based on
> /opt/perl/bin/perl ./Makefile.PL + options in correct order and solves all dependencies.
> It also holds path to MQ,DB2,Oracle etc software if needed.
>
Morten, would you be willing to share the above ?
That would really help us a lot.

I agree that you have to set up your own local repository. One other
reason is that many of our customer systems do not have direct access to
the internet.
Currently, I do solve that by just having a copy of the modules
(downloaded beforehand from CPAN) in a local directory, and using a
script that does the chdir, make, make install, one after the other.
But it would be more elegant to have a local CPAN repository, as you do.
I just don't know how to do that.

André

Re: distributing software built on mod_perl

am 10.09.2009 17:37:57 von Perrin Harkins

On Thu, Sep 10, 2009 at 9:55 AM, Mike Barborak wrote:
> So, can anyone point me at other software with these requirements and
> their solution to this problem?

You can look at the installer for Krang (http://krangcms.com) which
builds the whole stack. There are also several approaches on CPAN,
like Shipwright.

- Perrin

Re: distributing software built on mod_perl

am 10.09.2009 19:08:52 von Mike Barborak

Thanks for the thoughts. I think as Steven said, it's hard to give
specific recommendations without understanding the nature of the
software being distributed. I didn't mention it before because I
didn't want to bore anyone but I'll take that chance now.

So the web server module that I'm wanting to distribute is secondary
to the primary software. The primary software is at frontalcode.com
(still in somewhat pre-launch mode) and is something we've written
that can be thought of as HTML and Javascript for Flash. The idea is
to use a markup language to develop rich media sites. One of the
bonuses of using Frontal is that if you have an HTML-like description
of your rich media app then it's possible to transform that into an
actual HTML version that can be indexed by Google and the other search
engines - a general problem with Flash content.

This then is where this secondary software comes in. You plug this
module (the SEO Module) into your web server and it looks at each
requests it gets, finds the files that will satisfy it, searches them
for Frontal content, converts that content to HTML and inserts that
into the result.

So, given that product offering, the thought is that we would be
plugging into someone's existing infrastructure. That is, our
customers would likely already have a website/cms/etc. that they would
be loathe to part from but that they might be convinced to add a layer
to. That means that building our own stack wouldn't really work for us
- imagine that a page is being served by Krang for example and the
result has Frontal content in it.

Next, I think the price point we're looking at for this software
($500) prevents us from being able to spend too much time with each
customer getting them set up. In other words, doing an install for
each sale seems contrary to the business model especially since I want
my developers focused on Flash development rather than Apache
administration.

I think that last point, that this is secondary software and that we
don't want to be Perl experts makes building RPMs and installable Perl
bundles kind of unattractive.

I think then that the most attractive option is to write an
installation manual that says, "install mod_perl, install these
modules, put software here..." and then leaves it to the purchaser to
accomplish those steps. If they have in-house IT (which is probably
not far-fetched for the scenarios I'm imagining) then this isn't
unreasonable. And then if they don't have in-house IT, we refer them
to a pool of mod_perl and Apache savvy consultants who could follow
those sketchy steps.

So, is there a pool of mod_perl and Apache savvy consultants out there
available for tapping? Google gives results but if anyone wants to
recommend a group then I'd be happy to hear from them. (BTW, you can't
even buy the SEO module at the moment so this is about preparations.)

Thanks again,
Mike

Mike Barborak
Technical Director
barborak@basikgroup.com

------------------------------------

BASIK GROUP
1201 Broadway, #704
New York, NY 10001

office: 646 201 9347
cell: 646 263 7029
www.basikgroup.com

On Thu, Sep 10, 2009 at 11:37 AM, Perrin Harkins wrote=
:
> On Thu, Sep 10, 2009 at 9:55 AM, Mike Barborak =
wrote:
>> So, can anyone point me at other software with these requirements and
>> their solution to this problem?
>
> You can look at the installer for Krang (http://krangcms.com) which
> builds the whole stack. =A0There are also several approaches on CPAN,
> like Shipwright.
>
> - Perrin
>

Re: distributing software built on mod_perl

am 11.09.2009 18:49:02 von William T

Whenever I creating shrink-wrapped software I always make packaging
and distribution part of the development, qa and testing process. All
packages for the platforms that we will be supporting. The reason I
do this is to cut down on the customer support overhead. I've found
you get less calls and emails regarding installation. And more
importantly when it comes to supporting the customer they all have the
same installation per platform which often times makes things much
easier to debug (or at least removes alot of potential sources for
problems).

In practice what this means that we have an additional make target,
package which creates the appropriate package for the platform it's
running on. I also have automated unit test which pull apart the
installation verifying meta data and install bits, as well as running
the installation in a sandbox to verify installation occurs without
issue. It can be alot of upfront time to set this up, but very little
maintenance is required once it's up and running.

YMMV

-wjt

Re: distributing software built on mod_perl

am 25.09.2009 16:32:45 von Mike Barborak

I think I see what you're saying. It seems like a very expensive
problem to solve. It must be a barrier to people choosing mod_perl to
develop their apps because it might be the case that their
distribution and installation process is more complex to develop and
support than their actual applications. That's what it's looking like
in my situation and so I'm going to reevaluate using mod_perl.

But it seems like there's a lot of generally repurposable results in
your efforts that could make distributing mod_perl applications more
straightforward. It seems like at the core of what you do is a massive
distributable that contains:

* mod_perl binaries
* perl binaries
* necessary perl module binaries
* project-specific binaries

This massive distributable contains these binaries built for all the
supported platforms and all the supported versions of Apache. So if we
say that there are on the order of 10 platforms that need to be
supported to cover 95% of all installations and there are 5 versions
of Apache to support (just guessing with both numbers), then it would
be a matter of building:

* 50 versions of the mod_perl binaries
* 10 versions of the perl binaries (multiplatform installations
supported by perl already)
* 10 versions of the necessary perl module binaries (multiplatform
installations supported by perl already)
* 10 versions of the project-specific binaries

This would take a lot of effort to setup but doesn't seem unreasonable
to maintain. And with this binary repository in hand then it seems the
distribution problem really does become fairly straightforward; it's
just a matter of figuring out the target platform and apache version
and then producing an appropriate apache configuration snippet. That
seems a lot easier than trying to walk a client through the process of
getting the development version of apache, getting the necessary build
tools, building mod_perl, building perl modules, etc. It also seems
like it would make less of an impact on a client's existent system
that replacing their web server.

So is it a pipe dream to think that such a repository of
non-project-specific binaries could be made publicly available to
mod_perl application developers? Some of the problems I see are:

* Have I captured all the variants or would you also have to build
version for all the various versions of libc, mysql, etc.?
* For customized perl modules and your own application, what would the
resource be to build it on all the supported platforms? Perhaps this
is where someone could monetize this scheme.

I'm not expecting such a thing would be available any time soon and so
wouldn't affect my decisions but I just wanted to put this out there
as food for thought. It seems like my situation isn't that odd and so
is an interesting case study arguing that mod_perl isn't being as
broadly used as it might be.

Mike

On Fri, Sep 11, 2009 at 12:49 PM, William T wrote:
> Whenever I creating shrink-wrapped software I always make packaging
> and distribution part of the development, qa and testing process. =A0All
> packages for the platforms that we will be supporting. =A0The reason I
> do this is to cut down on the customer support overhead. =A0I've found
> you get less calls and emails regarding installation. =A0And more
> importantly when it comes to supporting the customer they all have the
> same installation per platform which often times makes things much
> easier to debug (or at least removes alot of potential sources for
> problems).
>
> In practice what this means that we have an additional make target,
> package which creates the appropriate package for the platform it's
> running on. =A0I also have automated unit test which pull apart the
> installation verifying meta data and install bits, as well as running
> the installation in a sandbox to verify installation occurs without
> issue. =A0It can be alot of upfront time to set this up, but very little
> maintenance is required once it's up and running.
>
> YMMV
>
> -wjt
>

Re: distributing software built on mod_perl

am 25.09.2009 20:34:24 von Fred Moyer

Re: distributing software built on mod_perl

am 26.09.2009 09:41:42 von William T

On Fri, Sep 25, 2009 at 7:32 AM, Mike Barborak wrote:
> I think I see what you're saying. It seems like a very expensive
> problem to solve. It must be a barrier to people choosing mod_perl to
> develop their apps because it might be the case that their
> distribution and installation process is more complex to develop and
> support than their actual applications. That's what it's looking like
> in my situation and so I'm going to reevaluate using mod_perl.

It's not limited to mod_perl. It's limited to any shrink wrapped
software. It's less of an issue with software that can be self
contained versus software that is shared on the system.

> But it seems like there's a lot of generally repurposable results in
> your efforts that could make distributing mod_perl applications more
> straightforward. It seems like at the core of what you do is a massive
> distributable that contains:
>
> * mod_perl binaries
> * perl binaries
> * necessary perl module binaries
> * project-specific binaries

I would split up the distributable by "platform" since that is the
granularity you care about.

I usually rely on package that are already provided by various
distributions. My target platforms are usually RedHat, Debian, and
FreeBSD which all have stable mod_perl+apache packages already, as
well as a significant number of perl packages available. I then
backfill any packages which are not available for that distro so
really all I have to maintain is the difference between the packages I
use, and what the distributions already provide.

> This massive distributable contains these binaries built for all the
> supported platforms and all the supported versions of Apache. So if we
> say that there are on the order of 10 platforms that need to be
> supported to cover 95% of all installations and there are 5 versions
> of Apache to support (just guessing with both numbers), then it would
> be a matter of building:
>
> * 50 versions of the mod_perl binaries
> * 10 versions of the perl binaries (multiplatform installations
> supported by perl already)
> * 10 versions of the necessary perl module binaries (multiplatform
> installations supported by perl already)
> * 10 versions of the project-specific binaries

10 platforms seems a bit excessive, but then again it depends what
your counting as a platform. ie. RedHat/Debian, RHEL 4.2/4.3, or
RHEL 4/5. If your platform is a distribution then your dependency
chain is different for each "platform" . If your platform is minor
revision of a particular distribution, then it may not matter so much,
it's probably the same dependency chain. If they are different major
versions, you can go either way depending on if it's common for both
major versions don't have too many differences in the dependency
chain.

In all cases though you want to leverage work already done by the
various distributions as well as third party package repositories to
reduce your own overhead.

> This would take a lot of effort to setup but doesn't seem unreasonable
> to maintain. And with this binary repository in hand then it seems the
> distribution problem really does become fairly straightforward; it's
> just a matter of figuring out the target platform and apache version
> and then producing an appropriate apache configuration snippet. That
> seems a lot easier than trying to walk a client through the process of
> getting the development version of apache, getting the necessary build
> tools, building mod_perl, building perl modules, etc. It also seems
> like it would make less of an impact on a client's existent system
> that replacing their web server.

It's a tradeoff of up front work to figure out which platforms your
going to support, and what that means in terms of your dependency
chain, versus an unknown dependency chain and trying to mitigate all
the variance you may encounter in your support calls.

That isn't to say you can't take a middle ground as well so long as
the risks are understood. It may in fact be cost prohibitive to do
the "right" thing" and track entire dep chains especially if your
trying to support a large number of platforms. So the question
becomes what is "good enough". How much can you maximize number of
supported platforms while minimizing support risk?

> So is it a pipe dream to think that such a repository of
> non-project-specific binaries could be made publicly available to
> mod_perl application developers? Some of the problems I see are:
>
> * Have I captured all the variants or would you also have to build
> version for all the various versions of libc, mysql, etc.?
> * For customized perl modules and your own application, what would the
> resource be to build it on all the supported platforms? Perhaps this
> is where someone could monetize this scheme.

There is is the idea of following your dependency chain until it is
"good enough" versus following it all the way to libc versions and
kernel versions. My rule of thumb is to only follow language/library
which is what I would consider prime dependencies. So I track what
versions of perl libraries I may use, or at least snapshot the package
repository, and checkin my custom packages. Then periodically I sync
up with the distribution packages to update package versions. For
each platform you want to support you create the package repository
snapshot.

-wjt