Problem relaying uploads

Dear all,

I have painted myself into a fairly bizarre corner for a fairly
bizarre client. I would be grateful if someone could please help shed
some light on what is going wrong.

Here's the situation in brief:

A. A company has a number of office machines (OMs) that can access the
company's intranet through a browser.

B. Their intranet web server (IWS) provides a web interface containing
a CGI upload form for uploading files from the OMs to a remote web
server (RWS) for processing, the output of which is to be sent back to
the IWS.

C. The OMs are not permitted to access the RWS directly.

Suffice to say that [I think] any upload from an OM has to be done in
two steps. The first is to upload the file to the IWS. The second
step is to upload the file from the IWS to the RWS. ie:

OM -> IWS -> RWS

The first step is working fine using the CGI upload command.. That
is, a user can upload the file from their office machine to a
directory on the IWS.

The second step is causing me grief. I was very much hoping that this
could simply be done by passing the upload formdata on the IWS into a
curl command for the RWS, but the file won't upload and I'm getting no
errors in the log.

I suppose my question is, is my strategy workable, or is there known
problems with CGI when uploading from one web server to another (as
opposed to uploading from one filesystem to a web-server (as in the
first step))?

Is it possible that I'm breaking the 'upload' filehandle/filename
structure by simply passing it in as formdata to curl?

I've already thrown many hours at this problem and I would like to
know if I should continue throwing more.

Thank you for any suggestions/advice.

Re: Problem relaying uploads

Here is some sample code to help clarify things. In this example,
each line of the file uploaded to the IWS is reversed by the RWS

IWS has something like the following:

upload.html:

Re: Problem relaying uploads

On Jan 23, 9:34 am, paktsardi...@gmail.com wrote:
> Here is some sample code to help clarify things. In this example,
> each line of the file uploaded to the IWS is reversed by the RWS
>
> IWS has something like the following:
>
> upload.html:
>
>
>
>
> upload.pl:
> my $filename=$q->param('input_file')
> my $fh = $q->upload('input_file');
> my $tmpfile="test.txt";
> open UPLOADFILE, ">$tmp_file" or die "Cannot open $!ile\n";
> while (<$fh>) {
> print UPLOADFILE;}
>
> close UPLOADFILE;
>
> # this works to this point.. then it tries the curl stuff
> $cmd_curl= "/usr/bin/curl -d \"file_upload=" . $q->param('input_file') ."\"http://RWS.domain.com/process.pl
>
> print `$cmd_curl`
>
> Then, on RWS, there's something like:
> process.pl:
> my $filename=$q->param('input_file')
> my $fh = $q->upload('input_file');
>
> while (<$fh>) {
> print reverse($_); # reverse the line, for example..
>
> }
>
> Hopefully this clarifies things a little more.


Your curl command is totally wrong. It needs to be
/usr/bin/curl -d input_file=test.txt

What you are actually sending is:
/usr/bin/curl -d file_upload=some_random_filename

Re: Problem relaying uploads

paktsardines@gmail.com wrote:
> Here is some sample code to help clarify things. In this example,
> each line of the file uploaded to the IWS is reversed by the RWS
>
> IWS has something like the following:
>
> upload.html:
>
>
>
>
> upload.pl:
> my $filename=$q->param('input_file')
> my $fh = $q->upload('input_file');
> my $tmpfile="test.txt";
> open UPLOADFILE, ">$tmp_file" or die "Cannot open $!ile\n";

Which is it, $tmpfile or $tmp_file? And what the heck is $!ile ?

> while (<$fh>) {
> print UPLOADFILE;
> }
> close UPLOADFILE;
>
> # this works to this point.. then it tries the curl stuff
> $cmd_curl= "/usr/bin/curl -d \"file_upload=" . $q-
> >param('input_file') ."\" http://RWS.domain.com/process.pl
> print `$cmd_curl`

No where here do you give curl $tmp_file or $tmpfile, so how is
curl supposed to find the file to upload it? Plus, that doesn't seem
to be how you do file uploads with curl, my reading of the man page
suggests you would use -F, not -d. (But that isn't a Perl issue).

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

Re: Problem relaying uploads

On Jan 23, 10:01 am, smallpond wrote:
> On Jan 23, 9:34 am, paktsardi...@gmail.com wrote:
>
>
>
> > Here is some sample code to help clarify things. In this example,
> > each line of the file uploaded to the IWS is reversed by the RWS
>
> > IWS has something like the following:
>
> > upload.html:
> >
> >
> >
>
> > upload.pl:
> > my $filename=$q->param('input_file')
> > my $fh = $q->upload('input_file');
> > my $tmpfile="test.txt";
> > open UPLOADFILE, ">$tmp_file" or die "Cannot open $!ile\n";
> > while (<$fh>) {
> > print UPLOADFILE;}
>
> > close UPLOADFILE;
>
> > # this works to this point.. then it tries the curl stuff
> > $cmd_curl= "/usr/bin/curl -d \"file_upload=" . $q->param('input_file') ."\"http://RWS.domain.com/process.pl
>
> > print `$cmd_curl`
>
> > Then, on RWS, there's something like:
> > process.pl:
> > my $filename=$q->param('input_file')
> > my $fh = $q->upload('input_file');
>
> > while (<$fh>) {
> > print reverse($_); # reverse the line, for example..
>
> > }
>
> > Hopefully this clarifies things a little more.
>
> Your curl command is totally wrong. It needs to be
> /usr/bin/curl -d input_file=test.txt
>
> What you are actually sending is:
> /usr/bin/curl -d file_upload=some_random_filename


Doh. What am I thinking. This will never work. curl
is sending form fields, not the file. When a form has
input type="file" name="somename" size="chars"
the client browser is packing up the file and sending
it along in a multipart request. curl has no clue how to
do that. You could write the whole thing in perl
and issue the request to the server. The basics of file
input type are here:

http://www.cs.tut.fi/~jkorpela/forms/file.html

It would require you to create the whole multipart form request
with the file properly MIME-encoded, then connect to the web
server and ship up the whole thing.
--S

Re: Problem relaying uploads

On Wed, 23 Jan 2008 05:54:11 -0800 (PST),
paktsardines@gmail.com wrote:

> A. A company has a number of office machines (OMs) that can access the
> company's intranet through a browser.
>
> B. Their intranet web server (IWS) provides a web interface containing
> a CGI upload form for uploading files from the OMs to a remote web
> server (RWS) for processing, the output of which is to be sent back to
> the IWS.
>
> C. The OMs are not permitted to access the RWS directly.

Can you use a HTTP proxy?

Martien
--
|
Martien Verbruggen | If it isn't broken, it doesn't have enough
| features yet.
|

Re: Problem relaying uploads

Quoth smallpond :
>
> Doh. What am I thinking. This will never work. curl
> is sending form fields, not the file. When a form has
> input type="file" name="somename" size="chars"
> the client browser is packing up the file and sending
> it along in a multipart request. curl has no clue how to
> do that. You could write the whole thing in perl
> and issue the request to the server. The basics of file
> input type are here:
>
> http://www.cs.tut.fi/~jkorpela/forms/file.html
>
> It would require you to create the whole multipart form request
> with the file properly MIME-encoded, then connect to the web
> server and ship up the whole thing.

LWP will handle this trivially.

Ben

Re: Problem relaying uploads

Apologies for:

1. not replying sooner
2. the numerous typos in my sample code, which was typed directly into
the post, rather of running perl across it first.


But, for those who were wondering (and you have my sympathies), the
problem was in the call to curl.. Changing '-d' to '-H' for cur's
form parameters got things working.

It's all so bloody obvious in hindsight.

Thanks for your help,

pakt.