How to bypass (pipe) curl_exec return value directly to a file?

Newbie question.
I need to download a very large amount of xml data from a site using CURL.

How to bypass (pipe) curl_exec return value directly to a file, without
using memory allocation?

set_time_limit(0);
$ch = curl_init($siteURL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$mixed = curl_exec($ch);

How to set/pipe $mixed as a (disk) file, so that data returned by curl_exec
is directly saved to the disk-file, and not involving memory allocation?

Thank you.

-PHP 5
-Windows XP


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: How to bypass (pipe) curl_exec return value directly to a

2009/10/12 m.hasibuan :
> Newbie question.
> I need to download a very large amount of xml data from a site using CURL.
>
> How to bypass (pipe) curl_exec return value directly to a file, without
> using memory allocation?
>
> set_time_limit(0);
> $ch = curl_init($siteURL);
> curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
> $mixed = curl_exec($ch);
>
> How to set/pipe $mixed as a (disk) file, so that data returned by curl_exec
> is directly saved to the disk-file, and not involving memory allocation?
>
> Thank you.

Use the CURLOPT_FILE option to set the output to write to the file
handle given by the option's value (which must be a writable file
handle). For instance:

$ch = curl_init($url);
$fp = fopen('/tmp/curl.out', 'w');
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_exec($ch);

Error checking etc. is of course left up to you. :)


Good luck,

Torben

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: How to bypass (pipe) curl_exec return value directly to afile?

--_fbff48bd-5ea9-46a9-a9b4-2b0166a25d7a_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable


I guess this should work

set_time_limit(0)=3B
$ch =3D curl_init($siteURL)=3B
curl_setopt($ch=2C CURLOPT_RETURNTRANSFER=2C false)=3B
curl_setopt($ch=2C CURLOPT_BINARYTRANSFER=2C true)=3B
curl_setopt($ch=2C CURLOPT_CONNECTTIMEOUT=2C 0)=3B
curl_setopt($ch=2C CURLOPT_FILE=2C 'stream.bin')=3B
curl_exec($ch)=3B

Regards

> From: magda.hasibuan@yahoo.co.uk
> To: php-general@lists.php.net
> Date: Tue=2C 13 Oct 2009 11:01:04 +0700
> Subject: [PHP] How to bypass (pipe) curl_exec return value directly to a =
file?
>=20
> Newbie question.
> I need to download a very large amount of xml data from a site using CURL=
..
>=20
> How to bypass (pipe) curl_exec return value directly to a file=2C without
> using memory allocation?
>=20
> set_time_limit(0)=3B
> $ch =3D curl_init($siteURL)=3B
> curl_setopt($ch=2C CURLOPT_RETURNTRANSFER=2C true)=3B
> $mixed =3D curl_exec($ch)=3B
>=20
> How to set/pipe $mixed as a (disk) file=2C so that data returned by curl_=
exec
> is directly saved to the disk-file=2C and not involving memory allocation=
?
>=20
> Thank you.
>=20
> -PHP 5
> -Windows XP
>=20
>=20
> --=20
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe=2C visit: http://www.php.net/unsub.php
>=20
=0A=
____________________________________________________________ _____=0A=
Windows Live: Make it easier for your friends to see what you=92re up to on=
Facebook.=0A=
http://www.microsoft.com/middleeast/windows/windowslive/see- it-in-action/so=
cial-network-basics.aspx?ocid=3DPID23461::T:WLMTAGL:ON:WL:en -xm:SI_SB_2:092=
009=

--_fbff48bd-5ea9-46a9-a9b4-2b0166a25d7a_--

RE: How to bypass (pipe) curl_exec return value directly to a file?

--_b71ef2dd-d863-4e98-8f15-f5d8c85e4968_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable



> $ch =3D curl_init($url)=3B
> $fp =3D fopen('/tmp/curl.out'=2C 'w')=3B
> curl_setopt($ch=2C CURLOPT_FILE=2C $fp)=3B
> curl_exec($ch)=3B
>=20
> Error checking etc. is of course left up to you. :)

oops=2C I sent directly the file name. Let me reformulate the code then:


set_time_limit(0)=3B
$fp =3D fopen('stream.bin'=2C 'wb')=3B
$ch =3D curl_init($siteURL)=3B
curl_setopt($ch=2C CURLOPT_RETURNTRANSFER=2C false)=3B
curl_setopt($ch=2C CURLOPT_BINARYTRANSFER=2C true)=3B
curl_setopt($ch=2C CURLOPT_CONNECTTIMEOUT=2C 0)=3B
curl_setopt($ch=2C CURLOPT_FILE=2C $fp)=3B
curl_exec($ch)=3B
fclose($fp)=3B

Apologize I did not test it before.

Regards

=0A=
____________________________________________________________ _____=0A=
Windows Live: Friends get your Flickr=2C Yelp=2C and Digg updates when they=
e-mail you.=0A=
http://www.microsoft.com/middleeast/windows/windowslive/see- it-in-action/so=
cial-network-basics.aspx?ocid=3DPID23461::T:WLMTAGL:ON:WL:en -xm:SI_SB_3:092=
010=

--_b71ef2dd-d863-4e98-8f15-f5d8c85e4968_--

Re: How to bypass (pipe) curl_exec return value directly to a

2009/10/13 Andrea Giammarchi :
>
>> $ch = curl_init($url);
>> $fp = fopen('/tmp/curl.out', 'w');
>> curl_setopt($ch, CURLOPT_FILE, $fp);
>> curl_exec($ch);
>>
>> Error checking etc. is of course left up to you. :)
>
> oops, I sent directly the file name. Let me reformulate the code then:
>
>
> set_time_limit(0);
> $fp = fopen('stream.bin', 'wb');
> $ch = curl_init($siteURL);
> curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
> curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);

If you're re-using a curl handle then it may be a good idea to set
these explicitly. However, these are also the default values so it's
not really necessary to set them for a new handle. You're right that
it's a good idea to include the 'b' in the fopen() mode.

> curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 0);

I wouldn't recommend setting this to 0 unless you're very sure that
the connection will succeed; otherwise, your script will hang
indefinitely waiting for the connection to be made.


Regards,

Torben

> curl_setopt($ch, CURLOPT_FILE, $fp);
> curl_exec($ch);
> fclose($fp);
>
> Apologize I did not test it before.
>
> Regards
>
>
> ________________________________
> Windows Live: Friends get your Flickr, Yelp, and Digg updates when they
> e-mail you.

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: How to bypass (pipe) curl_exec return value directly to a file?

--_7b385bfa-c97b-4727-83e7-7ae5062451a4_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable



> > curl_setopt($ch=2C CURLOPT_CONNECTTIMEOUT=2C 0)=3B
>=20
> I wouldn't recommend setting this to 0 unless you're very sure that
> the connection will succeed=3B otherwise=2C your script will hang
> indefinitely waiting for the connection to be made.

agreed=2C it's just he set timeout to zero so I guess he meant the curl con=
nection as well otherwise it does not make sense to set the timeout to 0 if=
curl has 10 seconds timeout :-)

Regards
=0A=
____________________________________________________________ _____=0A=
Windows Live Hotmail: Your friends can get your Facebook updates=2C right f=
rom Hotmail=AE.=0A=
http://www.microsoft.com/middleeast/windows/windowslive/see- it-in-action/so=
cial-network-basics.aspx?ocid=3DPID23461::T:WLMTAGL:ON:WL:en -xm:SI_SB_4:092=
009=

--_7b385bfa-c97b-4727-83e7-7ae5062451a4_--

Re: How to bypass (pipe) curl_exec return value directly to a

2009/10/13 Andrea Giammarchi :
>
>> > curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 0);
>>
>> I wouldn't recommend setting this to 0 unless you're very sure that
>> the connection will succeed; otherwise, your script will hang
>> indefinitely waiting for the connection to be made.
>
> agreed, it's just he set timeout to zero so I guess he meant the curl
> connection as well otherwise it does not make sense to set the timeout to 0
> if curl has 10 seconds timeout :-)
>
> Regards

If he wants to download a very large file then it would make sense to
set_time_limit(0) but leave the curl connect timeout enabled; he
wouldn't want the PHP script timing out partway through a large
download. :) The curl timeout isn't for the transfer; just for making
the connection.


Regards,

Torben

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

RE: How to bypass (pipe) curl_exec return value directly to a file?

--_6ff9267a-9f76-40b3-92f7-6be46492c0f1_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable


uhm=2C right=2C I should have better explain that option ... still=2C if ti=
melimit is 0=2C I guess connection timeout matters=2C maybe I am wrong.

Thanks.

Regards

> Date: Tue=2C 13 Oct 2009 11:31:00 -0700
> Subject: Re: [PHP] How to bypass (pipe) curl_exec return value directly t=
o a file?
> From: larstorben@gmail.com
> To: an_red@hotmail.com
> CC: magda.hasibuan@yahoo.co.uk=3B php-general@lists.php.net
>=20
> 2009/10/13 Andrea Giammarchi :
> >
> >> > curl_setopt($ch=2C CURLOPT_CONNECTTIMEOUT=2C 0)=3B
> >>
> >> I wouldn't recommend setting this to 0 unless you're very sure that
> >> the connection will succeed=3B otherwise=2C your script will hang
> >> indefinitely waiting for the connection to be made.
> >
> > agreed=2C it's just he set timeout to zero so I guess he meant the curl
> > connection as well otherwise it does not make sense to set the timeout =
to 0
> > if curl has 10 seconds timeout :-)
> >
> > Regards
>=20
> If he wants to download a very large file then it would make sense to
> set_time_limit(0) but leave the curl connect timeout enabled=3B he
> wouldn't want the PHP script timing out partway through a large
> download. :) The curl timeout isn't for the transfer=3B just for making
> the connection.
>=20
>=20
> Regards=2C
>=20
> Torben
=0A=
____________________________________________________________ _____=0A=
Windows Live: Friends get your Flickr=2C Yelp=2C and Digg updates when they=
e-mail you.=0A=
http://www.microsoft.com/middleeast/windows/windowslive/see- it-in-action/so=
cial-network-basics.aspx?ocid=3DPID23461::T:WLMTAGL:ON:WL:en -xm:SI_SB_3:092=
010=

--_6ff9267a-9f76-40b3-92f7-6be46492c0f1_--