Curl output

------=_Part_4709_4832042.1254440234883
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit


Hi, I am doing a basical curl call and can get the webpage I want. However
when it prints to the screen, it only prints the text, not css or any
javascript calls that run on page load.
Is there a way to make it do that?

thanks
using a basic curl call
$curl_handle=curl_init();
curl_setopt($curl_handle,CURLOPT_URL,'http://example.com');
curl_setopt($ch, CURLOPT_RETURNTRANSFER,0); // return into a variable
curl_exec($curl_handle);





--
View this message in context: http://www.nabble.com/Curl-output-tp25708385p25708385.html
Sent from the PHP - General mailing list archive at Nabble.com.

------=_Part_4709_4832042.1254440234883--

Re: Curl output

On Thu, Oct 01, 2009 at 04:37:14PM -0700, gbhumphrey wrote:

>
> Hi, I am doing a basical curl call and can get the webpage I want. However
> when it prints to the screen, it only prints the text, not css or any
> javascript calls that run on page load.
> Is there a way to make it do that?
>
> thanks
> using a basic curl call
> $curl_handle=curl_init();
> curl_setopt($curl_handle,CURLOPT_URL,'http://example.com');
> curl_setopt($ch, CURLOPT_RETURNTRANSFER,0); // return into a variable
> curl_exec($curl_handle);

I don't know about the javascript, but if the CSS is an external file,
then it wouldn't get dragged along with the original file, and thus
wouldn't be there to style anything on the HTML page. Just a guess.

Paul

--
Paul M. Foster

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Curl output

gbhumphrey wrote:
> Hi, I am doing a basical curl call and can get the webpage I want. However
> when it prints to the screen, it only prints the text, not css or any
> javascript calls that run on page load.
> Is there a way to make it do that?
>
> thanks
> using a basic curl call
> $curl_handle=curl_init();
> curl_setopt($curl_handle,CURLOPT_URL,'http://example.com');
> curl_setopt($ch, CURLOPT_RETURNTRANSFER,0); // return into a variable
> curl_exec($curl_handle);
>
>
>
>
>

Along the lines of what Paul has already alluded to, you will probably find upon further examination
of the html source that the "linked" files (js, css, etc...) are probably using relative paths, not
absolute paths including the domain name.

To fix this, you would have to do a little source code rewriting to add the full domain name and
directory path information to the "linked" in url structure.

I know if you do this with wget (cli app) that you can have it re-write the domain + path url
references. But, I'm not sure if cURL has an option for that. You might RTM on the cURL options
page to find out.

http://php.net/curl


--
Jim Lucas

"Some men are born to greatness, some achieve greatness,
and some have greatness thrust upon them."

Twelfth Night, Act II, Scene V
by William Shakespeare

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Curl output

using the http://www.w3schools.com/TAGS/tag_base.asp
but I am not sure if base tag works outside ....

try... before curl_init()
where URL is parsed using PHP's parse_url() function on http://example.com

in either case if there are any external js/css files cURL wont fetch
external files(and if they are internal/inline your browser should
display them already).. your browser will have to fetch them for you,
net console of firebug will be helpful in that case.

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Curl output

On Fri, 2009-10-02 at 12:51 +0530, kranthi wrote:
> using the > http://www.w3schools.com/TAGS/tag_base.asp
> but I am not sure if base tag works outside ....
>
> try... before curl_init()
> where URL is parsed using PHP's parse_url() function on http://example.com
>
> in either case if there are any external js/css files cURL wont fetch
> external files(and if they are internal/inline your browser should
> display them already).. your browser will have to fetch them for you,
> net console of firebug will be helpful in that case.
>

Some browser security settings may not allow you to run Javascript code
that exists on another server though. The tag can be used to
reference the external CSS and Javascript, but watch out for security
settings on client agents.

Thanks,
Ash
http://www.ashleysheridan.co.uk




--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Curl output

>> Some browser security settings may not allow you to run Javascript code
>> that exists on another server though
not many users use those kind of browsers, because if they do most of
the websites which use CDNs will not work.

Firstly, it is not a good idea to fetch an entire web page and snow it
to an user. (use iframes if this is a must)
Secondly, this idea may not be feasible if the web page in question
uses AJAX, because none of the browsers allow cross domain AJAX
requests

As a side note, use str_replace("", "",
$text) if base tag doesnot work

On 02/10/2009, Ashley Sheridan wrote:
> On Fri, 2009-10-02 at 12:51 +0530, kranthi wrote:
>> using the >> relative..
>> http://www.w3schools.com/TAGS/tag_base.asp
>> but I am not sure if base tag works outside ....
>>
>> try... before curl_init()
>> where URL is parsed using PHP's parse_url() function on http://example.com
>>
>> in either case if there are any external js/css files cURL wont fetch
>> external files(and if they are internal/inline your browser should
>> display them already).. your browser will have to fetch them for you,
>> net console of firebug will be helpful in that case.
>>
>
> Some browser security settings may not allow you to run Javascript code
> that exists on another server though. The tag can be used to
> reference the external CSS and Javascript, but watch out for security
> settings on client agents.
>
> Thanks,
> Ash
> http://www.ashleysheridan.co.uk
>
>
>
>


--
Kranthi.

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Curl output

On Fri, 2009-10-02 at 15:59 +0530, kranthi wrote:
> not many users use those kind of browsers, because if they do most of
> the websites which use CDNs will not work.

I've read that the upcoming Firefox 4 may have some features built in
for this sort of thing, and there are plugins out there for most
browsers that can do this as an added layer of security.

On this front though, has anyone else read about Microsofts Gazelle
browser/os? I'm not trying to plug it here or anything (I'm an anti
Windows person myself!) but the concept looks interesting. It goes
further than Chrome where each tab has it's own process by giving each
domain specific object its own process/sandbox.

Thanks,
Ash
http://www.ashleysheridan.co.uk




--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

Re: Curl output

>> I've read that the upcoming Firefox 4 may have some features built in
>> for this sort of thing, and there are plugins out there for most
>> browsers that can do this as an added layer of security.
Sorry but I could not understand what you meant by "this"

coming back to original problem... you should keep in mind that if
base tag is used, the links () in that page will become
absolute links instead of relative links

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

On Fri, 2009-10-02 at 18:07 +0530, kranthi wrote:
> >> I've read that the upcoming Firefox 4 may have some features built in
> >> for this sort of thing, and there are plugins out there for most
> >> browsers that can do this as an added layer of security.
> Sorry but I could not understand what you meant by "this"
>
> coming back to original problem... you should keep in mind that if
> base tag is used, the links () in that page will become
> absolute links instead of relative links
>

"this" is this:

"Some browser security settings may not allow you to run Javascript code
that exists on another server though"

Also, you won't have to worry about the links, unless you wanted the
links that were relative to the remote site to be relative to your own.
This would only work out in such a way if you'd scraped their whole site
for content, which would beg the question, "why would someone honestly
need to scrape an entire site?"

Thanks,
Ash
http://www.ashleysheridan.co.uk




--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php