packet sniffing with WinPcap/ SharpPcap: How to download FLV content noninteractively?

I have been investigating programmatically downloading FLV content from
various sites ("video scraping"??)

Many interactive GUI tools do this, such as the Orbit downloader. At the
heart of them are packet sniffers which inspect the ethernet traffic coming
over the wire. I would like to do this noninteractively in code.

There is a .NET specific flavor of WinPCap called SharpPCap:
http://www.tamirgal.com/home/dev.aspx?Item=SharpPcap
Article on the Code Project:
http://www.codeproject.com/KB/IP/sharppcap.aspx

The first question I have is if server-side visits by the WebClient class
will result in "inspectable" results for FLV content. In other words if you
tell WebClient to go visit a site with FLVs on the page, can the server
sniff its own activity using something like SharpPCap?

The second question I have is more general, and would apply to
noninteractive and interactive apps I might build. The question is how
exactly to use something like SharPCap to find a simple list of FLV links in
the network traffic. I have never used a packet sniffer or sniffing
framework and have no idea how you'd (totally guessing here) pull strings of
interest out of the raw traffic.

Thanks much for any help and insight you can offer.

-KF

RE: packet sniffing with WinPcap/ SharpPcap: How to download FLV content noninteractively?

Hi KF,

For the FLV sniffing and downloading code, do you means you want to
programmticaly crawl some websites and download the FLV links found on
those websites?

For the two questions you mentioned:

1. If the server-side you mentioned here is the website/web application.
Sure, it can get noticed of all those client requesters(both interactive
users or some crawler). Generally the "UserAgent" http header will indicate
this and such information is normally easily to get at application
layer(such as HTTP or ASP.NET webapplicaiton), the server-side even do not
need to hook into the raw network or transport layer to get the messages.

2. If what you want to do is find the FLV links, I think a straightforward
means is to parse the html resposne content of each web page and look for
the FLV links(through url extensions or