What is the best regular expression for finding urls in plain text
files?
(By urls I mean http://www.something.com, but also www.something.com,
or salve@somewhere.com)
Salve
Salve Håkedal wrote:
> What is the best regular expression for finding urls in plain text
> files?
> (By urls I mean http://www.something.com, but also www.something.com,
> or salve@somewhere.com)
>
> Salve
The simplest way is to use the one thing they all have in common. ".com".
strstr($text,'.com');
Salve Håkedal wrote:
> What is the best regular expression for finding urls in plain text
> files?
> (By urls I mean http://www.something.com, but also www.something.com,
> or salve@somewhere.com)
>
> Salve
I've used this before, but you're probably better off making your own
expression. Note that it's really loose and will get a lot of false positives -
especially file names - and it will cause havoc if you use it on HTML source. I
deliberately did not enter any Top-Level-Domain filtering, because there are so
many of them. You can replace the [a-z]{2,5} with something like (com|net|org)
if you don't need to worry about country codes.
The following expression should find strings that satisfy these conditions:
- optionally a http protocol identifier
- optionally a username(:password)@ string, which allows pretty much any
characters except for spaces and colons. This isn't RFC-standard, by the way.
- a hostname consisting of at least two and at most 34 labels, the last of which
has 2 to 5 alphabet letters (for weird new ones like aero and museum; you can
shorten it to 3 and still get the most common ones).
- optionally a path containing any characters apart from spaces, and /ending in
a non-punctuation character/. This last bit is vital because it avoids messing
up URLs at the end of a sentence.
(http:\/\/)?([^ :]+(:[^
]+)?@)?[a-z0-9]([a-z0-9i\-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0 -9\-]{0,61}[a-z0-9])?){0,32}\.[a-z]{2,5}(\/[^
]*[^" \.,;\)])?
(linebreaks are added by email client)
This is a case insensitive pattern, you'll need the i modifier.
--
Christoph Burschka
On 2007-11-02, Christoph Burschka
> Salve Håkedal wrote:
>> What is the best regular expression for finding urls in plain text
>> files?
>> (By urls I mean http://www.something.com, but also www.something.com,
>> or salve@somewhere.com)
>>
>> Salve
>
> I've used this before, but you're probably better off making your own
> expression. Note that it's really loose and will get a lot of false positives -
> especially file names - and it will cause havoc if you use it on HTML source. I
> deliberately did not enter any Top-Level-Domain filtering, because there are so
> many of them. You can replace the [a-z]{2,5} with something like (com|net|org)
> if you don't need to worry about country codes.
>
> The following expression should find strings that satisfy these conditions:
>
> - optionally a http protocol identifier
> - optionally a username(:password)@ string, which allows pretty much any
> characters except for spaces and colons. This isn't RFC-standard, by the way.
> - a hostname consisting of at least two and at most 34 labels, the last of which
> has 2 to 5 alphabet letters (for weird new ones like aero and museum; you can
> shorten it to 3 and still get the most common ones).
> - optionally a path containing any characters apart from spaces, and /ending in
> a non-punctuation character/. This last bit is vital because it avoids messing
> up URLs at the end of a sentence.
>
> (http:\/\/)?([^ :]+(:[^
> ]+)?@)?[a-z0-9]([a-z0-9i\-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0 -9\-]{0,61}[a-z0-9])?){0,32}\.[a-z]{2,5}(\/[^
> ]*[^" \.,;\)])?
>
> (linebreaks are added by email client)
>
> This is a case insensitive pattern, you'll need the i modifier.
>
> --
> Christoph Burschka
Thanks alot! I'll study this closely
--
Salve
..oO(Chris Gorospe)
>Salve Håkedal wrote:
>> What is the best regular expression for finding urls in plain text
>> files?
>> (By urls I mean http://www.something.com, but also www.something.com,
>> or salve@somewhere.com)
>>
>The simplest way is to use the one thing they all have in common. ".com".
>
>strstr($text,'.com');
What about the other TLDs? There are _some_ more ...
Micha
"Salve Håkedal"
news:2-2dnSvk0Nk307baRVnzvQA@telenor.com...
> On 2007-11-02, Christoph Burschka
> wrote:
>> The following expression should find strings that satisfy these
>> conditions:
>>
>> - optionally a http protocol identifier
>> - optionally a username(:password)@ string, which allows pretty much any
>> characters except for spaces and colons. This isn't RFC-standard, by the
>> way.
>> - a hostname consisting of at least two and at most 34 labels, the last
>> of which
>> has 2 to 5 alphabet letters (for weird new ones like aero and museum; you
>> can
>> shorten it to 3 and still get the most common ones).
>> - optionally a path containing any characters apart from spaces, and
>> /ending in
>> a non-punctuation character/. This last bit is vital because it avoids
>> messing
>> up URLs at the end of a sentence.
>>
>> (http:\/\/)?([^ :]+(:[^
>> ]+)?@)?[a-z0-9]([a-z0-9i\-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0 -9\-]{0,61}[a-z0-9])?){0,32}\.[a-z]{2,5}(\/[^
>> ]*[^" \.,;\)])?
>>
>> (linebreaks are added by email client)
>>
>> This is a case insensitive pattern, you'll need the i modifier.
>>
>> --
>> Christoph Burschka
>
> Thanks alot! I'll study this closely
Does it make your eyes and ears bleed,the way it does mine?
Been doin this stuff since the 70's - but regex still makes me cry.
On that note, I am at the moment, writing a function that could sure benefit
from some regex.
I just want to see if a string starts with "http(s)://", "news:", "mailto:",
"ftp:".
That's a pretty simple regex, right?
Hoooowwww?
"Sanders Kaufman"
> Does it make your eyes and ears bleed,the way it does mine?
> Been doin this stuff since the 70's - but regex still makes me cry.
>
> On that note, I am at the moment, writing a function that could sure benefit
> from some regex.
> I just want to see if a string starts with "http(s)://", "news:", "mailto:",
> "ftp:".
> That's a pretty simple regex, right?
> Hoooowwww?
This should get you started:
$pattern = '/^(http(s)?:\/\/|news:|mailto:|ftp:)/';
$tests = array('http://www.google.com', 'https://www.google.com',
'news:comp.lang', 'mailto:test@nowhere.com',
' http://www.google.com',
'bad_http://www.google.com',
'mailtobad:fdsa', 'ftp://ftp.host.net',
'ftpbad:', 'badftp://');
foreach($tests as $v) {
print "'".$v."'" . ' ~ ' .
(preg_match($pattern, $v)
? 'matches'
: 'doesn\'t match')."\n";
}
On 2007-11-02, Carl
> "Sanders Kaufman"
>> Does it make your eyes and ears bleed,the way it does mine?
>> Been doin this stuff since the 70's - but regex still makes me cry.
>>
>> On that note, I am at the moment, writing a function that could sure benefit
>> from some regex.
>> I just want to see if a string starts with "http(s)://", "news:", "mailto:",
>> "ftp:".
>> That's a pretty simple regex, right?
>> Hoooowwww?
>
> This should get you started:
>
> $pattern = '/^(http(s)?:\/\/|news:|mailto:|ftp:)/';
>
> $tests = array('http://www.google.com', 'https://www.google.com',
> 'news:comp.lang', 'mailto:test@nowhere.com',
> ' http://www.google.com',
> 'bad_http://www.google.com',
> 'mailtobad:fdsa', 'ftp://ftp.host.net',
> 'ftpbad:', 'badftp://');
>
> foreach($tests as $v) {
> print "'".$v."'" . ' ~ ' .
> (preg_match($pattern, $v)
> ? 'matches'
> : 'doesn\'t match')."\n";
> }
Thank you, Carl, for the script.
But the regexp there is as simple at I could have written myself. What
I need is something that can find urls in a text file, and convert them
to links. And by urls I mean, as I wrote in OT: http://something.org as
well as for example www.someother.anytopdm and the url in the original
text may be in parantheses or for example at the end of a sentence, so
it'll have a . in the end. So on..
Christoph Burschka's is still the most promising, but I haven't had time
to understand and to try it out yet.
Salve
"Carl"
> "Sanders Kaufman"
>> Does it make your eyes and ears bleed,the way it does mine?
>> Been doin this stuff since the 70's - but regex still makes me cry.
>>
>> On that note, I am at the moment, writing a function that could sure
>> benefit
>> from some regex.
>> I just want to see if a string starts with "http(s)://", "news:",
>> "mailto:",
>> "ftp:".
>> That's a pretty simple regex, right?
>> Hoooowwww?
>
> This should get you started:
>
> $pattern = '/^(http(s)?:\/\/|news:|mailto:|ftp:)/';
>
> $tests = array('http://www.google.com', 'https://www.google.com',
> 'news:comp.lang', 'mailto:test@nowhere.com',
> ' http://www.google.com',
> 'bad_http://www.google.com',
> 'mailtobad:fdsa', 'ftp://ftp.host.net',
> 'ftpbad:', 'badftp://');
>
> foreach($tests as $v) {
> print "'".$v."'" . ' ~ ' .
> (preg_match($pattern, $v)
> ? 'matches'
> : 'doesn\'t match')."\n";
> }
That helps a LOT.
Thanks.
Salve Håkedal
> On 2007-11-02, Carl
>> "Sanders Kaufman"
>>> Does it make your eyes and ears bleed,the way it does mine?
>>> Been doin this stuff since the 70's - but regex still makes me cry.
>>>
>>> On that note, I am at the moment, writing a function that could sure benefit
>>> from some regex.
>>> I just want to see if a string starts with "http(s)://", "news:", "mailto:",
>>> "ftp:".
>>> That's a pretty simple regex, right?
>>> Hoooowwww?
>>
>> This should get you started:
>>
>> $pattern = '/^(http(s)?:\/\/|news:|mailto:|ftp:)/';
>>
>> $tests = array('http://www.google.com', 'https://www.google.com',
>> 'news:comp.lang', 'mailto:test@nowhere.com',
>> ' http://www.google.com',
>> 'bad_http://www.google.com',
>> 'mailtobad:fdsa', 'ftp://ftp.host.net',
>> 'ftpbad:', 'badftp://');
>>
>> foreach($tests as $v) {
>> print "'".$v."'" . ' ~ ' .
>> (preg_match($pattern, $v)
>> ? 'matches'
>> : 'doesn\'t match')."\n";
>> }
>
> Thank you, Carl, for the script.
>
> But the regexp there is as simple at I could have written myself. What
> I need ...
--8<-- message cut -->8--
Salve,
My response was a followup to Sanders, not your O.P., I assumed that
the previous posts answered you question already.
The question you posted is quite common, and google'n should turn
up enough examples that you should'nt have to do much to get it working
decently enough.
For starters, this looks promising, though I haven't tested it (from
the 1st results page of a google search)
http://immike.net/blog/2007/04/06/5-regular-expressions-ever y-web-programmer-should-know/
--
Hope that helps,
Carl.
Salve H=E5kedal:
> What is the best regular expression for finding urls in plain text
> files?
Matching URLs of every scheme with a single regular expression would
be
incredibly complex and complicated. For example, matching mailto URLs
would entail the notorious regular expression for matching e-mail
addresses. Even matching URLs of individual schemes would require
careful study of both RFC3986 and the specification that governs that
particular URL scheme. Moreover, if you want to turn partial URLs
such as www.example.com into complete URLs, you would need to define
your own heuristics for doing so.
Upshot is, you're in a world of hurt.
--
Jock