Spelling suggestions for common words - ispell, etc.

I am looking for a way to, without custom defining a dictionary, to
get a list of suggested words for a misspelled word. Or better, "the"
most likely intended word for a misspelled word.

My base case to consider is:

dmr wjite saddle

which refers to a brand (DMR) and color (white) of a bike part
(saddle).

Ideally, dmr would return no suggestion, and wjite would return the
string "white" though I could certainly understand why "write" is
equally good a suggestion. I would be willing to define an add-on
dictionary to ignore certain words, such as brands and abbreviations
which are known to me, such as DMR, so that is possible to handle.

ispell -a yields:

$ echo "dmr wjite saddle" | ispell -a
@(#) International Ispell Version 3.1.20 (but really Aspell 0.50.5)
& dmr 28 0: Dr, Mr, DMD, DMZ, Dar, Der, Dir, Dur, Dem, dorm, Dame,
dame, demur, dime, dimer, dome, MDT, mfr, Dom, Dre, Dru, MRI, dam,
dim, dry, MD, Md, rm
& wjite 29 4: jute, kite, White, white, quite, Waite, write, Joete,
jitter, kiter, jet, Kit, jot, jut, kit, quiet, whiter, whitey, quote,
Whit, whit, Cote, Jude, Kate, cote, cute, quit, Wit, wit
*

from which I could easily pass on the dmr suggestions, but, scoring
and evaluating the suggestions for wjite is harder. "white" and
"write" are 'ranked' (I guess) 3rd, 4th, and 7th.

Does anyone know of an alternative that would return basic words as
suggestions? ispell is certainly a good start and I might be able to
use it, but I was thinking maybe there is something more human-
intuitive out there.

Thanks!
David

Re: Spelling suggestions for common words - ispell, etc.

sftriman wrote:
> get a list of suggested words for a misspelled word. Or better, "the"
> most likely intended word for a misspelled word.

Ever notice that Google does a pretty good job of that? So consider
Net::Google::Spelling:
http://search.cpan.org/~bstilwell/Net-Google-1.0.1/lib/Net/G oogle/Spelling.pm

--
David Filmer (http://DavidFilmer.com)

Re: Spelling suggestions for common words - ispell, etc.

sftriman writes:

> I am looking for a way to, without custom defining a dictionary, to
> get a list of suggested words for a misspelled word. Or better, "the"
> most likely intended word for a misspelled word.

You may find this article interesting:
http://norvig.com/spell-correct.html

You still need a list of "good" words, of course.

--
Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/

Re: Spelling suggestions for common words - ispell, etc.

On Apr 3, 6:47 pm, sftriman wrote:
> I am looking for a way to, without custom defining a dictionary, to
> get a list of suggested words for a misspelled word. Or better, "the"
> most likely intended word for a misspelled word.

> from which I could easily pass on the dmr suggestions, but, scoring
> and evaluating the suggestions for wjite is harder. "white" and
> "write" are 'ranked' (I guess) 3rd, 4th, and 7th.

One thing which might help you rank the strings is the "Levenshtein
distance". This gives you the "difference" between two strings as a
number. I don't know if it is on CPAN but there is a module found
here:

http://world.std.com/~swmcd/steven/perl/lib/String/Levenshte in/index.html

The documentation is here:

http://world.std.com/~swmcd/steven/perl/lib/String/Levenshte in/Levenshtein.html

Presumably the string with the smallest Levenshtein distance from the
input string would be the most likely candidate for the spelling
checker, although some very rare words might have small distances.

Re: Spelling suggestions for common words - ispell, etc.

On Thu, 3 Apr 2008 23:27:56 -0700 (PDT) Ben Bullock wrote:

BB> On Apr 3, 6:47 pm, sftriman wrote:
>> I am looking for a way to, without custom defining a dictionary, to
>> get a list of suggested words for a misspelled word. Or better, "the"
>> most likely intended word for a misspelled word.

>> from which I could easily pass on the dmr suggestions, but, scoring
>> and evaluating the suggestions for wjite is harder. "white" and
>> "write" are 'ranked' (I guess) 3rd, 4th, and 7th.

BB> One thing which might help you rank the strings is the "Levenshtein
BB> distance". This gives you the "difference" between two strings as a
BB> number. I don't know if it is on CPAN but there is a module found
BB> here:

BB> http://world.std.com/~swmcd/steven/perl/lib/String/Levenshte in/index.html

BB> The documentation is here:

BB> http://world.std.com/~swmcd/steven/perl/lib/String/Levenshte in/Levenshtein.html

BB> Presumably the string with the smallest Levenshtein distance from the
BB> input string would be the most likely candidate for the spelling
BB> checker, although some very rare words might have small distances.

It's useful to rank the distance in terms of how close keys are on the
keyboard. For example, h and j are more likely to be swapped than h and
r, for the white/write/wjite example above. On CPAN, I found:

String::Similarity
String::KeyboardDistance (see above)
String::Approx (very comprehensive, probably the right choice for the OP)
Text::DoubleMetaphone

Ted