forcing utf8 on selected values

There are several to-do items regarding utf8 that would be
great, but aren't here yet.

My database has utf8 columns, server/client character sets and
collation. Yet when I call fetchrow, resulting Perl scalars do
not have the utf8 flag set. That's really annoying. Am I
supposed to call

map { utf8::upgrade($_) } $every,$stupid,$little,$scalar

every time I do a select statement? Or map through the results
of any selectall_*ref's? This fixes the display problems when
printing a page to a browser with charset utf-8.

Is there an easier way to do this? Please tell me there is.

Mark

RE: forcing utf8 on selected values

------_=_NextPart_001_01C652F9.21C8F05C
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable


What database are you using? Perhaps the driver has an option to do =
this. I know Oracle does.

-----Original Message-----
From: Mark Hedges [mailto:hedges@ucsd.edu]
Sent: Tue 3/28/2006 9:39 PM
To: dbi-users@perl.org
Subject: forcing utf8 on selected values
=20

There are several to-do items regarding utf8 that would be=20
great, but aren't here yet.

My database has utf8 columns, server/client character sets and=20
collation. Yet when I call fetchrow, resulting Perl scalars do=20
not have the utf8 flag set. That's really annoying. Am I=20
supposed to call=20

map { utf8::upgrade($_) } $every,$stupid,$little,$scalar

every time I do a select statement? Or map through the results=20
of any selectall_*ref's? This fixes the display problems when=20
printing a page to a browser with charset utf-8.

Is there an easier way to do this? Please tell me there is.

Mark



------_=_NextPart_001_01C652F9.21C8F05C--

Re: forcing utf8 on selected values

Soe does a recent version of DBD::Pg (> = 1.40, pg_enable_utf8 database
handle attribute), and with my patch
(http://www.alexander-foken.de/unicode-patch.txt.gz,
http://www.alexander-foken.de/README.unicode-patch.html) also DBD::ODBC
1.13.

For proper Unicode support on Oracle, you should use at least Oracle 9
on both client and server, and you should read
http://search.cpan.org/~timb/DBD-Oracle-1.16/Oracle.pm#Unico de at least
twice.

Alexander

On 29.03.2006 08:21, Garrett, Philip (MAN-Corporate) wrote:

>What database are you using? Perhaps the driver has an option to do this. I know Oracle does.
>
>-----Original Message-----
>From: Mark Hedges [mailto:hedges@ucsd.edu]
>Sent: Tue 3/28/2006 9:39 PM
>To: dbi-users@perl.org
>Subject: forcing utf8 on selected values
>
>
>There are several to-do items regarding utf8 that would be
>great, but aren't here yet.
>
>My database has utf8 columns, server/client character sets and
>collation. Yet when I call fetchrow, resulting Perl scalars do
>not have the utf8 flag set. That's really annoying. Am I
>supposed to call
>
> map { utf8::upgrade($_) } $every,$stupid,$little,$scalar
>
>every time I do a select statement? Or map through the results
>of any selectall_*ref's? This fixes the display problems when
>printing a page to a browser with charset utf-8.
>
>Is there an easier way to do this? Please tell me there is.
>
>Mark
>
>
>
>
>


--
Alexander Foken
mailto:alexander@foken.de http://www.foken.de/alexander/

RE: forcing utf8 on selected values

On Wed, 29 Mar 2006, Garrett, Philip (MAN-Corporate) wrote:
>> My database has utf8 columns, server/client character sets and
>> collation. Yet when I call fetchrow, resulting Perl scalars do
>> not have the utf8 flag set. That's really annoying. Am I
>> supposed to call
>>
>> map { utf8::upgrade($_) } $every,$stupid,$little,$scalar
>>
>> every time I do a select statement? Or map through the results
>> of any selectall_*ref's? This fixes the display problems when
>> printing a page to a browser with charset utf-8.
>>
>> Is there an easier way to do this? Please tell me there is.
>
> What database are you using? Perhaps the driver has an option
> to do this. I know Oracle does.

mysql.

I've been told this might work upon connecting:

$dbh->do('SET NAMES utf8')

But it doesn't have any effect.

Hmm, for some reason this morning the Matrix has allowed the
selected characters to display correctly in the browser. This
doesn't make a whole lot of sense to me now.

The scalars still do not have the utf8 flag set, though.

Mark

RE: forcing utf8 on selected values

On Wed, 29 Mar 2006, Mark Hedges wrote:
>
> Hmm, for some reason this morning the Matrix has allowed the
> selected characters to display correctly in the browser. This
> doesn't make a whole lot of sense to me now.
>
> The scalars still do not have the utf8 flag set, though.

This is really weird. All the characters display correctly in
the browser now EXCEPT what used to be a generic apostrophe,
which now comes across as this non-printable character:

U+0092

General Character Properties

Unicode category: Other, Control

Various Useful Representations

UTF-8: 0xC2 0x92
Octal escaped UTF-8: \302\222
Decimal entity reference: ’

Annotations and Cross References

Alias names:
PRIVATE USE TWO

But this displays fine in mysql console on my terminal.

Argh.

Mark

RE: forcing utf8 on selected values

On Wed, 29 Mar 2006, Mark Hedges wrote:
> >
> > Hmm, for some reason this morning the Matrix has allowed the
> > selected characters to display correctly in the browser. This
> > doesn't make a whole lot of sense to me now.
> >
> > The scalars still do not have the utf8 flag set, though.
>
> This is really weird. All the characters display correctly in
> the browser now EXCEPT what used to be a generic apostrophe,
> which now comes across as this non-printable character:
>
> U+0092
>
> General Character Properties
>
> Unicode category: Other, Control
>
> Various Useful Representations
>
> UTF-8: 0xC2 0x92
> Octal escaped UTF-8: \302\222
> Decimal entity reference: ’
>
> Annotations and Cross References
>
> Alias names:
> PRIVATE USE TWO
>
> But this displays fine in mysql console on my terminal.

Here I am talking to myself in public again.

Scratch that -- the original source database is corrupt and
actually has this character stored in these fields. Why they
display correctly as quotes in the terminal or when viewed in
western encoding is a mystery.

Mark