What to do with UTF-8 data?

Hi,

I have a question regarding the handling of UTF-8 data using the
DBD-mysql driver.

I'm running Perl 5.8.0 with MySQL 3.23.56 (via DBD-mysql). Since (I
think) there is no native UTF-8 support in MySQL below 4.1.x, my plan
was to simply store the bytes of each UTF-8 character in the database
(converting Perl's UTF-8 strings to sequences of octets using
Encode::encode_utf8()), and then convert such octet sequences back to
UTF-8 strings when retrieving data from the database using
Encode::decode_utf8().

As long as I store *all* my data in this way, those conversions should
always succeed. (encode_utf8() never fails anyway, and decode_utf8()
will always work here because I'm always feeding it "valid" data.)

The problem is: How do I trap all input/output to/from DBI to do these
conversions?

I can easily do it "manually":

$dbh->do('INSERT INTO foo (bar) VALUES (?)', undef,
Encode::encode_utf8($input_utf8str));
...
my @octets_row = $dbh->selectrow_array('SELECT bar FROM foo');
my $output_utf8str2 = Encode::decode_utf8($octets_row[0]);

but that's way too tedious in practice. I want to have those
conversions done for me automatically.

I've asked about this on the dbi-users mailing list, and the answer
(from Tim Bunce, no less) was that it is really the responsibility of
the DBD driver to perform such conversions if the data in question is UTF-8.

He noted that MySQL 4.1.x provides information about the character set
of a given column, so DBD-mysql should be able to use that.

For those of us not wanting to take the plunge just yet with what is,
after all, still alpha software, he suggested that a driver private
option could be used to indicate that all char fields are UTF-8, or else
have some way of indicating that per-column, such as

$sth->bind_col(1, undef, { mysql_charset => 'utf8' });

Is there any such functionality currently in DBD-mysql, or any chance of
it or something similar being added soonish?

I could really use some means of telling DBD-mysql to do the conversions
outlined above on all my char data.

Thanks,
- Steve


--
MySQL Perl Mailing List
For list archives: http://lists.mysql.com/perl
To unsubscribe: http://lists.mysql.com/perl?unsub=gcdmp-msql-mysql-modules@m .gmane.org