utf8 encoding problem

------=_Part_21664_60370.1166681873824
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hello,

I had posted this question in CPAN forum but i was directed here for a
better answer.

Here is the problem I am facing. In the following piece of code I get a
customer name from the database and append a pound symbol to the customer
name and update the same in the database. The NLS_LANG parameter is set to
AMERICAN_AMERICA.WE8DEC (western European character set) When I query the
database to see if the name has been updated correctly I see some garbage
value appended to the name instead of just a pound symbol. Then I did some
research to find out the version oralce DBD version being used is 1.12. I
found some difference between 1.12 and 1.14 relating to unicode
documentation. All I want to know is whether this was a bug in 1.12 which
was fixed in the later version of 1.14 and will an upgrade to 1.14 fix this
problem. Could anyone please confirm me on this? The closer I could get was
to find this link http://search.cpan.org/dist/DBD-Oracle/Changes#___top.
However the work around was to use "use Encoding" (Which is commented out in
the following code) in perl which fixed the problem.

Cheers,

Anand.



use strict;

use encoding 'utf8';

use atadb;

use vars qw($opt_u);

my $db = atadb::connect($opt_u);

my $sql = " SELECT NODE_NAME FROM CUSTOMER_NODE_HISTORY WHERE
CUSTOMER_NODE_ID = 9295370 ";

my $csr = $db->prepare($sql) || ataerr::dbprepare($db);

$csr->execute() || ataerr::dbexecute($db);

my $node_name = $csr->fetchrow; $csr->finish;

$node_name = 'wre'; my $pound = " \xc2\xa3"; print "Pound =>$pound\n";

my $node_name_upper = "\U$node_name\E".$pound; $node_name.=$pound; print
"Node name".$node_name."\n";;

print $node_name_upper."\n"; my $sql1 = "UPDATE CUSTOMER_NODE_HISTORY SET
NODE_NAME = ? , NODE_NAME_UPPERCASE = ? WHERE CUSTOMER + +_NODE_ID = 9295370
";

my $csr1 = $db->prepare($sql1) || ataerr::dbprepare($db); print "Executing
.... \n"; #use Encode; #$node_name = encode($ENV{PERL_ENCODING}, $node_name);
#$node_name_upper = encode($ENV{PERL_ENCODING}, $node_name_upper);

$csr1->execute($node_name, $node_name_upper) || ataerr::dbexecute($db);

print "Committing ... \n";

$db->commit || die("commit: ".$db->errstr."\n");

$csr1->finish; $db->disconnect;

------=_Part_21664_60370.1166681873824--

Re: utf8 encoding problem

Unfortunety I cannot say for 100% sure this will solve you problem as there
is no way to exactly match your system's setup. Oracle
version/patches/Oracle client system enviornment etc.

There were a large number of improvements in UTF encoding since 1.12 so my
only sugestion is for you to install DBD::Oracle 1.19 locally on a seperate
machine and then test to see if the bug continues.

The UTF bug like this could be caused by the Oracle cleint or the Oracle DB
as well and not just DBD::Oracle.

What are you Oracle client and DB versions?

----- Original Message -----
From: "Anand.K.S."
To:
Sent: Thursday, December 21, 2006 1:17 AM
Subject: utf8 encoding problem


> Hello,
>
> I had posted this question in CPAN forum but i was directed here for a
> better answer.
>
> Here is the problem I am facing. In the following piece of code I get a
> customer name from the database and append a pound symbol to the customer
> name and update the same in the database. The NLS_LANG parameter is set to
> AMERICAN_AMERICA.WE8DEC (western European character set) When I query the
> database to see if the name has been updated correctly I see some garbage
> value appended to the name instead of just a pound symbol. Then I did some
> research to find out the version oralce DBD version being used is 1.12. I
> found some difference between 1.12 and 1.14 relating to unicode
> documentation. All I want to know is whether this was a bug in 1.12 which
> was fixed in the later version of 1.14 and will an upgrade to 1.14 fix
> this
> problem. Could anyone please confirm me on this? The closer I could get
> was
> to find this link http://search.cpan.org/dist/DBD-Oracle/Changes#___top.
> However the work around was to use "use Encoding" (Which is commented out
> in
> the following code) in perl which fixed the problem.
>
> Cheers,
>
> Anand.
>
>
>
> use strict;
>
> use encoding 'utf8';
>
> use atadb;
>
> use vars qw($opt_u);
>
> my $db = atadb::connect($opt_u);
>
> my $sql = " SELECT NODE_NAME FROM CUSTOMER_NODE_HISTORY WHERE
> CUSTOMER_NODE_ID = 9295370 ";
>
> my $csr = $db->prepare($sql) || ataerr::dbprepare($db);
>
> $csr->execute() || ataerr::dbexecute($db);
>
> my $node_name = $csr->fetchrow; $csr->finish;
>
> $node_name = 'wre'; my $pound = " \xc2\xa3"; print "Pound =>$pound\n";
>
> my $node_name_upper = "\U$node_name\E".$pound; $node_name.=$pound; print
> "Node name".$node_name."\n";;
>
> print $node_name_upper."\n"; my $sql1 = "UPDATE CUSTOMER_NODE_HISTORY SET
> NODE_NAME = ? , NODE_NAME_UPPERCASE = ? WHERE CUSTOMER + +_NODE_ID =
> 9295370
> ";
>
> my $csr1 = $db->prepare($sql1) || ataerr::dbprepare($db); print "Executing
> ... \n"; #use Encode; #$node_name = encode($ENV{PERL_ENCODING},
> $node_name);
> #$node_name_upper = encode($ENV{PERL_ENCODING}, $node_name_upper);
>
> $csr1->execute($node_name, $node_name_upper) || ataerr::dbexecute($db);
>
> print "Committing ... \n";
>
> $db->commit || die("commit: ".$db->errstr."\n");
>
> $csr1->finish; $db->disconnect;
>

RE: utf8 encoding problem

The easiest way to know is to try it out. If you want to just test it
without replacing your installation's DBD::Oracle, you can do a
temporary install of a newer version of DBD::Oracle by using the
INSTALLSITELIB parameter to Makefile.PL:

wget
http://www.perl.com/CPAN/modules/by-module/DBD/DBD-Oracle-1. 14.tar.gz
tar xvfz DBD-Oracle-1.14.tar.gz
cd DBD-Oracle-1.14.tar.gz
export ORACLE_HOME=3D/path/to/your/oracle/installation
export ORACLE_USERID=3Duser/pass
export ORACLE_SID=3Dyour-sid
perl Makefile.PL INSTALLSITELIB=3D$HOME/tmp-dbd-lib
make test
make install

If the build process was successful, you'll have DBD::Oracle installed
in $HOME/tmp-dbd-lib. You can try it out using the -I parameter for
perl:

perl -I$HOME/tmp-dbd-lib your_script.pl

If it doesn't help, just "rm -rf $HOME/tmp-dbd-lib". No harm, no foul!

Regards
Philip

-----Original Message-----
From: Anand.K.S. [mailto:ks.anand80@gmail.com]=20
Sent: Thursday, December 21, 2006 1:18 AM
To: dbi-users@perl.org
Subject: utf8 encoding problem

Hello,

I had posted this question in CPAN forum but i was directed here for a
better answer.

Here is the problem I am facing. In the following piece of code I get a
customer name from the database and append a pound symbol to the
customer
name and update the same in the database. The NLS_LANG parameter is set
to
AMERICAN_AMERICA.WE8DEC (western European character set) When I query
the
database to see if the name has been updated correctly I see some
garbage
value appended to the name instead of just a pound symbol. Then I did
some
research to find out the version oralce DBD version being used is 1.12.
I
found some difference between 1.12 and 1.14 relating to unicode
documentation. All I want to know is whether this was a bug in 1.12
which
was fixed in the later version of 1.14 and will an upgrade to 1.14 fix
this
problem. Could anyone please confirm me on this? The closer I could get
was
to find this link http://search.cpan.org/dist/DBD-Oracle/Changes#___top.
However the work around was to use "use Encoding" (Which is commented
out in
the following code) in perl which fixed the problem.

Cheers,

Anand.



use strict;

use encoding 'utf8';

use atadb;

use vars qw($opt_u);

my $db =3D atadb::connect($opt_u);

my $sql =3D " SELECT NODE_NAME FROM CUSTOMER_NODE_HISTORY WHERE
CUSTOMER_NODE_ID =3D 9295370 ";

my $csr =3D $db->prepare($sql) || ataerr::dbprepare($db);

$csr->execute() || ataerr::dbexecute($db);

my $node_name =3D $csr->fetchrow; $csr->finish;

$node_name =3D 'wre'; my $pound =3D " \xc2\xa3"; print "Pound =
=3D>$pound\n";

my $node_name_upper =3D "\U$node_name\E".$pound; $node_name.=3D$pound; =
print
"Node name".$node_name."\n";;

print $node_name_upper."\n"; my $sql1 =3D "UPDATE CUSTOMER_NODE_HISTORY
SET
NODE_NAME =3D ? , NODE_NAME_UPPERCASE =3D ? WHERE CUSTOMER + +_NODE_ID =
=3D
9295370
";

my $csr1 =3D $db->prepare($sql1) || ataerr::dbprepare($db); print
"Executing
.... \n"; #use Encode; #$node_name =3D encode($ENV{PERL_ENCODING},
$node_name);
#$node_name_upper =3D encode($ENV{PERL_ENCODING}, $node_name_upper);

$csr1->execute($node_name, $node_name_upper) || ataerr::dbexecute($db);

print "Committing ... \n";

$db->commit || die("commit: ".$db->errstr."\n");

$csr1->finish; $db->disconnect;

Re: utf8 encoding problem

------=_Part_30578_30132098.1166745718932
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Many thanks for your response.
Client version - SQL*Plus: Release 10.2.0.1.0 - Production on Thu Dec 21
16:31:14 2006
Oracle version - Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 -
64bi

Cheers,
Anand.

On 12/21/06, John Scoles wrote:
>
> Unfortunety I cannot say for 100% sure this will solve you problem as
> there
> is no way to exactly match your system's setup. Oracle
> version/patches/Oracle client system enviornment etc.
>
> There were a large number of improvements in UTF encoding since 1.12 so my
> only sugestion is for you to install DBD::Oracle 1.19 locally on a
> seperate
> machine and then test to see if the bug continues.
>
> The UTF bug like this could be caused by the Oracle cleint or the Oracle
> DB
> as well and not just DBD::Oracle.
>
> What are you Oracle client and DB versions?
>
> ----- Original Message -----
> From: "Anand.K.S."
> To:
> Sent: Thursday, December 21, 2006 1:17 AM
> Subject: utf8 encoding problem
>
>
> > Hello,
> >
> > I had posted this question in CPAN forum but i was directed here for a
> > better answer.
> >
> > Here is the problem I am facing. In the following piece of code I get a
> > customer name from the database and append a pound symbol to the
> customer
> > name and update the same in the database. The NLS_LANG parameter is set
> to
> > AMERICAN_AMERICA.WE8DEC (western European character set) When I query
> the
> > database to see if the name has been updated correctly I see some
> garbage
> > value appended to the name instead of just a pound symbol. Then I did
> some
> > research to find out the version oralce DBD version being used is 1.12.
> I
> > found some difference between 1.12 and 1.14 relating to unicode
> > documentation. All I want to know is whether this was a bug in 1.12which
> > was fixed in the later version of 1.14 and will an upgrade to 1.14 fix
> > this
> > problem. Could anyone please confirm me on this? The closer I could get
> > was
> > to find this link http://search.cpan.org/dist/DBD-Oracle/Changes#___top.
> > However the work around was to use "use Encoding" (Which is commented
> out
> > in
> > the following code) in perl which fixed the problem.
> >
> > Cheers,
> >
> > Anand.
> >
> >
> >
> > use strict;
> >
> > use encoding 'utf8';
> >
> > use atadb;
> >
> > use vars qw($opt_u);
> >
> > my $db = atadb::connect($opt_u);
> >
> > my $sql = " SELECT NODE_NAME FROM CUSTOMER_NODE_HISTORY WHERE
> > CUSTOMER_NODE_ID = 9295370 ";
> >
> > my $csr = $db->prepare($sql) || ataerr::dbprepare($db);
> >
> > $csr->execute() || ataerr::dbexecute($db);
> >
> > my $node_name = $csr->fetchrow; $csr->finish;
> >
> > $node_name = 'wre'; my $pound = " \xc2\xa3"; print "Pound =>$pound\n";
> >
> > my $node_name_upper = "\U$node_name\E".$pound; $node_name.=$pound; print
> > "Node name".$node_name."\n";;
> >
> > print $node_name_upper."\n"; my $sql1 = "UPDATE CUSTOMER_NODE_HISTORY
> SET
> > NODE_NAME = ? , NODE_NAME_UPPERCASE = ? WHERE CUSTOMER + +_NODE_ID =
> > 9295370
> > ";
> >
> > my $csr1 = $db->prepare($sql1) || ataerr::dbprepare($db); print
> "Executing
> > ... \n"; #use Encode; #$node_name = encode($ENV{PERL_ENCODING},
> > $node_name);
> > #$node_name_upper = encode($ENV{PERL_ENCODING}, $node_name_upper);
> >
> > $csr1->execute($node_name, $node_name_upper) || ataerr::dbexecute($db);
> >
> > print "Committing ... \n";
> >
> > $db->commit || die("commit: ".$db->errstr."\n");
> >
> > $csr1->finish; $db->disconnect;
> >
>
>

------=_Part_30578_30132098.1166745718932--

Re: utf8 encoding problem

--envbJBWh7q8WU6mo
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2006-12-21 16:17:53 +1000, Anand.K.S. wrote:
> Hello,
>=20
> I had posted this question in CPAN forum but i was directed here for a
> better answer.
>=20
> Here is the problem I am facing. In the following piece of code I get a
> customer name from the database and append a pound symbol to the customer
> name and update the same in the database. The NLS_LANG parameter is set to
> AMERICAN_AMERICA.WE8DEC (western European character set)



> $node_name =3D 'wre'; my $pound =3D " \xc2\xa3"; print "Pound =3D>$pound\=
n";

That doesn't look like a pound sign in WE8DEC. You need to set NLS_LANG
to the charset that you actually use, i.e., AMERICAN_AMERICA.UTF8 in
your case.

hp

PS: Why are you using WE8DEC at all, and not ISO-8859-1?

--=20
_ | Peter J. Holzer | If I wanted to be "academically correct",
|_|_) | Sysadmin WSR | I'd be programming in Java.
| | | hjp@wsr.ac.at | I don't, and I'm not.
__/ | http://www.hjp.at/ | -- Jesse Erlbaum on dbi-users

--envbJBWh7q8WU6mo
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iQDQAwUBRYud4lLjemazOuKpAQIaVgXSAgYROJ7IM52fsfUd9QUV7eUCoySy m5EA
6crilxnpBNZ1V9qQw7G954bNMFdablRCOzG7z8ki99uNr+6675KtChnIIdy1 bi34
W0V1cN0XLleOCHYbuf7MPy1l27pT8seEe23Ny5EpsVfkB30HPAT1FE0LXiwg CCSU
lM1Qg6YmnCDzZ2Z1wi//Fm4lxgUeWIhSGhP41PKmyHI35yYTvTYSmffUjgl5 Enlv
7I61iKvlQMMAZC5vCHNfrapOFg==
=597H
-----END PGP SIGNATURE-----

--envbJBWh7q8WU6mo--

Re: utf8 encoding problem

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Peter J. Holzer wrote:
>> $node_name = 'wre'; my $pound = " \xc2\xa3"; print "Pound =>$pound\n";
> That doesn't look like a pound sign in WE8DEC. You need to set NLS_LANG
> to the charset that you actually use, i.e., AMERICAN_AMERICA.UTF8 in
> your case.

Or "AMERICAN_AMERICA.AL32UTF8" - See
http://download-west.oracle.com/docs/cd/B10501_01/server.920 /a96529/ch5.htm#1004607

- --michael

- --
063A F25E B064 A98F A479 1690 78CD D023 5E2A 6688
http://zis.uibk.ac.at/.m/uibk.ac.at_pgp_pubkey.asc
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (MingW32)

iD8DBQFFi6FdeM3QI14qZogRAnP9AJ9Kupo++1xIX98Wqi9jScSPZXDYbwCf ahUz
jQGI9DtU4zKueT9eZsB8sfo=
=fsB4
-----END PGP SIGNATURE-----

Re: utf8 encoding problem

--sHrvAb52M6C8blB9
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2006-12-22 10:11:57 +0100, Michael Kröll wrote:
> Peter J. Holzer wrote:
> >> $node_name =3D 'wre'; my $pound =3D " \xc2\xa3"; print "Pound =3D>$pou=
nd\n";
> > That doesn't look like a pound sign in WE8DEC. You need to set NLS_LANG
> > to the charset that you actually use, i.e., AMERICAN_AMERICA.UTF8 in
> > your case.
>=20
> Or "AMERICAN_AMERICA.AL32UTF8" - See
> http://download-west.oracle.com/docs/cd/B10501_01/server.920 /a96529/ch5.h=
tm#1004607

Yes. AL32UTF8 is better (unless you're still running Oracle 8). Don't
know why I wrote UTF8.

hp

--=20
_ | Peter J. Holzer | If I wanted to be "academically correct",
|_|_) | Sysadmin WSR | I'd be programming in Java.
| | | hjp@wsr.ac.at | I don't, and I'm not.
__/ | http://www.hjp.at/ | -- Jesse Erlbaum on dbi-users

--sHrvAb52M6C8blB9
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iQDQAwUBRYupF1LjemazOuKpAQKsnwXUCIQmUQK2PjgZC/rysaM4MhomQUYW EQ45
L0mL1QBoy1upQLhR1/tYLqIPeIn367ftwTHbws7ybCw7/p+dLI4Lk0gLYoCC 6A2D
sJO4zfmlot/dpR3fYlp0KFb0rxk8wcyppQtl+r4/4DRpKs3aGvTsLrYvJtsR EadY
nwJqBs8+r65zaoEITuan3atUJ6U+9Rflm8p0xORYGMd6TTRQPu00HQnNkQke mUst
e91PIIRDV4V7qRy6mvO7Izrd8A==
=S029
-----END PGP SIGNATURE-----

--sHrvAb52M6C8blB9--