Future versions of psqlODBC

Future versions of psqlODBC

am 20.03.2006 16:19:31 von Dave Page

Hi All,

I recently polled the list for feedback of the 07.03.0260 experimental
release of psqlODBC vs. 08.01.0200. Based on the few responses on list,
and conversations I've had with other developers off-list, it seems that
the REL-07_03_ENHANCED code branch should be used as the basis for
future versions of psqlODBC.

I therefore propose the following:

1) The current CVS code is branched into a REL-08_01_PATCHES branch to
be used for bug fixes.

2) The REL-07_03_ENHANCED branch is merged into CVS tip, whilst
maintaing the current installer and documentation etc.

3) The driver is rebranded to
PostgreSQL+/psqlodbcplus.so/psqlodbcplus.lib to differentiate it from
all previous versions.

4) The version number is bumped to 08.02.xxxx in preparation for a
future formal release.

Thoughts/comments/objections?

Regards, Dave.

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Re: Future versions of psqlODBC

am 20.03.2006 19:22:53 von Andreas

Dave Page wrote:

[...] it seems that
the REL-07_03_ENHANCED code branch should be used as the basis for
future versions of psqlODBC.



Hi Dave,

as allways when there are ways to chose from, there should be some guide
nudging the ignorant user (=me) in the "right" direction.

Where is 7.3.x actually better than 8.1.2 ?
Who would be encouraged to switch and the 07-branch anywhere near to be
considered stable for productive use?
Is it faster transfering data with office apps like Access?

An informed pointing finger was nice. ;)


---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: Future versions of psqlODBC

am 21.03.2006 09:21:20 von Dave Page

=20

> -----Original Message-----
> From: Andreas [mailto:maps.on@gmx.net]=20
> Sent: 20 March 2006 18:23
> To: Dave Page
> Cc: pgsql-odbc@postgresql.org
> Subject: Re: [ODBC] Future versions of psqlODBC
>=20
> Dave Page wrote:
>=20
> [...] it seems that
> the REL-07_03_ENHANCED code branch should be used as the basis for
> future versions of psqlODBC.
>=20
>=20
>=20
> Hi Dave,
>=20
> as allways when there are ways to chose from, there should be=20
> some guide=20
> nudging the ignorant user (=3Dme) in the "right" direction.
>=20
> Where is 7.3.x actually better than 8.1.2 ?

It includes features such as Updateable cursors and savepoint support.

> Who would be encouraged to switch and the 07-branch anywhere=20
> near to be=20
> considered stable for productive use?

Dunno yet - it hasn't had particularly widespread testing.

> Is it faster transfering data with office apps like Access?

Also unknown at this point.

> An informed pointing finger was nice. ;)

Well when the time comes we will point users in the right direction, but
at the moment the choice is from a developers perspective, which driver
provides the best basis for future versions? It appears that the
enhanced 07.03 branch is most peoples choice, so we will look at basing
future versions of psqlODBC on that. At the moment though 08.01.0200 is
the officially supported version.

Regards, Dave

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Re: Future versions of psqlODBC

am 22.03.2006 12:57:51 von Ludek Finstrle

> I therefore propose the following:
>
> 1) The current CVS code is branched into a REL-08_01_PATCHES branch to
> be used for bug fixes.

Sure.

> 2) The REL-07_03_ENHANCED branch is merged into CVS tip, whilst
> maintaing the current installer and documentation etc.

Sure.

> 3) The driver is rebranded to
> PostgreSQL+/psqlodbcplus.so/psqlodbcplus.lib to differentiate it from
> all previous versions.

I disagree. If I remember it right there was psqlodbcplus already.
Why confuse users more? Why we need the differentiation? The driver
is based on 7.3 version ...

> 4) The version number is bumped to 08.02.xxxx in preparation for a
> future formal release.

I agree.

Regards,

Luf

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Re: Future versions of psqlODBC

am 22.03.2006 13:22:05 von Dave Page

=20

> -----Original Message-----
> From: pgsql-odbc-owner@postgresql.org=20
> [mailto:pgsql-odbc-owner@postgresql.org] On Behalf Of Ludek Finstrle
> Sent: 22 March 2006 11:58
> To: Dave Page
> Cc: pgsql-odbc@postgresql.org; Ludek Finstrle;=20
> anoopk@pervasive-postgres.com; Hiroshi Inoue; Hiroshi Saito
> Subject: Re: [ODBC] Future versions of psqlODBC
>=20
> > I therefore propose the following:
> >=20
> > 1) The current CVS code is branched into a=20
> REL-08_01_PATCHES branch to
> > be used for bug fixes.
>=20
> Sure.
>=20
> > 2) The REL-07_03_ENHANCED branch is merged into CVS tip, whilst
> > maintaing the current installer and documentation etc.
>=20
> Sure.
>=20
> > 3) The driver is rebranded to
> > PostgreSQL+/psqlodbcplus.so/psqlodbcplus.lib to=20
> differentiate it from
> > all previous versions.
>=20
> I disagree. If I remember it right there was psqlodbcplus already.
> Why confuse users more? Why we need the differentiation? The driver
> is based on 7.3 version ...

Well, either way it will be different from 08.01 because there is only
one driver in the 7.3E branch, so there is no ANSI/Unicode distinction.

As for the name, yes, there was a psqlodbcplus, but I thought they never
actually released anything. On double checking that's not the case so
that isn't such a good name as you say.

We could just return to plain old psqlodbc/PostgreSQL ?

Regards, Dave.

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Re: Future versions of psqlODBC

am 22.03.2006 14:06:26 von Hiroshi Saito

From: "Ludek Finstrle"

> > 4) The version number is bumped to 08.02.xxxx in preparation for a
> > future formal release.
>
> I agree.

Yea, I want your achievement to be poured there. You are admired
by many users. Furthermore, i wishes so again.

Regards,
Hiroshi Saito



---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: Future versions of psqlODBC

am 22.03.2006 15:35:31 von Ludek Finstrle

> > > 3) The driver is rebranded to
> > > PostgreSQL+/psqlodbcplus.so/psqlodbcplus.lib to
> > > differentiate it from
> > > all previous versions.
> >
> > I disagree. If I remember it right there was psqlodbcplus already.
> > Why confuse users more? Why we need the differentiation? The driver
> > is based on 7.3 version ...
>
> Well, either way it will be different from 08.01 because there is only
> one driver in the 7.3E branch, so there is no ANSI/Unicode distinction.

I don't know that we want release only one driver. It's ok to have only
one driver?
You wrote that there were many problems pointed by users when psqlodbc
switch to Unicode driver.

> As for the name, ...
>
> We could just return to plain old psqlodbc/PostgreSQL ?

It sounds good for me.

Regards,

Luf

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: Future versions of psqlODBC

am 22.03.2006 15:39:52 von Dave Page

=20

> -----Original Message-----
> From: pgsql-odbc-owner@postgresql.org=20
> [mailto:pgsql-odbc-owner@postgresql.org] On Behalf Of Ludek Finstrle
> Sent: 22 March 2006 14:36
> To: Dave Page
> Cc: Ludek Finstrle; pgsql-odbc@postgresql.org;=20
> anoopk@pervasive-postgres.com; Hiroshi Inoue; Hiroshi Saito
> Subject: Re: [ODBC] Future versions of psqlODBC
>=20
> I don't know that we want release only one driver. It's ok to=20
> have only
> one driver?
> You wrote that there were many problems pointed by users when psqlodbc
> switch to Unicode driver.

Yes, but they didn't have Hiroshi around to fix things for them then (he
wrote the Unicode code), and none of the rest of us fully understood the
issues.

I have asked Hiroshi about the need for two drivers, and he told me
there should be no problem with the 7.3E branch with just the one
driver.

Regards Dave.

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Re: Future versions of psqlODBC

am 22.03.2006 15:48:24 von Tom Lane

"Dave Page" writes:
> We could just return to plain old psqlodbc/PostgreSQL ?

Please. This is going to be confusing enough anyway.

Do you expect to resurrect the separate ANSI/Unicode driver versions
someday? If so, please consider making the build create both driver
names right away, even if there's not any distinction at the moment.
Otherwise you'll be creating useless thrash in the packaging and
people's odbcinst.ini files, as they have to change the driver name
only to have to change it again later.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Re: Future versions of psqlODBC

am 22.03.2006 15:55:27 von Dave Page

=20

> -----Original Message-----
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]=20
> Sent: 22 March 2006 14:48
> To: Dave Page
> Cc: Ludek Finstrle; pgsql-odbc@postgresql.org
> Subject: Re: [ODBC] Future versions of psqlODBC=20
>=20
> "Dave Page" writes:
> > We could just return to plain old psqlodbc/PostgreSQL ?
>=20
> Please. This is going to be confusing enough anyway.
>=20
> Do you expect to resurrect the separate ANSI/Unicode driver versions
> someday? If so, please consider making the build create both driver
> names right away, even if there's not any distinction at the moment.
> Otherwise you'll be creating useless thrash in the packaging and
> people's odbcinst.ini files, as they have to change the driver name
> only to have to change it again later.

Hiroshi tells me we won't need two drivers and that the 07 branch should
not suffer the problems we had with 08.00 that caused us to move back to
two. My only concern is the *nix build - other than OSX Panther which
doesn't have a Unicode DM, does anyone know if having a Unicode only
driver is going to cause any problems on any particular platforms?

Regards, Dave.

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Re: Future versions of psqlODBC

am 22.03.2006 18:17:48 von Hiroshi Inoue

Dave Page wrote:

>
>
>
>
>>-----Original Message-----
>>From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
>>Sent: 22 March 2006 14:48
>>To: Dave Page
>>Cc: Ludek Finstrle; pgsql-odbc@postgresql.org
>>Subject: Re: [ODBC] Future versions of psqlODBC
>>
>>"Dave Page" writes:
>>
>>
>>>We could just return to plain old psqlodbc/PostgreSQL ?
>>>
>>>
>>Please. This is going to be confusing enough anyway.
>>
>>Do you expect to resurrect the separate ANSI/Unicode driver versions
>>someday? If so, please consider making the build create both driver
>>names right away, even if there's not any distinction at the moment.
>>Otherwise you'll be creating useless thrash in the packaging and
>>people's odbcinst.ini files, as they have to change the driver name
>>only to have to change it again later.
>>
>>
>
>Hiroshi tells me we won't need two drivers and that the 07 branch should
>not suffer the problems we had with 08.00 that caused us to move back to
>two.
>

AFAIR I didn't tell you such a thing. Though I've heard no problem from
Japanese( e.g. about the
use with MS Access) so far , I don't know if it solves the problems
e.g. about BDE or under
other languages' environment.

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: Future versions of psqlODBC

am 22.03.2006 18:30:09 von Dave Page

=20

> -----Original Message-----
> From: Hiroshi Inoue [mailto:inoue@tpf.co.jp]=20
> Sent: 22 March 2006 17:18
> To: Dave Page
> Cc: Tom Lane; Ludek Finstrle; pgsql-odbc@postgresql.org
> Subject: Re: [ODBC] Future versions of psqlODBC
>=20
>=20
> AFAIR I didn't tell you such a thing. Though I've heard no=20
> problem from=20
> Japanese( e.g. about the
> use with MS Access) so far , I don't know if it solves the problems=20
> e.g. about BDE or under
> other languages' environment.

Oh, I apologise - I was misremembering what you said:

> What do you mean by *non-latin1* ? As for the combination of=20
> Japanase and
> MS Access, it seems to work well.

Can anyone who was suffering the latin character problems with the old
Unicode driver try the 07.03 demo version please?

Regards, Dave.

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Re: Future versions of psqlODBC

am 27.03.2006 11:01:20 von Dave Page

Ahh, thanks for testing Miguel. Are you hanging around for a while and prep=
ared to run any more tests if needed?

Regards,Dave.

-----Original Message-----
From: Miguel Juan [mailto:mjuan@cibal.es]
Sent: Mon 3/27/2006 9:59 AM
To: Dave Page; Hiroshi Inoue; pgsql-odbc@postgresql.org
Subject: Re: [ODBC] Future versions of psqlODBC
=20
Hi all,

I have tested the enhanced version (7.3.2.60) with BDE, and still have the=
=20
same problems with text fields that have version 7.3.2.0. Basically, all th=
e=20
text fields doesn't appear when you make a query.

The last working version with BDE is the 7.3.0.1

Regards,

Miguel Juan


NOTE: I send you a direct mail becouse my messages never appear on the list.



----- Original Message -----=20
From: "Dave Page"
To: "Hiroshi Inoue"
Cc: "Tom Lane" ; "Ludek Finstrle" ;=20

Sent: Wednesday, March 22, 2006 7:30 PM
Subject: Re: [ODBC] Future versions of psqlODBC




> -----Original Message-----
> From: Hiroshi Inoue [mailto:inoue@tpf.co.jp]
> Sent: 22 March 2006 17:18
> To: Dave Page
> Cc: Tom Lane; Ludek Finstrle; pgsql-odbc@postgresql.org
> Subject: Re: [ODBC] Future versions of psqlODBC
>
>
> AFAIR I didn't tell you such a thing. Though I've heard no
> problem from
> Japanese( e.g. about the
> use with MS Access) so far , I don't know if it solves the problems
> e.g. about BDE or under
> other languages' environment.

Oh, I apologise - I was misremembering what you said:

> What do you mean by *non-latin1* ? As for the combination of
> Japanase and
> MS Access, it seems to work well.

Can anyone who was suffering the latin character problems with the old
Unicode driver try the 07.03 demo version please?

Regards, Dave.

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings




---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Re: Future versions of psqlODBC

am 27.03.2006 11:40:33 von Hiroshi Inoue

Miguel Juan wrote:
> Hi all,
>
> I have tested the enhanced version (7.3.2.60) with BDE, and still have
> the same problems with text fields that have version 7.3.2.0. Basically,
> all the text fields doesn't appear when you make a query.

Thanks.
Could you send me the Mylog output ?

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Re: Future versions of psqlODBC

am 27.03.2006 12:20:03 von Hiroshi Inoue

Miguel Juan wrote:
> Hello Hiroshi,

Hi Miguel,

> here you have the MyLog output. I just opened a connection, and ha a
> look for some data in a table with 2 fields (varchar, text).

Thanks but it looks like you are turning on the global option only.
Could you turn on the DSN option also ?

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

psqlODBC-Driver Test

am 27.03.2006 14:31:19 von Johann Zuschlag

Dave Page schrieb:

>
>Can anyone who was suffering the latin character problems with the old
>Unicode driver try the 07.03 demo version please?
>
>Regards, Dave.
>
>---------------------------(end of broadcast)---------------------------
>TIP 5: don't forget to increase your free space map settings
>
>
>
Hi Dave,

a short (late) Test with:

PostgreSQL 8.0X on Debian Sarge
Unicode Database
Application with psqlodbc+ 7.02.02.60 on Win XP Sp2

Operation: Reading and writing of a, A, u, U, o, O-Umlaut and sz (german
characters)
Data type: varchar(40)
Remarks: I got some odd errors when leaving my application (only once)
Protocol: 7.4+ and 6.4

Result: success

That never worked with any unicode version so far.
The result is remarkable since I can use a unicode database now.

Congratulations!

I have to repeat the test with Win ME.

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

psqlODBC-Driver Test / text fields

am 27.03.2006 14:45:40 von Johann Zuschlag

Hi Dave,

here is another one, same problem like Miguel:

PostgreSQL 8.0X on Debian Sarge
Unicode Database
Application with psqlodbc+ 7.02.02.60 on Win XP Sp2

Operation: Reading and writing of a, A, u, U, o, O-Umlaut and sz (german
characters)
Data type: text
Remarks: none
Protocol: 7.4+
Result: failed, insert works, querying doesn't work

Remark: No errors in postgres.log

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Re: psqlODBC-Driver Test / text fields

am 27.03.2006 14:50:42 von Dave Page

=20

> -----Original Message-----
> From: Johann Zuschlag [mailto:zuschlag2@online.de]=20
> Sent: 27 March 2006 13:46
> To: Dave Page
> Cc: pgsql-odbc@postgresql.org
> Subject: psqlODBC-Driver Test / text fields
>=20
> Hi Dave,
>=20
> here is another one, same problem like Miguel:

Miguel's problem is somewhat unique to BDE from what I recall -
specifically it ignores Unicode text columns which the Unicode driver
will offer by default. The same problem has been reported with Oracle
and other DBMS's on usenet.

> PostgreSQL 8.0X on Debian Sarge
> Unicode Database
> Application with psqlodbc+ 7.02.02.60 on Win XP Sp2
>=20
> Operation: Reading and writing of a, A, u, U, o, O-Umlaut and=20
> sz (german=20
> characters)
> Data type: text
> Remarks: none
> Protocol: 7.4+
> Result: failed, insert works, querying doesn't work

OK, that sounds like the old bug we were seeing. What is the difference
between those and the previous results you reported a few minutes ago?
Is it just the text/varchar difference?

Regards, Dave.

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Re: psqlODBC-Driver Test / text fields

am 27.03.2006 15:00:32 von Johann Zuschlag

Dave Page schrieb:

>
> OK, that sounds like the old bug we were seeing. What is the difference
> between those and the previous results you reported a few minutes ago?
> Is it just the text/varchar difference?
>
>
>
Yes, I forgot to test the text field. I shall set debug higher to see
what is going on.

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: psqlODBC-Driver Test / text fields

am 27.03.2006 17:50:45 von Johann Zuschlag

Dave Page schrieb:

>OK, that sounds like the old bug we were seeing. What is the difference
>between those and the previous results you reported a few minutes ago?
>Is it just the text/varchar difference?
>
>Regards, Dave.
>
> =20
>

Hi Dave,

the problem doesn't seem to be related to the text field. Please note=20
the examples below:

1. Searching for a string starting with a 't' seems to work fine:

2006-03-27 16:49:48 [2931] LOG: statement: declare "SQL_CUR0210FD50"=20
cursor with hold for SELECT t6.* FROM KUNDE t6 WHERE t6.name >=3D 't' AN=
D=20
t6.name <=3D 'tz' ORDER BY t6.name ASC, t6.kundenid ASC

2. Searching for a string starting with o-Umlaut (german character)=20
doesn't return any results (in my app.).

2006-03-27 16:50:37 [2931] LOG: statement: declare "SQL_CUR0210FD50"=20
cursor with hold for SELECT t6.* FROM KUNDE t6 WHERE t6.name >=3D 'ö=
'=20
AND t6.name <=3D 'öz' ORDER BY t6.name ASC, t6.kundenid ASC

Maybe the WHERE-statement is not parsed by the driver. But the hex=20
representation of 'ö' is 'C3B6', that is the correct UTF8 code (not=20
unicode) of o-Umlaut.

3. Furthermore I noticed that I wouldn't get anything back if the=20
searched string contains any "Umlauts". So I guess there could be a=20
problem with the result set too.

SELECT without a WHERE-statement works.

Any ideas?

A mylog ist available, as well as the tiny data set (6 lines)..

Hope that helps.

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Re: psqlODBC-Driver Test / text fields

am 27.03.2006 20:44:09 von Hiroshi Inoue

Johann Zuschlag wrote:

> Dave Page schrieb:
>
>> OK, that sounds like the old bug we were seeing. What is the differenc=
e
>> between those and the previous results you reported a few minutes ago?
>> Is it just the text/varchar difference?
>>
>> Regards, Dave.
>>
>> =20
>>
>
> Hi Dave,
>
> the problem doesn't seem to be related to the text field. Please note=20
> the examples below:
>
> 2. Searching for a string starting with o-Umlaut (german character)=20
> doesn't return any results (in my app.).
>
> 2006-03-27 16:50:37 [2931] LOG: statement: declare "SQL_CUR0210FD50"=20
> cursor with hold for SELECT t6.* FROM KUNDE t6 WHERE t6.name >=3D '=C3=
=B6'=20
> AND t6.name <=3D 'öz' ORDER BY t6.name ASC, t6.kundenid ASC
>
> Maybe the WHERE-statement is not parsed by the driver. But the hex=20
> representation of 'ö' is 'C3B6', that is the correct UTF8 code (no=
t=20
> unicode) of o-Umlaut.

Hi Johann,
Could you try the same query using psql with the client_encoding 'UTF8' ?

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Re: psqlODBC-Driver Test / text fields

am 28.03.2006 16:01:05 von Johann Zuschlag

Hiroshi Inoue schrieb:

>
>>
>> 2. Searching for a string starting with o-Umlaut (german character)=20
>> doesn't return any results (in my app.).
>>
>> 2006-03-27 16:50:37 [2931] LOG: statement: declare "SQL_CUR0210FD50"=20
>> cursor with hold for SELECT t6.* FROM KUNDE t6 WHERE t6.name >=3D '=C3=
=B6'=20
>> AND t6.name <=3D 'öz' ORDER BY t6.name ASC, t6.kundenid ASC
>>
>> Maybe the WHERE-statement is not parsed by the driver. But the hex=20
>> representation of 'ö' is 'C3B6', that is the correct UTF8 code (n=
ot=20
>> unicode) of o-Umlaut.
>
>
> Hi Johann,
> Could you try the same query using psql with the client_encoding 'UTF8'=
?
>
> regards,
> Hiroshi Inoue
>

Hi Hiroshi,

Do you mean psql on the Linux-server?

my locales:
de_DE.ISO-8859-1 (default)
de_DE.UTF-8
de_DE.UTF-8@euro
de_DE.ISO-8859-15@euro

I call psql:
- set client_encoding=3D'UTF8';
- select name from kunde;
result e.g: 'öä-test' (=3Do-Umlaut, a-Umlaut, -test)

- select name from kunde where name >=3D 'ö' and name <=3D 'öz' order=
by=20
name asc;

result: ERROR: Unicode characters greater than or equal to 0x10000 are=20
not supported

export LANG=3Dde_DE.UTF-8 doesn't change the behavior.

(select name from kunde where name >=3D 'ö' and name <=3D 'öz' =
order by=20
name asc; works of course)

regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: psqlODBC-Driver Test / text fields

am 28.03.2006 16:17:19 von Johann Zuschlag

Johann Zuschlag schrieb:

>
>
> (select name from kunde where name >=3D 'ö' and name <=3D 'öz=
' order by=20
> name asc; works of course)
>
>
Maybe that is more precise:

select name from kunde where name >=3D 'ö' and name <=3D 'öz' o=
rder by=20
name asc;

Does not give an error, but 0 result lines.

regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Re: psqlODBC-Driver Test / text fields

am 28.03.2006 16:34:31 von Hiroshi Inoue

Johann Zuschlag wrote:

> Johann Zuschlag schrieb:
>
>>
>>
>> (select name from kunde where name >=3D 'ö' and name <=3D 'ö=
z' order by=20
>> name asc; works of course)
>>
>>
> Maybe that is more precise:
>
> select name from kunde where name >=3D 'ö' and name <=3D 'öz'=
order by=20
> name asc;
>
> Does not give an error, but 0 result lines.


Thanks.
Could you issue the following 2 queries

select name from kunde where name >=3D 'ö' order by name asc;

select name from kunde where name <=3D 'öz' order by name asc;

and see the results ?

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: psqlODBC-Driver Test / text fields

am 28.03.2006 16:54:09 von Johann Zuschlag

Hiroshi Inoue schrieb:
>
> Thanks.
> Could you issue the following 2 queries
>
> select name from kunde where name >=3D 'ö' order by name asc;
>
name
--------
öä-test
öäüÃÃ
ÃÃ-test
(2 Zeilen)

> select name from kunde where name <=3D 'öz' order by name asc;
>
name
--------
Hühne
Müller
Täst
test
test-2
(5 Zeilen)


The complete "data" set:

select name from kunde order by name asc;

name
---------------
Hühne
Müller
Täst
test
test-2
öä-test
öäüÃÃ
ÃÃ-test
(7 Zeilen)

Seems, that the query is not correct my application is sending.

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Re: psqlODBC-Driver Test / text fields

am 28.03.2006 17:38:21 von Hiroshi Inoue

Johann Zuschlag wrote:

> Hiroshi Inoue schrieb:
>
>>
>> Thanks.
>> Could you issue the following 2 queries
>>
>> select name from kunde where name >=3D 'ö' order by name asc;
>>
> name
> --------
> öä-test
> öäüÃÃ
> ÃÃ-test
> (2 Zeilen)
>
>> select name from kunde where name <=3D 'öz' order by name asc;
>>
> name
> --------
> Hühne
> Müller
> Täst
> test
> test-2
> (5 Zeilen)


Hmm utf8 code of a-umlaut seems bigger than 'z' .

Well how is the result of the following(original ?) query
under default encoding ?

select name from kunde where name >=3D 'ö' and name <=3D 'öz' order=
by=20
name asc;

regards,
Hiroshi inoue

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Re: psqlODBC-Driver Test / text fields

am 28.03.2006 18:05:53 von Greg Campbell

This is a multi-part message in MIME format.
--------------010200000002000606070507
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable

When I look at a character map, for instance with Windows Start, Run, All=
Programs, Accessories, System=20
Tools, Character Map and a extended character set like Arial.
I see that all the accented characters (umlauts, graves, etc.) are higher=
that unaccented letters. I also=20
can see the UTF8 equivalents.



Hiroshi Inoue wrote:
> Johann Zuschlag wrote:
>=20
>> Hiroshi Inoue schrieb:
>>
>>>
>>> Thanks.
>>> Could you issue the following 2 queries
>>>
>>> select name from kunde where name >=3D 'ö' order by name asc;
>>>
>> name
>> --------
>> öä-test
>> öäüÃÃ
>> ÃÃ-test
>> (2 Zeilen)
>>
>>> select name from kunde where name <=3D 'öz' order by name asc;
>>>
>> name
>> --------
>> Hühne
>> Müller
>> Täst
>> test
>> test-2
>> (5 Zeilen)
>=20
>=20
>=20
> Hmm utf8 code of a-umlaut seems bigger than 'z' .
>=20
> Well how is the result of the following(original ?) query
> under default encoding ?
>=20
> select name from kunde where name >=3D 'ö' and name <=3D 'öz' orde=
r by name=20
> asc;
>=20
> regards,
> Hiroshi inoue
>=20
> ---------------------------(end of broadcast)--------------------------=
-
> TIP 6: explain analyze is your friend

--------------010200000002000606070507
Content-Type: text/x-vcard; charset=utf-8;
name="greg.campbell.vcf"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="greg.campbell.vcf"

begin:vcard
fn:Greg Campbell
n:Campbell;Greg
org:Michelin North America - US5 Lexington;ENG-ASE
email;internet:greg.campbell@us.michelin.com
title:ASE Systems Engineer
tel;work:803-951-5561/x75561
x-mozilla-html:FALSE
version:2.1
end:vcard


--------------010200000002000606070507
Content-Type: text/plain
Content-Disposition: inline
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable


---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

--------------010200000002000606070507--

Re: psqlODBC-Driver Test / text fields

am 28.03.2006 18:43:13 von Johann Zuschlag

Hiroshi Inoue schrieb:
>
> Hmm utf8 code of a-umlaut seems bigger than 'z' .
>
> Well how is the result of the following(original ?) query
> under default encoding ?
>
> select name from kunde where name >=3D 'ö' and name <=3D 'öz' orde=
r by=20
> name asc;
>
ERROR: Unicode characters greater than or equal to 0x10000 are not=20
supported

:-)

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Re: psqlODBC-Driver Test / text fields

am 28.03.2006 19:44:47 von Hiroshi Inoue

Johann Zuschlag wrote:

> Hiroshi Inoue schrieb:
>
>>
>> Hmm utf8 code of a-umlaut seems bigger than 'z' .
>>
>> Well how is the result of the following(original ?) query
>> under default encoding ?
>>
>> select name from kunde where name >=3D 'ö' and name <=3D 'öz' ord=
er by=20
>> name asc;
>>
> ERROR: Unicode characters greater than or equal to 0x10000 are not=20
> supported


With the default encoding(ISO-8859-1 ?) ?

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Re: psqlODBC-Driver Test / text fields

am 28.03.2006 23:33:30 von Hiroshi Inoue

Campbell, Greg wrote:

> When I look at a character map, for instance with Windows Start, Run,
> All Programs, Accessories, System Tools, Character Map and a extended
> character set like Arial.
> I see that all the accented characters (umlauts, graves, etc.) are
> higher that unaccented letters. I also can see the UTF8 equivalents.
>

Do you mean the behabior reported by Johann is reasonable ?
I'm not familiar with LATIN encoding.

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: Future versions of psqlODBC

am 28.03.2006 23:40:59 von Hiroshi Inoue

Miguel Juan wrote:

> Hi,
>
>> Hi Miguel,
>>
>>> here you have the MyLog output. I just opened a connection, and ha a
>>> look for some data in a table with 2 fields (varchar, text).
>>
>>
>> Thanks but it looks like you are turning on the global option only.
>> Could you turn on the DSN option also ?
>>
>
> Yes, my fault. I attached the mylog file with both options checked


Thanks.
Judging from the log you can see 4 records for both queries.
Is it right ?

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Re: psqlODBC-Driver Test / text fields

am 28.03.2006 23:53:16 von Greg Campbell

This is a multi-part message in MIME format.
--------------060905030505000203030709
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

I think, yes.
When I look at characters and numeric representations of them, SORTS and comparison operators seem to be
behaving correctly.

Hiroshi Inoue wrote:

> Campbell, Greg wrote:
>
>> When I look at a character map, for instance with Windows Start, Run,
>> All Programs, Accessories, System Tools, Character Map and a extended
>> character set like Arial.
>> I see that all the accented characters (umlauts, graves, etc.) are
>> higher that unaccented letters. I also can see the UTF8 equivalents.
>>
>
> Do you mean the behabior reported by Johann is reasonable ?
> I'm not familiar with LATIN encoding.
>
> regards,
> Hiroshi Inoue
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster

--------------060905030505000203030709
Content-Type: text/x-vcard; charset=utf-8;
name="greg.campbell.vcf"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="greg.campbell.vcf"

begin:vcard
fn:Greg Campbell
n:Campbell;Greg
org:Michelin North America - US5 Lexington;ENG-ASE
email;internet:greg.campbell@us.michelin.com
title:ASE Systems Engineer
tel;work:803-951-5561/x75561
x-mozilla-html:FALSE
version:2.1
end:vcard


--------------060905030505000203030709
Content-Type: text/plain
Content-Disposition: inline
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

--------------060905030505000203030709--

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 11:17:50 von Marc Herbert

"Campbell, Greg" writes:

> When I look at a character map, for instance with Windows Start, Run,
> All Programs, Accessories, System Tools, Character Map and a extended
> character set like Arial.
> I see that all the accented characters (umlauts, graves, etc.) are
> higher that unaccented letters. I also can see the UTF8 equivalents.
>

Not 100% sure I understood this issue, but it seems you are confusing
encoding and collation.

See strcmp() and strcoll().




---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 15:16:21 von Johann Zuschlag

Hiroshi Inoue schrieb:
> Johann Zuschlag wrote:
>
>> Hiroshi Inoue schrieb:
>>
>>>
>>> Hmm utf8 code of a-umlaut seems bigger than 'z' .
>>>
>>> Well how is the result of the following(original ?) query
>>> under default encoding ?
>>>
>>> select name from kunde where name >=3D 'ö' and name <=3D 'öz' or=
der by=20
>>> name asc;
>>>
>> ERROR: Unicode characters greater than or equal to 0x10000 are not=20
>> supported
>
>
> With the default encoding(ISO-8859-1 ?) ?
>
Oh, I misunderstood you. Do you mean the database default encoding?
IIRC ISO-8859-1 would be equivalent to LATIN1?
I generated a new database with the same data in LATIN1:

test-latin1=3D# select name from kunde order by name asc;
name
--------------
Hühne
Müller
Täst
test
test-2
öä-test
öäüÖÄÜß-test
(7 Zeilen)

test-latin1=3D# select name from kunde where name >=3D 'ö' and name <=3D=
'öz'=20
order by name asc;
name
------
(0 Zeilen)

And please note the following:

test-latin1=3D# select name from kunde where name >=3D 'ö' and name <=3D=
'ö'=20
order by name asc;
name
------
(0 Zeilen)

Hmm...

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 16:34:50 von Ludek Finstrle

> test-latin1=3D# select name from kunde order by name asc;
> name
> --------------
> Hühne
> Müller
> Täst
> test
> test-2
> öä-test
> öäüÖÄÜß-test
> (7 Zeilen)
>=20
> test-latin1=3D# select name from kunde where name >=3D 'ö' and name <=
=3D 'öz'=20
> order by name asc;
> name
> ------
> (0 Zeilen)
>=20
> And please note the following:
>=20
> test-latin1=3D# select name from kunde where name >=3D 'ö' and name <=
=3D 'ö'=20
> order by name asc;
> name
> ------
> (0 Zeilen)
>=20
> Hmm...

The last query is equivalent with:
select name from kunde where name =3D 'ö'

So I'm not surprised with the result ;-)

What is the locale for backend? Isn't this the real problem?

When you have the problem in psql client feel free to ask
in another pgsql-* mailing list. It isn't odbc related bug.
Maybe we solve your problem but maybe you get the right advice
faster in another list.

Regards,

Luf

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 16:38:21 von Dave Page

=20

> -----Original Message-----
> From: Ludek Finstrle [mailto:luf@pzkagis.cz]=20
> Sent: 29 March 2006 15:35
> To: Johann Zuschlag
> Cc: Hiroshi Inoue; Dave Page; pgsql-odbc@postgresql.org
> Subject: Re: [ODBC] psqlODBC-Driver Test / text fields
>=20
> When you have the problem in psql client feel free to ask
> in another pgsql-* mailing list. It isn't odbc related bug.
> Maybe we solve your problem but maybe you get the right advice
> faster in another list.

Hi Luf,

Johann is trying out some queries in psql to help Hiroshi Inoue track
down a psqlODBC problem (the bug that caused us to release both ANSI and
Unicode builds of the driver).

Regards, Dave

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 16:40:22 von Ludek Finstrle

> > When you have the problem in psql client feel free to ask
> > in another pgsql-* mailing list. It isn't odbc related bug.
> > Maybe we solve your problem but maybe you get the right advice
> > faster in another list.
>
> Johann is trying out some queries in psql to help Hiroshi Inoue track
> down a psqlODBC problem (the bug that caused us to release both ANSI and
> Unicode builds of the driver).

Hello,

maybe I remember it wrong but I remember that there is same problem
in psql client.

Regards,

Luf

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 16:42:54 von Dave Page

=20

> -----Original Message-----
> From: Ludek Finstrle [mailto:luf@pzkagis.cz]=20
> Sent: 29 March 2006 15:40
> To: Dave Page
> Cc: Ludek Finstrle; Johann Zuschlag; Hiroshi Inoue;=20
> pgsql-odbc@postgresql.org
> Subject: Re: [ODBC] psqlODBC-Driver Test / text fields
>=20
> > > When you have the problem in psql client feel free to ask
> > > in another pgsql-* mailing list. It isn't odbc related bug.
> > > Maybe we solve your problem but maybe you get the right advice
> > > faster in another list.
> >=20
> > Johann is trying out some queries in psql to help Hiroshi=20
> Inoue track
> > down a psqlODBC problem (the bug that caused us to release=20
> both ANSI and
> > Unicode builds of the driver).
>=20
> Hello,
>=20
> maybe I remember it wrong but I remember that there is same problem
> in psql client.

Haven't heard of it. I would think that's the sort of thing that would
get fixed pretty quickly to be honest.

Regards, Dave

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 17:12:38 von Johann Zuschlag

Ludek Finstrle schrieb:
>> The last query is equivalent with:
>> select name from kunde where name =3D 'ö'
>>
>> So I'm not surprised with the result ;-)
>>
>> What is the locale for backend? Isn't this the real problem?
>> =20
Hi Luf,

sure, that is the same, but the result is 0 lines for both. :-)

test-latin1=3D# select name from kunde where name =3D 'ö';
name
------
(0 Zeilen)

the psql driver is definitely better than ever. In my (humble) opinion=20
we don't need two drivers anymore. Thanks a lot for your work. So far=20
all my tests with UNICODE, LATIN1 and SQL-ASCII databases seem to work=20
with the new 7.02.260 driver on Win XP, (i.e. insert, update, select)

My set-up is:

PostgreSQL 8.0X on Debian Sarge
Unicode or LATIN1 or SQL-ASCII Database

locale=3Dde_DE

de_DE.ISO-8859-1 (default)
de_DE.UTF-8
de_DE.UTF-8@euro
de_DE.ISO-8859-15@euro

changing to de_DE didn't change anything, client_encoding didn't help,=20
since the problem doesn't seem to be related to the driver.

Hiroshi tracked down the problem by doing the same query with the=20
psql.-client. So you may be right that this an old problem.

regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 17:22:42 von Ludek Finstrle

> > > > When you have the problem in psql client feel free to ask
> > > > in another pgsql-* mailing list. It isn't odbc related bug.
> > > > Maybe we solve your problem but maybe you get the right advice
> > > > faster in another list.
> > >
> > > Johann is trying out some queries in psql to help Hiroshi
> > Inoue track
> > > down a psqlODBC problem (the bug that caused us to release
> > both ANSI and
> > > Unicode builds of the driver).
> >
> > Hello,
> >
> > maybe I remember it wrong but I remember that there is same problem
> > in psql client.
>
> Haven't heard of it. I would think that's the sort of thing that would
> get fixed pretty quickly to be honest.

Let's try read and it's ancestor:
http://archives.postgresql.org/pgsql-odbc/2006-03/msg00188.p hp

Regards,

Luf

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 17:34:52 von Dave Page

=20

> -----Original Message-----
> From: Ludek Finstrle [mailto:luf@pzkagis.cz]=20
> Sent: 29 March 2006 16:23
> To: Dave Page
> Cc: Ludek Finstrle; Johann Zuschlag; Hiroshi Inoue;=20
> pgsql-odbc@postgresql.org
> Subject: Re: [ODBC] psqlODBC-Driver Test / text fields
>=20
> Let's try read and it's ancestor:
> http://archives.postgresql.org/pgsql-odbc/2006-03/msg00188.p hp

I'm not sure I understand that that test is actually valid anyway. Consider=
the test query:

select name from kunde where name >=3D 'ö';

If 'ö' is 'ö', then isn't the query above mixing single and a multib=
yte encoding? Ie. It should all be single byte - e.g.

select name from kunde where name >=3D 'ö' order by name asc;

Or all multibyte (displayed byte by byte) whatever that results in:

s*e*l*e*c*t* *n*a*m*e* *f*r*o*m* *k*u*n*d*e* *w*h*e*r*e* *n*a*m*e* *>*=3D* =
*'*ö'*;*

Of course, we all know how well I grok encoding issues :-)

Regards, Dave.

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 17:35:19 von Ludek Finstrle

> >>The last query is equivalent with:
> >>select name from kunde where name =3D 'ö'
> >>
> >>So I'm not surprised with the result ;-)
> >>
> >>What is the locale for backend? Isn't this the real problem?
>=20
> sure, that is the same, but the result is 0 lines for both. :-)

It's ok result. I see no 'ö' in data you have posted. I see only
'ösomething'.

> we don't need two drivers anymore. Thanks a lot for your work. So far=20

It's not my work. I only fixed 08.01 branch since 08.01.0102.
The enahnced branch is Hiroshi (Inoue and Saito) work.

> PostgreSQL 8.0X on Debian Sarge
> Unicode or LATIN1 or SQL-ASCII Database

You tried this all?

> locale=3Dde_DE

I think it could be the problem. You have unicode data but you try
sort it in de_DE.ISO-8859-1.
Could you try change the locale for _backend_ process to de_DE.UTF-8 or
de_DE.UTF-8@euro? (Maybe also in postmaster.conf configuration file
of postgresql). I'm not sure if it isn't even initdb time related.

> de_DE.ISO-8859-1 (default)
> de_DE.UTF-8
> de_DE.UTF-8@euro
> de_DE.ISO-8859-15@euro
>=20
> changing to de_DE didn't change anything, client_encoding didn't help,=20
> since the problem doesn't seem to be related to the driver.

I don't speak about psql (client or driver) locale. I speak about
backend locale (and settings in postmaster.conf).
The sort operation is performed in backend process.

I suggest to try ask for this (with psql client problem) in pgsql-general=
..

Regards,

Luf

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Re: psqlODBC-Driver Test / text fields

am 29.03.2006 17:43:30 von Ludek Finstrle

> I'm not sure I understand that that test is actually valid anyway.

....

> Of course, we all know how well I grok encoding issues :-)

I know the encoding issue (UTF8, UNICODE, ...) at same level as you.
My good knowledge ending at single byte encoding.

I see no progress for some time so I hope someone in pgsql-general
(maybe peter_e) could help when the (or similar) problem is in psql
client too.

Regards,

Luf

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

psqlODBC-Driver Test, ok and closed

am 29.03.2006 20:16:59 von Johann Zuschlag







Hi Luv,

Hi Hiroshi,

Hi Dave,



I'm sorry, but I made a mistake during my last tests.

type="cite">
It's not my work. I only fixed 08.01 branch since 08.01.0102.
The enahnced branch is Hiroshi (Inoue and Saito) work.



Yeah, I know. But I hope your work for 8.01.0102 can be included in the
future driver

type="cite">


PostgreSQL 8.0X on Debian Sarge
Unicode or LATIN1 or SQL-ASCII Database



You tried this all?



Yes, and while doing so many initdb's I made a mistake with my locales
for the last tests.

Thanks for pointing me in the right direction.



So my first test results where indeed correct. I'm sorry for keeping
you all so busy. But I wanted to supply you with complete test results.
So finally: concerning the ANSI/UTF8 problem the driver seems to work
(at least for me :-) ). WinME test will follow later.



Only some minor issues are left:



- sometimes my application gives an (memory) error when leaving. (Will
try to trace that)

- continuously I get 'unknown configuration parameter
"max_identifier_length"

- Typ "lo" is not existing (I don't think my app is doing that)



I never got the last two with the former drivers. But I'll check the
mylog.



For further tests I switched back my Debian test system to PostgreSQL
7.4.X since my production server is still running that version.



Regards,

Johann




Re: psqlODBC-Driver Test, ok and closed

am 30.03.2006 01:40:27 von Hiroshi Inoue

Johann Zuschlag wrote:

>Hi Luv,
>Hi Hiroshi,
>Hi Dave,
>
>I'm sorry, but I made a mistake during my last tests.
>
>>It's not my work. I only fixed 08.01 branch since 08.01.0102.
>>The enahnced branch is Hiroshi (Inoue and Saito) work.
>>
>>
>>
>Yeah, I know. But I hope your work for 8.01.0102 can be included in the future
>driver
>
>>>PostgreSQL 8.0X on Debian Sarge
>>>Unicode or LATIN1 or SQL-ASCII Database
>>>
>>>
>>
>>You tried this all?
>>
>>
>>
>Yes, and while doing so many initdb's I made a mistake with my locales for the
>last tests.
>Thanks for pointing me in the right direction.
>
>So my first test results where indeed correct. I'm sorry for keeping you all so
>busy. But I wanted to supply you with complete test results. So finally:
>concerning the ANSI/UTF8 problem the driver seems to work (at least for me :-)
>). WinME test will follow later.
>
>

Thanks for your testing.
What we've had so far is *locale* support which I've never used and
we've had
no *collation* support unfortunaltely.

>Only some minor issues are left:
>- sometimes my application gives an (memory) error when leaving. (Will try to
>trace that)
>
>

I think this is fixed in the dll at
http://www.geocities.jp/inocchichichi/psqlodbc/index.html .

>- continuously I get 'unknown configuration parameter "max_identifier_length"
>
>

7.4 doesn't seems to have the max_identifier_length parameter.
This may be fixed in the abov dll also..

>- Typ "lo" is not existing (I don't think my app is doing that)
>
>

This isn't an error at all, just checking the existence of lo type when
connecting to the server.

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Unicode is not UTF-8. was :psqlODBC-Driver Test / text fields

am 30.03.2006 21:41:06 von Johann Zuschlag

Dave Page schrieb:
> If 'ö' is 'ö', then isn't the query above mixing single and a mu=
ltibyte encoding? Ie. It should all be single byte - e.g.
>
> select name from kunde where name >=3D 'ö' order by name asc;
>
> Or all multibyte (displayed byte by byte) whatever that results in:
>
> s*e*l*e*c*t* *n*a*m*e* *f*r*o*m* *k*u*n*d*e* *w*h*e*r*e* *n*a*m*e* *>*=3D=
* *'*ö'*;*
>
> Of course, we all know how well I grok encoding issues :-)
> =20
Hi Dave,

I can understand you. This encoding issues drive me also crazy some=20
times. :-)

The problem with UTF-8 is that all ASCII characters are represented by=20
one byte and all non ASCII characters, e.g. German Umlauts, are=20
represented by two bytes. That's why UTF-8 is called a "variable-length=20
multibyte encoding". In a pure Unicode world, e.g. U+xxxx with two=20
bytes, every character is represented by two bytes (fixed-length=20
multibyte encoding). So Unicode is not equal to UTF-8, even though the=20
PostgreSQL documentation is stating that.

If you like, see: http://www.utf8-chartable.de/ or some explanation at=20
http://czyborra.com/utf/

Windows XP supports ANSI, UTF-8, Unicode and Unicode Big Endian.=20
Unfortunately (or fortunately?) Windows seems to use UTF-8 for European=20
languages. Hiroshi can you explain that? I guess the Japanese edition of=20
Windows XP is using pure 2 byte Unicode.

I can't say anything about psql. But the new psqlodbc driver 7.03.26X=20
seems to handle that situation very well.

So I suppose the test was valid to a certain extend, since the=20
characters are handled in this mixed way in Win XP. I still have some=20
funny behaviour with Unicode in psql (even after setting LC_COLLATE=20
correctly :-) ).

For my production machines I will anyway use ISO-8859-1 (or=20
ISO-8859-15). Then the driver will convert all characters to single byte=20
avoiding all kind of problems.

But feel free to ask me for tests... ;-)

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text fields

am 30.03.2006 21:45:43 von Dave Page

=20

> -----Original Message-----
> From: Johann Zuschlag [mailto:zuschlag2@online.de]=20
> Sent: 30 March 2006 20:41
> To: Dave Page
> Cc: Hiroshi Inoue; pgsql-odbc@postgresql.org
> Subject: Unicode is not UTF-8. was :psqlODBC-Driver Test / text fields
>=20
> Dave Page schrieb:
> > If 'ö' is 'ö', then isn't the query above mixing single=20
> and a multibyte encoding? Ie. It should all be single byte - e.g.
> >
> > select name from kunde where name >=3D 'ö' order by name asc;
> >
> > Or all multibyte (displayed byte by byte) whatever that results in:
> >
> > s*e*l*e*c*t* *n*a*m*e* *f*r*o*m* *k*u*n*d*e* *w*h*e*r*e* *n*a*m*e*=20
> > *>*=3D* *'*ö'*;*
> >
> > Of course, we all know how well I grok encoding issues :-)
> > =20
> Hi Dave,
>=20
> I can understand you. This encoding issues drive me also=20
> crazy some times. :-)
>=20
> The problem with UTF-8 is that all ASCII characters are=20
> represented by one byte and all non ASCII characters, e.g.=20
> German Umlauts, are represented by two bytes. That's why=20
> UTF-8 is called a "variable-length multibyte encoding". In a=20
> pure Unicode world, e.g. U+xxxx with two bytes, every=20
> character is represented by two bytes (fixed-length multibyte=20
> encoding). So Unicode is not equal to UTF-8, even though the=20
> PostgreSQL documentation is stating that.
>=20
> If you like, see: http://www.utf8-chartable.de/ or some=20
> explanation at http://czyborra.com/utf/

Ahh, thanks for the explanation.

> Windows XP supports ANSI, UTF-8, Unicode and Unicode Big Endian.=20
> Unfortunately (or fortunately?) Windows seems to use UTF-8=20
> for European languages. Hiroshi can you explain that? I guess=20
> the Japanese edition of Windows XP is using pure 2 byte Unicode.

Ahh, now I do know that Windows does not fully support UTF-8. That's the ve=
ry reason why it is not supported in PostgreSQL 8.0 on Windows, and in 8.1 =
and above requires conversion routines that were added to the server by Mag=
nus Hagander to convert to UCS2(?) before doing any sorting etc.

> I can't say anything about psql. But the new psqlodbc driver=20
> 7.03.26X seems to handle that situation very well.
>=20
> So I suppose the test was valid to a certain extend, since=20
> the characters are handled in this mixed way in Win XP. I=20
> still have some funny behaviour with Unicode in psql (even=20
> after setting LC_COLLATE correctly :-) ).
>=20
> For my production machines I will anyway use ISO-8859-1 (or=20
> ISO-8859-15). Then the driver will convert all characters to=20
> single byte avoiding all kind of problems.
>=20
> But feel free to ask me for tests... ;-)

I'll need to leave that to Hiroshi - we already know we're past my knowledg=
e on the subject :-)

Regards, Dave.

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text fields

am 30.03.2006 23:35:12 von Hiroshi Inoue

Johann Zuschlag wrote:

> Dave Page schrieb:
>
>> If 'ö' is 'ö', then isn't the query above mixing single and a=20
>> multibyte encoding? Ie. It should all be single byte - e.g.
>>
>> select name from kunde where name >=3D 'ö' order by name asc;
>>
>> Or all multibyte (displayed byte by byte) whatever that results in:
>>
>> s*e*l*e*c*t* *n*a*m*e* *f*r*o*m* *k*u*n*d*e* *w*h*e*r*e* *n*a*m*e*=20
>> *>*=3D* *'*ö'*;*
>>
>> Of course, we all know how well I grok encoding issues :-)
>> =20
>
> Hi Dave,
>
> I can understand you. This encoding issues drive me also crazy some=20
> times. :-)
>
> The problem with UTF-8 is that all ASCII characters are represented by=20
> one byte and all non ASCII characters, e.g. German Umlauts, are=20
> represented by two bytes. That's why UTF-8 is called a=20
> "variable-length multibyte encoding". In a pure Unicode world, e.g.=20
> U+xxxx with two bytes, every character is represented by two bytes=20
> (fixed-length multibyte encoding). So Unicode is not equal to UTF-8,=20
> even though the PostgreSQL documentation is stating that.
>
> If you like, see: http://www.utf8-chartable.de/ or some explanation at=20
> http://czyborra.com/utf/
>
> Windows XP supports ANSI, UTF-8, Unicode and Unicode Big Endian.=20
> Unfortunately (or fortunately?) Windows seems to use UTF-8 for=20
> European languages. Hiroshi can you explain that? I guess the Japanese=20
> edition of Windows XP is using pure 2 byte Unicode.


Unicode ODBC drivers handle UCS-2 not UTF-8 even in European environemt.=20
Unfortunately PostgreSQL doesn't handle UCS-2
directly(because it could contain NULL bytes in the string), the unicode=20
driver sets the client_encoding to UTF-8 automatically and
converts from UCS-2 data to UTF-8 data which the PostgreSQL backend=20
can understands when sending queries. So what you
can see in the backend log is UTF-8. Then the backend converts from=20
UTF-8 data to the server encoding data. After all, the locale
(especially LC_COLLATE) setting you need is the one which matches the=20
backend encoding.

>
> I can't say anything about psql. But the new psqlodbc driver 7.03.26X=20
> seems to handle that situation very well.
>
> So I suppose the test was valid to a certain extend,=20


Yes thanks. I can't test the LATINxx encoding by myself.

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 30.03.2006 23:36:44 von Bart Samwel

Johann Zuschlag wrote:
> The problem with UTF-8 is that all ASCII characters are represented by
> one byte and all non ASCII characters, e.g. German Umlauts, are
> represented by two bytes. That's why UTF-8 is called a "variable-length
> multibyte encoding". In a pure Unicode world, e.g. U+xxxx with two
> bytes, every character is represented by two bytes (fixed-length
> multibyte encoding). So Unicode is not equal to UTF-8, even though the
> PostgreSQL documentation is stating that.

Well, it's actually even more complicated, because Unicode is actually a
32-bit character set. There is actually UTF8 (variable-length multibyte,
8 bits per unit), UTF16 (variable-length multibyte) and UTF32
(fixed-length multibyte). There is also UCS2 (fixed-length 16-bit),
which is limited to the 16 bits of the Basic Multilingual Plane, and
UCS4, which is functionally identical to UTF32. UTF-8 actually supports
up to 4 bytes per character, so it is more complete than the purely
16-bit UCS-2. Any of the variable-length encodings, and the 32-bit
UTF-32 and UCS-4 encodings can represent the whole of the character set.
A pure Unicode world can use any of those encodings, so it's a tradeoff.
If you want a direct relationship between the number of characters in a
string and the number of bytes taken, use a fixed-length encoding. If
you want to be able to encode everything, use a variable-length encoding
or a 32-bit encoding. If you want to use little space, use an 8-bit
encoding. That's it.

> Windows XP supports ANSI, UTF-8, Unicode and Unicode Big Endian.
> Unfortunately (or fortunately?) Windows seems to use UTF-8 for European
> languages. Hiroshi can you explain that? I guess the Japanese edition of
> Windows XP is using pure 2 byte Unicode.

In fact, the Win32 API is UTF-16 even in European languages(started out
as UCS-2 but became UTF-16 when Unicode went 32-bit :-) ), but it
provides an 8-bit compatibility interface. Don't know if te 8-bit
encoding is UTF-8 or plain 8-bit code pages though.

Reference: http://en.wikipedia.org/wiki/Unicode

Cheers,
Bart

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text fields

am 31.03.2006 11:22:55 von Marc Herbert

"Dave Page" writes:

>
>> Windows XP supports ANSI, UTF-8, Unicode and Unicode Big Endian.
>> Unfortunately (or fortunately?) Windows seems to use UTF-8
>> for European languages. Hiroshi can you explain that? I guess
>> the Japanese edition of Windows XP is using pure 2 byte Unicode.

> Ahh, now I do know that Windows does not fully support UTF-8. That's
> the very reason why it is not supported in PostgreSQL 8.0 on
> Windows, and in 8.1 and above requires conversion routines that were
> added to the server by Magnus Hagander to convert to UCS2(?) before
> doing any sorting etc.


Do you mean CP_UTF8 does not exist on some asian releases of Windows?



Thanks in advance.

Cheers,

Marc.


---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text fields

am 31.03.2006 11:28:57 von Dave Page

=20

> -----Original Message-----
> From: pgsql-odbc-owner@postgresql.org=20
> [mailto:pgsql-odbc-owner@postgresql.org] On Behalf Of Marc Herbert
> Sent: 31 March 2006 10:23
> To: pgsql-odbc@postgresql.org
> Subject: Re: [ODBC] Unicode is not UTF-8. was=20
> :psqlODBC-Driver Test / text fields
>=20
> "Dave Page" writes:
>=20
> >
> >> Windows XP supports ANSI, UTF-8, Unicode and Unicode Big Endian.=20
> >> Unfortunately (or fortunately?) Windows seems to use UTF-8 for=20
> >> European languages. Hiroshi can you explain that? I guess the=20
> >> Japanese edition of Windows XP is using pure 2 byte Unicode.
>=20
> > Ahh, now I do know that Windows does not fully support=20
> UTF-8. That's=20
> > the very reason why it is not supported in PostgreSQL 8.0=20
> on Windows,=20
> > and in 8.1 and above requires conversion routines that were=20
> added to=20
> > the server by Magnus Hagander to convert to UCS2(?) before=20
> doing any=20
> > sorting etc.
>=20
>=20
> Do you mean CP_UTF8 does not exist on some asian releases of Windows?

I believe the problem is that the code pages exist, but there are no
corresponding NLS files - see the notes here:
http://pginstaller.projects.postgresql.org/faq/FAQ_windows.h tml#2.6

I don't know that that actually affects us from an ODBC point of view
though - but regardless, the ODBC API uses UCS-2 as Hiroshi reminded us.

Regards, Dave.

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 31.03.2006 18:51:07 von Johann Zuschlag

Hiroshi Inoue schrieb:
>
> Unicode ODBC drivers handle UCS-2 not UTF-8 even in European=20
> environemt. Unfortunately PostgreSQL doesn't handle UCS-2
> directly(because it could contain NULL bytes in the string), the=20
> unicode driver sets the client_encoding to UTF-8 automatically and
> converts from UCS-2 data to UTF-8 data which the PostgreSQL backend=20
> can understands when sending queries. So what you
> can see in the backend log is UTF-8. Then the backend converts from=20
> UTF-8 data to the server encoding data. After all, the locale
> (especially LC_COLLATE) setting you need is the one which matches the=20
> backend encoding.
>
Hmm..., so Windows XP uses UCS-2 or do be more correct (like Bart=20
mentioned) UTF-16 (which is nearly the same, except for the surrogates).=20
That is converted to UTF-8, sent to the backend and then converted to=20
the proper locale and stored. I've read about the problems with the NULL=20
bytes on Unix machines.

Let's have two examples:
1.
backend-1 =3D ISO8859-1
backend-2 =3D UTF-8

'A' =3D U+0041 (does windows use big-endian?)

Win UCS-2: U+0041
ODBC UTF-8: U+41
backend-1 stores =3D 0x41
backend-2 stores =3D U+41

2.
'Ä' =3D U+00C4 (german A-Umlaut)

Win UCS-2: U+00C4
ODBC UTF-8: U+C384
backend-1 stores =3D 0xC4
backend-2 stores =3D U+C384

Did I get that right? So I have to be really careful when testing.

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 31.03.2006 18:58:57 von Johann Zuschlag

Johann Zuschlag schrieb:
> Let's have two examples:
> 1.
> backend-1 =3D ISO8859-1
> backend-2 =3D UTF-8
>
> 'A' =3D U+0041 (does windows use big-endian?)
>
> Win UCS-2: U+0041
> ODBC UTF-8: U+41
> backend-1 stores =3D 0x41
> backend-2 stores =3D U+41
>
> 2.
> 'Ä' =3D U+00C4 (german A-Umlaut)
>
> Win UCS-2: U+00C4
> ODBC UTF-8: U+C384
> backend-1 stores =3D 0xC4
> backend-2 stores =3D U+C384
>
> Did I get that right? So I have to be really careful when testing.
>
No, again wrong. Or is it more like this:

1.
a) locale =3D ISO8859-1
backend-1 =3D LATIN1

b) locale =3D UTF-8
backend-2 =3D Unicode

'A' =3D U+0041 (does windows use big-endian?)

Win UCS-2: U+0041
ODBC UTF-8: U+41
backend-1 stores =3D U+41
backend-2 stores =3D U+0041

2.
'Ä' =3D U+00C4 (german A-Umlaut)

Win UCS-2: U+00C4
ODBC UTF-8: U+C384
backend-1 stores =3D 0xC4
backend-2 stores =3D U+00C4


Did I get that right?

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 31.03.2006 20:47:05 von Marc Herbert

Johann Zuschlag writes:

> 'A' = U+0041 (does windows use big-endian?)

Argh, please do not make it even more complex than it needs to be!

Endianness is by chance an _independent_ issue. You just care about it
at the low-low level when dealing with files or network sockets, but
then it's over and you never want to hear about it anymore at a higher
level.

So U+0041 is an integer whose value is:

zero thousand zero hundred forty one

and this is always true, whatever is the byte ordering used by the
processor. You don't need to know more than this, even when converting
to UTF-8 or anything else.




---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 31.03.2006 21:02:38 von Marc Herbert

Johann Zuschlag writes:

> Hmm..., so Windows XP uses UCS-2 or do be more correct (like Bart
> mentioned) UTF-16 (which is nearly the same, except for the
> surrogates).

It's nearly the same... but that makes a huge difference.

The reason why you use fixed-character length encoding in memory is
speed. This saves you a lot of time when computing string lengths,
look for some characters (isalnum(),...), collating etc.

If don't care about all this speed then you better stay in a
variable-length encoding like UTF-8 which saves you A LOT of space,
especially with small occidental alphabets.

I think that by moving from UCS-2 to UTF-16 you lose on BOTH sides
[insert some missing benchmarks here]

And you can be sure that it brings a lot of bugs: one bug every
time some string code has been "forgotten" and not updated, still
assuming UCS-2.

Anyway those bugs are only for far-away and unknown countries out of
the BMP so who cares? :-/

So it really looks like a poor compatibility hack to me (java does it
too).





---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 31.03.2006 21:12:13 von Marc Herbert

Johann Zuschlag writes:

> I've read about the problems with the NULL bytes on Unix machines.

This problem is not related to Unix at all but to the programming
language used. Most standard C functions use the zero byte convention
as a string terminator, so it becomes a forbidden character in C.

On the other hand String objects in C++ and Java use a separate length
field, and having NULLs inside a string is a no brainer there.

The ODBC API has been designed for C and Cobol. Cobol does not forbid
zero as a character either. When browsing the ODBC spec you'll notice
it carefully caters for the two ways.


Guess which programming language is used PostgreSQL.


I suspect unicode does not care at all about this. After all unicode
is just about characters not about strings.




---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 01.04.2006 03:26:58 von Bart Samwel

Marc Herbert wrote:
> Johann Zuschlag writes:
>
>> I've read about the problems with the NULL bytes on Unix machines.
>
> This problem is not related to Unix at all but to the programming
> language used. Most standard C functions use the zero byte convention
> as a string terminator, so it becomes a forbidden character in C.
>
> On the other hand String objects in C++ and Java use a separate length
> field, and having NULLs inside a string is a no brainer there.
>
> The ODBC API has been designed for C and Cobol. Cobol does not forbid
> zero as a character either. When browsing the ODBC spec you'll notice
> it carefully caters for the two ways.
>
>
> Guess which programming language is used PostgreSQL.

C++ even introduced a special alternative character type "wchar_t" for
this, just so that people could handle both 8-bit char* and 16-bit
wchar_t* strings. In wchar_t* strings, 8-bit NULs are not a problem
because only 16-bit NULs count (and AFAIK the Unicode standard does
allows this to be interpreted as a NUL aka end-of-string). The downside
of this solution is that no application actually uses it, and everybody
is stuck with 8-bit ASCII plus a random local codepage unless special
support is added. Why didn't they just upgrade chars to 32 bits and be
done with it... :-/

Cheers,
Bart

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 01.04.2006 03:35:37 von Hiroshi Inoue

Johann Zuschlag wrote:
> Johann Zuschlag schrieb:
>
>>
> No, again wrong. Or is it more like this:
>
> 1.
> a) locale = ISO8859-1
> backend-1 = LATIN1
>
> b) locale = UTF-8
> backend-2 = Unicode

What do you mean by the Unicode and are you really setting b)
as above ?

First note that in PostgreSQL the encoding has nothing to do
with the locale setting. Though PostgreSQL manages the encoding
settings by itself, as for the locale setting it completely relies
on the OS environment. There exists an essential flaw from the first.
Anyway you can change the encoding as you like per database at
createdb time but the locale setting LC_COLLATE and LC_CTYPE are
fixed at initdb time.

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 03.04.2006 10:55:30 von Marc Herbert

Bart Samwel writes:
>
> C++ even introduced a special alternative character type "wchar_t" for
> this, just so that people could handle both 8-bit char* and 16-bit
> wchar_t* strings. In wchar_t* strings, 8-bit NULs are not a problem
> because only 16-bit NULs count (and AFAIK the Unicode standard does
> allows this to be interpreted as a NUL aka end-of-string). The
> downside of this solution is that no application actually uses it, and
> everybody is stuck with 8-bit ASCII plus a random local codepage
> unless special support is added.

wchar_t is not defined as 16-bits, but as "wide enough to hold any
character of the platform". For instance if the platform uses UCS-4,
then wchar_t is 32 bits wide.

(UTF-16 wchar_t violates this)


I don't clearly see how you want to use a 8-bit NULL to terminate a
(wider) wchar_t array... ?


> Why didn't they just upgrade chars to 32 bits and be done with
> it... :-/

Because "char" was and is still used to store multibyte /
variable-length / encoded characters.



---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 03.04.2006 11:03:40 von Bart Samwel

Marc Herbert wrote:
> Bart Samwel writes:
> wchar_t is not defined as 16-bits, but as "wide enough to hold any
> character of the platform". For instance if the platform uses UCS-4,
> then wchar_t is 32 bits wide.
>
> (UTF-16 wchar_t violates this)

Ahhh, this explains a lot. The same assumption used to be true for char
until they came up with UTF-8 char. And they couldn't just upgrade char
because too much code assumed that char was one byte. Then platforms
started to use UCS-2 wchar_t, then upgraded those to UTF-16 because they
couldn't just upgrade wchar_t because too much code assumed that wchar_t
was two bytes. Same pattern. Time to introduce wwchar_t_t. :-)

> I don't clearly see how you want to use a 8-bit NULL to terminate a
> (wider) wchar_t array... ?

This was a backreference to a situation mentioned earlier in the
discussion, where wchar_t buffers couldn't be "tunneled through" a layer
that used char*, as the wider wchar_t characters may contain NUL bytes.

Cheers,
Bart

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 03.04.2006 11:17:03 von Johann Zuschlag

Hiroshi Inoue schrieb:
> Johann Zuschlag wrote:
>> Johann Zuschlag schrieb:
>>
>>>
>> No, again wrong. Or is it more like this:
>>
>> 1.
>> a) locale = ISO8859-1
>> backend-1 = LATIN1
>>
>> b) locale = UTF-8
>> backend-2 = Unicode
>
> What do you mean by the Unicode and are you really setting b)
> as above ?
>
Oh, typo! On my system (Debian Sarge) it is in fact:

b) locale = de_DE.UTF-8
backend-2 = Unicode

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Re: Unicode is not UTF-8. was :psqlODBC-Driver Test / text

am 04.04.2006 16:54:03 von Hiroshi Inoue

Johann Zuschlag wrote:

> Hiroshi Inoue schrieb:
>
>> Johann Zuschlag wrote:
>>
>>> Johann Zuschlag schrieb:
>>>
>>>>
>>> No, again wrong. Or is it more like this:
>>>
>>> 1.
>>> a) locale = ISO8859-1
>>> backend-1 = LATIN1
>>>
>>> b) locale = UTF-8
>>> backend-2 = Unicode
>>
>>
>> What do you mean by the Unicode and are you really setting b)
>> as above ?
>>
> Oh, typo! On my system (Debian Sarge) it is in fact:
>
> b) locale = de_DE.UTF-8
> backend-2 = Unicode


What's the result of
show server_encoding
?

regards,
Hiroshi Inoue


---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Re: Unicode is not UTF-8

am 06.04.2006 19:57:22 von Johann Zuschlag

Hiroshi Inoue schrieb:
> b) locale = de_DE.UTF-8
>> backend-2 = Unicode
>
>
> What's the result of
> show server_encoding
Hi Hiroshi,

when I use an unicode database it is UNICODE,
for LATIN1 it's of course LATIN1.
(and for LATIN9 it's LATIN9)

Thank you for your explanations. Now I'm understanding what is happening
with a windows client on one end and PostgreSQL on the other end.

I am just experimenting with the three above settings (initdb with
ISO8859-15, locale ISO8859-15 only). So far it seems to work pretty
well. No problems with an application using psqlodbc driver. All German
characters including euro (LATIN9, UNICODE) seem to work. Sorting is ok.
pgAdminIII is producing the same output. psql works as well, except psql
in conjunction with unicode behaves still a little bit strange. Maybe my
locale setting is not appropriate, but de_DE.UTF-8 doesn't change it.
Will check my console later.

Unfortunately I couldn't test 7.03.262 (psqlodbc35W.dll) since the zip
file has some kind of error.

Regards,
Johann


---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Re: Unicode is not UTF-8

am 07.04.2006 02:21:59 von Hiroshi Inoue

Johann Zuschlag wrote:
> Hiroshi Inoue schrieb:
>
>> b) locale = de_DE.UTF-8
>>
>>> backend-2 = Unicode
>>
>>
>>
>> What's the result of
>> show server_encoding
>
> Hi Hiroshi,

Hi Johann,

> Unfortunately I couldn't test 7.03.262 (psqlodbc35W.dll) since the zip
> file has some kind of error.

Sorry. Fixed.

regards,
Hiroshi Inoue



---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster