DBD::ODBC works in perl 5.8.7 but fails in 5.8.8 and above

Hi,

Back in March this year Jonathan Gillespie reported the following
error in DBD::ODBC test suite:

Can't change param 1 maxlen (51->50) after first bind at
>t/20SqlServer.t line 180.

(see http://www.mail-archive.com/dbi-users@perl.org/msg26946.html ).

The same version of DBI and DBD::ODBC works fine in perl 5.8.7 but
fails in perl 5.8.8 and later. I've even tried 5.9.4. I never really
got to the root of the problem but it appears:

in dbdimp.c did a:

svGrow(phs->sv, 50+1)

but

SvLEN(phs->sv) returns 52!

DBD::ODBC does not expect this so the test fails. Since this is
continuing to fail in all Perl versions since 5.8.8 I really would
like to get this sorted out. I've checked through the perl changes files and
I cannot see anything that looks a possibility. Does anyone know why

If phs->sv is a SVt_NULL and you do:
SvUPGRADE(phs->sv, SVt_PVNV)
svGrow(phs->sv, 50+1)
SvLEN(phs->sv) returns 52 in 5.8.8 onwards

but the same code returned 51 in 5.8.7 and earlier?

Any over ideas that would help me track this down?

Martin
--
Martin J. Evans
Easysoft Ltd, UK
http://www.easysoft.com

Re: DBD::ODBC works in perl 5.8.7 but fails in 5.8.8 and above

On Tue, Sep 19, 2006 at 02:47:18PM +0100, Martin J. Evans wrote:
> Hi,
>
> Back in March this year Jonathan Gillespie reported the following
> error in DBD::ODBC test suite:
>
> Can't change param 1 maxlen (51->50) after first bind at
> >t/20SqlServer.t line 180.
>
> (see http://www.mail-archive.com/dbi-users@perl.org/msg26946.html ).
>
> The same version of DBI and DBD::ODBC works fine in perl 5.8.7 but
> fails in perl 5.8.8 and later. I've even tried 5.9.4. I never really
> got to the root of the problem but it appears:
>
> in dbdimp.c did a:
>
> svGrow(phs->sv, 50+1)
>
> but
>
> SvLEN(phs->sv) returns 52!
>
> DBD::ODBC does not expect this so the test fails. Since this is
> continuing to fail in all Perl versions since 5.8.8 I really would
> like to get this sorted out. I've checked through the perl changes files and
> I cannot see anything that looks a possibility. Does anyone know why
>
> If phs->sv is a SVt_NULL and you do:
> SvUPGRADE(phs->sv, SVt_PVNV)
> svGrow(phs->sv, 50+1)

svGrow should really be SvGROW. In fact there's no svGrow in perl
[...later...] or in DBD::ODBC. I'll assume you meant SvGROW and your
shift key got out of step.

> SvLEN(phs->sv) returns 52 in 5.8.8 onwards

SvLEN is the size of the buffer allocated to the string, not the
length of the current string in the buffer (which is SvCUR).

> but the same code returned 51 in 5.8.7 and earlier?

I don't have 5.8.8 handy and no time to grab it now (satellite link
down so I only have slow ISDN at the moment).

Perhaps SvLEN(phs->sv) was 52 before SvGROW was called?

Tim.

Re: DBD::ODBC works in perl 5.8.7 but fails in 5.8.8 and above

On 19-Sep-2006 Steve Hay wrote:
> Martin J. Evans wrote:
>> On 19-Sep-2006 Tim Bunce wrote:
>>> On Tue, Sep 19, 2006 at 02:47:18PM +0100, Martin J. Evans wrote:
>>>> I never really
>>>> got to the root of the problem but it appears:
>>>>
>>>> in dbdimp.c did a:
>>>>
>>>> svGrow(phs->sv, 50+1)
>>>>
>>>> but
>>>>
>>>> SvLEN(phs->sv) returns 52!
>>>>
>>>> DBD::ODBC does not expect this so the test fails. Since this is
>>>> continuing to fail in all Perl versions since 5.8.8 I really would
>>>> like to get this sorted out. I've checked through the perl changes files
>>>> and
>>>> I cannot see anything that looks a possibility. Does anyone know why
>>>>
>>>> If phs->sv is a SVt_NULL and you do:
>>>> SvUPGRADE(phs->sv, SVt_PVNV)
>>>> svGrow(phs->sv, 50+1)
>>> svGrow should really be SvGROW. In fact there's no svGrow in perl
>>> [...later...] or in DBD::ODBC. I'll assume you meant SvGROW and your
>>> shift key got out of step.
>>
>> Yes I did.
>>
>>>> SvLEN(phs->sv) returns 52 in 5.8.8 onwards
>
> I've not been following this thread at all, but seeing the above made
> this spring to mind:
>
> http://public.activestate.com/cgi-bin/perlbrowse?show_patch= Show+Patch&patch_n
> um=24665
>
> Perl has always been at liberty to allocate more memory than was
> requested if it thinks that would be a good idea, and that is exactly
> what it does as of the above patch (#24665) in order to reduce the
> number of realloc()'s that might be required.
>
> That patch was integrated into 5.8.8.

Thanks Steve, this sounds like it would cause the affect I'm seeing and was
introduced in the right perl version too. With this information I should be
able to work on a fix to DBD::ODBC.

Thanks again.

Martin
--
Martin J. Evans
Easysoft Ltd, UK
http://www.easysoft.com

Re: DBD::ODBC works in perl 5.8.7 but fails in 5.8.8 and above

--=-QVZP27/g4eQ5wC1d1+fR
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

Attached is a patch which fixes the problem in t/20SqlServer.t in
DBD::ODBC 1.13 (patch actually against latest subversion) which causes:

Can't change param 1 maxlen (51->50) after first bind at
>t/20SqlServer.t line 180.

As Steve writes below, SvGROW was changed to allocate up to the
next long boundary in perl 5.8.8 and this breaks DBD::ODBC because it expects
SvLEN to return what it passes to SvGROW (thanks for remembering
this change to Perl Steve).

The actual patch is trivial - instead of comparing exactly with what
SvLEN returns it just makes sure the length passed to bind param is less
than the buffer we've already allocated (minus a 1 for trailing NUL).

The patch also fixes some "/*" in comments in dbdimp.h.

I've included Jonathan in the CC list as he reported the problem originally.

I'm sorry I didn't push this more at the time but since trying DBD::ODBC
with SQL Server 2005 there are many more test cases in 20SqlServer.t which
would fail if we could get past this issue so it sort of galvanised me
into pursuing this again. For anyone else using SQL Server 2005,
I think you can safely ignore the errors in 20SqlServer.t
(with this patch) so long as you don't use prints in procedures
- but I will try and follow this up later.

Jeff Urlwin (if you are reading), I hope you can include this
in the next DBD::ODBC as we are getting a lot of people asking about
this failure in the tests. Also, I mailed you earlier in the year
about a problem (with a fix) with obtaining ParamValues (I think)
but I see that has not been committed to DBD::ODBC in subversion
yet either. If you want the details again - let me know and I'll
dig them out.

Martin
--
Martin J. Evans
Easysoft Limited
http://www.easysoft.com

On Tue, 2006-09-19 at 17:23 +0100, Martin J. Evans wrote:
> On 19-Sep-2006 Steve Hay wrote:
> > Martin J. Evans wrote:
> >> On 19-Sep-2006 Tim Bunce wrote:
> >>> On Tue, Sep 19, 2006 at 02:47:18PM +0100, Martin J. Evans wrote:
> >>>> I never really
> >>>> got to the root of the problem but it appears:
> >>>>
> >>>> in dbdimp.c did a:
> >>>>
> >>>> svGrow(phs->sv, 50+1)
> >>>>
> >>>> but
> >>>>
> >>>> SvLEN(phs->sv) returns 52!
> >>>>
> >>>> DBD::ODBC does not expect this so the test fails. Since this is
> >>>> continuing to fail in all Perl versions since 5.8.8 I really would
> >>>> like to get this sorted out. I've checked through the perl changes files
> >>>> and
> >>>> I cannot see anything that looks a possibility. Does anyone know why
> >>>>
> >>>> If phs->sv is a SVt_NULL and you do:
> >>>> SvUPGRADE(phs->sv, SVt_PVNV)
> >>>> svGrow(phs->sv, 50+1)
> >>> svGrow should really be SvGROW. In fact there's no svGrow in perl
> >>> [...later...] or in DBD::ODBC. I'll assume you meant SvGROW and your
> >>> shift key got out of step.
> >>
> >> Yes I did.
> >>
> >>>> SvLEN(phs->sv) returns 52 in 5.8.8 onwards
> >
> > I've not been following this thread at all, but seeing the above made
> > this spring to mind:
> >
> > http://public.activestate.com/cgi-bin/perlbrowse?show_patch= Show+Patch&patch_n
> > um=24665
> >
> > Perl has always been at liberty to allocate more memory than was
> > requested if it thinks that would be a good idea, and that is exactly
> > what it does as of the above patch (#24665) in order to reduce the
> > number of realloc()'s that might be required.
> >
> > That patch was integrated into 5.8.8.
>
> Thanks Steve, this sounds like it would cause the affect I'm seeing and was
> introduced in the right perl version too. With this information I should be
> able to work on a fix to DBD::ODBC.
>
> Thanks again.
>
> Martin
> --
> Martin J. Evans
> Easysoft Ltd, UK
> http://www.easysoft.com
>
>


--=-QVZP27/g4eQ5wC1d1+fR
Content-Disposition: attachment; filename=dbd_odbc_patch
Content-Type: text/x-patch; name=dbd_odbc_patch; charset=UTF-8
Content-Transfer-Encoding: 7bit

Index: dbdimp.c
============================================================ =======
--- dbdimp.c (revision 7847)
+++ dbdimp.c (working copy)
@@ -2928,7 +2928,7 @@
croak("Can't rebind or change param %s in/out mode after first bind (%d => %d)",
phs->name, phs->is_inout, is_inout);
}
- else if (maxlen && maxlen != phs->maxlen) {
+ else if (maxlen && maxlen > phs->maxlen) {
croak("Can't change param %s maxlen (%ld->%ld) after first bind",
phs->name, phs->maxlen, maxlen);
}
Index: dbdimp.h
============================================================ =======
--- dbdimp.h (revision 7847)
+++ dbdimp.h (working copy)
@@ -37,8 +37,8 @@
int odbc_sqlmoreresults_supported; /* flag to see if SQLMoreResults is supported */
int odbc_defer_binding; /* flag to work around SQLServer bug and defer binding until */
/* last possible moment */
- int odbc_force_rebind; /* force rebinding the output columns after each execute to
- /* resolve some issues where certain stored procs can return
+ int odbc_force_rebind; /* force rebinding the output columns after each execute to */
+ /* resolve some issues where certain stored procs can return */
/* multiple result sets */
int odbc_query_timeout;
int odbc_async_exec; /* flag to set asynchronous execution */
@@ -86,8 +86,8 @@
int odbc_ignore_named_placeholders; /* flag to ignore named parameters */
int odbc_default_bind_type; /* flag to set default binding type (experimental) */
int odbc_exec_direct; /* flag for executing SQLExecDirect instead of SQLPrepare and SQLExecute. Magic happens at SQLExecute() */
- int odbc_force_rebind; /* force rebinding the output columns after each execute to
- /* resolve some issues where certain stored procs can return
+ int odbc_force_rebind; /* force rebinding the output columns after each execute to */
+ /* resolve some issues where certain stored procs can return */
/* multiple result sets */
int odbc_query_timeout;
};

--=-QVZP27/g4eQ5wC1d1+fR--