Interpreting cpantester results

Hi,

I hope this isn't too stupid a question, but I haven't found
answers by looking up FAQs and googling. I have uploaded a module
to CPAN and now the test results from CPAN Tester are coming in.
There's one failure I don't understand yet but which taught me
to use more verbose output for failed tests (in the hope that they
will show up in the reports). What I couldn't figure out is what
results marked as "NA" actually mean. My module depends on a few
other modules (nothing exotic but, of course, they could be not
installed on the testers machine) and I do some tests in the
Makefile.PL to check if a C compiler can be used and if a certain
system call exists on the machine - failures should result in an
error message. Unfortunately, nothing shows up in the reports for
the "NA" cases, only some information about the test machine.
Thus I am uncertain what could have led to these cases and how
to deal with them (and if there's a way to write the Makefile.PL
so that more information is returned).

Another point: I guess the testers are quite busy and could very
well understand if they are not too keen to also get flooded
with questions from authors of modules they were kind enough to
test. Is there a good way to get at more information without
bothering the testers too much?

Regards, Jens
--
\ Jens Thoms Toerring ___ jt@toerring.de
\__________________________ http://toerring.de

Re: Interpreting cpantester results

On 08/08/2007 06:37 PM, Jens Thoms Toerring wrote:
> Hi,
>
> I hope this isn't too stupid a question, but I haven't found
> answers by looking up FAQs and googling. I have uploaded a module
> to CPAN and now the test results from CPAN Tester are coming in.
> There's one failure I don't understand yet but which taught me
> to use more verbose output for failed tests (in the hope that they
> will show up in the reports). What I couldn't figure out is what
> results marked as "NA" actually mean. My module depends on a few
> other modules (nothing exotic but, of course, they could be not
> installed on the testers machine) and I do some tests in the
> Makefile.PL to check if a C compiler can be used and if a certain
> system call exists on the machine - failures should result in an
> error message. Unfortunately, nothing shows up in the reports for
> the "NA" cases, only some information about the test machine.
> Thus I am uncertain what could have led to these cases and how
> to deal with them (and if there's a way to write the Makefile.PL
> so that more information is returned).
>
> Another point: I guess the testers are quite busy and could very
> well understand if they are not too keen to also get flooded
> with questions from authors of modules they were kind enough to
> test. Is there a good way to get at more information without
> bothering the testers too much?
>
> Regards, Jens

Probably you've required modules that some testers don't have installed,
or your module is just not compatible with certain platforms. The
relevant section from the Test::Reporter docs has been clipped:

http://clipmarks.com/clipmark/1CCE9BCC-8444-4985-8C46-616EC7 CBCFB4/

Although you didn't say what module you were talking about, my ESP tells
me that it's here: http://search.cpan.org/~jtt/File-Fcntl_Lock-0.07/

You gave my ESP a workout ;-) , but I would've preferred that the link
was in your message.

Perhaps fcntl(2) is missing on Cygwin and Freebsd.

As far as bothering the testers is concerned, I wouldn't bother them.
Make your tests more exacting and more descriptive and read Test::Reporter.

Re: Interpreting cpantester results

"Jens Thoms Toerring" wrote in message
news:5hv2dbF3mvgn7U1@mid.uni-berlin.de...
..
> What I couldn't figure out is what
> results marked as "NA" actually mean.

As I understand it, it just means that the 'perl Makefile.PL' step did not
run to completion - ie no Makefile was written.

>
> Another point: I guess the testers are quite busy and could very
> well understand if they are not too keen to also get flooded
> with questions from authors of modules they were kind enough to
> test. Is there a good way to get at more information without
> bothering the testers too much?
>

I think, in this case, I'd be sending off an email to the tester to find out
what happened.
Keep it short and polite, and you'll probably get a helpful response.

Unless you can deduce what went wrong by looking at the Makefile.PL, I don't
know of any other way of obtaining the info you seek.

Cheers,
Rob

Re: Interpreting cpantester results

On Wed, 08 Aug 2007 21:17:54 -0500 Mumia W. wrote:

> As far as bothering the testers is concerned, I wouldn't bother them.
> Make your tests more exacting and more descriptive and read Test::Reporter.

Having released my first module to CPAN some weeks ago, I faced the same
problems as the original poster. There were some test reports that failed
and all the testers I contacted were actually very friendly and gave
helpful comments.

One tester found out that his installation was broken -- something had
messed up his combinations of Test::More and Test::Builder. So contacting
the tester actually was helpful for the tester, too ;)

Re: Interpreting cpantester results

Hi,

thank you all very much for your answers. It definitely helped
me to isolate where the problem probably is (it look as if trying
to lock stdin or stdout on (Net|Open|Free)BSD doesn't work for
some reasons I don't yet understand but which would nicely ex-
plain why already the make process fails) and I hope I can avoid
to bother the testers;-)
Best regards, Jens
--
\ Jens Thoms Toerring ___ jt@toerring.de
\__________________________ http://toerring.de

Re: Interpreting cpantester results

"Jens Thoms Toerring" wrote in message
news:5i2nthF3ngimsU2@mid.uni-berlin.de...
..
..
> .... would nicely ex-plain why already the make process fails ...

If the 'make process' (as opposed to the 'perl Makefile.PL' process) fails,
then the test report should be a "FAIL" instead of an "NA" - and the 'make'
output should be available.

Seems this shortcoming in the CPAN Testers reporting of 'perl Makefile.PL'
failures is being addressed. See
http://www.perlmonks.org/index.pl?node_id=631524 .

Cheers,
Rob

Re: Interpreting cpantester results

Sisyphus wrote:

> "Jens Thoms Toerring" wrote in message
> news:5i2nthF3ngimsU2@mid.uni-berlin.de...
> .
> > .... would nicely ex-plain why already the make process fails ...

> If the 'make process' (as opposed to the 'perl Makefile.PL' process) fails,
> then the test report should be a "FAIL" instead of an "NA" - and the 'make'
> output should be available.

Sorry, I was wrong in my choice of words, with "make process" I actu-
ally meant the 'perl Makefile.PL' part of the procedure.

> Seems this shortcoming in the CPAN Testers reporting of 'perl Makefile.PL'
> failures is being addressed. See
> http://www.perlmonks.org/index.pl?node_id=631524 .

Thanks, it's good to see that this already has been addressed!

Regards, Jens
--
\ Jens Thoms Toerring ___ jt@toerring.de
\__________________________ http://toerring.de