FAQ 4.3 Why isn"t my octal data interpreted correctly?

This message is one of several periodic postings to comp.lang.perl.misc
intended to make it easier for perl programmers to find answers to
common questions. The core of this message represents an excerpt
from the documentation provided with Perl.

------------------------------------------------------------ --------

4.3: Why isn't my octal data interpreted correctly?

Perl only understands octal and hex numbers as such when they occur as
literals in your program. Octal literals in perl must start with a
leading "0" and hexadecimal literals must start with a leading "0x". If
they are read in from somewhere and assigned, no automatic conversion
takes place. You must explicitly use oct() or hex() if you want the
values converted to decimal. oct() interprets hex ("0x350"), octal
("0350" or even without the leading "0", like "377") and binary
("0b1010") numbers, while hex() only converts hexadecimal ones, with or
without a leading "0x", like "0x255", "3A", "ff", or "deadbeef". The
inverse mapping from decimal to octal can be done with either the "%o"
or "%O" sprintf() formats.

This problem shows up most often when people try using chmod(), mkdir(),
umask(), or sysopen(), which by widespread tradition typically take
permissions in octal.

chmod(644, $file); # WRONG
chmod(0644, $file); # right

Note the mistake in the first line was specifying the decimal literal
644, rather than the intended octal literal 0644. The problem can be
seen with:

printf("%#o",644); # prints 01204

Surely you had not intended "chmod(01204, $file);" - did you? If you
want to use numeric literals as arguments to chmod() et al. then please
try to express them as octal constants, that is with a leading zero and
with the following digits restricted to the set 0..7.



------------------------------------------------------------ --------

Documents such as this have been called "Answers to Frequently
Asked Questions" or FAQ for short. They represent an important
part of the Usenet tradition. They serve to reduce the volume of
redundant traffic on a news group by providing quality answers to
questions that keep coming up.

If you are some how irritated by seeing these postings you are free
to ignore them or add the sender to your killfile. If you find
errors or other problems with these postings please send corrections
or comments to the posting email address or to the maintainers as
directed in the perlfaq manual page.

Note that the FAQ text posted by this server may have been modified
from that distributed in the stable Perl release. It may have been
edited to reflect the additions, changes and corrections provided
by respondents, reviewers, and critics to previous postings of
these FAQ. Complete text of these FAQ are available on request.

The perlfaq manual page contains the following copyright notice.

AUTHOR AND COPYRIGHT

Copyright (c) 1997-2002 Tom Christiansen and Nathan
Torkington, and other contributors as noted. All rights
reserved.

This posting is provided in the hope that it will be useful but
does not represent a commitment or contract of any kind on the part
of the contributers, authors or their agents.

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

In article , Peter J.
Holzer wrote:

> On 2007-12-16 16:59, brian d foy wrote:
> > In article , Peter J.
> > Holzer wrote:
> >> On 2007-12-15 08:03, PerlFAQ Server wrote:
> >> > 4.3: Why isn't my octal data interpreted correctly?
> >> >
> >> > Perl only understands octal and hex numbers as such when they occur
> >> > as
> >> > literals in your program. Octal literals in perl must start with a
> >> > leading 0 and hexadecimal literals must start with a leading "0x". If
> >> > they are read in from somewhere and assigned, no automatic conversion
> >> > takes place. You must explicitly use "oct()" or "hex()" if you want
> >> > the
> >> > values converted to decimal.
> >> ^^^^^^^
> >> Perl numbers aren't decimal,
>
> It might help if you read a little more than the first half sentence.
> All of your objections were answered in the next few lines.

There's no need to be a jerk. I read the entire message you posted and
clarified the issue. You said that numbers were binary. As you said in
response to my message, that is wrong. That's why I responded to your
message. Remember that many more people than the two of us read our
messages, and I'm including all of the information in my reply so that
others see the things that you already know

I still say that Perl numbers are decimal unless we say otherwise. Perl
will treat them as decimal and print them as decimal unless given
instructions not to. That's the point of the FAQ.

I'm sure that the next time we might disagree that we can do it
respectfully. :)

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

On 2007-12-17 19:55, brian d foy wrote:
> In article , Peter J.
> Holzer wrote:
>
>> On 2007-12-16 16:59, brian d foy wrote:
>> > In article , Peter J.
>> > Holzer wrote:
>> >> On 2007-12-15 08:03, PerlFAQ Server wrote:
>> >> > 4.3: Why isn't my octal data interpreted correctly?
>> >> >
>> >> > Perl only understands octal and hex numbers as such when
>> >> > they occur as literals in your program. Octal literals in
>> >> > perl must start with a leading 0 and hexadecimal literals
>> >> > must start with a leading "0x". If they are read in from
>> >> > somewhere and assigned, no automatic conversion takes place.
>> >> > You must explicitly use "oct()" or "hex()" if you want the
>> >> > values converted to decimal.
>> >> ^^^^^^^
>> >> Perl numbers aren't decimal,
>>
>> It might help if you read a little more than the first half sentence.
>> All of your objections were answered in the next few lines.
>
> There's no need to be a jerk.

Sorry. You pushed a sensitive spot there.


> I read the entire message you posted and clarified the issue.

I don't see how quoting a half-sentence out of a paragraph and then
presenting issues as new which were already addressed in the same
paragraph clarifies anything. At the very least, that's a rather sloppy
discussion style.


> You said that numbers were binary. As you said in response to my
> message, that is wrong.

I hope nobody except you read my response that way, because I certainly
didn't mean that. Perl numbers are binary, it's just rather irrelevant
for this question.


> I still say that Perl numbers are decimal unless we say otherwise.

That's easy to disprove:

% perl -l -e 'for ($i = 0; $i != 1; $i += 0.1) { print $i }'
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2
2.1
^C

Obviously not decimal.

I could accept an argument that perl numbers aren't binary because Perl
(the language) doesn't make any guarantees and they could be anything,
so there might be platform where perl (the interpreter) really uses
decimal numbers. But since this is rather hypothetical and Perl is
defined by the implementation and not a formal standard, I find even
that argument rather tenuous.


> Perl will treat them as decimal

No. Perl treats *strings* as decimal when used in a numeric context.
That is, it will convert from the decimal representation to an internal
(binary) representation. The number will than always be treated as the
binary number it is, because anything else would just result in garbage.
When a number is used in a string context, it is converted to decimal.

> and print them as decimal unless given instructions not to. That's the
> point of the FAQ.

The point of the FAQ is conversion from strings to numbers (and back) in
some bases. Base 10 is clearly privileged as just using a string in a
numeric context (or a number in a string context) invokes the
conversion, but claiming that perl numbers are decimal, or that hex()
and oct() "convert to decimal" is just wrong.

> I'm sure that the next time we might disagree that we can do it
> respectfully. :)

I consider properly addressing the points made by the previous poster as
an important part of disagreeing respectfully.

hp

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

In article , Peter J.
Holzer wrote:

> > I still say that Perl numbers are decimal unless we say otherwise.
>
> That's easy to disprove:
>
> % perl -l -e 'for ($i = 0; $i != 1; $i += 0.1) { print $i }'

I see Perl using the digits 0 to 9.

There are underlying implementation details based on the architecture,
but that's implementation. The language is not the processor or the
storage. That there is some round-off error because of the limitations
of computers doesn't change the fact that Perl is using the digits 0 to
9.

> > Perl will treat them as decimal
>
> No. Perl treats *strings* as decimal when used in a numeric context.

No. Perl treats literal numbers as decimal. That's why we have other
ways to denote bases 2, 8, 16. When you type just 100, that's 10 base
10 groups of 10 base 10, not 8 or 64 or 256. The only exception I know
is in a \nnn sequence in a double quoted string context, which is
treated as octal.

It matters too, because some things in Perl take octal numbers. Ask
chmod, mkdir, and a few other things what a literal 755 is. You'll see
Perl uses it as base 10 and people get the wrong thing.

In the language called Perl, literal numbers are base 10 unless we say
otherwise.

When you talk about binary, you're talking about computer storage.
That's something external to Perl. I'm just talking about Perl.

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

brian d foy wrote:
> In article , Peter J.
> Holzer wrote:
>
>>> I still say that Perl numbers are decimal unless we say
>>> otherwise.
>> That's easy to disprove:
>>
>> % perl -l -e 'for ($i = 0; $i != 1; $i += 0.1) { print $i }'
>
> I see Perl using the digits 0 to 9.
[snip]
> When you talk about binary, you're talking about computer storage.
> That's something external to Perl. I'm just talking about Perl.

You and Peter seem to be arguing at cross-purposes. I believe Peter was
objecting to the final words of this part of the FAQ:

>> Perl only understands octal and hex numbers as such when they occur
>> as literals in your program. Octal literals in perl must start
>> with a leading 0 and hexadecimal literals must start with a leading
>> "0x". If they are read in from somewhere and assigned, no
>> automatic conversion takes place. You must explicitly use "oct()"
>> or "hex()" if you want the values converted to decimal.

If, as I think you say, Perl doesn't care whether the internal
representation of a number in perl (or some hypothetical other
implementation) is decimal or binary, it might be clearer to end that
paragraph thus:

You must explicitly use "oct()" or "hex()" if you want the
values converted from octal or hexadecimal representations.

If we accept that Perl is somehow independant of perl: If Perl has
nothing to say about the base of $n in "my $n = 17 + 017 + 0x17;" it is
nevertheless true that the result is represented exactly and that Perl's
"print" function will convert it to a decimal representation unless we
tell it otherwise.

Just my 0.02 worth.

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

On 2007-12-19 17:11, brian d foy wrote:
> In article , Peter J.
> Holzer wrote:
>
>> > I still say that Perl numbers are decimal unless we say otherwise.
>>
>> That's easy to disprove:
>>
>> % perl -l -e 'for ($i = 0; $i != 1; $i += 0.1) { print $i }'
>
> I see Perl using the digits 0 to 9.
>
> There are underlying implementation details based on the architecture,
> but that's implementation. The language is not the processor or the
> storage. That there is some round-off error because of the limitations
> of computers doesn't change the fact that Perl is using the digits 0 to
> 9.

You are confusing Source code and run-time. I was talking about
run-time.


>> > Perl will treat them as decimal
>>
>> No. Perl treats *strings* as decimal when used in a numeric context.
>
> No. Perl treats literal numbers as decimal.

Literal numbers are completely irrelevant to the matter at hand, which
is what the hex and oct functions do. Whether a literal is written in
decimal, hex or octal doesn't matter after the compiler is done with it.
The hex and oct functions don't have anything to do with literal
numbers. They take a string argument and return a number.


> When you talk about binary, you're talking about computer storage.
> That's something external to Perl. I'm just talking about Perl.

I am talking about the behaviour of perl at run-time. Which is what the
sentence in the FAQ entry I remarked upon talked about.
You seem to be talking about perl source code, which is a completely
different matter.

hp

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

On 2007-12-19 17:57, RedGrittyBrick wrote:
> You and Peter seem to be arguing at cross-purposes. I believe Peter was
> objecting to the final words of this part of the FAQ:
>
>>> Perl only understands octal and hex numbers as such when they occur
>>> as literals in your program. Octal literals in perl must start
>>> with a leading 0 and hexadecimal literals must start with a leading
>>> "0x". If they are read in from somewhere and assigned, no
>>> automatic conversion takes place. You must explicitly use "oct()"
>>> or "hex()" if you want the values converted to decimal.

Exactly.


> If, as I think you say, Perl doesn't care whether the internal
> representation of a number in perl (or some hypothetical other
> implementation) is decimal or binary, it might be clearer to end that
> paragraph thus:
>
> You must explicitly use "oct()" or "hex()" if you want the
> values converted from octal or hexadecimal representations.

Which is quite similar to the sentence I suggested, yes.

hp

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

In article <47695b7e$0$13929$fa0fcedb@news.zen.co.uk>, RedGrittyBrick
wrote:

> brian d foy wrote:
> > In article , Peter J.
> > Holzer wrote:
> >
> >>> I still say that Perl numbers are decimal unless we say
> >>> otherwise.
> >> That's easy to disprove:
> >>
> >> % perl -l -e 'for ($i = 0; $i != 1; $i += 0.1) { print $i }'
> >
> > I see Perl using the digits 0 to 9.
> [snip]
> > When you talk about binary, you're talking about computer storage.
> > That's something external to Perl. I'm just talking about Perl.
>
> You and Peter seem to be arguing at cross-purposes. I believe Peter was
> objecting to the final words of this part of the FAQ:

That would make sense. I couldn't (an still can't) tell that from his
original message, which quoted the entire answer.

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

In article , Peter J.
Holzer wrote:

> On 2007-12-19 17:11, brian d foy wrote:
> > In article , Peter J.
> > Holzer wrote:
> >
> >> > I still say that Perl numbers are decimal unless we say otherwise.
> >>
> >> That's easy to disprove:
> >>
> >> % perl -l -e 'for ($i = 0; $i != 1; $i += 0.1) { print $i }'
> >
> > I see Perl using the digits 0 to 9.
> >
> > There are underlying implementation details based on the architecture,
> > but that's implementation. The language is not the processor or the
> > storage. That there is some round-off error because of the limitations
> > of computers doesn't change the fact that Perl is using the digits 0 to
> > 9.
>
> You are confusing Source code and run-time. I was talking about
> run-time.

I'm not confusing them at all. The run time is not the implementation.
You're not talking about run time either. You're talking about the
implementation on binary computers. The run time is just the bit that
does what the op codes say. By that time, Perl has already decided to
parse numbers. It's not a problem with phase.

I now see where the our confusion is. In your original message, you
quote the entire FAQ answer. It's much easier to establish context if
you trim it to the parts that you're responding too.

The answer does need some work, and I'll look at that.

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

On 2007-12-20 15:59, brian d foy wrote:
> In article <47695b7e$0$13929$fa0fcedb@news.zen.co.uk>, RedGrittyBrick
> wrote:
>> You and Peter seem to be arguing at cross-purposes. I believe Peter was
>> objecting to the final words of this part of the FAQ:
>
> That would make sense. I couldn't (an still can't) tell that from his
> original message, which quoted the entire answer.

For very small values of "entire":
The entire answer has 29 lines. I quoted the first five lines and part
of the sixth, which amounts to about 19 %. I could have snipped the
first sentence, but that would have removed the contrast between
literals in source-code and strings being processed at run-time, which I
considered relevant.

I also underlined the word "decimal". I pointed out that I didn't care
about the actual storage format, just that there is a difference between
a number and its representation as a string, and I suggested two
improvements. I don't see how I could have been more specific.

hp

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

On 2007-12-20 16:03, brian d foy wrote:
> In article , Peter J.
> Holzer wrote:
>> On 2007-12-19 17:11, brian d foy wrote:
>> > In article , Peter J.
>> > Holzer wrote:
>> >> > I still say that Perl numbers are decimal unless we say otherwise.
>> >>
>> >> That's easy to disprove:
>> >>
>> >> % perl -l -e 'for ($i = 0; $i != 1; $i += 0.1) { print $i }'
>> >
>> > I see Perl using the digits 0 to 9.
>> >
>> > There are underlying implementation details based on the architecture,
>> > but that's implementation. The language is not the processor or the
>> > storage. That there is some round-off error because of the limitations
>> > of computers doesn't change the fact that Perl is using the digits 0 to
>> > 9.
>>
>> You are confusing Source code and run-time. I was talking about
>> run-time.
>
> I'm not confusing them at all. The run time is not the implementation.
> You're not talking about run time either. You're talking about the
> implementation on binary computers.

The difference between "Perl" and "perl" is rather academic. Unlike
languages like C, which have a formal standard, the abstract language
"Perl" is only defined by its implementation "perl" and the assorted
documentation.

The perl documentation has little to say about numbers. Mainly it says
that scalars may be strings or numbers (or references, but we can ignore
that in this context) and that there are automatic conversions between
these two types and that an application cannot tell whether a scalar is
a string or a number (in fact, it may be both at the same time). It
doesn't define any specific format, although it does drop some hints
(for example it says that "~0 is 32 bits wide on a 32-bit platform, but
64 bits wide on a 64-bit platform", or that "printf "%.20g\n",
123456789123456789; produces 123456789123456784". An implementation with
base 10 would probably be allowed, but it certainly isn't required, and
I don't know any implementation of perl which does use base 10. You may
call that a bug, an imperfection, a deviation from the ideal Perl, but I
don't. I think this is quite intentional, and I think that a Perl
programmer should be aware that perl numbers are not decimal, despite
the automatic conversion between numbers and strings. He shouldn't
assume ANY specific representation, but he has to (some people don't
seem to be able to think in abstractions) he should get used to thinking
of Perl numbers as binary, because that's what they are on any platform
he's likely to use. Otherwise he will get bitten badly.

> The run time is just the bit that does what the op codes say. By that
> time, Perl has already decided to parse numbers. It's not a problem
> with phase.

Do you really not understand the difference between

my $x = 0377;

and

my $s = '0377';
my $x = oct($s);
?

In the first case the octal number is parsed by the compiler and
converted to some internal form. When the interpreter gets to that
statement it does't know whether the number was written in octal, hex,
or decimal in the source code. All it knows is that it has a numeric
constant.

In the second case the function oct is called with a scalar argument
which happens to be a string (I used a variable which has been assigned
a string literal in this example, but the string could have been read
from a file or a database, or be the result of some other operation).
When the interpreter reaches that statement, it calls the oct function,
which parses the string and computes a number with the equivalent value
which it then returns.

The parsing of the numbers happens at different phases of the program's
execution and it is done by different functions.


hp

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

In article , Peter J.
Holzer wrote:


> Do you really not understand the difference between
>
> my $x = 0377;
>
> and
>
> my $s = '0377';
> my $x = oct($s);
> ?

Well, if you still want to be an asshole, I guess I'll just ignore you.

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

On 2007-12-21 19:51, Peter J. Holzer wrote:
> The difference between "Perl" and "perl" is rather academic. Unlike
> languages like C, which have a formal standard, the abstract language
> "Perl" is only defined by its implementation "perl" and the assorted
> documentation.
>
> The perl documentation has little to say about numbers.
[...]
> An implementation with base 10 would probably be allowed, but it
> certainly isn't required, and I don't know any implementation of perl
> which does use base 10.

As a sort of postscriptum to this thread:

Yesterday I learned that the Power6 architecture supports decimal
floating point numbers in hardware. Assuming that these types are also
supported by IBM's C compiler, there is indeed now a reasonably common
architecture, where a perl with decimal numbers could be built.

hp

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

>>>>> "PJH" == Peter J Holzer writes:

PJH> On 2007-12-21 19:51, Peter J. Holzer wrote:
>> The difference between "Perl" and "perl" is rather academic. Unlike
>> languages like C, which have a formal standard, the abstract language
>> "Perl" is only defined by its implementation "perl" and the assorted
>> documentation.
>>
>> The perl documentation has little to say about numbers.
PJH> [...]
>> An implementation with base 10 would probably be allowed, but it
>> certainly isn't required, and I don't know any implementation of perl
>> which does use base 10.

PJH> As a sort of postscriptum to this thread:

PJH> Yesterday I learned that the Power6 architecture supports decimal
PJH> floating point numbers in hardware. Assuming that these types are also
PJH> supported by IBM's C compiler, there is indeed now a reasonably common
PJH> architecture, where a perl with decimal numbers could be built.

hah! i worked on the first power6 models porting PL/I to it. it was
meant to be a vax killer in its day (about 1985). our cpu was about 8mhz
with 8 megs of ram. took up 3 x 4 ft high cabinets including a 9 track
drive. are they still around today in any form?

i wrote a decimal math lib for that project but didn't use any decimal
instructions since it was in pure c and meant to be somewhat portable. i
don't even recall decimal instructions at all but i could be wrong or
they added them in later models. the only uses of that cpu it seems were
in telcos and for a while BSD was developed on it (tahoe was the nick
for that cpu).

and you can write fairly efficient decimal math code using binary
instructions with a technique called excess 3 bcd. i have the start of a
new one meant for p6 but is has languished for years. it does basic +/-
and the design for the rest is done. if anyone wants to play with it,
let me know. written in pure c and should port to any cpu with any word
length.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

On 2008-01-03 21:53, Uri Guttman wrote:
>>>>>> "PJH" == Peter J Holzer writes:
>
> PJH> On 2007-12-21 19:51, Peter J. Holzer wrote:
> >> The perl documentation has little to say about numbers.
> PJH> [...]
> >> An implementation with base 10 would probably be allowed, but it
> >> certainly isn't required, and I don't know any implementation of perl
> >> which does use base 10.
>
> PJH> As a sort of postscriptum to this thread:
>
> PJH> Yesterday I learned that the Power6 architecture supports decimal
> PJH> floating point numbers in hardware. Assuming that these types are also
> PJH> supported by IBM's C compiler, there is indeed now a reasonably common
> PJH> architecture, where a perl with decimal numbers could be built.
>
> hah! i worked on the first power6 models porting PL/I to it. it was
> meant to be a vax killer in its day (about 1985).

Power6 (the 6th revision of the Power architecture) was only released a
few months ago, so I doubt you worked on in in 1985. That was probably
a PC/RT (IBM's first RISC chip and IIRC the predecessor of the POWER
series). Or something completely different which just had the same name.

> our cpu was about 8mhz with 8 megs of ram. took up 3 x 4 ft high
> cabinets including a 9 track drive. are they still around today in
> any form?

The Power architecture is used in IBM's p-series servers and
workstations, some game consoles and quite a lot of embedded devices.
It was also used by Apple until 2006.


> i wrote a decimal math lib for that project but didn't use any decimal
> instructions since it was in pure c and meant to be somewhat portable. i
> don't even recall decimal instructions at all but i could be wrong or
> they added them in later models.

I think DFP is new in Power6, so it has only been available for a few
months. The IBM guy I was talking to speculated that it was added to
improve SAP performance.

hp

Re: FAQ 4.3 Why isn"t my octal data interpreted correctly?

>>>>> "PJH" == Peter J Holzer writes:

>> hah! i worked on the first power6 models porting PL/I to it. it was
>> meant to be a vax killer in its day (about 1985).

PJH> Power6 (the 6th revision of the Power architecture) was only released a
PJH> few months ago, so I doubt you worked on in in 1985. That was probably
PJH> a PC/RT (IBM's first RISC chip and IIRC the predecessor of the POWER
PJH> series). Or something completely different which just had the same name.

totally different and preceded the ppc chip as i would call the power
risc chips. but the name tahoe was used for it too.


>> our cpu was about 8mhz with 8 megs of ram. took up 3 x 4 ft high
>> cabinets including a 9 track drive. are they still around today in
>> any form?

PJH> The Power architecture is used in IBM's p-series servers and
PJH> workstations, some game consoles and quite a lot of embedded devices.
PJH> It was also used by Apple until 2006.

yeah, that is ppc by another name.

>> i wrote a decimal math lib for that project but didn't use any decimal
>> instructions since it was in pure c and meant to be somewhat portable. i
>> don't even recall decimal instructions at all but i could be wrong or
>> they added them in later models.

PJH> I think DFP is new in Power6, so it has only been available for a few
PJH> months. The IBM guy I was talking to speculated that it was added to
PJH> improve SAP performance.

hah.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org