Outputting a fixed number of bytes from a binary file

I am using a bourne compatible shell. I want to output only a certain number
of bytes from a binary file. The cat command can be used to output the entire
file, but it does not appear to have an option to stop after a certain number
of bytes have been read:

cat -y 500 bigbinary.fil > myfile.bin

(I made up the -y option for number of bytes, it does not exist)

I could add some sort of filter, but which tool is best suited for this?

cat bigbinary.bin|foobarfilter -b 500 >myfile.bin

The cut tool works on lines, rather than on a complete binary file. I really
want to truncate the output at a set number of bytes (500 in these examples).
I do not want to pad the file, so it will be output in its entirity, if its
length is less than the specified count. The files are not text files, and
contain non-printable characters, which must not be affected in any way.

Thanks in advance.

Mark.

--
Mark Hobley
393 Quinton Road West
QUINTON
Birmingham
B32 1QE

Email: markhobley at hotpop dot donottypethisbit com

http://markhobley.yi.org/

Re: Outputting a fixed number of bytes from a binary file

Mark Hobley wrote:
> I am using a bourne compatible shell. I want to output only a certain number
> of bytes from a binary file. The cat command can be used to output the entire
> file, but it does not appear to have an option to stop after a certain number
> of bytes have been read:

The dd command is your friend. For example...

dd if=bigbinary.fil of=myfile.bin bs=500 count=1

See man dd for more options.

Janis

>
> cat -y 500 bigbinary.fil > myfile.bin
>
> (I made up the -y option for number of bytes, it does not exist)
>
> I could add some sort of filter, but which tool is best suited for this?
>
> cat bigbinary.bin|foobarfilter -b 500 >myfile.bin
>
> The cut tool works on lines, rather than on a complete binary file. I really
> want to truncate the output at a set number of bytes (500 in these examples).
> I do not want to pad the file, so it will be output in its entirity, if its
> length is less than the specified count. The files are not text files, and
> contain non-printable characters, which must not be affected in any way.
>
> Thanks in advance.
>
> Mark.
>

Re: Outputting a fixed number of bytes from a binary file

Mark Hobley wrote:
> I am using a bourne compatible shell. I want to output only a certain number
> of bytes from a binary file. The cat command can be used to output the entire
> file, but it does not appear to have an option to stop after a certain number
> of bytes have been read:
>
> cat -y 500 bigbinary.fil > myfile.bin
>
> (I made up the -y option for number of bytes, it does not exist)
>
> I could add some sort of filter, but which tool is best suited for this?
>
> cat bigbinary.bin|foobarfilter -b 500 >myfile.bin
>
> The cut tool works on lines, rather than on a complete binary file. I really
> want to truncate the output at a set number of bytes (500 in these examples).
> I do not want to pad the file, so it will be output in its entirity, if its
> length is less than the specified count. The files are not text files, and
> contain non-printable characters, which must not be affected in any way.
>
> Thanks in advance.
>
> Mark.
>

Take a look at "dd" (http://www.rt.com/man/dd.1.html).

Ed.

Re: Outputting a fixed number of bytes from a binary file

Janis Papanagnou wrote:

> dd if=bigbinary.fil of=myfile.bin bs=500 count=1

Is there any limit to the block size?

Suppose I want a count of 13383236383 bytes. Can I use the following? Or is
there a better tool for the job?

dd if=bigbinary.fil bs=13383236383 count=1

(I know I could do a division and multiplication, but I was really hoping to
avoid this, and work with a raw byte count.)

Mark.

--
Mark Hobley
393 Quinton Road West
QUINTON
Birmingham
B32 1QE

Email: markhobley at hotpop dot donottypethisbit com

http://markhobley.yi.org/

Re: Outputting a fixed number of bytes from a binary file

On 24 Sep., 09:08, markhob...@hotpop.deletethisbit.com (Mark Hobley)
wrote:
> Janis Papanagnou wrote:
> > dd if=bigbinary.fil of=myfile.bin bs=500 count=1
>
> Is there any limit to the block size?

I'd think there are memory limitations, but, frankly, I don't know.

> Suppose I want a count of 13383236383 bytes. Can I use the following? Or is
> there a better tool for the job?
>
> dd if=bigbinary.fil bs=13383236383 count=1

You could of course swap the numbers for bs and count, but I'd expect
that to become very very slow.

> (I know I could do a division and multiplication, but I was really hoping to
> avoid this, and work with a raw byte count.)

The division and remainder calculation is no big deal and can be done
even inline using $(( $size / $bufsize )) and $(( $size % $bufsize )).

In a similar case I had splitted the command in two dd commands, the
first one was a multiple of some "optimal" buffer size, the second
command used a skip= and seek= to append.

You can encapsulate the two commands in a function or separate script
if you think it's too bulky.

Janis

> Mark.
>
> --
> Mark Hobley
> 393 Quinton Road West
> QUINTON
> Birmingham
> B32 1QE
>
> Email: markhobley at hotpop dot donottypethisbit com
>
> http://markhobley.yi.org/

Re: Outputting a fixed number of bytes from a binary file

On Sep 24, 8:08 am, markhob...@hotpop.deletethisbit.com (Mark Hobley)
wrote:
> I am using a bourne compatible shell. I want to output only a certain number
> of bytes from a binary file. The cat command can be used to output the entire
> file, but it does not appear to have an option to stop after a certain number
> of bytes have been read:
>
> cat -y 500 bigbinary.fil > myfile.bin
>
> (I made up the -y option for number of bytes, it does not exist)
>
> I could add some sort of filter, but which tool is best suited for this?
>
> cat bigbinary.bin|foobarfilter -b 500 >myfile.bin
>
> The cut tool works on lines, rather than on a complete binary file. I really
> want to truncate the output at a set number of bytes (500 in these examples).
> I do not want to pad the file, so it will be output in its entirity, if its
> length is less than the specified count. The files are not text files, and
> contain non-printable characters, which must not be affected in any way.
>
> Thanks in advance.
>
> Mark.
>
> --
> Mark Hobley
> 393 Quinton Road West
> QUINTON
> Birmingham
> B32 1QE
>
> Email: markhobley at hotpop dot donottypethisbit com
>
> http://markhobley.yi.org/

if you can use Python:

#!/usr/bin/python
data=open("/path/binaryfile").read(200) #read 200 bytes
print data

Re: Outputting a fixed number of bytes from a binary file

2007-09-24, 07:08(+00), Mark Hobley:
> Janis Papanagnou wrote:
>
>> dd if=bigbinary.fil of=myfile.bin bs=500 count=1
>
> Is there any limit to the block size?
>
> Suppose I want a count of 13383236383 bytes. Can I use the following? Or is
> there a better tool for the job?
>
> dd if=bigbinary.fil bs=13383236383 count=1
>
> (I know I could do a division and multiplication, but I was really hoping to
> avoid this, and work with a raw byte count.)
[...]

That's one of the biggest problems of dd. It's really a raw
interface to read(2) and write(2).

The above would do 1 read of that big size and 1 write of the
same size, using a lot of memory and will not work if for
instance the input file is a pipe or more generally not a
regular file.

dd if=bigbinary.fil bs=1 count=13383236383

doesn't have this problem but is going to be very slow.

Some implementations of head (namely GNU head) support binary
files and a -c option, so that you can do

head -c13383236383 < bigbinary.file

Otherwise, as someone else suggested, you can use dd with
blocksizes, and write a helper function for that, something
like:

head_c() {
bytes=$1
blksize=4096
nblks=$(($1 / $blksize))
rembytes=$(($1 % $blksize))
{
dd bs="$blksize" count="$nblks" &&
dd bs="$rembytes" count=1
} 2> /dev/null
}

head_c 13383236383 < bigbinary.file

But beware! dd is not going to make more than $nblks+1 read()
calls, so may not read as many bytes as requested if the input
file is not a regular file.

$ {echo abcdef; sleep 1; echo abcdef} | dd bs=10 count=1 2> /dev/null | wc -c
7

$ {echo abcdef; sleep 1; echo abcdef} | head -c10 | wc -c
10

--
Stéphane