edit_file and edit_file_lines

hi all,

As with other releases of File::Slurp I think this list should be told
about it since this module is so easy to use and makes your Perl much
simpler and also faster.

uri

Have you ever wanted to use perl -pi inside perl? did you have the guts
to localize $^I and @ARGV to do that? now you can do that with a simple
call to edit_file or edit_file_lines in the new .018 release of
File::Slurp. Now you can modify a file in place with a simple call.

edit_file reads a whole file into $_, calls its code block argument and
writes $_ back out the file. These groups are equivalent operations:

perl -0777 -pi -e 's/foo/bar/g' filename

use File::Slurp qw( edit_file ) ;

edit_file { s/foo/bar/g } 'filename' ;

edit_file sub { s/foo/bar/g }, 'filename' ;

edit_file \&replace_foo, 'filename' ;
sub replace_foo { s/foo/bar/g }

edit_file_lines reads a whole file and puts each line into $_, calls its
code block argument and writes each $_ back out the file. These groups are
equivalent operations:

perl -pi -e '$_ = "" if /foo/' filename

use File::Slurp qw( edit_file_lines ) ;

edit_file_lines { $_ = '' if /foo/ } 'filename' ;

edit_file_lines sub { $_ = '' if /foo/ }, 'filename' ;

edit_file \&delete_foo, 'filename' ;
sub delete_foo { $_ = '' if /foo/ }

So now when someone asks for a simple way to modify a file from inside
Perl, you have an easy answer to give them.


--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

From: "Uri Guttman"
>=20
> hi all,
>=20
> As with other releases of File::Slurp I think this list should be told
> about it since this module is so easy to use and makes your Perl much
> simpler and also faster.
>=20
> uri
>=20
> Have you ever wanted to use perl -pi inside perl? did you have the =
guts
> to localize $^I and @ARGV to do that? now you can do that with a =
simple
> call to edit_file or edit_file_lines in the new .018 release of
> File::Slurp. Now you can modify a file in place with a simple call.
>=20
> edit_file reads a whole file into $_, calls its code block argument =
and
> writes $_ back out the file.=20


Hi Uri,

Congratulations for the idea of adding this helpful feature to =
File::Slurp.

It is also useful for adding a string at the beginning of the file in an =
easy way, using something like:

edit_file { $_ =3D "first line\n" . $_ } 'file_name';

Octavian


--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

>>>>> "OR" == Octavian Rasnita writes:


OR> Congratulations for the idea of adding this helpful feature to
OR> File::Slurp.

OR> It is also useful for adding a string at the beginning of the file
OR> in an easy way, using something like:

OR> edit_file { $_ = "first line\n" . $_ } 'file_name';

that is already supported by prepend_file.

prepend_file 'file_name', "first line\n" ;

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

On 15 May 2011 01:28, Uri Guttman wrote:
>
> hi all,
>
> As with other releases of File::Slurp I think this list should be told
> about it since this module is so easy to use and makes your Perl much
> simpler and also faster.
>
> uri
>
> Have you ever wanted to use perl -pi inside perl? did you have the guts
> to localize $^I and @ARGV to do that? now you can do that with a simple
> call to edit_file or edit_file_lines in the new .018 release of
> File::Slurp. Now you can modify a file in place with a simple call.
>
> edit_file reads a whole file into $_, calls its code block argument and
> writes $_ back out the file. These groups are equivalent operations:
>
> =A0 =A0 =A0 =A0perl -0777 -pi -e 's/foo/bar/g' filename
>
> =A0 =A0 =A0 =A0use File::Slurp qw( edit_file ) ;
>
> =A0 =A0 =A0 =A0edit_file { s/foo/bar/g } 'filename' ;
>
> =A0 =A0 =A0 =A0edit_file sub { s/foo/bar/g }, 'filename' ;
>
> =A0 =A0 =A0 =A0edit_file \&replace_foo, 'filename' ;
> =A0 =A0 =A0 =A0sub replace_foo { s/foo/bar/g }
>
> edit_file_lines reads a whole file and puts each line into $_, calls its
> code block argument and writes each $_ back out the file. These groups ar=
e
> equivalent operations:
>
> =A0 =A0 =A0 =A0perl -pi -e '$_ =3D "" if /foo/' filename
>
> =A0 =A0 =A0 =A0use File::Slurp qw( edit_file_lines ) ;
>
> =A0 =A0 =A0 =A0edit_file_lines { $_ =3D '' if /foo/ } 'filename' ;
>
> =A0 =A0 =A0 =A0edit_file_lines sub { $_ =3D '' if /foo/ }, 'filename' ;
>
> =A0 =A0 =A0 =A0edit_file \&delete_foo, 'filename' ;
> =A0 =A0 =A0 =A0sub delete_foo { $_ =3D '' if /foo/ }
>
> So now when someone asks for a simple way to modify a file from inside
> Perl, you have an easy answer to give them.

this is very nice, don't need to call perl in perl anymore..

perl -pi -e '$_ =3D "" if /foo/' filename

but what if the file is very large? slurping the file in to memory will
be ok?

or is there any other alternatives you are aware of? or you can dismiss?

Thanks!

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

>>>>> "JG" == Jim Green writes:

JG> this is very nice, don't need to call perl in perl anymore..

JG> perl -pi -e '$_ = "" if /foo/' filename

JG> but what if the file is very large? slurping the file in to memory will
JG> be ok?

in the slurp distro is an article i wrote for perl.com that covers that
issue.

http://search.cpan.org/~uri/File-Slurp-9999.18/extras/slurp_ article.pod

but look around at all the files you see - text, source, config, html,
control, etc. almost all are under 1mb and that is nothing in today's
ram of 1GB and up. the issue comes to fore with the large files like
logs, genetics, database dumps etc. and you should know that is the type
of file you are dealing with and not slurp them. but for maybe 99% of
the files out there, you can slurp with no ill effects. also the maximum
size you can slurp is really up to you and your ram size. i can see even
slurping 10's of MB without too much pain on a decent box.

JG> or is there any other alternatives you are aware of? or you can dismiss?

for the very large files, you need to stick to line by line or buffered
block reads. not much else you can do. for log files if you want to read
the lines from the end, File::ReadBackwards (another of my modules) is
the way to go and it is very popular.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

On Sun, May 15, 2011 at 01:31:47PM -0400, Jim Green wrote:

> this is very nice, don't need to call perl in perl anymore..
>
> perl -pi -e '$_ = "" if /foo/' filename
>
> but what if the file is very large? slurping the file in to memory will
> be ok?
>
> or is there any other alternatives you are aware of? or you can dismiss?

{
local ($^I, @ARGV) = ("", "filename");
while (<>)
{
print unless /foo/;
}
}

is the idiomatic way to do that. Well, to do:

perl -ni -e 'print unless /foo/' filename

which is the idiomatic way to do what you had written. For a more
direct translation, substitute the loop contents with

$_ = "" if /foo/;
print;

--
Paul Johnson - paul@pjcj.net
http://www.pjcj.net

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

>>>>> "PJ" == Paul Johnson writes:

PJ> On Sun, May 15, 2011 at 01:31:47PM -0400, Jim Green wrote:
>> this is very nice, don't need to call perl in perl anymore..
>>
>> perl -pi -e '$_ = "" if /foo/' filename
>>
>> but what if the file is very large? slurping the file in to memory will
>> be ok?
>>
>> or is there any other alternatives you are aware of? or you can dismiss?

PJ> {
PJ> local ($^I, @ARGV) = ("", "filename");
PJ> while (<>)
PJ> {
PJ> print unless /foo/;
PJ> }
PJ> }

PJ> is the idiomatic way to do that. Well, to do:

and also very slow and also clunky. as i said, you need to localize
those vars and also not be using <> elsewhere. it is just a bad idiom
and now it can be laid to rest. :)


PJ> perl -ni -e 'print unless /foo/' filename

about the same as my version. i wanted to show how you would do the same
logic with both -pi and edit_file_lines. my sub doesn't have a print
command so you can invert the logic which is why i used -p and not -n.

PJ> which is the idiomatic way to do what you had written. For a more
PJ> direct translation, substitute the loop contents with

PJ> $_ = "" if /foo/;
PJ> print;

and you don't need the print if you use -p.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

On May 15, 7:13=A0pm, u...@StemSystems.com ("Uri Guttman") wrote:
> >>>>> "PJ" == Paul Johnson writes:
>
> =A0 PJ> On Sun, May 15, 2011 at 01:31:47PM -0400, Jim Green wrote:
> =A0 >> this is very nice, don't need to call perl in perl anymore..
> =A0 >>
> =A0 >> perl -pi -e '$_ =3D "" if /foo/' filename
> =A0 >>
> =A0 >> but what if the file is very large? slurping the file in to memory=
will
> =A0 >> be ok?
> =A0 >>
> =A0 >> or is there any other alternatives you are aware of? or you can di=
smiss?
>
> =A0 PJ> {
> =A0 PJ> =A0 =A0 local ($^I, @ARGV) =3D ("", "filename");
> =A0 PJ> =A0 =A0 while (<>)
> =A0 PJ> =A0 =A0 {
> =A0 PJ> =A0 =A0 =A0 =A0 print unless /foo/;
> =A0 PJ> =A0 =A0 }
> =A0 PJ> }
>
> =A0 PJ> is the idiomatic way to do that. =A0Well, to do:
>
> and also very slow and also clunky. as i said, you need to localize
> those vars and also not be using <> elsewhere. it is just a bad idiom
> and now it can be laid to rest. :)
>
> =A0 PJ> =A0 perl -ni -e 'print unless /foo/' filename
>
> about the same as my version. i wanted to show how you would do the same
> logic with both -pi and edit_file_lines. my sub doesn't have a print
> command so you can invert the logic which is why i used -p and not -n.
>
> =A0 PJ> which is the idiomatic way to do what you had written. =A0For a m=
ore
> =A0 PJ> direct translation, substitute the loop contents with
>
> =A0 PJ> =A0 =A0 =A0 =A0 $_ =3D "" if /foo/;
> =A0 PJ> =A0 =A0 =A0 =A0 print;
>
> and you don't need the print if you use -p.

Is there a preferred way to append a text file to the end of gzipped
file? the api of File::Slurp append_file is nice, but doesn't work
with gzip file...

Thanks!

>
> uri
>
> --
> Uri Guttman =A0------ =A0u...@stemsystems.com =A0-------- =A0http://www.s=
ysarch.com--
> ----- =A0Perl Code Review , Architecture, Development, Training, Support =
------
> --------- =A0Gourmet Hot Cocoa Mix =A0---- =A0http://bestfriendscocoa.com=
---------


--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

>>>>> "JG" == Jim Green writes:

JG> Is there a preferred way to append a text file to the end of gzipped
JG> file? the api of File::Slurp append_file is nice, but doesn't work
JG> with gzip file...

wow, that is an odd request. there is no direct way i think with
append_file. maybe with edit_file and calling a gzip module to unzip,
append the text and gzip the result into $_ could do it. you could also
do soemthing like this (rough code, untested and i think the -c option
is zip/unzip to stdio - fix as needed)


open my $in, "gunzip -c $file|" or die ;
open my $out, "| gzip -c $file|" or die ;
write_file $out, read_file( $in ) . $append_text ) ;

the same with edit_file could be like this (again very rough code as i
don't know the gzip module api so fix this)

use Compress:gzip ; # or whatever the real module is
use File::Slurp qw( edit_file ) ;

edit_file { my $text = uncompress $_ ; $_ = compress "$_$append_text" } $file ;

hope that helps.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

> Is there a preferred way to append a text file to the end of gzipped
> file? the api of File::Slurp append_file is nice, but doesn't work
> with gzip file...

The concatenation of two gzip'ed files is a valid gzip file, which
when ungzip'd will produce a single file containing the contents of
the first followed by the contents of the second -- so just gzip the
stuff you want to append and concatenate the files together.

j.

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

>>>>> "JSA" == John SJ Anderson writes:

>> Is there a preferred way to append a text file to the end of gzipped
>> file? the api of File::Slurp append_file is nice, but doesn't work
>> with gzip file...

JSA> The concatenation of two gzip'ed files is a valid gzip file, which
JSA> when ungzip'd will produce a single file containing the contents of
JSA> the first followed by the contents of the second -- so just gzip the
JSA> stuff you want to append and concatenate the files together.

that means append_file can be used. to correct one thing i forgot in my
other post, you need to set binmode => ':raw' for that to work on
windows due to line endings. so this (again rough code) should be close
to what is needed:

append_file( $file, { binmode => ':raw' }, compress $append_text ) ;

that assumes compress is an imported gzip function. edit as needed.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

--485b3973ea897bc62d04a4c10fd4
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Jun 2, 2011 3:58 PM, "Uri Guttman" wrote:
>
> >>>>> "JG" == Jim Green writes:
>
> JG> Is there a preferred way to append a text file to the end of gzipped
> JG> file? the api of File::Slurp append_file is nice, but doesn't work
> JG> with gzip file...
>
> wow, that is an odd request. there is no direct way i think with
> append_file. maybe with edit_file and calling a gzip module to unzip,
> append the text and gzip the result into $_ could do it. you could also
> do soemthing like this (rough code, untested and i think the -c option
> is zip/unzip to stdio - fix as needed)
>
>
> open my $in, "gunzip -c $file|" or die ;
> open my $out, "| gzip -c $file|" or die ;
> write_file $out, read_file( $in ) . $append_text ) ;
>
> the same with edit_file could be like this (again very rough code as i
> don't know the gzip module api so fix this)
>
> use Compress:gzip ; # or whatever the real module is
> use File::Slurp qw( edit_file ) ;
>
> edit_file { my $text =3D uncompress $_ ; $_ =3D compress "$_$append_text"=
}
$file ;
>

That looks to be pretty much the same as:
zcat file =A6 script =A6 gzip -c -
(Untested)

.... you'd need in place editing as a part of a gzip module. IIRC gzip allow=
s
such things (or maybe that was tar).

Nice addition to the module BTW. Haven't had the need for File::Slurp (used
more specialized modules such as Text::CSV_XS or Web::Scraper) but this
feature is definitely good. If not just because it makes how to do in place
edits a no brainer.

--485b3973ea897bc62d04a4c10fd4--

Re: edit_file and edit_file_lines

>>>>> "sw" == shawn wilson writes:

sw> That looks to be pretty much the same as:
sw> zcat file =A6 script =A6 gzip -c -
sw> (Untested)

yes it is. and a perl one liner like perl -0777p le '$_ .=3D "text"' is
all you need there. the issue is if text is large and/or from a file.

sw> Nice addition to the module BTW. Haven't had the need for
sw> File::Slurp (used more specialized modules such as Text::CSV_XS or
sw> Web::Scraper) but this feature is definitely good. If not just
sw> because it makes how to do in place edits a no brainer.

file::slurp can make those modules faster as most typical modules use
either slow slurping or line by line reading when they don't need to. as
i have said, decades ago line by line was required but with today's ram
size it isn't and for most files slurping wins big. you can either pass
a list of lines directly to modules that you already slurped or do a
foreach loop over those slurped lines and you could gain a nice amount
of speed.

uri

--=20
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com =
--
----- Perl Code Review , Architecture, Development, Training, Support ----=
--
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com -------=
--

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

--002215975fe2f3dfac04a4c2641e
Content-Type: text/plain; charset=ISO-8859-1

On Jun 2, 2011 5:35 PM, "Uri Guttman" wrote:
>
> >>>>> "sw" == shawn wilson writes:
>
> sw> Nice addition to the module BTW. Haven't had the need for
> sw> File::Slurp (used more specialized modules such as Text::CSV_XS or
> sw> Web::Scraper) but this feature is definitely good. If not just
> sw> because it makes how to do in place edits a no brainer.
>
> file::slurp can make those modules faster as most typical modules use
> either slow slurping or line by line reading when they don't need to. as
> i have said, decades ago line by line was required but with today's ram
> size it isn't and for most files slurping wins big. you can either pass
> a list of lines directly to modules that you already slurped or do a
> foreach loop over those slurped lines and you could gain a nice amount
> of speed.
>

Never considered that. Thanks, good call.

--002215975fe2f3dfac04a4c2641e--

Re: edit_file and edit_file_lines

On May 15, 1:28=A0am, u...@StemSystems.com ("Uri Guttman") wrote:
> hi all,
>
> As with other releases of File::Slurp I think this list should be told
> about it since this module is so easy to use and makes your Perl much
> simpler and also faster.
>
> uri
>
> Have you ever wanted to use perl -pi inside perl? did you have the guts
> to localize $^I and @ARGV to do that? now you can do that with a simple
> call toedit_fileor edit_file_lines in the new .018 release of
> File::Slurp. Now you can modify a file in place with a simple call.
>
> edit_filereads a whole file into $_, calls its code block argument and
> writes $_ back out the file. These groups are equivalent operations:
>
> =A0 =A0 =A0 =A0 perl -0777 -pi -e 's/foo/bar/g' filename
>
> =A0 =A0 =A0 =A0 use File::Slurp qw(edit_file) ;
>
> =A0 =A0 =A0 =A0edit_file{ s/foo/bar/g } 'filename' ;
>
> =A0 =A0 =A0 =A0edit_filesub { s/foo/bar/g }, 'filename' ;
>
> =A0 =A0 =A0 =A0edit_file\&replace_foo, 'filename' ;
> =A0 =A0 =A0 =A0 sub replace_foo { s/foo/bar/g }
>
> edit_file_lines reads a whole file and puts each line into $_, calls its
> code block argument and writes each $_ back out the file. These groups ar=
e
> equivalent operations:
>
> =A0 =A0 =A0 =A0 perl -pi -e '$_ =3D "" if /foo/' filename
>
> =A0 =A0 =A0 =A0 use File::Slurp qw( edit_file_lines ) ;
>
> =A0 =A0 =A0 =A0 edit_file_lines { $_ =3D '' if /foo/ } 'filename' ;
>
> =A0 =A0 =A0 =A0 edit_file_lines sub { $_ =3D '' if /foo/ }, 'filename' ;
>
> =A0 =A0 =A0 =A0edit_file\&delete_foo, 'filename' ;
> =A0 =A0 =A0 =A0 sub delete_foo { $_ =3D '' if /foo/ }
>
> So now when someone asks for a simple way to modify a file from inside
> Perl, you have an easy answer to give them.

I tried my @texts =3D read_file( 'zcat filename.gz|' ) but it doesn't
work, could someone help with this syntax? I would use PerlIO::gz for
large files but prefer a simple call to get the content I want for
smaller files.

Thanks!
Jim.
>
> --
> Uri Guttman =A0------ =A0u...@stemsystems.com =A0-------- =A0http://www.s=
ysarch.com--
> ----- =A0Perl Code Review , Architecture, Development, Training, Support =
------
> --------- =A0Gourmet Hot Cocoa Mix =A0---- =A0http://bestfriendscocoa.com=
---------


--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/

Re: edit_file and edit_file_lines

>>>>> "JG" == Jim Green writes:

JG> I tried my @texts = read_file( 'zcat filename.gz|' ) but it doesn't
JG> work, could someone help with this syntax? I would use PerlIO::gz for
JG> large files but prefer a simple call to get the content I want for
JG> smaller files.

read_file uses sysopen internally for speed and that doesn't support all
the features of open.

i can see two ways to do what you want. one is to call open on that pipe
and pass the handle to read_file. the other is to call read_file on the
file and pass the results to an module to uncompress the
data. IO::Uncompress::Gunzip exports a gunzip sub which can do that.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
http://learn.perl.org/