run perl program when there is new file?

Hi,

I have a folder where new files will be generated when some other
programs finished working.

Now, I need to write a perl program to store some text output files
generated from the other programs into a MySQL database. i think that
i could just write a perl code to generate a sql command file and
insert text into database.

The challenge is that this perl program needs to know what files have
already been processed, while only process on NEW files.. Maybe in
the mysql database, i could create a field to record the time of the
last run, then compare that time with the timestamp of all files to
find what files are new and therefore needs to be processed. if this
is the way to go, can you please let me know how to get the timestamp
of files? Also, to retrieve the time recorded in mySQL, the idea in my
mind now is to use something like below. There might be a better way,
i believe...
the_times = `mysql -u me - p the_password; select last_time from
dummp_table`

Thanks!!

Jie

Re: run perl program when there is new file?

On Oct 1, 1:26 pm, Jie wrote:
> Hi,
>
> I have a folder where new files will be generated when some other
> programs finished working.
>
> Now, I need to write a perl program to store some text output files
> generated from the other programs into a MySQL database. i think that
> i could just write a perl code to generate a sql command file and
> insert text into database.
>
> The challenge is that this perl program needs to know what files have
> already been processed, while only process on NEW files.. Maybe in
> the mysql database, i could create a field to record the time of the
> last run, then compare that time with the timestamp of all files to
> find what files are new and therefore needs to be processed. if this
> is the way to go, can you please let me know how to get the timestamp
> of files? Also, to retrieve the time recorded in mySQL, the idea in my
> mind now is to use something like below. There might be a better way,
> i believe...
> the_times = `mysql -u me - p the_password; select last_time from
> dummp_table`
>
> Thanks!!
>
> Jie

Can you move the files once they have been processed by Perl ?

For the perl script, maybe you can add an entry to the crontab to
check every minutes if there is new files in the specified directory
and if there is new files, put the content in the MySQL database and
move the files somewhere else.

Or you can create a table of files and if the files listed in the
directory are not in the table, put it in the file list table and add
their content to the table where it should be.

If you can't play with the crontab, maybe you can add the SQL stuff at
the end of your other program.. or if you don't have the source you
can create a wrapper for this application that will run the
application and once it's done, add the output to the database

I'm pretty sure there is MANY other ways to do the job too.

--Ben

Re: run perl program when there is new file?

Quoth Jie :
>
> I have a folder where new files will be generated when some other
> programs finished working.
>
> Now, I need to write a perl program to store some text output files
> generated from the other programs into a MySQL database. i think that
> i could just write a perl code to generate a sql command file and
> insert text into database.

Better would be to use

perldoc DBI
perldoc DBD::mysql

If you don't have DBI installed, start with

perldoc CPAN

basically, you need to run

perl -MCPAN -e'install DBI'

and answer all the prompts. Defaults are usually good if you don't
understand the questions :).

> The challenge is that this perl program needs to know what files have
> already been processed, while only process on NEW files.. Maybe in
> the mysql database, i could create a field to record the time of the
> last run, then compare that time with the timestamp of all files to
> find what files are new and therefore needs to be processed. if this
> is the way to go, can you please let me know how to get the timestamp
> of files?

perldoc -f stat

and also I find

perldoc File::stat

very helpful.


> Also, to retrieve the time recorded in mySQL, the idea in my
> mind now is to use something like below. There might be a better way,
> i believe...

DBI again. The answer to almost any database-related question in Perl
involves DBI :).

Ben

Re: run perl program when there is new file?

Jie schreef:

> The challenge is that this perl program needs to know what files have
> already been processed, while only process on NEW files.

Mimic how maildir-type delivery works (new. tmp, cur).

--
Affijn, Ruud

"Gewoon is een tijger."