I've a txt file that is 1GB large !!!
Problem is that my provider set the timeout to 30sec and that can't be
changed.
It is possible to avoid timeout by sql using to import fragment of the huge
file and then restart where it stopped when the php page is reloaded (in
fact when the mysql query is started again).
Bob
"Bob Bedford"
> I've a txt file that is 1GB large !!!
1GB isn't large.
> Problem is that my provider set the timeout to 30sec and that can't be
> changed.
What timeout? Talking about PHP? Using phpMyAdmin?
If the latter: forget phpMyAdmin! You just discovered why it can't be
used for serious database administration.
> It is possible to avoid timeout by sql using to import fragment of the huge
> file and then restart where it stopped when the php page is reloaded (in
> fact when the mysql query is started again).
If no other solution apply, you can split your data file and load it in
parts. Usually it's much better to use the command line tools of MySQL.
You can do so by using your SSH access to your machine or by writing a
small PHP script using the system() funktion. The execution time limit
of PHP does not apply to actions done via system().
XL
Hi Axel thanks for your reply,
>> Problem is that my provider set the timeout to 30sec and that can't be
>> changed.
>
> What timeout? Talking about PHP? Using phpMyAdmin?
PHP timeout.
> If the latter: forget phpMyAdmin! You just discovered why it can't be
> used for serious database administration.
but what to use ?
>> It is possible to avoid timeout by sql using to import fragment of the
>> huge
>> file and then restart where it stopped when the php page is reloaded (in
>> fact when the mysql query is started again).
>
> If no other solution apply, you can split your data file and load it in
> parts. Usually it's much better to use the command line tools of MySQL.
Do you have a solution to split the file and let mysql treat them as I don't
have access to mysql command line tool
> You can do so by using your SSH access to your machine or by writing a
> small PHP script using the system() funktion. The execution time limit
> of PHP does not apply to actions done via system().
system() function is also disabled, to avoid infinite loops....
How to do then ?
Bob
Bob Bedford schrieb:
> Hi Axel thanks for your reply,
>
>
> How to do then ?
You can make a deal with a better provider.
Greetings
Dirk
Bob Bedford wrote:
> Do you have a solution to split the file and let mysql treat
> them as I don't have access to mysql command line tool
What kind of file is this?
A CSV file loaded with LOAD DATA INFILE?
In that case you can simply split it by lines.
A SQL file created using mysqldump?
You can usually split these by lines, too, unless you have text data
with newline characters in them.
A SQL file created by some other tool?
Statements may span multiple lines here but it is still possible to
split by statements.
In any case: find a text editor that is able to handle large files
and use it to break the file in several smaller pieces containing
a smaller set of lines (CSV) or statements (SQL) ...
--
Hartmut Holzgraefe, Senior Support Engineer .
MySQL AB, www.mysql.com
http://www.mysql.com/support/
"Bob Bedford"
>> If no other solution apply, you can split your data file and load it in
>> parts. Usually it's much better to use the command line tools of MySQL.
> Do you have a solution to split the file and let mysql treat them as I don't
> have access to mysql command line tool
As you were talking about LOAD DATA INFILE, I expect you have CSV
or TSV data in your file. Then you can split the file at any line break.
On UNIX systems, the "split" command comes handy for such tasks:
~ $whatis split
split (1) - split a file into pieces
>> ... use your SSH access to your machine or by writing a
>> small PHP script using the system() funktion. The execution time limit
>> of PHP does not apply to actions done via system().
> system() function is also disabled, to avoid infinite loops....
>
> How to do then ?
Switch to another (less paranoid) service provider. If you want to do
serious things with your database, you need SSH access.
XL