MySQL Parallel Inserts

--00163630ed43322f0c049755a3ba
Content-Type: text/plain; charset=ISO-8859-1

Greetings everyone.

I am in a situation where I need to do parallel inserts into MySQL database
from inside my Perl program. Basically, I have several million records to
insert into the database, and hence I would rather do them in parallel than
doing them one at a time. I looked around but did not find any information
on doing this. Does MySQL not support parallel reads/writes?

Thanks in advance.

Andy

--00163630ed43322f0c049755a3ba--

Re: MySQL Parallel Inserts

On 12/13/2010 21:32, Andy wrote:
> Greetings everyone.
>
> I am in a situation where I need to do parallel inserts into MySQL database
> from inside my Perl program. Basically, I have several million records to
> insert into the database, and hence I would rather do them in parallel than
> doing them one at a time. I looked around but did not find any information
> on doing this. Does MySQL not support parallel reads/writes?
>
> Thanks in advance.
>
> Andy
>

Actually, you would be better off using the LOAD DATA INFILE... command
for your bulk loading needs. The more contention you create for the same
data and the more times you force an index rebuild, the slower it will
move. Single-threaded, bulk loading is what works best for MySQL.

Some light reading to help you along:
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
http://dev.mysql.com/doc/refman/5.1/en/non-select-optimizati on.html
http://dev.mysql.com/doc/refman/5.1/en/csv-storage-engine.ht ml

--
Shawn Green
MySQL Principal Technical Support Engineer
Oracle USA, Inc.
Office: Blountville, TN

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: MySQL Parallel Inserts

--001636697571e9f2db04975e953a
Content-Type: text/plain; charset=ISO-8859-1

Hi Andy,

I agree, prefer LOAD DATA INFILE... command for bulk loading. It is 30%
faster than normal inserts.

Krishna

On Tue, Dec 14, 2010 at 8:02 AM, Andy wrote:

> Greetings everyone.
>
> I am in a situation where I need to do parallel inserts into MySQL database
> from inside my Perl program. Basically, I have several million records to
> insert into the database, and hence I would rather do them in parallel than
> doing them one at a time. I looked around but did not find any information
> on doing this. Does MySQL not support parallel reads/writes?
>
> Thanks in advance.
>
> Andy
>

--001636697571e9f2db04975e953a--

Re: MySQL Parallel Inserts

--001636284baa23b1e304978dd3d8
Content-Type: text/plain; charset=ISO-8859-1

Shawn/Krishna,

Thank you. I will try this.

-Andy


On Tue, Dec 14, 2010 at 8:13 AM, Krishna Chandra Prajapati <
prajapatikc@gmail.com> wrote:

> Hi Andy,
>
> I agree, prefer LOAD DATA INFILE... command for bulk loading. It is 30%
> faster than normal inserts.
>
> Krishna
>
>
> On Tue, Dec 14, 2010 at 8:02 AM, Andy wrote:
>
>> Greetings everyone.
>>
>> I am in a situation where I need to do parallel inserts into MySQL
>> database
>> from inside my Perl program. Basically, I have several million records to
>> insert into the database, and hence I would rather do them in parallel
>> than
>> doing them one at a time. I looked around but did not find any information
>> on doing this. Does MySQL not support parallel reads/writes?
>>
>> Thanks in advance.
>>
>> Andy
>>
>
>

--001636284baa23b1e304978dd3d8--

Re: MySQL Parallel Inserts

--20cf3043430067e33a0497e891da
Content-Type: text/plain; charset=UTF-8

Hi.
MySQL do the sql parse in the single way, just one thread. So if there
are parallel insert statement, it will not faster than before.
David Yeung, In China, Beijing.
My First Blog:http://yueliangdao0608.cublog.cn
My Second Blog:http://yueliangdao0608.blog.51cto.com


2010/12/17 Andy

> Shawn/Krishna,
>
> Thank you. I will try this.
>
> -Andy
>
>
> On Tue, Dec 14, 2010 at 8:13 AM, Krishna Chandra Prajapati <
> prajapatikc@gmail.com> wrote:
>
> > Hi Andy,
> >
> > I agree, prefer LOAD DATA INFILE... command for bulk loading. It is 30%
> > faster than normal inserts.
> >
> > Krishna
> >
> >
> > On Tue, Dec 14, 2010 at 8:02 AM, Andy wrote:
> >
> >> Greetings everyone.
> >>
> >> I am in a situation where I need to do parallel inserts into MySQL
> >> database
> >> from inside my Perl program. Basically, I have several million records
> to
> >> insert into the database, and hence I would rather do them in parallel
> >> than
> >> doing them one at a time. I looked around but did not find any
> information
> >> on doing this. Does MySQL not support parallel reads/writes?
> >>
> >> Thanks in advance.
> >>
> >> Andy
> >>
> >
> >
>

--20cf3043430067e33a0497e891da--