--0016e64764367e12fc047dabf7cc
Content-Type: text/plain; charset=ISO-8859-1
Hi list,
I want to insert 10000 records/sec into table. There can be n number of
tables with unique data in each. What are the possible ways to do ?
Thanks,
Krishna
--0016e64764367e12fc047dabf7cc--
Krishna Chandra Prajapati wrote:
> Hi list,
>
> I want to insert 10000 records/sec into table. There can be n number of
> tables with unique data in each. What are the possible ways to do ?
>
> Thanks,
> Krishna
>
The manual is your friend. It doesn't hurt to consult it.
INSERT ...
http://dev.mysql.com/doc/refman/5.1/en/insert.html
LOAD DATA INFILE ...
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
And we even have both general and specific suggestions on improving
INSERT performance in our Optimization chapter:
http://dev.mysql.com/doc/refman/5.1/en/optimization.html
Warmest regards,
--
Shawn Green, MySQL Senior Support Engineer
Sun Microsystems, Inc.
Office: Blountville, TN
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org
Krishna Chandra Prajapati schrieb:
> Hi list,
>
> I want to insert 10000 records/sec into table. There can be n number of
> tables with unique data in each. What are the possible ways to do ?
>
i prefer mysqlimport. just sort your output into a file that is named like the table
you wish to import. Basic unix scripting is sufficient.
re,
wh
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org
--001485eaffaa411450047dd20819
Content-Type: text/plain; charset=ISO-8859-1
Hi shawn,
As the data grows to 20 millions the insert rate will become very slow. In
such case i am getting 2000 insert/seconds only.
Therefore my objective is not achieved.
I cannot slow up the insert rate of 10,000/second. I am getting data
(inserted by users at this rate)
Is there any other way to do so. (distributed servers)
Thanks,
Krishna
On Thu, Jan 21, 2010 at 7:39 PM, Shawn Green
> Krishna Chandra Prajapati wrote:
>
>> Hi list,
>>
>> I want to insert 10000 records/sec into table. There can be n number of
>> tables with unique data in each. What are the possible ways to do ?
>>
>> Thanks,
>> Krishna
>>
>> The manual is your friend. It doesn't hurt to consult it.
>
> INSERT ...
> http://dev.mysql.com/doc/refman/5.1/en/insert.html
>
> LOAD DATA INFILE ...
> http://dev.mysql.com/doc/refman/5.1/en/load-data.html
>
> And we even have both general and specific suggestions on improving INSERT
> performance in our Optimization chapter:
> http://dev.mysql.com/doc/refman/5.1/en/optimization.html
>
> Warmest regards,
> --
> Shawn Green, MySQL Senior Support Engineer
> Sun Microsystems, Inc.
> Office: Blountville, TN
>
>
>
--001485eaffaa411450047dd20819--
non-linearity in the insert rate means you have indexes on some columns.
depending on your situation, mysql can be more efficient if drop those
indexes, do bulk inserts, and then add the indexes again.
On 1/23/10 5:02 AM, "Krishna Chandra Prajapati"
wrote:
> Hi shawn,
>
> As the data grows to 20 millions the insert rate will become very slow. In
> such case i am getting 2000 insert/seconds only.
>
> Therefore my objective is not achieved.
>
> I cannot slow up the insert rate of 10,000/second. I am getting data
> (inserted by users at this rate)
>
> Is there any other way to do so. (distributed servers)
>
> Thanks,
> Krishna
>
> On Thu, Jan 21, 2010 at 7:39 PM, Shawn Green
>
>> Krishna Chandra Prajapati wrote:
>>
>>> Hi list,
>>>
>>> I want to insert 10000 records/sec into table. There can be n number of
>>> tables with unique data in each. What are the possible ways to do ?
>>>
>>> Thanks,
>>> Krishna
>>>
>>> The manual is your friend. It doesn't hurt to consult it.
>>
>> INSERT ...
>> http://dev.mysql.com/doc/refman/5.1/en/insert.html
>>
>> LOAD DATA INFILE ...
>> http://dev.mysql.com/doc/refman/5.1/en/load-data.html
>>
>> And we even have both general and specific suggestions on improving INSERT
>> performance in our Optimization chapter:
>> http://dev.mysql.com/doc/refman/5.1/en/optimization.html
>>
>> Warmest regards,
>> --
>> Shawn Green, MySQL Senior Support Engineer
>> Sun Microsystems, Inc.
>> Office: Blountville, TN
>>
>>
>>
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org
If you only need very fast INSERTs, you might try to use ARCHIVE
storage engine (
http://dev.mysql.com/tech-resources/articles/storage-engine. html ). It
was developed for handling INSERTs very fast. Many peoples use it, for
example, for storing logs.
> Hi list,
>
> I want to insert 10000 records/sec into table. =A0There can be n number o=
f
> tables with unique data in each. What are the possible ways to do ?
>
> Thanks,
> Krishna
>
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgcdmg-mysql-2@m.gmane.o rg
Krishna Chandra Prajapati wrote:
> Hi shawn,
>
> As the data grows to 20 millions the insert rate will become very slow.
> In such case i am getting 2000 insert/seconds only.
>
> Therefore my objective is not achieved.
>
> I cannot slow up the insert rate of 10,000/second. I am getting data
> (inserted by users at this rate)
>
> Is there any other way to do so. (distributed servers)
>
As mentioned in
http://dev.mysql.com/doc/refman/5.1/en/optimization.html
Many things can affect the speed at which you can import data:
* The actual size and data types of the rows you are inserting.
* The storage engine you are using
* The method you use to insert the rows (INSERT vs. INSERT (extended
format) vs LOAD DATA INFILE ...)
* The number of and types of indexes on the table you are inserting into
* The CPU or disk contention from other processes on the same machine.
* The speed of your network and or storage devices
* The use of DRBD
....
Full treatment of all of those factors is beyond the level of assistance
I am permitted to supply in these forums. However if you care to ask a
specific question or two I may be able to bend the rules a little.
--
Shawn Green, MySQL Senior Support Engineer
Sun Microsystems, Inc.
Office: Blountville, TN
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org