Replaying the mysqld.log file from production onto QA???

--0-1266536781-1283970158=:63639
Content-Type: text/plain; charset=us-ascii

Hello all, I am trying to "REPLAY" the mysqld.log file from a production 5.1.4
server onto a QA server with 5.5 running and was wondering if there is an easy
way of replaying the mysqld.log file? Just want to make sure all of the kinds
of inserts, updates, selects and deletes work just as well on the 5.5 box, esp
since we are turning on replication and copying tables from another server onto
this server. Any ideas please?

TIA...

Nunzio




--0-1266536781-1283970158=:63639--

Re: Replaying the mysqld.log file from production onto QA???

--_000_D73BD5A6C60A42DD85640D28A27FE208nokiacom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Try using a mysqldump.

----- Reply message -----
From: "ext Nunzio Daveri"
Date: Wed, Sep 8, 2010 19:25
Subject: Replaying the mysqld.log file from production onto QA???
To: "mysql@lists.mysql.com"

Hello all, I am trying to "REPLAY" the mysqld.log file from a production 5.=
1.4
server onto a QA server with 5.5 running and was wondering if there is an e=
asy
way of replaying the mysqld.log file? Just want to make sure all of the ki=
nds
of inserts, updates, selects and deletes work just as well on the 5.5 box, =
esp
since we are turning on replication and copying tables from another server =
onto
this server. Any ideas please?

TIA...

Nunzio





--_000_D73BD5A6C60A42DD85640D28A27FE208nokiacom_--

Re: Replaying the mysqld.log file from production onto QA???

--0-198420161-1284041528=:71988
Content-Type: text/plain; charset=us-ascii

So.. I am trying to mimic replaying production like queries so joins, temp
tables etc... are stuff I am trying to test as well. Just doing a dump and
import is no more than export and importing, I also want to test selects,
updates :-) Thanks for replying :-)

Nunzio




________________________________
From: "andrew.2.moore@nokia.com"
To: nunziodaveri@yahoo.com; mysql@lists.mysql.com
Sent: Wed, September 8, 2010 4:19:19 PM
Subject: Re: Replaying the mysqld.log file from production onto QA???

Try using a mysqldump.


----- Reply message -----
From: "ext Nunzio Daveri"
Date: Wed, Sep 8, 2010 19:25
Subject: Replaying the mysqld.log file from production onto QA???
To: "mysql@lists.mysql.com"

Hello all, I am trying to "REPLAY" the mysqld.log file from a production 5.1.4
server onto a QA server with 5.5 running and was wondering if there is an easy
way of replaying the mysqld.log file? Just want to make sure all of the kinds
of inserts, updates, selects and deletes work just as well on the 5.5 box, esp
since we are turning on replication and copying tables from another server onto
this server. Any ideas please?

TIA...

Nunzio



--0-198420161-1284041528=:71988--

Re: Replaying the mysqld.log file from production onto QA???

--00032557552aabb15e048fe64f3d
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Sep 9, 2010 at 4:12 PM, Nunzio Daveri wrote:

> So.. I am trying to mimic replaying production like queries so joins, temp
> tables etc... are stuff I am trying to test as well. Just doing a dump and
> import is no more than export and importing, I also want to test selects,
> updates :-) Thanks for replying :-)
>


Then you'll have to first activate full logging on your production server.
This may take quite some diskpace, and IOs, so keep the logfiles on separate
spindles.

Extracting the queries from the log shouldn't be all that hard; but there's
too little timing information in there to do a time-true replay - and I
don't know of any app that does that, either.




>
> Nunzio
>
>
>
>
> ________________________________
> From: "andrew.2.moore@nokia.com"
> To: nunziodaveri@yahoo.com; mysql@lists.mysql.com
> Sent: Wed, September 8, 2010 4:19:19 PM
> Subject: Re: Replaying the mysqld.log file from production onto QA???
>
> Try using a mysqldump.
>
>
> ----- Reply message -----
> From: "ext Nunzio Daveri"
> Date: Wed, Sep 8, 2010 19:25
> Subject: Replaying the mysqld.log file from production onto QA???
> To: "mysql@lists.mysql.com"
>
> Hello all, I am trying to "REPLAY" the mysqld.log file from a production
> 5.1.4
> server onto a QA server with 5.5 running and was wondering if there is an
> easy
> way of replaying the mysqld.log file? Just want to make sure all of the
> kinds
> of inserts, updates, selects and deletes work just as well on the 5.5 box,
> esp
> since we are turning on replication and copying tables from another server
> onto
> this server. Any ideas please?
>
> TIA...
>
> Nunzio
>
>
>
>



--
Bier met grenadyn
Is als mosterd by den wyn
Sy die't drinkt, is eene kwezel
Hy die't drinkt, is ras een ezel

--00032557552aabb15e048fe64f3d--

Re: Replaying the mysqld.log file from production onto QA???

Hello Johan,

On 9/10/2010 7:47 AM, Johan De Meersman wrote:
> On Thu, Sep 9, 2010 at 4:12 PM, Nunzio Daveri wrote:
>
>> So.. I am trying to mimic replaying production like queries so joins, temp
>> tables etc... are stuff I am trying to test as well. Just doing a dump and
>> import is no more than export and importing, I also want to test selects,
>> updates :-) Thanks for replying :-)
>>
>
>
> Then you'll have to first activate full logging on your production server.
> This may take quite some diskpace, and IOs, so keep the logfiles on separate
> spindles.
>
> Extracting the queries from the log shouldn't be all that hard; but there's
> too little timing information in there to do a time-true replay - and I
> don't know of any app that does that, either.
>

He already did! Those are the logs he needs to replay. He has the logs
already but needs tools to extract the commands and repeat them as a
load test.

Do you have any techniques you can share?

--
Shawn Green
MySQL Principal Technical Support Engineer
Oracle USA, Inc.
Office: Blountville, TN

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: Replaying the mysqld.log file from production onto QA???

--000325575efeb3e79d048fe7dcbc
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Sep 10, 2010 at 1:56 PM, Shawn Green (MySQL) <
shawn.l.green@oracle.com> wrote:

> He already did! Those are the logs he needs to replay. He has the logs
> already but needs tools to extract the commands and repeat them as a load
> test.
>
> Do you have any techniques you can share?
>

Alas, no. I suspect the rather expensive Avalanches we got might be able to,
but that's not particularly helpful to him :-)

I wrote a little thing many years ago that pulled Apache logs from
production over SSH and multithreaded them at the staging server, but SQL
stuff would be much more time-critical - can't select before the insert :-)



--
Bier met grenadyn
Is als mosterd by den wyn
Sy die't drinkt, is eene kwezel
Hy die't drinkt, is ras een ezel

--000325575efeb3e79d048fe7dcbc--

RE: Replaying the mysqld.log file from production onto QA???

If possible, use web server logging to get the timings.

As I explained in my previous essay, this is not at all simple or easy.

If you run a benchmark more than once, be sure to reboot the system between
runs.

Regards,

Jerry Schwartz
Global Information Incorporated
195 Farmington Ave.
Farmington, CT 06032

860.674.8796 / FAX: 860.674.8341
E-mail: jerry@gii.co.jp
Web site: www.the-infoshop.com

>-----Original Message-----
>From: Shawn Green (MySQL) [mailto:shawn.l.green@oracle.com]
>Sent: Friday, September 10, 2010 7:56 AM
>To: Johan De Meersman
>Cc: Nunzio Daveri; andrew.2.moore@nokia.com; mysql@lists.mysql.com
>Subject: Re: Replaying the mysqld.log file from production onto QA???
>
>Hello Johan,
>
>On 9/10/2010 7:47 AM, Johan De Meersman wrote:
>> On Thu, Sep 9, 2010 at 4:12 PM, Nunzio Daveri
>> wrote:
>>
>>> So.. I am trying to mimic replaying production like queries so joins, temp
>>> tables etc... are stuff I am trying to test as well. Just doing a dump
>>> and
>>> import is no more than export and importing, I also want to test selects,
>>> updates :-) Thanks for replying :-)
>>>
>>
>>
>> Then you'll have to first activate full logging on your production server.
>> This may take quite some diskpace, and IOs, so keep the logfiles on
>> separate
>> spindles.
>>
>> Extracting the queries from the log shouldn't be all that hard; but there's
>> too little timing information in there to do a time-true replay - and I
>> don't know of any app that does that, either.
>>
>
>He already did! Those are the logs he needs to replay. He has the logs
>already but needs tools to extract the commands and repeat them as a
>load test.
>
>Do you have any techniques you can share?
>
>--
>Shawn Green
>MySQL Principal Technical Support Engineer
>Oracle USA, Inc.
>Office: Blountville, TN
>
>--
>MySQL General Mailing List
>For list archives: http://lists.mysql.com/mysql
>To unsubscribe: http://lists.mysql.com/mysql?unsub=jerry@gii.co.jp





--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql-2@m.gmane.org

Re: Replaying the mysqld.log file from production onto QA???

--001485f90ec0e8d34c048feb3789
Content-Type: text/plain; charset=ISO-8859-1

Would something like: http://www.maatkit.org/doc/mk-log-player.html

Work? I know the timings
won't be the same as how the queries actually came in, but .... it could be
a start,

JW


On Fri, Sep 10, 2010 at 8:38 AM, Johan De Meersman wrote:

> On Fri, Sep 10, 2010 at 1:56 PM, Shawn Green (MySQL) <
> shawn.l.green@oracle.com> wrote:
>
> > He already did! Those are the logs he needs to replay. He has the logs
> > already but needs tools to extract the commands and repeat them as a load
> > test.
> >
> > Do you have any techniques you can share?
> >
>
> Alas, no. I suspect the rather expensive Avalanches we got might be able
> to,
> but that's not particularly helpful to him :-)
>
> I wrote a little thing many years ago that pulled Apache logs from
> production over SSH and multithreaded them at the staging server, but SQL
> stuff would be much more time-critical - can't select before the insert :-)
>
>
>
> --
> Bier met grenadyn
> Is als mosterd by den wyn
> Sy die't drinkt, is eene kwezel
> Hy die't drinkt, is ras een ezel
>



--
-----------------------------
Johnny Withers
601.209.4985
johnny@pixelated.net

--001485f90ec0e8d34c048feb3789--

Re: Replaying the mysqld.log file from production onto QA???

The mysqld.log isn't in an executable format, but mk-query-digest can
understand it and convert it into a "slow query log" format. Then you
can use mk-upgrade to validate that your queries produce the same
result. Good for you for thinking to do this -- most people just
upgrade and then panic when something doesn't work right.

On Wed, Sep 8, 2010 at 2:22 PM, Nunzio Daveri wrot=
e:
> Hello all, I am trying to "REPLAY" the mysqld.log file from a production =
5.1.4
> server onto a QA server with 5.5 running and was wondering if there is an=
easy
> way of replaying the mysqld.log file? =A0Just want to make sure all of th=
e kinds
> of inserts, updates, selects and deletes work just as well on the 5.5 box=
, esp
> since we are turning on replication and copying tables from another serve=
r onto
> this server. =A0Any ideas please?
>
> TIA...
>
> Nunzio
>
>
>
>



--=20
Baron Schwartz
Percona Inc
Consulting, Training, Support & Services for MySQL

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=3Dgcdmg-mysql-2@m.gmane.o rg