compacting / data size

Hi all!

I am (still) running some tests (dispite my problems), and I'd like to
know the size of data I have in order to calculate growth.

As a test I added some data, 1.69 Gb to be exact to a database, working
with it, then I deleted app 66% of it. When deleting it grew to 1.97
Gb.

I'd like to schrik that, as it will never again hold that amount of
data.
But more important - I'd like to know how much of that is actual data.

The point is, that Id like to delete even more data, and see the size
(of data) again.

How do I do that?

Sonnich

Re: compacting / data size

"Sonnich" wrote in message
news:1141319874.714208.264340@v46g2000cwv.googlegroups.com.. .
> As a test I added some data, 1.69 Gb to be exact to a database, working
> with it, then I deleted app 66% of it. When deleting it grew to 1.97 Gb.

Read about OPTIMIZE TABLE, to reclaim free space and defragment the data
file.

http://dev.mysql.com/doc/refman/5.0/en/optimize-table.html

Regards,
Bill K.

Re: compacting / data size

Bill Karwin wrote:
> "Sonnich" wrote in message
> news:1141319874.714208.264340@v46g2000cwv.googlegroups.com.. .
> > As a test I added some data, 1.69 Gb to be exact to a database, working
> > with it, then I deleted app 66% of it. When deleting it grew to 1.97 Gb.
>
> Read about OPTIMIZE TABLE, to reclaim free space and defragment the data
> file.
>
> http://dev.mysql.com/doc/refman/5.0/en/optimize-table.html

I have tried this before, and the result is none-existing. My DB is
1.97 GB, and remains. I can free up more than one GB, I just need to
know how.

I read about it, wrote "optimize table mytable", the table remains,
looking at the page mentioned here and looking in the NG has not brough
me to any solution yet.

What exactly should I do?

BR
Sonnich

Re: compacting / data size

Sonnich wrote:
> Bill Karwin wrote:
>> "Sonnich" wrote in message
>> news:1141319874.714208.264340@v46g2000cwv.googlegroups.com.. .
>>> As a test I added some data, 1.69 Gb to be exact to a database, working
>>> with it, then I deleted app 66% of it. When deleting it grew to 1.97 Gb.
>> Read about OPTIMIZE TABLE, to reclaim free space and defragment the data
>> file.
>>
>> http://dev.mysql.com/doc/refman/5.0/en/optimize-table.html
>
> I have tried this before, and the result is none-existing. My DB is
> 1.97 GB, and remains. I can free up more than one GB, I just need to
> know how.
>
> I read about it, wrote "optimize table mytable", the table remains,
> looking at the page mentioned here and looking in the NG has not brough
> me to any solution yet.
>
> What exactly should I do?
>
> BR
> Sonnich
>

How many tables are there in the database? Did you optimize all of them?

Marc

Re: compacting / data size

"Sonnich" wrote in message
news:1141376777.328010.115900@i39g2000cwa.googlegroups.com.. .
>
> I have tried this before, and the result is none-existing. My DB is
> 1.97 GB, and remains. I can free up more than one GB, I just need to
> know how.

Hmm. Apparently OPTIMIZE TABLE defragments the free space in a table, but
the file size remains the same. I suppose making the free space contiguous
doesn't necessarily mean the free space will be at the end of the file.

Is the table MyISAM or InnoDB? If it is MyISAM, you could copy the table to
a new table of identical structure. Then drop the old table.

If it is InnoDB, you can perform a "dummy" ALTER TABLE, which will cause the
table to be rebuilt (this may take a long time, and temporarily require
additional disk space).

For either storage engine, you can dump the table to a text file, drop the
table, and the restore the table from the dump you made.

See also the commentary threads in these two bugs:
http://bugs.mysql.com/bug.php?id=1287
http://bugs.mysql.com/bug.php?id=1341

Regards,
Bill K.

Re: compacting / data size

Marc DVer wrote:
> Sonnich wrote:
> > Bill Karwin wrote:
> >> "Sonnich" wrote in message
> >> news:1141319874.714208.264340@v46g2000cwv.googlegroups.com.. .
> >>> As a test I added some data, 1.69 Gb to be exact to a database, working
> >>> with it, then I deleted app 66% of it. When deleting it grew to 1.97 Gb.
> >> Read about OPTIMIZE TABLE, to reclaim free space and defragment the data
> >> file.
> >>
> >> http://dev.mysql.com/doc/refman/5.0/en/optimize-table.html
> >
> > I have tried this before, and the result is none-existing. My DB is
> > 1.97 GB, and remains. I can free up more than one GB, I just need to
> > know how.
> >
> > I read about it, wrote "optimize table mytable", the table remains,
> > looking at the page mentioned here and looking in the NG has not brough
> > me to any solution yet.
> >
> > What exactly should I do?
>
> How many tables are there in the database? Did you optimize all of them?

I have 15 tables, where:

3 are the ones, where data are added to for some temporary time. Once
too old or processed, data are deleted from here again. These tables
are app. 99% of all data I have. They hold 1-4 millions rows each.
2 tables get data when something happens (like a log). Data are not
added often.
4 tables hold "processed data" from the 3 tables above. After 3-4
months of testing they have 1-4000 rows each.
6 tables are lookup data (strings). Mostly less than 10 rows, one has
app 100 rows.

Only the three first ones are being optimized. Thet are the only ones,
where data is being deleted from.

BR
Sonnich

Re: compacting / data size

Bill Karwin wrote:
> "Sonnich" wrote in message
> news:1141376777.328010.115900@i39g2000cwa.googlegroups.com.. .
> >
> > I have tried this before, and the result is none-existing. My DB is
> > 1.97 GB, and remains. I can free up more than one GB, I just need to
> > know how.
>
> Hmm. Apparently OPTIMIZE TABLE defragments the free space in a table, but
> the file size remains the same. I suppose making the free space contiguous
> doesn't necessarily mean the free space will be at the end of the file.
>
> Is the table MyISAM or InnoDB? If it is MyISAM, you could copy the table to
> a new table of identical structure. Then drop the old table.

InnoDB I asume.

MySQL 5.0, odbcs driver 3.51 (5.0/5.1 does not work properly with
Delphi)

> If it is InnoDB, you can perform a "dummy" ALTER TABLE, which will cause the
> table to be rebuilt (this may take a long time, and temporarily require
> additional disk space).

like adding a field, then dropping it again?
I get the point, but it is too risky, as if it fails, and the programme
is restarted, it might get problems saving data, as I save data this
way:

insert into sfgdfsgdsf values(a,b,c)

If I add "d", then the line above might fail... and then I have system
which does not work.

>
> For either storage engine, you can dump the table to a text file, drop the
> table, and the restore the table from the dump you made.

I will taka a look as this...

>
> See also the commentary threads in these two bugs:
> http://bugs.mysql.com/bug.php?id=1287
> http://bugs.mysql.com/bug.php?id=1341
>
> Regards,
> Bill K.

Re: compacting / data size

"Sonnich" wrote in message
news:1141913779.103395.200190@v46g2000cwv.googlegroups.com.. .
>> If it is InnoDB, you can perform a "dummy" ALTER TABLE, which will cause
>> the
>> table to be rebuilt (this may take a long time, and temporarily require
>> additional disk space).
>
> like adding a field, then dropping it again?

Sorry, I meant to include an example:

ALTER TABLE reallybigtable TYPE=InnoDB;

Assuming the table was already InnoDB to begin with, this makes no change,
but triggers a complete rebuild of the table. As does adding or dropping an
index, etc.

You can check what storage type you're using for a given table called
reallybigtable, as follows:

SHOW TABLE STATUS LIKE 'reallybigtable';

The 'Engine' column shows the storage engine type, e.g. MyISAM or InnoDB.

Regards,
Bill K.

Re: compacting / data size

Bill Karwin wrote:
> "Sonnich" wrote in message
> news:1141913779.103395.200190@v46g2000cwv.googlegroups.com.. .
> >> If it is InnoDB, you can perform a "dummy" ALTER TABLE, which will cause
> >> the
> >> table to be rebuilt (this may take a long time, and temporarily require
> >> additional disk space).
> > like adding a field, then dropping it again?
> Sorry, I meant to include an example:
> ALTER TABLE reallybigtable TYPE=InnoDB;
>
> Assuming the table was already InnoDB to begin with, this makes no change,
> but triggers a complete rebuild of the table. As does adding or dropping an
> index, etc.
>
> You can check what storage type you're using for a given table called
> reallybigtable, as follows:
> SHOW TABLE STATUS LIKE 'reallybigtable';
> The 'Engine' column shows the storage engine type, e.g. MyISAM or InnoDB.
>
> Regards,
> Bill K.

Well they are indeed InnoDB's, but this does still not change
anything... The only result I got was that I got the file ~200 Mb
larger.
I even tried to alter all 15 tables I have, and the result did not
show.

I think I better realise, that there hardly is any solution after
having looking for one for some time, and move on to something else.

BR
Sonnich

Re: compacting / data size

"Sonnich" wrote in message
news:1141995628.338652.242320@v46g2000cwv.googlegroups.com.. .
> I think I better realise, that there hardly is any solution after
> having looking for one for some time, and move on to something else.

It seems you are right. The MySQL bug #1341 has no resolution at this time.

The ultimate solution might be:
1. Back up _all_ InnoDB databases you have on the system (since they are all
stored in the one file /ibdata1)
2. Remove the ibdata1 file with 'rm'
3. Restore your databases

Also consider configuring InnoDB to store databases in one file per table
before you restore the databases. That may make it easier to recover space
on a table-by-table basis in the future.
See http://dev.mysql.com/doc/refman/5.0/en/multiple-tablespaces. html

Regards,
Bill K.

Re: compacting / data size

Bill Karwin wrote:
> "Sonnich" wrote in message
> news:1141995628.338652.242320@v46g2000cwv.googlegroups.com.. .
> > I think I better realise, that there hardly is any solution after
> > having looking for one for some time, and move on to something else.
>
> It seems you are right. The MySQL bug #1341 has no resolution at this time.
>
> The ultimate solution might be:
> 1. Back up _all_ InnoDB databases you have on the system (since they are all
> stored in the one file /ibdata1)
> 2. Remove the ibdata1 file with 'rm'
> 3. Restore your databases
>
> Also consider configuring InnoDB to store databases in one file per table
> before you restore the databases. That may make it easier to recover space
> on a table-by-table basis in the future.
> See http://dev.mysql.com/doc/refman/5.0/en/multiple-tablespaces. html

This might work, but in the project I have, this is not possible.
Unfortunately, I have been thinking of doing this too, but another
reason not to, is that to delete the file I have to stop the service...
this will be something a bit to specific.... and I'll end up speding
too much time on it.

So it might be better to find something else before this gets enterely
out of hand (but sad, as MySQL was the 2nd best option we had).

/S