Array read 3 time slower to read after data fully loaded

Hi,
Can anyone please explain this to me?

=46or the last >30 hours I have been loading some 2.5TB of 200-400MB ti=
ff=20
images, from another PC on our internal network, onto this new array.
Yesterday, during this load process, I copied 10 of these files (about=20
2.5GB) from the array onto the /home directory of the same system.

It took 33 seconds, equating to some 75MB/s.

This morning I came in to find that the load process had finished, so I=
=20
deleted the 10 test files from /home, and redid the identical copy test=
=20
(using bash-history, so: identical).

It took 123 seconds, equating to some 20MB/s.

Que? (as in Manuel from Faulty Towers :-) )


Thanks & regards,
Zoltan


root@gs0:/home/geograph# uname -a
Linux gs0 2.6.32-25-generic #45-Ubuntu SMP Sat Oct 16 19:52:42 UTC 2010=
=20
x86_64 GNU/Linux
root@gs0:/home/geograph# mdadm -V
mdadm - v3.1.4 - 31st August 2010
root@gs0:/home/geograph# lvm version
LVM version: 2.02.54(1) (2009-10-26)
Library version: 1.02.39 (2009-10-26)
Driver version: 4.15.0
root@gs0:/home/geograph# df #ext4
=46ilesystem 1K-blocks Used Available Use% Mounted on
/dev/sdi7 426559760 9494844 395396872 3% /home
/dev/mapper/geo_vg0-geo_lvm0
11536635672 2530042680 8889387468 23% /mnt/geo_l=
vm0
root@gs0:/home/geograph#


--=20

==================== =====
===================3D
Zoltan Szecsei PrGISc [PGP0031]
Geograph (Pty) Ltd.
P.O. Box 7, Muizenberg 7950, South Africa.

65 Main Road, Muizenberg 7945
Western Cape, South Africa.

34° 6'16.35"S 18°28'5.62"E

Tel: +27-21-7884897 Mobile: +27-83-6004028
=46ax: +27-86-6115323 www.geograph.co.za
==================== =====
===================3D



-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 10.0.1153 / Virus Database: 424/3262 - Release Date: 11/17/10

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" i=
n
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Array read 3 time slower to read after data fully loaded

On 10-11-18 06:30, Zoltan Szecsei wrote:
> Hi,
> Can anyone please explain this to me?
>
> For the last >30 hours I have been loading some 2.5TB of 200-400MB tiff
> images, from another PC on our internal network, onto this new array.
> Yesterday, during this load process, I copied 10 of these files (about
> 2.5GB) from the array onto the /home directory of the same system.
>
> It took 33 seconds, equating to some 75MB/s.
>
> This morning I came in to find that the load process had finished, so I
> deleted the 10 test files from /home, and redid the identical copy test
> (using bash-history, so: identical).
>
> It took 123 seconds, equating to some 20MB/s.
>
> Que? (as in Manuel from Faulty Towers :-) )
>

Is the situation with reading into /dev/null instead of /home the same ?

As this is reading issue, one thing that came to my mind:

- make sure the kernel is _not_ compiled with CONFIG_MULTICORE_RAID456.
It's afaik still experimental and can cause severe slowdowns

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

RE: Array read 3 time slower to read after data fully loaded

Throughput is a function of block size & chunk size. For operations such as copying large files, then larger block/chunk sizes mean
higher throughput .. at the COST of I/Os per second.

There is no free lunch. Before doing any tuning, ask yourself what the normal mix of random, sequential, read, write, large and small
block I/O is. If you are constantly moving large files around, then by all means redo the RAID and file system setup, but if this exercise is an exception, then you may be better off leaving things as they are.
________________________________________
From: linux-raid-owner@vger.kernel.org [linux-raid-owner@vger.kernel.org] On Behalf Of Michal Soltys [soltys@ziu.info]
Sent: Thursday, November 18, 2010 2:43 PM
To: Zoltan Szecsei
Cc: linux-raid@vger.kernel.org
Subject: Re: Array read 3 time slower to read after data fully loaded

On 10-11-18 06:30, Zoltan Szecsei wrote:
> Hi,
> Can anyone please explain this to me?
>
> For the last >30 hours I have been loading some 2.5TB of 200-400MB tiff
> images, from another PC on our internal network, onto this new array.
> Yesterday, during this load process, I copied 10 of these files (about
> 2.5GB) from the array onto the /home directory of the same system.
>
> It took 33 seconds, equating to some 75MB/s.
>
> This morning I came in to find that the load process had finished, so I
> deleted the 10 test files from /home, and redid the identical copy test
> (using bash-history, so: identical).
>
> It took 123 seconds, equating to some 20MB/s.
>
> Que? (as in Manuel from Faulty Towers :-) )
>

Is the situation with reading into /dev/null instead of /home the same ?

As this is reading issue, one thing that came to my mind:

- make sure the kernel is _not_ compiled with CONFIG_MULTICORE_RAID456.
It's afaik still experimental and can cause severe slowdowns

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Array read 3 time slower to read after data fully loaded

On Thu, 18 Nov 2010 07:30:14 +0200
Zoltan Szecsei wrote:

> Hi,
> Can anyone please explain this to me?
>
> For the last >30 hours I have been loading some 2.5TB of 200-400MB tiff
> images, from another PC on our internal network, onto this new array.
> Yesterday, during this load process, I copied 10 of these files (about
> 2.5GB) from the array onto the /home directory of the same system.
>
> It took 33 seconds, equating to some 75MB/s.
>
> This morning I came in to find that the load process had finished, so I
> deleted the 10 test files from /home, and redid the identical copy test
> (using bash-history, so: identical).
>
> It took 123 seconds, equating to some 20MB/s.
>
> Que? (as in Manuel from Faulty Towers :-) )

Maybe the first time some of the files were in RAM... how much RAM do you
have, and how recently that the files been loaded before your copied them?

NeilBrown - who has never been to Barcelona.

>
>
> Thanks & regards,
> Zoltan
>
>
> root@gs0:/home/geograph# uname -a
> Linux gs0 2.6.32-25-generic #45-Ubuntu SMP Sat Oct 16 19:52:42 UTC 2010
> x86_64 GNU/Linux
> root@gs0:/home/geograph# mdadm -V
> mdadm - v3.1.4 - 31st August 2010
> root@gs0:/home/geograph# lvm version
> LVM version: 2.02.54(1) (2009-10-26)
> Library version: 1.02.39 (2009-10-26)
> Driver version: 4.15.0
> root@gs0:/home/geograph# df #ext4
> Filesystem 1K-blocks Used Available Use% Mounted on
> /dev/sdi7 426559760 9494844 395396872 3% /home
> /dev/mapper/geo_vg0-geo_lvm0
> 11536635672 2530042680 8889387468 23% /mnt/geo_lvm0
> root@gs0:/home/geograph#
>
>

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html