[Wlug] large file performance on linux
karl at hiramoto.org
Wed May 16 15:35:55 EDT 2007
brad noyes wrote:
> On Wed, May 16, 2007 at 02:40:38PM -0400, Jeff Moyer wrote:
>> ==> On Wed, 16 May 2007 12:59:54 -0400, Jeff Moyer <jmoyer at redhat.com> said:
>> Jeff> Look under /proc/sys/vm. Documentation for these variables might be
>> Jeff> in Documentation/filesystems/proc.txt (it's not always up-to-date).
>> Jeff> But, as I said, I don't think this is the right avenue to explore.
>> Jeff> You can get more predictable results by using AIO+O_DIRECT (or maybe
>> Jeff> even O_SYNC as another mentioned).
>> One other thing worth mentioning is that you should be doing I/O in
>> large block sizes. What size are you currently using for your write
> i'm writing in 16777216 byte chunks. That happens to be evenly divisible by 512
> for the O_DIRECT flag. However every time i try to use that flag the file gets
> created, but nothing gets written. I've been looking online for an example.
> I don't know if this means anything, but i ran
> hdparm -T /dev/sdb1
> Timing cached reads: 1369MB in 2.00 seconds == 698.14MB/sec
> hdparm -T --direct /dev/sdb1
> Timing O_DIRECT cached reads: 136 in 2.00 seconds == 66.54MB/sec
> It really seems that 53MB/s shouldn't be hard. I have fairly heavy hardware,
> scsi320 in a raid1 configuration.
> -- Brad
That is CACHED read.. your uncached write will be slower.. run a
bonnie++ benchmark on your disks. You might want to switch to raid 0 or
And use a 64 bit kernel to be able to allocate more than 3G of ram.
Karl Hiramoto http://karl.hiramoto.org/
US VOIP: (1) 603.966.4448
Spain Casa (34) 951.273.347
Spain Mobil (34) 617.463.826
Yahoo_IM = karl_hiramoto GTalk= karl.hiramoto [at] gmail [d0t] com
Everyone complains of his memory, no one of his judgement.
More information about the Wlug