[Wlug] large file performance on linux

[Jeff Moyer]

==> On Wed, 16 May 2007 19:19:45 +0000, brad noyes <maitre at ccs.neu.edu> said:

brad> On Wed, May 16, 2007 at 02:40:38PM -0400, Jeff Moyer wrote:
brad> > ==> On Wed, 16 May 2007 12:59:54 -0400, Jeff Moyer <jmoyer at redhat.com> said:
brad> > 
brad> > Jeff> Look under /proc/sys/vm.  Documentation for these variables might be
brad> > Jeff> in Documentation/filesystems/proc.txt (it's not always up-to-date).
brad> > Jeff> But, as I said, I don't think this is the right avenue to explore.
brad> > Jeff> You can get more predictable results by using AIO+O_DIRECT (or maybe
brad> > Jeff> even O_SYNC as another mentioned).
brad> > 
brad> > One other thing worth mentioning is that you should be doing I/O in
brad> > large block sizes.  What size are you currently using for your write
brad> > buffers?
brad> > 
brad> i'm writing in 16777216 byte chunks. That happens to be evenly divisible by 512
brad> for the O_DIRECT flag. However every time i try to use that flag the file gets
brad> created, but nothing gets written. I've been looking online for an example.

Are you aligning your buffers on a 512 byte boundary?  Man posix_memalign.

brad> I don't know if this means anything, but i ran 
brad>   hdparm -T /dev/sdb1
brad>     Timing cached reads: 1369MB in 2.00 seconds == 698.14MB/sec
brad>   hdparm -T --direct /dev/sdb1
brad>     Timing O_DIRECT cached reads: 136 in 2.00 seconds == 66.54MB/sec

brad> It really seems that 53MB/s shouldn't be hard. I have fairly heavy hardware, 
brad> scsi320 in a raid1 configuration. 

So long as nothing else is going on at the time.  =)

-Jeff