HLRB-II: Recommendations on tuning of large scale I/O

[LRZ aktuell]

Recently, we have seen intermittent degradation of I/O performance on
 the HLRB-II. The exact root cause of this degradation is being
 investigated. However, there are measures that you as the users of the
 system can take to optimize your own program's I/O performance.
 
 For tuning of I/O, SGI provides a special shared library called FFIO
 (Fast and Flexible I/O), the use of which is recommended. In
 particular, FFIO allows to perform unbuffered I/O, which for large
 files should give much better performance. The FFIO facility requires
 the following information, which is stored inside environment
 variables:
 
  1. A pattern for the file names for which FFIO should be used
  2. Some parameters for tuning (note that FFIO will need additional
     memory resources)
 
 For running MPI programs together with FFIO a special run script
 mpiexec_ffio must be used.
 
 A more detailed description on how to use FFIO is available at http://
 www.lrz.de/services/compute/hlrb/files/ under the heading Efficient use
 of IO -> FFIO layer.
 
 If you encounter any difficulties with using FFIO, or if you see that I
 /O performance does not improve even with FFIO switched on, please
 contact the LRZ service desk and provide a detailed description of your
 problem.


 This information is also available on our web server
 http://www.lrz-muenchen.de/services/compute/hlrb/aktuell/ali3965/

 Reinhold Bader