Dear users of the fat node system at LRZ, earlier known as "SuperMIG", due to the integration of the fat node system into the SuperMUC infrastructure, there will be some items you will need to take care of before re-starting your job processing on this part of the system. 1. Login procedure: The name via which the login node is accessed has changed to supermuc-fat.lrz.de 2. Scratch and Project Data Handling: Instead of the previously used NAS volumes, the fat nodes are now provisioned with the same GPFS file system as the thin nodes. The NAS volumes will remain available in Read-Only mode until mid-November 2013. Please migrate any still needed data to GPFS by copying them to GPFS on a login node. For example: cd $SCRATCH cp -a $SCRATCH_MIGRATION/<dirname> <new_dirname> or cd $WORK cp -a $WORK_MIGRATION/<dirname> <new_dirname> It is probably a good idea for <new_dirname> to have the same value as <dirname> (if no name clashes occur). The paths pointed to by the variables SCRATCH_MIGRATION and WORK_MIGRATION will be available only on the login nodes. Also do not forget to adjust your batch scripts if you either have explicit path names in them, or if your directory structure has changed through the copying procedure. 3. LoadLeveler job classes and scripts: Because the same instance of LoadLeveler as for the thin nodes is now used, you will need to adjust the job class names for scripts that should run on the fat nodes: Please replace "general" by "fat" and "test" by "fattest". For jobs in the "special" class, please add the specification #@ requirements (Island == "island01") to force them to run on the fat node island. 4. Software updates: A new version of the operating system as well as the HPC software stack is now deployed. In particular, the MPICH2-based version of IBM MPI (PE 1.3) replaces the legacy 5.2 version. We consider it advisable to recompile all applications that previously were built for the fat nodes (see also next section). 5. Compilation procedure: Due to the different architecture of the thin and fat nodes, an executable that is compiled with defaults on the fat node does not take advantage of the AVX instructions on the thin nodes. Therefore we recommend to compile your code with "-axAVX -xSSE4.2" to target both types. This option tells the compiler to generate multiple, processor-specific auto-dispatch code paths for Intel processors. For details consult the following documents: + http://www.lrz.de/services/compute/supermuc/access_and_login/ + http://www.lrz.de/services/compute/supermuc/loadleveler/ + http://www.lrz.de/services/compute/supermuc/filesystems/ This information is also available on our web server http://www.lrz-muenchen.de/services/compute/supermuc/aktuell/ali4626/ Reinhold Bader