JUROPA MPI Buffer on demand

To enable huge runs with lots of MPI ranks you have to disable the per default allocated all-to-all send buffer on the NEC- Nehalem Cluster Juropa at FZ Jülich.

Here is an excerpt from the official docu:

Most MPI programs do not need every connection

• Nearest neighbor communication
  
• Scatter/Gather and Allreduce based on binary trees
  
• Typically just a few dozen connections when having hundreds of
  processes
  
• ParaStation MPI supports this with ?on demand connections?
  
• export PSP_ONDEMAND=1
  
• was used for the Linpack runs (np > 24000)
  
• mpiexec --ondemand
  
• Backdraw
  
• Late all-to-all communication might fail due to short memory
  
• Default on JuRoPA is not to use ?on demand connections?
  

Links

Juropa Introduction @ FZJ