Interaction with the Message-passing Interface

To improve performance of cluster applications, it is critical for Intel® oneAPI Math Kernel Library to use the optimal number of threads, as well as the correct thread affinity. Usually, the optimal number is the number of available cores per node divided by the number of MPI processes per node. You can set the number of threads using one of the available methods, described inTechniques to Set the Number of Threads.

If the number of threads is not set, Intel® oneAPI Math Kernel Library checks whether it runs under MPI provided by the Intel® MPI Library. If this is true, the following environment variables define Intel® oneAPI Math Kernel Library threading behavior:

• I_MPI_THREAD_LEVEL
• MKL_MPI_PPN
• I_MPI_NUMBER_OF_MPI_PROCESSES_PER_NODE
• I_MPI_PIN_MAPPING
• OMPI_COMM_WORLD_LOCAL_SIZE
• MPI_LOCALNRANKS

The threading behavior depends on the value of I_MPI_THREAD_LEVEL as follows:

• 0 or undefined.

  Intel® oneAPI Math Kernel Library considers that thread support level of Intel MPI Library isMPI_THREAD_SINGLE and defaults to sequential execution.

• 1, 2, or 3.

  This value determines Intel® oneAPI Math Kernel Library conclusion of the thread support level:

  • 1 - MPI_THREAD_FUNNELED
  • 2 - MPI_THREAD_SERIALIZED
  • 3 - MPI_THREAD_MULTIPLE

  In all these cases, Intel® oneAPI Math Kernel Library determines the number of MPI processes per node using the other environment variables listed and defaults to the number of threads equal to the number of available cores per node divided by the number of MPI processes per node.