ESMF (Earth System Modeling Framework) v9

The Earth System Modeling Framework (ESMF) is a high-performance, flexible software infrastructure for building and coupling weather, climate, and related Earth science applications.

Overview 

ESMF Version: v9.0.0b06

Official Repository: https://github.com/esmf-org/esmf

Official Build Documentation: https://earthsystemmodeling.org/docs/nightly/develop/ESMF_usrdoc/node10.html

Important

This guide is specific to ESMF version 9.x. The build system and compilation methods may differ for other major versions. For example, older versions may only support Intel Classic compilers.

This guide demonstrates how to compile ESMF using the HPC module system with multiple compiler toolchains. ESMF v9.0.0b06 supports AMD AOCC and Intel Classic compilers.

Prerequisites 

Required Software Components 

ESMF requires the following software components:

C/CXX/Fortran compiler toolset (Intel Classic or AMD AOCC)
An MPI implementation (Intel MPI or OpenMPI)
NetCDF-C (with parallel I/O support)
NetCDF-Fortran
Parallel-NetCDF (PnetCDF)
ParallelIO (PIO)
HDF5 (with parallel I/O support)
CMake (for ParallelIO dependency)

Caution

As tested on 2025-12-09, the Intel OneAPI compilers (icx, icpx, ifx) are not compatible with ESMF v9.0.0b06 when used with either Intel MPI or OpenMPI for missing -lmpi++ linkage. There is no known workaround at this time.

Use Intel Classic compilers (icc, icpc, ifort) instead.

System Requirements 

Sufficient disk space (~3GB for source code and build)
Memory: At least 8GB RAM for serial compilation, more for parallelized builds
Time: Compilation takes approx. 1 hour

Supported Compiler Toolchains 

The following table summarizes the supported compiler toolchains and their configurations:

Compiler Toolchain Configuration
Configuration	AMD AOCC	Intel Classic
Compiler Module	`aocc/5`	`intel-oneapi-compilers-classic`
MPI Module	`openmpi/5`	`intel-oneapi-mpi`
`ESMF_COMPILER`	`aocc`	`intel`
`ESMF_COMM`	`openmpi`	`intelmpi`
`ESMF_C`	(unset)	`icc`
`ESMF_CXX`	(unset)	`icpc`
`ESMF_F90`	(unset)	`ifort`
Additional Flags	None	`CFLAGS="-diag-disable=10441"` `CXXFLAGS="-diag-disable=10441"`

Warning

Intel OneAPI Compatibility Issue: The Intel OneAPI compilers (icx, icpx, ifx) are currently not compatible with ESMF v9.0.0b06 when used with either Intel MPI or OpenMPI. There is no known workaround at this time. Use Intel Classic compilers (icc, icpc, ifort) instead.

Download ESMF Source Code 

# Create a working directory
mkdir -p ~/esmf-build
cd ~/esmf-build

# Clone ESMF repository
git clone https://github.com/esmf-org/esmf.git
cd esmf

# Checkout the desired version
git checkout v9.0.0b06

Compilation Steps 

Important

Do not compile on login nodes! Compilation is resource-intensive and should be performed on compute nodes.

Request Interactive Compute Node 

Before compiling, request an interactive session on a compute node. For detailed instructions and examples, see:

How to Request Interactive Sessions

Once the interactive session starts, you’ll be on a compute node where you can safely compile ESMF.

Note

For the value of ${SPACK_ROOT}, please refer to Spack Instances for the installation path.

Option 1: Build with AMD AOCC + OpenMPI 

This configuration uses the AMD Optimizing C/C++ and Fortran Compilers with OpenMPI.

# Navigate to ESMF source directory
cd ~/esmf-build/esmf

# Activate Spack environment
export SPACK_ROOT="/opt/shared/.spack-edge"
source "${SPACK_ROOT}/dist/bin/setup-env.sh" -y

# Load build tools first
module purge
module load cmake

# Load AMD AOCC compiler and OpenMPI
module load aocc/5
module load openmpi/5

# Load required libraries
module load hdf5
module load netcdf-c
module load netcdf-fortran
module load parallel-netcdf
module load parallelio

# Verify modules are loaded
module list

# Set ESMF build environment variables
export ESMF_DIR="$(pwd)"
export ESMF_SITE="hkust_hpc4"
export ESMF_OS="Linux"
export ESMF_COMPILER="aocc"
export ESMF_COMM="openmpi"
export ESMF_ABI="64"

# Set library paths
export ESMF_NETCDF="nc-config"
export ESMF_PNETCDF="pnetcdf-config"
export ESMF_PIO="external"
export ESMF_NUMA="ON"

# Compile ESMF (use all available cores)
make -j $(nproc) 2>&1 | tee hpc4_build.log

Option 2: Build with Intel Classic Compilers + Intel MPI 

This configuration uses the classic Intel compilers (icc, icpc, ifort) with Intel MPI.

# Navigate to ESMF source directory
cd ~/esmf-build/esmf

# Activate Spack environment
export SPACK_ROOT="/opt/shared/.spack-edge"
source "${SPACK_ROOT}/dist/bin/setup-env.sh" -y

# Load build tools first
module purge
module load cmake

# Load Intel Classic compilers and MPI
module load intel-oneapi-compilers-classic
module load intel-oneapi-mpi

# Load required libraries
module load hdf5
module load netcdf-c
module load netcdf-fortran
module load parallel-netcdf
module load parallelio

# Verify modules are loaded
module list

# Set ESMF build environment variables
export ESMF_DIR="$(pwd)"
export ESMF_SITE="hkust_hpc4"
export ESMF_OS="Linux"
export ESMF_COMPILER="intel"
export ESMF_COMM="intelmpi"
export ESMF_ABI="64"

# Explicitly set Intel Classic compilers
export ESMF_C="icc"
export ESMF_CXX="icpc"
export ESMF_F90="ifort"

# Suppress Intel Classic compiler deprecation warnings
export CFLAGS="-diag-disable=10441"
export CXXFLAGS="-diag-disable=10441"

# Set library paths
export ESMF_NETCDF="nc-config"
export ESMF_PNETCDF="pnetcdf-config"
export ESMF_PIO="external"
export ESMF_NUMA="ON"

# Set optimization level
export ESMF_BOPT="O"

# Compile ESMF library only (use all available cores)
make -j $(nproc) lib 2>&1 | tee hpc4_build.log

Tip

Compilation typically takes approximately 1 hour. You can monitor progress by watching the build log file.

Verify Compilation 

After compilation completes, verify that ESMF was built successfully. At the end of the build log, you should see a message similar to:

ESMF library built successfully on Mon Dec 8 17:39:53 HKT 2025
To verify, build and run the unit and system tests with: make check
or the more extensive: make all_tests

You can check the build status and library files as follows:

# Check build status
make info

# The output should show:
# - ESMF_VERSION_STRING
# - Compiler settings
# - Library locations

# Check for library files
ls -lh lib/

# You should see ESMF library files such as:
# - libesmf.a
# - libesmf.so (if shared libraries were built)

Tip

Verify Build Quality with Tests

To ensure ESMF was built correctly and all functionality works as expected, run the test suites:

Quick verification (recommended): Run make check to execute unit and system tests. This typically takes 15-30 minutes.
Comprehensive testing: Run make all_tests for the complete test suite, which includes additional validation tests. This may take several hours.

Both test commands should be run in the same environment (with the same modules loaded) used for compilation. Test failures may indicate compatibility issues or missing dependencies.

Using ESMF in Your Application 

Setting Up ESMF Environment 

After building ESMF, you need to set up the environment to use it in your applications:

# Set ESMF installation directory
export ESMFMKFILE=/path/to/esmf/lib/libO/Linux.intel.64.intelmpi.default/esmf.mk

# Verify the file exists
ls -l $ESMFMKFILE

Note

The path to esmf.mk depends on your build configuration. For example:

Intel: .../lib/libO/Linux.intel.64.intelmpi.default/esmf.mk
AMD AOCC: .../lib/libO/Linux.aocc.64.openmpi.default/esmf.mk

Check the lib/libO/ directory after compilation for the correct path corresponding to your ESMF_COMPILER, ESMF_COMM, and ESMF_ABI settings.

Compiling Applications with ESMF 

To compile applications that use ESMF, include the ESMF makefile fragment:

# Include ESMF configuration
include $(ESMFMKFILE)

# Use ESMF variables in your build
my_app: my_app.o
    $(ESMF_F90LINKER) $(ESMF_F90LINKOPTS) -o $@ $^ $(ESMF_F90LINKPATHS) $(ESMF_F90LINKLIBS)

my_app.o: my_app.F90
    $(ESMF_F90COMPILER) -c $(ESMF_F90COMPILEOPTS) $(ESMF_F90COMPILEPATHS) $<

ESMF Environment Variables Reference 

Core Build Variables 

Essential ESMF Environment Variables
Variable	Description	Example Value
`ESMF_DIR`	Root directory of ESMF source code	`$(pwd)`
`ESMF_SITE`	Site identifier for this build	`hkust_hpc4`
`ESMF_OS`	Operating system	`Linux`
`ESMF_COMPILER`	Compiler family (`aocc`, `intel`, etc.)	`intel`
`ESMF_COMM`	Communication layer (`openmpi`, `intelmpi`, `mpiuni`)	`intelmpi`
`ESMF_ABI`	Application Binary Interface (32 or 64 bit)	`64`

Compiler-Specific Variables 

Compiler Selection Variables
Variable	Description	Example Value
`ESMF_C`	C compiler executable	`icx` or `icc`
`ESMF_CXX`	C++ compiler executable	`icpx` or `icpc`
`ESMF_F90`	Fortran compiler executable	`ifx` or `ifort`

Library Configuration Variables 

Library Integration Variables
Variable	Description	Example Value
`ESMF_NETCDF`	NetCDF configuration method	`nc-config`
`ESMF_PNETCDF`	Parallel-NetCDF configuration method	`pnetcdf-config`
`ESMF_PIO`	ParallelIO integration (`external` or `internal`)	`external`
`ESMF_NUMA`	NUMA-awareness support (`ON` or `OFF`)	`ON`
`ESMF_BOPT`	Build optimization (`O` for optimized, `g` for debug)	`O`

For a complete list of ESMF environment variables and build options, refer to the official ESMF build documentation.

Performance Optimization 

Build Optimization Levels 

ESMF supports different optimization levels through the ESMF_BOPT variable:

# Optimized build (recommended for production)
export ESMF_BOPT="O"

# Debug build (for development and troubleshooting)
export ESMF_BOPT="g"

Compiler-Specific Optimizations 

For advanced users, you can add compiler-specific optimization flags:

Intel Compilers:

# Add to your build environment
export ESMF_F90COMPILEOPTS="-O3 -xHost -ip"
export ESMF_CXXCOMPILEOPTS="-O3 -xHost -ip"

AMD AOCC:

# Add to your build environment
export ESMF_F90COMPILEOPTS="-O3 -march=native"
export ESMF_CXXCOMPILEOPTS="-O3 -march=native"

Warning

Custom compiler flags may affect portability. Test thoroughly before deploying to production.

NUMA Optimization 

For systems with NUMA architecture, enabling NUMA support can improve performance:

export ESMF_NUMA="ON"

This is particularly beneficial for multi-socket systems where memory locality matters.

Support and Resources 

ESMF Documentation

Compatible Architectures

For a complete list of supported platforms, compilers, and configurations, see the Supported Platforms section in the official documentation.

Community Support

Version-Specific Notes

Important

This guide is specific to ESMF v9.x. For other versions:

ESMF v8.x may have different build requirements
Older versions may only support Intel Classic compilers
Check the version-specific documentation for compatibility information