parallel build linker issue

Hi, I’m trying a cmake configuration script like this on Ubuntu 20.04:

export BASEDIR=$HOME/ngsuite rm -rf $BASEDIR/ngsolve-build rm -rf $BASEDIR/ngsolve-install mkdir $BASEDIR/ngsolve-build mkdir $BASEDIR/ngsolve-install cd $BASEDIR/ngsolve-build cmake -DUSE_JPEG:BOOL=ON \ -DUSE_MKL:BOOL=ON \ -DMKL_ROOT:PATH=$HOME/intel/oneapi/mkl/2021.2.0 \ -DUSE_UMFPACK:BOOL=ON \ -DUSE_MPI:BOOL=ON \ -DUSE_MUMPS:BOOL=ON \ -DCMAKE_INSTALL_PREFIX:PATH=${BASEDIR}/ngsolve-install ${BASEDIR}/ngsolve-src

I really just wanted to try MUMPS as a sparse solver in a non-MPI context, but I don’t mind having a parallel build. I noticed that if the MUMPS flag is set without the MPI flag, CMake fails to configure

CMake Error at cmake/external_projects/mumps.cmake:9 (find_package):
By not providing “FindPARMETIS.cmake” in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by “PARMETIS”,
but CMake did not find one.

Could not find a package configuration file provided by “PARMETIS” with any
of the following names:

PARMETISConfig.cmake
parmetis-config.cmake

Add the installation prefix of “PARMETIS” to CMAKE_PREFIX_PATH or set
“PARMETIS_DIR” to a directory containing one of the above files. If
“PARMETIS” provides a separate development package or SDK, be sure it has
been installed.
Call Stack (most recent call first):
cmake/SuperBuild.cmake:212 (include)
CMakeLists.txt:57 (include)

If I remove both MUMPS and MPI, the build always works, and the Pardiso inverse from MKL is available. I have a local Pardiso installed from Intel OneAPI

With the configuration at the top of this post (both MUMPS and MKL set), I don’t see any issues in the configuration step, Netgen and NGSolve build except for the final step when NGSolve gives a linker error about missing libblacs-related definitions.

[ 99%] Linking CXX executable ngs
/usr/bin/ld: …/linalg/libngla.so: undefined reference to blacs_gridinfo_' /usr/bin/ld: ../linalg/libngla.so: undefined reference to pzpotrf_’
/usr/bin/ld: …/linalg/libngla.so: undefined reference to pdpotrf_' /usr/bin/ld: ../linalg/libngla.so: undefined reference to pdgetrs_’
/usr/bin/ld: …/linalg/libngla.so: undefined reference to blacs_gridexit_' /usr/bin/ld: ../linalg/libngla.so: undefined reference to numroc_’
/usr/bin/ld: …/linalg/libngla.so: undefined reference to pdgetrf_' /usr/bin/ld: ../linalg/libngla.so: undefined reference to pzgetrs_’
/usr/bin/ld: …/linalg/libngla.so: undefined reference to blacs_gridinit_' /usr/bin/ld: ../linalg/libngla.so: undefined reference to descinit_’
/usr/bin/ld: …/linalg/libngla.so: undefined reference to pzgetrf_' /usr/bin/ld: ../linalg/libngla.so: undefined reference to pdpotrs_’
/usr/bin/ld: …/linalg/libngla.so: undefined reference to `pzpotrs_’
collect2: error: ld returned 1 exit status

I have openmpi3 installed through apt. I tried installing scalapack through apt also, but that didn’t seem to help with the linker issue. I see libmkl-blacs-openmpi packages available, but I haven’t tried them. I’m not sure if my installation of the Intel OneAPI might also include a blacs implementation.

If anyone has ideas on how to get past this linker error, that would be great. I’m attaching a file with the full build output.

Thanks!
Dow

Attachment: build-dump.txt

Hi Dow,

I could solve this mumps-related linker issue by using this cmake flag:

-DSCALAPACK_LIBRARY=/usr/lib/x86_64-linux-gnu/libscalapack-openmpi.so

This worked for me on Ubuntu 18.04, but I guess it would work on any distro with any scalapack library.

Best,
Julius

Thanks Julius! That makes perfect sense. I assumed CMake would have found it in such a standard location, and must not have checked the CMakeCache.txt carefully enough.

Best,
Dow