Dear NGSolve Developers,
I am trying to understand how ngsolve works on a hpc cluster. I launch the mpi_poisson.py from here with:
mpiexec -n 4 python3 mpi_poisson.py
on one of the nodes. I would have expected the code to print the dofs owned by each rank at the end and save the solution in 4 different vtk files as dictated by the partitions. However what i see is 4 copies of the code being run thus it’s always the rank 0 which owns all the dofs and the output being saved to a single vtk file:
Hello from rank 0 of 1
rank 0 has 1288 of 1288 dofs!
L2-error 1.3801833471401903e-07
Hello from rank 0 of 1
rank 0 has 1288 of 1288 dofs!
L2-error 1.380183347140096e-07
Hello from rank 0 of 1
rank 0 has 1288 of 1288 dofs!
L2-error 1.3801833471401694e-07
Hello from rank 0 of 1
rank 0 has 1288 of 1288 dofs!
L2-error 1.3801833471401066e-07
How do i make sure the mpi is enabled when using ngsolve on hpc? I can see that on the cluster, build/CMakeCache.txt has -DUSE_MPI=OFF as well as i can see that NETGEN_CMAKE_ARGS has:
-DCMAKE_CXX_COMPILER=/opt/gridware/depots/3dc76222/el7/pkg/compilers/gcc/8.3.0/bin/g++;-DCMAKE_C_COMPILER=/opt/gridware/depots/3dc76222/el7/pkg/compilers/gcc/8.3.0/bin/gcc;
I would have expected the c++ compiler to be mpicc and not g++. Is this the problem? I am attaching the CMakeCache.txt in build and build/netgen for your reference. Thanks a lot for your time and help!
https://ngsolve.org/media/kunena/attachments/1052/CMakeCache_buildnetgen.txt
https://ngsolve.org/media/kunena/attachments/1052/MakeCache_build.txt
Attachment: CMakeCache_buildnetgen.txt
Attachment: MakeCache_build.txt