MultiVector class and MPI compatibility


I am converting an existing library to run in parallel with MPI. The library makes extensive use of the MultiVector class.
I have found that taking the InnerProduct of a MultiVector with parallel vectors computes the inner product of the local components for each rank. Is this the desired functionality or is MultiVector not compatible with MPI? Can I apply parallel operators, such as mumps inverse, to parallel MultiVectors?


[code]from netgen.geom2d import unit_square
from ngsolve import grad, dx
import ngsolve as ng
import netgen
from mpi4py import MPI
import numpy as np
import logging

format=‘%(asctime)s %(funcName)s:%(lineno)d\n%(message)s’

h = 0.1
if comm.rank == 0:
ngmesh = unit_square.GenerateMesh(maxh=h).Distribute(comm)
ngmesh = netgen.meshing.Mesh.Receive(comm)
mesh = ng.Mesh(ngmesh)

fes = ng.H1(mesh, order=3, dirichlet=“.*”)
u,v = fes.TnT()
a = ng.BilinearForm(grad(u)*grad(v)*dx)
with ng.TaskManager():
n = fes.ndof
logging.debug(f’Degrees of freedom of H1 on current rank: {n}\n’ +
f’FreeDofs bit array: {fes.FreeDofs()}\n’)

Create random value MultiVectors

gfu = ng.GridFunction(fes)
mv0 = ng.MultiVector(gfu.vec, 2)
mv1 = ng.MultiVector(gfu.vec, 2)
rng = np.random.default_rng()
random_vecs = rng.random((4, n))

mv0[0].FV().NumPy()[:] = random_vecs[0, :]
mv0[1].FV().NumPy()[:] = random_vecs[1, :]
mv1[0].FV().NumPy()[:] = random_vecs[2, :]
mv1[1].FV().NumPy()[:] = random_vecs[3, :]

MultiVector inner product test

inner = ng.InnerProduct(mv0, mv1)
logging.debug(f’Inner product of 2 MultiVectors:\n{inner}\n’)

Inner product of MultiVector local components

local_inner = np.zeros((2, 2))
for i in range(2):
for j in range(2):
local_inner[i, j] = ng.InnerProduct(mv0[i].local_vec, mv1[j].local_vec)
logging.debug(f’Inner product of 2 MultiVectors local components:\n{local_inner}\n’)

Manual inner product of MultiVector

manual_inner = np.zeros((2,2))
for i in range(2):
for j in range(2):
manual_inner = ng.InnerProduct(mv0, mv1[j])
logging.debug(f’Pairwise inner product of parallel vectors:\n{manual_inner}')


[code]2022-03-02 13:31:26,952 main:39
Degrees of freedom of H1 on current rank: 46
FreeDofs bit array: 0: 1111111111111111111111111111111111111111111111

2022-03-02 13:31:26,953 main:55
Inner product of 2 MultiVectors:
10.517 10.3929
10.6328 11.6122

2022-03-02 13:31:26,954 main:62
Inner product of 2 MultiVectors local components:
[[10.51703725 10.3928831 ]
[10.63276183 11.61222237]]

2022-03-02 13:31:26,955 main:69
Pairwise inner product of parallel vectors:
[[270.4623489 276.32998277]
[284.52705911 289.39745476]][/code]

Best regards,

Hi Austen,
thank you for the bug report, is now fixed,