Build from source on Windows

Main Forum Old-Forum - do not post here
1

Hi, I’m getting a compile error when I try to build NGSolve from source on Windows 7 with Visual Studio 2015 C++ and Python 3.7. I get the same error if I try to build using the CMake command or from inside Visual Studio.

The error is:

c:\users\dow\ngsuite\src\external_dependencies\netgen\libsrc\core\type_traits.hpp(46): fatal error C1001: An internal error has occurred in the compiler. [C:\Users\Dow\ngsuite\build\netgen\netgen\libsrc\core\ngcore.vcxproj] [C:\Users\Dow\ngsuite\build\netgen\netgen.vcxproj] 8> (compiler file 'f:\dd\vctools\compiler\cxxfe\sl\p1\c\template.cpp', line 13144) 8> To work around this problem, try simplifying or changing the program near the locations listed above. 8> Please choose the Technical Support command on the Visual C++ 8> Help menu, or open the Technical Support help file for more information (compiling source file C:\Users\Dow\ngsuite\src\external_dependencies\netgen\libsrc\core\localheap.cpp) 8> c:\users\dow\ngsuite\src\external_dependencies\netgen\libsrc\core\type_traits.hpp(52): note: see reference to class template instantiation 'ngcore::detail::has_Range<std::T>' being compiled 8> with 8> [ 8> T=std::size_t 8> ] (compiling source file C:\Users\Dow\ngsuite\src\external_dependencies\netgen\libsrc\core\localheap.cpp) 8> c:\users\dow\ngsuite\src\external_dependencies\netgen\libsrc\core\type_traits.hpp(46): note: see reference to variable template 'const bool has_range<unsigned __int64>' being compiled (compiling source file C:\Users\Dow\ngsuite\src\external_dependencies\netgen\libsrc\core\localheap.cpp) 8> c:\users\dow\ngsuite\src\external_dependencies\netgen\libsrc\core\array.hpp(333): note: see reference to class template instantiation 'ngcore::T_Range<std::size_t>' being compiled (compiling source file C:\Users\Dow\ngsuite\src\external_dependencies\netgen\libsrc\core\localheap.cpp)
This is not an urgent issue for me, but I was wondering if there is a work-around. The nightly build works, so I’m guessing that there is just some issue with my build environment.

I’m attaching my CMakeCache.txt. Please let me know if there is more information I can provide.

Best,
Dow

Attachment: CMakeCache_2020-06-27.txt

2

I think VS2015 is too old, they do not support cpp17 fully afaik. And obviously crash on it… Can you use VS 2019? - community edition is free
Best
Christopher

3

Thanks, Christopher! That makes sense. I was following the instructions on the Build on Windows page, which maybe could be updated. They mention:

and

cmake "../src" -DCMAKE_INSTALL_PREFIX="BASEDIR/install" -G "Visual Studio 14 Win64"

Best,
Dow

EDIT: I replaced Python 3.7 with Python 3.8, assuming that the documentation was out of date in that respect also. Then this worked for me to build NGSolve:

 cmake "../src" -DCMAKE_INSTALL_PREFIX="BASEDIR/install" -G "Visual Studio 16 2019" -A "x64"

where BASEDIR in my case was C:/Users/Dow/ngsuite
Then I did the step to set environment variables:

cmake --build . --config Release --target set_environment_variables

This prepended C:/Users/Dow/ngsuite/install to my PYTHONPATH, but then netgen wasn’t able to find the python modules:

C:\Users\Dow>netgen
NETGEN-6.2.2006-8-g2800d6c2
Developed by Joachim Schoeberl at
2010-xxxx Vienna University of Technology
2006-2010 RWTH Aachen University
1996-2006 Johannes Kepler University Linz
optfile ./ng.opt does not exist - using default values
togl-version : 2
loading ngsolve library
NGSolve-6.2.2006-49-ga4dfcd3b
Using Lapack
Including sparse direct solver UMFPACK
Running parallel using 8 thread(s)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'ngsolve'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'ngsolve'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'netgen'
Thank you for using NGSolve[/code]

I tried changing the PYTHONPATH to include  [code]C:/Users/Dow/ngsuite/install/lib/site-packages[/code]
Then netgen ran correctly, and I was able to import ngsolve from the netgen python console and run a python script using [code]netgen myscript.py[/code]
The only remaining issue is that if I try to import ngsolve in a regular python shell or in a Jupyter notebook, or run my script using [code]python myscript.py[/code], I get the error:
[code]C:\Users\Dow>python test.py
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    from ngsolve import *
  File "C:\Users\Dow\ngsuite\install\lib\site-packages\ngsolve\__init__.py", lin
e 12, in <module>
    import netgen
  File "C:\Users\Dow\ngsuite\install\lib\site-packages\netgen\__init__.py", line
 13, in <module>
    from . import libngpy
ImportError: DLL load failed while importing libngpy: The specified module could
 not be found.

The package “C:\Users\Dow\ngsuite\install\lib\site-packages\netgen\libngpy.lib” exists as a static library, but the Python 3.8 interpreter seems to be expecting a dynamic load library (dll) perhaps?

Best,
Dow

4

Sorry – I was wrong yesterday when I reported that the build completed correctly. I deleted the build and install directories and repeated the build steps.

cmake "../src" -DCMAKE_INSTALL_PREFIX="BASEDIR/install" -G "Visual Studio 16 2019" -A "x64" cmake --build . --config Release --target install
This time I noticed an error in the build that I had missed.

... -- Installing: C:/UsersDow/ngsuite/install/Lib/site-packages/ngsolve/labextension/labextension -- Installing: C:/UsersDow/ngsuite/install/Lib/site-packages/ngsolve/labextension/labextension/plugin.js -- Installing: C:/UsersDow/ngsuite/install/Lib/site-packages/ngsolve/labextension/labextension/plugin.js.map -- Installing: C:/UsersDow/ngsuite/install/Lib/site-packages/ngsolve/labextension/package.json Traceback (most recent call last): File "c:\users\dow\appdata\local\programs\python\python38\lib\runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "c:\users\dow\appdata\local\programs\python\python38\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "C:\Users\Dow\AppData\Local\Programs\Python\Python38\Scripts\jupyter-nbextension.exe\__main__.py", line 7, in <module> File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\jupyter_core\application.py", line 270, in launch_instance return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs) File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\traitlets\config\application.py", line 664, in launch_instance app.start() File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 983, in start super(NBExtensionApp, self).start() File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\jupyter_core\application.py", line 259, in start self.subapp.start() File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 711, in start self.install_extensions() File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 682, in install_extensions full_dests = install(self.extra_args[0], File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 206, in install_nbextension_python m, nbexts = _get_nbextension_metadata(module) File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 1117, in _get_nbextension_metadata m = import_item(module) File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\traitlets\utils\importstring.py", line 42, in import_item return __import__(parts[0]) ModuleNotFoundError: No module named 'ngsolve' Traceback (most recent call last): File "c:\users\dow\appdata\local\programs\python\python38\lib\runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "c:\users\dow\appdata\local\programs\python\python38\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "C:\Users\Dow\AppData\Local\Programs\Python\Python38\Scripts\jupyter-nbextension.exe\__main__.py", line 7, in <module> File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\jupyter_core\application.py", line 270, in launch_instance return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs) File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\traitlets\config\application.py", line 664, in launch_instance app.start() File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 983, in start super(NBExtensionApp, self).start() File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\jupyter_core\application.py", line 259, in start self.subapp.start() File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 891, in start self.toggle_nbextension_python(self.extra_args[0]) File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 864, in toggle_nbextension_python return toggle(module, File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 477, in enable_nbextension_python return _set_nbextension_state_python(True, module, user, sys_prefix, File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 375, in _set_nbextension_state_python m, nbexts = _get_nbextension_metadata(module) File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\notebook\nbextensions.py", line 1117, in _get_nbextension_metadata m = import_item(module) File "c:\users\dow\appdata\local\programs\python\python38\lib\site-packages\traitlets\utils\importstring.py", line 42, in import_item return __import__(parts[0]) ModuleNotFoundError: No module named 'ngsolve' ...

I can’t find any error preceding this, but there were many warnings about “inconsistent dll linkage”.
I’m attaching the configure/build output along with my CMakeCache.txt

I also tried deleting the build and install directories and building in Visual Studio. The documentation says:

The only ‘.sln’ file in the build directory was ‘ngsuite.sln’, which I assume is the new name for ‘SUPERBUILD.sln’. I built the “INSTALL” project as instructed. The same warnings and errors were generated and the behavior of the installed application was the same (i.e. netgen runs but importing ngsolve in a python shell fails).

I don’t know what else to try at this point.

Best,
Dow

Attachment: build-output.txt

Attachment: CMakeCache_2020-06-29.txt

5

Hi Dow,

I think you need to set the environment variables (like PYTHONPATH):

cmake --build . --config Release --target set_environment_variables

I wasn’t aware this is now already needed during the build, we will move this up in the documentation.

Best,
Matthias

6

Thanks, Matthias!

Your point makes sense. But I think there may also be an issue with the directory that is being added to PYTHONPATH by set_environment_variables.

I just tried removing references to ngsolve directories from my PATH and PYTHONPATH, then running

cmake --build . --config Release --target set_environment_variables cmake --build . --config Release --target install
The build failed in the same way.

7

What happens when you do following?

python -c ‘import ngsolve’
echo %PYTHONPATH%

Best,
Matthias

8

I think the issue may be that the PYTHONPATH did not get sourced in the ‘git bash’ mingw shell I used to do the build. Maybe if I closed and re-opened the shell after doing

cmake --build . --config Release --target set_environment_variables

but before doing the install step it would work? I can try that.

In the mingw shell I used to perform the (failed) build, if I cd to my home directory:

$ pwd /c/Users/Dow $ python -c 'import ngsolve' Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'ngsolve'
If I go back to the build directory and try, I get

$ cd ngsuite/build
$ pwd
/c/Users/Dow/ngsuite/build
$ python -c 'import ngsolve'

So the import succeeds. If I do

$ echo $PYTHONPATH C:\Users\Dow;C:\Users\Dow\python;
(doesn’t have ngsolve).

Now if I open a new ‘git bash’ shell, I see

$ echo $PYTHONPATH C:\Users\Dow\ngsuite\install\;C:\Users\Dow;C:\Users\Dow\python;
so the environment variable is set, but in that same shell,

$ python -c 'import ngsolve' Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'ngsolve'
unless I cd to the build directory.

I see this same behavior if I open a regular command terminal

[code]C:\Users\Dow>python -c “import ngsolve”
Traceback (most recent call last):
File “”, line 1, in
ModuleNotFoundError: No module named ‘ngsolve’

C:\Users\Dow>cd ngsuite/build
C:\Users\Dow\ngsuite\build>python -c “import ngsolve”

C:\Users\Dow\ngsuite\build>echo %PYTHONPATH%
C:\Users\Dow\ngsuite\install;C:\Users\Dow;C:\Users\Dow\python;
[/code]

Best,
Dow

9

[quote]Maybe if I closed and re-opened the shell after doing
cmake --build . --config Release --target set_environment_variables
but before doing the install step it would work? I can try that.[/quote]
I tried this just now, but got the same error in the install step

Best,
Dow

10

Hi Dow,

In your case, PYTHONPATH should have the value
C:/UsersDow/ngsuite/install/Lib/site-packages
This is where the python packages/modules are installed. (See the output.txt you sent).

I will test on Windows if there is an issue with the set_environment_variables target in
CMake.

Best,
Matthias

11

Hi Dow,

I just tried it on Windows and it seems there is a problem with Python 3.8 (might be releated to Tcl/Tk).
Please downgrade to Python 3.7 and try again.

Best,
Matthias

12

Thanks, Matthias! Downgrading to Python 3.7 eliminated the build error.

Then to get things working, I just had to
add %NETGENDIR% to my path and
add C:\Users\dow\ngsuite\install\lib\site-packages to my PYTHONPATH

Best,
Dow