Running functions from MOD files via LLVM JIT

[georgemitenkov]

This is a draft PR to gather some feedback and suggestions :)

It introduces:

1. A new JITDriver class that, given a LLVM IR module, set ups the JIT compiler and is able to look up a function and execute it. Currently, the result of the execution is printed to stdout and can be a double only.
2. nmodl_llvm_runner executable that can take a mod file and a specified entry-point function, and run it via LLVM code generation pipeline and JITDriver.

For example, for MOD file like

// foo.mod
FUNCTION one() {
    one = 1
}

FUNCTION bar() {
    bar = one() + exp(1)
}

running nmodl_llvm_runner -f foo.mod -e bar gives

Result: 3.718282

Current status:
[x] An executable to run functions from MOD files
[x] JIT class to wrap around LLVM's JIT, which can be used in executable(mentioned above) or testing
[ ] Function results and optimisations tests

fixes #482

[bbpbuildbot]

Can one of the admins verify this patch?

[pramodk]

@georgemitenkov : which version of cmake-format you have? it should be 0.6.0. For this PR, on my machine I see:

± |georgemitenkov/llvm-jit {2} U:1 ✗| → cmake-format --version
0.6.0

± |georgemitenkov/llvm-jit {2} U:1 ✗| → make cmake-format
Built target cmake-format

± |georgemitenkov/llvm-jit {2} U:2 ✗| → git diff
diff --git a/ext/pybind11 b/ext/pybind11
--- a/ext/pybind11
+++ b/ext/pybind11
@@ -1 +1 @@
-Subproject commit f1abf5d9159b805674197f6bc443592e631c9130
+Subproject commit f1abf5d9159b805674197f6bc443592e631c9130-dirty
diff --git a/src/codegen/llvm/CMakeLists.txt b/src/codegen/llvm/CMakeLists.txt
index e8d4296..8cfc269 100644
--- a/src/codegen/llvm/CMakeLists.txt
+++ b/src/codegen/llvm/CMakeLists.txt
@@ -3,8 +3,7 @@
 # =============================================================================
 set(LLVM_CODEGEN_SOURCE_FILES
     ${CMAKE_CURRENT_SOURCE_DIR}/codegen_llvm_visitor.cpp
-    ${CMAKE_CURRENT_SOURCE_DIR}/codegen_llvm_visitor.hpp
-    ${CMAKE_CURRENT_SOURCE_DIR}/jit_driver.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/codegen_llvm_visitor.hpp ${CMAKE_CURRENT_SOURCE_DIR}/jit_driver.cpp
     ${CMAKE_CURRENT_SOURCE_DIR}/jit_driver.hpp)

 # =============================================================================
@@ -41,6 +40,5 @@ endif()
 # =============================================================================

 if(NOT NMODL_AS_SUBPROJECT)
-  install(TARGETS nmodl_llvm_runner
-          DESTINATION ${NMODL_INSTALL_DIR_SUFFIX}bin/llvm)
+  install(TARGETS nmodl_llvm_runner DESTINATION ${NMODL_INSTALL_DIR_SUFFIX}bin/llvm)
 endif()

[pramodk]

I quickly skimmed though, very nice! ⭐

I will take a detailed look at JIT driver tomorrow.

[georgemitenkov]

cmake-format

That's the issue then! I have 0.6.13

[pramodk]

@georgemitenkov : I see following error in CI :

/jenkins/05/workspace/hpc.nmodl/platform/bb5/src/codegen/llvm/jit_driver.cpp(38): error: no suitable user-defined conversion from "lambda [](llvm::orc::JITTargetMachineBuilder)->llvm::Expected<std::unique_ptr<<error-type>, std::default_delete<<error-type>>>>" to "llvm::orc::LLJITBuilderState::CompileFunctionCreator" exists
          llvm::orc::LLJITBuilder().setCompileFunctionCreator(compile_function_creator).create());
                                                         ^

Which LLVM version you are using? master or 11.0 ? If it's hassle to be compatible with 9.0, we can just drop it and move to 11.0. I can update our Jenkins CI.

[georgemitenkov]

I am using LLVM 11. I think it makes sense to upgrade to 11 on CI? As far as I am aware LLVM tries to keep up to date with new C++ standards, so lambdas, callbacks, etc. are used more and more frequently.

If that's a problem, I can try to rewrite it in the more standard way too.

[pramodk]

If that's a problem, I can try to rewrite it in the more standard way too.

I would upgrade to v11.0 in CI for two reasons:

• We don't expect end users of NMODL to build against LLVM (this will be pre-build package)
• LLVM API changes (often) and I don't see need to spend efforts to make it backward compatible

I will update CI shortly and re-trigger build.

[pramodk]

Retest this please

[pramodk]

@georgemitenkov : overall, this looks sufficiently good starting point and ready to merge in llvm.

I have merged rest of my PRs into llvm branch. I haven't rebased and pushed this PR branch in case you have any local commits. You can rebase on llvm branch and push it. There are two minor conflicts which should be easy to solve.

[georgemitenkov]

@pramodk I have rebased on your recent PR, so this should be ready for the merge now!

I have made some additions to the previous draft version, particularly:

• Now there is a Runner class that wraps around JIT driver. It helps to initialise JIT with LLVM IR module only once, and then we can run multiple functions from it. For example:

// Create runner only once
Runner runner(...);

runner.run("foo");
runner.run("bar");
...

• Set up framework for some other return types with templates, but currently we only use doubles.
• Added some tests to check the results of executing LLVM IR module functions
• Added tests to verify optimisation does not break anything. This may be changed if we change the way we optimise our IR.

[pramodk]

2 minimal comments, LGTM!

Feel free to squash & merge.