[LLVM] SLEEF and libsystem_m vector libraries support

[georgemitenkov]

This PR adds support for libsystem_m and SLEEF vector libraries. The first is supported by LLVM internally, so it comes for free with LLVM 13. For SLEEF, basic support was added for AArch64 architecture. We currently support only fixed type vectors and exponential/power functions.

Also, corresponding IR checks were added.

[bbpbuildbot]

Can one of the admins verify this patch?

[pramodk]

thanks! 👌

I will give a try locally!

[pramodk]

To test this:

• Install sleef from source

Spack users on BB5 can do : spack install sleef (updating sleef in BlueBrain/spack#1179)
I temporarily copied installation under /gpfs/bbp.cscs.ch/apps/hpc/llvm-install/0621/sleef-3.5.1/.

• And run test as

test.mod had following in DERIVATIVE block: mtau = exp(m) + exp(minf) + (m / minf)

$ ./bin/nmodl  ../test.mod -o output llvm --ir --vector-width 8 --veclib SLEEF --opt \
    benchmark --run --instance-size 10000000 --repeat 4 \
     --libs /gpfs/bbp.cscs.ch/apps/hpc/llvm-install/0621/sleef-3.5.1/lib64/libsleefgnuabi.so \
     --opt-level-ir 3 --opt-level-codegen  3  --backend default
...
[NMODL] [info] :: Benchmarking kernel 'nrn_state_hh' with 953.6744613647461 MBs dataset
[NMODL] [info] :: Experiment 0 compute time = 0.158433 sec
[NMODL] [info] :: Experiment 1 compute time = 0.049783 sec
[NMODL] [info] :: Experiment 2 compute time = 0.050062 sec
[NMODL] [info] :: Experiment 3 compute time = 0.049977 sec
...
$ ./bin/nmodl  ../test.mod -o output llvm --ir --vector-width 2 --veclib SLEEF --opt \
    benchmark --run --instance-size 10000000 --repeat 4 \
     --libs /gpfs/bbp.cscs.ch/apps/hpc/llvm-install/0621/sleef-3.5.1/lib64/libsleefgnuabi.so \
     --opt-level-ir 3 --opt-level-codegen  3  --backend default
...
[NMODL] [info] :: Running LLVM optimisation passes
[NMODL] [info] :: Running LLVM benchmark
[NMODL] [info] :: Created LLVM IR module from NMODL AST in 5.2643e-05 sec
[NMODL] [info] :: Backend: default
[NMODL] [info] :: Benchmarking kernel 'nrn_state_hh' with 953.6744613647461 MBs dataset
[NMODL] [info] :: Experiment 0 compute time = 0.416231 sec
[NMODL] [info] :: Experiment 1 compute time = 0.265908 sec
[NMODL] [info] :: Experiment 2 compute time = 0.264754 sec
[NMODL] [info] :: Experiment 3 compute time = 0.265929 sec
[NMODL] [info] :: Average compute time = 0.303206
[NMODL] [info] :: Compute time variance = 0.00425849
[NMODL] [info] :: Minimum compute time = 0.264754
[NMODL] [info] :: Maximum compute time = 0.416231
...

# SVML 
$ ./bin/nmodl  ../test.mod -o output llvm --ir --vector-width 8 --veclib SVML --opt benchmark --run --instance-size 10000000 --repeat 4  --libs /gpfs/bbp.cscs.ch/apps/hpc/llvm-install/0621/sleef-3.5.1/lib64/libsleefgnuabi.so /gpfs/bbp.cscs.ch/ssd/apps/hpc/jenkins/deploy/compilers/2021-01-06/linux-rhel7-x86_64/gcc-4.8.5/intel-20.0.2-ilowey/compilers_and_libraries_2020.2.254/linux/compiler/lib/intel64_lin/libsvml.so --opt-level-ir 3 --opt-level-codegen  3  --backend default
....
[NMODL] [info] :: Running LLVM optimisation passes
[NMODL] [info] :: Running LLVM benchmark
[NMODL] [info] :: Created LLVM IR module from NMODL AST in 5.2221e-05 sec
[NMODL] [info] :: Backend: default
[NMODL] [info] :: Benchmarking kernel 'nrn_state_hh' with 953.6744613647461 MBs dataset
[NMODL] [info] :: Experiment 0 compute time = 0.142812 sec
[NMODL] [info] :: Experiment 1 compute time = 0.040704 sec
[NMODL] [info] :: Experiment 2 compute time = 0.040823 sec
[NMODL] [info] :: Experiment 3 compute time = 0.039614 sec

And .ll file has correct replacement:

$ grep exp output/v8_test_opt.ll | grep exp
@llvm.compiler.used = appending global [1 x i8*] [i8* bitcast (<8 x double> (<8 x double>)* @_ZGVeN8v_exp to i8*)], section "llvm.metadata"
  %44 = tail call double @llvm.exp.f64(double %40)
  %45 = tail call double @llvm.exp.f64(double %42)
  %69 = tail call <8 x double> @_ZGVeN8v_exp(<8 x double> %64)
  %70 = tail call <8 x double> @_ZGVeN8v_exp(<8 x double> %67)
  %89 = tail call double @llvm.exp.f64(double %85)
  %90 = tail call double @llvm.exp.f64(double %87)
  %106 = tail call double @llvm.exp.f64(double %102)
  %107 = tail call double @llvm.exp.f64(double %104)
declare double @llvm.exp.f64(double) #3
declare <8 x double> @_ZGVeN8v_exp(<8 x double>) #3
...
$ grep exp output/v2_test_opt.ll | grep exp
@llvm.compiler.used = appending global [1 x i8*] [i8* bitcast (<2 x double> (<2 x double>)* @_ZGVbN2v_exp to i8*)], section "llvm.metadata"
  %44 = tail call double @llvm.exp.f64(double %40)
  %45 = tail call double @llvm.exp.f64(double %42)
  %69 = tail call <2 x double> @_ZGVbN2v_exp(<2 x double> %64)
  %70 = tail call <2 x double> @_ZGVbN2v_exp(<2 x double> %67)
  %89 = tail call double @llvm.exp.f64(double %85)
  %90 = tail call double @llvm.exp.f64(double %87)
  %106 = tail call double @llvm.exp.f64(double %102)
  %107 = tail call double @llvm.exp.f64(double %104)
declare double @llvm.exp.f64(double) #3
declare <2 x double> @_ZGVbN2v_exp(<2 x double>) #3

Didn't check performance or any other aspects in too detail though.

cc: @iomaganaris @castigli

[pramodk]

LGTM!

@georgemitenkov : check if my changes are good and then good to go for merge!

[pramodk]

Also note that LLVM install path on BB5 changed with new version:

-DLLVM_DIR=/gpfs/bbp.cscs.ch/apps/hpc/llvm-install/0621/lib/cmake/llvm

[georgemitenkov]

@pramodk Everything looks good! Thanks for fixing the tests:) I will merge this then