Improve NNPack Binding

[tqchen]

Cross ref @Maratyszcza 's comment at #2986

• Use new cmake system for latest version
• Use pre-allocated workspace
• Caching packing(might need changes in nnvm)
• Possible fused kernel pattern

The first two items seems to be low hanging fruit and can be done easily.

[dabraude]

I don't mind adding NNPack to the CMake. Only question I have is should NNPack be a submodule under 3rd party?

I don't know enough about the internals to do the rest.

[Maratyszcza]

@dabraude if you want to build NNPACK together with MXNet, you'd need to put it as a submodule. Most likely, you'd also want to submodule NNPACK dependencies (otherwise they will be downloaded at configuration time, and many people don't like that):

• Maratyszcza/FP16
• Maratyszcza/FXdiv
• Maratyszcza/cpuinfo
• Maratyszcza/psimd
• Maratyszcza/pthreadpool
• Maratyszcza/PeachPy
• benjaminp/six
• PeachPy/enum34

Then you setup some variables to tell NNPACK where to find these dependencies, and call add_subdirectory, which would create nnpack target in CMake. See https://github.com/caffe2/caffe2/blob/c787c79f180dc283d0af28674e0c1472168c7216/cmake/External/nnpack.cmake#L55-L86 for reference

[Roshrini]

@sandeep-krishnamurthy Can you add labels

• Feature Request
• Call for contribution