add mkldnn softmax backward by rongzha1 · Pull Request #17170 · apache/mxnet

rongzha1 · 2019-12-25T08:19:19Z

Description

add mkldnn softmax backward implementation
unitest pass

Should fix #13365

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at https://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

Feature1, tests, (and when applicable, API doc)
Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

@PatricZhao @TaoLv

TaoLv

Please take a look at #17152 and cache primitive for backward pass. Thanks!

TaoLv · 2019-12-25T08:30:39Z

+    { MKLDNN_ARG_DIFF_SRC, *out_mem.second },
+  };
+
+  stream->RegisterPrimArgs(bwd_pd, args);


will change it to cache primitive pd.

I mean here you need give a primitive not a primitive descriptor. Please check the definition of RegisterPrimArgs.

yes, you're right.
Although it is a pd, it call implicit constructor
inline primitive::primitive(const primitive_desc &pd) : primitive(pd.get()) {}
to get a primitive

I will make it more clearly by adding constructor before used.

TaoLv · 2019-12-25T08:31:17Z

+                                    const std::vector<NDArray>& inputs,
+                                    const std::vector<OpReqType>& req,
+                                    const std::vector<NDArray>& outputs) {
+  // It seems MKLDNN softmax doesn't support training.


Can you elaborate?

Will remove this out-dated comments

TaoLv · 2019-12-25T08:32:03Z

                                      std::vector<int> *in_attrs,
                                      std::vector<int> *out_attrs) {
  const SoftmaxParam& param = nnvm::get<SoftmaxParam>(attrs.parsed);
-  CHECK_EQ(in_attrs->size(), (param.use_length.value()) ? 2U : 1U);


Why remove this?

this check will result in backward check fail.
Will recover this check and add another function for backward.

pengzhao-intel · 2020-01-03T00:50:22Z

@rongzha1 please retrigger the PR @TaoLv could you help review again?

TaoLv · 2020-01-14T15:21:45Z

+  std::shared_ptr<mkldnn::softmax_backward> bwd_;
+};
+
+typedef ParamOpSign<SoftmaxParam> MKLDNNSoftmaxSignature;


Same as L99?

exactly same, add extra one when rebase, will remove this line Thanks

TaoLv · 2020-01-14T15:23:39Z

+void MKLDNNSoftmaxBackward(const nnvm::NodeAttrs& attrs,
+                           const OpContext &ctx,
+                           const std::vector<NDArray> &in_data,
+                           const std::vector<OpReqType>& req,


Suggested change

const std::vector<OpReqType>& req,

const std::vector<OpReqType> &req,

TaoLv · 2020-01-14T15:37:09Z

+  auto data_mem = in_data[1].GetMKLDNNData();
+  auto bwd = GetSoftmaxBwd(param, axis, in_data, out_data);
+
+  auto out_mem = CreateMKLDNNMem(out_data[0], bwd.pd.diff_src_desc(), req[0]);


Please check if you want to support req=kAddTo;

softmax backward primitive should support in-place calculation so no need to create additional buffer.

you are right, create mem is used to support kAddTo

Does the original softmax backward support kAddTo?

yes, original softmax bwd support kAddTo : SoftmaxGradCompute-->SoftmaxGrad -->KERNEL_ASSIGN(igrad[base + j*sa], Req, final_result);

pengzhao-intel · 2020-02-08T10:05:17Z

@TaoLv were all concerns resolved now?

* add mkldnn softmax backward * add primitive cache for softmax bwd * fix preci failed test * rm duplicate line

TaoLv suggested changes Dec 25, 2019

View reviewed changes

TaoLv added the MKLDNN label Dec 25, 2019

TaoLv reviewed Jan 14, 2020

View reviewed changes

rongzha1 added 4 commits January 16, 2020 10:04

add mkldnn softmax backward

9c7ef9e

add primitive cache for softmax bwd

98efa1e

fix preci failed test

7a76d7e

rm duplicate line

05302b0

TaoLv approved these changes Feb 9, 2020

View reviewed changes

TaoLv merged commit 9883b99 into apache:master Feb 9, 2020

TaoLv mentioned this pull request Feb 12, 2020

Fix storage type infer of softmax backward #17576

Merged

7 tasks

zheyuye pushed a commit to zheyuye/incubator-mxnet that referenced this pull request Feb 19, 2020

add mkldnn softmax backward (apache#17170)

d9f5165

* add mkldnn softmax backward * add primitive cache for softmax bwd * fix preci failed test * rm duplicate line

anirudh2290 pushed a commit to anirudh2290/mxnet that referenced this pull request May 29, 2020

add mkldnn softmax backward (apache#17170)

17b46c8

* add mkldnn softmax backward * add primitive cache for softmax bwd * fix preci failed test * rm duplicate line

	const std::vector<OpReqType>& req,
	const std::vector<OpReqType> &req,

Conversation

rongzha1 commented Dec 25, 2019 • edited by TaoLv Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Essentials

Changes

Comments

Uh oh!

TaoLv left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pengzhao-intel commented Jan 3, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pengzhao-intel commented Feb 8, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rongzha1 commented Dec 25, 2019 •

edited by TaoLv

Loading