Added A General Swish Activation Function inNeural Networks#10415
Added A General Swish Activation Function inNeural Networks#10415tianyizheng02 merged 4 commits intoTheAlgorithms:masterfrom
Conversation
tianyizheng02
left a comment
There was a problem hiding this comment.
Thanks for your contribution, but could you modify neural_network/activation_functions/sigmoid_linear_unit.py rather than making a separate file for the general swish function? Since the SiLU function is just the swish function with a parameter of 1, I think it'd make much more sense if you could rename that file to swish.py and generalize the existing function.
|
@tianyizheng02 Sir, I added the general swish model in the Sigmoid Linear Unit Model and renamed it as 'swish.py'. Now in order to maintain learning through these algorithms, I didn't find it appealing enough to delete the existing SiLU function to let others know that it is a popular name for the swish function of parameter 1. Hence I just added both functions. Could you please look into this and merge my PR? |
|
Thanks for your contribution! |
Describe your change:
Checklist: