https://ml-cheatsheet.readthedocs.io/en/latest/calculus.html#chain-rule

Basics and with math, but not too much: https://cs231n.github.io/neural-networks-1/

https://ml-cheatsheet.readthedocs.io/en/latest/index.html

• https://ml-cheatsheet.readthedocs.io/en/latest/forwardpropagation.html

Activation functions:

• Why:
  • Map input to 0..1
  • Inject non-linearity to the network for more complex learnable relationships
    • $Wx+b$ derived is W -> degree of 1
    • Higher complexity
• Resources:
  • Nice list and graphs:
    • https://towardsdatascience.com/everything-you-need-to-know-about-activation-functions-in-deep-learning-models-84ba9f82c253
    • https://ml-cheatsheet.readthedocs.io/en/latest/activation_functions.html