Reverse mode
Running the chain rule backward from the output, which is how autodiff libraries get all gradients for about the cost of one extra pass.
Running the chain rule backward from the output, which is how autodiff libraries get all gradients for about the cost of one extra pass.