Automatic differentiation discussions

[yewalenikhil65]

@pravinjagtap @Nikesh14

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.

➡️ You can convert issues to discussions either individually or bulk by labels. Looking at you, issues labeled “question” or “discussion”.

Follow guidelines strictly - Write pseudocode only for proposing ideas , preferably following MATLAB/Julia like notations. For the topics that do not involve parallelization as focus but only as demonstrations of concepts, we will commit in Julia and/or MATLAB that are much readable.

[yewalenikhil65]

To-do task list for automatic differentiation(AD)
Expect demonstration to be in a form of blog on github gist or as plain markdown documentation, which can be easily linked our repository later

For viewing docs, make sure you have added a chrome extension to view rendered math equations in your chrome browser

• Snag in finite differencing --> improvement with Complex step differentiation. Here.
• Motivation from Complex step differentiation --> Dual numbers(Introduction and demo)
• Automatic Differentiation(AD), chain rule(Introduction and demo using dual numbers ?)
  • Forward mode AD
  • Reverse mode AD
  • Backpropagation using Vector-Jacobian Products
• How finite -difference fares with AD ?
• Conclude

[yewalenikhil65]

@pravinjagtap @Nikesh14 @Satyajit92 updated a doc.. see above comment and a competed task sometime back. You can edit it if u want

[pravinjagtap]

Hello @yewalenikhil65,
You did a great job explaining it comprehensively.