autograd-engine

A minimal scalar-valued autograd engine built from scratch in Python — similar to Karpathy's micrograd.

What it does

Implements reverse-mode automatic differentiation (backpropagation) on a dynamically built computation graph. Each Value node tracks its data, gradient, and how it was created, so gradients can flow backward through any expression.

Supported operations

Operation	Method
Addition	__add__, __radd__
Subtraction	__sub__, __rsub__
Multiplication	__mul__, __rmul__
Division	__truediv__, __rtruediv__
Power	__pow__
ReLU	relu()
Negation	__neg__

How it works

from source import Value

a = Value(2.0)
b = Value(-3.0)
c = Value(10.0)

d = a * b + c
d.backward()

print(a.grad)  # ∂d/∂a
print(b.grad)  # ∂d/∂b

Calling .backward() performs a topological sort of the computation graph and applies the chain rule at each node in reverse order.

Visualization

Uses Graphviz to render the computation graph with data and gradient values at each node.

from utils.graph import draw_dot
draw_dot(d)

Requirements

graphviz
numpy

Structure

source.ipynb   # Value class + examples
utils/
  graph.py     # computation graph visualization