-
Yaae (Yet another autodiff engine) is an automatic differentiation engine written in Numpy which comes with a small neural networks library. It supports scalar operations as well as tensors operations and comes with various functions such as exponential, relu, sigmoid ... For educational puprose only.
-
Here is my blog post explaining how an automatic differentiation works and my design/implementation choices.
- Let's compare a simple example using Yaae and Pytorch.
# Yaae. w1 = Node(2, requires_grad=True) w2 = Node(3, requires_grad=True) w3 = w2 * w1 w4 = w1.sin() w5 = w3 + w4 z = w5 z.backward() w1_yaae, w2_yaae, z_yaae = w1, w2, z # Pytorch. w1 = torch.Tensor([2]).double() w1.requires_grad = True w2 = torch.Tensor([3]).double() w2.requires_grad = True w3 = w2 * w1 w4 = w1.sin() w5 = w3 + w4 z = w5 z.backward() w1_torch, w2_torch, z_torch = w1, w2, z # Forward pass. assert z_yaae.data == z_torch.data.item() # True. # Backward pass. assert w1_yaae.grad.data == w1_torch.grad.item() # True. assert w2_yaae.grad.data == w2_torch.grad.item() # True.
- The files demo_regression.ipynb and demo_classification.ipynb are simple regression/classification problems solved using Yaae. As shown in the notebooks, here are the the results:
- If you are still skeptical, here is my GAN implemented with Yaae.
- Create a virtual environment in the root folder using virtualenv and activate it.
# On Linux terminal, using virtualenv. virtualenv my_yaae_env # Activate it. source my_yaee_env/bin/activate
- Install requirements.txt.
pip install -r requirements.txt
# Tidy up the root folder.
python3 setup.py clean