I am reading this paper and wanted to replicate some of the results as part of a learning process. I'm more of a digital guy but obviously have had some analog design as part of my studies.
Implementing a neural network model to learn the XOR operation is quite easy in Python, for example. In the analog domain, you have to worry about (1) how to multiply the activations with the weights (2) how to add the result (3) how to apply a non-linear function to the sum (4) how to propagate the errors from the output to the inputs (5) ensure that the amplitude does not degrade as it flows through the network in both directions.
The paper I mentioned above offers a solution to the above problem in the form of the block diagram shown below:
Multiplication and addition is done using a resistive array. This is straight-forward to do (for just the inference).
A sigma non-linearity is applied using a diodes.
Bidirectional amplifiers ensure that the signal quality is maintained in both directions.
A current source at the output injects some current into the network in proportion to the error. This is only done during training.
The author of the paper did the analog simulation for inference in SPICE, read-out the results in Python, computed the error in Python, applied a current in proportion to the error, and allowed the circuit to run again in SPICE.
The result of the implementation of the XOR operation is shown below:
I would like to replicate the results for the system shown above. Even though I have taken a few courses in analog electronics, they have been focused more towards theory and my analog design skills are not great.
My main question is designing the neuron as shown in the first image. How do I design the non-linearity and bidirectional amplifier ?