A simple analog neural network to implement the XOR operation

Question

I am reading this paper and wanted to replicate some of the results as part of a learning process. I'm more of a digital guy but obviously have had some analog design as part of my studies.

Implementing a neural network model to learn the XOR operation is quite easy in Python, for example. In the analog domain, you have to worry about (1) how to multiply the activations with the weights (2) how to add the result (3) how to apply a non-linear function to the sum (4) how to propagate the errors from the output to the inputs (5) ensure that the amplitude does not degrade as it flows through the network in both directions.

The paper I mentioned above offers a solution to the above problem in the form of the block diagram shown below:

1. Multiplication and addition is done using a resistive array. This is straight-forward to do (for just the inference).

2. A sigma non-linearity is applied using a diodes.

3. Bidirectional amplifiers ensure that the signal quality is maintained in both directions.

4. A current source at the output injects some current into the network in proportion to the error. This is only done during training.

The author of the paper did the analog simulation for inference in SPICE, read-out the results in Python, computed the error in Python, applied a current in proportion to the error, and allowed the circuit to run again in SPICE.

The result of the implementation of the XOR operation is shown below:

I would like to replicate the results for the system shown above. Even though I have taken a few courses in analog electronics, they have been focused more towards theory and my analog design skills are not great.

My main question is designing the neuron as shown in the first image. How do I design the non-linearity and bidirectional amplifier ?

Answer 1

The paper has an appendix where they name the elements they used and a few other details. I'll just focus on your question:

How do I design the non-linearity and bidirectional amplifier ?

For the nonlinearity, they added two anti-parallel diodes with voltage sources for pre-polarization. For diodes, appendix C.2 gives Is=1u and n=2, so you can add a diode with this .model card:

.model D D Is=1u n=2 ; Rs=10m Cjo=10p

The extra Rs and Cjo are minor additions that can help with convergence or with possible unwanted oscillations, if needed (just uncomment them by removing the ;). Rs is mostly to have a series resistance (damping), and Cjo helps by smoothing out sharp derivatives to a sufficiently small transition. As I said, "if". The sources are given as 0.3 V and -0.7 V.

The VCVS-CCCS pair is known as an "ideal transformer", or a "DC transformer", and it acts like an inductorless transformer: the load draws current from the VCVS, and the CCCS takes that back to the source. And "DC" because it will happily deliver DC.

The whole "nonlinear amplifier" looks like this (the values for V1 and I1 are random):

I wanted to recreate the example on page 29, but they are using iterations to reach those values and the output currents are not shown. That's not something you can just throw in values in there and hope to work, so that's a part for another day.