Abstract
From logical reasoning to mental simulation, biological and artificial neural systems possess an incredible capacity for computation. Such neural computers offer a fundamentally novel computing paradigm by representing data continuously and processing information in a natively parallel and distributed manner. To harness this computation, prior work has developed extensive training techniques to understand existing neural networks. However, the lack of a concrete and low-level machine code for neural networks precludes us from taking full advantage of a neural computing framework. Here we provide such a machine code along with a programming framework by using a recurrent neural network—a reservoir computer—to decompile, code and compile analogue computations. By decompiling the reservoir’s internal representation and dynamics into an analytic basis of its inputs, we define a low-level neural machine code that we use to program the reservoir to solve complex equations and store chaotic dynamical systems as random-access memory. We further provide a fully distributed neural implementation of software virtualization and logical circuits, and even program a playable game of pong inside of a reservoir computer. Importantly, all of these functions are programmed without requiring any example data or sampling of state space. Finally, we demonstrate that we can accurately decompile the analytic, internal representations of a full-rank reservoir computer that has been conventionally trained using data. Taken together, we define an implementation of neural computation that can both decompile computations from existing neural connectivity and compile distributed programs as new connections.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Main
Neural systems possess an incredible capacity for computation. From biological brains that learn to manipulate numeric symbols and run mental simulations1,2,3 to artificial neural networks that are trained to master complex strategy games4,5, neural networks are outstanding computers. What makes these neural computers so compelling is that they are exceptional in different ways from modern-day silicon computers: the latter relies on binary representations, rapid sequential processing6, and segregated memory and central processing unit7, while the former utilizes continuum representations8,9, parallel and distributed processing10,11, and distributed memory12. To harness these distinct computational abilities, prior work has studied a vast array of different network architectures13,14, learning algorithms15,16 and information-theoretic frameworks17,18,19 in both biological and artificial neural networks. Despite these substantial advances, the relationship between neural computers and modern-day silicon computers remains an analogy due to our lack of a concrete and low-level neural machine code, thereby limiting our access to neural computation.
To bring this analogy to reality, we seek a neural network with a simple set of governing equations that demonstrates many computer-like capabilities20. One such network is a reservoir computer (RC), which is a recurrent neural network (RNN) that receives inputs, evolves a set of internal states forward in time and outputs a weighted sum of its states21,22. True to its namesake, RCs can be trained to perform fundamental computing functions such as memory storage23,24 and manipulation25,26, prediction of chaotic systems22 and model-free control27. Owing to the simplicity of the governing equations, the theoretical mechanism of training is understood, and recent advances have dramatically shortened training requirements by using a more efficient and expanded set of input training data 28. But can we skip the training altogether and program RCs without any sampling or simulation just as we do for silicon computers?
These ideas have a rich history of exploration, notably including the Neural Engineering Framework29 and system hierarchies for brain-inspired computing30. The former defines the guiding principles—representation, transformation and dynamics—to implement complex computations and dynamics in neuron models31. The latter defines an extension of Turing completeness to neuromorphic completeness, and builds an interface between neuromorphic software and hardware for program-level portability. Our work sits at the intersection of these two areas by defining an extension of the former for RCs to program computations in existing RCs with full-rank connectivity, decompile the computations performed by conventionally trained RCs and omit any sampling of state space in the optimization procedure (Supplementary Section X). Combined with the substantial advances in experimental RC platforms32, it is now timely to formalize a programming framework to develop neural software atop RC hardware.
In this Article, we provide two such programming frameworks—state neural programming (SNP) and dynamic neural programming (DNP)—by constructing an analytic representational basis of the RC neurons, alongside two architectures: open loop and closed loop. Through SNP, we program RCs to perform operations and solve analytic equations. Through DNP, we program RCs to store chaotic dynamical systems as random-access memories (dRAM), virtualize the dynamics of a guest RC, implement neural logic AND, NAND, OR, NOR, XOR and XNOR, and construct neural logic circuits such as a binary adder, flip-flop latch and multivibrator circuit. Using these circuits, we define a simple scheme for game development on RC architectures by programming an RC to simulate a variant of the game ‘pong’. Finally, we decompile computations from conventionally trained RCs as analytic functions.
Open-loop architecture with SNP for output functions
We begin with the simplest architecture, which is an open-loop neural computer architecture that treats the RNN as a function approximator. We conceptualize an RNN as a computing machine comprising N neurons r, which receive k inputs x and produce m outputs o (Fig. 1a). This machine typically supports the basic instructions of multiplication of the neural states and inputs by weights A, B and W, the addition of these weighted states with each other and some bias term d, the transformation of the resultant quantities through an activation function g, and evolution in time (equations (6) and (7)). The output is given by o = Wr (Fig. 1b). Hence, the weights B, A, d and W specify the set of instructions for the RNN to run, which we conceptualize as the low-level neural machine code (Fig. 1c). Unlike conventional computers, these instructions are simultaneously evaluated in parallel by the global set of neurons. We randomly instantiate B, A and d, and program W.
a, An RNN conceptualized as a computing machine with inputs x, neurons r and outputs o, which are all treated as variables. b, The low-level instructions supported by this RNN. c, We randomly instantiate B, A and d, and compile only the output matrix W. d, To convert the machine code into an algebraic form (that is, as a function of x), we decompile the machine code into the SNP matrix Rc by first approximating the RNN state as a function of the powers and the time derivatives of the inputs x (equations (8)–(10)), and then by taking the Taylor series expansion of h. Hence, the (i, j) term of Rc corresponds to the Taylor series coefficient of neuron i and term j. e, This SNP matrix is used to program a desired output (for example, a rotation about the x3 axis) by filling in the entries of the source code matrix Oc, which correspond to the coefficients in front of the desired output functions. f, This program is compiled by training an output matrix Wc that maps Rc to Oc (equation (1)). The RNN output, Wcr(t), successfully rotates the three-dimensional Thomas attractor input x(t). g, The discrete-time decompiler is also composed of a Taylor series expansion, but the function h is now from equations (11) and (12), and the time derivative terms are replaced by time lags. h,i, We program highpass filters of various cut-off frequencies by weighting the appropriate time-lagged terms (h), which yields high-passed versions of the inputs (i).
To define a programming framework, we take the approach of translating the machine code into a representation that is meaningful to the user. We choose that representation to be the input variables x, as the inputs are usually meaningful and interpretable in most applications. This translation from the low-level to the high-level programming matrix is referred to as decompiling, and involves writing the neural states r as a function h of the inputs x given the machine code B, A and d (Methods, equations (8)–(10)). To write h in a more understandable form, we perform a Taylor series expansion of h with respect to all of the input variables \({{{\boldsymbol{x}}}},\dot{{{{\boldsymbol{x}}}}},\cdots \,\), thereby writing the state of every neuron as a weighted sum of linear, quadratic and higher-order polynomial terms of \({{{\boldsymbol{x}}}},\dot{{{{\boldsymbol{x}}}}},\cdots \,\). The weights that multiply these terms are precisely the coefficients of the Taylor series expansion, and form an N × K matrix of coefficients Rc, where K is the number of terms in the expansion (Fig. 1d). This matrix of coefficients R is our SNP matrix in the open-loop architecture.
Next, we define the source code, which is the set of programmable output functions given by the rowspace of R. This is because the output of our RNN is determined by a linear combination of neural states, W, which are weighted sums of the K expansion terms from the decompiler. Here, programming refers to specifying m outputs as a weighted sum of the K terms from the decompiler, which forms an m × K matrix Oc: the source code matrix. In this example, the RNN receives three inputs: x1, x2 and x3. To program a 90° rotation about x3, we specify the coefficients in matrix Oc for three outputs o1, o2 and o3 (Fig. 1e). Finally, to compile the source code Oc into machine code W, we solve
When we drive the RNN with the complex, chaotic time series x(t) from the Thomas attractor (Fig. 1f, blue), the RNN output is a rotated attractor Wr(t) (Fig. 1f, orange).
This process of decompiling, programming and compiling also holds in discrete-time RNNs (Fig. 1g; see Methods, equations (11) and (12)). Our programming matrix now consists of polynomial powers of time-lagged inputs, which allows us to program operations such as highpass filters (Fig. 1h). After compiling the program Od into W, the RNN outputs (Fig. 1i, orange, yellow and purple) filter away the lower-frequency components of an input signal (Fig. 1i, blue). See Supplementary Section IX for the parameters used for all examples.
Closed-loop architecture with SNP to solve algorithms
To increase the computational power of these RNNs, we use the same SNP, but introduce a closed-loop neural architecture. The idea is to use SNP to program some output function of the inputs as \(\bar{W}{{{\boldsymbol{r}}}}={{{\boldsymbol{f}}}}(\bar{{{{\boldsymbol{x}}}}},{{{\boldsymbol{x}}}})\), and then feed that output back into the inputs to define an equivalence relation between the outputs and the inputs to solve \(\bar{{{{\boldsymbol{x}}}}}={{{\boldsymbol{f}}}}(\bar{{{{\boldsymbol{x}}}}},{{{\boldsymbol{x}}}})\). This feedback modifies the internal recurrent weights in the RNN, allowing it to store and modify a history of its states and inputs to store and run algorithms. Hence, the closed-loop architecture is no longer solely a function approximator. We consider the same RNN as in Fig. 1a, except now with two sets of inputs, \(\bar{{{{\boldsymbol{x}}}}}\in {{\mathbb{R}}}^{n}\) and \({{{\boldsymbol{x}}}}\in {{\mathbb{R}}}^{k}\), and two sets of outputs, \(\bar{{{{\boldsymbol{o}}}}}\in {{\mathbb{R}}}^{n}\) and \({{{\boldsymbol{o}}}}\in {{\mathbb{R}}}^{m}\) (Fig. 2a and equation (13)). We feed one set of outputs back into the inputs such that \(\bar{{{{\boldsymbol{o}}}}}=\bar{{{{\boldsymbol{x}}}}}\), which will determine the internal connectivity of the RNN. Using SNP, we program an output \(\bar{{{{\boldsymbol{o}}}}}=\bar{W}{{{\boldsymbol{r}}}}=\bar{{{{\boldsymbol{f}}}}}(\bar{{{{\boldsymbol{x}}}}},{{{\boldsymbol{x}}}})\) and perform feedback as \(A=\bar{A}+\bar{B}\bar{W}\) (Fig. 2b,c and equation (14)).
a, Schematic of the closed-loop architecture with two input sets, \(\bar{{{{\boldsymbol{x}}}}}\) and x, and two output sets, \(\bar{{{{\boldsymbol{o}}}}}\) and o, where feedback occurs by setting \(\bar{{{{\boldsymbol{x}}}}}=\bar{{{{\boldsymbol{o}}}}}\). b, Instruction before and after feedback for continuous-time and discrete-time RNNs. c, \(\bar{B},B,\bar{A}\) and d are randomly initialized, while \(\bar{W}\) and W are programmed. d, A scaled subset of the decompiled matrix Rc containing constant, linear, and quadratic terms in the inputs. e, These terms are used to program the left-hand side of the Lyapunov equation into the source code matrix Oc. f, After compiling Oc into W and performing feedback, a new RNN is defined with recurrent connections \(A=\bar{A}+\bar{B}\bar{W}\), and when this RNN is driven with input matrix X, its output converges to the solution of the Lyapunov equation. Here, we set \(W=\bar{W}\) such that output o matched the feedback output \(\bar{{{{\boldsymbol{o}}}}}\). g, Decompiled Rd for the discrete-time RNN. h, A sample lag operator is programmed for the output used for feedback, \(\bar{{{{\boldsymbol{o}}}}}\), and a Fourier transformation is programmed into the output that is not fed back, o. i, When both outputs are compiled and \(\bar{W}\) is fed back, then driving the resultant RNN with a noisy sawtooth successfully lags the input xt and generates the short-time Fourier transform of the input.
As a demonstration, we program an RNN to solve the continuous Lyapunov equation,
where X and \(\bar{X}\) are 6 × 6 matrices such that the pre-programmed RNN receives n = 36 inputs as \(\bar{{{{\boldsymbol{x}}}}}\) and k = 36 inputs as x. Given X, the solution \(\bar{X}\) to equation (2) is important for control theory33 and neuroscience34, and is often referred to as the controllability Gramian. To program equation (2) into our RNN, we first decompile the neural states r into the SNP matrix Rc with respect to our input variables \(\bar{{{{\boldsymbol{x}}}}}\) and x (Fig. 2d). Next, we fill in matrix Oc with the coefficients of all constant, linear and quadratic terms from equation (2) (Fig. 2e). Then, we compile the program Oc by solving for \({{{{\rm{argmin}}}}}_{\bar{W}}\parallel \bar{W}{R}_{c}-{O}_{c}\parallel\), and define the recurrent connections of a new feedback RNN, \(A=\bar{A}+\bar{B}\bar{W}\), which evolves as equation (14). By driving this new RNN with a matrix X, the output converges to the solution \(\bar{X}\) of equation (2) (Fig. 2f).
As a demonstration for discrete-time systems, we program an RNN to store a substantial time history of a stochastic, non-differentiable time series xt, and perform a short-time Fourier transform. Starting with our decompiled RNN states (Fig. 2g), we write a program, \({\bar{O}}_{d}\), to store time history across n = 49 inputs for \({\bar{{{{\boldsymbol{x}}}}}}_{t}\), by defining a sample lag operator that shifts the state of all inputs down by one index (Fig. 2h and equation (15)). Then, using this lagged history, we write another program, Od, that outputs a short-time Fourier transform (equation (16)) with a sliding window of length n. We compile these two programs by minimizing \(\parallel \bar{W}{R}_{d}-{\bar{O}}_{d}\parallel\) and ∥WRd − Od∥, and define the recurrent connectivity of a new feedback RNN—where \(A=\bar{A}+\bar{B}\bar{W}\)—according to equation (14). By driving this RNN with a stochastic sawtooth wave of amplitude 0.2, period of eight samples and noise that is uniformly distributed about 0 with a width of 0.1, the RNN matches the true short-time Fourier transform (for a performance comparison with conventional RCs and FORCE, see Supplementary Section VII).
Closed-loop RNN with DNP to simulate and virtualize
Here we define a second, dynamical programming framework, DNP, which allows explicit programming of time history for continuous-time RNNs (for an extended discussion, see Supplementary Section VIII). Building on SNP where we decompiled the neural states r, we now decompile the activation function g, which encodes both state and dynamic information through a rearrangement of equation (6). We substitute \({{{\boldsymbol{r}}}}\approx {{{\boldsymbol{h}}}}(\bar{{{{\boldsymbol{x}}}}},{{{\boldsymbol{x}}}})\) into equation (13) as
Hence, our DNP decompiler takes the Taylor series coefficients of g instead of h, allowing us to program not only output functions of the input states, but also the input time history.
As a demonstration, we consider an RNN with 15 states, \({{{{\boldsymbol{r}}}}}^{\circ }\in {{\mathbb{R}}}^{15}\), which receives three inputs \(\bar{{{{\boldsymbol{x}}}}}\in {{\mathbb{R}}}^{3}\). We will use the closed-loop architecture where \({\bar{A}}^{\circ }\), \({\bar{B}}^{\circ }\) and d∘ are randomly initialized, which is decompiled according to equation (3) (Fig. 3a). Because our decompiled code consists of analytic state and time-derivative variables, we can compile \({\bar{W}}^{\circ }\) to map RNN states to input states, and RNN time derivatives to input time derivatives. Prior work has trained RNNs to simulate time-evolving systems by copying exemplars22 or sampling the dynamical state space35. Here we achieve the same simulation without any samples in a chaotic Lorenz attractor that evolves according to \(\dot{\bar{{{{\boldsymbol{x}}}}}}=f(\bar{{{{\boldsymbol{x}}}}})\), such that
Here, ‘programming’ refers to the construction of a matrix \({\bar{O}}_{c}^{\circ }\) comprising the coefficients of \(\bar{{{{\boldsymbol{x}}}}}+\frac{1}{\gamma }\,f(\bar{{{{\boldsymbol{x}}}}})\) preceding the variables \(1,{\bar{x}}_{1},{\bar{x}}_{2},{\bar{x}}_{3},{\bar{x}}_{1}{\bar{x}}_{2},\cdots \,\) of the program matrix \({G}_{c}^{\circ }\) (Fig. 3b). Once we compile this code and perform feedback by defining new connectivity \(A=\bar{A}+\bar{B}\bar{W}\), the evolution of the RNN simulates the Lorenz attractor (Fig. 3c).
a, The machine code (\({\bar{B}}^{\circ },{{{{\boldsymbol{d}}}}}^{\circ }\)) of the guest RNN with 14 neurons and 3 inputs is decompiled into DNP matrix \({G}_{c}^{\circ }\) by taking the Taylor series coefficients of g∘ as equation (3). b, Therefore, the programs we write are not output functions of \(\bar{{{{\boldsymbol{x}}}}}\), but are rather given by \(\bar{{{{\boldsymbol{x}}}}}+\frac{1}{{\gamma }^{\circ }}\dot{\bar{{{{\boldsymbol{x}}}}}}\), where \(\dot{\bar{{{{\boldsymbol{x}}}}}}=f(\bar{{{{\boldsymbol{x}}}}})\) for a dynamical system. c, After compiling into \({\bar{W}}^{\circ }\) and performing feedback, the evolution of the guest RNN projects onto the programmed Lorenz dynamical system. d, The machine code (\(\bar{B},{{{\boldsymbol{d}}}}\)) of the 2,000 neuron host RNN with 15 inputs is decompiled into DNP matrix Gc. e, The program is the set of coefficients for the state and dynamics \({{{{\boldsymbol{r}}}}}^{\circ }+\frac{1}{\gamma }{\dot{{{{\boldsymbol{r}}}}}}^{\circ }\) of the guest RNN. f, After compiling and feedback, the time evolution of the host RNN emulates the time-evolution of the guest RNN, which itself is simulating a Lorenz attractor.
More generally, DNP allows us to program systems of the form \(\dot{\bar{{{{\boldsymbol{x}}}}}}=f(\bar{{{{\boldsymbol{x}}}}})\), which raises an interesting phenomenon. We program another RNN \({{{\boldsymbol{r}}}}\in {{\mathbb{R}}}^{2000}\)—the host—to emulate the dynamics of the feedback RNN \({{{{\boldsymbol{r}}}}}^{\circ }\in {{\mathbb{R}}}^{15}\)—the guest—that itself was programmed to evolve about the Lorenz attractor. We decompile the host RNN using DNP (Fig. 3d), write the code of the guest RNN in the format \({{{{\boldsymbol{r}}}}}^{\circ }+\frac{1}{\gamma }{\dot{{{{\boldsymbol{r}}}}}}^{\circ }\) (Fig. 3e), and compile the code into matrix \(\bar{W}\) (Fig. 3f). The 2,000 state host RNN emulates the 15 state guest RNN, which is simulating a Lorenz attractor. A larger host can emulate multiple guests as a virtual machine36.
Op-codes, composition and dynamic RAM
Here we extend more of the functionality of general purpose computers to RNNs. The first functionality is support for op-codes, which is typically a string of bits specifying which instruction to run. We add control inputs c as a string of 0s and 1s such that
which pushes the pre-programmed RNN to different fixed points, thereby generating a unique SNP at each point (Fig. 4a). Then, we program different matrix operations (Fig. 4b), and simultaneously compile each source code at a different SNP (Fig. 4c) into matrix \(\bar{W}\). When we drive our RNN with matrices P and Q at different c, the RNN outputs each operation (Fig. 4d).
a, Biasing an RNN with different control inputs c1, c2 produces a unique SNP \({R}_{{c}_{1},{c}_{2}}\). b, Source code for programming four matrix operations: elementwise addition, subtraction and multiplication, and matrix multiplication. c, The code is simultaneously compiled by concatenating all SNPs into R and all source codes into \(\bar{O}\), and solving \({{{{\rm{argmin}}}}}_{\bar{W}}\parallel \bar{W}R-\bar{O}\parallel\). d, The RNN is driven by two matrices, P and Q, while switching the op-code c1, c2 to yield the desired operation. e, A neural processing unit (NPU) that performs matrix multiplication and vector addition can be connected to another NPU such that their composition yields a more complex algebraic equation. Here the equation is for least-squares regression, and the programmed RNN composition evolves forward in time to solve the least-squares problem. f, Biasing an RNN with different control inputs c1, c2 produces a unique DNP \({G}_{{c}_{1},{c}_{2}}\). g, Source code for different chaotic attractors. h, The code is simultaneously compiled by concatenating all DNPs into G, and all source codes into \(\bar{O}\), and solving \({{{{\rm{argmin}}}}}_{\bar{W}}\parallel \bar{W}G-\bar{O}\parallel\). i, By changing c1 and c2, we retrieve the stored memories, thereby forming our dRAM.
The second functionality is the ability to compose more complicated programs from simpler programs. We note that, in SNP, the output is programmed and compiled to perform an operation on the inputs, such as a matrix multiplication and vector addition for a neural processing unit (NPU, Fig. 4e). By feeding these outputs into another NPU, we can perform a successive series of feedback operations to define and solve more complex equations, such as least-squares regression (NPU, Fig. 4e).
The third functionality is the random access of chaotic dynamical memories. The control inputs c drive the RNN to different fixed points, thereby generating unique DNPs \({G}_{{c}_{1},{c}_{2}}\) (Fig. 4f). By compiling a single matrix W that maps each DNP to a unique attractor (Fig. 4g,h), the feedback RNN with internal connectivity \(A=\bar{A}+\bar{B}\bar{W}\) autonomously evolves about each of the four chaotic attractors at different values of c (Fig. 4i).
A logical calculus using recurrent neural circuits
This dynamical programming framework allows us to greatly expand the computational capability of our RNNs by programming neural implementations of logic gates. While prior work has established the ability of biological and artificial networks to perform computations, here we provide an implementation that makes full use of existing computing frameworks. We program logic gates into distributed RNNs by using a simple dynamical system
where a, b and z are parameters. This particular system has the nice property of hysteresis, where when z = 0.1, the value of x converges to x = 0.1, but when z = −0.1, the value of x jumps discontinuously to converge at x = −0.1 (Fig. 5a). This property enables us to program logic gates (Fig. 5b). Specifically, by defining the variable z as a product of two input variables p and q, we can program in the dynamics in equation (5) to evolve to −0.1 or 0.1 for different patterns of p and q.
a, Phase diagram of a cubic dynamical system. When z = 0.1, the variable x tends towards the stable fixed point x* = 0.1. When z = −0.1, the system bifurcates, and a new stable fixed point emerges at x* = −0.1. b, By setting z equal to various products of two input variables p and q, the output evolves according to different Boolean logic gates, and we program these logic gate dynamics into our RNNs. c–e, By connecting these neural logic gates, we can form neural circuits that add Boolean numbers (c), store persistent Boolean states according to a SR latch with output o and hidden variable \({o}^{{\prime} }\) (d), and oscillate at a fixed phase difference due to the propagation delay of inversion operations (e).
These logic gates can now take full advantage of existing computing frameworks. For example, we can construct a full adder using neural circuits that take Boolean values p and q as the two numbers to be added, and a ‘carry’ value from the previous addition operation. The adder outputs the sum s and the output carry v. We show the inputs and outputs of a fully neural adder in Fig. 5c, forming the basis of our ability to program neural logic units, which are neural analogues of existing arithmetic logic units.
The emulation of these neural logic gates to circuit design extends even to recurrent circuit architectures. For example, the set–reset (SR) latch—commonly referred to as a flip-flop—is a circuit that holds persistent memory, and is used extensively in computer random-access memory (RAM). We construct a neural SR latch using two NOR gates with two inputs, p and q (Fig. 5d). When p = 0.1 is pulsed high, the output o = − 0.1 changes to low. When q is pulsed high, the output changes to high. When both p and q are held low, then the output is fixed at its most recent value (Fig. 5d). As another example, we can chain an odd number of inverting gates (that is, NAND, NOR and XOR) to construct a multivibrator circuit that generates oscillations (Fig. 5e). Because the output of each gate will be the inverse of its input, if p is high, then q is low and o is high. However, if we use o as the input to the first gate, then p must switch to low. This discrepancy produces constant fluctuations in the states of p, q and o, which generate oscillations that are offset by the same phase (Fig. 5e).
Game development and decompiling trained RNNs
To demonstrate the flexibility and capacity of our framework, we program a variant of the game ‘pong’ into our RNN as a dynamical system. We begin with the game design by defining the relevant variables and behaviours (Fig. 6a). The variables are the coordinates of the ball, x, y, the velocity of the ball, \(\dot{x},\dot{y}\), and the position of the paddle, xp. Additionally, we have the variables that determine contact with the left, right and upper walls as cl, cr and cu, respectively, and the variable that determines contact with the paddle, cp. The behaviour that we want is for the ball to travel with a constant velocity until it hits either a wall or the paddle, at which point the ball should reverse direction.
a, Design of a pong variant. The wall positions (xl, xr, yu) and the paddle’s y-coordinate are fixed as constants. The variables are the ball’s position (x, y) and velocity (\(\dot{x},\dot{y}\)), the paddle’s position (xp), and the variables determining contact with the walls and paddle (cl, cr, cu, cp). The code matrix \({\bar{O}}_{c}\) (scaled for visualization) is shown. b, Contact detection with the right wall is implemented using a supercritical pitchfork bifurcation by scaling the b term in equation (5) by x − xr. When x < xr, the contact variable cr goes to 0. When x > xr, a bifurcation occurs and cr becomes non-zero. c, These contact variables are used to drive an SR latch whose output is the ball’s velocity. d, An RNN simulating a playable game of pong in its head. e, The colour from blue to yellow represents the evolution of time. The bottom square is the movement of the paddle, and the circle is the movement of the marker. f, Conventional training of a reservoir by first driving it with an input time series to generate the reservoir time series r(t), and then training an output matrix W to reconstruct highpass-filtered versions of the input. g, The decompiled analytic outputs of the trained reservoir closely match the true highpass filter coefficients.
Here we run into our first problem: how do we represent contact detection—a fundamentally discontinuous behaviour—using continuous dynamics? Recall that we have already done so to program logic gates in Fig. 5a by using the bifurcation of the cubic dynamical system in equation (5). Here we will use the exact same equation, except rather than changing the parameter z to shift the dynamics up and down (Fig. 5a), we will set the parameter b to skew the shape. As an example, for the right-wall contact cr, we will let b = x − xr (Fig. 6b). When the ball is to the left such that x < xr, then cr approaches 0. When the ball is to the right such that x > xr, then cr becomes non-zero. To set the velocity of the ball, we use the SR latch developed in Fig. 5d. When neither wall is in contact, then cr and cl are both low, and the latch’s output does not change. When either the right or the left wall is in contact, then either cr or cl pulses the latch, producing a shift in the velocity (Fig. 6c). Combining these dynamical equations together produces the code for our pong program (Fig. 6d), and the time evolution of our programmed RNN simulates a game of pong (Fig. 6e).
To demonstrate the capacity of our programming framework beyond compiling programs, we decompile the internal representation of a reservoir that has been trained to perform an operation. We instantiate a reservoir with random input matrix B and random recurrent matrix A, drive the reservoir with a sum of sinusoids—thereby generating a time series r(t)—and train the output weights W to reconstruct highpass-filtered versions of the inputs (Fig. 6f). To understand what the reservoir has learned, we decompile the reservoir state given A and B into the SNP matrix Rd according to equation (12), and find that the output WRd as an analytic function of the inputs and time derivatives closely matches the true filter coefficients (Fig. 6g).
Discussion
Neural computation exists simultaneously at the core of and the intersection between many fields of study. From the differential binding of neurexins in molecular biology37 and neural circuits in neuroscience38,39,40,41,42, to the RNNs in dynamical systems43 and neural replicas of computer architectures in machine learning44, the analogy between neural and silicon computers has generated growing and interdisciplinary interest. Our work provides one possible realization of this analogy by defining a dynamical programming framework for RNNs that takes full advantage of their natively continuous and analytic representation, their parallel and distributed processing, and their dynamical evolution.
This work also makes an important contribution to the increasing interest in alternative computation. A clear example is the vast array of systems—such as molecular45, DNA46 and single photon47—that implement Boolean logic gates. Other examples include the design of materials that compute48,49 and store memories50,51. Perhaps of most relevance are physical instantiations of reservoir computing in electronic, photonic, mechanical and biological systems32. Our work demonstrates the potential of alternative computing frameworks to be fully programmable, thereby shifting paradigms away from imitating silicon computer hardware6, and towards defining native programming frameworks that bring out the full computational capability of each system.
One of the main current limitations is the linear approximation of the RC dynamics. While prior work demonstrates substantial computational ability for RCs with largely fluctuating dynamics (that is, computation at the edge of chaos52), the approximation used in this work requires that the RC states stay reasonably close to the operating points. While we are able to program a single RC at multiple operating points that are far apart, the linearization is a prominent limitation. Future extensions would use more advanced dynamical approximations into the bilinear regime using Volterra kernels53 or Koopman composition operators54 to better capture non-linear behaviours.
Finally, we report in the Supplementary Section XI an analysis of the gender and the racial makeup of the authors we cited in a Citation Diversity Statement.
Methods
Open-loop architecture with SNP
In our framework, we conceptualize an RNN comprising N neurons \({{{\boldsymbol{r}}}}\in {{\mathbb{R}}}^{N}\), which receive k inputs \({{{\boldsymbol{x}}}}\in {{\mathbb{R}}}^{k}\) and produce m outputs \({{{\boldsymbol{o}}}}\in {{\mathbb{R}}}^{m}\). This machine has weights \(A\in {{\mathbb{R}}}^{N\times N}\), \(B\in {{\mathbb{R}}}^{N\times k}\), and \(W\in {{\mathbb{R}}}^{m\times N}\), and some bias term \({{{\boldsymbol{d}}}}\in {{\mathbb{R}}}^{N\times 1}\). If the RNN evolves in continuous time, the instructions are
where 1/γ is a time constant. If the RNN evolves in discrete time, these instructions are
We decompile the neural states r as a function h of the inputs x given the machine code B, A and d in three steps. First, we linearize the dynamics in equation (6) about a stable fixed point r* and an operating point x* to yield
where A* = (dg(Ar* + Bx* + d)∘A − I). Second, because our system is now linear, we can write the neural states as the convolution of the impulse response and the inputs as
Third, to obtain r(t) as an algebraic function without an integral, we perform a Taylor series expansion of this convolution with respect to t to yield
We provide a detailed analytical derivation of h in the Supplementary Sections I–III, and demonstrate the goodness of the approximations in Supplementary Sections IV–VI.
To decompile discrete-time RNNs, first we linearize equation (7) about a stable fixed point r* and operating point x*:
where A* = dg(Ar* + Bx* + d)∘A. Second, we write rt+1 as the convolved sum of inputs
which we Taylor series expand to yield the N × K coefficient matrix for K expansion terms.
Closed-loop architecture with SNP
For the closed-loop architecture with SNP, we begin with the pre-programmed RNNs,
for continuous-time and discrete-time systems, respectively (Fig. 2b). Using SNP, we program an output \(\bar{{{{\boldsymbol{o}}}}}=\bar{W}{{{\boldsymbol{r}}}}=\bar{{{{\boldsymbol{f}}}}}(\bar{{{{\boldsymbol{x}}}}},{{{\boldsymbol{x}}}})\) and perform feedback as \(A=\bar{A}+\bar{B}\bar{W}\) to yield
for continuous-time and discrete-time systems, respectively.
The sample lag operator is defined as
which shifts the state of all inputs down by one index. The short-time Fourier transform with a sliding window of length n is defined as
Data availability
There are no data with mandated deposition used in the manuscript or supplement. All data in the main text and Supplementary Information are generated by the code that is publicly available online.
Code availability
All figures were directly generated in MATLAB from the code available on Code Ocean, available upon publication at https://codeocean.com/capsule/7809611/tree/v1 (ref. 55).
Change history
28 June 2023
A Correction to this paper has been published: https://doi.org/10.1038/s42256-023-00693-7
References
Nieder, A. & Dehaene, S. Representation of number in the brain. Annu. Rev. Neurosci. 32, 185–208 (2009).
Salmelin, R., Hari, R., Lounasmaa, O. V. & Sams, M. Dynamics of brain activation during picture naming. Nature 368, 463–465 (1994).
Hegarty, M. Mechanical reasoning by mental simulation. Trends Cogn. Sci. 8, 280–285 (2004).
Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016).
Silver, D. et al. Mastering the game of Go without human knowledge. Nature 550, 354–359 (2017).
Patterson, D. A. & Hennessy, J. L. Computer Organization and Design ARM Edition: The Hardware Software Interface (Morgan Kaufmann, 2016).
Von Neumann, J. First draft of a report on the EDVAC. IEEE Ann. Hist. Comput. 15, 27–75 (1993).
Singh, C. & Levy, W. B. A consensus layer V pyramidal neuron can sustain interpulse-interval coding. PLoS ONE 12, e0180839 (2017).
Gollisch, T. & Meister, M. Rapid neural coding in the retina with relative spike latencies. Science 319, 1108–1111 (2008).
Sigman, M. & Dehaene, S. Brain mechanisms of serial and parallel processing during dual-task performance. J. Neurosci. 28, 7585–7598 (2008).
Nassi, J. J. & Callaway, E. M. Parallel processing strategies of the primate visual system. Nat. Rev. Neurosci. 10, 360–372 (2009).
Rissman, J. & Wagner, A. D. Distributed representations in memory: insights from functional brain imaging. Annu. Rev. Psychol. 63, 101–128 (2012).
Cho, K., Van Merriënboer, B., Bahdanau, D. & Bengio, Y. On the properties of neural machine translation: encoder–decoder approaches. Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation 103–111 (2014).
Towlson, E. K., Vértes, P. E., Ahnert, S. E., Schafer, W. R. & Bullmore, E. T. The rich club of the C. elegans neuronal connectome. J. Neurosci. 33, 6380–6387 (2013).
Werbos, P. J. Backpropagation through time: what it does and how to do it. Proc. IEEE 78, 1550–1560 (1990).
Caporale, N. & Dan, Y. Spike timing–dependent plasticity: a Hebbian learning rule. Annu. Rev. Neurosci. 31, 25–46 (2008).
Tishby, N., Pereira, F. C. & Bialek, W. The information bottleneck method. Preprint at arXiv https://doi.org/10.48550/arXiv.physics/0004057 (2000).
Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).
Kline, A. G. & Palmer, S. Gaussian information bottleneck and the non-perturbative renormalization group. New J. Phys. 24, 033007 (2021).
Lukoševičius, M., Jaeger, H. & Schrauwen, B. Reservoir computing trends. Künstl. Intell. 26, 365–371 (2012).
Jaeger, H. The “echo state” approach to analysing and training recurrent neural networks—with an erratum note. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report 148, 13 (2001).
Sussillo, D. & Abbott, L. F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009).
Lu, Z., Hunt, B. R. & Ott, E. Attractor reconstruction by machine learning. Chaos 28, 061104 (2018).
Kocarev, L. & Parlitz, U. Generalized synchronization, predictability, and equivalence of unidirectionally coupled dynamical systems. Phys. Rev. Lett. 76, 1816 (1996).
Smith, L. M., Kim, J. Z., Lu, Z. & Bassett, D. S. Learning continuous chaotic attractors with a reservoir computer. Chaos 32, 011101 (2022).
Kim, J. Z., Lu, Z., Nozari, E., Pappas, G. J. & Bassett, D. S. Teaching recurrent neural networks to infer global temporal structure from local examples. Nat. Mach. Intell. 3, 316–323 (2021).
Canaday, D., Pomerance, A. & Gauthier, D. J. Model-free control of dynamical systems with deep reservoir computing. J. Phys. Complex. 2, 035025 (2021).
Gauthier, D. J., Bollt, E., Griffith, A. & Barbosa, W. A. Next generation reservoir computing. Nat. Commun. 12, 5564 (2021).
Eliasmith, C. & Anderson, C. H. Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems (MIT Press, 2003).
Zhang, Y. et al. A system hierarchy for brain-inspired computing. Nature 586, 378–384 (2020).
Eliasmith, C. et al. A large-scale model of the functioning brain. Science 338, 1202–1205 (2012).
Tanaka, G. et al. Recent advances in physical reservoir computing: a review. Neural Netw. 115, 100–123 (2019).
Pasqualetti, F., Zampieri, S. & Bullo, F. Controllability metrics, limitations and algorithms for complex networks. IEEE Trans. Control Netw. Syst. 1, 40–52 (2014).
Karrer, T. M. et al. A practical guide to methodological considerations in the controllability of structural brain networks. J. Neural Eng. 17, 026031 (2020).
Bekolay, T. et al. Nengo: a Python tool for building large-scale functional brain models. Front. Neuroinform. 7, 48 (2014).
Rosenblum, M. & Garfinkel, T. Virtual machine monitors: current technology and future trends. Computer 38, 39–47 (2005).
Südhof, T. C. Synaptic neurexin complexes: a molecular code for the logic of neural circuits. Cell 171, 745–769 (2017).
Lerner, T. N., Ye, L. & Deisseroth, K. Communication in neural circuits: tools, opportunities, and challenges. Cell 164, 1136–1150 (2016).
Feller, M. B. Spontaneous correlated activity in developing neural circuits. Neuron 22, 653–656 (1999).
Calhoon, G. G. & Tye, K. M. Resolving the neural circuits of anxiety. Nat. Neurosci. 18, 1394–1404 (2015).
Maass, W., Joshi, P. & Sontag, E. D. Computational aspects of feedback in neural circuits. PLoS Comput. Biol. 3, e165 (2007).
Clarke, L. E. & Barres, B. A. Emerging roles of astrocytes in neural circuit development. Nat. Rev. Neurosci. 14, 311–321 (2013).
Sussillo, D. Neural circuits as computational dynamical systems. Curr. Opin. Neurobiol. 25, 156–163 (2014).
Graves, A. et al. Hybrid computing using a neural network with dynamic external memory. Nature 538, 471–476 (2016).
Kompa, K. & Levine, R. A molecular logic gate. Proc. Natl Acad. Sci. USA 98, 410–414 (2001).
Zhang, M. & Ye, B.-C. A reversible fluorescent DNA logic gate based on graphene oxide and its application for iodide sensing. Chem. Commun. 48, 3647–3649 (2012).
Pittman, T., Fitch, M., Jacobs, B. & Franson, J. Experimental controlled–not logic gate for single photons in the coincidence basis. Phys. Rev. A 68, 032316 (2003).
Fang, Y., Yashin, V. V., Levitan, S. P. & Balazs, A. C. Pattern recognition with “materials that compute”. Sci. Adv. 2, e1601114 (2016).
Stern, M., Hexner, D., Rocks, J. W. & Liu, A. J. Supervised learning in physical networks: from machine learning to learning machines. Phys. Rev. X 11, 021045 (2021).
Pashine, N., Hexner, D., Liu, A. J. & Nagel, S. R. Directed aging, memory, and nature’s greed. Sci. Adv. 5, eaax4215 (2019).
Chen, T., Pauly, M. & Reis, P. M. A reprogrammable mechanical metamaterial with stable memory. Nature 589, 386–390 (2021).
Boedecker, J., Obst, O., Lizier, J. T., Mayer, N. M. & Asada, M. Information processing in echo state networks at the edge of chaos. Theory Biosci. 131, 205–213 (2012).
Svoronos, S., Stephanopoulos, G. & Aris, R. Bilinear approximation of general non-linear dynamic systems with linear inputs. Int. J. Control 31, 109–126 (1980).
Bevanda, P., Sosnowski, S. & Hirche, S. Koopman operator dynamical models: learning, analysis and control. Annu. Rev. Control 52, 197–212 (2021).
Kim, J. Z. & Bassett, D. S. A neural machine code and programming framework for the reservoir computer. Code Ocean https://doi.org/10.24433/CO.7077387.v1 (2023).
Acknowledgements
We gratefully acknowledge M. X. Lim, K. A. Murphy, H. Ju, D. Zhou and J. Stiso for conversations and comments on the manuscript. J.Z.K. acknowledges support from the National Science Foundation Graduate Research Fellowship No. DGE-1321851, and the Cornell Bethe/KIC/Wilkins Theory Postdoctoral Fellowship. D.S.B. acknowledges support from the John D. and Catherine T. MacArthur Foundation, the ISI Foundation, the Alfred P. Sloan Foundation, an NSF CAREER award PHY-1554488, and the NSF through the University of Pennsylvania Materials Research Science and Engineering Center (MRSEC) DMR-1720530.
Author information
Authors and Affiliations
Contributions
J.Z.K. conceived the initial idea and developed the analyses in conversation with D.S.B. J.Z.K. and D.S.B. prepared the manuscript. All authors contributed to discussions and approved the manuscript.
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Machine Intelligence thanks Adrian Valente, Brian DePasquale and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
Supplementary Figs. 1–12, discussion and derivations of equations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kim, J.Z., Bassett, D.S. A neural machine code and programming framework for the reservoir computer. Nat Mach Intell 5, 622–630 (2023). https://doi.org/10.1038/s42256-023-00668-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s42256-023-00668-8
- Springer Nature Limited
This article is cited by
-
An exact mathematical description of computation with transient spatiotemporal dynamics in a complex-valued neural network
Communications Physics (2024)
-
Learning noise-induced transitions by multi-scaling reservoir computing
Nature Communications (2024)
-
Reservoir-computing based associative memory and itinerancy for complex dynamical attractors
Nature Communications (2024)
-
Higher-order Granger reservoir computing: simultaneously achieving scalable complex structures inference and accurate dynamics prediction
Nature Communications (2024)
-
A ‘programming’ framework for recurrent neural networks
Nature Machine Intelligence (2023)