Cellular neural network

Post-publication activity

Dr. Tamás Roska, SZTAKI, Hungarian Academy of Sciences & Pázmány University, Budapest, Hungary
Dr. Giovanni E. Pazienza, SZTAKI, Hungarian Academy of Sciences & Pázmány University, Budapest, Hungary

Figure 1: Two-dimensional CNN with radius of neighborhood r=1': the red cell has nine neighbors (the eight blue cells and itself ).

A Cellular Neural Network (CNN), also known as Cellular Nonlinear Network, is an array of dynamical systems (cells) or coupled networks with local connections only. Cells can be arranged in several conﬁgurations; however, the most popular is the two-dimensional CNNs organized in an eight-neighbor rectangular grid. Each cell has an input, a state, and an output, and it interacts directly only with the cells within its radius of neighborhood r: when r = 1, which is a common assumption, the neighborhood includes the cell itself and its eight nearest neighboring cells (see Figure 1). In general, the state of each cell, and hence its output, depends only on the input and the output of its neighbor cells, and the initial state of the network. By varying the values of the connections among cells (i.e., its interaction weights), a CNN can present a large number of dynamics, as proven by Gilli et al. (2002).

Among the numerous applications of CNNs, image processing is probably the most widespread (vision systems based on CNN computers are commercially available), even though recent studies have proved that they can be also used for simulations in fluid dynamics (Kocsárdi et al., 2009) and statistical physics (Ercsey-Ravasz et al., 2008).

Standard CNN model

The CNN dynamics is described by a system of nonlinear differential equations. Using the simplest first-order cell dynamics and linear interactions, the state equation of a cell in position (i,j) is as follows\[ \frac{dx_{ij}(t)} {dt} = - x_{ij}(t) + \sum_{(k,l)\in \mathcal{N} (i,j)} A(i,j;k,l) \cdot y_{kl}(t) + \sum_{(k,l)\in \mathcal{N} (i,j)} B(i,j;k,l) \cdot u_{kl} + z(i,j;k,l) \]

where u_ij, x_ij, and y_ij are the input, the state, and the output of the cell in position (i,j), respectively; the indices k and l denote a generic cell belonging to the neighborhood \(\mathcal{N} (i,j)\) of the cell in position (i,j). All variables are continuous. The set of matrices and the threshold {A,B,z}, which contains the weights of the neural/nonlinear network, is called the cloning template and it defines the operation performed by the network. When the values of the cloning template do not depend on the position of the cell, the CNN is called space-invariant. In this case, the dynamic behavior of the network depends only on a few parameters; for instance, for a two-dimensional CNN with a radius of neighborhood r=1, A and B are 3\(\times\)3 matrices, while \(z\) is a scalar; therefore, in total just 19 real numbers determines the CNN dynamics.

Figure 2: Standard nonlinearity for the output equation.

The expression for the output y_ij is

\( y_{ij}(t) = f(x_{ij}(t))=\frac{1}{2} ( \left| x_{ij}(t)+1 \right| - \left| x_{ij}(t) -1 \right|) \)

which corresponds to the nonlinear function in Figure 2. Note that other functions \(f(\cdot)\) are also possible (see Fig. 2 in Chua and Roska, 1993).

Figure 3: Local connections of an edge cell. Observe that three of its neighbors are boundary cells (dashed).

A standard two-dimensional CNN is defined on an M\(\times\)N rectangular array. Therefore, in order to calculate the state of the cells on the boundary of the array, it is necessary to define the boundary conditions of the network, as shown in Figure 3. In the standard model, the boundary conditions can be:

Fixed (or Dirichlet), if the value of the boundary cells is a prescribed constant;
Zero-flux (or Neumann), if the value of the boundary cells is the same as the edge cells;
Periodic (or toroidal), if the value of the boundary cells is the same as the edge cells on the opposite side (e.g., top boundary cells have the value of bottom edge cells).

Given a cloning template meeting certain conditions of convergence, the CNN converges to a steady state which depends on the initial state and the input.
Clearly, the concept of Cellular Neural/Nonlinear Networks is closely related to that of Cellular Automata: in particular, every binary Cellular Automaton of any spatial dimension is a special case of a CNN with the same neighborhood size, as proven by Chua (2006). However, the CNN has input, as well.

CNN - Universal Machine and Cellular Wave Computing

Figure 4: Architecture of the CNN - Universal Machine (click to enlarge).

The potential applications of a Cellular Neural/Nonlinear Network are fully exploited in the CNN-Universal Machine (CNN-UM), whose architecture is shown in Figure 4. A CNN-UM (see Roska and Chua, 1993) consists essentially of a grid of CNN nuclei, which have local memories and basic local processing capabilities, controlled by a global programming unit. A typical CNN-UM program is composed of a sequence of operations (logic and analog) deﬁned by cloning templates. Since each single cell of the network has its own input, state, and output, CNNs can easily integrate sensing and processing. For example, if the CNN cells have visual sensors, then an image can be processed directly at the focal plane without requiring any data transfer to a central processing unit, as instead occurs in standard computers. It has been proved that the mammalian retina operation can be modeled according to similar principles (Bálya et al., 2002).

A Cellular Wave Computer (Roska, 2003) is an extension of the Cellular Neural Network - Universal Machine in which input and output are ﬂows and elementary instructions are solutions of partial difference-differential equations. The philosophy of this paradigm is completely different from that of a standard digital computer. Complex problems, like a typical reaction-diffusion equation, become trivial in this architecture, since the problem is solved by a single instruction.

It is possible to prove rigorously that both the CNN-UM and the Cellular Wave Computing machine are universal in the Turing sense.

Biological and technological motivations

Cellular Neural/Nonlinear Networks were conceived by Chua and Yang (1988) as a particular model of Neural Networks. Nevertheless, the recent advances in Neuroscience have confirmed that CNNs can model the working principles of many sensory parts of the brain. For example, the Cellular Wave Computing paradigm is based on several of the key properties of brain-like systems, such as:

Continuous-time continuous-valued (analog) signal arrays (flows)
Several two-dimensional strata of analog processors (neurons)
Vertical interconnections between a few strata of neuron processors
Typically, mainly local or sparse global interconnections
Integration of sensing and processing
Spatial-temporal active waves
Events are patterns in space and/or time

These features cannot be found in current standard computers which work according to the principles conceived by Von Neumann – logic machines acting on discrete-valued data programmable algorithmically by using logic sequences of arithmetic and logic operations – which were in agreement with the model of neurons as threshold logic envisaged a few decades ago.

Furthermore, technology has changed dramatically in the past few years: nowadays, it allows the integration of multiple, even thousands, processors on a chip. The high working frequencies of such processors force to use mainly local communications which have the further advantage of reducing the wiring required and power consumed.

References

Bálya, Dávid; Roska, Botond; Roska, Tamás and Werblin, Frank S. (2002). A CNN framework for modeling parallel processing in a mammalian retina. International Journal of Circuit Theory and Applications 30(2): 363-393. doi:10.1002/cta.204.
Chua(1988). Cellular neural networks: theory. IEEE Transactions on Circuits and Systems 35(10): 1257-1272. doi:10.1109/31.7600.
Chua(1993). The CNN paradigm. IEEE Transactions on Circuits and Systems 40(3): 147-156. doi:10.1109/81.222795.
Chua, Leon O. (2006, 2007, 2009). A nonlinear dynamics perspective of Wolfram's new kind of science (Volumes I, II, III). World Scientific, Singapore .
Ercsey-Ravasz, Maria-Magdolna; Néda, Zoltan and Roska, Tamas (2008). Statistical Physics on Cellular Neural Network Computers. Physica D: Nonlinear Phenomena 237(9): 2051-2068. doi:10.1016/j.physd.2008.03.028.
Gilli, Marco; Roska, Tamas; Chua, Leon O. and Civalleri, Pier Paolo (2002). CNN dynamics represents a broader class than PDEs. International Journal of Bifurcation and Chaos 12(10): 2051-2068.
Kocsárdi, Sándor; Nagy, Zoltán; Csík, Árpád and Szolgay, Péter (2009). Two-dimensional Compressible Flow Simulation on Emulated Digital CNN-UM. International Journal of Circuit Theory and Applications 37(4): 569-585. doi:10.1002/cta.565.
Roska(1993). The CNN universal machine: an analogic array computer. IEEE Transactions on Circuits and Systems II 40(3): 163-173. doi:10.1109/82.222815.
Roska, Tamás (2003). Computational and computer complexity of analogic cellular wave computers Journal of Circuits, Systems, and Computers 12(4): 539–562. doi:10.1142/S0218126603001021.

Internal references

Valentino Braitenberg (2007) Brain. Scholarpedia, 2(11):2918.

Olaf Sporns (2007) Complexity. Scholarpedia, 2(10):1623.

James Meiss (2007) Dynamical systems. Scholarpedia, 2(2):1629.

Rodolfo Llinas (2008) Neuron. Scholarpedia, 3(8):1490.

Dale Purves (2009) Neuroscience. Scholarpedia, 4(8):7204.

Gregoire Nicolis and Anne De Wit (2007) Reaction-diffusion systems. Scholarpedia, 2(9):1475.

Philip Holmes and Eric T. Shea-Brown (2006) Stability. Scholarpedia, 1(10):1838.

Cellular neural network

Contents

Standard CNN model

CNN - Universal Machine and Cellular Wave Computing

Biological and technological motivations

References

Recommended readings

See also

Personal tools

Namespaces

Variants

Views

Actions

Search

Navigation

Focal areas

Activity

Tools