User:Dr. Andras J. Pellionisz/Proposed/Tensor network theory
Tensor Network Theory
by Dr. Andras J. Pellionisz
Tensor Network Theory (TNT) is an approach and algorithm in Neural Networks aimed at explaining existing biological (more closely, sensorimotor) neural networks. TNT is part of the ouvre of Biophysicist Dr. Pellionisz [1], a pioneer of geometrization of biology.
Mathematization of the function of sensorimotor neural networks was developed in the protective environment of Dr. Pellionisz as a refugee from communist Hungary to New York, carried on grants of Dr. Llinas. Although Dr. Pellionisz fought out his own NIH grant, the revolution of geometrization of biology could be temporarily suppressed for too much resistance and not enough support. When his fractal model of development of a Purkinje neuron was published in 1989 the work invoked the "double heresy" of overturning both axioms of genomics (the "Junk DNA" and "Central Dogma" misnomers). As a result, NIH support of Dr. Pellionisz was discontinued and his new grant proposal was denied. On his own and against all odds Dr. Pellionisz accomplished over two further decades a geometrical unification of neuroscience and genomics [2], where in the smooth and derivable spacetime manifold tensor geometry governs, while in the fractal/chaotic domains nonlinear dynamics rules (Tensor Network Theory and FractoGene are based on the common principle of recursion).
The most concise description of Tensor Network Theory is found in the "Encyclopedia of Neuroscience" [3]
A list of relevant publications of Tensor Network Theory (a good number available in full electronically) can be found at [4]
The fundamental tenet of geometrization of biology is that living systems can, and thus need to be described in terms of mathematical (geometrical) abstraction, similar to other branches of Natural Sciences, most particularly in physics. For "NeuroGeometry", this axiom is comprised in "NeuroPhilosophy" by Prof. Patricia Churchland; [5] exemplified by the need and the possibility of explaining known function of the cerebellar neural network ("sensorimotor coordination", vital for species where survival depends on fast and precise movements; the cerebellum as a "neural network" emerged with the sharks about 400 million years ago).
Mathematics of Tensor Network Theory is based on the realization that biological sensorimotor activities can be described from an extrinsic viewpoint in the x,y,z,t physical space as movements (physical space-time vectors), but intrinsically neither the sensory- nor the motor systems of biological organisms are necessarily using the "conventional" Cartesian (orthogonal) system of x,y,z,t coordinates. It is an undeniable fact, for instance, that e.g. head movements are measured by the vestibular semicircular canals that (contrary to widespread misinterpretation) do *not* constitute an orthogonal frame of reference. Likewise, it is unthinkable that motoneurons driving e.g. hand movements use many dozens of muscles generate the movement in a coordinate system that is intrinsic to biological neural networks.
Tensor Network Theory, therefore realizes that the same physical vector of e.g. a head movement, measured in a sensory coordinate system (of the vestibular apparatus) is compensated by e.g. an eye movement (driven by the extraocular eye muscles) where the two (sensory and motor) coordinate systems are not identical to one-another, and both are different from the Cartesian x,y,z,t frame of reference that is used for simplicity and convenience in engineering.
It is well known in physics, that expressions of the same physical vector in any number of generalized (typically non-orthogonal) frames of reference can be defined by tensors. A tensor is a generalized vector-relationship, that mathematically appears as a matrix, which in turn, is materialized by a many-to-many system of connections (a network) between e.g. sensory neurons and motoneurons.
Though the conceptual symplicity and undeniable physiological and anatomical facts (e.g. of vestibular systems and extraocular eye movements, or of vestibular systems and neck-movements expressed in different generalized coordinates) are very difficult to deny, Tensor Network Theory becomes perhaps more difficult to understand once mathematically untrained sensorimotor physiologists or morphologists are reminded that in non-orthogonal (generalized) coordinate systems the very same physical vector (e.g. a movement) can be expressed in two very different type of mathematical vectors. Pellionisz (1980) put forward the realization that the "orthogonal projection-type" (sensory) components conform with the (known) so-called "covariant" vector components, while the "paralelogam-type" (motor) component conform with the (known) so-called "contravariant" vector components. (Sensory covariants can be established independently and in ignorance of other sensory measurements, but they do not physically add up to compose the physical vector, e.g. a movement. Motor contravariants are interdependent, since for paralelogram-components for each coordinate axis the other axes must be known as well - but the great advantage is that contravariant components add up to the physical vector, e.g. a movement.)
Tensor Network Theory, while mathematically abstract, is one of the conceptual approaches to living systems that was developed in close cooperation with experts of biology; most particularly of the cerebellum and sensorimotor systems. Early "neural modeling" of cerebellar neural networks started with a 7-year collaboration with Dr. John Szentagothai (Budapest, Hungary), and was followed by a 14-year collaboration with Dr. Rodolfo Llinas (New York Medical Center, N.Y., USA). Tensor Network Theory provided quantitative, experimentally verifiable predictions that were supported by laboratories in the Netherlands (Dr. Stan Gielen and Zuylen) in Chicago (Dr. Barry Peterson), and in Paris (Dr. Alain Berthoz). Applications to gaze-control sensorimotor systems included collaboration with Drs. Graf, Ostriker, Daunicht. TNT provided an explanation of cerebellar sensorimotor networks that the independently (covariantly) measured sensory coordinates transform into motor intention vectors, but a covariant-to-contravariant transformation (via the cerebellar metric tensor) is needed that expresses the movement by executable (mathematically, contravariant) components. Thus, the cerebellum is a space-time metric tensor; where the known cerebellar function (coordination) was, for the first time, explained in intrinsic coordinates used by cerebellar neural networks.
A recent review of Tensor Network Theory (2005) is provided by John S. Barlow, Cambridge University Press (2005) In "The Cerebellum and Adaptive Control" pp.178-179 as follows:
+++++ "13.2.9. Pellionisz' Tensor Theory
Against a background of prior work directed at comuter simulations and neuronal net modeling of the cerebellum and its components (Pellionisz 1970; Pellionisz and Szentágothai 1973, 1974) Pellionisz drew attention, in a series of papers (e.g. Pellionisz and Llinás 1979, 1980, 1982, 1984), to the problem for the cerebral cortex planning execution of movements in one space or system of coordinates, and executing them in another. Thus, the planning space could be ordinarilz extrinsic three-dimensional space, whereas the space for execution could be that of the limbs-on-limbs-on-trunk, a multidimensional space or hyperspace (more than one dimensions), a quite different frame of reference. For the solution of this problem, Pellionisy proposed the cerebellum as a tensor (tensorial) transformer.
In tensor theory, the coordinate axes, unlike the conventional mutually perpendicular three-dimensional xyz axes, are not necessarily mutually perpendicular. This is the case, for example, in a corner of a parallelogram. Tensor analysis arises, for example, in deformations of a medium that has different properties (e.g. compressibility, heat transfer) in one direction from another (i.e. it is anisotropic). (The general question of movement in space, frames of reference, and coordinate systems was considered by Soechting and Flanders(1992), see also Andersen et al. (1993) and Stein (1992), concerning representations of space and their transformations (or transformations of sensory vectors) by distributed neural networks).
A specific example entailing such transformations is that the VOR, in which the coordinate systems for the tensions exerted by the extraocular muscles are different from those for the semicircular canals, along roll, pitch and yaw axes. In tensor terminology (Pellionisz and Graf 1987, Pellionisz and Llinás 1980, Pellionisz, Peterson and Tomko 1990), the cerebellum would function by transforming the covariant (sensory frame of reference) components of the intended movements into their contravariant (motor-executive frame of reference) components. Computer simulation of use of covariant components of an intended movement, that is, without proper transformation, was reported to result in the equivalent of ataxia. Further, by adding time as another dimension, the possibility of predictive cerebellar action arises (Pellionisz and Llinas 1982). It should be noted that the prediction was based on higher derivatives of a Taylor expansion (representation), rather than on (adaptive) linear prediction (see Arbib and Amari, 1985...)
According to Arbib and Amari (1985), however, the depiction by Pellionisz and Llinás (1980) of the transformation of motor planning into motor execution in terms of tensor theory was miscast, because the latter authors had not demonstrated that the cerebellum implements a metric tensor (i.e. accomplishes a transformation of coordinates according to tensor theory). In the face of results from neural network modeling of the VOR, Robinson (1992a, 1992b) raised questions about the utility for understanding of brain function at a detailed level of such specific mathematical formulations as tensor theory."
++++
Dr. Barlow's review can be augmented here by the following facts:
Tensor Network Theory met its early criticism by Dr. Amari and Arbib (1985), as known to them by the 1980 paper (that they criticized), but their criticism was immediately refuted in "notes added in proof" of Pellionisz and Llinas (1985) that Drs. Amari and Arbib could not yet see thus could not, and ever since have not criticized. Moreover, within a few years (1991) Dr. Amari himself adopted the use of covariant and contravariant tensors (Amari, S: 1991 "Dualistic Geometry of the Manifold of Higher-Order Neurons. Neural Networks" 4(4): 443-451) with a slew of major sensorimotor experts to follow:
Contemporary application of Tensor Network Theory for Gaze Control is best exemplified by "Neural Control of Rotational Kinematics Within Realistic Vestibuloocular Coordinate Systems" (1998) by Michael A. Smith and J. Douglas Crawford, Centre for Vision Research and Department of Psychology and Department of Biology, York University, Toronto, Ontario M3J 1P3, Canada http://jn.physiology.org/cgi/content/full/80/5/2295
The paper by Drs. Smith and Crawford (1998) significantly furthers the tensorial approach to gaze control with proper references to Tensor Network Theory, and in its appendix provides an excellent "tutorial" for the basics of Tensor Theory (for gaze applications).
Over the somewhat hastily put forward criticisms by Drs. Amari and Arbib(not knowing the paper by Pellionisz and Llinas, 1985), for his Tensor Network Theory, Dr. Pellionisz received the Alexander von Humboldt Prize for Senior Distinguished American Scientists from Germany (1990), and Dr. Pellionisz, as a Senior Research Associate of the National Academy of Science to NASA Ames Research Center submitted a plan for flight control by artificial neural networks (plan submitted 1990, F15 fighter jet tested by NASA about a decade later).
Therefore, Tensor Network Theory originated by Pellionisz stands as one of the very few "neural network" theories that were derived from actual biological neural networks (of the cerebellum and gaze control neural circuits), that produced quantitative predictions with experimental results supportive of the theory, and the theory was put to use for engineering projects of significance. At the time of its preparation (from 1970) and through its gradual development (till 1990) Tensor Netwok Theory as "geometrization of biology" was ahead of its time, but still resulted in a rather large number of published collaborative results with neuroscientists, and can show distinct followership.
In a larger sense, Tensor Network Theory significantly contributed to development by Drs. Patricia and Paul Churchland of a new discipline "Neurophilosophy".
