Synfire Chains

• Dr. Moshe Abeles, Bar-Ilan University, Ramat Gan, Israel

Figure 1: A synfire chain. Each neuron is depicted as a circle (the cell body), with a dendrite protruding to the left, and a branching axon to the right. Synapses are symbolized by small triangles. The bold- faced drawing at the top left depicts one neuron. The neurons of each pool are drawn one under the other. The layout of the chain does not represent the anatomy, but rather the order of activation in time. The same neuron can be part of more than one pool. For example the two red neurons shown in the figure may be physically the same neuron. Repeated participation of the same neurons can occur up to the limit of the synfire chain’s memory capacity.

A synfire chain is essentially a feed-forward network of neurons with many layers (or pools). Each neuron in one pool feeds many excitatory connections to neurons in the next pool, and each neuron in the receiving pool is excited by many neurons in the previous pool. When activity in such a cascade of pools is arranged like a volley of spikes propagating synchronously from pool to pool it is called a synfire chain. Figure 1 is an illustration of the connectivity in a synfire chain. The mean number of neurons in each pool is the width (w) of the chain. The mean number of synapses that each neuron in one pool receives from neurons in the previous pool (or generate upon neurons in the next pool) is the multiplicity of connections (m). In Figure 1, w=6 and m=4. In real cortical networks w and m are assumed to be much larger.

Activity in synfire chains

Activity along a synfire chain may propagate in either a synchronous or an asynchronous mode. In the asynchronous mode an elevated firing rate in one pool will summate over space and time and cause an elevation in the firing rate in the next pool. In the synchronous mode a synchronous volley in one pool will elicit a synchronous volley in the other after one synaptic delay. It has been argued that the only stable mode of transmission is the synchronous mode (Abeles 1982, 1991). This was shown to be true theoretically (Hertz 1997, Goedeke and Diesmann 2008), through numerical calculations (Diesmann et al. 1999), simulations (Abeles et al. 1993, Gewaltig et al. 2001, Hayon et al. 2004), and in the slice (Reyes 2003).

It has been claimed that if the pools also include inhibitory neurons, and the degree of connectivity overlap is below 30% (that is, two neurons in one pool share at most 30% of their inputs with the previous pool), then asynchronous transmission along a chain is possible (Shadlen and Newsome 1998). However, it has been shown that this assumption is not correct when realistic shapes of synaptic potentials are used (Litvak et al. 2003). Even with a 10% overlap, the correlation between neurons increases along the chain until synchronous volleys are generated. However, if the activity of each neuron is periodic or close to periodic (van Rossum et al. 2002), asynchronous activity may be transmitted along the chain.

Thus, in a multi-layered feed-forward network of spiking neurons, when each neuron fires at random times, the only stable mode of transmission is by synchronous volleys. Within each volley there is a small random dispersion of spike-times but this dispersion does not increase as the volleys propagate along the chain.

Historical notes

The term “synfire chain” was coined by Abeles (1982) to account for the appearance of precise firing sequences with long inter-spike delays which resisted explanation in terms of the known properties of cortical physiology. This structure, with every neuron in one pool exciting all neurons in the second pool, was suggested by Griffith (1963) as a structure that can guarantee a fixed level of activity in a network of excitatory neurons. He called this structure a “complete transmission line”. Griffith did not study its properties in any detail. Similar structures were presumably used by Grossberg (1969) to learn and reproduce complicated space-time patterns. Bienenstock (1995) suggested that the transmission delays need not be equal. In such a system a connection with a long delay may skip over one or more pools. He called such a network synfire braids. The governing rule for synchronous transmission in such a braid is that for any two neurons along the braid, the sum of all delays along each of the multitude (multi-neuronal) trajectories, by which one neuron may excite the other, adds up to the same value. Izhikevich (2006) termed the time-locked but not synchronous spiking activity within each braid polychrony.

Experimental evidence

Direct experimental support for the existence of synfire chains requires simultaneous recording of several neurons from one pool and several from the next pool under conditions in which the synfire chain is repeatedly activated. Using current techniques, one can simultaneously record up to ~100 neurons; however these will be dispersed over a few cortical hyper-columns. This is far too diluted a sampling. Thus, the most one can hope for is to record a few neurons from the same chain simultaneously. In this case, whenever this chain is activated, one expects to see a precise firing sequence of these neurons. Indeed there have been multiple reports in the literature about observations of precisely repeating firing patterns (see a partial list in Abeles and Gat, 2001).

With the synfire memory capacity described below, each neuron may take part in up to 100 chains within a hyper-column, which contains 200 chains. Thus, the chances that a given neuron will participate in a given chain is approximately 0.5. The chances that 3 neurons will participate in this chain are 0.125. Considering the fact that there are many different synfire chains and that one can record from more than 3 neurons simultaneously, there could be numerous different precise firing sequences in any recording site. This was reported to be the case in the slice (Ikegaya et al 2004) and in behaving monkeys. In behaving monkeys (Prut et al. 1998), some of these patterns were strongly related to the monkey’s behavior, but never in a one-to-one fashion. A weaker, but still clear, association of pair-wise patterns with behavior was reported by Shmiel et al. (2005, 2006).

It should be noted that activity in synfire chains must result in precisely repeating firing patterns, but the existence of such patterns does not prove that activity is organized along synfire chains. There could be other networks that produce precise firing patterns.

Memory capacity for synfire chains

Bienenstock (1995) and Herrmann et al. (1995) examined how many synfire pools can be embedded in a network of N neurons without interfering with the reproducibility of the activation. Although they used two very different approaches, both came to similar conclusions; namely, the number of pools (P) is proportional to the network size (N), it decreases when the average firing rate in the network increases, and it depends on the width (w) of the pools. A cortical network with N neurons, an average firing rate of 5 spikes per second, and with w=100, may sustain N synfire pools (P=N). This means that each neuron can take part in 100 synfire pools(!).

These pools can be concatenated into one huge synfire chain or into many shorter chains. As pointed out by Maass, any structure that can be wired from individual neurons could, just as easily, be composed of synfire pools. The advantages would be more reliable operation, shorter input-output delays, and immunity to fallout.

Thus an average cortical hyper-column of 20,000 pyramidal neurons may contain 200 synfire chains, each 100 pools long and 100 neurons wide. At 5 spikes per second per neuron, this column would have 300 neurons firing every 3 ms. If the average transmission time between two pools is 3 ms, there could be 3 synfire waves active concomitantly. These may be distributed over 3 synfire chains or confined to repeated activation of one chain.

Although appealing, this idea failed to materialize when tested in a simulation of a large network of integrate and fire neuromimes in which excitation and inhibition are balanced (see Brunel 2000 for an elegant analysis of the behavior of such a random network). Even when only one synfire chain was embedded in such a balanced network, the network went into a mode of synchronous oscillations (Tetzlaff et al. 2002, Aviel et al. 2002). Attempts to remedy this failure took two forms. One solution was to also include inhibitory neurons in each pool. The output of these inhibitory neurons was not fed forward to the subsequent pool but rather fed back to the entire network (Hayon et al. 2004, Aviel et al. 2003). The other solution, found by Diesmann and collaborators, was to use dispersed parameters for the neuromimes. In particular, using dispersed thresholds, synaptic potential sizes, and delays for the inhibitory neurons was very useful for avoiding global synchronization. It is interesting to point out that anatomists (Economo, 1929) classified cortical neurons into over 60 types, one of which was the pyramidal cell (the excitatory neuron comprising 70-90 percent of cortical neurons) whereas all the others were non pyramidal cells, almost all of which are the inhibitory neurons in the cortex.

The problem of synchronous oscillations does not exist in spiking networks with conduction delays: even if a few neurons fire synchronously, their spikes arrive to the targets asynchronously thanks to different delays. As a result, such spiking networks can have more synfire braids than the number of neurons or even the number of synapses in the network (Izhikevich 2006).

When the same neuron participates in several pools within the same synfire chain, the chain is not a strictly feed-forward network any more. At some level of such feedback connections, activity may start to reverberate within the synfire chain (Abeles et al. 1993). But here too, the reverberating activity is organized in synchronous volleys.

Computing with synfire chains

There have only been a few applications using synfire chains. Arnoldi (1999) used them to find invariances in a picture, Jacquemin (1994) used them to parse and classify French sentences, and Wrigley (1999) used synfire chains to parse an auditory scene. However, the main application seems to lie in Bienenstock’s suggestion (1996) to use synfire chains for implementing “compositional” systems. He posits that only a few cross links between two synfire chains that obey appropriate timing constraints are necessary to assure that both synfire chains will act as a wider and more stable synfire chain. This in turn could lock-in to activities of other synfire chains, etc.. Thus a large structure of synfire chains can be dynamically generated to represent binding of many simple components into a meaningful composite mental representation. This type of binding is dynamic, so that a single synfire chain may, under different conditions, lock-in to a number of other chains and therefore be part of numerous distinct composite representations. In Bienenstock’s view, the most important feature of such dynamic binding is that it can be “vertical”, where a higher- level concept binds to the lower level elements that compose it. Izhikevich (2006) applied Bienenstock's ideas to other cognitive computations.

Hayon (Hayon et al 2004, 2005) has shown by way of simulation, and by numerical analysis, that hierarchical systems of synfire chains can be used to separate figure from ground, to implement a minimal description length (MDL) principle and exhibit the property of compositionality. However, these demonstrations were for “toy” problems. It remains to be shown whether synfire chains can efficiently solve such problems in real world situations.

Abeles et al. (1993), have shown by way of simulations that two synfire chains with random connections may learn to lock-in to each other if activated synchronously several times, and if the synaptic modifications follow a time dependent synaptic plasticity (i.e. synapses are strengthened if the pre synaptic spike precedes the post synaptic, and weakened if the time order is reversed). Such learning rules were experimentally reported (e.g. Zhang et al. 1998) and are dubbed STDP. It should be noted that to a limited extent even a single neuron can learn to recognize specific spatio-temporal firing patterns, as was demonstrated by the “tempotron” of Gutig and Sompolinsky (2006).

Self organization of synfire chain

Bienenstock and Doursat (Bienenstock 1991) found that in a random network of excitatory neurons, with simple learning rules, if one seeds synchrony by exciting w neurons synchronously from time to time, a synfire chain will spontaneously be formed. In their model the total synaptic strength that a neuron can produce or accept was limited. Prugel-Bennett and Hertz (1996) tried to grow a synfire chain in a random network. They found that parameters had to be fine- tuned in order to succeed, and even then the synfire chains tended to be short and to form closed loops. Similar results were obtained by Levy et al. (2001), who employed STDP learning rules. The network would typically become organized into short cyclical synfire chains, which caused the network to generate cyclical activation of groups of neurons. They called this activity mode “distributed synchrony”. Apparently the optimal solution is to combine both the STDP learning rule and limit the total cumulative synaptic strength that neurons may produce and receive to spontaneously grow long synfire chains. However, no such study has been carried out to date.

Izhikevich (2006) showed that synfire braids appeared spontaneously in spiking networks with a distribution of conduction delays and spike-timing dependent plasticity. There were many more synfire braids than the number of neurons in the network, though in his simulations he used an unrealistic assumption that a few presynaptic spikes were needed to excite a postsynaptic cell.

Criticism

The claim that the reported precise firing patterns were real was challenged by Richmond and Oram (1999) and by Baker and Lemon (2003). They argued that the statistics used to detect such a pattern were flawed. An elegant solution was provided by Bienenstock and Geman (Hastopuolos et al. 2003). According to their approach, the null hypothesis, to be tested, is that there is nothing precise about spike timing to within a time window of W ms. If true, then any statistic derived from the spike trains should not be significantly different from the same statistic derived after teetering each spike at random within W ms.

In order to test the null hypothesis, we randomly teeter the spike trains many times, compute our preferred statistic after each teetering, and build a histogram of the statistics. If the statistic of the real data falls in the a percentile of the teetered values, then we can reject the null hypothesis with a significance level of 1-a. This method also allows for estimating the time precision of the spikes. By making the teetering window W smaller and smaller we reach a critical W at which the null hypothesis can no longer be rejected. This critical W is an upper bound for precision. Potentially, with another statistic, better precision could be found. Figure 2 illustrates these ideas.

Figure 2: Estimating the precision of spikes. A statistic based on the most prominent peaks in cross-correlations was extracted to describe the time precision in the data sets. Left – the distribution of 5000 estimations of the statistic when times were teetered within 10 ms and the statistic of the actual data. Clearly the probability of obtaining the statistic for real data by chance is less than 1/5000. Right - the probability that the statistic was obtained by chance as a function of the teetering window for 5 recording days (Shmiel et al. 2005, 2006).

A synfire chain is a very (over?) simplified structure. There might be other structures, with less specific connections, which can generate precise firing sequences. Indeed, any network that can produce and recognize precise firing sequences would have all the advantages of a synfire chain. In a random diluted network, if one chooses w and m appropriately, one can find an almost infinite number of connectivity schemes similar to synfire chains by chance (Abeles 1991). However, the activity in such a network is not reproducible. If we start the activity by activating the same w neurons twice, even a very small amount of noise will cause the activity patterns to diverge very fast. Van Vreeswijk and Sompolinsky (1996) showed that a random network with balanced excitation and inhibition behaves in this “chaotic” manner.

Luczak et al. (2007) showed that reproducible precise firing patterns can appear at the beginning of transitions from down- to up-states in vivo in anesthetized and awake rats (see UP and DOWN States). The exact mechanism of generation of such patterns is not known, but it most probably relies on the fact that pyramidal neurons exhibit reproducible latencies (delays up to 500 ms) to the first spike when stimulated with an injected step of dc-current or injected conductance (via dynamic clamp). Different neurons have different latencies resulting in stereotypical patterns of transition from down- to up-states.

In summary, classical anatomy and physiology of the cortex sustain the idea that activity may be organized in synfire chains. Synfire chains generate and can learn to recognize sequences of co-activated neurons. Therefore, one can create compositional systems from synfire chains. It remains for future work to show if, when, and where cortical activity is organized in synfire- like modes, and whether real world problems can be solved by synfire chains.

References

• Abeles M. (1982) Local Cortical Circuits: An Electrophysiological study. Springer, Berlin.

• Abeles M. (1991) Corticonics: Neural Circuits of the Cerebral Cortex. Cambridge University Press, New-York.

• Abeles M., Vaadia E., Bergman H., Prut Y., Haalman I. and Slovin H. (1993) Dynamics of neuronal interactions in the frontal cortex of behaving monkeys. Concepts in Neuroscience 4:131-158.

• Abeles M. and Gat I. (2001) Detecting precise firing sequences in experimental data. J. Neurosci. Methods. 107:141-154.

• Arnoldi H.M., Englemeier R.K. and Brauer W. (1999) Invariant pattern recognition based on synfire chains. Biol. Cybern. 80:433-447.

• Aviel Y., Pavlov E., Abeles M., and Horn D. (2002 ) Synfire chain in a balanced network. Neurocomput. 44:285-292

• Aviel Y., Horn D. and Abeles M. (2003) Synfire waves in small balanced networks. Neurocomput., 58:123-127.

• Baker S.N. and Lemon R.G. (2003) Precise spatiotemporal repeating patterns in monkey primary and supplementary motor areas occur at chance levels. J Neurophysiol 84:1870-1880.

• Bienenstock E. (1991) Notes on the growth of composition machine. In: (Andler E., Bienenstock E. and Laks B. eds.) Proceeding of first interdisciplinary workshop on Compositionality in cognition and neural networks, Abbaye de Royaumont, France. pp25-43.

• Bienenstock E. (1995) A model of neocortex. Network: Computation in Neural systems, 6:179-224.

• Bienenstock E. (1996) Composition. In: Aertsen A. and Braitenberg V. (eds.) Brain Theory: Biological Basis and Computational Theory of Vision. Elsevier, pp 269-300.

• Brunel N. (2000) Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J Comput Neurosci. 8:183-208.

• Diesmann, M., Gewaltig M.-O. and Aertsen A. (1999) Stable propagation of synchronous spiking in cortical neural networks. Nature, 402:529-533.

• Economo C. (1929) The Cytoarchitectonics of the Human Cerebral Cortex. Oxford University Press.

• Gewaltig M.-O., Diesmann M. and Aertsen A. (2001) Propagation of cortical synfire activity: survival probability in single trials and stability in the mean. Neural networks 14:657-673.

• Goedeke S. and Diesmann M. (2008) The mechanism of synchronization in feed-forward neuronal networks. New J. Physics 10:015007.

• Griffith J.S. (1963) On the stability of brain-like structures. Biophys. J. 3:299-308.

• Grossberg, S. (1969), Some networks that can learn, remember, and reproduce any number of complicated space-time patterns. I. J. Math. Mech., 19:53-91.

• Gutig R. and Sompolinsky H. (2006) The tempotron: a neuron that learns spike timing-based decisions. Nat Neurosci. 9:420-429.

• Hastopoulos N.G., Geman S., Amarasingham A. and Bienenstock E. (2003) At what time scale does the nervous system operate? Neurocomput. 52-54:25-29.

• Hayon G., Abeles M. and Lehmann D. (2004) Modeling compositionality by dynamic binding of synfire chains. J. Comp. Neurosci. 17:179-201.

• Hayon G., Abeles M., Lehmann D. (2005) A model for representing the dynamics of a system of synfire chains. J. Comp. Neurosci. 18: 41-53.

• Herrmann M., Prugel-Bennett A. and Hertz J. (1995) Analysis of synfire chains. Network: Computation in Neural systems, 6:403-414.

• Hertz J. (1997) Modelling synfire processing. In: Wong K-Y M. King I. and Yeung D-Y (eds) Theoretical Aspects of Neural Computation. Springer-Verlag, pp 135-144.

• Horn, D., Levy N., Meilijson I. and Ruppin E. (2000) Distributed synchrony of spiking neurons in a Hebbian cell assembly. NIPS 12:129-135.

• Ikegaya Y., Aaron G., Cossart R., Aronov D., Lampel I., Ferster D. and Yuste R. (2004) Synfire chains and cortical songs: Temporal modules of cortical activity. Science 304:559-564.

• Izhikevich E.M. (2006) Polychronization: computation with spikes. Neural Comput. 18:245-282.

• Jacquemin C. (1994) A temporal connectionist approach to natural language. Sigart Bull., 5(3):12-22.

• Levy, N., Horn, D., Meilijson, I. and Ruppin, E., 2001, Distributed synchrony in a cell assembly of spiking neurons. Network, 14:815-824.

• Luczak A., Barthóf P., Marguet S.L. , Buzsáki G., and Harris K.D. (2007) Sequential structure of neocortical spontaneous activity in vivo. PNAS, 104:347-352.

• Oram, M.W., Wiener M.C., Lestienne R. and Richmond B. J. (1999) Stochastic nature of precisely timed spike patterns in visual system neuronal responses. J Nerophysiol., 81:3021-3033.

• Prugel-Bennett A. and Hertz J. (1996) Learning synfire chains. Int J Neural Systems, 7:445-450.

• Prut Y., Vaadia E., Bergman H., Haalman I., Slovin H., and Abeles M. (1998) Spatio-temporal structure of cortical activity, properties and behavioral relevance. J. Neurophysiol. 79: 2857-2874.

• Reyes A.D. (2003) Synchrony dependent propagation of firing rates in iteratively constructed networks in vivo. Nature Neurosci 6:593-599.

• Shadlen, M.N. and Newsome W.T. (1998) The variable discharge of cortical neurons: Implications for connectivity, computation and information coding. J. Neurosci., 18:3870-3896.

• Shmiel T., Drori R., Shmiel O., Ben-Shaul Y., Nadasdy Z., Shemesh M. Teicher M. and Abeles M. (2005) Neurons of the cerebral cortex exhibit precise inter-spike timing in correspondence to behavior. PNAS 102:18655-18657.

• Shmiel T., Drori R., Shmiel O., Ben-Shaul Y., Nadasdy Z., Shemesh M., Teicher M. and Abeles M. (2006) Temporally precise cortical firing patterns are associated with distinct action segments. J. Neurophysiol. 96: 2645-2652.

• Tetzlaff T., Geisel T. and Diesmann M. (2002) The ground state of cortical feed-forward networks. Neurocomputing 44-46:673-678.

• van Rossum M.C., Turrigiano G.G. and Nelson S.B. (2002) Fast propagation of firing rates through layered networks of noisy neurons. J Neurosci 1:1956-1966.

• van Vreeswijk C. and Sompolinsky H. (1998) Chaotic balanced state in a model of cortical circuits. Neural Comput. 15:1321-1371.

• Wrigley N.S. (1999). Synfire chains as a neural mechanism for auditory grouping. Tech. Report CS-99-11, U. Sheffield. S.wrigley@dcs.shef.ac.uk

• Zhang, L.I., Tao H.W., Holt C.E., Harris W.A. and Poo M. (1998) A critical window for cooperation and competition among developing retinotectal synapses. Nature, 395:37-44.

Internal references

• Alain Destexhe (2007) High-conductance state. Scholarpedia, 2(11):1341.

• Howard Eichenbaum (2008) Memory. Scholarpedia, 3(3):1747.

• Peter Jonas and Gyorgy Buzsaki (2007) Neural inhibition. Scholarpedia, 2(9):3286.

• Rodolfo Llinas (2008) Neuron. Scholarpedia, 3(8):1490.

• Kendall E. Atkinson (2007) Numerical analysis. Scholarpedia, 2(8):3163.

• Jeff Moehlis, Kresimir Josic, Eric T. Shea-Brown (2006) Periodic orbit. Scholarpedia, 1(7):1358.

• Nelson Spruston (2009) Pyramidal neuron. Scholarpedia, 4(5):6130.

• Philip Holmes and Eric T. Shea-Brown (2006) Stability. Scholarpedia, 1(10):1838.

• Arkady Pikovsky and Michael Rosenblum (2007) Synchronization. Scholarpedia, 2(12):1459.

• Charles Wilson (2008) Up and down states. Scholarpedia, 3(6):1410.

See Also

High-Conductance State, Neuronal Synchronization, Spike-Timing-Dependent Plasticity, Spiking Neurons

Sponsored by: Eugene M. Izhikevich, Editor-in-Chief of Scholarpedia, the peer-reviewed open-access encyclopedia

Sponsored by: Dr. Marc-Oliver Gewaltig, Blue Brain Project, École Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland

Reviewed by: Dr. Tom Tetzlaff, Nowegian University of Life-Sciences (UMB)

Reviewed by: Eugene M. Izhikevich, Editor-in-Chief of Scholarpedia, the peer-reviewed open-access encyclopedia

Accepted on: 2009-07-21 08:07:27 GMT