81
views
2
recommends
+1 Recommend
0
shares
• Review: found
Is Open Access

Review of 'Hypotheses of neural code and the information model of the neuron-detector'

Bookmark
2
 Average rating:     Rated 2 of 5. Level of importance:     Rated 2 of 5. Level of validity:     Rated 2 of 5. Level of completeness:     Rated 2 of 5. Level of comprehensibility:     Rated 2 of 5. Competing interests: None

Reviewed article

• Record: found
• Abstract: found
• Article: found
Is Open Access

Hypotheses of neural code and the information model of the neuron-detector

Yuri Parzhin (corresponding) (2014)
This paper deals with the problem of neural code solving. On the basis of the formulated hypotheses the information model of a neuron-detector is suggested, the detector being one of the basic elements of an artificial neural network (ANN). The paper subjects the connectionist paradigm of ANN building to criticism and suggests a new presentation paradigm for ANN building and neuroelements (NE) learning. The adequacy of the suggested model is proved by the fact that is does not contradict the modern propositions of neuropsychology and neurophysiology.
Bookmark

Review information

10.14293/S2199-1006.1.SOR-COMPSCI.AP5TO7.v1.RBMLQQ

This work has been published open access under Creative Commons Attribution License CC BY 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Conditions, terms of use and publishing policy can be found at www.scienceopen.com.

 ScienceOpen disciplines: Computer science Keywords: artificial intelligence, neural code, artificial neural network, neuron-detector

Review text

This paper is an interesting article on a model of a neuron detector for the neural code solving problem. Importantly, neuron detectors are one of the basic elements of an Artificial Neural Network (ANN).

I appreciate very much the study and implementation of ANN inspired by biological neurons and it is my hope that the following criticism will be considered as a constructive contribution to improving this kind of approach.

As for the majority of papers that I have read on this topic, the present work suffers from a large approximation of the biological reality. Of course, increasing the approximation, the probability to implement a good model inspired to real neurons decreases. Moreover, this paper appears confused and some assumptions seem to contradict others. Basic knowledge on how real neurons work in brain modules also seems to conflict with present knowledge on how single neurons and brain modules likely process information.

Historically, ANN have been inspired by real neurons and by their integration in modules of brain activity. Therefore, the idea to reproduce brain activity is intrinsic to the idea of ANN. ANN research and design has benefited from following the progress of knowledge on the single neuron and brain activity. Researchers have been able to improve ANN implementation and the ability of ANN to perform intended tasks. However, this does not mean that ANN really resemble the neuronal activity. It means only that, ideas arising from knowledge on single neurons helps developers of ANN in producing better algorithms. In fact, it is not a necessary condition that ANN resemble exactly the single neuron or brain module activity in order to perform a specific task. It is a sufficient condition that the used algorithm solves adequately the proposed problem or performs the intended specific tasks. Different algorithms can perform the same task or solve a problem independently of the degree to which they resemble the algorithms used by the real neuron.

In the present paper, however, the author uses several hypotheses and assumptions to describe a system, which may or may not accurately model the neural code (real neuron algorithm) used to process information. In other words, there is the pretention to unveil the code formation of the real neurons. If this is the goal, then the authors would need to attend more closely to the actual state of knowledge concerning neuronal and brain activity. In my opinion, the larger part of the paper is based on wrong (or at lease unclear) ideas of how a real neuron works and on how modules integrate the activity of many co-operating neurons. This is a common problem when ANN strive to be strongly related to real neuron activity and is common in those papers where there is not strong cooperation between ANN scientists and neuroscientists, cooperation that I think is very important to address the topic proposed by this article.

Because of my previous considerations, I will not enter into the details of the neuron model, of how the modules work and on the algorithm proposed for the ANN. I will restrict my considerations on what, in my opinion, is wrong in the premises and what is not adequately considered for the goal of the paper.

As a starting general consideration, it is my opinion that to relate the information processing (neural coding) in a single neuron to neuropsychological mechanisms is a big mistake. Neuropsychological level is, several orders of magnitude more complex than the activity of a single neuron of a single brain module.

Another mistake is to consider the reflex arch as a general system of information processing in the brain. The reflex arch is the most basic example of action/reaction mechanisms in living animals. It works also in case of brainless animals (animals surgically deprived of the brain) because it does not need any kind of computation by brain. The action/reaction relationship in the reflex arch simply consists of a sensory stimulation (for example stretching or compression of a muscle) which produces excitation of a neuronal sensor. The sensor produces a spike train that, in some way, is proportional to the intensity of stimulus. The sensory neuron excites a motor neuron into the spinal cord that, in turn, closes the arc by releasing Acetylcholine (Ach) on the muscle producing contraction. The output is a fast reaction mechanism of response to a possible dangerous stimulus. It is not voluntary and this is why information processing by the brain is not called for. The brain is only informed of the event through the ascending path of the spinal cord in order to elaborate the event and to take, if necessary, further voluntary actions. To compare this simple mechanism with visual perception is very hazardous. In the retina, before visual information enters the brain to be processed, a very complex integration activity of the stimulus occurs. This involves several different types of cells that make a pre-processing action on the input information. As for any sensory system, the perception of stimulus occur essentially following a pathway with three different types of cells. Sensory cell (rods and cones in retina), are directly excited by the stimulus and transmit information to a second level cell (bipolar cells in retina) that pass it to the third type of cell type (ganglion cells in retina) that carry information to the appropriate area (module) of the brain. In the retina, for any 1000 receptor cell (rods and cones) there are 100 bipolar-cells and 10 ganglion cells (ration 100:10:1). This means that the information of environmental light (images) involving 100 receptors is, before entering the brain, integrated onto a single ganglion cell. But this system is not so simple. Two more types of cells (amacrine and horizontal cells) produce a sort of modulation on the connection between the three cell types of the visual pathway in the retina. In addition, fast eye movements (saccadic movements with a mean duration of 100ms) produce the continuous shift of the stimulus between different sets of receptors. This basic schema of the retina structure, already gives an idea of the great difference in the complexity of information processing between the reflex arc and visual perception.

Now let us have a look to the proposed hypotheses which, as I said before, are rather confused, confusing for the reader and, perhaps, wrong in principle. The hypothesis seems to consider visual perception mechanisms.

Probably a wrong way to explain this hypothesis, what I understand is that codification of an input image can be constrained in the level of excitation of a single neuron (or a group of a few neurons). Where in the brain? I have already shown that, at the perceptual level in the retina, the system implies the activity of several different cell types performing a pre-processing of the input information both serially (visual pathway) and in parallel (horizontal and amacrine). Now let me say something about the properties of the input that needs to be processed. For a simple point in 2D, for example, a position on the plane (if it is a single input object), a relative position (if more than one input is presented), a level of brightness and a level of contrast with respect to background are the minimal information that need to be processed. Increasing the number of dots, additional information processing is needed as, for example, the spatial frequency (some cells have different responses depending on the dot spatial frequency) and the relative distance. It is very difficult to estimate the number of parameters needed to process all the information for more complex images. To gain fast responses to visual stimuli, part of the information is serially processed and part is processed in parallel in the brain and the whole process involves many different areas of the visual and associative cortex. By these simple considerations, the statement “The ensemble consist of neurons of different modules. Each modules perform a specific function of information processing. Only one neuron can be excited at a time …...” at least would need a better explanation. The meaning is not clear. Does it means that for different types of information needed to process a 2D dot (contrast, position, and intensity) all that is needed is excitation of a single neuron? Or, alternatively, does it means that a single “neuron at a time“ encodes all these different aspects for a dot? From how it is presented I think this aspect is not clear for me.

For the implementation of an algorithm for ANN, this aspect is not so important. If the test is to perceive the stimulus given by the symbol “a”, we can decide that when a neuron “A” belonging to the ANN fires with frequency $$\Phi$$ it is perceiving “a”. The system can easily learn that for any time “a” appears, it must fire with frequency $$\Phi$$ and at any time this frequency is observed it means “a” is being perceived by the system. But I do not think this is the way the brain modules process information for “a”. If a single neuron would code for “a” or alternatively for any of the properties of “a”, the loss of this neuron would correspond to the loss of this kind of information. Firing of a single neuron does not depend only on stimulus activation and not even the spike frequency. Also the frequency can vary and even more can decrease in time if the stimulus is presented for a long time (adaptation). Frequency then is not dependent only on the stimulus or one of its characteristics but also on the duration of the stimulus as well as on other factors. This is true also for the reflex arc where --for long stimulation-- the rules of proportionality of the intensity of stimulus with the spike frequency lose validity since for long lasting stimulation frequency can reduce up to almost zero.
Something more about the spike frequency. As the author says, variability of frequency response is well known. What does an increase (or decrease) of the frequency for a stimulus presentation mean? For example, for the perception of the symbol “a” does it mean a different property of “a” (bigger or smaller)? Or a different shape (italic or bold)? In contrast, different neurons fire in a module for different properties of the input so that brightness and contrast are processed by two different neurons of the module, what is the meaning of frequency variability in the single real neuron? This part of the paper would need a better explanation of the hypothesis and of the meaning of some assumptions. Once again, my opinion is that processing of information for stimulus “a” in brain does not follow the proposed schema. This schema can work well for an ANN where neurons never die, where we can develop an algorithm for the optimal processing of the information for “a”, but this does not mean that we are using the same algorithm the brain uses. My personal opinion on this topic is that, for any information of a stimulus there are many neurons working at the same time and it is not the activity of a single neuron but a pattern of activity generated by many neurons that processes any single information about “a”. In this way, losing a single neuron does not imply that any information for “a” is lost. If the whole system can still produce all the patterns of activity related to “a” it can still process all information for “a”. This idea, I think, better agrees with the clinical findings. Usually we do not lose the meaning of a word or of a letter. We can lose them for large area destruction of brain as, for example, after a brain stroke.

Also hypothesis (3) has some obscure points that are based on a wrong idea about how a real neuron works. The proposed concept of “address” does not correspond to the neurophysiology of neuron in cortex. The relation “address->neurotransmitters” is not realistic. Although it is true that many neurotransmitters have been identified with different roles in the regulation of specific areas of brain, more than 80% of the excitatory synapses of brain use a unique neurotransmitter: Glutamate (Glut) which, in contrast with what the author says, is diffusively present in all areas of brain and mostly in the cortex and hippocampus. The targets of this neurotransmitter are only 2 different types of receptors which are co-localized in the excitatory synapses: AMPA receptor and NMDA receptor. These receptors and Glut are considered the basic elements for Long Term Potentiation, Long Term Depression, learning and memory (i.e., Glut synapse, by a single type of neurotransmitter, constitute the basic system for the most important information processing activity of the brain). This fact contradicts point 3 of this hypothesis. Excitatory neurons using different neurotransmitters (Ach for example) are a minority. Several neurotransmitter, have essentially regulatory activity on the action of other synapses and consequently cannot play the role of “address”. Dopamine, for example, induces phosphorylation of AMPA and NMDA receptors in Glut synapse of spiny neuron (GABAergic and hence inhibitory) of nuclei of the Striatum. The phosphorylation increases the synaptic efficacy (weight) increasing the excitation of GABAergic neuron so that the final real output is an enhanced inhibition of the whole system (see for example, Gerfen, 2000 and Umemiya and Raymond, 1997). This does not mean that Dopamine has this unique role, it is just an example to show that many neurons releasing different neurotransmitters do not produce simple excitation but regulate (or tune) other factors and a fortiori the idea of a relationship “neurotransmitter > address” is not very plausible. In the following, by describing how a single cortical neuron integrate inputs, the weakness of this assumption will be even more evident.

The proposal on offer for how neurons produce spikes and of the spike frequencies is not very accurate. The level of excitation does not depend directly on the level of the threshold but on the ability of the inputs to maintain the membrane potential ($$V_{m}$$) over the threshold. The simple crossing of threshold produces a single spike. If input can be over the threshold for a period enough long, then a train of spikes or bursts can be produced. Of course a higher threshold needs a stronger input to be reached, but the frequency and spike sequence in general depend on the time the system can be over the threshold (of course, never forgetting that spike sequence can be affected by adaptation (see above)). Also the concept of information encoded in the spike frequency is a matter of discussion, although there is near consensus. Some neurons, in fact, respond to stimuli in a tonic way. When crossing the threshold they give only a single spike (independently of the size of $$V_{m}$$ and of the duration they remain over the threshold). Others give a single (or a few) spikes phase locked to sinusoidal-like variation of $$V_{m}$$. A case of the latter type are Dopaminergic neurons mentioned above.

Data used for neuron frequency are also completely wrong. Real neurons cannot fire with a frequency of 1000Hz. The range 100-1000Hz exceeds the obverted biophysical properties of biological neurons. The theoretical limit for the spike frequency is 500Hz. This limit is computed considering that the fastest spike has a duration of 1ms and that a refractory period, that has a minimal duration of 1 ms, always follows the spike. The whole event then cannot have a duration less than 2ms and this give a maximal theoretical frequency of 500Hz (1000/2 events per second). In my knowledge, the theoretical limit of 500Hz has never been observed in electrophysiological recordings. Moreover, spike duration is not 1 ms in all neurons nor is a 1ms refractory period. Spike duration depends on the dynamics of voltage dependent $$Na^{+}$$ and $$K^{+}$$ channels and, in some cases, on the contribution of $$Ca^{2+}$$ channels. Cell membrane properties (resistance, capacitance, axial resistance and so on) also play a significant role which can be different among different neurons. Refractory period essentially depends on the inactivation time of $$Na^{+}$$ channels which are of many different types (almost 50 different types with different activation and inactivation dynamics). Moreover, stochastic processes in the channel dynamics can produces spontaneous (not stimulus induced) crossing of threshold with consequences for spike production. In some neurons the spontaneous spike generation can have a frequency up to 1-2Hz. In general, frequency response of synaptically stimulated neurons is in the rage of tens (not hundreds) of Hz and, interestingly, the neurons with higher spike frequency are not the excitatory neurons but the inhibitory ones (around 60Hz, for GABAergic neurons). I remember only a single case (I read about it more than 15 years ago) where in some neurons of the visual cortex a spike frequency of almost 400Hz was observed for “sub-burst of bursts”. This event was reported in a paper on visual stimulation with recording in area V1 of macaque. The authors, analyzing some bursts with spike frequency of 50-60Hz observed that inside these bursts, some smaller burst of 3-4 spikes were present and that the inter-spike interval of these sub-bursts had a frequency close to 400Hz. They deduced that probably the information code of these neurons was in the “inter-burst bursts” frequency and sequence. In my knowledge, this idea has had no further development.

Also the way synaptic input works seems to be confused in this paper, probably because of the original sin of considering the reflex arch as a good example of the basic working system.

A Pyramidal Neuron (90% of neurons of cortex and of hippocampus are of this type) has a number of synapses ranging from $$5\cdot10^{3}$$ to $$5\cdot10^{4}$$. Pyramidal neurons are excitatory glutamatergic neurons and 90% of synapses they receive are Glut synapses. These numbers already suggest that the assumption that the level of excitation of such a neuron can depend on the amount of a single (other) neuron neurotransmitter release very weak. Nevertheless, what was assumed before seems in partial contradiction with the premises for the hypothesis 4. In these premises, the computation of the number of inputs necessary to cross the threshold seems to be 20-40, which should correspond to the simultaneous activation of many presynaptic neurons, not as is suggested of a single neuron. So it is not very clear if a single presynaptic neuron or 20-40 neurons are the condition necessary for the firing and code representation. However, as I said before, the simple crossing of threshold produces only a single spike. To produce a sequence, it is necessary that the membrane potential $$V_{m}$$ stays over the threshold for an adequate time and quantity. If at any time we need 20-40 active synapses, to maintain $$V_{m}$$ over the threshold for a long period we need the activation in time (time summation) of many more presynaptic neurons. Alternatively, they should fire all with high frequency and for a long time. In addition to the limits of spike frequency of presynaptic neurons, there are limitations also in the synaptic activity that must be considered. A single glutamatergic synapse has a number of receptors which are usually less than 100. These receptors have their dynamics of binding to Glut. Once a first input (Glut release) has occurred and glutamate has been bound it remains bound for period that can be 2-5 ms for AMPA and up to 500ms for NMDA. This means that, probably, after the first spikes arriving in the presynaptic neurons, the receptors got completely saturated. A high spike frequency of the presynaptic neuron, does not necessarily imply that a given synapse increases its current. This occurs when the number of receptors increases (LTP). The fast activation of AMPA receptors quickly decays. Furthermore, both AMPA and NMDA have their inactivation times. So the computation of 20-40 active synaptic inputs is underestimated (probably a result of not considering appropriately the requirements for spatial and temporal integration). Single synapse activity is a very complex machinery with very fine regulation of many different parameters to be considered (Ventriglia and Di Maio 2000a,b; 2002a,b; 2003; 2013a,b).

Moreover, a question arises. What a single neuron has $$10^{4}$$ glutamatergic synaptic connections if only 20-40 are condition necessary for his coding? Additionally, not only is the single synaptic activity important, but the mutual interaction of synapses and their relative position with respect to the soma of the cell (Di Maio, 2008). Reality is more complicated than what is describe in this paper and some parameters on the synaptic contribution to the potential are not appropriately presented. As a single example, usually resting potential does not arrive to -100mV. It does not cross the voltage level of $$K^{+}$$ equilibrium potential which is -80 -90mV. This crossing would produce the effect that opening of $$K^{+}$$ channels would give a depolarizing current while the role of this cannels in spike generation is to produce repolarizing of $$V_{m}$$. But, even more important, the relative position of the input that produces the spatial summation is not adequately considered. This position, more than the neurotransmitter nature, could be interesting for the research and the definition of the “address” of the input.

Let me describe in short some basic properties of a cortical neuron (but valid for any neuron) so as to give a basic simplified idea of how a neuron is a complicated system of integration of signals.

The structure of the main cortical neuron type is defined “pyramidal” because of the shape of the body. From the body departs a system— like a tree— which can extend for some millimeters. Let’s call this the “dendritic tree”. Each neuron has a unique output system which is the axon that carries the information in the form of spikes up to the end where it releases neurotransmitter to other neurons. Spikes are generated in the junction point (hillock) between the body (soma) and the axon. Excitatory synaptic inputs are located on the dendritic tree and positioned on the top of micro structures called spines where AMPA and NMDA receptors are usually co-localized in the same synapse. The larger part of the excitatory inputs ($$10^{4}$$ on average) arrives on spines where, the release of presynaptic glutamate, produce the increase of AMPA (first) and NMDA (later and dependent on $$V_{m}$$ conductance that produces a current that, at peak, range 5-65pA with a mean of 25pA (Forti et al, 1997). Let me stress here that: 1) excitatory synapses have different distance from the point where spike is generate; 2) they can give responses with very high variability. On shafts of dendrites, there are Inhibitory synapses (GABAergic) that represent almost 10% of the total inputs but are located in strategic positions. Dendrites, with spines, have different diameter depending on the distance from the soma almost as branches of a tree have lower diameter far from the body. Electrical properties of the dendrites depends on the size, on the type of channels they have and on the characteristics of the cell membrane. The current produced at spine level induces a variation of few $$V_{m}$$ 1-10mV. This generates a wave that must travel up to the body (hillock) to give its contribution to the spike generation. Usually a neuron at resting (not active) has a membrane potential around -65mV and threshold level which is 10-15mV more positive. Only a small fraction of the variation of the membrane potential generated at the origin (synapse) arrives to the hillock. Dendrites, according to cable theory proposed by Rall since 1957 (only to start have a look on Wikipedia: http://en.wikipedia.org/wiki/Wilfrid_Rall) produce a filtering action such that the Voltage at the origin decreases in time and space depending on the electrotonic properties of the dendrites. Hence, depending on the distance, the path (types and dimension of dendrites) the potential decay in time and space according the following equations of exponential type:

$$V(t) = V_{0}e^{\frac{-t}{\tau}}$$

$$V(x) = V_{0}e^{\frac{-x}{\lambda}}$$

where $$V_{0}$$ is the voltage at origin, $$\tau$$ is a time constant ($$\tau$$ = $$R_{m}C_{m}$$) with $$R_{m}$$ being the membrane resistance and $$C_{m}$$ the membrane capacitance (1µF), $$x$$ is the distance and $$\lambda$$ = sqrt ($$R_{m}$$/$$R_{I}i$$) is the space constant with $$R_{I}$$ being the axial resistance. It follows that an EPSP of few mVs at the synaptic level, arrives to the hillock strongly reduced in amplitude and delayed in time. So far, the effectiveness of an input produced by a single synapse depends on many factors including the position of the dendritic tree, the time of activation, the pathway the information follows to arrive at the soma, a stochastic variability intrinsic to the synaptic mechanism (Ventriglia and Di Maio, 2002a, 2002b), the presence of $$Ca^{2+}$$ channels along the pathway, the value of threshold, the types of voltage gated channels in the axon, and many others.
My opinion is that the process of synaptic integration and of information processing is much more complicated than proposed in this paper. To give an idea of the behavior of a single neuron connected to $$3{\cdot}10^{4}$$ synapses I suggest Ventriglia and Di Maio (2006) which was our attempt (among many others) to produce a simplified, but realistic model, of how a neuron with so many connections can produce spikes. We consider this model already over-simplified with respect to the reality. As additional considerations on the single neuron code production, I would not neglect the activity of the inhibitory neurons on cortical and hippocampal excitatory neurons. Although in cortical neurons they represent only 10% of the synaptic input, the position of inhibitory synapses on the shaft of the dendrites (each among many excitatory synapses) and their ability to fire at high frequency, give them an amplified power in controlling the excitatory processes.

Conclusions
As said in the premise, I appreciate very much and give the right consideration to this paper and to the approach to the problem of neural coding presented. Nevertheless, a better formulation and a better explanation of hypothesis and premises are needed because if the foundation is faulty, then a model which can be good for ANN does not reach the goal of contributing to the unveiling of neural coding as seems to be the goal of this work.

Among all the possible objections to the paper, I think that the most important one is against the idea of “address” based on the neurotransmitter type. The large number of synaptic contacts on a single cortical neuron and the fact that almost all of them use the same kind of neurotransmitter makes the starting hypothesis not very realistic.

As said at beginning I do not enter into the details of the goodness of the part related to the ANN and on its logical schema. The system can easily work well for an ANN, but I do not think that this is because the model resembles real neurons and I am confident that the “neuron detector” cannot explain or unveil or decode the neural code of real neurons at least not in the present formulation and with the proposed starting hypothesis.

References
[1] Di Maio V. (2008) Regulation of information passing by synaptic transmission: A short review. Brain Research 1225:26-38.
[2] Forti L., Bossi M., Bergamaschi, A., Villa A., Malgaroli A. (1997). Loose-patch recordings of single quanta at individual hippocampal synapse. Nature 388: 874-878.
[3] Gerfen, C.R. (2000). Molecular effects of dopamine on striatal-projection pathways. Trends Neurosci. 23, S64-S70.
[4] Umemiya, M. and Raymond, L.A. (1997). Dopaminergic modulation of excitatory post-synaptic currents in rat neostriatal neurons. J. Neurophysiol. 78, 1248-55.
[5] Ventriglia, F. and Di Maio, V. (2000a) A Brownian simulation model of glutamate synaptic diffusion in the femtosecond time scale. Biol. Cybernet. 83, 93-109.
[6] Ventriglia, F. and Di Maio, V. (2000b) A Brownian model of glutamate diffusion in excitatory synapses of hippocampus. Biosystems. 58, 67-74.
[7] Ventriglia, F. and Di Maio, V. (2002a) Stochastic fluctuation of the quantal EPSC amplitude in computer simulated excitatory synapses of hippocampus. Biosystems, 71, 195-204.
[8] Ventriglia, F. and Di Maio, V. (2002b). Stochastic fluctuation of the synaptic function. Biosystems, 67, 287-294.
[9] Ventriglia, F. and Di Maio, V. (2003). Synaptic fusion pore structure and AMPA receptors activation according to Brownian simulation of glutamate diffusion. Biol. Cybernet. 88, 201-209.
[10] Ventriglia F. and Di Maio V. (2006) Multisynaptic activity in a pyramidal neuron model and neural code. Biosystems 86: 18-26.
[11] Ventriglia, F. and Di Maio, V, (2013a) Effects of AMPARs trafficking and glutamate-receptor binding probability on stochastic variability of EPSC. Biosystems 112: 298-304.
[12] Ventriglia, F. and Di Maio, V. (2013b) Glutamate-AMPA interaction in a model of synaptic transmission. Brain Res. 1536: 168-176.