Hebbian theory is a neuropsychological theory claiming that an increase in synaptic efficacy arises from a presynaptic cell's repeated and persistent stimulation of a postsynaptic cell. It is an attempt to explain synaptic plasticity, the adaptation of brain neurons during the learning process. It was introduced by Donald Hebb in his 1949 book The Organization of Behavior.[1] The theory is also called Hebb's rule, Hebb's postulate, and cell assembly theory. Hebb states it as follows:
Let us assume that the persistence or repetition of a reverberatory activity (or "trace") tends to induce lasting cellular changes that add to its stability. ... When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.[1]: 62
The theory is often summarized as "Neurons that fire together, wire together."[2] However, Hebb emphasized that cell A needs to "take part in firing" cell B, and such causality can occur only if cell A fires just before, not at the same time as, cell B. This aspect of causation in Hebb's work foreshadowed what is now known about spike-timing-dependent plasticity, which requires temporal precedence.[3]
The theory attempts to explain associative or Hebbian learning, in which simultaneous activation of cells leads to pronounced increases in synaptic strength between those cells. It also provides a biological basis for errorless learning methods for education and memory rehabilitation. In the study of neural networks in cognitive function, it is often regarded as the neuronal basis of unsupervised learning.
Hebbian engrams and cell assembly theory
Hebbian theory concerns how neurons might connect themselves to become engrams. Hebb's theories on the form and function of cell assemblies can be understood from the following:[1]: 70
The general idea is an old one, that any two cells or systems of cells that are repeatedly active at the same time will tend to become 'associated' so that activity in one facilitates activity in the other.
When one cell repeatedly assists in firing another, the axon of the first cell develops synaptic knobs (or enlarges them if they already exist) in contact with the soma of the second cell.
[D. Alan Allport] posits additional ideas regarding cell assembly theory and its role in forming engrams, along the lines of the concept of auto-association, described as follows:
If the inputs to a system cause the same pattern of activity to occur repeatedly, the set of active elements constituting that pattern will become increasingly strongly inter-associated. That is, each element will tend to turn on every other element and (with negative weights) to turn off the elements that do not form part of the pattern. To put it another way, the pattern as a whole will become 'auto-associated'. We may call a learned (auto-associated) pattern an engram.[4]: 44
Work in the laboratory of Eric Kandel has provided evidence for the involvement of Hebbian learning mechanisms at synapses in the marine gastropodAplysia californica.[citation needed] Experiments on Hebbian synapse modification mechanisms at the central nervous system synapses of vertebrates are much more difficult to control than are experiments with the relatively simple peripheral nervous system synapses studied in marine invertebrates. Much of the work on long-lasting synaptic changes between vertebrate neurons (such as long-term potentiation) involves the use of non-physiological experimental stimulation of brain cells. However, some of the physiologically relevant synapse modification mechanisms that have been studied in vertebrate brains do seem to be examples of Hebbian processes. One such study[5] reviews results from experiments that indicate that long-lasting changes in synaptic strengths can be induced by physiologically relevant synaptic activity working through both Hebbian and non-Hebbian mechanisms.
Principles
From the point of view of artificial neurons and artificial neural networks, Hebb's principle can be described as a method of determining how to alter the weights between model neurons. The weight between two neurons increases if the two neurons activate simultaneously, and reduces if they activate separately. Nodes that tend to be either both positive or both negative at the same time have strong positive weights, while those that tend to be opposite have strong negative weights.
The following is a formulaic description of Hebbian learning: (many other descriptions are possible)
where is the weight of the connection from neuron to neuron and the input for neuron . Note that this is pattern learning (weights updated after every training example). In a Hopfield network, connections are set to zero if (no reflexive connections allowed). With binary neurons (activations either 0 or 1), connections would be set to 1 if the connected neurons have the same activation for a pattern.
When several training patterns are used the expression becomes an average of individual ones:
where is the weight of the connection from neuron to neuron , is the number of training patterns and the -th input for neuron . This is learning by epoch (weights updated after all the training examples are presented), being last term applicable to both discrete and continuous training sets. Again, in a Hopfield network, connections are set to zero if (no reflexive connections).
A variation of Hebbian learning that takes into account phenomena such as blocking and many other neural learning phenomena is the mathematical model of Harry Klopf.[6]Klopf's model reproduces a great many biological phenomena, and is also simple to implement.
Relationship to unsupervised learning, stability, and generalization
Because of the simple nature of Hebbian learning, based only on the coincidence of pre- and post-synaptic activity, it may not be intuitively clear why this form of plasticity leads to meaningful learning. However, it can be shown that Hebbian plasticity does pick up the statistical properties of the input in a way that can be categorized as unsupervised learning.
This can be mathematically shown in a simplified example. Let us work under the simplifying assumption of a single rate-based neuron of rate , whose inputs have rates . The response of the neuron is usually described as a linear combination of its input, , followed by a response function:
As defined in the previous sections, Hebbian plasticity describes the evolution in time of the synaptic weight :
Assuming, for simplicity, an identity response function , we can write
As in the previous chapter, if training by epoch is done an average over discrete or continuous (time) training set of can be done:where is the correlation matrix of the input under the additional assumption that (i.e. the average of the inputs is zero). This is a system of coupled linear differential equations. Since is symmetric, it is also diagonalizable, and the solution can be found, by working in its eigenvectors basis, to be of the form
where are arbitrary constants, are the eigenvectors of and their corresponding eigen values.
Since a correlation matrix is always a positive-definite matrix, the eigenvalues are all positive, and one can easily see how the above solution is always exponentially divergent in time.
This is an intrinsic problem due to this version of Hebb's rule being unstable, as in any network with a dominant signal the synaptic weights will increase or decrease exponentially. Intuitively, this is because whenever the presynaptic neuron excites the postsynaptic neuron, the weight between them is reinforced, causing an even stronger excitation in the future, and so forth, in a self-reinforcing way. One may think a solution is to limit the firing rate of the postsynaptic neuron by adding a non-linear, saturating response function , but in fact, it can be shown that for any neuron model, Hebb's rule is unstable.[7] Therefore, network models of neurons usually employ other learning theories such as BCM theory, Oja's rule,[8] or the generalized Hebbian algorithm.
Regardless, even for the unstable solution above, one can see that, when sufficient time has passed, one of the terms dominates over the others, and
where is the largest eigenvalue of . At this time, the postsynaptic neuron performs the following operation:
Because, again, is the eigenvector corresponding to the largest eigenvalue of the correlation matrix between the s, this corresponds exactly to computing the first principal component of the input.
This mechanism can be extended to performing a full PCA (principal component analysis) of the input by adding further postsynaptic neurons, provided the postsynaptic neurons are prevented from all picking up the same principal component, for example by adding lateral inhibition in the postsynaptic layer. We have thus connected Hebbian learning to PCA, which is an elementary form of unsupervised learning, in the sense that the network can pick up useful statistical aspects of the input, and "describe" them in a distilled way in its output.[9]
Limitations
Despite the common use of Hebbian models for long-term potentiation, Hebb's principle does not cover all forms of synaptic long-term plasticity. Hebb did not postulate any rules for inhibitory synapses, nor did he make predictions for anti-causal spike sequences (presynaptic neuron fires after the postsynaptic neuron). Synaptic modification may not simply occur only between activated neurons A and B, but at neighboring synapses as well.[10] All forms of hetero synaptic and homeostatic plasticity are therefore considered non-Hebbian. An example is retrograde signaling to presynaptic terminals.[11] The compound most commonly identified as fulfilling this retrograde transmitter role is nitric oxide, which, due to its high solubility and diffusivity, often exerts effects on nearby neurons.[12] This type of diffuse synaptic modification, known as volume learning, is not included in the traditional Hebbian model.[13]
Hebbian learning account of mirror neurons
Hebbian learning and spike-timing-dependent plasticity have been used in an influential theory of how mirror neurons emerge.[14][15] Mirror neurons are neurons that fire both when an individual performs an action and when the individual sees[16] or hears[17] another perform a similar action. The discovery of these neurons has been very influential in explaining how individuals make sense of the actions of others, by showing that, when a person perceives the actions of others, the person activates the motor programs which they would use to perform similar actions. The activation of these motor programs then adds information to the perception and helps predict what the person will do next based on the perceiver's own motor program. A challenge has been to explain how individuals come to have neurons that respond both while performing an action and while hearing or seeing another perform similar actions.
Christian Keysers and David Perrett suggested that as an individual performs a particular action, the individual will see, hear, and feel the performing of the action. These re-afferent sensory signals will trigger activity in neurons responding to the sight, sound, and feel of the action. Because the activity of these sensory neurons will consistently overlap in time with those of the motor neurons that caused the action, Hebbian learning predicts that the synapses connecting neurons responding to the sight, sound, and feel of an action and those of the neurons triggering the action should be potentiated. The same is true while people look at themselves in the mirror, hear themselves babble, or are imitated by others. After repeated experience of this re-afference, the synapses connecting the sensory and motor representations of an action are so strong that the motor neurons start firing to the sound or the vision of the action, and a mirror neuron is created.
Evidence for that perspective comes from many experiments that show that motor programs can be triggered by novel auditory or visual stimuli after repeated pairing of the stimulus with the execution of the motor program (for a review of the evidence, see Giudice et al., 2009[18]). For instance, people who have never played the piano do not activate brain regions involved in playing the piano when listening to piano music. Five hours of piano lessons, in which the participant is exposed to the sound of the piano each time they press a key is proven sufficient to trigger activity in motor regions of the brain upon listening to piano music when heard at a later time.[19] Consistent with the fact that spike-timing-dependent plasticity occurs only if the presynaptic neuron's firing predicts the post-synaptic neuron's firing,[20] the link between sensory stimuli and motor programs also only seem to be potentiated if the stimulus is contingent on the motor program.
^Allport, D.A. (1985). "Distributed memory, modular systems and dysphasia". In Newman, S.K.; Epstein R. (eds.). Current Perspectives in Dysphasia. Edinburgh: Churchill Livingstone. ISBN978-0-443-03039-0.
^Shouval, Harel (2005-01-03). "The Physics of the Brain". The Synaptic basis for Learning and Memory: A theoretical approach. The University of Texas Health Science Center at Houston. Archived from the original on 2007-06-10. Retrieved 2007-11-14.
^Bauer EP; LeDoux JE; Nader K (2001). "Fear conditioning and LTP in the lateral amygdala are sensitive to the same stimulus contingencies". Nat Neurosci. 4 (7): 687–688. doi:10.1038/89465. PMID11426221. S2CID33130204.
Further reading
Hebb, D.O. (1961). "Distinctive features of learning in the higher animal". In J. F. Delafresnaye (ed.). Brain Mechanisms and Learning. London: Oxford University Press.
Hebb, D. O. (1940). "Human Behavior After Extensive Bilateral Removal from the Frontal Lobes". Archives of Neurology and Psychiatry. 44 (2): 421–438. doi:10.1001/archneurpsyc.1940.02280080181011.
Bishop, C.M. (1995). Neural Networks for Pattern Recognition. Oxford: Oxford University Press. ISBN978-0-19-853849-3.
For primaries in other races, see 2020 United States House of Representatives elections in Texas and 2020 United States Senate election in Texas. 2020 Texas Republican presidential primary ← 2016 March 3, 2020 2024 → ← TNUT → Candidate Donald Trump Uncommitted Home state Florida[1] N/A Delegate count 117 0 Popular vote 1,898,664[2] 71,803 Percentage 94.13% 3.56% Elections in Texas Federal government Presidential...
هذه المقالة يتيمة إذ تصل إليها مقالات أخرى قليلة جدًا. فضلًا، ساعد بإضافة وصلة إليها في مقالات متعلقة بها. (يوليو 2013) في هذه المقالة ألفاظ تعظيم تمدح موضوع المقالة، وهذا مخالف لأسلوب الكتابة الموسوعية. فضلاً، أَزِل ألفاظ التفخيم واكتفِ بعرض الحقائق بصورة موضوعية ومجردة ودو...
American college basketball season 1933–34 Wisconsin Badgers men's basketballConferenceBig Ten ConferenceRecord14–6 (8–4 Big Ten)Head coachWalter MeanwellHome arenaUW FieldhouseSeasons← 1932–331934–35 → 1933–34 Big Ten Conference men's basketball standings vte Conf Overall Team W L PCT W L PCT Purdue 10 – 2 .833 17 – 3 .850 Wisconsin 8 – 4 .667 14 – 6 .700 Northwestern 8...
University of BonnRheinische Friedrich-Wilhelms-Universität BonnJenisUmumDidirikan1818PresidenJürgen FohrmannStaf administrasi6,500[1]Jumlah mahasiswa27,800.[1]LokasiBonn, JermanKampusPerkotaanAfiliasiEuropaeumSitus webhttp://www.uni-bonn.deData hingga Juni 2009[update] Universitas Bonn (Jerman: Rheinische Friedrich-Wilhelms-Universität Bonn) adalah sebuah universitas riset publik yang terletak di Bonn, Jerman. Didirikan tahun 1818, Universitas Bonn adalah ...
Pemilihan umum Lesotho 20152012202028 Februari 2015 (2015-02-28)120 kursi untuk Majelis Nasional61 kursi untuk meraih status mayoritasKandidat Partai pertama Partai kedua Partai ketiga Ketua Pakalitha Mosisili Tom Thabane Mothetjoa Metsing Partai Kongres Demokratik All Basotho Convention Kongres Lesotho untuk Demokrasi Kursi sebelumnya 48 30 26 Kursi yang dimenangkan 47 46 12 Perubahan kursi 1 16 14 Suara Popular 218,573 215,022 56,477 Persentase 38...
2021 7-part limited series directed by Kemi Adetiba King of Boys: The Return of the KingGenrePolitical crime dramaCreated byKemi AdetibaDirected byKemi AdetibaStarringSola SobowaleToni TonesCountry of originNigeriaOriginal languagesEnglishIgboYorubaHausaNo. of seasons1No. of episodes7ProductionProducersRemi Adetiba Kemi Adetiba Joy Nnamdi-YusufRunning time60 minutesOriginal releaseNetworkNetflixRelease27 August 2021 (2021-08-27)RelatedKing of Boys King of Boys: The Return Of Th...
Japanese manga series This article is about the manga series and its adaptations. For other uses, see Fable (disambiguation). The FableFirst tankōbon volume coverザ・ファブル(Za Faburu)GenreDrama[1]Thriller[1] MangaWritten byKatsuhisa MinamiPublished byKodanshaEnglish publisherNA: Kodansha USAImprintYoung Magazine KCMagazineWeekly Young MagazineDemographicSeinenOriginal runNovember 1, 2014 – November 18, 2019Volumes22 (List of volumes) MangaWritten byKa...
Swedish discus thrower Daniel StåhlDaniel Ståhl during the Finland-Sweden athletics international 2019 in StockholmPersonal informationNationalitySwedishBorn (1992-08-27) 27 August 1992 (age 31)Solna, SwedenHeight2.00 m (6 ft 7 in)Weight155 kg (342 lb)SportCountrySwedenSportAthleticsEvent(s)Discus throw, shot putClubSpårvägens FK[1] Medal record Men's athletics Representing Sweden Olympic Games 2020 Tokyo Discus throw World Championships 2019 Do...
Overview of Neoclassicism in France Neoclassicism in FranceYears activec. 1760–1830CountryFrance Neoclassicism is a movement in architecture, design and the arts which was dominant in France between about 1760 to 1830. It emerged as a reaction to the frivolity and excessive ornament of the baroque and rococo styles. In architecture it featured sobriety, straight lines, and forms, such as the pediment and colonnade, based on Ancient Greek and Roman models. In painting it featured heroi...
Bilateral relationsNorway-United Kingdom relations Norway United Kingdom Diplomatic missionEmbassy of Norway, LondonEmbassy of the United Kingdom, Oslo The Norwegian stone in Hyde Park, London Norway–United Kingdom relations are foreign relations between Norway and the United Kingdom. The two nations have enjoyed very close cultural, economic, military and political cooperation since Norwegian independence in 1905.[1] Both countries are central allies in NATO, and also have many bil...
2013 AdvoCare 500 Race details[1] Race 25 of 36 in the 2013 NASCAR Sprint Cup Series Atlanta Motor Speedway track layoutDate September 1, 2013 (2013-09-01)Location Atlanta Motor SpeedwayHampton, Georgia, United StatesCourse Permanent racing facility1.54 mi (2.48 km)Distance 325 laps, 500.5 mi (805.476 km)Weather Temperatures up to 90 °F (32 °C); wind speeds up to 14 miles per hour (23 km/h)[2]Average speed 135.128 mph (217.467 km/h)Pol...
Administrative division of the Russian Empire For the original Azov Governorate established in 1708, see Voronezh Governorate. Azov GovernorateАзовская губернияGovernorate of Russian Empire1775–1783HistoryHistory • Established 1775• Disestablished 1783 Succeeded by Voronezh Governorate Today part ofUkraineRussia Azov Governorate (Russian: Азовская губерния, Azovskaya guberniya) was an administrative division (a guberniya) of the Russian Emp...
Men's long jumpat the Games of the XI OlympiadJesse OwensVenueOlympiastadion: Berlin, GermanyDateAugust 4Competitors43 from 27 nationsWinning distance8.06 ORMedalists Jesse Owens United States Luz Long Germany Naoto Tajima Japan← 19321948 → Athletics at the1936 Summer OlympicsTrack events100 mmenwomen200 mmen400 mmen800 mmen1500 mmen5000 mmen10,000 mmen80 m hurdleswomen110 m hurdlesmen400 m hurdlesmen3000 m steeplemen4 × 100 m relaymenwomen4 × ...