Inception (deep learning architecture)

Inception
Original author(s)Google AI
Initial release2014
Stable release
v4 / 2017
Repositorygithub.com/tensorflow/models/blob/master/research/slim/README.md
Type
LicenseApache 2.0

Inception[1] is a family of convolutional neural network (CNN) for computer vision, introduced by researchers at Google in 2014 as GoogLeNet (later renamed Inception v1). The series was historically important as an early CNN that separates the stem (data ingest), body (data processing), and head (prediction), an architectural design that persists in all modern CNN.[2]

Inception-v3 model.

Version history

Inception v1

GoogLeNet architecture.

In 2014, a team at Google developed the GoogLeNet architecture, an instance of which won the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14).[1][3]

The name came from the LeNet of 1998, since both LeNet and GoogLeNet are CNNs. They also called it "Inception" after a "we need to go deeper" internet meme, a phrase from Inception (2010) the film.[1] Because later, more versions were released, the original Inception architecture was renamed again as "Inception v1".

The models and the code were released under Apache 2.0 license on GitHub.[4]

An individual Inception module. On the left is a standard module, and on the right is a dimension-reduced module.
A single Inception dimension-reduced module.

The Inception v1 architecture is a deep CNN composed of 22 layers. Most of these layers were "Inception modules". The original paper stated that Inception modules are a "logical culmination" of Network in Network[5] and (Arora et al, 2014).[6]

Since Inception v1 is deep, it suffered from the vanishing gradient problem. The team solved it by using two "auxiliary classifiers", which are linear-softmax classifiers inserted at 1/3-deep and 2/3-deep within the network, and the loss function is a weighted sum of all three:

These were removed after training was complete. This was later solved by the ResNet architecture.

The architecture consists of three parts stacked on top of one another:[2]

  • The stem (data ingestion): The first few convolutional layers perform data preprocessing to downscale images to a smaller size.
  • The body (data processing): The next many Inception modules perform the bulk of data processing.
  • The head (prediction): The final fully-connected layer and softmax produces a probability distribution for image classification.

This structure is used in most modern CNN architectures.

Inception v2

Inception v2 was released in 2015, in a paper that is more famous for proposing batch normalization.[7][8] It had 13.6 million parameters.

It improves on Inception v1 by adding batch normalization, and removing dropout and local response normalization which they found became unnecessary when batch normalization is used.

Inception v3

Inception v3 was released in 2016.[7][9] It improves on Inception v2 by using factorized convolutions.

As an example, a single 5×5 convolution can be factored into 3×3 stacked on top of another 3×3. Both has a receptive field of size 5×5. The 5×5 convolution kernel has 25 parameters, compared to just 18 in the factorized version. Thus, the 5×5 convolution is strictly more powerful than the factorized version. However, this power is not necessarily needed. Empirically, the research team found that factorized convolutions help.

It also uses a form of dimension-reduction by concatenating the output from a convolutional layer and a pooling layer. As an example, a tensor of size can be downscaled by a convolution with stride 2 to , and by maxpooling with pool size to . These are then concatenated to .

Other than this, it also removed the lowest auxiliary classifier during training. They found that the auxiliary head worked as a form of regularization.

They also proposed label-smoothing regularization in classification. For an image with label , instead of making the model to predict the probability distribution , they made the model predict the smoothed distribution where is the total number of classes.

Inception v4

In 2017, the team released Inception v4, Inception ResNet v1, and Inception ResNet v2.[10]

Inception v4 is an incremental update with even more factorized convolutions, and other complications that were empirically found to improve benchmarks.

Inception ResNet v1 and v2 are both modifications of Inception v4, where residual connections are added to each Inception module, inspired by the ResNet architecture.[11]

Xception

Xception ("Extreme Inception") was published in 2017.[12] It is a linear stack of depthwise separable convolution layers with residual connections. The design was proposed on the hypothesis that in a CNN, the cross-channels correlations and spatial correlations in the feature maps can be entirely decoupled.

References

  1. ^ a b c Szegedy, Christian; Wei Liu; Yangqing Jia; Sermanet, Pierre; Reed, Scott; Anguelov, Dragomir; Erhan, Dumitru; Vanhoucke, Vincent; Rabinovich, Andrew (June 2015). "Going deeper with convolutions". 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. pp. 1–9. arXiv:1409.4842. doi:10.1109/CVPR.2015.7298594. ISBN 978-1-4673-6964-0.
  2. ^ a b Zhang, Aston; Lipton, Zachary; Li, Mu; Smola, Alexander J. (2024). "8.4. Multi-Branch Networks (GoogLeNet)". Dive into deep learning. Cambridge New York Port Melbourne New Delhi Singapore: Cambridge University Press. ISBN 978-1-009-38943-3.
  3. ^ Official repo of Inception V1 on Kaggle, published by Google.
  4. ^ "google/inception". Google. 2024-08-19. Retrieved 2024-08-19.
  5. ^ Lin, Min; Chen, Qiang; Yan, Shuicheng (2014-03-04). "Network In Network". arXiv:1312.4400 [cs.NE].
  6. ^ Arora, Sanjeev; Bhaskara, Aditya; Ge, Rong; Ma, Tengyu (2014-01-27). "Provable Bounds for Learning Some Deep Representations". Proceedings of the 31st International Conference on Machine Learning. PMLR: 584–592.
  7. ^ a b Szegedy, Christian; Vanhoucke, Vincent; Ioffe, Sergey; Shlens, Jon; Wojna, Zbigniew (2016). "Rethinking the Inception Architecture for Computer Vision". Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR): 2818–2826.
  8. ^ Official repo of Inception V2 on Kaggle, published by Google.
  9. ^ Official repo of Inception V3 on Kaggle, published by Google.
  10. ^ Szegedy, Christian; Ioffe, Sergey; Vanhoucke, Vincent; Alemi, Alexander (2017-02-12). "Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning". Proceedings of the AAAI Conference on Artificial Intelligence. 31 (1). arXiv:1602.07261. doi:10.1609/aaai.v31i1.11231. ISSN 2374-3468.
  11. ^ He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing; Sun, Jian (10 Dec 2015). Deep Residual Learning for Image Recognition. arXiv:1512.03385.
  12. ^ Chollet, Francois (2017). "Xception: Deep Learning With Depthwise Separable Convolutions". Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR): 1251–1258.

Read other articles:

Kawasan Benteng Tatas di Banjarmasin. Benteng Tatas (bahasa Inggris: Fort Tatas) adalah bekas benteng historis sebagai pusat pertahanan militer Belanda di Kalimantan. Situs Benteng ini terletak di pusat kota banjarmasin, sekarang menjadi lahan Masjid Raya Sabilal Muhtadin.[1][2][3] Catatan kaki ^ Dari Benteng Tatas, Tata Kota Banjarmasin Digagas. jejakrekam.com. Diakses tanggal 2020-02-06.  ^ http://kebudayaan.kemdikbud.go.id/bpcbkaltim/lanjutan-pola-tata-ruang-ko...

 

Artikel ini sebatang kara, artinya tidak ada artikel lain yang memiliki pranala balik ke halaman ini.Bantulah menambah pranala ke artikel ini dari artikel yang berhubungan atau coba peralatan pencari pranala.Tag ini diberikan pada Desember 2022. Seorang pengrajin batik sedang membuat batik Batik Malaysia adalah seni tekstil yang berasal dari Malaysia, khususnya di pesisir timur Malaysia (Kelantan, Terengganu dan Pahang). Kebanyakan motif populernya adalah dedaunan dan bunga. Batik Malaysia ya...

 

Emaar PropertiesJenisPublikIndustriReal estateDidirikan1997KantorpusatDubai, Uni Emirat ArabTokohkunciJamal Bin Theniyah(pemimpin)Ahmed Jawa(Wakil pemimpinZishan Baig(MD - India)JasaPengembangan properti hunian dan komersialPendapatanUS$8,5 MiliarKaryawan6,600AnakusahaEmaar Developments, Memaar Building Systems, Emaar International, Emaar Hospitality, Emaar Malls, Emaar Hotels & Resorts, Emaar Retail, Emaar Community Management, Emaar Technologies, Emaar Industries and Investment, OBC El-...

Luca SpicolaL'emblema dell'Ordine: la croce gigliata bianca e nera con il motto laudare, benedicere, praedicare Religioso domenicano  NascitaPontecorvo, 1425 circa MortePontecorvo, 1490 Venerato daChiesa cattolica (attualmente non riconosciuto) BeatificazioneSanta Sede (data non nota) Santuario principaleAttualmente nessuno Ricorrenza22 aprile Patrono dinon noto Manuale Il beato Luca Spicola (Pontecorvo, 1425 circa – Pontecorvo, 1490) è stato un religioso italiano appartenente al...

 

статуя Ангероны во дворце Шёнбрунн в Вене. Ангеро́на (лат. Angerona или Angeronia) — в римской мифологии богиня, изображавшаяся с прижатым к губам пальцем. Её значение было неясно самим древним римлянам. Ангерону связывали с манами, таинственными силами, утешением в горе, исц...

 

Species of fungus Sarcoscypha coccinea Scientific classification Domain: Eukaryota Kingdom: Fungi Division: Ascomycota Class: Pezizomycetes Order: Pezizales Family: Sarcoscyphaceae Genus: Sarcoscypha Species: S. coccinea Binomial name Sarcoscypha coccinea(Scop.) Lambotte (1889) Synonyms[1] List Helvella coccinea Schaeff. (1772) Peziza coccinea Jacq. (1774) Peziza cochleata Batsch (1783) Peziza dichroa Holmsk. (1799) Peziza coccinea Jacq. (1800) Peziza aurantia Schumach. (1803) Ma...

Reservoir in California, United States This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.Find sources: Crowley Lake – news · newspapers · books · scholar · JSTOR (November 2013) (Learn how and when to remove this message) Crowley LakeCrowley LakeShow map of CaliforniaCrowley LakeShow map of the United StatesLocationMono Cou...

 

Coppa Italia Dilettanti 1981-1982 Competizione Coppa Italia Dilettanti Sport Calcio Edizione 16ª Organizzatore Lega Nazionale Dilettanti Date dal 6 settembre 1981al 29 maggio 1982 Luogo  Italia Formula Eliminazione diretta Risultati Vincitore Leffe(1º titolo) Secondo Pro Palazzolo Semi-finalisti Trevigliese e Canosa Cronologia della competizione 1980-1981 1982-1983 Manuale La Coppa Italia Dilettanti 1981-1982 è stata la 16ª edizione di questa competizione calcistica italiana...

 

† Палеопропитеки Научная классификация Домен:ЭукариотыЦарство:ЖивотныеПодцарство:ЭуметазоиБез ранга:Двусторонне-симметричныеБез ранга:ВторичноротыеТип:ХордовыеПодтип:ПозвоночныеИнфратип:ЧелюстноротыеНадкласс:ЧетвероногиеКлада:АмниотыКлада:СинапсидыКласс:�...

Dagger, Ceremonial Knife Kujang Kujang, a typical weapon of Sundanese.TypeDagger, Ceremonial KnifePlace of originIndonesiaService historyUsed bySundanese peopleSpecificationsMass300 g (11 oz) approximatelyLength20–25 cm (7.9–9.8 in)Blade typeSingle edge, convex grindHilt typeWater buffalo horn, woodenScabbard/sheathWater buffalo horn, wooden The kujang is a bladed weapon native to the Sundanese people of Indonesia. The earliest kujang made is f...

 

يفتقر محتوى هذه المقالة إلى الاستشهاد بمصادر. فضلاً، ساهم في تطوير هذه المقالة من خلال إضافة مصادر موثوق بها. أي معلومات غير موثقة يمكن التشكيك بها وإزالتها. (مايو 2021) هذه المقالة يتيمة إذ تصل إليها مقالات أخرى قليلة جدًا. فضلًا، ساعد بإضافة وصلة إليها في مقالات متعلقة بها. (�...

 

South African Airways' first Airbus A350-900 arriving at John F. Kennedy International Airport in New York. As of December 2022[update] the airline does not serve this destination.[1] This is a list of South African Airways destinations, as of January 2024[update].[2] As of June 2016[update], South African Airways served eight destinations outside Africa. By that time, the top five international routes led from Johannesburg to New York C...

Texas settler, born 1795 For other people named George Fisher, see George Fisher (disambiguation). George Fisher Đorđe ŠagićGeorge Fisher, ca. 1843Native nameЂорђе ШагићBirth nameĐorđe ŠagićNickname(s)Đorđe RibarBorn(1795-03-30)30 March 1795Stolni Beograd, Kingdom of Hungary, Habsburg EmpireDied11 June 1873(1873-06-11) (aged 78)San Francisco, California, U.S.Allegiance Revolutionary Serbia (1813) Mexico (1829–1835) Republic of Texas (1836–1846) Unit...

 

Archaeological site in Ohio, United States United States historic placeTower SiteU.S. National Register of Historic Places The site in March 2010Show map of OhioShow map of the United StatesLocationOn a hilltop 4 miles northwest of Barnesville[2]Nearest cityBarnesville, OhioCoordinates40°1′23″N 81°13′16″W / 40.02306°N 81.22111°W / 40.02306; -81.22111Area3.5 acres (1.4 ha)NRHP reference No.82003544[1]Added to NRHPJune 11, 1982 ...

 

MenduaGenre Drama Roman BerdasarkanDoctor FosterDitulis oleh Dono Indarto Sinar Ayu Massie SutradaraPritagita ArianegaraPemeran Adinia Wirasti Chicco Jerikho Tatjana Saphira Bima Azriel Widyawati Penggubah lagu tema Adrian Rahmat Purwanto Keke Kananta Mahalini Marco Steffiano Lagu penutupKisah Sempurna — MahaliniPenata musik Aghi Narottama Bemby Gusti Tony Merle Negara asalIndonesiaBahasa asliBahasa IndonesiaJmlh. musim1Jmlh. episode8ProduksiProduser eksekutifAnthony BuncioProduserWi...

Illustration by W. H. Overend, Illustrated London News, 20 December 1884 On Saturday 13 December 1884 two American-Irish Republicans carried out a dynamite attack on London Bridge as part of the Fenian dynamite campaign. The bomb went off prematurely while the men were in a boat attaching it to a bridge pier at 5.45 pm during the evening rush hour.[1] There was little damage to the bridge, and no casualties other than the bombers, who died instantly. However, there was considerable c...

 

1881 Iowa Senate election ← 1879 October 11, 1881 1883 → 25 out of 50 seats in the Iowa State Senate26 seats needed for a majority   Majority party Minority party Third party   Party Republican Democratic Greenback Last election 41 7 2 Seats after 45 2 2 Seat change 4 5   Fourth party   Party Independent Last election 0 Seats after 1 Seat change 1 Elections in Iowa Federal government U.S. Presidential elections 1848 1852...

 

Cet article est une ébauche concernant la Moldavie et l’Union européenne. Vous pouvez partager vos connaissances en l’améliorant (comment ?) selon les recommandations des projets correspondants. Procédure d'adhésion de la Moldavie à l'Union européenne La Moldavie (en orange)et l'Union européenne (en vert). Données clés Dépôt de candidature 3 mars 2022 Reconnaissance du statut de candidat 23 juin 2022 Chapitre(s) ouvert(s) 0 Chapitre(s) clos 0 Informations Populations res...

هذه المقالة يتيمة إذ تصل إليها مقالات أخرى قليلة جدًا. فضلًا، ساعد بإضافة وصلة إليها في مقالات متعلقة بها. (يوليو 2019) كريستينا ماي معلومات شخصية الميلاد 3 سبتمبر 1961 (63 سنة)  دورتموند  مواطنة ألمانيا  الحياة العملية المهنة منافسة ألعاب القوى  اللغات الألمانية  الر...

 

Disambiguazione – Se stai cercando altri omonimi, vedi Cosimo de' Medici (disambigua). Cosimo I de' MediciBronzino, Ritratto di Cosimo I de' Medici in armatura, olio su tavola, 1545 circa, Galleria degli UffiziGranduca di ToscanaStemma In carica21 agosto 1569 –21 aprile 1574 Incoronazione5 marzo 1570, Roma[1] Predecessoresé stesso come duca di Firenze SuccessoreFrancesco I Duca di FirenzeIn carica6 gennaio 1537 –21 agosto 1569 PredecessoreAlessandro Successoresé ste...