Letter frequency is the number of times letters of the alphabet appear on average in written language. Letter frequency analysis dates back to the Arab mathematician Al-Kindi (c. 801–873 AD), who formally developed the method to break ciphers. Letter frequency analysis gained importance in Europe with the development of movable type in 1450 AD, where one must estimate the amount of type required for each letterform. Linguists use letter frequency analysis as a rudimentary technique for language identification, where it is particularly effective as an indication of whether an unknown writing system is alphabetic, syllabic, or ideographic.
The use of letter frequencies and frequency analysis plays a fundamental role in cryptograms and several word puzzle games, including hangman, Scrabble, Wordle[2] and the television game show Wheel of Fortune. One of the earliest descriptions in classical literature of applying the knowledge of English letter frequency to solving a cryptogram is found in Edgar Allan Poe's famous story "The Gold-Bug", where the method is successfully applied to decipher a message giving the location of a treasure hidden by Captain Kidd.[3][citation needed]
Herbert S. Zim, in his classic introductory cryptography text Codes and Secret Writing, gives the English letter frequency sequence as "ETAON RISHD LFCMU GYPWB VKJXZQ", the most common letter pairs as "TH HE AN RE ER IN ON AT ND ST ES EN OF TE ED OR TI HI AS TO", and the most common doubled letters as "LL EE SS OO TT FF RR NN PP CC".[4] Different ways of counting can produce somewhat different orders.
The frequency of letters in text has been studied for use in cryptanalysis, and frequency analysis in particular, dating back to the Arab mathematician al-Kindi (c. 801–873 AD), who formally developed the method (the ciphers breakable by this technique go back at least to the Caesar cipher used by Julius Caesar,[citation needed] so this method could have been explored in classical times). Letter frequency analysis gained additional importance in Europe with the development of movable type in 1450 AD, where one must estimate the amount of type required for each letterform, as evidenced by the variations in letter compartment size in typographer's type cases.
No exact letter frequency distribution underlies a given language, since all writers write slightly differently. However, most languages have a characteristic distribution which is strongly apparent in longer texts. Even language changes as extreme as from Old English to modern English (regarded as mutually unintelligible) show strong trends in related letter frequencies: over a small sample of Biblical passages, from most frequent to least frequent, enaid sorhm tgþlwu æcfy ðbpxz of Old English compares to eotha sinrd luymw fgcbp kvjqxz of modern English, with the most extreme differences concerning letterforms not shared.[5]
Linotype machines for the English language assumed the letter order, from most to least common, to be etaoin shrdlu cmfwyp vbgkqj xz based on the experience and custom of manual compositors. The equivalent for the French language was elaoin sdrétu cmfhyp vbgwqj xz.
Arranging the alphabet in Morse into groups of letters that require equal amounts of time to transmit, and then sorting these groups in increasing order, yields e it san hurdm wgvlfbk opxcz jyq.[a] Letter frequency was used by other telegraph systems, such as the Murray Code.
Letter frequencies, like word frequencies, tend to vary, both by writer and by subject. For instance, ⟨d⟩ occurs with greater frequency in fiction, as most fiction is written in past tense and thus most verbs will end in the inflectional suffix -ed / -d. One cannot write an essay about x-rays without using ⟨x⟩ frequently. Different authors have habits which can be reflected in their use of letters. Hemingway's writing style, for example, is visibly different from Faulkner's. Letter, bigram, trigram, word frequencies, word length, and sentence length can be calculated for specific authors, and used to prove or disprove authorship of texts, even for authors whose styles are not so divergent.
Accurate average letter frequencies can only be gleaned by analyzing a large amount of representative text. With the availability of modern computing and collections of large text corpora, such calculations are easily made. Examples can be drawn from a variety of sources (press reporting, religious texts, scientific texts and general fiction) and there are differences especially for general fiction with the position of ⟨h⟩ and ⟨i⟩, with ⟨h⟩ becoming more common.
Different dialects of a language will also affect a letter's frequency. For example, an author in the United States would produce something in which ⟨z⟩ is more common than an author in the United Kingdom writing on the same topic: words like "analyze", "apologize", and "recognize" contain the letter in American English, whereas the same words are spelled "analyse", "apologise", and "recognise" in British English. This would highly affect the frequency of the letter ⟨z⟩, as it is rarely used by British writers in the English language.[6]
The "top twelve" letters constitute about 80% of the total usage. The "top eight" letters constitute about 65% of the total usage. Letter frequency as a function of rank can be fitted well by several rank functions, with the two-parameter Cocho/Beta rank function being the best.[7] Another rank function with no adjustable free parameter also fits the letter frequency distribution reasonably well[8] (the same function has been used to fit the amino acid frequency in protein sequences.[9]) A spy using the VIC cipher or some other cipher based on a straddling checkerboard typically uses a mnemonic such as "a sin to err" (dropping the second "r")[10][11] or "at one sir"[12] to remember the top eight characters.
Relative frequencies of letters in the English language
There are three ways to count letter frequency that result in very different charts for common letters. The first method, used in the chart below, is to count letter frequency in lemmas of a dictionary. The lemma is the word in its canonical form. The second method is to include all word variants when counting, such as "abstracts", "abstracted" and "abstracting" and not just the lemma of "abstract". This second method results in letters like ⟨s⟩ appearing much more frequently, such as when counting letters from lists of the most used English words on the Internet. ⟨s⟩ is especially common in inflected words (non-lemma forms) because it is added to form plurals and third person singular present tense verbs. A final method is to count letters based on their frequency of use in actual texts, resulting in certain letter combinations like ⟨th⟩ becoming more common due to the frequent use of common words like "the", "then", "both", "this", etc. Absolute usage frequency measures like this are used when creating keyboard layouts or letter frequencies in old fashioned printing presses.
An analysis of entries in the Concise Oxford dictionary, ignoring frequency of word use, gives an order of "EARIOTNSLCUDPMHGBFYWKVXZJQ".[13]
The letter-frequency table below is taken from Pavel Mička's website, which cites Robert Lewand's Cryptological Mathematics.[14]
According to Lewand, arranged from most to least common in appearance, the letters are: etaoinshrdlcumwfgypbvkjxqz. Lewand's ordering differs slightly from others, such as Cornell University Math Explorer's Project, which produced a table after measuring 40,000 words.[15]
In English, the space character occurs almost twice as frequently as the top letter (⟨e⟩)[16] and the non-alphabetic characters (digits, punctuation, etc.) collectively occupy the fourth position (having already included the space) between ⟨t⟩ and ⟨a⟩.[17]
Relative frequencies of the first letters of a word in English language
Letter
Relative frequency as the first letter of an English word[citation needed]
Texts
Dictionaries
A
11.7%
11.7
5.7%
5.7
B
4.4%
4.4
6%
6
C
5.2%
5.2
9.4%
9.4
D
3.2%
3.2
6.1%
6.1
E
2.8%
2.8
3.9%
3.9
F
4%
4
4.1%
4.1
G
1.6%
1.6
3.3%
3.3
H
4.2%
4.2
3.7%
3.7
I
7.3%
7.3
3.9%
3.9
J
0.51%
0.51
1.1%
1.1
K
0.86%
0.86
1%
1
L
2.4%
2.4
3.1%
3.1
M
3.8%
3.8
5.6%
5.6
N
2.3%
2.3
2.2%
2.2
O
7.6%
7.6
2.5%
2.5
P
4.3%
4.3
7.7%
7.7
Q
0.22%
0.22
0.49%
0.49
R
2.8%
2.8
6%
6
S
6.7%
6.7
11%
11
T
16%
16
5%
5
U
1.2%
1.2
2.9%
2.9
V
0.82%
0.82
1.5%
1.5
W
5.5%
5.5
2.7%
2.7
X
0.045%
0.045
0.05%
0.05
Y
0.76%
0.76
0.36%
0.36
Z
0.045%
0.045
0.24%
0.24
The frequency of the first letters of words or names is helpful in pre-assigning space in physical files and indexes.[18] Given 26 filing cabinet drawers, rather than a 1:1 assignment of one drawer to one letter of the alphabet, it is often useful to use a more equal-frequency-letter code by assigning several low-frequency letters to the same drawer (often one drawer is labeled VWXYZ), and to split up the most-frequent initial letters (⟨s, a, c⟩) into several drawers (often 6 drawers Aa-An, Ao-Az, Ca-Cj, Ck-Cz, Sa-Si, Sj-Sz). The same system is used in some multi-volume works such as some encyclopedias. Cutter numbers, another mapping of names to a more equal-frequency code, are used in some libraries.
Both the overall letter distribution and the word-initial letter distribution approximately match the Zipf distribution and even more closely match the Yule distribution.[19]
Often the frequency distribution of the first digit in each datum is significantly different from the overall frequency of all the digits in a set of numeric data, an observation known as Benford's law.
The figure below illustrates the frequency distributions of the 26 most common Latin letters across some languages. All of these languages use a similar 25+ character alphabet.
^American Morse code was developed in the 1830s by Alfred Vail, based on English-language letter frequencies, to encode the most frequent letters with the shortest symbols. Some efficiency was lost in the reformed version now used: the International Morse Code.
References
^Mička, Pavel. "Letter frequency (English)". Algoritmy.net. Archived from the original on 4 March 2021. Retrieved 14 June 2022. Source is Leland, Robert. Cryptological mathematics. [s.l.] : The Mathematical Association of America, 2000. 199 p. ISBN 0-88385-719-7
^Li, Wentian; Miramontes, Pedro (2011). "Fitting ranked English and Spanish letter frequency distribution in US and Mexican presidential speeches". Journal of Quantitative Linguistics. 18 (4): 359. arXiv:1103.2950. doi:10.1080/09296174.2011.608606. S2CID1716455.
^Gusein-Zade, S.M. (1988). "Frequency distribution of letters in the Russian language". Probl. Peredachi Inf. 24 (4): 102–107.
^Serengil, Sefik Ilkin; Akin, Murat (20–22 February 2011). Attacking Turkish Texts Encrypted by Homophonic Cipher(PDF). Proceedings of the 10th WSEAS International Conference on Electronics, Hardware, Wireless and Optical Communications. Cambridge, UK. pp. 123–126.
Useful tables for single letter, digram, trigram, tetragram, and pentagram frequencies based on 20,000 words that take into account word-length and letter-position combinations for words 3 to 7 letters in length:
Mayzner, M.S.; Tresselt, M.E.; Wolin, B.R. (1965). "Tables of single-letter and digram frequency counts for various word-length and letter-position combinations". Psychonomic Monograph Supplements. 1 (2): 13–32. OCLC639975358.
Mayzner, M.S.; Tresselt, M.E.; Wolin, B.R. (1965). "Tables of trigram frequency counts for various word-length and letter-position combinations". Psychonomic Monograph Supplements. 1 (3): 33–78.
Mayzner, M.S.; Tresselt, M.E.; Wolin, B.R. (1965). "Tables of tetragram frequency counts for various word-length and letter-position combinations". Psychonomic Monograph Supplements. 1 (4): 79–143.
Mayzner, M.S.; Tresselt, M.E.; Wolin, B.R. (1965). "Tables of pentagram frequency counts for various word-length and letter-position combinations". Psychonomic Monograph Supplements. 1 (5): 144–190.
Artikel ini menjelaskan secara umum. Untuk bentuk bahasa yang dituturkan pada tahap tertentu, lihat Pra-Helenik, Proto-Helenik, Yunani Mikenai, Yunani Kuno, Yunani Koine, Yunani Pertengahan, dan Yunani Modern. Cari artikel bahasa Cari berdasarkan kode ISO 639 (Uji coba) Kolom pencarian ini hanya didukung oleh beberapa antarmuka Halaman bahasa acak Bahasa Yunani Ἑλληνική/ΕλληνικάHellēnikḗ/Ellinika Pengucapan[hel.lɛː.ni.ke] atau [eliniˈka]Dituturkan diYun...
Pour les articles homonymes, voir Hanotaux. Gabriel HanotauxFonctionsFauteuil 29 de l'Académie française1er avril 1897 - 11 avril 1944Paul-Armand Challemel-LacourAndré SiegfriedMinistre des Affaires étrangères29 avril 1896 - 28 juin 1898Ministre des Affaires étrangères30 mai 1894 - 1er novembre 1895Député de l'Aisne18 avril 1886 - 14 octobre 1889PrésidentFrance-AmériquesBiographieNaissance 19 novembre 1853BeaurevoirDécès 11 avril 1944 (à 90 ans)ParisSépulture Cimetière d...
Medieval kingdom in Scotland Political centres in Scotland in the early Middle Ages The Kingdom of Alba (Latin: Scotia; Scottish Gaelic: Alba) was the Kingdom of Scotland between the deaths of Donald II in 900 and of Alexander III in 1286. The latter's death led indirectly to an invasion of Scotland by Edward I of England in 1296 and the First War of Scottish Independence. Alba included Dalriada, but initially excluded large parts of the present-day Scottish Lowlands, which were then divided ...
UFC mixed martial arts event in 2020 UFC on ESPN: Poirier vs. HookerThe poster for UFC on ESPN: Poirier vs. HookerInformationPromotionUltimate Fighting ChampionshipDateJune 27, 2020 (2020-06-27)VenueUFC ApexCityEnterprise, Nevada, United StatesAttendanceNone (behind closed doors)[1]Event chronology UFC on ESPN: Blaydes vs. Volkov UFC on ESPN: Poirier vs. Hooker UFC 251: Usman vs. Masvidal UFC on ESPN: Poirier vs. Hooker (also known as UFC on ESPN 12 and UFC Vegas 4) was...
Ця стаття потребує додаткових посилань на джерела для поліпшення її перевірності. Будь ласка, допоможіть удосконалити цю статтю, додавши посилання на надійні (авторитетні) джерела. Зверніться на сторінку обговорення за поясненнями та допоможіть виправити недоліки. Мат...
River in Staffordshire, England River BlitheBlithe near to The BlytheConfluence with the River Trent shown in StaffordshireLocationCountryEnglandCountiesStaffordshirePhysical characteristicsSource • locationHeywood Grange Farm, Staffordshire Mouth • locationKing's Bromley, Staffordshire • coordinates52°45′21″N 1°49′54″W / 52.75580°N 1.83163°W / 52.75580; -1.83163Length29 km (18 mi)Ba...
سليم بيغ تقسيم إداري البلد إيران [1] التقسيم الأعلى محافظة أذربيجان الغربية إحداثيات 37°24′40″N 44°55′36″E / 37.41111111°N 44.92666667°E / 37.41111111; 44.92666667 الرمز الجغرافي 20858 تعديل مصدري - تعديل سليم بيغ هي قرية في مقاطعة أرومية، إيران. عدد سكان هذه القرية هو...
1966 novel by Bernard Malamud For other uses, see The Fixer (disambiguation). The Fixer (novel) First editionAuthorBernard MalamudCountryUnited StatesLanguageEnglishPublisherFarrar, Straus & GirouxPublication date1966Media typePrintPreceded byIdiots First (1963) Followed byPictures of Fidelman (1969) The Fixer is a novel by Bernard Malamud published in 1966 by Farrar, Straus & Giroux.[1] It won the U.S. National Book Award for Fiction (his second)[...
Bagian dari seri PolitikBentuk dasar dari pemerintahan Struktur kekuatan Konfederasi Federasi Hegemoni Kerajaan Negara kesatuan Sumber kekuatan Demokrasi Langsung Perwakilan Semi lainnya Kerajaan Mutlak Konstitusi Oligarki Aristokrasi Junta militer Kleptokrasi Plutokrasi Stratokrasi Timokrasi Otokrasi Otoritarianisme Despotisme Diktatur (Kediktatoran) Totalitarianisme Republik Parlementer Presidensial Semi presidensial Lainnya Anarki Anokrasi Khilafah Kritarsi Meritokrasi Oklokrasi Parti...
This article is about Barmer district. For Barmer headquarter, see Barmer, Rajasthan. District of Rajasthan in IndiaBarmer district बाड़मेर जिलाDistrict of RajasthanClockwise from top-left: Kiradu Mandir, Siwana Fort, Nakodaji, Viratra Vankal Mata Temple,Barmer, A view of Sams Sand Dunes, Siwana FortBarmer District in RajasthanCoordinates (Barmer, Rajasthan|Barmer): 25°45′N 71°23′E / 25.75°N 71.38°E / 25.75; 71.38Country IndiaStat...
Voce principale: Eccellenza 1993-1994. Eccellenza Trentino-Alto Adige(DE) Oberliga Trentino-Südtirol1993-1994 Competizione Eccellenza Trentino-Alto Adige Sport Calcio Edizione 3ª Organizzatore FIGC - LNDComitato Regionale Trentino-Alto Adige Luogo Italia Cronologia della competizione 1992-1993 1994-1995 Manuale Il campionato italiano di calcio di Eccellenza regionale 1993-1994 è stato il terzo organizzato in Italia. Rappresenta il sesto livello del calcio italiano. Questo è il camp...
Reruntuhan Gereja Santo Paulus Reruntuhan St. Paul Nama Tionghoa Hanzi: 大三巴牌坊 Alih aksara Mandarin - Hanyu Pinyin: Dàsānbā Páifāng Yue (Kantonis) - Romanisasi Yale: daaih sāam bā pàaihfōng - Jyutping: daai6 saam1 baa1 paai4 fong1 Nama Portugis Portugis: Ruínas de São Paulo Wikimedia Commons memiliki media mengenai Cathedral of Saint Paul in Macau. Reruntuhan St.Paul (Hanzi: 大三巴牌坊; Pinyin: Dàsānbā Páifāng) merujuk kepada reruntuhan dari sebuah komple...
Pour les articles homonymes, voir Sol Invictus. Cet article est une ébauche concernant la Rome antique. Vous pouvez partager vos connaissances en l’améliorant (comment ?) selon les recommandations des projets correspondants. Sol Invictus Dieu de la mythologie romaine Disque dédié à Sol Invictusportant la couronne radiée,(Pessinonte, IIIe siècle). Caractéristiques Autre(s) nom(s) Helgabal Fonction principale Dieu de la lumière, de la justice et de la vérité Fonction sec...
Peruvian water skier (born 1997) Natalia CuglievanPersonal informationNational team PeruBorn (1997-01-20) 20 January 1997 (age 27) Medal record Representing Peru Pan American Games Women's tricks 2015 Toronto 2019 Lima Natalia Cuglievan (born 20 March 1997) is a Peruvian water skier. She won gold medals for Peru at the 2015 and 2019 Pan American Games. Biography Natalia Cuglievan was born on 20 March 1997[1] in Peru. She began water skiing in 2005,[2] prac...
Tacoma-class patrol frigate USS Sausalito (PF-4), bow view, taken in dry dock while she was being prepared for transfer to the Republic of Korea, c. 1952, Yokosuka, Japan. History United States NameSausalito NamesakeCity of Sausalito, California Orderedas a Type S2-S2-AQ1 hull, MCE hull 1422[1] BuilderPermanente Metals Richmond Shipyard #4, Richmond, California Yard number47[1] Laid down7 April 1943 ReclassifiedPatrol Frigate (PF), 15 April 1943 Launched20 July 1943 Sponsored&...
This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages) This article's lead section may be too short to adequately summarize the key points. Please consider expanding the lead to provide an accessible overview of all important aspects of the article. (July 2013) This article contains content that is written like an advertisement. Please help improve it by removing promotional content and inappropriate ext...
Group intelligence that emerges from collective efforts Types of collective intelligence Part of a series onCollective intelligence Concepts Collaborative intelligence Collective wisdom Common sense Intelligence assessment Background concepts Collective vte Recommender systems Concepts Collective intelligence Relevance Star ratings Long tail Methods and challenges Cold start Collaborative filtering Dimensionality reduction Implicit data collection Item-item collaborative filtering Matrix fact...
This article is about Southampton players with at least 100 appearances. For Southampton players with between 25 and 99 appearances, see List of Southampton F.C. players (25–99 appearances). For Southampton players with fewer than 25 appearances, see List of Southampton F.C. players (1–24 appearances). Matt Le Tissier is the second-highest all-time goalscorer and fourth-highest appearance-maker for Southampton. Southampton Football Club is an English association football club based in So...
British politician Howard in 1895. Joseph Howard (9 May 1834 – 2 March 1923) was an English barrister and Conservative politician who sat in the House of Commons from 1885 to 1906. Howard was born in Tottenham, the son of John Eliot Howard and Mary Crewdson and was a grandson of Luke Howard. His father was a chemist noted for pioneering work with quinine and a Quaker.[1] Howard was educated at London University and was called to the bar at Lincoln's Inn. He was a J.P. for Middlesex ...