Mean reciprocal rank

The mean reciprocal rank is a statistic measure for evaluating any process that produces a list of possible responses to a sample of queries, ordered by probability of correctness. The reciprocal rank of a query response is the multiplicative inverse of the rank of the first correct answer: 1 for first place, 1⁄2 for second place, 1⁄3 for third place and so on. The mean reciprocal rank is the average of the reciprocal ranks of results for a sample of queries Q:[1][2]

where refers to the rank position of the first relevant document for the i-th query.

The reciprocal value of the mean reciprocal rank corresponds to the harmonic mean of the ranks.

Example

Suppose we have the following three queries for a system that tries to translate English words to their plurals. In each case, the system makes three guesses, with the first one being the one it thinks is most likely correct:

Query Proposed Results Correct response Rank Reciprocal rank
cat catten, cati, cats cats 3 1/3
torus torii, tori, toruses tori 2 1/2
virus viruses, virii, viri viruses 1 1

Given those three samples, we could calculate the mean reciprocal rank as , or approximately 0.61.

If none of the proposed results are correct, the reciprocal rank is 0.[1] Note that only the rank of the first relevant answer is considered, and possible further relevant answers are ignored. If users are also interested in further relevant items, mean average precision is a potential alternative metric.

See also

References

  1. ^ a b E.M. Voorhees (1999). "Proceedings of the 8th Text Retrieval Conference" (PDF). TREC-8 Question Answering Track Report. pp. 77–82.
  2. ^ D. R. Radev; H. Qi; H. Wu; W. Fan (2002). "Evaluating web-based question answering systems" (PDF). Proceedings of LREC.

Read other articles:

Rai 2

Rai 2Diluncurkan4 November 1961; 62 tahun lalu (1961-11-04)PemilikRaiPangsa pemirsa6.85% (Mei 2014, [1])NegaraItaliaBahasaItaliaKantor pusatRoma, ItaiaSaluran seindukRai 1Rai 3Rai 4Rai 5Rai GulpRai MovieRai News 24Rai PremiumRai ScuolaRai SportRai StoriaRai YoyoRai LadiniaRai SüdtirolRai ItaliaSitus webrai.it/rai2Televisi InternetRai.tvLive Streaming Rai 2 (bahasa Italia: Rai Due) adalah sebuah saluran televisi publik kedua yang disiarkan oleh penyiar milik negara Italia, RA...

 

الهجوم على صحيفة شارلي إبدو

الهجوم على صحيفة شارلي إبدو الصحفيون والشرطة والطواقم الطبية في موقع الحادثة بعد عدة ساعات المعلومات البلد فرنسا  الموقع الدائرة 11 في باريس، فرنسا[1] الإحداثيات 48°51′33″N 2°22′13″E / 48.859246°N 2.370258°E / 48.859246; 2.370258 التاريخ 7 يناير 2015 الهدف موظفي صحيفة شارلي إبدو ...

 

Jalan Metro Pondok Indah (Jakarta)

Skybridge penghubung Pondok Indah Mall 1 dan 2 difoto dari Jalan Metro Pondok Indah. Foto diambil pada tahun 2008, terlihat pembangunan Halte Pondok Indah 2 pada sebelah kanan foto. Mal Pondok Indah yang terletak di Jalan Metro Pondok Indah. Jalan Metro Pondok Indah adalah nama salah satu jalan utama Jakarta. Jalan ini melewati kompleks Pondok Indah sepanjang 3,1 km dari perbatasan Jalan Margaguna Raya (Kali Grogol) sampai persimpangan Jalan Raden Ajeng Kartini (Lebak Bulus). Jalan ini b...

Protettorato francese del Marocco

Protettorato francese del Marocco Bandiera mercantile (dettagli) (dettagli) Protettorato francese del Marocco - LocalizzazioneConquista francese del Marocco, 1907-1927 Dati amministrativiNome ufficiale(FR) Protectorat français au Maroc(AR) حماية فرنسا في المغرب Lingue ufficialiarabo, francese Lingue parlatearabo marocchino, berbero, francese InnoLa Marsigliese CapitaleRabat Dipendente da Terza Repubblica Francia di Vichy Francia libera GPRF Quarta Repubblica Polit...

 

趙家驤

追晉陸軍二級上將趙家驤將軍个人资料出生1910年 大清河南省衛輝府汲縣逝世1958年8月23日(1958歲—08—23)（47—48歲） † 中華民國福建省金門縣国籍 中華民國政党 中國國民黨获奖 青天白日勳章（追贈）军事背景效忠 中華民國服役 國民革命軍 中華民國陸軍服役时间1924年－1958年军衔 二級上將 （追晉）部队四十七師指挥東北剿匪總司令部參謀長陸軍�...

 

Lessons in Chemistry

Lessons in ChemistryGenreDrama sejarahBerdasarkanLessons in Chemistryoleh Bonnie GarmusPengembangLee EisenbergPemeran Brie Larson Lewis Pullman Aja Naomi King Stephanie Koenig Patrick Walker Penggubah lagu temaCarlos Rafael Rivera[1]Negara asalAmerika SerikatBahasa asliInggrisJmlh. episode8ProduksiProduser eksekutif Jason Bateman Michael Costigan Susannah Grant Brie Larson Lee Eisenberg Natalie Sandy Sinematografi Zachary Galler Jason Oldak Penyunting Matthew James Barbato Jack Cunni...

Neuhütten

Neuhütten Lambang kebesaranLetak Neuhütten NegaraJermanNegara bagianBayernWilayahUnterfrankenKreisMain-SpessartMunicipal assoc.Partenstein Pemerintahan • MayorEdmund Wirzberger (CSU)Luas • Total5,95 km2 (230 sq mi)Ketinggian270 m (890 ft)Populasi (2013-12-31)[1] • Total1.175 • Kepadatan2,0/km2 (5,1/sq mi)Zona waktuWET/WMPET (UTC+1/+2)Kode pos97843Kode area telepon06020Pelat kendaraanMSPSitus webwww.ne...

 

Siege of Virovitica (1684)

A siege of Slavonian town of Virovitica during Great Turkish War Siege of ViroviticaPart of Croatian-Slavonian-Dalmatian theater in Great Turkish WarDepiction of the siege from Karlsruhe archive.DateJuly 1684LocationSanjak of Pojega, modern day Slavonia, CroatiaResult Croatian-Habsburg victory Croatian-Habsburg capture of townBelligerents Army of Croatian ban Imperial Habsburg Army Ottoman TurksCommanders and leaders gen. James Leslie gen. Sigismund Trautmansdorf Nikola Erdody, ban of Croatia...

 

نصير الدين الطوسي

  ميّز عن شرف الدين الطوسي.   هذه المقالة عن نصير الدين الطوسي. لمعانٍ أخرى، طالع الطوسي (توضيح). نصير الدين الطوسي (بالفارسية: نصيرالدین طوسی)‏  صورة تخيلية لنصير الدين الطوسي مشتقة من طابع أذري صدر في عام 2001. معلومات شخصية اسم الولادة مُحمد بن مُحمد بن الحسن الط�...

Big Spring, Texas

City in Texas, United StatesBig Spring, TexasCityThe Municipal Auditorium in Big SpringNickname: The Spring CityBig SpringShow map of TexasBig SpringShow map of the United StatesCoordinates: 32°14′36″N 101°28′31″W / 32.24333°N 101.47528°W / 32.24333; -101.47528Country United StatesState TexasCountyHowardGovernment • TypeCouncil-Manager • City CouncilMayor Robert Moore Nick Ornelas Diane Yanez Cody Hughes Gloria McDona...

 

Lithography

Printing technique This article is about a printing method. For rock types, see Lithology. For the microfabrication process, see microlithography. A lithograph of Charles Marion Russell's The Custer Fight (1903), with the range of tones fading toward the edges Part of a series on theHistory of printing TechniquesWoodblock printing200Movable type1040Intaglio (printmaking)1430Printing pressc. 1440Etchingc. 1515Mezzotint1642Relief printing1690Aquatint1772Lithography1796Chromolithogra...

 

Indian elephant

Most widespread subspecies of Asian elephant For the species also referred to as the Indian elephant, see Asian elephant. Indian elephantTemporal range: Pleistocene – Recent[1] Male Female Conservation status Endangered  (IUCN 3.1)[2] (Elephas maximus) Scientific classification Domain: Eukaryota Kingdom: Animalia Phylum: Chordata Class: Mammalia Order: Proboscidea Family: Elephantidae Genus: Elephas Species: E. maximus Subspecies: E. m. indicus Trinomial name ...

Elettronegatività

L'elettronegatività, simbolo χ, è una proprietà chimica che descrive la tendenza di un atomo ad attrarre verso di sé elettroni condivisi.[1] Al livello più elementare, l'elettronegatività è determinata da fattori come la carica nucleare (più protoni ha un atomo, più attrarrà gli elettroni) e il numero e posizione degli altri elettroni presenti nei vari orbitali atomici (più elettroni ha un atomo, più sono lontani dal nucleo gli elettroni di valenza, che saranno quindi sog...

 

Granary Square

Urban square in the London Borough of Camden Road bridge to Granary Square with Central Saint Martins college behind Granary Square is a large open space in the London Borough of Camden. Comparable in size to Trafalgar Square,[1] it is part of the larger King's Cross Central development. It has been cited as an example of a privately owned public space in London.[2] The new Central Saint Martins complex is adjacent. Visitor attractions in the square include Queer Britain, the ...

 

العلاقات اليمنية التركية

يفتقر محتوى هذه المقالة إلى الاستشهاد بمصادر. فضلاً، ساهم في تطوير هذه المقالة من خلال إضافة مصادر موثوق بها. أي معلومات غير موثقة يمكن التشكيك بها وإزالتها. (ديسمبر 2018) العلاقات اليمنية التركية     تركيا   اليمن السفارات سفارة اليمن في تركيا سفارة تركيا في اليم�...

Border (cricket team)

Cricket team Border CricketPersonnelCaptainJason NiemandCoachTumelo BodibeChief executiveOmphile RamelaTeam informationFounded1898Home groundBuffalo Park , East LondonOfficial websiteOfficial website Border is the team representing the Border region in domestic first-class cricket in South Africa. The team began playing in March 1898. When Cricket South Africa introduced the franchise system in 2004, Border merged with Eastern Province to form the Warriors. Honours Currie Cup (0) -  ; sh...

 

Robert Strettell

Irish-born mayor of Philadelphia, Pennsylvania Robert Strettell (1693–1762) was a city councilman and mayor of Philadelphia. He was born in Dublin, Ireland, in 1693, the son of Amos Strettell and moved to London shortly thereafter.[1] In 1736 he came with his family to Philadelphia, where he worked as a merchant. He was a member of the Common Council of the City of Philadelphia, a member of the Governor's Council, and Mayor of Philadelphia. Strettell was a trustee of the Academy and...

 

Caduta di Kabul (2021)

Caduta di Kabulparte dell'offensiva talebana del 2021Talebani pattugliano Kabul con un Humvee, 17 agosto 2021Data15 agosto - 16 agosto 2021 LuogoKabul, Afghanistan Esito Occupazione di Kabul; Breve controllo degli Stati Uniti e dei suoi alleati sull'Aeroporto Internazionale Hamid Karzai ai fini dell’evacuazione. Modifiche territorialiConquista della capitale afghana da parte dei Talebani Schieramenti AfghanistanSupporto per l'evacuazione: Stati Uniti Regno Unito Italia&#...

八神純子

このページのノートに、このページに関する注意があります。注意の要約：本人の写真掲載について 八神 純子出生名 八神 純子別名 ジューン・スタンレー生誕 (1958-01-05) 1958年1月5日（66歳）出身地 日本・愛知県名古屋市千種区学歴 愛知淑徳高等学校卒[1]ジャンル ニューミュージック[2]シティ・ポップ[3][4][5]J-POP職業 歌手・シンガーソング�...

 

العريش (مادبا)

  لمعانٍ أخرى، طالع العريش (توضيح). العريش  - منطقة سكنية -  تقسيم إداري البلد الأردن  المحافظة محافظة مادبا لواء لواء قصبة مادبا قضاء قضاء جرينة السكان التعداد السكاني 1726 نسمة (إحصاء 2015)   • الذكور 892   • الإناث 834   • عدد الأسر 353 معلومات أخرى التوقيت �...