Von Mises–Fisher distribution

In directional statistics, the von Mises–Fisher distribution (named after Richard von Mises and Ronald Fisher), is a probability distribution on the -sphere in . If the distribution reduces to the von Mises distribution on the circle.

Definition

The probability density function of the von Mises–Fisher distribution for the random p-dimensional unit vector is given by:

where and the normalization constant is equal to

where denotes the modified Bessel function of the first kind at order . If , the normalization constant reduces to

The parameters and are called the mean direction and concentration parameter, respectively. The greater the value of , the higher the concentration of the distribution around the mean direction . The distribution is unimodal for , and is uniform on the sphere for .

The von Mises–Fisher distribution for is also called the Fisher distribution.[1][2] It was first used to model the interaction of electric dipoles in an electric field.[3] Other applications are found in geology, bioinformatics, and text mining.

Note on the normalization constant

In the textbook, Directional Statistics [3] by Mardia and Jupp, the normalization constant given for the Von Mises Fisher probability density is apparently different from the one given here: . In that book, the normalization constant is specified as:

where is the gamma function. This is resolved by noting that Mardia and Jupp give the density "with respect to the uniform distribution", while the density here is specified in the usual way, with respect to Lebesgue measure. The density (w.r.t. Lebesgue measure) of the uniform distribution is the reciprocal of the surface area of the (p-1)-sphere, so that the uniform density function is given by the constant:

It then follows that:

While the value for was derived above via the surface area, the same result may be obtained by setting in the above formula for . This can be done by noting that the series expansion for divided by has but one non-zero term at . (To evaluate that term, one needs to use the definition .)

Support

The support of the Von Mises–Fisher distribution is the hypersphere, or more specifically, the -sphere, denoted as

This is a -dimensional manifold embedded in -dimensional Euclidean space, .

Relation to normal distribution

Starting from a normal distribution with isotropic covariance and mean of length , whose density function is:

the Von Mises–Fisher distribution is obtained by conditioning on . By expanding

and using the fact that the first two right-hand-side terms are fixed, the Von Mises-Fisher density, is recovered by recomputing the normalization constant by integrating over the unit sphere. If , we get the uniform distribution, with density .

More succinctly, the restriction of any isotropic multivariate normal density to the unit hypersphere, gives a Von Mises-Fisher density, up to normalization.

This construction can be generalized by starting with a normal distribution with a general covariance matrix, in which case conditioning on gives the Fisher-Bingham distribution.

Estimation of parameters

Mean direction

A series of N independent unit vectors are drawn from a von Mises–Fisher distribution. The maximum likelihood estimates of the mean direction is simply the normalized arithmetic mean, a sufficient statistic:[3]

Concentration parameter

Use the modified Bessel function of the first kind to define

Then:

Thus is the solution to

A simple approximation to is (Sra, 2011)

A more accurate inversion can be obtained by iterating the Newton method a few times

Standard error

For N ≥ 25, the estimated spherical standard error of the sample mean direction can be computed as:[4]

where

It is then possible to approximate a a spherical confidence interval (a confidence cone) about with semi-vertical angle:

where

For example, for a 95% confidence cone, and thus

Expected value

The expected value of the Von Mises–Fisher distribution is not on the unit hypersphere, but instead has a length of less than one. This length is given by as defined above. For a Von Mises–Fisher distribution with mean direction and concentration , the expected value is:

.

For , the expected value is at the origin. For finite , the length of the expected value is strictly between zero and one and is a monotonic rising function of .

The empirical mean (arithmetic average) of a collection of points on the unit hypersphere behaves in a similar manner, being close to the origin for widely spread data and close to the sphere for concentrated data. Indeed, for the Von Mises–Fisher distribution, the expected value of the maximum-likelihood estimate based on a collection of points is equal to the empirical mean of those points.

Entropy and KL divergence

The expected value can be used to compute differential entropy and KL divergence.

The differential entropy of is:

where the angle brackets denote expectation. Notice that the entropy is a function of only.

The KL divergence between and is:

Transformation

Von Mises-Fisher (VMF) distributions are closed under orthogonal linear transforms. Let be a -by- orthogonal matrix. Let and apply the invertible linear transform: . The inverse transform is , because the inverse of an orthogonal matrix is its transpose: . The Jacobian of the transform is , for which the absolute value of its determinant is 1, also because of the orthogonality. Using these facts and the form of the VMF density, it follows that:

One may verify that since and are unit vectors, then by the orthogonality, so are and .

Pseudo-random number generation

General case

An algorithm for drawing pseudo-random samples from the Von Mises Fisher (VMF) distribution was given by Ulrich[5] and later corrected by Wood.[6] An implementation in R is given by Hornik and Grün;[7] and a fast Python implementation is described by Pinzón and Jung.[8]

To simulate from a VMF distribution on the -dimensional unitsphere, , with mean direction , these algorithms use the following radial-tangential decomposition for a point  :

where lives in the tangential -dimensional unit-subsphere that is centered at and perpendicular to ; while . To draw a sample from a VMF with parameters and , must be drawn from the uniform distribution on the tangential subsphere; and the radial component, , must be drawn independently from the distribution with density:

where . The normalization constant for this density may be verified by using:

as given in Appendix 1 (A.3) in Directional Statistics.[3] Drawing the samples from this density by using a rejection sampling algorithm is explained in the above references. To draw the uniform samples perpendicular to , see the algorithm in,[8] or otherwise a Householder transform can be used as explained in Algorithm 1 in.[9]

3-D sphere

To generate a Von Mises–Fisher distributed pseudo-random spherical 3-D unit vector[10][11] on the sphere for a given and , define

where is the polar angle, the azimuthal angle, and the distance to the center of the sphere

for the pseudo-random triplet is then given by

where is sampled from the continuous uniform distribution with lower bound and upper bound

and

where is sampled from the standard continuous uniform distribution

here, should be set to when and rotated to match any other desired .

Distribution of polar angle

For , the angle θ between and satisfies . It has the distribution

,

which can be easily evaluated as

.

For the general case, , the distribution for the cosine of this angle:

is given by , as explained above.

The uniform hypersphere distribution

When , the Von Mises–Fisher distribution, on simplifies to the uniform distribution on . The density is constant with value . Pseudo-random samples can be generated by generating samples in from the standard multivariate normal distribution, followed by normalization to unit norm.

Component marginal of uniform distribution

For , let be any component of . The marginal distribution for has the density:[12][13]

where is the beta function. This distribution may be better understood by highlighting its relation to the beta distribution:

where the Legendre duplication formula is useful to understand the relationships between the normalization constants of the various densities above.

Note that the components of are not independent, so that the uniform density is not the product of the marginal densities; and cannot be assembled by independent sampling of the components.

Distribution of dot-products

In machine learning, especially in image classification, to-be-classified inputs (e.g. images) are often compared using cosine similarity, which is the dot product between intermediate representations in the form of unitvectors (termed embeddings). The dimensionality is typically high, with at least several hundreds. The deep neural networks that extract embeddings for classification should learn to spread the classes as far apart as possible and ideally this should give classes that are uniformly distributed on .[14] For a better statistical understanding of across-class cosine similarity, the distribution of dot-products between unitvectors independently sampled from the uniform distribution may be helpful.


Let be unitvectors in , independently sampled from the uniform distribution. Define:

where is the dot-product and are transformed versions of it. Then the distribution for is the same as the marginal component distribution given above;[13] the distribution for is symmetric beta and the distribution for is symmetric logistic-beta:

The means and variances are:

and

where is the first polygamma function. The variances decrease, the distributions of all three variables become more Gaussian, and the final approximation gets better as the dimensionality, , is increased.

Generalizations

Matrix Von Mises-Fisher

The matrix von Mises-Fisher distribution (also known as matrix Langevin distribution[15][16]) has the density

supported on the Stiefel manifold of orthonormal p-frames , where is an arbitrary real matrix.[17][18]

Saw distributions

Ulrich,[5] in designing an algorithm for sampling from the VMF distribution, makes use of a family of distributions named after and explored by John G. Saw.[19] A Saw distribution is a distribution on the -sphere, , with modal vector and concentration , and of which the density function has the form:

where is a non-negative, increasing function; and where is the normalization constant. The above-mentioned radial-tangential decomposition generalizes to the Saw family and the radial compoment, has the density:

where is the beta function. Also notice that the left-hand factor of the radial density is the surface area of .

By setting , one recovers the VMF distribution.

Weighted Rademacher Distribution

The definition of the Von Mises-Fisher distribution can be extended to include also the case where , so that the support is the 0-dimensional hypersphere, which when embedded into 1-dimensional Euclidean space is the discrete set, . The mean direction is and the concentration is . The probability mass function, for is:

where is the logistic sigmoid. The expected value is . In the uniform case, at , this distribution degenerates to the Rademacher distribution.

See also

References

  1. ^ Fisher, R. A. (1953). "Dispersion on a sphere". Proc. R. Soc. Lond. A. 217 (1130): 295–305. Bibcode:1953RSPSA.217..295F. doi:10.1098/rspa.1953.0064. S2CID 123166853.
  2. ^ Watson, G. S. (1980). "Distributions on the Circle and on the Sphere". J. Appl. Probab. 19: 265–280. doi:10.2307/3213566. JSTOR 3213566. S2CID 222325569.
  3. ^ a b c d Mardia, Kanti; Jupp, P. E. (1999). Directional Statistics. John Wiley & Sons Ltd. ISBN 978-0-471-95333-3.
  4. ^ Embleton, N. I. Fisher, T. Lewis, B. J. J. (1993). Statistical analysis of spherical data (1st pbk. ed.). Cambridge: Cambridge University Press. pp. 115–116. ISBN 0-521-45699-1.{{cite book}}: CS1 maint: multiple names: authors list (link)
  5. ^ a b Ulrich, Gary (1984). "Computer generation of distributions on the m-sphere". Applied Statistics. 33 (2): 158–163. doi:10.2307/2347441. JSTOR 2347441.
  6. ^ Wood, Andrew T (1994). "Simulation of the Von Mises Fisher distribution". Communications in Statistics - Simulation and Computation. 23 (1): 157–164. doi:10.1080/03610919408813161.
  7. ^ Hornik, Kurt; Grün, Bettina (2014). "movMF: An R Package for Fitting Mixtures of Von Mises-Fisher Distributions". Journal of Statistical Software. 58 (10). doi:10.18637/jss.v058.i10. S2CID 13171102.
  8. ^ a b Pinzón, Carlos; Jung, Kangsoo (2023-03-03), Fast Python sampler for the von Mises Fisher distribution, retrieved 2023-03-30
  9. ^ De Cao, Nicola; Aziz, Wilker (13 Feb 2023). "The Power Spherical distribution". arXiv:2006.04437 [stat.ML].
  10. ^ Pakyuz-Charrier, Evren; Lindsay, Mark; Ogarko, Vitaliy; Giraud, Jeremie; Jessell, Mark (2018-04-06). "Monte Carlo simulation for uncertainty estimation on structural data in implicit 3-D geological modeling, a guide for disturbance distribution selection and parameterization". Solid Earth. 9 (2): 385–402. Bibcode:2018SolE....9..385P. doi:10.5194/se-9-385-2018. ISSN 1869-9510.
  11. ^ A., Wood, Andrew T. (1992). Simulation of the Von Mises Fisher distribution. Centre for Mathematics & its Applications, Australian National University. OCLC 221030477.{{cite book}}: CS1 maint: multiple names: authors list (link)
  12. ^ Gosmann, J; Eliasmith, C (2016). "Optimizing Semantic Pointer Representations for Symbol-Like Processing in Spiking Neural Networks". PLOS ONE. 11 (2): e0149928. Bibcode:2016PLoSO..1149928G. doi:10.1371/journal.pone.0149928. PMC 4762696. PMID 26900931.
  13. ^ a b Voelker, Aaron R.; Gosmann, Jan; Stewart, Terrence C. "Efficiently sampling vectors and coordinates from the n-sphere and n-ball" (PDF). Centre for Theoretical Neuroscience – Technical Report, 2017. Retrieved 22 April 2023.
  14. ^ Wang, Tongzhou; Isola, Phillip (2020). "Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere". International Conference on Machine Learning (ICML). arXiv:2005.10242.
  15. ^ Pal, Subhadip; Sengupta, Subhajit; Mitra, Riten; Banerjee, Arunava (2020). "Conjugate Priors and Posterior Inference for the Matrix Langevin Distribution on the Stiefel Manifold". Bayesian Analysis. 15 (3): 871–908. doi:10.1214/19-BA1176. ISSN 1936-0975.
  16. ^ Chikuse, Yasuko (1 May 2003). "Concentrated matrix Langevin distributions". Journal of Multivariate Analysis. 85 (2): 375–394. doi:10.1016/S0047-259X(02)00065-9. ISSN 0047-259X.
  17. ^ Jupp (1979). "Maximum likelihood estimators for the matrix von Mises-Fisher and Bingham distributions". The Annals of Statistics. 7 (3): 599–606. doi:10.1214/aos/1176344681.
  18. ^ Downs (1972). "Orientational statistics". Biometrika. 59 (3): 665–676. doi:10.1093/biomet/59.3.665.
  19. ^ Saw, John G (1978). "A family of distributions on the m-sphere and some hypothesis tests". Biometrika. 65 (`): 69–73. doi:10.2307/2335278. JSTOR 2335278.

Further reading

  • Dhillon, I., Sra, S. (2003) "Modeling Data using Directional Distributions". Tech. rep., University of Texas, Austin.
  • Banerjee, A., Dhillon, I. S., Ghosh, J., & Sra, S. (2005). "Clustering on the unit hypersphere using von Mises-Fisher distributions". Journal of Machine Learning Research, 6(Sep), 1345-1382.
  • Sra, S. (2011). "A short note on parameter approximation for von Mises-Fisher distributions: And a fast implementation of I_s(x)". Computational Statistics. 27: 177–190. CiteSeerX 10.1.1.186.1887. doi:10.1007/s00180-011-0232-x. S2CID 3654195.

Read other articles:

Ediakara~635 – 538.8 ± 0.2 Ma Had'n Arkean Proterozoikum Pha. Peta dunia pada Ediakara Akhir.EtimologiNama resmiFormalNama ratifikasi1990Informasi penggunaanBenda angkasaBumiPenggunaan regionalGlobal (ICS)Skala waktuICS Time ScaleRentang waktuSatuan kronologisPeriodeSatuan stratigrafikSistemJangka waktu resmiFormalAmbang batas jangka waktu Bawah Karbonat tutup yang berbeda di seluruh dunia. Permulaan dari pola khas perubahan sekuler dalam isotop karbon. Versi GSSPBagian Enorama Creek, Flin...

 

Kaisar Hongzhi Kaisar Hongzhi (Hanzi: 弘治, 30 Juli 1470-8 Juni 1505) adalah kaisar Dinasti Ming yang memerintah dari tahun 1487 sampai 1505. Terlahir dengan nama Zhu Youtang (朱祐樘), adalah anak dari Kaisar Chenghua dari selirnya yang bermarga Ji. Dia adalah kaisar masa pertengahan Ming yang bijaksana dan cinta damai, selama masa pemerintahannya Tiongkok stabil dan makmur sehingga masa ini disebut abad perak Hongzhi. Kehidupan awal Hongzhi lahir dan dibesarkan di bawah penganiayaan Sel...

 

Armería adalah sebuah munisipalitas di bagian selatan negara bagian Colima, Meksiko. Ciudad de Armería adalah kota dan kursi munisipalitas Armería. Munisipalitas tersebut dikabarkan memiliki 29,599 penduduk dalam sensus tahun 2015 dan memiliki wilayah seluas 341.60 km² (131.89 sq mi).[1] Kursi munisipalnya adalah Ciudad de Armería.[2] Referensi ^ Número de habitantes. INEGI. Diakses tanggal 2017-07-15.  ^ Armería. Enciclopedia de los Municipios de México (dal...

Cyclostomi Periode Lochkovium - Masa kini 419.2–0 jtyl PreЄ Є O S D C P T J K Pg N Cyclostomata Perik laut dari SwediaTaksonomiKerajaanAnimaliaFilumChordataSuperkelasCyclostomata Duméril, 1806 Kelas Myxini (Belut lintah) Hyperoartia (perik) lbs Cyclostomi, terkadang dirujuk sebagai Cyclostomata /sɪkloʊˈstɒmətə/, adalah sebuah kelompok hewan vertebrata yang mencakup ikan tak berahang yang masih hidup: lamprey (perik) dan ikan hag (belut lintah). Kedua kelompok memiliki mulut ta...

 

Current delegationMarco Rubio (R)Rick Scott (R) Florida was admitted to the Union on March 3, 1845, and elects its U.S. senators to class 1 and class 3. Florida's U.S. Senate seats were declared vacant in March 1861, due to its secession from the Union. They were filled again in July 1868. The state is currently represented by Republicans Marco Rubio (serving since 2011) and Rick Scott (serving since 2019). Duncan U. Fletcher was Florida's longest-serving senator (1909–1936). Florida is on...

 

Mungyeong 문경시KotaTranskripsi Korean • Hangul문경시 • Hanja聞慶市 • Revised RomanizationMungyeong-si • McCune-ReischauerMun'gyŏng-siPaviliun Gyogijeong di Saejae Bendera MungyeongBenderaLokasi di Korea SelatanNegara Korea SelatanRegionYeongnamPembagian administratif2 eup, 7 myeon, 5 dongLuas • Total911,73 km2 (35,202 sq mi)Populasi (31 Desember 2011) • Total77.304 • Ke...

خماسي المساري صمام خماسي طراز EF86 النوع فعّال المخترع برنارد تيليجين إعداد الدبوس خمسة أطراف (مصعد,مهبط,شبكة تحكم,شبكة حجب,شبكة كبت) تعديل مصدري - تعديل   خماسي المساري أوالصمام الخماسي (بالإنجليزية: Pentode)‏ نبيطة تطورت من الصمام الرباعي الذي تطور من الصمام الثلاثي وهو مكو�...

 

United States historic placeEumenean Hall, Davidson CollegeU.S. National Register of Historic Places Eumenean Hall, October 2011Show map of North CarolinaShow map of the United StatesLocationDavidson College campus, Davidson, North CarolinaCoordinates35°30′0″N 80°50′51″W / 35.50000°N 80.84750°W / 35.50000; -80.84750Area0.5 acres (0.20 ha)Built1849 (1849)ArchitectDinkins, Lewis; Alexander, DanielArchitectural styleGreek RevivalNRHP refere...

 

Maya archaeological site in Belize XunantunichEl Castillo at XunantunichLocation within MesoamericaShow map of MesoamericaLocation within BelizeShow map of BelizeLocationSan Jose Succotz, BelizeRegionCayo DistrictCoordinates17°05′21″N 89°08′29″W / 17.089059°N 89.141427°W / 17.089059; -89.141427HistoryPeriodsPreclassic to Postclassic occupationCulturesMayaSite notesArchaeologistsThomas Gann, Sir J. Eric S. Thompson, A. H. Anderson, Linton Satterhwaite, ...

Invasi Italia ke AlbaniaPasukan Italia di Albania, 7 April 1939Tanggal7 April 1939 – 12 April 1939 (5 Hari)LokasiAlbaniaHasil Kemenangan ItaliaPerubahanwilayah Italia menduduki AlbaniaPihak terlibat Kerajaan Italia Kerajaan AlbaniaTokoh dan pemimpin Alfredo Guzzoni Benito Mussolini Giovanni Messe Zog I Xhemal Aranitasi Abaz Kupi Mujo UlqinakuKekuatan 40,000 Tentara400 Pesawat 2 kapal Perang3 Kapal Penjelajah Berat9 Kapal Perusak14 Kapal Torpedo10 Kapal Tambahan25 Tank 15,000 Tentara 8,000 P...

 

Belkin TuneCast transmitter, for use with any device which has a 3.5mm headphone jack. Frequency range is 88.1 - 88.3 - 88.5 - 88.7 MHz Belkin TuneCastII FM Transmitter with a modified antenna connected to an iPod music player. A personal FM transmitter is a low-power FM radio transmitter that broadcasts a signal from a portable audio device (such as an MP3 player) to a standard FM radio. Most of these transmitters plug into the device's headphone jack and then broadcast the signal over an FM...

 

This article’s introduction, “General data”, “Emergencies”, “Cooperation with the army” and “Academic participation”, sections contains content that is written like an advertisement. Please help improve it by removing promotional content and inappropriate external links, and by adding encyclopedic content written from a neutral point of view. (April 2022) (Learn how and when to remove this message) Hospital in Haifa, IsraelBnai Zion Medical CenterGeographyLocationHaifa, ...

莎拉·阿什頓-西里洛2023年8月,阿什頓-西里洛穿著軍服出生 (1977-07-09) 1977年7月9日(46歲) 美國佛羅里達州国籍 美國别名莎拉·阿什頓(Sarah Ashton)莎拉·西里洛(Sarah Cirillo)金髮女郎(Blonde)职业記者、活動家、政治活動家和候選人、軍醫活跃时期2020年—雇主內華達州共和黨候選人(2020年)《Political.tips》(2020年—)《LGBTQ國度》(2022年3月—2022年10月)烏克蘭媒�...

 

D-Link CorporationD-Link Systems, Inc.JenisPublikKode emitenTWSE: 2332IndustriPeralatan jaringan komputerDidirikanMaret 1986; 38 tahun lalu (1986-03)(sebagai Datex Systems Inc.)1994 (sebagai D-Link Corporation)KantorpusatTaipei, TaiwanTokohkunciVictor Kuo (Chairman)ProdukRouter, DSL/Cable Gateways, Switch, Hotspot wi-fi, Storage dan IP camera keamananPendapatan NT$ 19,383 juta (2018)Karyawan~2,000 (2018)Situs webdlink.com D-Link Corporation adalah perusahaan produsen peralatan jaringan k...

 

Scottish rugby union team Rugby teamHawickFull nameHawick Rugby Football ClubUnionSRUFounded1885; 139 years ago (1885)LocationHawick, ScotlandGround(s)Mansfield Park (Capacity: 5,000)PresidentIan LandlesCoach(es)Head Coach: Graham Hogg Team Manager: Mick Gillan Director Of Rugby: Gary MuirCaptain(s)Shawn MuirLeague(s)Scottish Premiership2023–241st (Runners up) Team kit Official websitehawickrfc.co.uk Hawick Rugby Football Club is an semi-pro rugby union side, currently pla...

Polish sociocultural movement (c. 1820 - 1864) Part of a series on theCulture of Poland History Middle Ages Renaissance Baroque Enlightenment Romanticism Positivism Young Poland Interbellum World War II Polish People's Republic Modern-day People Poles Ethnic minorities Refugees Crime Education Health care Languages Languages Polish Yiddish German Lithuanian Ruthenian Romani (Baltic Romani North Central Romani Sinte Romani Vlax Romani) Silesian Kashubian Vilamovian Traditions Mythology Cuisine...

 

Shaman worship of mother goddesses in VietnamThis article contains too many pictures for its overall length. Relevant discussion may be found on the talk page. Please improve this article by removing indiscriminate collections of images or adjusting images that are sandwiching text in accordance with the Manual of Style on use of images. (Learn how and when to remove this message)The painting Chúng Dân Chi Mẫu portraying the four Mother Goddesses in the Four Palaces denomination, Đạo M...

 

Form of communication over the internet For other uses, see Messaging application. This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.Find sources: Instant messaging – news · newspapers · books · scholar · JSTOR (October 2016) (Learn how and when to remove this message) A classic example of instant messaging on a desktop com...

Nggalat (Rubus moluccanus), adalah spesies tumbuhan dari famili rosaceae. Tumbuhan ini sering dijumpai hidup di pegunungan pada ketinggian lebih dari 1.000 mdpl. Tumbuhan ini menghasilkan buah agregat berwarna merah yang dapat dimakan. Tumbuhan ini kadang dijumpai hidup bersama arbei gunung Nggalat Rubus moluccanus TaksonomiSuperkerajaanEukaryotaKerajaanPlantaeDivisiTracheophytaOrdoRosalesFamiliRosaceaeGenusRubusSpesiesRubus moluccanus Linnaeus, 1753 Tata namaSinonim taksonRubus wawrai Kuntze...

 

由於受到破坏,依據方針,本条目页已獲半保護。 請参閱保护方针及保护日志以获取更多信息。如果您不能修改此条目页,您可以请求修改、在讨论页提出修改提议、申请解除保护、登录或创建账号。 此條目可参照英語維基百科相應條目来扩充。 (2021年1月5日)若您熟悉来源语言和主题,请协助参考外语维基百科扩充条目。请勿直接提交机械翻译,也不要翻译不可靠、低品�...