Probabilistic programming

Probabilistic programming (PP) is a programming paradigm in which probabilistic models are specified and inference for these models is performed automatically.[1] It represents an attempt to unify probabilistic modeling and traditional general purpose programming in order to make the former easier and more widely applicable.[2][3] It can be used to create systems that help make decisions in the face of uncertainty.

Programming languages used for probabilistic programming are referred to as "probabilistic programming languages" (PPLs).

Applications

Probabilistic reasoning has been used for a wide variety of tasks such as predicting stock prices, recommending movies, diagnosing computers, detecting cyber intrusions and image detection.[4] However, until recently (partially due to limited computing power), probabilistic programming was limited in scope, and most inference algorithms had to be written manually for each task.

Nevertheless, in 2015, a 50-line probabilistic computer vision program was used to generate 3D models of human faces based on 2D images of those faces. The program used inverse graphics as the basis of its inference method, and was built using the Picture package in Julia.[4] This made possible "in 50 lines of code what used to take thousands".[5][6]

The Gen probabilistic programming library (also written in Julia) has been applied to vision and robotics tasks.[7]

More recently, the probabilistic programming system Turing.jl has been applied in various pharmaceutical[8] and economics applications.[9]

Probabilistic programming in Julia has also been combined with differentiable programming by combining the Julia package Zygote.jl with Turing.jl. [10]

Probabilistic programming languages are also commonly used in Bayesian cognitive science to develop and evaluate models of cognition. [11]

Probabilistic programming languages

PPLs often extend from a basic language. For instance, Turing.jl[12] is based on Julia, Infer.NET is based on .NET Framework,[13] while PRISM extends from Prolog.[14] However, some PPLs, such as WinBUGS, offer a self-contained language that maps closely to the mathematical representation of the statistical models, with no obvious origin in another programming language.[15][16]

The language for WinBUGS was implemented to perform Bayesian computation using Gibbs Sampling and related algorithms. Although implemented in a relatively unknown programming language (Component Pascal), this language permits Bayesian inference for a wide variety of statistical models using a flexible computational approach. The same BUGS language may be used to specify Bayesian models for inference via different computational choices ("samplers") and conventions or defaults, using a standalone program WinBUGS (or related R packages, rbugs and r2winbugs) and JAGS (Just Another Gibbs Sampler, another standalone program with related R packages including rjags, R2jags, and runjags). More recently, other languages to support Bayesian model specification and inference allow different or more efficient choices for the underlying Bayesian computation, and are accessible from the R data analysis and programming environment, e.g.: Stan, NIMBLE and NUTS. The influence of the BUGS language is evident in these later languages, which even use the same syntax for some aspects of model specification.

Several PPLs are in active development, including some in beta test. Two popular tools are Stan and PyMC.[17]

Relational

A probabilistic relational programming language (PRPL) is a PPL specially designed to describe and infer with probabilistic relational models (PRMs).

A PRM is usually developed with a set of algorithms for reducing, inference about and discovery of concerned distributions, which are embedded into the corresponding PRPL.

Probabilistic logic programming

Main article: Probabilistic logic programming

Probabilistic logic programming is a programming paradigm that extends logic programming with probabilities.

Most approaches to probabilistic logic programming are based on the distribution semantics, which splits a program into a set of probabilistic facts and a logic program. It defines a probability distribution on interpretations of the Herbrand universe of the program.[18]

List of probabilistic programming languages

This list summarises the variety of PPLs that are currently available, and clarifies their origins.

Name Extends from Host language
Analytica[19] C++
bayesloop[20][21] Python Python
Bean Machine[22] PyTorch Python
Venture[23] Scheme C++
BayesDB[24] SQLite, Python
PRISM[14] B-Prolog
Infer.NET[13] .NET Framework .NET Framework
diff-SAT[25] Answer set programming, SAT (DIMACS CNF)
PSQL[26] SQL
BUGS[15] Component Pascal
Dyna[27] Prolog
Figaro[28] Scala Scala
ProbLog[29] Prolog Python
ProBT[30] C++, Python
Stan[16] BUGS C++
Hakaru[31] Haskell Haskell
BAli-Phy (software)[32] Haskell C++
ProbCog[33] Java, Python
PyMC[34] Python Python
Rainier[35][36] Scala Scala
greta[37] TensorFlow R
pomegranate[38] Python Python
Lea[39] Python Python
WebPPL[40] JavaScript JavaScript
Picture[4] Julia Julia
Turing.jl[12] Julia Julia
Gen[41] Julia Julia
Edward[42] TensorFlow Python
TensorFlow Probability[43] TensorFlow Python
Edward2[44] TensorFlow Probability Python
Pyro[45] PyTorch Python
NumPyro[46] JAX Python
Birch[47] C++
PSI[48] D
Blang[49]
MultiVerse[50] Python Python
Anglican[51] Clojure Clojure

Difficulty

  • Reasoning about variables as probability distributions causes difficulties for novice programmers, but these difficulties can be addressed through use of Bayesian network visualizations and graphs of variable distributions embedded within the source code editor.[52]
  • As many PPLs rely on the specification of priors on the variables of interest, specifying informed priors is often difficult for novices. In some cases, libraries such as PyMC provide automated methods to find the parameterization of informed priors.[53]

See also

Notes

  1. ^ "Probabilistic programming does in 50 lines of code what used to take thousands". phys.org. April 13, 2015. Retrieved April 13, 2015.
  2. ^ "Probabilistic Programming". probabilistic-programming.org. Archived from the original on January 10, 2016. Retrieved December 24, 2013.
  3. ^ Pfeffer, Avrom (2014), Practical Probabilistic Programming, Manning Publications. p.28. ISBN 978-1 6172-9233-0
  4. ^ a b c "Short probabilistic programming machine-learning code replaces complex programs for computer-vision tasks". KurzweilAI. April 13, 2015. Retrieved November 27, 2017.
  5. ^ Hardesty, Larry (April 13, 2015). "Graphics in reverse".
  6. ^ "MIT shows off machine-learning script to make CREEPY HEADS". The Register.
  7. ^ "MIT's Gen programming system flattens the learning curve for AI projects". VentureBeat. June 27, 2019. Retrieved June 27, 2019.
  8. ^ Semenova, Elizaveta; Williams, Dominic P.; Afzal, Avid M.; Lazic, Stanley E. (November 1, 2020). "A Bayesian neural network for toxicity prediction". Computational Toxicology. 16: 100133. doi:10.1016/j.comtox.2020.100133. ISSN 2468-1113. S2CID 225362130.
  9. ^ Williams, Dominic P.; Lazic, Stanley E.; Foster, Alison J.; Semenova, Elizaveta; Morgan, Paul (2020), "Predicting Drug-Induced Liver Injury with Bayesian Machine Learning", Chemical Research in Toxicology, 33 (1): 239–248, doi:10.1021/acs.chemrestox.9b00264, PMID 31535850, S2CID 202689667
  10. ^ Innes, Mike; Edelman, Alan; Fischer, Keno; Rackauckas, Chris; Saba, Elliot; Viral B Shah; Tebbutt, Will (2019). "∂P: A Differentiable Programming System to Bridge Machine Learning and Scientific Computing". arXiv:1907.07587 [cs.PL].
  11. ^ Goodman, Noah D; Tenenbaum, Joshua B; Buchsbaum, Daphna; Hartshorne, Joshua; Hawkins, Robert; O'Donnell, Timothy J; Tessler, Michael Henry. "Probabilistic Models of Cognition". Probabilistic Models of Cognition - 2nd Edition. Retrieved May 27, 2023.
  12. ^ a b "The Turing language for probabilistic programming". GitHub. December 28, 2021.
  13. ^ a b "Infer.NET". microsoft.com. Microsoft.
  14. ^ a b "PRISM: PRogramming In Statistical Modeling". rjida.meijo-u.ac.jp. Archived from the original on March 1, 2015. Retrieved July 8, 2015.
  15. ^ a b "The BUGS Project - MRC Biostatistics Unit". cam.ac.uk. Archived from the original on March 14, 2014. Retrieved January 12, 2011.
  16. ^ a b "Stan". mc-stan.org. Archived from the original on September 3, 2012.
  17. ^ "The Algorithms Behind Probabilistic Programming". Retrieved March 10, 2017.
  18. ^ De Raedt, Luc; Kimmig, Angelika (July 1, 2015). "Probabilistic (logic) programming concepts". Machine Learning. 100 (1): 5–47. doi:10.1007/s10994-015-5494-z. ISSN 1573-0565.
  19. ^ "Analytica-- A Probabilistic Modeling Language". lumina.com.
  20. ^ "bayesloop - Probabilistic programming framework". bayesloop.com.
  21. ^ "GitHub -- bayesloop". GitHub. December 7, 2021.
  22. ^ "Bean Machine - A universal probabilistic programming language to enable fast and accurate Bayesian analysis". beanmachine.org.
  23. ^ "Venture -- a general-purpose probabilistic programming platform". mit.edu. Archived from the original on January 25, 2016. Retrieved September 20, 2014.
  24. ^ "BayesDB on SQLite. A Bayesian database table for querying the probable implications of data as easily as SQL databases query the data itself". GitHub. December 26, 2021.
  25. ^ "diff-SAT (probabilistic SAT/ASP)". GitHub. October 8, 2021.
  26. ^ Dey, Debabrata; Sarkar, Sumit (1998). "PSQL: A query language for probabilistic relational data". Data & Knowledge Engineering. 28: 107–120. doi:10.1016/S0169-023X(98)00015-9.
  27. ^ "Dyna". www.dyna.org. Archived from the original on January 17, 2016. Retrieved January 12, 2011.
  28. ^ "Charles River Analytics - Probabilistic Modeling Services". cra.com. February 9, 2017.
  29. ^ "ProbLog: Probabilistic Programming". dtai.cs.kuleuven.be.
  30. ^ ProbaYes. "ProbaYes - Ensemble, nous valorisations vos données". probayes.com. Archived from the original on March 5, 2016. Retrieved November 26, 2013.
  31. ^ "Hakaru Home Page". hakaru-dev.github.io/.
  32. ^ "BAli-Phy Home Page". bali-phy.org.
  33. ^ "ProbCog". GitHub.
  34. ^ PyMC devs. "PyMC". pymc-devs.github.io.
  35. ^ stripe/rainier, Stripe, August 19, 2020, retrieved August 26, 2020
  36. ^ "Rainier · Bayesian inference for Scala". samplerainier.com. Retrieved August 26, 2020.
  37. ^ "greta: simple and scalable statistical modelling in R". GitHub. Retrieved October 2, 2018.
  38. ^ "Home — pomegranate 0.10.0 documentation". pomegranate.readthedocs.io. Retrieved October 2, 2018.
  39. ^ "Lea Home Page". bitbucket.org.
  40. ^ "WebPPL Home Page". github.com/probmods/webppl.
  41. ^ "Gen: A General Purpose Probabilistic Programming Language with Programmable Inference". Retrieved June 11, 2024.
  42. ^ "Edward – Home". edwardlib.org. Retrieved January 17, 2017.
  43. ^ TensorFlow (April 11, 2018). "Introducing TensorFlow Probability". TensorFlow. Retrieved October 2, 2018.
  44. ^ "'Edward2' TensorFlow Probability module". GitHub. Retrieved June 11, 2024.
  45. ^ "Pyro". pyro.ai. Retrieved February 9, 2018.
  46. ^ "NumPyro". pyro.ai. Retrieved July 23, 2021.
  47. ^ "Probabilistic Programming in Birch". birch-lang.org. Retrieved April 20, 2018.
  48. ^ "PSI Solver - Exact inference for probabilistic programs". psisolver.org. Retrieved August 18, 2019.
  49. ^ "Home". www.stat.ubc.ca.
  50. ^ Perov, Yura; Graham, Logan; Gourgoulias, Kostis; Richens, Jonathan G.; Lee, Ciarán M.; Baker, Adam; Johri, Saurabh (January 28, 2020), MultiVerse: Causal Reasoning using Importance Sampling in Probabilistic Programming, arXiv:1910.08091
  51. ^ "The Anglican Probabilistic Programming System". probprog.github.io. Retrieved October 7, 2024.
  52. ^ Gorinova, Maria I.; Sarkar, Advait; Blackwell, Alan F.; Syme, Don (January 1, 2016). "A Live, Multiple-Representation Probabilistic Programming Environment for Novices". Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. CHI '16. New York, NY, USA: ACM. pp. 2533–2537. doi:10.1145/2858036.2858221. ISBN 9781450333627. S2CID 3201542.
  53. ^ "pymc.find_constrained_prior — PyMC dev documentation". www.pymc.io. Retrieved October 23, 2024.

Read other articles:

A kara

A karaAksara BaliHuruf LatinAIASTAFonem[a], [ə][1]UnicodeU+1B05 , U+Warga aksarakanthya A atau A kara adalah salah satu aksara swara (huruf vokal) dalam sistem penulisan aksara Bali. Aksara ini melambangkan bunyi /a/, sama halnya seperti aksara अ (A) dalam aksara Dewanagari, huruf A dalam alfabet Latin, atau huruf alfa (α) dalam alfabet Yunani. Bentuk Huruf A dalam aksara Brahmi telah menurunkan aksara Grantha, Pallawa dan bentuknya telah mengalami perubahan. A kara dalam aksara ...

 

 

Jesper Grønkjær

Jesper Grønkjær Informasi pribadiTanggal lahir 12 Agustus 1977 (umur 46)Tempat lahir Nuuk, GreenlandTinggi 1,87 m (6 ft 1+1⁄2 in)Posisi bermain Winger / Second penyerangKarier junior Thisted FCKarier senior*Tahun Tim Tampil (Gol)1995–1998 AaB 86 (10)1998–2000 Ajax 55 (12)2000–2004 Chelsea 88 (7)2004–2005 Birmingham City 16 (0)2005 Atlético Madrid 16 (0)2005–2006 VfB Stuttgart 25 (0)2006–2011 F.C. Copenhagen 114 (16)Total 400 (45)Tim nasional1993 Denma...

 

 

Velimir Valenta

Yugoslav-Croatian rower Velimir ValentaValenta (left) at the 1952 OlympicsPersonal informationBorn21 April 1929Klis, Kingdom of SCSDied27 November 2004 (aged 75)Mendrisio, SwitzerlandSportSportRowingClubHVK Gusar, Split Medal record Representing  Yugoslavia Olympic Games 1952 Helsinki Coxless four Velimir Valenta (21 April 1929 – 27 November 2004) was a Yugoslav/Croatian rower who won a gold medal in the coxless four event at the 1952 Summer Olympics.[1] References ^ Velimir Va...

Modèle:Infobox Observatoire

 Documentation[voir] [modifier] [historique] [purger] Ce modèle utilise le module Infobox/Observatoire, un script écrit dans le langage de programmation Lua. Toute expérimentation devrait être conduite d'abord via une sous-page bac à sable. Voir le projet Scribunto si vous voulez en savoir plus. Ce modèle utilise les données de Wikidata (aide). Un paramètre laissé vide dans le wikicode fera appel à l'élément wikidata correspondant de l'article...

 

 

Pasquale Fornara

Pasquale Fornara Nazionalità  Italia Ciclismo Specialità Strada Termine carriera 1961 - ciclista Carriera Squadre di club 1946-1947S.C. Crennese1948Individuale1949-1951 Legnano1951-1953 Cilo1952 Bianchi1952-1954 Bottecchia1955 Bottecchia1955Leo-Chlorodont1956 Saint-Raphaël1956 Allegro1956-1957 Arbos1956-1958 Cilo1958 Ignis1959-1960 Emi1960 Mondia1961 Vov Nazionale 1953-1956 Italia Carriera da allenatore 1965 Cynar ...

 

 

The Helix, Dublin

This article includes a list of general references, but it lacks sufficient corresponding inline citations. Please help to improve this article by introducing more precise citations. (April 2022) (Learn how and when to remove this message) The HelixThe Helix in August 2007Full nameThe Helix Centre for the Performing ArtsFormer namesNorth Dublin Arts CentreAddressDCU Main Campus, Collins Avenue ExtensionDublin D9D09 FW22IrelandLocationGlasnevinCoordinates53°23′11″N 6°15′34″W ...

Krešimir Zubak

Bosnian Croat politician Krešimir ZubakZubak in 19971st Croat Member of the Presidency of Bosnia and HerzegovinaIn office5 October 1996 – 15 November 1998Preceded byStjepan Kljuić Ivo Komšić (as members of the Presidency of the Republic of Bosnia and Herzegovina)Succeeded byAnte Jelavić1st President of the Federation of Bosnia and HerzegovinaIn office31 May 1994 – 18 March 1997Prime MinisterHaris Silajdžić Izudin KapetanovićVice PresidentEjup GanićPreceded byOff...

 

 

Matsuyama

For other uses, see Matsuyama (disambiguation). Core city in Shikoku, JapanMatsuyama 松山市Core cityFrom top left:Dōgo Onsen Honkan, Stone monument of Shiki Masaoka, Matsuyama Castle, Botchan train, The gate of Ishite-ji, Iyotetsu Matsuyama-shi Station, Gintengai Street FlagEmblemLocation of Matsuyama in Ehime PrefectureMatsuyamaLocation in JapanCoordinates: 33°50′N 132°46′E / 33.833°N 132.767°E / 33.833; 132.767CountryJapanRegionShikokuPrefectureEhimeGover...

 

 

Quasi famosi

Quasi famosiPatrick Fugit e Kate Hudson in una scena del filmTitolo originaleAlmost Famous Paese di produzioneStati Uniti d'America Anno2000 Durata122 min (versione originale)161 min (versione Untitled) Generecommedia, drammatico, musicale RegiaCameron Crowe SceneggiaturaCameron Crowe ProduttoreCameron Crowe, Ian Bryce Casa di produzioneColumbia Pictures, DreamWorks Pictures, Vynil Films FotografiaJohn Toll MontaggioJoe Hutshing, Saar Klein Effetti specialiJohn Frazier, Ed Jones Music...

Guardiani della Galassia (film)

Guardiani della GalassiaI Guardiani della Galassia in una scena del filmTitolo originaleGuardians of the Galaxy Lingua originaleinglese Paese di produzioneStati Uniti d'America Anno2014 Durata121 min Rapporto2,35:1 Genereazione, fantascienza, avventura, commedia RegiaJames Gunn Soggettopersonaggi creati da Dan Abnett e Andy Lanningstoria di James Gunn e Nicole Perlman SceneggiaturaJames Gunn, Nicole Perlman ProduttoreKevin Feige Produttore esecutivoLouis D'Esposito, Victoria Alons...

 

 

内华达州

内華達州 美國联邦州State of Nevada 州旗州徽綽號：產銀之州、起戰之州地图中高亮部分为内華達州坐标：35°N－42°N, 114°W－120°W国家 美國建州前內華達领地加入聯邦1864年10月31日（第36个加入联邦）首府卡森城最大城市拉斯维加斯政府 • 州长（英语：List of Governors of {{{Name}}}]]） • 副州长（英语：List of lieutenant governors of {{{Name}}}]]）喬·隆巴爾多（R斯塔...

 

 

Simona Molinari

Simona MolinariSimona Molinari a Lanciano nel 2012 Nazionalità Italia GenereElectro swingJazzSwingBossa nova[1]PopMusical Periodo di attività musicale2005 – in attività Strumentovoce, pianoforte EtichettaIsola degli Artisti/Universal (2009-2010)Warner Music (2010-2015)Isola degli Artisti (2018)BMG Rights Management (2021-) Album pubblicati6 Studio6 Sito ufficiale Modifica dati su Wikidata · Manuale Simona Molinari (Napoli, 23 febbraio 1983) è una ca...

アプリケーションソフトウェア

「アプリケーション」はこの項目へ転送されています。英語の意味については「wikt:応用」、「wikt:application」をご覧ください。 この記事には複数の問題があります。改善やノートページでの議論にご協力ください。 出典がまったく示されていないか不十分です。内容に関する文献や情報源が必要です。（2018年4月） 古い情報を更新する必要があります。（2021年3月）出...

 

 

Agama kepala merah (reptil)

Agama kepala merah Agama agama Status konservasiRisiko rendahIUCN172799 TaksonomiKerajaanAnimaliaFilumChordataKelasReptiliaOrdoSquamataFamiliAgamidaeGenusAgamaSpesiesAgama agama (Linnaeus, 1758) Tata namaProtonimLacerta agama lbs Agama kepala merah, agama pelangi, atau agama batu berkepala merah (Agama agama) adalah spesies kadal agama dari suku Agamidae yang tersebar di daerah Afrika bagian tengah hingga selatan wilayah gurun Sahara.[1][2][3] Morfologi Panjang tubuh a...

 

 

Sandra Bullock

Sandra Bullock nel 2013 Oscar alla miglior attrice 2010 Sandra Annette Bullock (Arlington, 26 luglio 1964) è un'attrice e produttrice cinematografica statunitense. Secondo la rivista Forbes è stata l'attrice più pagata del mondo nel 2010, con un guadagno di circa 56 milioni di dollari,[1][2][3] e nel 2014 con un guadagno di 51 milioni.[4] Nel 2012 è stata inserita nel libro del Guinness dei primati come l'attrice più pagata del mondo tra il 2010 e il 2011&...

X86

Family of instruction set architectures This article is about the Intel microprocessor architecture in general. For the 32-bit generation of this architecture that is also referred to as x86, see IA-32. x86DesignerIntel, AMDBits16-bit, 32-bit and 64-bitIntroduced1978 (16-bit), 1985 (32-bit), 2003 (64-bit)DesignCISCTypeRegister–memoryEncodingVariable (1 to 15 bytes)BranchingCondition codeEndiannessLittlePage size8086–i286: Nonei386, i486: 4 KB pagesP5 Pentium: added 4 MB pages(Le...

 

 

Reflection (專輯)

本條目存在以下問題，請協助改善本條目或在討論頁針對議題發表看法。 此條目可能包含原创研究。 (2019年7月27日)请协助補充参考资料、添加相关内联标签和删除原创研究内容以改善这篇条目。详细情况请参见讨论页。 此條目没有列出任何参考或来源。 (2019年7月27日)維基百科所有的內容都應該可供查證。请协助補充可靠来源以改善这篇条目。无法查证的內容可能會因為異�...

 

 

Kolohousenka

KolohousenkaKolohousenka KH-50DescrizioneTipocarro armato leggero Equipaggio2 (comandante/mitragliere e pilota) ProgettistaJoseph Vollmer CostruttoreŠkoda, ČKD, Tatra Data impostazione1923 Dimensioni e pesoLunghezza4,50 m Larghezza2,39 m Altezza2,53 m Peso7,5 t Propulsione e tecnicaMotoreHanomag WD 50PS PotenzaKH-50: 50 hp KH-60: 60 hp KH-70: 70 hp Trazionecingolata Sospensionia balestra PrestazioniVelocità su strada15 km/h su cingoli, 35 km/h su ruote Autonomia300 km Armamento e...

تي-84

يفتقر محتوى هذه المقالة إلى الاستشهاد بمصادر. فضلاً، ساهم في تطوير هذه المقالة من خلال إضافة مصادر موثوق بها. أي معلومات غير موثقة يمكن التشكيك بها وإزالتها. (ديسمبر 2018) تي-84 النوع دبابة قتال رئيسية بلد الأصل  أوكرانيا تاريخ الاستخدام فترة الاستخدام 1999–حتى الان المستخدم�...

 

 

Gil Scott-Heron

Gil Scott-HeronGil Scott-Heron nel 1986 Nazionalità Stati Uniti GenereSoulJazz-funkProto-rapFolk Periodo di attività musicale1969 – 2011 Strumentovoce, piano elettrico, chitarra Sito ufficiale Modifica dati su Wikidata · Manuale Gil Scott-Heron (Chicago, 1º aprile 1949 – New York, 27 maggio 2011) è stato un poeta, musicista e attivista statunitense, conosciuto principalmente per i suoi lavori della fine degli anni sessanta e inizi degli anni settanta come...