Apache Ignite

Apache Ignite
Original author(s)GridGain Systems
Developer(s)Apache Software Foundation
Initial release24 March 2015; 9 years ago (2015-03-24)
Stable release
2.16.0 / 25 December 2023; 12 months ago (2023-12-25)[1]
Preview release
3.0.0 (alpha 5) / 15 June 2022; 2 years ago (2022-06-15)[1]
RepositoryIgnite Repository
Written inJava, C#, C++, SQL
Operating systemCross-platform
PlatformIA-32, x86-64, PowerPC, SPARC, Java platform, .NET Framework
TypeDatabase, computing platform
LicenseApache License 2.0
Websiteignite.apache.org

Apache Ignite is a distributed database management system for high-performance computing.

Apache Ignite's database uses RAM as the default storage and processing tier, thus, belonging to the class of in-memory computing platforms.[2] The disk tier is optional but, once enabled, will hold the full data set whereas the memory tier[3] will cache full or partial data set depending on its capacity.

Data in Ignite is stored in the form of key-value pairs. The database component distributes key-value pairs across the cluster in such a way that every node owns a portion of the overall data set. Data is rebalanced automatically whenever a node is added to or removed from the cluster.

Apache Ignite cluster can be deployed on-premise on a commodity hardware, in the cloud (e.g. Microsoft Azure, AWS, Google Compute Engine) or in a containerized and provisioning environments such as Kubernetes, Docker, Apache Mesos, VMware.[4][5]

History

Apache Ignite was developed by GridGain Systems, Inc. and made open source in 2014. GridGain continues to be the main contributor to the source code, and offers both a commercial version and professional services around Apache Ignite.

Once donated as open source, Ignite was accepted in the Apache Incubator program in October 2014.[6][7] The Ignite project graduated from the incubator program to become a top-level Apache project on September 18, 2015.[7]

In recent years, Apache Ignite has become one of the top 5 most active projects[8] by some metrics, including user base activity and repository size.

GridGain was founded in 2010 by Nikita Ivanov and Dmitriy Setrakyan in Pleasanton, California. A funding round of $2 to $3 million was disclosed in November, 2011.[9] By 2013 it was located in Foster City, California when it disclosed funding of $10 million.[10]

Clustering

Apache Ignite clustering component uses a shared nothing architecture. Server nodes are storage and computational units of the cluster that hold both data and indexes and process incoming requests along with computations. Server nodes are also known as data nodes.[11]

Client nodes are connection points from applications and services to the distributed database on a cluster of server nodes. Client nodes are usually embedded in the application code written in Java, C# or C++ that have special libraries developed. On top of its distributed foundation, Apache Ignite supports interfaces including JCache-compliant key-value APIs, ANSI-99 SQL with joins, ACID transactions, as well as MapReduce like computations. Ignite provides ODBC,[12] JDBC[13] and REST drivers as a way to work with the database from other programming languages or tools. The drivers use either client nodes or low-level socket connections internally in order to communicate to the cluster.

Partitioning and replication

Ignite database organizes data in the form of key-value pairs in distributed "caches" (the cache notion is used for historical reasons because initially, the database supported the memory tier). Generally, each cache represents one entity type such as an employee or organization.

Every cache is split into a fixed set of "partitions" that are evenly distributed among cluster nodes using the rendezvous hashing algorithm. There is always one primary and zero or more backup copies of a partition. The number of copies is configured with a replication factor parameter.[14] If the full replication mode is configured, then every cluster node will store a partition's copy. The partitions are rebalanced[15] automatically if a node is added to or removed from the cluster in order to achieve an even data distribution and spread the workload.

The key-value pairs are kept in the partitions. Apache Ignite maps a pair to a partition by taking the key's value and passing it to a special hash function.

Memory architecture

The memory architecture in Apache Ignite consists of two storage tiers and is called "durable memory". Internally, it uses paging for memory space management and data reference,[16] similar to the virtual memory of systems like Unix. However, one significant difference between the durable and virtual memory architectures is that the former always keeps the whole data set with indexes on disk (assuming that the disk tier is enabled), while the virtual memory uses the disk when it runs out of RAM, for swapping purposes only.

The first tier of the memory architecture, memory tier, keeps data and indexes in RAM out of Java heap in so-called "off-heap regions". The regions are preallocated and managed by the database on its own which prevents Java heap utilization for storage needs, as a result, helping to avoid long garbage collection pauses. The regions are split into pages of fixed size that store data, indexes, and system metadata.[17]

Apache Ignite is fully operational from the memory tier but it is always possible to use the second tier, disk tier, for the sake of durability. The database comes with its own native persistence and, plus, can use RDBMS, NoSQL or Hadoop databases as its disk tier.

Native persistence

Apache Ignite native persistence is a distributed and strongly consistent disk store that always holds a superset of data and indexes on disk. The memory tier [3] will only cache as much data as it can depending on its capacity. For example, if there are 1000 entries and the memory tier can fit only 300 of them, then all 1000 will be stored on disk and only 300 will be cached in RAM.

Persistence uses the write-ahead logging (WAL) technique for keeping immediate data modifications on disk.[18] In the background, the store runs the "checkpointing process" which purpose is to copy dirty pages from the memory tier to the partition files. A dirty page is a page that is modified in memory with the modification recorded in WAL but not written to a respective partition file. The checkpointing allows removing outdated WAL segments over the time and reduces cluster restart time replaying only that part of WAL that has not been applied to the partition files.[19]

Third-party persistence

The native persistence became available starting version 2.1.[20] Before that Apache Ignite supported only third-party databases as its disk tier.

Apache Ignite can be configured as the in-memory tier on top of RDBMS, NoSQL or Hadoop databases speeding up the latter.[21] However, there are some limitations in comparison to the native persistence. For instance, SQL queries will be executed only on the data that is in RAM, thus, requiring to preload all the data set from disk to memory beforehand.

Swap space

When using pure memory storage, it is possible for the data size to exceed the physical RAM size, leading to Out-Of-Memory Errors (OOMEs). To avoid this, the ideal approach would be to enable Ignite native persistence or use third-party persistence. However, if you do not want to use native or third-party persistence, you can enable swapping, in which case, Ignite in-memory data will be moved to the swap space located on disk. Note that Ignite does not provide its own implementation of swap space. Instead, it takes advantage of the swapping functionality provided by the operating system (OS). When swap space is enabled, Ignites stores data in memory mapped files (MMF) whose content will be swapped to disk by the OS depending on the current RAM consumption

Consistency

Apache Ignite is a strongly consistent platform that implements two-phase commit protocol.[22] The consistency guarantees are met for both memory and disk tiers. Transactions in Apache Ignite are ACID-compliant and can span multiple cluster nodes and caches. The database supports pessimistic and optimistic concurrency modes, deadlock-free transactions and deadlock detection techniques.

In the scenarios where transactional guarantees are optional, Apache Ignite allows executing queries in the atomic mode that provides better performance.

Distributed SQL

Apache Ignite can be accessed using SQL APIs exposed via JDBC and ODBC drivers, and native libraries developed for Java, C#, C++ programming languages. Both data manipulation and data definition languages' syntax complies with ANSI-99 specification.

Being a distributed database, Apache Ignite supports both distributed collocated and non-collocated joins.[23] When the data is collocated, joins are executed on the local data of cluster nodes avoiding data movement across the network. Non-collocated joins might move the data sets around the network in order to prepare a consistent result set.

Machine Learning

Apache Ignite provides machine learning training and inference functionality as well as data preprocessing and model quality estimation.[24] It natively supports classical training algorithms such as Linear Regression, Decision Trees, Random Forest, Gradient Boosting, SVM, K-Means and others. In addition to that, Apache Ignite has a deep integration with TensorFlow.[25] This integrations allows to train neural networks on a data stored in Apache Ignite in a single-node or distributed manner.

The key idea of Apache Ignite Machine Learning toolkit is an ability to perform distributed training and inference instantly without massive data transmissions. It's based on MapReduce approach, resilient to node failures and data rebalances, allows to avoid data transfers and so that speed up preprocessing and model training.[26]

References

  1. ^ a b "Downloads - Apache Ignite". ignite.apache.org. Retrieved 2024-04-09.
  2. ^ "Nikita Ivanov on Apache Ignite In-Memory Computing Platform". InfoQ. Retrieved 2017-10-11.
  3. ^ a b "Apache Ignite Native Persistence, a Brief Overview - DZone Big Data". dzone.com. Retrieved 2017-10-11.
  4. ^ "Deploying Apache Ignite in Kubernetes on Microsoft Azure - DZone Cloud". dzone.com. Retrieved 2017-10-11.
  5. ^ "Real-time in-memory OLTP and Analytics with Apache Ignite on AWS | Amazon Web Services". Amazon Web Services. 2016-05-14. Retrieved 2017-10-11.
  6. ^ "Nikita Ivanov on Apache Ignite In-Memory Computing Platform". InfoQ. Retrieved 2017-11-02.
  7. ^ a b "Ignite Status - Apache Incubator". incubator.apache.org. Retrieved 2017-11-02.
  8. ^ "Apache Ignite Momentum: Highlights from 2020-2021 : Apache Ignite". blogs.apache.org. 14 September 2021. Retrieved 2022-06-13.
  9. ^ "Form D - Notice of Exempt Offering of Securities". United States Securities and Exchange Commission. November 8, 2011. Retrieved February 16, 2022.
  10. ^ "Form D - Notice of Exempt Offering of Securities". United States Securities and Exchange Commission. May 7, 2013. Retrieved February 16, 2022.
  11. ^ "Clients and Servers". apacheignite.readme.io. Retrieved 2017-10-11.
  12. ^ "ODBC Driver". apacheignite.readme.io. Retrieved 2017-10-11.
  13. ^ "JDBC Driver". apacheignite.readme.io. Retrieved 2017-10-11.
  14. ^ "Primary & Backup Copies". apacheignite.readme.io. Retrieved 2017-10-11.
  15. ^ "Data Rebalancing". apacheignite.readme.io. Retrieved 2017-10-11.
  16. ^ "Apache Ignite 2.0: Redesigned Off-heap Memory, DDL and Machine Learning : Apache Ignite". blogs.apache.org. 5 May 2017. Retrieved 2017-10-11.
  17. ^ "Memory Architecture". apacheignite.readme.io. Retrieved 2017-10-11.
  18. ^ "Ignite Persistence". apacheignite.readme.io. Retrieved 2017-10-11.
  19. ^ "Ignite Persistence". apacheignite.readme.io. Retrieved 2017-10-11.
  20. ^ "Apache Ignite 2.1 - A Leap from In-Memory to Memory-Centric Architecture : Apache Ignite". blogs.apache.org. 27 July 2017. Retrieved 2017-10-11.
  21. ^ "Apache Ignite for Database Caching - DZone Database". dzone.com. Retrieved 2017-10-11.
  22. ^ "Distributed Thoughts". Retrieved 2017-10-11.
  23. ^ "Apache Ignite 1.7: Welcome Non-Collocated Distributed Joins! - DZone Database". dzone.com. Retrieved 2017-10-11.
  24. ^ "Machine Learning". apacheignite.readme.io. Retrieved 2018-12-27.
  25. ^ "TensorFlow: Apache Ignite Integration". github.com. Retrieved 2018-12-27.
  26. ^ "Partition Based Dataset". apacheignite.readme.io. Retrieved 2018-12-27.

Read other articles:

Hapagfly

Artikel ini perlu dikembangkan agar dapat memenuhi kriteria sebagai entri Wikipedia.Bantulah untuk mengembangkan artikel ini. Jika tidak dikembangkan, artikel ini akan dihapus. Hapagfly Boeing 737-800 Hapagfly Airbus A310 Hapagfly adalah nama maskapai penerbangan Jerman. Kode IATA HF dan kode ICAO HLF. lbsAnggota European Low Fares Airline Association (ELFAA) easyJet · Flybe · MyAir · Norwegian Air Shuttle · Ryanair · SkyEurope ·...

 

 

Lambang Nepal

Lambang negara Nepal. Lambang negara Nepal adalah lambang yang digunakan negara Nepal dan mulai digunakan sejak periode rekonsiliasi setelah Perang Saudara Nepal. Pada tanggal 30 Desember 2006, lambang negara baru ini diperkenalkan. Lambang negara ini menampilkan bendera Nepal, Puncak Everest, perbukitan hijau melambangkan kawasan pergunungan Nepal, dan warna kuning melambangkan kawasan terai yang subur. Tangan pria dan wanita berjabat tangan melambangkan kesetaraan jender, dan rangkaian bung...

 

 

Ramón Rodrigo de Freitas

Ramón Nazionalità  Brasile Altezza 186 cm Peso 74 kg Calcio Ruolo Centrocampista Termine carriera 2019 Carriera Giovanili ?-2005 América-MG Squadre di club1 2005 América-MG? (?)2006→  Grêmio30 (7)2006-2007 Grêmio23 (2)2008→  Portuguesa0 (0)2008→  Figueirense20 (2)2009 Coritiba2 (0)2009→  South China4 (0)2010 Coritiba9 (1)2011 Botafogo-SP0 (0)[1]2011 Boa Esporte17 (2)2012-2015 Goiás99 (12)[2]2017-2...

Shushicë, Elbasan

Municipal unit in Elbasan, AlbaniaShushicëMunicipal unitShushicëCoordinates: 41°6′N 20°9′E / 41.100°N 20.150°E / 41.100; 20.150Country AlbaniaCountyElbasanMunicipalityElbasanPopulation (2011) • Municipal unit8,731Time zoneUTC+1 (CET) • Summer (DST)UTC+2 (CEST) Shushicë is a village and a former municipality in the Elbasan County, central Albania. At the 2015 local government reform it became a subdivision of the municipality...

 

 

FHL5

تحتاج هذه المقالة إلى الاستشهاد بمصادر إضافية لتحسين وثوقيتها. فضلاً ساهم في تطوير هذه المقالة بإضافة استشهادات من مصادر موثوق بها. من الممكن التشكيك بالمعلومات غير المنسوبة إلى مصدر وإزالتها. (ديسمبر 2022) FHL5 التراكيب المتوفرة بنك بيانات البروتينOrtholog search: PDBe RCSB قائمة رموز م...

 

 

Lomatia

Genus of flowering plants This article is about the plant genus. For the genus of insects, see Bombyliidae and Lomatia belzebul. Lomatiinae redirects here. For the insect subfamily, see Lomatiinae (fly). Lomatia L. silaifolia Scientific classification Kingdom: Plantae Clade: Tracheophytes Clade: Angiosperms Clade: Eudicots Order: Proteales Family: Proteaceae Subfamily: Grevilleoideae Tribe: Embothrieae Subtribe: Lomatiinae Genus: LomatiaR.Br. Species See text Synonyms Tricondylus Salisb. ex K...

University of Extremadura

This article includes a list of references, related reading, or external links, but its sources remain unclear because it lacks inline citations. Please help improve this article by introducing more precise citations. (July 2015) (Learn how and when to remove this message) University of ExtremaduraUniversidad de ExtremaduraLatin: Universitas ExtrematurensisOther nameUEXTypePublic UniversityEstablished1973RectorPedro M. Fernández SalgueroAdministrative staff860Undergraduates22.510Postgraduate...

 

 

Lakehead Junior Hockey League

Lakehead Junior Hockey LeagueFederation(s)Hockey Northwestern OntarioPresidentRon WhiteheadVice PresidentJosh GribbenFormer name(s)Thunder Bay Junior B/Juvenile/AAA Hockey LeagueFounded1993No. of teams5Recent ChampionsCurrent River Storm (2024)Most successful clubThunder Bay Northern Hawks 12Websitehttps://www.theljhl.com/ The Lakehead Junior Hockey League is a Canadian Junior ice hockey league in Northwestern Ontario, sanctioned by Hockey Northwestern Ontario[1] and Hockey Canada. An...

 

 

Palazzo della Questura (Firenze)

Palazzo della QuesturaPalazzo della Questura, prospetto su via ZaraLocalizzazioneStato Italia RegioneToscana LocalitàFirenze Indirizzovia Zara 2 Coordinate43°46′53.12″N 11°15′33.57″E / 43.781422°N 11.259325°E43.781422; 11.259325Coordinate: 43°46′53.12″N 11°15′33.57″E / 43.781422°N 11.259325°E43.781422; 11.259325 Informazioni generaliCondizioniIn uso Modifica dati su Wikidata · Manuale Il Palazzo della Questura di Firenze si t...

François Neveux

Pour les articles homonymes, voir Neveux. Cet article est une ébauche concernant un historien français. Vous pouvez partager vos connaissances en l’améliorant (comment ?) selon les recommandations des projets correspondants. François NeveuxFrançois Neveux et Claire Ruelle, conférence du 20 avril 2013 au château de Caen portant sur Bertrand Du Guesclin.FonctionPrésidentFédération des sociétés historiques et archéologiques de Normandie (d)BiographieNaissance 26 août 1944 (...

 

 

LGBT rights in Norway

LGBT rights in NorwayLocation of Norway (dark green)in Europe (dark grey)  –  [Legend]StatusLegal since 1972Gender identityTransgender people allowed to change legal sex based on self-determinationMilitaryLGBT people allowed to serve openly.Discrimination protectionsSexual orientation, gender identity/expression, intersex status protections (see below)Family rightsRecognition of relationshipsSame-sex marriage since 2009AdoptionFull adoption rights since 200...

 

 

Еврейские беженцы от нацизма

Часть серии статей о ХолокостеИдеология и политика Расовая гигиена Расовый антисемитизм Нацистская расовая политика Нюрнбергские расовые законы Шоа Лагеря смерти Белжец Дахау Майданек Малый Тростенец Маутхаузен Освенцим Собибор Треблинка Хелмно Юнгфернхоф Ясеновац...

Pierre Moscovici

Pierre Moscovici Komisioner Eropa untuk Ekonomi dan Urusan Keuangan, Perpajakan dan PabeanDiangkatMulai menjabat1 November 2014PresidenJean-Claude JunckerMenggantikanJyrki Katainen (Ekonomi dan Urusan Keuangan dan Euro)Algirdas Šemeta (Perpajakan dan Persatuan Pelanggan, Audit dan Anti-Penipuan)Menteri KeuanganMasa jabatan16 Mei 2012 – 2 April 2014Perdana MenteriJean-Marc AyraultPendahuluFrançois BaroinPenggantiMichel Sapin Informasi pribadiLahir16 September 1957 (umur 66)Pa...

 

 

Daftar keuskupan di Brasil

Episcopal regions in Brazil (CNBB). Total: 19 regions.Pembagian provinsi gerejawi di Brasil.Daftar keuskupan di Brasil adalah sebuah daftar yang memuat dan menjabarkan pembagian terhadap wilayah administratif Gereja Katolik Roma yang dipimpin oleh seorang uskup ataupun ordinaris di Brasil. Konferensi para uskup Brasil bergabung dalam Konferensi Waligereja Brasil. Per Juni 2020, terdapat 278 buah yurisdiksi, di mana 272 merupakan yurisdiksi Ritus Latin dan 6 lainnya merupakan yurisdiksi Gereja...

 

 

Charles A. Talcott

American politician (1857–1920)This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.Find sources: Charles A. Talcott – news · newspapers · books · scholar · JSTOR (March 2024) (Learn how and when to remove this message) Charles A. TalcottMember of the U.S. House of Representativesfrom New YorkIn officeMarch 4, 19...

Trichophyton

Trichophyton Trichophyton rubrum Klasifikasi ilmiah Kerajaan: Fungi Divisi: Deuteromycota Kelas: Eurotiomycetes Ordo: Onygenales Famili: Arthrodermataceae Genus: Trichophyton Trichophyton adalah suatu dermatofita yang hidup di tanah, binatang atau manusia. Berdasarkan tempat tinggal terdiri atas anthropophilic, zoophilic, dan geophilic. Trichophyton concentricum adalah endemic pulau Pacifik, Bagian tenggara Asia, dan Amerika Pusat. Trichophyton adalah satu penyebab infeksi pada rambut, kulit...

 

 

Peter Naur

Danish computer science pioneer Peter NaurNaur in 2008Born(1928-10-25)25 October 1928Frederiksberg, DenmarkDied3 January 2016(2016-01-03) (aged 87)Herlev, DenmarkNationalityDanishKnown forALGOLBackus–Naur formSpouseChristiane FloydAwardsComputer Pioneer Award (1986)Turing Award (2005)Scientific careerFieldsComputer science, informaticsInstitutionsRegnecentralenNiels Bohr InstituteTechnical University of DenmarkUniversity of Copenhagen Peter Naur (25 October 1928 – 3 January 2016...

 

 

Nokia 6103

Nokia 6103 adalah produk telepon genggam yang dirilis oleh perusahaan Nokia. Telepon genggam ini memiliki dimensi 85 x 45 x 24 mm dengan berat 97 gram. Fitur Kamera digital VGA SMS MMS Obrolan Pesan suara Polifonik Permainan Radio FM Internet GPRS Internet Inframerah Pranala luar informasi di GSMarena lbsNokiaAnak usaha Nokia Bell Labs Nokia Networks Nokia Technologies Unit lain NGP Capital Nuage Networks Akuisisi Alcatel-Lucent Dopplr earthmine Enpocket F5 Networks Intellisync Ipsilon N...

Washington (state)

U.S. state Washington state redirects here. For the proposed statehood of Washington, D.C., see District of Columbia statehood movement. For other uses, see Washington (disambiguation) and Washington State (disambiguation). Not to be confused with Wa State. State in the United StatesWashingtonState FlagSealNickname: The Evergreen State (unofficial)[1]Motto(s): Alki (Chinook jargon for 'By and By')Anthem: Washington, My Home Map of the United States with Washington hi...

 

 

BK Derby

BK DerbyGrundad15 april 1912Hemort Linköping, SverigeHemmaarenaKarlbergsplan, Division 4 västraOrdförandeJonas LundbergTränareRiki Skopljak Hemmaställ Bortaställ MeriterAllsvenskasäsonger1 (senast 1977)Placering iallsvenskansmaratontabell58:aSäsonger iSveriges nästhögsta division30 (senast 1979)ÖvrigtSupportrarGreen FoxesWebbplatsDerby.se BK Derby är en idrottsförening från Linköping bildad 15 april 1912.[1] Föreningen har bland annat bedrivit fotboll, bandy, handboll, ishocke...