Draft:Semantic Search
Submission declined on 13 July 2026 by ChrysGalley (talk).
Where to get help
How to improve a draft
You can also browse Wikipedia:Featured articles and Wikipedia:Good articles to find examples of Wikipedia's best writing on topics similar to your proposed article. Improving your odds of a speedy review To improve your odds of a faster review, tag your draft with relevant WikiProject tags using the button below. This will let reviewers know a new draft has been submitted in their area of interest. For instance, if you wrote about a female astronomer, you would want to add the Biography, Astronomy, and Women scientists tags. Editor resources
|  |
Comment: We cannot accept LLM submission. In addition please read WP:YFA to see how to write a good draft, and in particular the need for good quality sources and references. ChrysGalley (talk) 12:21, 13 July 2026 (UTC)
Semantic search is an approach to information retrieval that attempts to represent and match the meaning of a user's query and the information contained in documents, rather than relying only on the occurrence of identical words or character strings.Bast, Hannah; Buchhold, Björn; Haussmann, Elmar (2016). "Semantic Search on Text and Knowledge Bases". Foundations and Trends in Information Retrieval. 10 (2–3): 119–271. doi:10.1561/1500000032. It is a broad category rather than a single algorithm and may involve linguistic analysis, statistical representations, structured knowledge bases, machine-learned embeddings, or combinations of these methods.
In conventional lexical retrieval, documents are commonly ranked using signals based on terms appearing in both the query and the document. Examples include TF–IDF and BM25.Manning, Christopher D.; Raghavan, Prabhakar; Schütze, Hinrich (2008). Introduction to Information Retrieval. Cambridge University Press. ISBN 978-0-521-86571-5. Semantic search methods attempt to reduce the vocabulary mismatch that occurs when a relevant document expresses a concept using words different from those used in the query. Lexical and semantic retrieval are not mutually exclusive, and many systems combine signals from both.
Scope
The term semantic search has been used for several related research areas. In research associated with the Semantic Web, it has often referred to searching structured data represented through entities, relationships, taxonomies, and ontologies. In information retrieval and natural language processing, it also refers to methods that learn representations of queries and documents in a continuous vector space.
Meaning may be modelled at different parts of the retrieval process. A system may interpret the intent or entities in a query, enrich or disambiguate the indexed information, or calculate relevance using representations intended to capture contextual or conceptual similarity.
Development
Early information-retrieval systems were largely based on lexical statistics and manually constructed vocabularies. Techniques such as stemming, query expansion, thesauri, and relevance feedback were developed to address variation in how the same information need could be expressed.
Latent semantic analysis, introduced for document indexing in 1990, represented terms and documents in a lower-dimensional space derived from their patterns of co-occurrence.Deerwester, Scott; Dumais, Susan T.; Furnas, George W.; Landauer, Thomas K.; Harshman, Richard (1990). "Indexing by Latent Semantic Analysis". Journal of the American Society for Information Science. 41 (6): 391–407. This allowed retrieval to use statistical relationships between terms even when a query and a document did not contain exactly the same vocabulary.
Research on the Semantic Web subsequently emphasized explicit representations of concepts and relationships through metadata, ontologies, linked data, and knowledge bases. These representations allowed queries to refer to entities or relations rather than only unstructured strings.
During the late 2010s and early 2020s, pretrained language models led to increased use of dense vector representations for retrieval. Models could be trained to place semantically related queries, sentences, passages, or documents near one another in a vector space.Zhao, Wayne Xin; Liu, Jing; Ren, Ruiyang; Wen, Ji-Rong (2024). "Dense Text Retrieval Based on Pretrained Language Models: A Survey". ACM Transactions on Information Systems. 42 (4) 89: 1–60. doi:10.1145/3637870.
Approaches
Knowledge-based search
Knowledge-based semantic search uses structured representations of entities, categories, attributes, and relationships. Queries may be processed using techniques such as named-entity recognition, entity linking, word-sense disambiguation, query expansion, or logical inference.
For example, a system may distinguish between different entities that share a name, relate a query term to a broader category, or retrieve information through relationships stored in a knowledge graph. The usefulness of these methods depends partly on the coverage, accuracy, and maintenance of the underlying structured knowledge.
Latent semantic methods
Latent semantic methods derive representations from statistical patterns in a collection. Latent semantic analysis applies singular value decomposition to a term–document matrix and projects terms and documents into a lower-dimensional space.
Later methods used probabilistic topic models, distributed word representations, and neural networks to learn other forms of latent representation. These approaches seek to represent relationships that are not captured by direct term matching alone.
Dense retrieval
Dense retrieval represents queries and documents as comparatively low-dimensional numerical vectors. In a common dual-encoder architecture, one encoder produces a query vector and another produces document or passage vectors. Candidate results are ranked using a similarity function such as cosine similarity or a vector dot product.
Sentence-BERT adapted the BERT language model to generate sentence embeddings that could be compared efficiently, rather than jointly processing every possible pair of sentences.Reimers, Nils; Gurevych, Iryna (2019). "Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks". Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Association for Computational Linguistics. pp. 3982–3992. doi:10.18653/v1/D19-1410.
Dense Passage Retrieval applied a dual-encoder architecture to open-domain question answering and demonstrated that learned dense representations could serve as a first-stage passage-retrieval mechanism.Karpukhin, Vladimir; Oğuz, Barlas; Min, Sewon; Lewis, Patrick; Wu, Ledell; Edunov, Sergey; Chen, Danqi; Yih, Wen-tau (2020). "Dense Passage Retrieval for Open-Domain Question Answering". Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. pp. 6769–6781. doi:10.18653/v1/2020.emnlp-main.550.
For large collections, exact comparison between a query and every stored vector may be computationally expensive. Systems therefore commonly use approximate nearest-neighbour search indexes. These indexes reduce retrieval time by accepting a configurable trade-off between search speed and the probability of returning every true nearest neighbour.Wang, Mengzhao; Xu, Xiaoliang; Yue, Qiang; Wang, Yuxiang (2021). "A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Search". Proceedings of the VLDB Endowment. 14 (11): 1964–1978. doi:10.14778/3476249.3476255.
Reranking and hybrid retrieval
In a multi-stage retrieval system, a relatively fast method first selects a set of candidate documents. A more computationally intensive model may then rerank those candidates. Cross-encoder models, for example, process a query and candidate document together to estimate their relevance.
Hybrid retrieval combines lexical and semantic signals. A system may independently retrieve results using BM25 and a dense model and then merge or rerank the result lists. Lexical methods can preserve exact matches for names, codes, dates, or uncommon terms, while dense models may retrieve conceptually related text with little vocabulary overlap.
Results vary by dataset and domain. Evaluations using the BEIR benchmark found BM25 to be a robust zero-shot baseline, while reranking and late-interaction models produced strong average results at higher computational cost. Dense retrievers did not uniformly outperform lexical approaches across all evaluated datasets.Thakur, Nandan; Reimers, Nils; Rücklé, Andreas; Srivastava, Abhishek; Gurevych, Iryna (2021). "BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models". Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. arXiv:2104.08663.
Indexing and retrieval
A semantic retrieval system commonly separates document processing from query processing. During indexing, documents may be divided into passages, normalized, annotated with entities or metadata, and converted into lexical or dense representations. The resulting representations are stored in an index.
At query time, the system generates a compatible representation of the query and retrieves candidate items. Additional stages may apply metadata filters, query expansion, relevance feedback, or reranking. The final ordering may combine semantic similarity with lexical relevance, document quality, freshness, authority, or application-specific constraints.
The division of documents into passages affects retrieval granularity. Small passages may provide precise matches but omit surrounding context, while larger passages preserve context but may contain several unrelated subjects. This choice is normally evaluated for the collection and application in which the search system is used.
Evaluation
Semantic-search systems are generally evaluated using the same relevance measures used in information retrieval, including precision and recall, mean reciprocal rank, mean average precision, and normalized discounted cumulative gain. Evaluation datasets contain queries together with human or automatically constructed relevance judgements.
Performance on a single dataset may not indicate how well a system will operate in another domain. Differences in vocabulary, document structure, query length, language, and relevance criteria can affect results. Heterogeneous benchmarks such as BEIR were introduced partly to evaluate the ability of retrieval models to generalize across tasks and subject areas.
Operational evaluations may also measure query latency, indexing time, memory usage, storage requirements, and the recall of an approximate-neighbour index.
Applications
Semantic search is used in web and enterprise search, digital libraries, scientific-literature discovery, question-answering systems, customer-support systems, and search over organizational documents.
It can also be used as the retrieval component of systems that identify passages before further classification, summarization, or answer generation. In such applications, retrieval errors can affect every later stage of processing.
Limitations
Semantic similarity does not by itself establish factual correctness, authority, or usefulness. A document may be conceptually related to a query while still being outdated, unreliable, or unsuitable for the user's information need.
Dense models may perform poorly when transferred to domains or query distributions that differ substantially from their training data. They may also be less dependable than lexical retrieval for exact identifiers, rare terminology, newly introduced names, or strings that were poorly represented during training.
Vector-based retrieval also introduces computational and storage costs. Approximate nearest-neighbour indexes improve scalability but may omit relevant vectors, creating a trade-off between latency, resource use, and retrieval recall.
Knowledge-based systems face different limitations, including incomplete ontologies, entity-linking errors, inconsistent metadata, and the cost of maintaining structured knowledge. Consequently, the suitability of a semantic-search approach depends on the language, collection, domain, query types, available training data, and relevance requirements of the application.
See also
Natural language search engine
References
Semantic Search documentation at Sentence Transformers
External links
Semantic Search documentation at Sentence Transformers
References
Content Disclaimer
Informasi ini disarikan dari Wikipedia dan disajikan kembali untuk tujuan edukasi. Konten tersedia di bawah lisensi CC BY-SA 3.0. Kami tidak bertanggung jawab atas ketidakakuratan data yang bersumber dari kontribusi publik tersebut.
- The information displayed on this website is sourced in part or in whole from Wikipedia and has been adapted for the purpose of restating it. We strive to provide accurate and relevant information, however:
- There is no guarantee of absolute accuracy. Wikipedia is an open, collaborative project that can be edited by anyone, so information is subject to change.
- It is not intended to constitute professional advice. The content displayed is for informational and educational purposes only. For important decisions (e.g., medical, legal, or financial), please consult a professional.
- Content copyright. Wikipedia is licensed under the Creative Commons Attribution-ShareAlike License (CC BY-SA). This means that content may be reused with appropriate attribution and shared under a similar license.
- Responsible use. Any risk arising from the use of information from this website is entirely the responsibility of the user.
LLM-generated pages with certain obvious signs of being machine generated may be deleted without notice.
Instead, only summarize in your own words a range of independent, reliable, published sources that discuss the subject.
See the advice page on large language models for more information.