The Mona Lisa is the most famous painting in the world. Most people know who she is and generally what she looks like.
If you asked 10 people to describe her, while they’ll all arrive at similar shapes and phrases, they’ll use different words to describe the same painting.
Even if you asked 10 people to describe what the Mona Lisa is, they might use different words like: artwork, painting, or masterpiece.
This is called synonymy. And if you’re using traditional keyword search to analyze text, synonymy is a problem.
Traditional keyword search is a type of information retrieval method where a search engine or database looks for exact matches of the keywords or phrases input by the user. Here's a simple breakdown of how it works:
Indexing: The search engine first "indexes" all the documents (web pages, articles, etc.) in its database. This involves scanning all documents and creating a list of words or terms that appear in them.
Querying: When a user inputs a search term or keyword, the search engine looks for an exact match of these terms in its index.
Ranking: The search engine then ranks the search results based on their relevance, which is typically determined by how many times the search term appears in the document, whether it appears in the title or headers, its proximity to other search terms, among other factors.
Returning results: Finally, the search engine returns the ranked list of search results to the user.
Traditional keyword search can be very effective for finding documents that contain specific words or phrases. However, it also has significant limitations, and synonymy is one of them.
The problem arises because traditional keyword search engines are not good at understanding the semantics or meaning behind words. They look for exact matches of search terms and do not understand that different words can have the same or similar meanings.
For example, if a user searches for "automobile," a traditional search engine would not return documents that only contain the word "car," even though "car" and "automobile" are synonyms and essentially mean the same thing. This means the search engine could miss out on relevant documents simply because they use different words for the same concept.
This issue becomes even more problematic when dealing with highly contextual or domain-specific languages, like customer feedback, and different terms might be used to describe the same concept.
In previous posts we talked about contextual text embeddings and their benefits. One of the major implications of turning text into text embeddings is to conduct semantic search (like we do at CustomerIQ).
Semantic search is an approach to locating information that uses embeddings to understand the searcher's intent and the contextual meaning of the search query in order to provide more relevant results.
Unlike traditional keyword search, which looks for exact matches of search terms in the content, semantic search considers the context, synonyms, intent, and even the tone of the query to find the best match.
A few key features of semantic search
Semantic search provides a more sophisticated, nuanced, and relevant set of search results compared to traditional keyword search.
Just as 10 people use different words to describe Mona Lisa, your customers are going to use different words to describe your features, benefits, and jobs you help them do.
By using semantic search to search across your database of customer insights we can accomplish a number of things we couldn’t accomplish with traditional keyword search:
Overall, semantic search using embeddings has the potential to revolutionize how we search and retrieve information, making the process more intuitive, accurate, and efficient. It can provide a more natural way to interact with the textual data we collect, leading to a better user experience and more relevant search results.
As we’ve mentioned, when qualitative data is stored in CustomerIQ folders, we extract the insights and turn them into text embeddings. This helps us perform an array of game-changing analyses including semantic search.
Within the action bar on any view you’ll see a search field called “Topic search.” You can use this field to filter all the insights in view by topic. If you know what you’re looking for, write it explicitly. If you don’t, pose a simple question. CustomerIQ’s topic search will return the most relevant insights in order.
Using CustomerIQ’s topic search can help you uncover themes, identify trends, and address specific issues more effectively. Here are some examples of how semantic search can be employed in this context:
At CustomerIQ, semantic search is a big part of how we help teams build and market products people love. In minutes you can setup a database of customer insights for you to search and discover new themes to guide your roadmap and messaging.