It’s Tuesday morning. You have a meeting tomorrow to review next month's product roadmap and priorities. You're staring down a list of requests from sales, issues from support, comments on recent reviews from the CEO, and ideas from that one overly creative colleague, Brad.
What do you do now?
If you value customer feedback, you'll want to organize that pile of feedback by theme and a value metric, then prioritize what fits your product vision and drives the most value.
But unfortunately it's not that easy. If you're lucky enough to have tons of feedback, it's a huge pain to organize. If you're not as fortunate, like in the case of B2B teams, you have to go digging for feedback in longform content like recorded calls, meetings, and CRM notes. And after you've found it, it's a huge pain to organize.
When it comes to analyzing calls, for every 1 hour of text transcript it takes a researcher about 3 hours to analyze. The bulk of that time is spent extracting notes, quotes, and highlights from the text.
In this post we’ll discuss how CustomerIQ uses a family of AI models, including Large Language Models, to shorten the time it takes to extract, organize, and summarize product feedback from hours to seconds.
Extracting insights from longform text presents significant challenges due to the nature of analyzing lengthy transcripts and reviews. The sheer volume of information contained within these texts can be overwhelming, making it difficult to extract meaningful insights efficiently, especially considering how exhausting it can be to manually read through.
Traditional approaches to analyzing longform text, such as manual reading or basic keyword searches, fall short in extracting the broader context and uncovering valuable insights. Manual reading is subjective, prone to biases, and inefficient when dealing with large volumes of text. Basic keyword searches can be limited in capturing the nuanced meanings and contexts embedded within the feedback.
Without effective techniques in place, organizations may miss out on crucial insights hidden within longform text, impacting their ability to make informed decisions and improve their products or services based on customer feedback.
Fortunately, advancements in natural language processing (NLP) and the utilization of Large Language Models (LLMs) offer a solution to overcome these challenges. LLMs provide powerful capabilities to automatically extract key points, sentiments, and themes from lengthy customer feedback, enabling organizations to unlock valuable insights efficiently and at scale.
An LLM is an artificial intelligence model that has been trained on a vast amount of text data. These models are designed to generate or synthesize human-like text based on the input they receive. They do this by learning the statistical patterns in the data they're trained on. For example, if a model is trained on a lot of English text data, it will learn the general rules of English grammar, common phrases, and even some factual information, all from the patterns in the data.
At a high level, here's how these models work:
One of the remarkable capabilities of LLMs is their capacity to understand context. They can comprehend the meaning of words and phrases based on their surrounding context, which enhances their ability to extract meaningful insights from longform text. LLMs consider the broader context of the feedback, including the relationships between words, sentence structures, and even the overall document, to capture the intended message accurately.
When we say that a large language model (LLM) "synthesizes" text, it typically means the model is being used to produce text that serves a specific purpose or fulfills a particular task. In the case of CustomerIQ, that task is to extract the important phrases out of longer bodies of text (customer feedback).
While the underlying mechanics of the model don't change, the way we use the model does. This use case involves the model taking in some input (the text submission), processing it, and outputting a response that is contextually relevant and fulfills the task - summarizing the most important parts.
When we ask the model to extract insights from longform text, it doesn't simply generate text based on the patterns it has learned; it tries to understand (in a statistical sense) what we mean by “insight” and provide a relevant answer. It uses the patterns it has learned to synthesize an answer that's appropriate for the input question.
With this contextual understanding, LLMs can identify not only explicit key points but also subtle nuances, implied sentiments, and underlying themes embedded within the text. This enables organizations to gain a comprehensive understanding of customer feedback and extract deep insights that were previously challenging, or exhausting, to uncover.
LLMs represent a significant breakthrough in text extraction. Their advanced natural language processing capabilities, combined with their contextual understanding and ability to extract meaningful insights, empower organizations to leverage longform text effectively and uncover valuable information that drives product improvements and customer-centric decision-making.
The magic behind our Folders at CustomerIQ is in our use of LLMs to process and analyze the longform text submissions.
When data is submitted to a folder, a number of key functions are run:
Later, in Views, we're able to organize all these insights automatically by theme, classify them into different categories (ie. UX issues, feature requests, bugs, etc.), or search and monitor specific topics like "What are people saying about the new pricing model?"
Here’s a bit more detail on each step:
CustomerIQ initiates the text extraction process by transforming non-textual feedback into transcribed and diarized text. This involves converting audio recordings, video content, or any other non-textual formats into written text. By transcribing and diarizing the feedback, CustomerIQ ensures that all data is in a text format, ready for further analysis.
To tackle the challenge of analyzing lengthy customer feedback, CustomerIQ breaks down the text into manageable chunks called snippets. This step allows for a more focused analysis, prevents information overload, and enables efficient processing of each section. Breaking down the text also enhances the effectiveness of subsequent steps in the extraction process.
CustomerIQ leverages the power of LLMs to extract crucial key points from each text snippet. By utilizing LLMs' advanced natural language processing capabilities, the system can identify the most significant and relevant insights within the text. These key points capture the essence of the customer feedback, including important themes, sentiments, and specific details that inform product improvements and decision-making.
The transformative role of CustomerIQ's process lies in the conversion of insights into embeddings. Embeddings are numerical representations of textual data that capture the semantic meaning and relationships between words or phrases. By transforming key points into embeddings, CustomerIQ unlocks the potential for advanced analysis techniques such as cluster analysis, classification, and semantic search.
Considering the speed, accuracy, and scalability of using LLMs to extract insights, the question shifts from, “how do we analyze feedback?” to “Where can we find more?”
Consider for a moment all the different interactions members of your team have with customers where customers might provide insight into future product or marketing opportunities. Support calls, sales calls, customer success calls, survey responses, support tickets, product reviews, tweets, Reddit posts, community forums, focus groups… the sources are nearly endless.
Using folders, you could store each of these interactions, pulling out relevant insights and converting them into embeddings.
Using views you could discover themes and summarize the value of each theme for prioritizing work.
Using a new view, you could monitor a particular theme to watch how it changes over time.
And you can do all of this in minutes.