<h2 dir="ltr" style="line-height:1.38;margin-top:18pt;margin-bottom:6pt;"><span style="font-size: 16pt; font-family: Arial, sans-serif; font-weight: 400; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">The Pervasive Problem of Semantic Search</span></h2><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In the expansive digital age where information is not only vast but grows at an exponential rate, the quest for accurate and relevant search results has never been more critical. Within this context, Yahoo Mail, serving millions of users, understood the transformative potential of semantic search. By leveraging the prowess of OpenAI embeddings, we embarked on a journey to provide search results that would understand and match user intent, going beyond the conventional keyword-based approach. And while the results were commendable, they weren't devoid of hurdles:</span></p><ol style="margin-top:0;margin-bottom:0;padding-inline-start:48px;"> <li dir="ltr" style="list-style-type: decimal; font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre;" aria-level="1"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" role="presentation"><span style="font-size: 11pt; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; text-wrap: wrap;">Performance Bottlenecks: The integration of OpenAI embeddings, though powerful, significantly slowed down our search process.</span> </p></li> <li dir="ltr" style="list-style-type: decimal; font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre;" aria-level="1"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" role="presentation"><span style="font-size: 11pt; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; text-wrap: wrap;">User Experience: The new system demanded users to type extensively, often more than they were used to, leading to potential user dissatisfaction.</span> </p></li> <li dir="ltr" style="list-style-type: decimal; font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre;" aria-level="1"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" role="presentation"><span style="font-size: 11pt; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; text-wrap: wrap;">Habit Change: Introducing a paradigm shift in search behaviors meant we were not just altering algorithms but challenging years of user habits.</span> </p></li></ol><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Our objective was crystal clear yet daunting: We wanted to augment the semantic search with suggestions that were rapid, economically viable, and seamlessly integrated into the user's natural search behavior.</span></p><br><br><h2 dir="ltr" style="line-height:1.38;margin-top:18pt;margin-bottom:6pt;"><span style="font-size: 16pt; font-family: Arial, sans-serif; font-weight: 400; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Approach: Exploration Phase</span></h2><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Enticed by the idea of real-time suggestions via large language models (LLMs), we soon realized the impracticality of such an approach, primarily due to the speed constraints. The challenge demanded a solution that operated offline but mirrored the capabilities of real-time systems.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Our exploration led us to task the LLM to frame and answer all conceivable questions for every email a user received. While theoretically sound, the financial implications were prohibitive. Moreover, the risk of the LLM generating "hallucinations" or inaccurate results couldn't be ignored.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">It was amidst this exploration that a revelatory idea emerged. We were already equipped with a sophisticated extraction pipeline capable of gleaning crucial information from emails. This was achieved using a blend of human curated regex parsing and meticulously fine-tuned AI models. This became the key to powering our search suggestions.</span></p><br><br><br><h2 dir="ltr" style="line-height:1.38;margin-top:18pt;margin-bottom:6pt;"><span style="font-size: 16pt; font-family: Arial, sans-serif; font-weight: 400; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Implementation Challenges: Transitioning from Conceptualization to Real-World Application</span></h2><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">1. The Intricacies of Indexing: One of the more pronounced challenges we encountered revolved around the intricacies of over-indexing. Let's delve into a hypothetical yet common scenario to elucidate this. Imagine a user intending to search for the term "staples." As they begin their search with the initial letters "sta", an all-encompassing approach to indexing, which takes into account every conceivable keyword, might mistakenly steer the user towards unrelated terms like "statement." Such deviations, although seemingly minor, can significantly hamper the user experience. Recognizing the paramount importance of ensuring that our search suggestions remained razor-sharp in their precision and highly relevant, we embarked on a methodical approach. Our resolution was to meticulously handpick and index only a curated set of keywords, ensuring that every suggestion offered was in perfect alignment with the user's intent.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">2. The Quest for Relevance in Suggestions: Another challenge that frequently emerged was ensuring the highest degree of relevance in our search suggestions. This challenge becomes particularly pronounced when one considers a situation where a user's inbox is populated with multiple items that bear a resemblance to each other, say multiple flight confirmations. The conundrum we faced was discerning which of these similar items was of immediate interest to the user. Our breakthrough came in the form of an innovative approach centered on the extraction card date. Rather than basing our suggestions on the date the email was received, we shifted our focus to the date of the event described within the email, like a flight's departure date. This nuanced change enabled us to consistently zero in on and prioritize the most timely and pertinent result for the user.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">3. Embracing Dynamism and Adaptability: When we first conceptualized our approach, our methodology was anchored in generating questions and answers during the email delivery phase, which were then indexed. However, as we delved deeper, it became evident that this approach, while robust, was somewhat inflexible and lacked the dynamism that modern search paradigms demand. Determined to infuse our system with greater adaptability, we pioneered the Just-in-Time question generation mechanism. With this refined approach, while the foundational search indexes are crafted at the point of delivery, the actual questions are dynamically constructed in real-time, tailored to the user's specific queries and the prevailing temporal context. This rejuvenated approach not only elevated the flexibility quotient of our system but also enhanced operational efficiency, ensuring that users always received the most pertinent suggestions.</span></p><h2 dir="ltr" style="line-height:1.38;text-align: justify;margin-top:18pt;margin-bottom:6pt;"><span style="font-size: 16pt; font-family: Arial, sans-serif; font-weight: 400; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Implementation</span></h2><h3 dir="ltr" style="line-height:1.38;margin-top:16pt;margin-bottom:4pt;"><span style="font-size: 14pt; font-family: Arial, sans-serif; color: rgb(67, 67, 67); font-weight: 400; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">At delivery time</span></h3><ul style="margin-top:0;margin-bottom:0;padding-inline-start:48px;"> <li dir="ltr" style="list-style-type: disc; font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre;" aria-level="1"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" role="presentation"><span style="font-size: 11pt; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; text-wrap: wrap;">Here we extract important information, create cards from the emails and save it in our BE store.</span> </p></li></ul><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="border:none;display:inline-block;overflow:hidden;width:624px;height:203px;"><img src="https://lh7-us.googleusercontent.com/XlIgexwxvY16iECtJhtFb9BRo_K0Qpp8c96HDscEeTC4sXeAoFuzlLoFBuRbbBn7oTaMrRiWlpMa9arKZRgo1nc6M-CdYNsCrhX5CVTGMSHElz_0egTEQ61xNaEROaMVLWepGQM67mHIQpAmdpjOuOk" width="624" height="203" style="margin-left:0px;margin-top:0px;"></span></span></p><h3 dir="ltr" style="line-height:1.38;margin-top:16pt;margin-bottom:4pt;"> <span style="font-size: 14pt; font-family: Arial, sans-serif; color: rgb(67, 67, 67); font-weight: 400; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Semantic Search Indexing</span></h3><ul style="margin-top:0;margin-bottom:0;padding-inline-start:48px;"> <li dir="ltr" style="list-style-type: disc; font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre;" aria-level="1"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" role="presentation"><span style="font-size: 11pt; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; text-wrap: wrap;">Fetch/Update the extracted cards from BE DB. Index by extracting the keywords and storing in Semantic Search Index - DB</span> </p></li></ul><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="border:none;display:inline-block;overflow:hidden;width:624px;height:215px;"><img src="https://lh7-us.googleusercontent.com/jqk3LhXr48R60DZccCiV8NUl0M8le9JfYkdjukvGiArSMv6PpskJMhW3wvFdW6bL0800osfB2GsIlPflonGJl_nwhEZsz_8p1xidFhYGI1Xkpy20nDbIG7_fnfyvIYYmfFCUx1Q0sgfUn2ASh1Q-AUU" width="624" height="215" style="margin-left:0px;margin-top:0px;"></span></span></p><h3 dir="ltr" style="line-height:1.38;margin-top:16pt;margin-bottom:4pt;"> <span style="font-size: 14pt; font-family: Arial, sans-serif; color: rgb(67, 67, 67); font-weight: 400; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Retrieval</span></h3><ul style="margin-top:0;margin-bottom:0;padding-inline-start:48px;"> <li dir="ltr" style="list-style-type: disc; font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre;" aria-level="1"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" role="presentation"><span style="font-size: 11pt; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; text-wrap: wrap;">When the user makes the search, we make a server call which inturn will find the best matching extraction card for the query.</span> </p></li> <li dir="ltr" style="list-style-type: disc; font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre;" aria-level="1"><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" role="presentation"><span style="font-size: 11pt; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; text-wrap: wrap;">This will then be used for generating the suggestions for the semantic search.</span> </p></li></ul><h2 dir="ltr" style="line-height:1.38;text-align: justify;margin-top:18pt;margin-bottom:6pt;"><span style="font-size: 16pt; font-family: Arial, sans-serif; font-weight: 400; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="border:none;display:inline-block;overflow:hidden;width:624px;height:235px;"><img src="https://lh7-us.googleusercontent.com/uQLJEFuqU6Q48uDdqxOpD-sZPWvhf_nC0gYWHq7T12OF7XfaORsWEfnWnUDBYAzMOlSGCQnkAVzd9jvfI1SU0Uz2U8jfPocsw3H0TdnOHmIFhLGqAiSUTXb_AfcnVDwKjkAiyUq-4LVv1FjnB6VSALg" width="624" height="235" style="margin-left:0px;margin-top:0px;"></span></span></h2><br><br><h2 dir="ltr" style="line-height:1.38;margin-top:18pt;margin-bottom:6pt;"> <span style="font-size: 16pt; font-family: Arial, sans-serif; font-weight: 400; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Conclusion</span></h2><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Our innovative foray into enhancing search suggestions bore fruit in a remarkably short span of 30 days, even as we navigated the intricacies of a completely new tech stack. The benefits were manifold, an enriched user experience and 10% of semantic search traffic handled by search suggestions.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In the rapidly evolving realm of AI, challenges are omnipresent. However, our journey at Yahoo underscores the potential of lateral thinking and a commitment to User Experience. Through our experiences, we hope to galvanize the broader tech community, encouraging them to ideate and implement solutions that are not just effective, but also economically prudent.</span></p><div><span style="font-size: 11pt; font-family: Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br></span></div><div><b>Contributors</b> <div>Kevin Patel(<a href="mailto:patelkev@yahooinc.com">patelkev@yahooinc.com</a>) + Renganathan Dhanogopal(<a href="mailto:renga@yahooinc.com">renga@yahooinc.com</a>) - Architecture + Tech Implementation </div> <div>Josh Jacobson + Sam Bouguerra(<a href="mailto:sbouguerra@yahooinc.com">sbouguerra@yahooinc.com</a>) - Product </div></div><br/><div><b>Author</b> <div>Kevin Patel(<a href="mailto:patelkev@yahooinc.com">patelkev@yahooinc.com</a>) - Director of Engineering Yahoo </div></div>