To deeply understand these new workflows and pain points, we built our own support bot, which we’ve named WandBot. In this article, we’ll discuss the implementation of a Q&A bot for Weights & Biases (W&B) using GPT-4, Langchain, OpenAI embeddings, and FAISS. If you’d like to take it for a spin, it’s live on our Discord in thewandbot channel. Let us know what you think! (View Highlight)
After preprocessing, the data is ingested into the Langchain framework. The code demonstrates the use of HydeEmbeddings instead of simple embeddings. HydeEmbeddings are a more advanced type of embedding based on the Hypothetical Document Embeddings (HyDE) method, which seeks to improve search results by leveraging hypothetical answers generated by an LLM like ChatGPT instead of just keywords. Compared to simple embeddings, HyDE embeddings have real benefits for this project:
• Higher dimensionality: HydeEmbeddings use a higher-dimensional vector space, allowing them to capture more nuanced relationships between words and phrases.
• Context-awareness: HydeEmbeddings are designed to incorporate contextual information, resulting in a better understanding of the meaning and intent behind the text. By using an LLM-generated document based on a specific question or topic, HydeEmbeddings can capture relevant patterns that help find similar documents in a trusted knowledge base
• Robustness: HydeEmbeddings are more resistant to noise and ambiguity, making them better suited for handling complex language structures and diverse document formats. The use of hypothetical answers in the HyDE method helps mitigate the risk of “hallucinations” from the LLM, which can be especially useful in sensitive applications where precise information is critical, such as medicine. (View Highlight)
Using HydeEmbeddings allows the Q&A bot to take advantage of the full context of the knowledge base without the need for fine-tuning or exceeding token limits, enhancing the overall user experience. By utilizing HydeEmbeddings, our Q&A bot saw improved performance and understanding of the text. These embeddings are then used to create and store documents with metadata, forming the basis for the bot’s knowledge and response generation capabilities. (View Highlight)
The next step is to create a FAISS index, which is a powerful and efficient similarity search library for high-dimensional data (FAISS stands for Facebook AI Similarity Search).
In the complementary code, we subclassed Langchain’s FAISS class to also return the similarity scores for the retrieved document. We called it FAISSWithScore, and we used this to store document embeddings in the FAISS index. This allows for efficient document retrieval based on user queries and filtering of the retrieved documents based on similarity scores. We also updated the retriever to VectorStoreRetrieverWithScore to utilize the FAISS index for document and score retrieval, adapting to the changes in the Langchain framework. (View Highlight)
To ensure the desired behavior (and output format) from the language model, the code utilizes Langchain’s ChatPromptTemplate class. This class enables users to design a custom prompt tailored to the specific requirements of the Q&A bot. By using the ChatPromptTemplate, developers can provide context, specify desired answer format, and manage token constraints. This ensures that the model’s output is not only relevant but also well-structured. The relevant code: (View Highlight)
Our Q&A pipeline was created using RetrievalQAWithSourcesChainWithScore in Langchain, replacing the earlier VectorDBQAWithSourcesChain. This pipeline leverages the power of OpenAI embeddings and the FAISS index for efficient document retrieval. The benefits of using RetrievalQAWithSourcesChainWithScore include:
• Improved search efficiency: By using the FAISS index and OpenAI embeddings, the pipeline can search through a large number of documents quickly, yielding relevant results.
• Contextual understanding: Since the pipeline incorporates the HydeEmbeddings, it has a better understanding of the context and can provide more accurate responses.
• Scoring mechanism: The RetrievalQAWithSourcesChainWithScore class also provides a scoring mechanism, allowing the system to rank the relevance of the retrieved documents and filter the documents by the similarity scores.
• Usage with Weights & Biases: Storing the pipeline components–such as the FAISS index and embeddings–in W&B Artifacts allows for better version control, collaboration, and data portability. The use of W&B Artifacts ensures that the pipeline can be easily updated and shared among team members, facilitating the continuous improvement of the Q&A bot. In the code, the pipeline loads the artifacts using the run.use_artifact method, which simplifies the process of accessing the required data and components. (View Highlight)
The Chat class in the code serves as the chat interface (surprise!), providing stateful storage for user inputs and model responses. This is particularly useful for maintaining context during an ongoing conversation. The benefits of using the Chat class include:
• Interactive experience: The stateful storage can enable a more interactive and dynamic chat experience, as the model can eventually be improved to generate context-aware responses based on previous interactions with the user. This is great for diving deeper or refining queries to get the answer you really want.
• Flexibility: The Chat class can be easily adapted to work with various user interfaces, such as Discord and Slack applications, allowing developers to integrate the Q&A bot into different platforms seamlessly. (View Highlight)
• Service Continuity: By using GPT-4 as the primary model and GPT-3.5 Turbo as a fallback, the Q&A bot can ensure continuous operation even if the primary model is unavailable or encounters issues. This is particularly important for maintaining a consistent user experience and preventing downtime, which can negatively impact user satisfaction and trust in the system.
• Performance Optimization: GPT-4 provides state-of-the-art performance in natural language understanding and generation. By default, our Q&A bot leverages this model to deliver the highest-quality responses to user queries. However, GPT-3.5 Turbo, while slightly less powerful, still offers a high level of performance. Utilizing this fallback mechanism allows the Q&A bot to maintain its effectiveness even when the primary model is not available. Speaking of which:
• Resource Management: In some cases, the availability of the primary model, GPT-4, might be limited due to resource constraints or other factors. By incorporating a fallback mechanism, the Q&A bot can seamlessly switch to GPT-3.5 Turbo, ensuring that users continue to receive responses to their queries without being negatively impacted by resource limitations.
• Flexibility and Scalability: The inclusion of a fallback mechanism provides the Q&A bot with the flexibility to adapt to changes in the underlying language models or infrastructure. This makes it easier to scale the system, accommodate new models or updates, and ensure that the bot remains up-to-date with the latest advancements in natural language processing. (View Highlight)
