{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LLM Observability for Langchain with Agenta"
   ]
  },
  {
   "attachments": {
    "CleanShot%202024-11-18%20at%2020.13.46@2x.png": {
     "image/png": "[encoded data, 617412 characters]"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "This guide shows you how to set up tracing for a RAG application in Langchain using Agenta, the open-source LLMOps platform.\n",
    "\n",
    "Tracing allows us to debug effectively complex LLM applications. It allows us to view exact prompts sent and contexts retrieved. \n",
    "\n",
    "\n",
    "We'll build a simple Q&A RAG application that answers questions about Agenta's documentation. Then, we'll instrument the calls (i.e., trace them) and view the trace in Agenta.\n",
    "\n",
    "This guide uses a Jupyter notebook which you can find here.\n",
    "\n",
    "![CleanShot%202024-11-18%20at%2020.13.46@2x.png](attachment:CleanShot%202024-11-18%20at%2020.13.46@2x.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "First, let's install the langchain, Agenta, and instrumentation dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "pip install --quiet agenta langchain langchain_community langchain langchain-chroma langchain-openai opentelemetry-instrumentation-langchain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenAI API key:········\n",
      "Agenta API key:········\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import getpass\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(prompt=\"OpenAI API key:\") \n",
    "# Create an Agenta API key under https://cloud.agenta.ai/settings?tab=apiKeys\n",
    "os.environ[\"AGENTA_API_KEY\"] = getpass.getpass(prompt=\"Agenta API key:\") \n",
    "os.environ[\"AGENTA_HOST\"] = \"https://cloud.agenta.ai\" # Change for self-hosted\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup tracing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "---------------------------\n",
      "Agenta SDK - using version: 0.27.3\n",
      "---------------------------\n",
      "--------------------------------------------\n",
      "Agenta SDK - connecting to otlp receiver at: https://cloud.agenta.ai/api/observability/v1/otlp/traces\n",
      "--------------------------------------------\n",
      "Success: traces will be exported.\n",
      "--------------------------------------------\n",
      "Overriding of current TracerProvider is not allowed\n",
      "Attempting to instrument while already instrumented\n"
     ]
    }
   ],
   "source": [
    "import agenta as ag\n",
    "from opentelemetry.instrumentation.langchain import LangchainInstrumentor\n",
    "\n",
    "ag.init()\n",
    "\n",
    "LangchainInstrumentor().instrument()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Q&A RAG Application\n",
    "\n",
    "This Langchain RAG application:\n",
    "\n",
    "1. Ingests a page from our documentation (chunks and embeds it)\n",
    "2. Runs the user's query and retrieves relevant context from the page\n",
    "3. Creates a prompt based on the question and retrieved context, then returns an answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'To save a new version of a prompt in Agenta, you need to create a variant, which acts like a branch in git for versioning. After making your changes, commit them to the variant. Finally, you can deploy the specific version of your variant to the desired environment.'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "\n",
    "import bs4\n",
    "from langchain import hub\n",
    "from langchain_chroma import Chroma\n",
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.runnables import RunnablePassthrough\n",
    "from langchain_openai import OpenAIEmbeddings\n",
    "from langchain_text_splitters import RecursiveCharacterTextSplitter\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt = \"\"\"\n",
    "You are an assistant for question-answering tasks.\n",
    "Use the following pieces of retrieved context to answer the question.\n",
    "If you don't know the answer, just say that you don't know.\n",
    "Use three sentences maximum and keep the answer concise and to the point.\n",
    "\n",
    "Question: {question} \n",
    "\n",
    "Context: {context} \n",
    "\n",
    "Answer:\n",
    "\"\"\"\n",
    "\n",
    "prompt_template = ChatPromptTemplate([\n",
    "    (\"human\", prompt),\n",
    "])\n",
    "\n",
    "llm = ChatOpenAI(model=\"gpt-4o-mini\")\n",
    "\n",
    "loader = WebBaseLoader(\n",
    "    web_paths=(\"https://docs.agenta.ai/prompt-management/prompt-management-sdk\",),\n",
    "    bs_kwargs=dict(\n",
    "        parse_only=bs4.SoupStrainer('article')  # Only parse the core\n",
    "    ),\n",
    ")\n",
    "docs = loader.load()\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)\n",
    "splits = text_splitter.split_documents(docs)\n",
    "vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())\n",
    "\n",
    "# Retrieve and generate using the relevant snippets of the blog.\n",
    "retriever = vectorstore.as_retriever()\n",
    "\n",
    "\n",
    "rag_chain = (\n",
    "    {\"context\": retriever, \"question\": RunnablePassthrough()}\n",
    "    | prompt_template\n",
    "    | llm\n",
    "    | StrOutputParser()\n",
    ")\n",
    "\n",
    "rag_chain.invoke(\"How can I save a new version of a prompt in Agenta?\")\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
