Skip to main content
`Ctrl`+`K`

# How to use the async API for Agents

LangChain provides async support for Agents by leveraging the asyncio library.

Async methods are currently supported for the following `Tools`: `SerpAPIWrapper` and `LLMMathChain`. Async support for other agent tools are on the roadmap.

For `Tool`s that have a `coroutine` implemented (the two mentioned above), the `AgentExecutor` will `await` them directly. Otherwise, the `AgentExecutor` will call the `Tool`’s `func` via `asyncio.get_event_loop().run_in_executor` to avoid blocking the main runloop.

You can use `arun` to call an `AgentExecutor` asynchronously.

## Serial vs. Concurrent Execution

In this example, we kick off agents to answer some questions serially vs. concurrently. You can see that concurrent execution significantly speeds this up.

    import asyncio
    import time

    from langchain.agents import initialize_agent, load_tools
    from langchain.agents import AgentType
    from langchain.llms import OpenAI
    from langchain.callbacks.stdout import StdOutCallbackHandler
    from langchain.callbacks.base import CallbackManager
    from langchain.callbacks.tracers import LangChainTracer
    from aiohttp import ClientSession

    questions = [
        "Who won the US Open men's final in 2019? What is his age raised to the 0.334 power?",
        "Who is Olivia Wilde's boyfriend? What is his current age raised to the 0.23 power?",
        "Who won the most recent formula 1 grand prix? What is their age raised to the 0.23 power?",
        "Who won the US Open women's final in 2019? What is her age raised to the 0.34 power?",
        "Who is Beyonce's husband? What is his age raised to the 0.19 power?"
    ]

    def generate_serially():
        for q in questions:
            llm = OpenAI(temperature=0)
            tools = load_tools(["llm-math", "serpapi"], llm=llm)
            agent = initialize_agent(
                tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True
            )
            agent.run(q)

    s = time.perf_counter()
    generate_serially()
    elapsed = time.perf_counter() - s
    print(f"Serial executed in {elapsed:0.2f} seconds.")

    > Entering new AgentExecutor chain...
     I need to find out who won the US Open men's final in 2019 and then calculate his age raised to the 0.334 power.
    Action: Search
    Action Input: "US Open men's final 2019 winner"
    Observation: Rafael Nadal
    Thought: I need to find out Rafael Nadal's age
    Action: Search
    Action Input: "Rafael Nadal age"
    Observation: 36 years
    Thought: I need to calculate 36 raised to the 0.334 power
    Action: Calculator
    Action Input: 36^0.334
    Observation: Answer: 3.3098250249682484

    Thought: I now know the final answer
    Final Answer: Rafael Nadal, aged 36, won the US Open men's final in 2019 and his age raised to the 0.334 power is 3.3098250249682484.

    > Finished chain.

    > Entering new AgentExecutor chain...
     I need to find out who Olivia Wilde's boyfriend is and then calculate his age raised to the 0.23 power.
    Action: Search
    Action Input: "Olivia Wilde boyfriend"
    Observation: Jason Sudeikis
    Thought: I need to find out Jason Sudeikis' age
    Action: Search
    Action Input: "Jason Sudeikis age"
    Observation: 47 years
    Thought: I need to calculate 47 raised to the 0.23 power
    Action: Calculator
    Action Input: 47^0.23
    Observation: Answer: 2.4242784855673896

    Thought: I now know the final answer
    Final Answer: Jason Sudeikis, Olivia Wilde's boyfriend, is 47 years old and his age raised to the 0.23 power is 2.4242784855673896.

    > Finished chain.

    > Entering new AgentExecutor chain...
     I need to find out who won the grand prix and then calculate their age raised to the 0.23 power.
    Action: Search
    Action Input: "Formula 1 Grand Prix Winner"
    Observation: Max Verstappen
    Thought: I need to find out Max Verstappen's age
    Action: Search
    Action Input: "Max Verstappen Age"
    Observation: 25 years
    Thought: I need to calculate 25 raised to the 0.23 power
    Action: Calculator
    Action Input: 25^0.23
    Observation: Answer: 1.84599359907945
    Thought: I now know the final answer
    Final Answer: Max Verstappen, 25 years old, raised to the 0.23 power is 1.84599359907945.

    > Finished chain.

    > Entering new AgentExecutor chain...
     I need to find out who won the US Open women's final in 2019 and then calculate her age raised to the 0.34 power.
    Action: Search
    Action Input: "US Open women's final 2019 winner"
    Observation: Bianca Andreescu defeated Serena Williams in the final, 6–3, 7–5 to win the women's singles tennis title at the 2019 US Open. It was her first major title, and she became the first Canadian, as well as the first player born in the 2000s, to win a major singles title.
    Thought: I need to find out Bianca Andreescu's age.
    Action: Search
    Action Input: "Bianca Andreescu age"
    Observation: 22 years
    Thought: I now know the age of Bianca Andreescu and can calculate her age raised to the 0.34 power.
    Action: Calculator
    Action Input: 22^0.34
    Observation: Answer: 2.8603798598506933

    Thought: I now know the final answer.
    Final Answer: Bianca Andreescu won the US Open women's final in 2019 and her age raised to the 0.34 power is 2.8603798598506933.

    > Finished chain.

    > Entering new AgentExecutor chain...
     I need to find out who Beyonce's husband is and then calculate his age raised to the 0.19 power.
    Action: Search
    Action Input: "Who is Beyonce's husband?"
    Observation: Jay-Z
    Thought: I need to find out Jay-Z's age
    Action: Search
    Action Input: "How old is Jay-Z?"
    Observation: 53 years
    Thought: I need to calculate 53 raised to the 0.19 power
    Action: Calculator
    Action Input: 53^0.19
    Observation: Answer: 2.12624064206896

    Thought: I now know the final answer
    Final Answer: Jay-Z is Beyonce's husband and his age raised to the 0.19 power is 2.12624064206896.

    > Finished chain.
    Serial executed in 65.11 seconds.

    async def generate_concurrently():
        agents = []
        # To make async requests in Tools more efficient, you can pass in your own aiohttp.ClientSession,
        # but you must manually close the client session at the end of your program/event loop
        aiosession = ClientSession()
        for _ in questions:
            manager = CallbackManager([StdOutCallbackHandler()])
            llm = OpenAI(temperature=0, callback_manager=manager)
            async_tools = load_tools(["llm-math", "serpapi"], llm=llm, aiosession=aiosession, callback_manager=manager)
            agents.append(
                initialize_agent(async_tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True, callback_manager=manager)
            )
        tasks = [async_agent.arun(q) for async_agent, q in zip(agents, questions)]
        await asyncio.gather(*tasks)
        await aiosession.close()

    s = time.perf_counter()
    # If running this outside of Jupyter, use asyncio.run(generate_concurrently())
    await generate_concurrently()
    elapsed = time.perf_counter() - s
    print(f"Concurrent executed in {elapsed:0.2f} seconds.")

    > Entering new AgentExecutor chain...

    > Entering new AgentExecutor chain...

    > Entering new AgentExecutor chain...

    > Entering new AgentExecutor chain...

    > Entering new AgentExecutor chain...
     I need to find out who Olivia Wilde's boyfriend is and then calculate his age raised to the 0.23 power.
    Action: Search
    Action Input: "Olivia Wilde boyfriend" I need to find out who Beyonce's husband is and then calculate his age raised to the 0.19 power.
    Action: Search
    Action Input: "Who is Beyonce's husband?"
    Observation: Jay-Z
    Thought: I need to find out who won the grand prix and then calculate their age raised to the 0.23 power.
    Action: Search
    Action Input: "Formula 1 Grand Prix Winner" I need to find out who won the US Open women's final in 2019 and then calculate her age raised to the 0.34 power.
    Action: Search
    Action Input: "US Open women's final 2019 winner"
    Observation: Jason Sudeikis
    Thought:
    Observation: Max Verstappen
    Thought:
    Observation: Bianca Andreescu defeated Serena Williams in the final, 6–3, 7–5 to win the women's singles tennis title at the 2019 US Open. It was her first major title, and she became the first Canadian, as well as the first player born in the 2000s, to win a major singles title.
    Thought: I need to find out Jason Sudeikis' age
    Action: Search
    Action Input: "Jason Sudeikis age" I need to find out Jay-Z's age
    Action: Search
    Action Input: "How old is Jay-Z?"
    Observation: 53 years
    Thought: I need to find out who won the US Open men's final in 2019 and then calculate his age raised to the 0.334 power.
    Action: Search
    Action Input: "US Open men's final 2019 winner"
    Observation: Rafael Nadal defeated Daniil Medvedev in the final, 7–5, 6–3, 5–7, 4–6, 6–4 to win the men's singles tennis title at the 2019 US Open. It was his fourth US ...
    Thought:
    Observation: 47 years
    Thought: I need to find out Max Verstappen's age
    Action: Search
    Action Input: "Max Verstappen Age"
    Observation: 25 years
    Thought: I need to find out Bianca Andreescu's age.
    Action: Search
    Action Input: "Bianca Andreescu age"
    Observation: 22 years
    Thought: I need to calculate 53 raised to the 0.19 power
    Action: Calculator
    Action Input: 53^0.19 I need to find out the age of the winner
    Action: Search
    Action Input: "Rafael Nadal age" I need to calculate 47 raised to the 0.23 power
    Action: Calculator
    Action Input: 47^0.23
    Observation: 36 years
    Thought: I need to calculate 25 raised to the 0.23 power
    Action: Calculator
    Action Input: 25^0.23
    Observation: Answer: 2.12624064206896

    Thought: I now know the age of Bianca Andreescu and can calculate her age raised to the 0.34 power.
    Action: Calculator
    Action Input: 22^0.34
    Observation: Answer: 1.84599359907945
    Thought:
    Observation: Answer: 2.4242784855673896

    Thought: I now need to calculate his age raised to the 0.334 power
    Action: Calculator
    Action Input: 36^0.334
    Observation: Answer: 2.8603798598506933

    Thought: I now know the final answer
    Final Answer: Jay-Z is Beyonce's husband and his age raised to the 0.19 power is 2.12624064206896.

    > Finished chain.
     I now know the final answer
    Final Answer: Max Verstappen, 25 years old, raised to the 0.23 power is 1.84599359907945.

    > Finished chain.

    Observation: Answer: 3.3098250249682484

    Thought: I now know the final answer
    Final Answer: Jason Sudeikis, Olivia Wilde's boyfriend, is 47 years old and his age raised to the 0.23 power is 2.4242784855673896.

    > Finished chain.
     I now know the final answer.
    Final Answer: Bianca Andreescu won the US Open women's final in 2019 and her age raised to the 0.34 power is 2.8603798598506933.

    > Finished chain.
     I now know the final answer
    Final Answer: Rafael Nadal, aged 36, won the US Open men's final in 2019 and his age raised to the 0.334 power is 3.3098250249682484.

    > Finished chain.
    Concurrent executed in 12.38 seconds.

## Using Tracing with Asynchronous Agents

To use tracing with async agents, you must pass in a custom `CallbackManager` with `LangChainTracer` to each agent running asynchronously. This way, you avoid collisions while the trace is being collected.

    # To make async requests in Tools more efficient, you can pass in your own aiohttp.ClientSession,
    # but you must manually close the client session at the end of your program/event loop
    aiosession = ClientSession()
    tracer = LangChainTracer()
    tracer.load_default_session()
    manager = CallbackManager([StdOutCallbackHandler(), tracer])

    # Pass the manager into the llm if you want llm calls traced.
    llm = OpenAI(temperature=0, callback_manager=manager)

    async_tools = load_tools(["llm-math", "serpapi"], llm=llm, aiosession=aiosession)
    async_agent = initialize_agent(async_tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True, callback_manager=manager)
    await async_agent.arun(questions[0])
    await aiosession.close()

    > Entering new AgentExecutor chain...
     I need to find out who won the US Open men's final in 2019 and then calculate his age raised to the 0.334 power.
    Action: Search
    Action Input: "US Open men's final 2019 winner"
    Observation: Rafael Nadal
    Thought: I need to find out Rafael Nadal's age
    Action: Search
    Action Input: "Rafael Nadal age"
    Observation: 36 years
    Thought: I need to calculate 36 raised to the 0.334 power
    Action: Calculator
    Action Input: 36^0.334
    Observation: Answer: 3.3098250249682484

    Thought: I now know the final answer
    Final Answer: Rafael Nadal, aged 36, won the US Open men's final in 2019 and his age raised to the 0.334 power is 3.3098250249682484.

    > Finished chain.