Build an AI app that takes a customer ID and a support question, checks the customer’s membership and the store’s policy, and answers with a citation before its deadline. Use a Python CLI, a local Ollama model, and three read-only tools over sample JSON data. Use asyncio to overlap independent lookups while keeping the model turns in order.

What your app must do

  • Accept a question and a customer ID. For the same return question, Sam’s Plus membership should yield 45 days from delivery and Alex’s Standard membership should yield 30 days from delivery. Both answers should cite [returns].
  • Give the model three tools. One retrieves the selected customer and order, one finds candidate policies by keyword, and one loads the full text of a policy by ID. Use the customer record to pick the correct rule and the policy text to support the answer.
  • Run independent tool calls from the same model turn concurrently. If a later call needs a value from an earlier result, send that result back to the model before making the dependent call.
  • Put the full conversation under one configurable deadline. Allow at most eight model turns and 12 tool calls in a turn. Support Ctrl+C and offer a --trace flag so learners can see the tools chosen at each turn.

Build the app in your own directory. Use the finished repository to compare your implementation and verify the observable behavior.

Architecture

One Python process with two I/O boundaries
  1. CLI · __main__.pyRead the question and settings, start the event loop, and print the final answer.
  2. Agent · workflow.pyOwn the message history, deadline, and tasks for each batch of tools.
  • Ollama · ollama.pyThe async HTTP adapter sends messages and tool schemas to a separate local model process. The process returns tool requests or answer text.
  • Evidence · data/*.jsonThe agent invokes registered tools. Those tools read customer and policy files in worker threads and return JSON data to the agent.

The model chooses what evidence to request. The Python process controls which functions can run and which customer record is in scope. Each invocation gets one HTTP client, one conversation history, and one SupportTools instance. The four lesson modules are small experiments you can run from the same CLI to learn the concurrency primitives in isolation. The tests reuse the real workflow and transport with a scripted model.

Follow one request

Use Sam’s returns question to trace one successful run. The automated round-trip test fixes this exact sequence so you can debug it without Ollama.

How Sam’s question becomes a cited answer
  1. CLI“How long do I have to return my order?”Start the event loop and set one deadline for the conversation.
  2. Ollama · Turn 1Request the profile and a policy searchThe agent sends Sam’s question and the available tool schemas.
  3. Run both tools concurrently in one TaskGroup

    • Look up SamCustomer tier · Plus
    • Search policiesArticle ID · returns

    Wait for both results, then append them in call order

  4. Ollama · Turn 2Request the full returns articleThe agent loads the text using the ID from the search results.
  5. Ollama · Turn 3“Plus orders have 45 days from delivery [returns].”The reply has no tool calls, so the CLI prints the answer.
  • Start the request. main() starts the event loop and ask() opens the HTTP client. answer_question() binds the chosen customer to the tools and starts the shared deadline. The model receives a system instruction, the user’s question, and three tool schemas.
  • Turn 1 · Look up the customer and search for a policy. The model requests fetch_customer() and search_vector_db("returns") in the same reply. Python starts two named tasks in one TaskGroup. One finds Sam’s membership tier and the other finds the returns article. Once both finish, Python adds their results to the history in the original call order.
  • Turn 2 · Read the policy. The model sees the Plus tier and the returns ID and requests fetch_knowledge_base("returns"). Python adds the full article to the conversation. In this trace, the article lookup depends on the preceding search result.
  • Turn 3 · Answer with evidence. The model sees the full policy and returns an answer with [returns] and no tool calls. The agent returns the nonempty text, ask() closes the client, and main() prints the answer.

The live model may divide the requests differently. The agent can overlap calls only when the model requests them in the same turn. Each new model request waits for the previous tool batch because its results are part of the next request’s input.

Python documentation

Keep the standard library reference open when you work through the guide. asyncio is part of Python, and the app uses features available from Python 3.11. Python asyncio library documentation · Coroutines and tasks API

Set up your workspace

You’ll need Python 3.11 or later and familiarity with functions, dictionaries, exceptions, and the terminal. The first two milestones use the standard library. The reference has optional examples that exercise features through Python 3.15.

The shell commands below assume macOS or Linux. On Windows, activate the environment with .venv\Scripts\Activate.ps1 in PowerShell.

mkdir -p support-agent/asyncio_lab/data
cd support-agent
touch asyncio_lab/__init__.py
python3 -m venv .venv
source .venv/bin/activate

1. Overlap two independent lookups

Save this experiment as timing.py. The two sleeps stand in for I/O. Each coroutine logs when it starts and finishes so you can see when work overlaps.

import asyncio
from time import perf_counter


async def lookup(name, delay):
    print(f"start {name}")
    await asyncio.sleep(delay)
    print(f"finish {name}")
    return name


async def sequential():
    await lookup("customer", 0.3)
    await lookup("policy", 0.2)


async def concurrent():
    async with asyncio.TaskGroup() as group:
        customer = group.create_task(lookup("customer", 0.3), name="customer")
        policy = group.create_task(lookup("policy", 0.2), name="policy")
    return customer.result(), policy.result()


async def main():
    for run in (sequential, concurrent):
        start = perf_counter()
        await run()
        print(f"{run.__name__}: {perf_counter() - start:.2f}s")


asyncio.run(main())

Run python timing.py. The sequential run should take roughly 0.50 seconds. In the concurrent run, both lookups should start before either finishes and the total should be closer to 0.30 seconds. The policy lookup finishes first, while the returned tuple still puts the customer result first.

FunctionWhat it doesConcepts used
lookup()Logs when a simulated lookup starts and finishes, then returns its name.asyncio.sleep() suspends this coroutine so the loop can run other work.
sequential()Completes the customer lookup before starting the policy lookup.Direct await expresses a dependency. The two delays add up.
concurrent()Starts both lookups and reads their results after both finish.TaskGroup owns both tasks. Task results can be read after the group exits.
main()Times both versions with perf_counter().asyncio.run(main()) creates and closes the loop at the program boundary.

Import sleep from time and replace await asyncio.sleep(delay) with sleep(delay). Watch how blocking the loop removes the overlap.

2. Build three read-only tools

Save the reference customer fixtures and policy fixtures as customers.json and policies.json in asyncio_lab/data/.

Implement these functions and methods first. They give you evidence you can verify before you introduce a language model. Retrieval source in workflow.py

FunctionWhat it doesConcepts used
read_json()Resolves data/ beside the module with Path(__file__), reads UTF-8 text, and parses JSON.Synchronous file I/O. Async callers move the read and decode off the loop with to_thread().
embed()Case-folds text, extracts word tokens with a regex, and counts them with Counter.Sparse term-frequency vectors for local keyword retrieval.
similarity()Divides the dot product by the product of the Euclidean norms from math.hypot(). Returns 0.0 if either vector is empty.Cosine similarity. Identical nonempty vectors score about 1.0. Disjoint vocabularies score 0.0.
SupportTools.__init__()Records the CLI-selected customer ID and initializes an empty cache.Per-request state. A later call can reuse the profile without sharing it across customers.
SupportTools.fetch_customer()Reads customers.json on a cache miss and caches the matched profile or Customer not found result.to_thread() offloads the file read. A cache hit returns without suspending.
SupportTools.search_vector_db()Reads policies.json, scores the query against each title and body, and returns up to three positive matches with IDs, titles, and rounded scores.to_thread() for I/O and synchronous cosine ranking over a small local catalog.
SupportTools.fetch_knowledge_base()Returns the full policy for a given ID, or Article not found. The full text supports the answer’s citation.to_thread() for the catalog read and dependent lookup using an ID returned by search.

The three tools are async because they wait for file I/O. Tokenization, scoring, and sorting stay synchronous because they are small computations over three policies. An async def declaration alone cannot make a blocking file read or a long computation yield to the loop.

Save this check as check_tools.py and run it with python check_tools.py.

import asyncio

from asyncio_lab.workflow import SupportTools


async def main():
    tools = SupportTools("sam")
    assert (await tools.fetch_customer())["tier"] == "plus"
    matches = await tools.search_vector_db("returns")
    assert matches[0]["id"] == "returns"
    policy = await tools.fetch_knowledge_base(matches[0]["id"])
    assert "45 days" in policy["text"]
    print("Customer and policy lookups passed.")


asyncio.run(main())

3. Build the conversation loop

Build the rest of workflow.py around an injected async callable, chat(messages, tools). A scripted callable can test the workflow before you connect an HTTP client. Orchestration source in workflow.py

FunctionWhat it doesConcepts used
tool_schema()Builds one of the function descriptions in TOOLS. Customer lookup takes no arguments. Search requires query and article lookup requires article_id.JSON Schema describes required string arguments and disallows extra properties.
execute_tool()Matches the tool name to one of three handlers, checks exact keys and nonblank strings, and returns a tool message with tool_name and JSON content.Explicit dispatch, input validation, and await for a handler that can suspend during I/O. Unknown tools and invalid arguments become structured tool results.
answer_question()Owns per-request state, awaits model turns in order, runs each tool batch together, and returns the first nonempty reply without tool calls.An injected async callable, structured concurrency with TaskGroup, and one asyncio.timeout() for the conversation.

AgentError is the exception the CLI uses for failures such as an empty reply, malformed model output, or a tool-loop limit. Tool results can report missing evidence and let the conversation continue.

  1. Start the history with a system message and the user’s question. Ask the model to use the selected customer for personalized questions, read full policies before making claims, cite article IDs in square brackets, and say when evidence is missing.
  2. Await chat(messages, TOOLS). If there are no tool calls, return the nonempty answer. If there are tool calls, append the assistant message to the history.
  3. When tracing is enabled, print the turn number and the tool names. Start the calls in a single task group and use each tool name as its task name.
  4. After the group exits, append the results in the model’s original call order. Then ask the model for its next turn with the expanded history.
async with asyncio.TaskGroup() as group:
    tasks = [
        group.create_task(execute_tool(tools, call), name=call["function"]["name"])
        for call in calls
    ]
messages.extend(task.result() for task in tasks)

Use TaskGroup because the tools within a turn share a lifetime. Exiting the group waits for the whole batch. If a child raises an unexpected exception, the group cancels unfinished siblings, waits for their cleanup, and raises the grouped failure. A loop that directly awaits each handler would serialize the calls. Default asyncio.gather() propagates a child failure while other awaitables may continue running. as_completed() fits workflows that consume individual results on arrival. Here the next prompt needs the batch, and retaining the task list preserves call order regardless of finish order.

Save this as check_workflow.py and run python check_workflow.py. It uses the real retrieval functions and a scripted model to test the message contract without a server.

import asyncio
import json

from asyncio_lab.workflow import answer_question


def call(name, **arguments):
    return {"function": {"name": name, "arguments": arguments}}


async def scripted_chat(messages, tools):
    assert len(tools) == 3
    if messages[-1]["role"] == "user":
        return {
            "role": "assistant",
            "tool_calls": [
                call("fetch_customer"),
                call("search_vector_db", query="returns"),
            ],
        }

    last = messages[-1]
    if last["tool_name"] == "search_vector_db":
        profile = messages[-2]
        assert profile["tool_name"] == "fetch_customer"
        assert json.loads(profile["content"])["tier"] == "plus"
        article_id = json.loads(last["content"])[0]["id"]
        return {
            "role": "assistant",
            "tool_calls": [call("fetch_knowledge_base", article_id=article_id)],
        }

    assert last["tool_name"] == "fetch_knowledge_base"
    assert "45 days" in json.loads(last["content"])["text"]
    return {"role": "assistant", "content": "Plus orders have 45 days [returns]."}


answer = asyncio.run(
    answer_question(scripted_chat, "Can I return my order?", "sam", timeout=2, trace=True)
)
assert answer == "Plus orders have 45 days [returns]."
print(answer)
FunctionWhat it doesConcepts used
call()Builds a tool request with the same name and argument fields as Ollama.Test data builder for the tool-call protocol.
scripted_chat()Checks the message history before returning the next tool request or answer.An async test double whose replies depend on evidence returned by earlier turns.

Expect two tool-trace lines followed by the answer. The two-event test in the verification section checks that the first two tools actually overlap.

4. Add a deadline and connect Ollama

Wrap the entire model-and-tool loop in async with asyncio.timeout(timeout). Put the context outside for turn in range(1, 9) so each model request and every tool batch spends from the same time budget. Reject more than 12 calls before starting any tasks for that turn. The turn cap also stops a fast model that keeps requesting tools without producing an answer.

When the deadline expires, the timeout context cancels the current task. That cancellation reaches the active await and lets async cleanup run. The context then surfaces TimeoutError to the caller outside the block. A fresh wait_for() around each model call would apply a new timer to each await and would leave tool batches outside that timer. HTTPX also has its own network timeout, which covers transport waits. The shared asyncio deadline covers the entire conversation.

To exercise cancellation, make the fake model wait on an unset asyncio.Event inside a try block and set a flag in finally. Verify that a short request deadline raises TimeoutError and that the cleanup flag is set. If the cancellation arrives while awaiting a worker thread or the Ollama server, the Python task can stop waiting while work already started outside the event loop may continue.

Install HTTPX and Ollama, then pull the tool-capable model used by the reference. If you use the Ollama CLI without the desktop app, run ollama serve in a separate terminal first. Install Ollama

python -m pip install 'httpx>=0.28,<1'
ollama pull qwen3:4b

The Ollama adapter

FunctionWhat it doesConcepts used
OllamaChat.__init__()Stores the configured HTTP client and model.Dependency injection lets the CLI own the client and the tests provide httpx.MockTransport.
OllamaChat.__call__()Posts model, messages, tools, and stream: false to /api/chat. Checks HTTP status, assistant role, text content, and tool names.An async callable. await client.post() yields during network I/O. One complete reply gives the agent enough information to dispatch tools or finish.

Adapter source in ollama.py · Ollama tool-calling documentation

The CLI entry point

FunctionWhat it doesConcepts used
main()Parses ask and lesson, validates the question and timeout, prints results or errors, and returns status 130 for Ctrl+C.asyncio.run() owns the event loop at the CLI boundary. inspect.iscoroutinefunction() identifies which lessons need that runner.
ask()Creates one httpx.AsyncClient, wraps it in OllamaChat, and awaits answer_question().async with closes the client on success, failure, or cancellation. Reusing it allows connection pooling across model turns.

The ask options are --customer, --model, --url, --timeout, and --trace. The defaults are alex, qwen3:4b, http://127.0.0.1:11434, and 120 seconds. LESSONS maps commands to demonstrations. The synchronous runner() lesson manages its own loops. HTTPX is imported only for ask, so lessons can run with the standard library.

CLI entry point source

Run the finished reference

Clone the reference beside your own project and check out the exact revision used in this guide. Completed source on GitHub

cd ..
git clone https://github.com/vinit-devops/python-asyncio.git
cd python-asyncio
git checkout --detach b03607c2b4296355d3c53816c5707ecacf79adce
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python -m asyncio_lab ask --customer sam --trace \
  "Look up my customer profile to identify my membership tier. How many days from delivery do I have to return my order?"

An example grounded answer is “Sam, as a Plus member, you can return your order within 45 days of delivery [returns].” Wording and tool order can vary. The model may ask for the full article in the first batch if it already has the article ID. Sam’s sample order is in transit, so there is no delivery date from which to calculate a calendar deadline.

Use the same command in your own project and compare the behavior. Repeat it with --customer alex to check that changing customer data changes the answer.

Verify your project

Your implementation is finished when you can demonstrate all four behaviors.

  • Grounded answer. The trace shows customer and policy retrieval. Sam gets 45 days and Alex gets 30 days, both measured from delivery and cited as [returns].
  • Concurrent tools. A scripted model requests two independent tools in one turn. Both start before either is allowed to finish. The next chat call receives the results in the original tool-call order.
  • Dependent retrieval. In a scripted run, the model searches first and then reads the returned article ID in a later turn. Its final answer follows receipt of the full article.
  • Bounded execution. A stalled chat call runs its cleanup when the deadline expires. A fake model that keeps requesting tools stops at the turn limit.

Run the reference checks from the cloned repository. The test suite uses a fake model response with HTTPX’s mock transport, so it runs without an active Ollama server.

python -W error -m unittest discover -s tests -v
python -W error -m asyncio_lab lesson all

The reference suite runs three tests and should report OK. Read the functions in tests/test_agent.py as a second pass through the architecture. Tests on GitHub

FunctionWhat it doesConcepts used
call()Constructs a model-style tool request from a name and keyword arguments.Test data builder for the tool-call protocol.
test_model_tool_round_trip_and_concurrent_retrieval()Runs the real OllamaChat adapter with a fake server and checks the three requests and final answer.httpx.MockTransport, async integration testing, and deterministic concurrency checks.
customer()Signals that the customer tool started, waits for search to start, then calls the real customer handler.Two asyncio.Event objects detect overlap. A serial implementation cannot reach the second signal.
search()Signals that search started, waits for the customer tool to start, then calls the real search handler.Cooperative synchronization without timing assertions.
respond()Checks the URL, request payload, and accumulated evidence before returning each scripted HTTP reply.A mock transport handler can test the real wire format without a server.
test_deadline_cancels_active_model_request()Expects TimeoutError and verifies that the fake chat ran its cleanup.Cancellation propagation and an asyncio.Event as a cleanup signal.
chat() in the deadline testWaits on an unset event and signals cleanup in finally.A suspended coroutine provides a deterministic timeout target.
test_tool_loop_is_bounded()Expects an AgentError after the fake model uses all eight turns.Bounded request loops and exception assertions.
chat() in the loop-limit testRepeatedly requests an unregistered tool and checks the resulting error message.Structured tool errors and a test double that never produces a final answer.

Make two additional checks in your own project. Return 13 calls from a fake model and verify the agent raises a per-turn limit error before any tool runs. Then arrange for one tool to fail while a second tool is waiting, and verify the second tool’s finally block runs before TaskGroup exits. The task-groups lesson demonstrates that cleanup pattern.

Explore the executable lessons

Run an individual lesson with python -m asyncio_lab lesson <name>. Start with tasks, task-groups, timeouts, and threads to see the scheduling, cancellation, and I/O patterns used in the agent.

foundations.py

FunctionWhat it doesConcepts usedLesson
runner()Runs individual coroutines, inspects their lifecycle, and shows what happens on reuse or when a coroutine is dropped unawaited.asyncio.run(), loop ownership, and single-use coroutine objects.runner
identify()Checks that the current task wraps the original coroutine and returns its event loop for inspection after shutdown.current_task(), get_coro(), and get_running_loop().runner
prepare_coffee()Awaits the simulated service and returns its coffee result.Coroutine composition with direct await.runner
service()Logs the start and end of a simulated I/O delay and returns the supplied result.Cooperative suspension with asyncio.sleep().runner, tasks
coroutines()Compares creating a coroutine, awaiting it, and scheduling it. Reads a finished task twice and triggers the nested-runner error.create_task() schedules work. asyncio.run() belongs outside the active loop.coroutines
awaitables()Awaits a coroutine, a task, a future completed by a loop callback, and an executor future.The awaitable protocol, Future, and run_in_executor().awaitables
calculate()Returns 42 as the common example for the coroutine and task.A coroutine can complete without an internal suspension.awaitables
tasks()Times sequential and overlapping service calls, cleans up outstanding tasks, and runs a task with a different request ID.create_task(), sleep(0), cancellation, and per-task context.tasks
fetch_customer()Returns a mock customer after a simulated API delay.Async I/O simulation and an awaitable result.tasks
fetch_orders()Returns a mock order list after a separate simulated API delay.Independent work that can overlap the customer lookup.tasks
read_request_id()Returns the current task’s request ID so it can be compared with the caller’s ID.A task can run in its own ContextVar context.tasks

Foundations source

concurrency.py

FunctionWhat it doesConcepts usedLesson
cancellation()Starts a task, waits until it is running, cancels it, and awaits it to observe cleanup.Task.cancel() requests cancellation. Awaiting the task observes its outcome.cancellation
cancellable_job()Signals startup, waits, re-raises CancelledError, and signals cleanup in finally.Cooperative cancellation and cleanup during unwinding.cancellation, task-groups
task_groups()Reads a child result after group exit, then triggers a failure and checks that the sibling cleaned up.Structured concurrency, ExceptionGroup, and explicit group cancellation on Python 3.15.task-groups
fail_after()Waits for another task to start before deliberately raising ValueError.asyncio.Event makes the failure order deterministic.task-groups, gather
sleeping()Compares blocking and cooperative sleep, then starts a heartbeat that interleaves with foreground work.time.sleep() blocks the event loop. asyncio.sleep() suspends the coroutine.sleep
compare_pause()Starts a peer task and checks whether the peer ran during the supplied pause.An event exposes the scheduling effect of blocking versus yielding.sleep
peer()Sets the event that records whether the peer got to run.Event.set() signals progress from another task.sleep
heartbeat()Emits a series of messages that interleave with foreground steps.sleep(0) yields to the event loop.sleep
gathering()Compares ordered results, first-error propagation, errors returned as values, and cancellation of the aggregate.gather(), return_exceptions=True, and behavior that differs from TaskGroup on a child failure.gather
eager()Shows when a cache hit can finish during task creation and exercises both standard and custom task factories.Eager factories on Python 3.12 or later. Per-task eager start and keyword forwarding on Python 3.14 or later.eager
cached()Returns a cache hit immediately for the scheduling comparison.An eager task can finish synchronously when its coroutine never suspends.eager
tagged_factory()Builds a NamedTask and records a tag from the scheduling call.Custom task factory and keyword forwarding.eager

Concurrency source

reliability.py

FunctionWhat it doesConcepts usedLesson
shielding()Starts a confirmation save and a request task, cancels the request, then waits for the save to finish.Task ownership. The caller retains and joins work that should outlive a cancelled request.shield
save_confirmation()Writes a sample confirmation after a short delay.to_thread() moves a synchronous file write off the event loop.shield
request()Signals that it started and awaits the save through shield().asyncio.shield() protects the saved task while cancellation still reaches the waiter.shield
timeouts()Compares relative, absolute, and rescheduled deadlines and verifies that wait_for() waits for cleanup.timeout(), timeout_at(), reschedule(), and the loop clock.timeouts
waiting()Inspects completed and pending tasks, then consumes independent results in completion order.wait(), as_completed(), and explicit cleanup after an early return or timeout.waiting
research_tasks()Creates three named tasks with staggered delays for the waiting examples.Named tasks make completion order and pending work visible.waiting
cancel_pending()Cancels unfinished tasks, then awaits the batch to collect results and exceptions.Task.cancel() and gather(return_exceptions=True) for cleanup.waiting

Reliability source

runtime.py

FunctionWhat it doesConcepts usedLesson
threads()Offloads a file read while an event-loop heartbeat runs, then restores the request context.to_thread(), context propagation, and overlapping I/O with active event-loop work.threads
read()Simulates a blocking SDK and reports the worker thread, copied request ID, and file contents.A synchronous helper runs in a worker with the caller’s ContextVar values.threads
cross_thread()Captures the running loop and awaits a worker that submits work back to it.get_running_loop() and to_thread() model the two directions of the thread boundary.cross-thread
submit()Schedules a coroutine from the worker, waits for its result there, and cancels it on timeout.run_coroutine_threadsafe() returns a thread-safe future with a blocking result() method.cross-thread
introspection()Identifies a coroutine, schedules it as a named task, and lists unfinished tasks.iscoroutine(), create_task(), and all_tasks().introspection
inspect_worker()Prints its own task name from inside the scheduled coroutine.current_task() connects running code to the task name visible in diagnostics.introspection
task_object()Inspects task names, stack, state, results, callbacks, and exceptions across several outcomes.Task introspection, completion callbacks, and cancellation state.task-object
report()Produces a successful task result after a short delay.A task transitions from pending to done, exposing a value through result().task-object
completed()Prints the name of the task it receives on completion.add_done_callback() passes the finished task to a synchronous callback.task-object
removed()Raises an assertion if the callback runs after the lesson removes it.remove_done_callback() unregisters a callback before completion.task-object
recover_deliberately()Catches cancellation, inspects the cancellation count, clears it, and returns a value.Cancellation suppression with cancelling() and uncancel() as an explicit demonstration.task-object
fail()Raises ValueError inside a scheduled task.await propagates the failure, and Task.exception() exposes it afterward.task-object

Runtime source