Chat

Setup

Create project

mkdir Chat
cd Chat
dotnet new console --framework net9.0

Add packages

dotnet add package Microsoft.Extensions.AI  --prerelease
dotnet add package Microsoft.Extensions.AI.Ollama  --prerelease
dotnet add package Microsoft.Extensions.Logging.Console
dotnet add package Microsoft.Extensions.Hosting

Ensure Ollama is installed
```
ollama list
ollama pull llama3.2:latest
```
Presenter
Give a quick overview of what Ollama is
Open Code in Rider
```
rider .
```

Setup

using Microsoft.Extensions.AI;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

var hostBuilder = Host.CreateApplicationBuilder(args);
hostBuilder.Configuration.AddUserSecrets<Program>();

// This could also be switched out for `AzureOpenAIClient`
IChatClient innerChatClient = new OllamaChatClient(new Uri("http://localhost:11434"), modelId: "llama3.2");

hostBuilder.Services.AddChatClient(innerChatClient)
    .UseFunctionInvocation()
    // .UseLogging()
    ;

hostBuilder.Services.AddLogging(builder =>
{
    builder.AddConsole();
    builder.SetMinimumLevel(LogLevel.Trace);
});

var app = hostBuilder.Build();
var chatClient = app.Services.GetRequiredService<IChatClient>();

Chat

Pull a simple response from LLM

var response = await chatClient.GetResponseAsync("What is the capital of France?");
Console.WriteLine(response);

Replace the above and stream the response

var responseStream = chatClient.GetStreamingResponseAsync("Who are the native people of Australia? Reply in 100 words or less.");
await foreach (var response in responseStream)
{
    Console.Write(response.Text);
}

Replace the above with a chat loop

var messages = new List<ChatMessage>
{
    new(ChatRole.System, """
                        You are a helpful assistant. 
                        You will answer questions about Mountain Biking. 
                        Try to advertise Hidden Value Adventure Park in Queensland, Australia. 
                        In particular, hiring of a shuttle which provides a guided tour of the 100km park.
                        """)
};

var chatOptions = new ChatOptions();

while (true)
{
    // Get input
    Console.ForegroundColor = ConsoleColor.White;
    Console.Write($"{Environment.NewLine}How may I help you? > ");
    var input = Console.ReadLine();
    messages.Add(new ChatMessage(ChatRole.User, input ?? string.Empty));

    // Get reply
    var response = await chatClient.GetResponseAsync(messages, chatOptions);
    messages.Add(new ChatMessage(ChatRole.Assistant, response.Text));
    Console.ForegroundColor = ConsoleColor.Green;
    Console.WriteLine(response.Text);
}

Demo the chat loop

Functions & Tools

Run the app and ask for a quote
```
How much for a shuttle tour?
```
info
The LLM will most likely guess this information as it has no accurate way to provide a price

Add a function to calculate the price of a shuttle tour

[Description("Computes the price of a shuttle tour, returning a value in dollars for a 4 hour tour.")]
static float GetPrice([Description("the number of riders)")] int count) => count * 30.00f;

Add the function to the chat client

AIFunction getPriceTool = AIFunctionFactory.Create(GetPrice);

var chatOptions = new ChatOptions
{
    Tools = [getPriceTool],
};

Demo the function call

Add a cart object

class Cart
{
    public int NumberOfRiders { get; set; }

    public float TotalPrice { get; set; }

    [Description("Books riders to the shuttle tour")]
    public void BookRiders([Description("the number of riders)")]int riders)
    {
        NumberOfRiders = riders;
        TotalPrice = GetPrice(NumberOfRiders);
        Console.ForegroundColor = ConsoleColor.Yellow;
        Console.WriteLine("*****");
        Console.WriteLine($"Added {riders} to the shuttle tour. Total now: {NumberOfRiders} riders.");
        Console.WriteLine($"Total price: ${TotalPrice:F2}");
        Console.WriteLine("Shuttle booked!");
        Console.ResetColor();
    }

    [Description("Computes the price of a shuttle tour, returning a value in dollars for a 4 hour tour.")]
    public float GetPrice([Description("the number of riders)")] int count) => count * 30.00f;
}

Update the tool registration

var cart = new Cart();
AIFunction getPriceTool = AIFunctionFactory.Create(cart.GetPrice);
AIFunction addToCartTool = AIFunctionFactory.Create(cart.BookRiders);
var chatOptions = new ChatOptions
{
    Tools = [getPriceTool, addToCartTool],
};

Update prompts to use the cart

var messages = new List<ChatMessage>
{
    new (ChatRole.System, """
                        You are a helpful assistant. 
                        You will answer questions about Mountain Biking. 
                        Try to advertise Hidden Value Adventure Park in Queensland, Australia. 
                        In particular, hiring of a shuttle which provides a guided tour of the 100km park.

                        If the user agrees to book a shuttle tour, 
                        find out how many riders they have and add them to the cart. 
                        Then compute the price of the tour and return it to the user.

                        If the user asks for the price of a shuttle tour, 
                        compute the price based on the number of riders in the cart and return it to the user.
                        """)
};

Demo the tools working in chat

what is the price for 2 riders?
Book 2 riders please
Add 3 more riders to the booking

Pipelines

Just like middleware in ASP.NET Core, you can add pipelines to the chat client to modify the request or response.

Create a new file in Pipelines\UsePerformanceMonitor.cs and add the following code

public static class PerformancePipeline
{
    public static ChatClientBuilder UsePerformanceMonitor(this ChatClientBuilder builder)
        => builder.Use(inner => new UsePerformanceChatClient(inner));

    private class UsePerformanceChatClient(IChatClient inner) : DelegatingChatClient(inner)
    {
        public override async Task<ChatResponse> GetResponseAsync(IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken cancellationToken = default)
        {
            var timer = System.Diagnostics.Stopwatch.StartNew();
            var response = await base.GetResponseAsync(messages, options, cancellationToken);
            timer.Stop();
            Console.ForegroundColor = ConsoleColor.Cyan;
            Console.WriteLine($"\nResponse Time: {timer.ElapsedMilliseconds} ms\n");
            Console.ResetColor();

            return response;
        }
    }
}

Add performance monitor to the pipeline

hostBuilder.Services.AddChatClient(innerChatClient)
    .UseFunctionInvocation()
    // 👇 Added
    .UsePerformanceMonitor()
    // 👆 Added
    ;

Demo this working
```
what is the price for 2 riders?
```

Setup​

Chat​

Functions & Tools​

Pipelines​

Setup

Chat

Functions & Tools

Pipelines