How to intercept generated prompts of kernel in SemanticKernel?

Question

In SemanticKernel I want to see what are the final rendered prompts that are being sent to OpenAI. I've found the PromptRendered event:

kernel.PromptRendered += (sender, args) => Console.WriteLine(args.RenderedPrompt);

But unfortunately, it can't be used anymore in 1.0.0-rc4.

How can I intercept the final rendered prompts for versions later than 1.0.0-rc4?

Answer 1

As of now, PromptRendered is still available in the latest Semantic Kernel version 1.0.1. However, pre and post hooks (FunctionInvoked, PromptRendering, PromptRendered) are marked as Experimental. To use them, you'll need to suppress the Error SKEXP0004. You can do this by adding the following line of code

    #pragma warning disable SKEXP0004
    kernel.PromptRendered += (sender, args) => Console.WriteLine(args.RenderedPrompt);

If your goal is solely to capture the rendered prompt, setting the logging to trace level will achieve this for you. I've put together a quick demo for reference.

using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel;
using Microsoft.Extensions.DependencyInjection;
using System.Threading;
using Moq;
using Microsoft.Extensions.Logging;

var chatMessage = new ChatMessageContent(AuthorRole.Assistant, "mocked chat message response");

var chatCompletion = new Mock<IChatCompletionService>();
chatCompletion
        .Setup(cc => cc.GetChatMessageContentsAsync(It.IsAny<ChatHistory>(), It.IsAny<PromptExecutionSettings>(), It.IsAny<Kernel>(), It.IsAny<CancellationToken>()))
        .ReturnsAsync(new List<ChatMessageContent> { chatMessage });
        
var logFactory = LoggerFactory.Create(builder => builder.AddConsole().SetMinimumLevel(LogLevel.Trace) );

var builder = Kernel.CreateBuilder();
builder.Services.AddSingleton<IChatCompletionService>(chatCompletion.Object);
builder.Services.AddSingleton<ILoggerFactory>(logFactory);
var kernel = builder.Build();

#pragma warning disable SKEXP0004
kernel.PromptRendered += (sender, args) => Console.WriteLine(args.RenderedPrompt);

var function = kernel.CreateFunctionFromPrompt("This is a function to test {{$topic}}");
FunctionResult actual = await kernel.InvokeAsync(function, arguments: new()
           {
               { "topic", "rendering template" },
           });

output

trce: Microsoft.SemanticKernel.KernelPromptTemplate[0]
      Extracting blocks from template: This is a function to test {{$topic}}
info: func04d92c3cd832433d8e92925b3ccad9bd[0]
      Function func04d92c3cd832433d8e92925b3ccad9bd invoking.
trce: func04d92c3cd832433d8e92925b3ccad9bd[0]
      Function arguments: {"topic":"rendering template"}
trce: Microsoft.SemanticKernel.KernelFunctionFactory[0]
      Rendered prompt: This is a function to test rendering template
This is a function to test rendering template
info: Microsoft.SemanticKernel.KernelFunctionFactory[0]
      No model ID provided to capture usage details.
info: func04d92c3cd832433d8e92925b3ccad9bd[0]
      Function func04d92c3cd832433d8e92925b3ccad9bd succeeded.
trce: func04d92c3cd832433d8e92925b3ccad9bd[0]
      Function result: {"Role":{"Label":"assistant"},"Content":"mocked chat message response","Items":null,"ModelId":null,"Metadata":null}
info: func04d92c3cd832433d8e92925b3ccad9bd[0]
      Function completed. Duration: 0.0004493s