How to Render Prompt String in Semantic Kernel for Evaluation with Prompt Flow SDK in Python?

Question

I am currently working on a project where I utilize the Semantic Kernel with a plugins directory and multiple skills. My ultimate goal is to develop a GitHu. workflow to evaluate my prompts using the Prompt Flow SDK. To achieve this, I need a way to access the final prompt string value before it is sent to the AI model.

My Question:

How can I retrieve the rendered prompt string in Semantic Kernel before it's sent to the AI model? Is there any existing functionality, method, or workaround to achieve this.

Answer 1

In C# you can use filters. Search for IPromptRenderFilter ... I am not using Python so your mileage may vary.

This is an example of how to display the Rendered Template.

internal class PromptRenderFilter : IPromptRenderFilter
{
    private readonly AppSettings _settings;

    public PromptRenderFilter(AppSettings settings)
    {
        _settings = settings;
    }

    public async Task OnPromptRenderAsync(PromptRenderContext context, Func<PromptRenderContext, Task> next)
    {
        AnsiConsole.WriteLine($"Rendering prompt for {context.Function.Name}");

        await next(context);

        if(_settings.DisplayRenderedPrompt)
            AnsiConsole.WriteLine($"Rendered prompt: {context.RenderedPrompt}");
    }
}