While researching how to empower your application with AI capabilities, I tried to take some first steps with “Semantic Kernel“. I found it a bit difficult to get started, so I decided to write this article to help others. This article is based on the current preview version 0.16.230615.1 of Semantic Kernel.
What is Semantic Kernel?
Semantic Kernel is a library from Microsoft that can be leveraged to use AI features in your application. There are two versions of it available. One for Python and one for .NET. As our main backend technology stack is .NET based, I will focus on the .NET version for this article.
What can you do with Semantic Kernel?
As we know from the current hype, we can generally do pretty much anything with AI, and Semantic Kernel tries to provide an abstraction layer that makes it easier to use AI in your application. The main building block of Semantic Kernel is, you might have guessed it, a “Kernel”. So we build, or configure, a “Kernel”. That kernel can be configured to make use of different AI models for different jobs, i.e. generating or retrieving embeddings or responding to queries, and it can also be configured to use different data sources, i.e. a vector database as “Memory”, to search for similar items. Besides that, you can also add skills to your kernel that will be available to the model. You also define a pipeline of different steps the Kernel will process. You give it the initial input, and every step in the pipeline produces an output that will be passed into the next step. The last step in the pipeline will hopefully be your desired output.
What should we build?
One of the easiest things we can do with Semantic Kernel is to mimic chat GPT. So let’s go for that. As we will be using the OpenAI API to access the actual GPT model, you will need an OpenAI API Key in order to follow along with this example. I will also omit all non-necessary code in this post, you can see the full implementation in the repository for this example.
Before we start, let’s take a second and think about the general approach we would like to take.
The chatbot should be a console application. I like to build console applications that run for some time and need to keep track of state with an IHostedService and use the .NET infrastructure for configuration, user secrets and DI, so we will need Microsoft.Extensions.Hosting as one dependency. As we want to build on Semantic Kernel with the OpenAI model we also need those dependencies.
The chatbot needs a way to get its input, so we need a skill where it can ask the user for a question on the console. With that input, we need to go to the chat model, but we also need to make sure the model knows what we were already talking about, so it needs some type of short-term memory with the previous prompts and the previous answers, to keep track of the context. We wrap this functionality in a chat skill. Then the kernel needs some way to present the output, so it needs the skill to write the output on the console.
In the actual sample implementation, I also added a way to end the conversation, but I will also omit this from this post.
Starting our project
First, we create a new project and add our required dependencies. Please note that SemanticKernel is in preview and it is a fast-moving target. It might be that the next prerelease version will introduce breaking changes, so you might really want to check out the example repository and the specific versions used there. So with all that out of the way, let’s start with the code:
dotnet new console -n SemanticChat -o . -f net7.0
dotnet user-secrets init
dotnet add package Microsoft.Extensions.Hosting
dotnet add package Microsoft.SemanticKernel --prerelease
dotnet add package Microsoft.SemanticKernel.Connectors.AI.OpenAI --prerelease
Code language: Bash (bash)You want to also add your Open API Api key to your user secrets so that you don’t accidentally check it into your repo:
dotnet user-secrets set "OpenAI:ApiKey" "..your apikey.."
Code language: Bash (bash)Be aware that .NET only applies the user secrets when running in the development environment, so make sure to set your DOTNET_ENVIRONMENT environment variable to Development or configure that variable in your launchSettings.json profile.
Skills
Let’s start with the skills, as they do only one thing each. The Semantic Kernel library needs to be able to know what a skill can do, so the methods in our classes for the skills will need some Semantic Kernel-specific attributes on them to explain the semantics of the skills.
Skill One: Console input
We wait for the user to input a line, and pass it back. Since the kernel skills should be asynchronous, we use the In-stream of the console directly to be able to read in an async fashion, but the rest is really straightforward.
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.SkillDefinition;
internal class ConsoleInputSkill
{
[SKFunction("Get input from the console")]
[SKFunctionName("GetInput")]
public async Task<string> GetInputAsync(SKContext context)
{
var line = String.Empty;
while (String.IsNullOrWhiteSpace(line))
{
line = await Console.In.ReadLineAsync();
}
return line;
}
}
Code language: C# (cs)Skill two: Console output
We print the response to the console. Again, as skills are called asynchronously, we use the Out-stream of the console to write the response.
using Microsoft.SemanticKernel.Orchestration;
using Microsoft.SemanticKernel.SkillDefinition;
internal class ConsoleOutputSkill
{
[SKFunction("Write output to the console.")]
[SKFunctionName("WriteOutput")]
public async Task WriteOutputAsync(string output, SKContext context)
{
await Console.Out.WriteLineAsync(output);
}
}
Code language: C# (cs)Skill three: the actual chatting
This is a bit more involved. We will need to keep track of the chat history so far, and we also need to pass some parameters to the OpenAI chat api to control what the model does.
The Semantic Kernel library already provides us with corresponding settings objects and also with types for keeping track of the history, but we need manually use these.
For the configuration of the chat system, the library provides the ChatRequestSettings object. And you can see from the name, that it is for some reason not called ChatRequestOptions. The whole Semantic Kernel does not follow the .NET-isms we all know and love from .NET 5+ and ASP.NET Core, so we will need to work around that a bit. That said, the ChatRequestSettings are a simple object only keeping track of some setting properties, so we will use these as an Options object and configure it in the main method a bit later.
The kernel also provides us with an IChatCompletion service, that we need to use to talk to the model. So let’s bring it together:
We create a new chat and we initialize this with a system prompt that will set the initial context for the model. We also take that from the configuration, where we will tell the model what it is and how it should act. We now get a history tracking object back for that specific chat that is already initialized with the system prompt.
Whenever our skill is invoked with the latest message from the user, we will add this to the history. We now ask for an answer by providing the history as well as the request settings to the model, wait for that API call to return, and then adding the response we received to the history too so that we will remember what the last answer from the model was.
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Options;
using Microsoft.SemanticKernel.AI.ChatCompletion;
using Microsoft.SemanticKernel.SkillDefinition;
internal class OpenAIChatSkill
{
private readonly ChatRequestSettings _chatRequestSettings;
private readonly IChatCompletion _chatCompletion;
private readonly ChatHistory _history;
public OpenAIChatSkill(IConfiguration configuration, IOptions<ChatRequestSettings> chatRequestSettings, IChatCompletion chatCompletion)
{
_chatRequestSettings = chatRequestSettings.Value;
_chatCompletion = chatCompletion;
_history = _chatCompletion.CreateNewChat(configuration["OpenAI:SystemPrompt"]);
}
[SKFunction("Get answer from the OpenAI chat model")]
[SKFunctionName("AskModel")]
public async Task<string> GetAnswerFromModelAsync(string userMessage)
{
try
{
_history.AddUserMessage(userMessage);
var reply = await _chatCompletion.GenerateMessageAsync(_history, _chatRequestSettings);
_history.AddAssistantMessage(reply);
return reply;
}
catch (Exception ex)
{
return "Sorry, I'm having technical difficulties answering your question. Please try again later.";
}
}
}
Code language: C# (cs)The chatbot background worker class
Now, we need to work on the kernel. We need to use the three skills to act like a chatbot. The kernel will be configured a bit later, here we’re going to only use it. That said, there is a small caveat: As I mentioned when we implemented the OpenAIChatSkill, the IChatCompletion service is provided by the Kernel itself. We cannot, however, configure the kernel with a skill that itself needs a configured kernel to get a service from. This is sort of a chicken and egg problem. Because of this, we will configure the kernel only with the first two skills, and then add the third skill right before using it in our background worker.
So we will get the IKernel injected as well as the OpenAIChatSkill and an instance of an IHostApplicationLifetime to be notified when our console application is terminated (i.e. by pressing Ctrl+C). This example is a bit flaky, so in the example implementation in the repository you will see a bit more complex but also more resilient handling of the application lifetime.
In the constructor, we will add the OpenAIChatSkill to the IKernel. In the lifecycle methods of our ChatBotService we will simply start an endless loop that works the kernel. The Semantic Kernel allows us to retrieve a skill by name and invoke it with its corresponding parameters. So first, we tell the user manually what we expect from him (chatting with the bot), and we invoke the kernel to tell it to use the output skill to send the initial message to the user.
We then build a pipeline, which is an array of our input, chat, and output skills, and tell the kernel to run this pipeline until the application is stopping.
using Microsoft.Extensions.Hosting;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.SkillDefinition;
internal class ChatBotService : IHostedService
{
private readonly IKernel _kernel;
private readonly IHostApplicationLifetime _lifetime;
public ChatBotService(IKernel kernel, OpenAIChatSkill chatSkill, IHostApplicationLifetime lifetime)
{
_kernel = kernel;
_lifetime = lifetime;
// Need to import the skill, as we couldn't do that at service registration time
_kernel.ImportSkill(chatSkill);
}
public Task StartAsync(CancellationToken cancellationToken)
{
ExecuteAsync(_lifetime.ApplicationStopping);
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
protected async Task ExecuteAsync(CancellationToken stoppingToken)
{
await _kernel.RunAsync("Hello, I'm Sparkles, the Unicorn. Talk to me!", _kernel.Skills.GetFunction("WriteOutput"));
// prepare the pipeline
var pipeline = new ISKFunction[] {
_kernel.Skills.GetFunction("GetInput"),
_kernel.Skills.GetFunction("AskModel"),
_kernel.Skills.GetFunction("WriteOutput")
};
while (!stoppingToken.IsCancellationRequested)
{
await _kernel.RunAsync(pipeline);
}
}
}
Code language: C# (cs)Bringing it all together: Our main method
So this is where we will make this come alive. First, we build our application host object. We use the default builder from Microsoft.Extensions.Hosting, configure that to use the default console lifetime (which will handle Ctrl-C for us to end the process), and then configure the dependency injection container. The default host builder also provides us with the default configuration and logging infrastructure.
So the first thing I like to do is handle configuration. We use the .Configure method to bind a configuration section in our appsettings.json file to the ChatRequestSettings. This will enable us to change things like the temperature in between runs. We also add the initial system prompt to the configuration file. Let’s see that first:
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Warning"
}
},
"OpenAI": {
"ApiKey": "", // comes from user-secrets for this demo
"ChatModel": "gpt-3.5-turbo",
"SystemPrompt": "Your name is Sparkles. You are a unicorn and want to bring more glitter and love to the world. You are friendly and love rainbows. Try to be funny in your responses.",
"ChatRequestSettings" : {
// Randomness of the response, the higher the more "creative"
"Temperature": 0.9, // 0.0 up to 1.0
// Diversity of the response, use this OR temperature, but not both
"TopP": 0.0, // 0.0 up to 1.0
// Tokens already in the prompt can be penalized for new creations (the higher the more likely new topics are brought up)
"PresencePenalty": 0.0, // 0.0 up to 2.0
// Tokens already in the prompt can be penalized for new creations (the higher the less likely will the model repeat itself)
"FrequencyPenalty": 0.0, // -2.0 up to 2.0
// Maximum number of tokens to generate
"MaxTokens": 1500
}
}
}
Code language: JSON / JSON with Comments (json)Then we will add our three skills to the DI. I already mentioned that Semantic Kernel does not really fit in with .NET-isms and especially not with the DI, so we need a bit of plumbing for that to work. Since the IChatCompletion service class is not in the DI but can only be retrieved from an IKernel, and the kernel itself also isn’t automatically part of the DI, we need to do that. So we add a service for the IChatCompletion to the DI, and as a resolver function, we add a method that will retrieve an IKernel from the DI and then returns the IChatCompletion service from that kernel instance.
We then add an IKernel as a singleton with a resolver function too. In this resolver function, we use the KernelBuilder object from the Semantic Kernel SDK. On that builder, we add the OpenAIChatComplectionService and pass the model we want to use and the API key to use along, from our configuration. We then build the kernel, and now, since we have access to the IServiceProvider in our resolver function, we can resolve the console input and output skills and add them to the kernel.
We can’t resolve the chat service here, as this requires the IChatCompletion , and as we already saw this will need to resolve the IKernel and we are currently in the resolver function of that kernel, and this would recursively call us again. This is why we added this single skill in our background service.
Talking about that, we also need this and we need to use the .AddHostedService<ChatBotService>() method for that to be automatically resolved by the host.
The last thing to do is to start our .NET application host, and have a chat with our lovely Sparkles.
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.AI.ChatCompletion;
var host = Host.CreateDefaultBuilder(args)
.UseConsoleLifetime()
.ConfigureServices((ctx, services) => {
// Configuration settings
services.Configure<ChatRequestSettings>(opt => ctx.Configuration.GetSection("OpenAI:ChatRequestSettings").Bind(opt));
// services
services.AddSingleton<ConsoleInputSkill>();
services.AddSingleton<ConsoleOutputSkill>();
services.AddSingleton<OpenAIChatSkill>();
// don't want to use the kernel to retrieve the chat completion in the chat skill, so we add it here
services.AddSingleton<IChatCompletion>(sp => sp.GetRequiredService<IKernel>().GetService<IChatCompletion>());
services.AddSingleton<IKernel>(sp => {
var kernel = new KernelBuilder()
// Use OpenAI API for chat completion
.WithOpenAIChatCompletionService(
modelId: ctx.Configuration.GetValue<string>("OpenAI:ChatModel") ?? throw new InvalidOperationException("OpenAI:ChatModel is not configured."),
apiKey: ctx.Configuration.GetValue<string>("OpenAI:ApiKey") ?? throw new InvalidOperationException("OpenAI:ApiKey is not configured.")
)
.Build();
// can't add chat skill yet as chat skill needs Kernel
kernel.ImportSkill(sp.GetRequiredService<ConsoleInputSkill>());
kernel.ImportSkill(sp.GetRequiredService<ConsoleOutputSkill>());
return kernel;
});
services.AddHostedService<ChatBotService>();
})
.Build();
await host.RunAsync();
return 0;
Code language: C# (cs)Summary
In this post, we did some initial baby steps with Semantic Kernel. You can find the source code here.
We configured Semantic Kernel to use the OpenAI API, and added three skills to it. The first skill was reading a user’s input from the console, the second one calls the actual OpenAI chat model and keeps track of the history, and the third writes the models’ response back to the console.
In a background worker class in our .NET application, we then configured a pipeline of these three skills and told the kernel to execute that pipeline.
Based on the (configurable) system prompt, we now can chat with a lovely unicorn called Sparkles, which will try to enlighten our days.
