Add QnA Pairs in QnA Maker Knowledge Base using the Bot in C#

This is Part 1 of the series “Auto Updater QnA Maker KB Bot“. Update your QnA Maker Knowledge using ChatBot. Add new question-answer pairs to your KB with the help of a bot interactive environment. You can also update and delete existing KB with this bot. In this part, we will be inserting the QnA pairs in the KB using the Bot.

QnA Maker is a cloud-based Natural Language Processing (NLP) service that easily creates a natural conversational layer over your data. It can be used to find the most appropriate answer for any given natural language input, from your custom knowledge base (KB) of information.

A client application for QnA Maker is any conversational application that communicates with a user in natural language to answer a question. Examples of client applications include social media apps, chatbots, and speech-enabled desktop applications.

For more information, refer Microsoft Documentation.

Pre-requisites

Visual Studio
Bot Emulator
Bot Framework SDK Templates
QnA Maker KB

You can get the links to the first 3 requirements on the Downloads page. If you have not created a Knowledge Base (KB) before, you can refer to my post on “Create QnA Maker Bot without writing Code using Azure Bot Service“.

Outcome

Creating a Core Bot project in Visual Studio

Kindly watch the below video to create a Core Bot project using the templates. Will be continuing from the created core bot project. I have used the Core Bot template with .NET Core 3.1 version.

Editing the Project Structure

Remove the following files and folder which are not required for our requirement –

BookingDetails.cs
FlightBookingRecognizer.cs
BookingDialog.cs
CancelAndHelpDialog.cs
DateResolverDialog.cs
CognitiveModels Folder

Remove the following lines of code from Startup.cs

// Register LUIS recognizer
services.AddSingleton<FlightBookingRecognizer>();

// Register the BookingDialog.
services.AddSingleton<BookingDialog>();

Remove all the method content from all methods in MainDialog.cs. Also, remove the imports which are showing error. Ignore the error for methods where return statements are missing. Below is what the MainDialog class will look like after the above removal.

public class MainDialog : ComponentDialog
    {
        protected readonly ILogger Logger;

        // Dependency injection uses this constructor to instantiate MainDialog
        public MainDialog(ILogger<MainDialog> logger)
            : base(nameof(MainDialog))
        {
            Logger = logger;

            AddDialog(new TextPrompt(nameof(TextPrompt)));
            AddDialog(new WaterfallDialog(nameof(WaterfallDialog), new WaterfallStep[]
            {
                IntroStepAsync,
                ActStepAsync,
                FinalStepAsync,
            }));

            // The initial child Dialog to run.
            InitialDialogId = nameof(WaterfallDialog);
        }

        private async Task<DialogTurnResult> IntroStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            
        }

        private async Task<DialogTurnResult> ActStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            
        }

        private async Task<DialogTurnResult> FinalStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            
        }
    }

Adding new class files and folders

I will be using the namespace “QnAUpdaterDemo” for all the posts in the series.

Create a new class files “User.cs” and “QnAData.cs” under QnAUpdaterDemo namespace.
Create a new folder “Utilities” under the QnAUpdaterDemo namespace.
Create a new class file “QnAClient.cs” under QnAUpdaterDemo.Utilities namespace.
Create a new folder “Operations” under QnAUpdaterDemo.Dialogs namespace.
Create a new class files “AddDialog.cs” and “QuestionPhraseDialog.cs” under QnAUpdaterDemo.Dialogs.Operations namespace.

We have now created the structure we will use for this demo. The end structure will look like this –

Creating the Bot with Add Operation

Open the file welcomeCard.json under QnAUpdaterDemo.Cards namespace and replace the content with the following to create a new Adaptive card that will show up when the user joins the conversation:

{
  "type": "AdaptiveCard",
  "body": [
    {
      "type": "Container",
      "items": [
        {
          "type": "TextBlock",
          "text": "Auto Updater QnA Maker KB Bot",
          "size": "ExtraLarge",
          "wrap": true,
          "weight": "Bolder",
          "color": "Dark"
        },
        {
          "type": "TextBlock",
          "text": "Update your QnA Maker Knowledge using the ChatBot. Add new question answer pairs to your KB with the help of bot interactive environment. You can also update and delete existing KB with this bot.",
          "wrap": true
        }
      ],
      "style": "good",
      "bleed": true
    }
  ],
  "actions": [
    {
      "type": "Action.OpenUrl",
      "title": "Go to QnA Maker Portal",
      "url": "https://www.qnamaker.ai/"
    },
    {
      "type": "Action.OpenUrl",
      "title": "Read Microsoft Documentation",
      "url": "https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/overview/overview"
    }
  ],
  "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
  "version": "1.2"
}

If you want to learn creating attractive adaptive cards for your bot, refer my post on “Create Adaptive Cards“. Open appsettings.json and add the configuration details.

{
  "MicrosoftAppId": "",
  "MicrosoftAppPassword": "",
  "KnowledgeBaseId": "<KB ID>",
  "Host": "https://<Resource Name>.azurewebsites.net/qnamaker",
  "EndpointKey": "<Endpoint Key>",
  "ResourceName": "<Resource Name>",
  "Key": "<Subscription Key>"
}

For testing the bot locally, MicrosoftAppId and Password are not required. You can get the Host, Endpoint Key, and KB Id from the publishing page of QnA Maker KB.

You can get the subscription Key from your Azure cognitive service resource. You can take any one key out of two.

Replace the code in MainDialog.cs with the below code.

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.
//
// Generated with Bot Builder V4 SDK Template for Visual Studio CoreBot v4.9.2

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using AdaptiveCards;
using Microsoft.Bot.Builder;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Builder.Dialogs.Choices;
using Microsoft.Bot.Schema;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json.Linq;
using QnAUpdaterDemo.Dialogs.Operations;

namespace QnAUpdaterDemo.Dialogs
{
    public class MainDialog : ComponentDialog
    {
        protected readonly ILogger Logger;
        protected readonly IConfiguration Configuration;

        // Dependency injection uses this constructor to instantiate MainDialog
        public MainDialog(ILogger<MainDialog> logger, IConfiguration configuration)
            : base(nameof(MainDialog))
        {
            Logger = logger;
            Configuration = configuration;
            AddDialog(new TextPrompt(nameof(TextPrompt)));
            AddDialog(new ChoicePrompt(nameof(ChoicePrompt)));
            AddDialog(new ConfirmPrompt(nameof(ConfirmPrompt)));
            AddDialog(new AddDialog(configuration));
            AddDialog(new WaterfallDialog(nameof(WaterfallDialog), new WaterfallStep[]
            {
                IntroStepAsync,
                ActStepAsync,
                FinalStepAsync,
            }));

            // The initial child Dialog to run.
            InitialDialogId = nameof(WaterfallDialog);
        }

        private async Task<DialogTurnResult> IntroStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            await stepContext.Context.SendActivityAsync(
                MessageFactory.Text("What operation you would like to perform?"), cancellationToken);

            List<string> operationList = new List<string> { "Add", "Update", "Delete" };
            // Create card
            var card = new AdaptiveCard(new AdaptiveSchemaVersion(1, 0))
            {
                // Use LINQ to turn the choices into submit actions
                Actions = operationList.Select(choice => new AdaptiveSubmitAction
                {
                    Title = choice,
                    Data = choice,  // This will be a string
                }).ToList<AdaptiveAction>(),
            };
            // Prompt
            return await stepContext.PromptAsync(nameof(ChoicePrompt), new PromptOptions
            {
                Prompt = (Activity)MessageFactory.Attachment(new Attachment
                {
                    ContentType = AdaptiveCard.ContentType,
                    // Convert the AdaptiveCard to a JObject
                    Content = JObject.FromObject(card),
                }),
                Choices = ChoiceFactory.ToChoices(operationList),
                // Don't render the choices outside the card
                Style = ListStyle.None,
            },
                cancellationToken);
        }

        private async Task<DialogTurnResult> ActStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            stepContext.Values["Operation"] = ((FoundChoice)stepContext.Result).Value;
            string operation = (string)stepContext.Values["Operation"];

            if (operation.Equals("Add"))
            {
                await stepContext.Context.SendActivityAsync(MessageFactory.Text("You selected Add Operation. Please provide following details to add a new question answer pair."), cancellationToken);
                return await stepContext.BeginDialogAsync(nameof(AddDialog), new User(), cancellationToken);
            }
            else if (operation.Equals("Update"))
            {
                await stepContext.Context.SendActivityAsync(MessageFactory.Text("You selected Update Operation. Please provide following details to update the existing question answer pair."), cancellationToken);
            }
            else if (operation.Equals("Delete"))
            {
                await stepContext.Context.SendActivityAsync(MessageFactory.Text("You selected Delete Operation. Please provide following details to delete the existing question answer pair."), cancellationToken); 
            }
            else
            {

            }
            return await stepContext.NextAsync(null, cancellationToken);
        }

        private async Task<DialogTurnResult> FinalStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            QnAData.QuestionPhrase.Clear();
            var promptMessage = "What else can I do for you?";
            return await stepContext.ReplaceDialogAsync(InitialDialogId, promptMessage, cancellationToken);
        }
    }
}

Install the nuget packages “AdaptiveCards” and “Microsoft.Azure.CognitiveServices.Knowledge.QnAMaker”.

Add the below code in AddDialog.cs.

using Microsoft.Azure.CognitiveServices.Knowledge.QnAMaker;
using Microsoft.Bot.Builder;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Extensions.Configuration;
using QnAUpdaterDemo.Utilities;
using System.Threading;
using System.Threading.Tasks;

namespace QnAUpdaterDemo.Dialogs.Operations
{
    public class AddDialog : ComponentDialog
    {
        protected readonly IConfiguration Configuration;
        public AddDialog(IConfiguration configuration) : base(nameof(AddDialog))
        {
            Configuration = configuration;
            var waterfallSteps = new WaterfallStep[]
            {
                QuestionStepAsync,
                QuestionPhraseStepAsync,
                ActStepAsync,
                AnswerStepAsync,
                ConfirmStepAsync,
                SummaryStepAsync,
            };

            AddDialog(new WaterfallDialog(nameof(WaterfallDialog), waterfallSteps));
            AddDialog(new TextPrompt(nameof(TextPrompt)));
            AddDialog(new NumberPrompt<int>(nameof(NumberPrompt<int>)));
            AddDialog(new ChoicePrompt(nameof(ChoicePrompt)));
            AddDialog(new ConfirmPrompt(nameof(ConfirmPrompt)));
            AddDialog(new QuestionPhraseDialog());

            InitialDialogId = nameof(WaterfallDialog);
        }



        private async Task<DialogTurnResult> QuestionStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            return await stepContext.PromptAsync(nameof(TextPrompt), new PromptOptions
            {
                Prompt = MessageFactory.Text("Please enter the question.")
            }, cancellationToken);
        }
        private async Task<DialogTurnResult> QuestionPhraseStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            stepContext.Values["Question"] = (string)stepContext.Result;
            QnAData.QuestionPhrase.Add((string)stepContext.Values["Question"]);
            return await stepContext.PromptAsync(nameof(ConfirmPrompt), new PromptOptions
            {
                Prompt = MessageFactory.Text("Would you like to Add more phrases for your question?")
            }, cancellationToken);
        }

        private async Task<DialogTurnResult> ActStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            if ((bool)stepContext.Result)
            {
                return await stepContext.BeginDialogAsync(nameof(QuestionPhraseDialog), new User(), cancellationToken);
            }
            else
            {
                return await stepContext.NextAsync(stepContext, cancellationToken);
            }
        }

        private async Task<DialogTurnResult> AnswerStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {

            return await stepContext.PromptAsync(nameof(TextPrompt), new PromptOptions
            {
                Prompt = MessageFactory.Text("Please enter the answer.")
            }, cancellationToken);
        }

        private async Task<DialogTurnResult> ConfirmStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            stepContext.Values["Answer"] = (string)stepContext.Result;

            return await stepContext.PromptAsync(nameof(ConfirmPrompt), new PromptOptions
            {
                Prompt = MessageFactory.Text("Would you like to Confirm?")
            }, cancellationToken);
        }

        private async Task<DialogTurnResult> SummaryStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            if ((bool)stepContext.Result)
            {
                await stepContext.Context.SendActivityAsync(MessageFactory.Text("Here are the details you provided."), cancellationToken);

                await stepContext.Context.SendActivityAsync(MessageFactory.Text("Questions - "), cancellationToken);

                for (int i = 0; i < QnAData.QuestionPhrase.Count; i++)
                {
                    await stepContext.Context.SendActivityAsync(MessageFactory.Text(QnAData.QuestionPhrase[i]), cancellationToken);
                }


                await stepContext.Context.SendActivityAsync(MessageFactory.Text("Answer - " + (string)stepContext.Values["Answer"]), cancellationToken);

                await stepContext.Context.SendActivityAsync(MessageFactory.Text("Please wait while I update your Knowledge Base."), cancellationToken);

                var authoringURL = $"https://{Configuration["ResourceName"]}.cognitiveservices.azure.com";

                // <AuthorizationAuthor>
                var client = new QnAMakerClient(new ApiKeyServiceClientCredentials(Configuration["Key"]))
                { Endpoint = authoringURL };
                // </AuthorizationAuthor>

                QnAClient.UpdateKB(client, Configuration["KnowledgeBaseId"], (string)stepContext.Values["Answer"]).Wait();
                QnAClient.PublishKb(client, Configuration["KnowledgeBaseId"]).Wait();

                await stepContext.Context.SendActivityAsync(MessageFactory.Text("I have added your qna pair in the Knowledge Base. Thank you for using QnA Updator Bot Service."));

                return await stepContext.EndDialogAsync(null, cancellationToken);
            }
            else
            {
                await stepContext.Context.SendActivityAsync(MessageFactory.Text("Request Not Confirmed."));
                return await stepContext.EndDialogAsync(cancellationToken: cancellationToken);
            }
        }
    }
}

Add the below code in QuestionPhraseDialog.cs.

using Microsoft.Bot.Builder;
using Microsoft.Bot.Builder.Dialogs;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

namespace QnAUpdaterDemo.Dialogs.Operations
{
    public class QuestionPhraseDialog : ComponentDialog
    {
        public QuestionPhraseDialog() : base(nameof(QuestionPhraseDialog))
        {
            var waterfallSteps = new WaterfallStep[]
            {
                QuestionStepAsync,
                ConfirmStepAsync,
                SummaryStepAsync,
            };

            AddDialog(new WaterfallDialog(nameof(WaterfallDialog), waterfallSteps));
            AddDialog(new TextPrompt(nameof(TextPrompt)));
            AddDialog(new NumberPrompt<int>(nameof(NumberPrompt<int>)));
            AddDialog(new ChoicePrompt(nameof(ChoicePrompt)));
            AddDialog(new ConfirmPrompt(nameof(ConfirmPrompt)));

            InitialDialogId = nameof(WaterfallDialog);
        }

        private async Task<DialogTurnResult> QuestionStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            return await stepContext.PromptAsync(nameof(TextPrompt), new PromptOptions
            {
                Prompt = MessageFactory.Text("Please enter the question phrase.")
            }, cancellationToken);
        }

        private async Task<DialogTurnResult> ConfirmStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            stepContext.Values["Phrase"] = (string)stepContext.Result;
            QnAData.QuestionPhrase.Add((string)stepContext.Values["Phrase"]);
            return await stepContext.PromptAsync(nameof(ConfirmPrompt), new PromptOptions
            {
                Prompt = MessageFactory.Text("Would you like to Add more phrases for your question?")
            }, cancellationToken);
        }

        private async Task<DialogTurnResult> SummaryStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
        {
            if ((bool)stepContext.Result)
            {
                var promptMessage = "What else can I do for you?";
                return await stepContext.ReplaceDialogAsync(InitialDialogId, promptMessage, cancellationToken);
            }
            else
            {
                await stepContext.Context.SendActivityAsync(MessageFactory.Text("Ok."));
                return await stepContext.EndDialogAsync(cancellationToken: cancellationToken);
            }
        }
    }
}

Add the below code in QnAClient.cs

using Microsoft.Azure.CognitiveServices.Knowledge.QnAMaker;
using Microsoft.Azure.CognitiveServices.Knowledge.QnAMaker.Models;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace QnAUpdaterDemo.Utilities
{
    public static class QnAClient
    {

        // <UpdateKBMethod>
        public static async Task UpdateKB(IQnAMakerClient client, string kbId, string answer)
        {

            var updateOp = await client.Knowledgebase.UpdateAsync(kbId, new UpdateKbOperationDTO
            {
                // Create JSON of changes
                Add = new UpdateKbOperationDTOAdd
                {
                    QnaList = new List<QnADTO> {
                        new QnADTO {
                            Questions = QnAData.QuestionPhrase,
                            Answer = answer,
                        }
                    },
                },
                Update = null,
                Delete = null
            }); ;

            // Loop while operation is success
            updateOp = await MonitorOperation(client, updateOp);
        }
        // </UpdateKBMethod>

        // <PublishKB>
        public static async Task PublishKb(IQnAMakerClient client, string kbId)
        {
            await client.Knowledgebase.PublishAsync(kbId);
        }
        // </PublishKB>

        // <MonitorOperation>
        private static async Task<Operation> MonitorOperation(IQnAMakerClient client, Operation operation)
        {
            // Loop while operation is success
            for (int i = 0;
                i < 20 && (operation.OperationState == OperationStateType.NotStarted || operation.OperationState == OperationStateType.Running);
                i++)
            {
                Console.WriteLine("Waiting for operation: {0} to complete.", operation.OperationId);
                await Task.Delay(5000);
                operation = await client.Operations.GetDetailsAsync(operation.OperationId);
            }

            if (operation.OperationState != OperationStateType.Succeeded)
            {
                throw new Exception($"Operation {operation.OperationId} failed to completed.");
            }
            return operation;
        }
        // </MonitorOperation>
    }
}

Add the below code in QnAData.cs.

using System.Collections.Generic;

namespace QnAUpdaterDemo
{
    public static class QnAData
    {
        public static List<string> QuestionPhrase = new List<string>();
    }
}

Add the following service in Startup.cs.

services.AddSingleton<ICredentialProvider, ConfigurationCredentialProvider>();

Import the below libraries in Startup.cs.

using Microsoft.Bot.Builder.BotFramework;
using Microsoft.Bot.Connector.Authentication;

Run the project using Cntrl+F5. Open your bot emulator and give the bot URL “http://localhost:3978/api/messages” to connect. Check the port number on the webpage opened on running the project.

Below is the output you will get when user joins the conversation.

Start updating your KB by following the conversation. In the next part, we will delete the pairs from KB. Get the source code for this demo from the Downloads Page or GitHub.