We Keep Coding

C# Microsoft Bot Framework with luis result directing to QNA Maker and graph api

I made a Bot using Microsoft bot framework and made use of Luis for matching intents. Some of the intents directs it to QNA and and some other intents directs it to graph api.
My Question is what is the best approach for identifying whether it should go to qna for searching the related intents in qna or whether it should go to graph api for fetching results.
As of now i did it using multiple Luis Intents for matching the correct intent and then redirect it according to the intent functionality needed(whether to direct it to qna dialog or graph api dialog).
`
[LuisModel("model id", "key")]
[Serializable]
public class RootDialog : DispatchDialog
{
//this intent directs to graph api dialog
[LuisIntent(DialogMatches.GraphApiIntent)]
public async Task RunGraphapiIntent(IDialogContext context, IActivity activity)
{
UserMessage = (activity as IMessageActivity).Text;
await context.Forward(new GraphApiDailog(), EndDialog, context.Activity, CancellationToken.None);
}
//This intent directs to qna dialog
[LuisIntent(DialogMatches.BlogMatch)]
public async Task RunBlogDialogQna(IDialogContext context, LuisResult result)
{
var userQuestion = (context.Activity as Activity).Text;
(context.Activity as Activity).Text = DialogMatches.BlogMatch;
await context.Forward(new BasicQnAMakerDialog(), this.EndDialog, context.Activity, CancellationToken.None);
}
`
But this approach requires me to match every intents using [LuisIntent("intentstring")].. Since i can have 50 or 100's of intent, this is not practical to write 50 functions for 50 intents.
I found out a way to call an api for fetching intent from utterances in Quickstart: Analyze text using C#
it makes use of "https://westus.api.cognitive.microsoft.com/luis/v2.0/apps/df67dcdb-c37d-46af-88e1-8b97951ca1c2?subscription-key=&q=turn on the bedroom light"
api for fetching intent.
Again My Question is how will i know whether i should redirect it to QnaDialog or Graph Api Dialog for fetching data using the intent result that i got?
Thanks in advance

If you want things to scale you will be better off by writing your own Nlp service that calls the Luis API to detect the intent. I think the best way to handle dialog redirection by intent is to make something like an IntentDetectorDialog whose only job is to analyze the user utterance and forwarding to the dialog that corresponds with the detected intent.
Here's a neat approach I've been using for a while:
public abstract class BaseDialog : IDialog<BaseResult>
{
public bool DialogForwarded { get; protected set; }
public async Task StartAsync(IDialogContext context)
{
context.Wait(OnMessageReceivedAsync);
}
public async Task OnMessageReceivedAsync(
IDialogContext context,
IAwaitable<IMessageActivity> result)
{
var message = await result;
var dialogFinished = await HandleMessageAsync(context, message);
if (DialogForwarded) return;
if (!dialogFinished)
{
context.Wait(OnMessageReceivedAsync);
}
else
{
context.Done(new DefaultDialogResult());
}
}
protected abstract Task<bool> HandleMessageAsync(IDialogContext context, IMessageActivity message);
protected async Task ForwardToDialog(IDialogContext context,
IMessageActivity message, BaseDialog dialog)
{
DialogForwarded = true;
await context.Forward(dialog, (dialogContext, result) =>
{
// Dialog resume callback
// this method gets called when the child dialog calls context.Done()
DialogForwarded = false;
return Task.CompletedTask;
}, message);
}
}
The base dialog, parent of all other dialogs, will handle the general flow of the dialog. If the dialog has not yet finished, it will notify the bot framework by calling context.Wait otherwise it will end the dialog with context.Done. It will also force all the child dialogs to implement the method HandleMessageAsync which returns a bool indicating whether the dialog has finished or not. And also exposes a reusable method ForwardToDialog that our IntentDetectorDialog will use to handle intent redirection.
public class IntentDetectorDialog : BaseDialog
{
private readonly INlpService _nlpService;
public IntentDetectorDialog(INlpService nlpService)
{
_nlpService = nlpService;
}
protected override async Task<bool> HandleMessageAsync(IDialogContext context, IMessageActivity message)
{
var intentName = await _nlpService.AnalyzeAsync(message.Text);
switch (intentName)
{
case "GoToQnaDialog":
await ForwardToDialog(context, message, new QnaDialog());
break;
case "GoToGraphDialog":
await ForwardToDialog(context, message, new GraphDialog());
break;
}
return false;
}
}
That is the IntentRedetectorDialog: son of BaseDialog whose only job is to detect the intent and forward to the corresponding dialog. To make things more scalable you could implement a IntentDialogFactory which can build dialogs based on the detected intent.
public class QnaDialog : BaseDialog
{
protected override async Task<bool> HandleMessageAsync(IDialogContext context, IMessageActivity message)
{
if (message.Text == "My name is Javier")
{
await context.PostAsync("What a cool name!");
// question was answered -> end the dialog
return true;
}
else
{
await context.PostAsync("What is your name?");
// wait for the user response
return false;
}
}
}
And finally we have our QnaDialog: also son of BaseDialog whose only job is to ask for the user's name and wait for the response.
Edit
Based on your comments, in your NlpService you can have:
public class NlpServiceDispatcher : INlpService
{
public async Task<NlpResult> AnalyzeAsync(string utterance)
{
var qnaResult = await _qnaMakerService.AnalyzeAsync(utterance);
var luisResult = await _luisService.AnalyzeAsync(utterance);
if (qnaResult.ConfidenceThreshold > luisResult.ConfidenceThreshold)
{
return qnaResult;
}
else
{
return luisResult;
}
}
}
Then change the IntentDetectorDialog to:
public class UtteranceAnalyzerDialog : BaseDialog
{
private readonly INlpService _nlpService;
public UtteranceAnalyzerDialog(INlpService nlpService)
{
_nlpService = nlpService;
}
protected override async Task<bool> HandleMessageAsync(IDialogContext context, IMessageActivity message)
{
var nlpResult = await _nlpService.AnalyzeAsync(message.Text);
switch (nlpResult)
{
case QnaMakerResult qnaResult:
await context.PostAsync(qnaResult.Answer);
return true;
case LuisResult luisResult:
var dialog = _dialogFactory.BuildDialogByIntentName(luisResult.IntentName);
await ForwardToDialog(context, message, dialog);
break;
}
return false;
}
}
And there you have it! You don't need to repeat utterances in Luis and QnaMaker, you can just use both and set your strategy based on the more confident result!

"Sorry my bot code is having issues" message on calling PromptDialog.Choice()

Code flow is as follows:
I have a DialogA which calls a qnaMaker Dialog in its StartAsync method.
The qnaMaker Dialog callback calls promptDialog function in a helper class(which is not a Dialog by the way)
Once user selects appropriate choice on the prompt, a callback is called which post a message
Error appears after the message("Helpful"/ "Not helpful") has been posted
My first incination was to use context.Wait() after the message has been posted from OnOptionSelectedForPrompt callback, bit it seems that calling wait here somehow executes the ResumeAfter callback for DialogA.
Code Sample: DialogA.cs
[Serializable]
public class DialogA : IDialog<object>
{
private string tool = "sometool";
private string userInput = String.Empty;
private bool sendToQnaMaker = false;
public DialogA(string userQuery)
{
if (userQuery.Length > 0)
{
sendToQnaMaker = true;
userInput = userQuery;
}
}
public async Task StartAsync(IDialogContext context)
{
if(sendToQnaMaker)
{
await context.Forward(new QnaDialog(), QnaMakerDialogCallback, BotHelper.CreateMessageActivityForQnaMaker(userInput), CancellationToken.None);
} else
{
await BotHelper.ShowWelcomeMessageOnMenuSection(context, tool);
context.Wait(OnMessageReceived);
}
}
private async Task QnaMakerDialogCallback(IDialogContext context, IAwaitable<object> result)
{
var response = await result;
bool isInQnaMaker = Convert.ToBoolean(response);
BotHelper botHelper = new BotHelper(tool);
botHelper.showPrompt(context)
}
BotHelper.cs
public void showPrompt(IDialogContext context)
{
PromptDialog.Choice(context, OnOptionSelectedForPrompt, Constants.promptChoices, MiscellaneousResponses.answerFeedbackMessage, MiscellaneousResponses.invalidOptionMessage, 3);
}
public async Task OnOptionSelectedForPrompt(IDialogContext context, IAwaitable<string> result)
{
string optionSelected = await result;
switch (optionSelected)
{
case MiscellaneousResponses.helpfulMessage:
string botreply = "Helpful";
await context.PostAsync(botreply);
break;
case MiscellaneousResponses.notHelpfulMessage:
string botResponse = "Not Helpful";
await context.PostAsync(botResponse);
break;
}
}
}

Created a separate dialog instead of using the BotHelper class and properly handled dialog stack using Context.Done and Context.Wait.

Real-time notification upon registration using SignalR

I want to send real-time notification in ASP.NET Boilerplate. Notification is saving successfully in Abp.NotificationSubscription table on subscription. When I publish the notification, the notification got saved in the Abp.Notification table but it is not displayed to the user in real-time.
My server-side code:
public async Task<RegisterOutput> Register(RegisterInput input)
{
public async Task<RegisterOutput> Register(RegisterInput input)
{
var user = await _userRegistrationManager.RegisterAsync(
input.Name,
input.Surname,
input.EmailAddress,
input.UserName,
input.Password,
true
);
_notificationSubscriptionManager.SubscribeToAllAvailableNotifications(user.ToUserIdentifier());
await _appNotifier.WelcomeToTheApplicationAsync(user);
var notification = _userNotificationManager.GetUserNotifications(user.ToUserIdentifier());
await _realTimeNotifier.SendNotificationsAsync(notification.ToArray());
// ...
}
}
_appNotifier implements WelcomeToTheApplicationAsync(user):
public async Task WelcomeToTheApplicationAsync(User user)
{
await _notificationPublisher.PublishAsync(
AppNotificationName.WelcomeToTheApplication,
new SendNotificationData("Naeem", "Hello I have sended this notification to you"),
severity: NotificationSeverity.Success,
userIds: new[] { user.ToUserIdentifier() }
);
}
SendNotificationData inherits from NotificationData:
public class SendNotificationData : NotificationData
{
public string name { get; set; }
public string message { get; set; }
public SendNotificationData(string _name, string _message)
{
name = _name;
message = _message;
}
}

I want to sent welcome notification to the user when he registers itself by using the register link on the login page. In the CloudbaseLine.Application project ... we have AccountAppService which contain the code for registering the user and sending notification to it after successful registration but the problem is notification go saved in Abp.NotificationSubscriptionTable and UserNotificationTable but user cannot received them.
public async Task<RegisterOutput> Register(RegisterInput input)
{
var user = await _userRegistrationManager.RegisterAsync(
input.Name,
input.Surname,
input.EmailAddress,
input.UserName,
input.Password,
true
);
_notificationSubscriptionManager.SubscribeToAllAvailableNotifications(user.ToUserIdentifier());
await _appNotifier.WelcomeToTheApplicationAsync(user);
var notification = _userNotificationManager.GetUserNotifications(user.ToUserIdentifier());
await _realTimeNotifier.SendNotificationsAsync(notification.ToArray());
// ...
}
public async Task WelcomeToTheApplicationAsync(User user)
{
await _notificationPublisher.PublishAsync(
AppNotificationName.WelcomeToTheApplication,
new SendNotificationData("Naeem", "Hello I have sended this notification to you"),
severity: NotificationSeverity.Success,
userIds: new[] { user.ToUserIdentifier() }
);
}
The user is not connected to the SignalR hub in the Register method.
One way to handle that is to enqueue a background job:
await _backgroundJobManager.EnqueueAsync<WelcomeNotificationJob, UserIdentifier>(
user.ToUserIdentifier(),
delay: TimeSpan.FromSeconds(5)
);
public class WelcomeNotificationJob : BackgroundJob<UserIdentifier>, ITransientDependency
{
private readonly IRealTimeNotifier _realTimeNotifier;
private readonly IUserNotificationManager _userNotificationManager;
public WelcomeNotificationJob(
IRealTimeNotifier realTimeNotifier,
IUserNotificationManager userNotificationManager)
{
_realTimeNotifier = realTimeNotifier;
_userNotificationManager = userNotificationManager;
}
[UnitOfWork]
public override void Execute(UserIdentifier args)
{
var notifications = _userNotificationManager.GetUserNotifications(args);
AsyncHelper.RunSync(() => _realTimeNotifier.SendNotificationsAsync(notifications.ToArray()));
}
}
Don't forget to register data formatters for custom notification data types, on the client side:
abp.notifications.messageFormatters['CloudBaseLine.Notification.SendNotificationData'] = function (userNotification) {
return userNotification.notification.data.message;
}

Reusing FormBuilder

My bot receives some user info like order no and few other details based on that it searches the orders. If more that 1 order is returned then it asks user to provide more info so asks for 2 additional details.
Now my problem is that as I have already build the form using FormBuilder and on completion I ask user to provide more info so that should be the approach to ask for additional info. Should I create another form and request info or is there any way to add the additional fields to the same form in the completion method.
I want to ask for more user inputs after initial search is complete and msg for more than 1 order found is displayed.
As my properties SubOrderNumber & SubOrderVersion are in OrderQuery class so Should i build new form for OrderQuery or is there anyother way to add these two to existing Order form.
using System;
using Microsoft.Bot.Builder.FormFlow;
namespace Bot
{
[Serializable]
public class OrderSearchQuery
{
[Prompt("Please enter {&}")]
public string OrderNumber { get; set; }
[Prompt("Please enter {&}?")]
public String Location { get; set; }
[Prompt("Please provide last status {&}?")]
public string Status{ get; set; }
[Prompt("Please enter {&}?")]
public string SubOrderNumber { get; set; }
[Prompt("Please enter {&}?")]
public string SubOrderVersion { get; set; }
}
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Web;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Builder.FormFlow;
using Microsoft.Bot.Connector;
namespace Bot.Dialogs
{
[Serializable]
public class OrderDialog : IDialog<object>
{
public async Task StartAsync(IDialogContext context)
{
await context.PostAsync("Welcome to the Order helper!");
var orderFormDialog = FormDialog.FromForm(this.BuildOrderForm, FormOptions.PromptInStart);
context.Call(orderFormDialog, this.ResumeAfterOrdersFormDialog);
}
private IForm<OrderSearchQuery> BuildOrderForm()
{
OnCompletionAsyncDelegate<OrderSearchQuery> processOrderSearch = async (context, state) =>
{
await context.PostAsync($"Ok. Searching for Orders with Number: {state.OrderNumber}...");
};
return new FormBuilder<OrderSearchQuery>()
.Field(nameof(OrderSearchQuery.OrderNumber))
.AddRemainingFields(new string[] { nameof(RequiredDWPSearchQuery.SubOrderNumber), nameof(RequiredDWPSearchQuery.SubOrderVersion) })
.OnCompletion(processOrderSearch)
.Build();
}
private async Task ResumeAfterOrdersFormDialog(IDialogContext context, IAwaitable<OrderSearchQuery> result)
{
try
{
var searchQuery = await result;
await context.PostAsync($"I found total of 100 Ordes");
await context.PostAsync($"To get Order details, you will need to provide more info...");
}
catch (FormCanceledException ex)
{
string reply;
if (ex.InnerException == null)
{
reply = "You have canceled the operation. Quitting from the Required DWP Search";
}
else
{
reply = $"Oops! Something went wrong :( Technical Details: {ex.InnerException.Message}";
}
await context.PostAsync(reply);
}
finally
{
context.Done<object>(null);
}
}
}
}

While there are conditional parameters for fields and the possibility of doing logic in validation, I don't believe that would be the cleanest way to go. I would take a two dialog approach.
Here's the original form, with additional parameters removed:
using Microsoft.Bot.Builder.FormFlow;
using System;
namespace TestChatbot.Dialogs
{
[Serializable]
public class OrderSearchQuery
{
[Prompt("Please enter {&}")]
public string OrderNumber { get; set; }
[Prompt("Please enter {&}?")]
public String Location { get; set; }
[Prompt("Please provide last status {&}?")]
public string Status { get; set; }
}
}
Instead, I moved the additional properties to a new form:
using Microsoft.Bot.Builder.FormFlow;
using System;
namespace TestChatbot.Dialogs
{
[Serializable]
public class OrderFollowUp
{
[Prompt("Please enter {&}?")]
public string SubOrderNumber { get; set; }
[Prompt("Please enter {&}?")]
public string SubOrderVersion { get; set; }
}
}
Then I added logic to handle the second form, depending on the results of the search:
using System;
using System.Threading.Tasks;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Builder.FormFlow;
namespace TestChatbot.Dialogs
{
[Serializable]
public class RootDialog : IDialog<object>
{
public async Task StartAsync(IDialogContext context)
{
await context.PostAsync("Welcome to the Order helper!");
var orderFormDialog = FormDialog.FromForm(this.BuildOrderForm, FormOptions.PromptInStart);
context.Call(orderFormDialog, this.ResumeAfterOrdersFormDialog);
}
private IForm<OrderSearchQuery> BuildOrderForm()
{
return new FormBuilder<OrderSearchQuery>()
.Build();
}
private IForm<OrderFollowUp> BuildFollowUpForm()
{
return new FormBuilder<OrderFollowUp>()
.Build();
}
private async Task ResumeAfterOrdersFormDialog(IDialogContext context, IAwaitable<OrderSearchQuery> result)
{
try
{
var searchQuery = await result;
await context.PostAsync($"Ok. Searching for Orders with Number: {searchQuery.OrderNumber}...");
await context.PostAsync($"I found total of 100 Ordes");
int searchResultCount = 2;
if (searchResultCount > 1)
{
await context.PostAsync($"To get Order details, you will need to provide more info...");
var followUpDialog = FormDialog.FromForm(this.BuildFollowUpForm, FormOptions.PromptInStart);
context.Call(followUpDialog, this.ResumeAfterFollowUpDialog);
}
}
catch (FormCanceledException ex)
{
string reply;
if (ex.InnerException == null)
{
reply = "You have canceled the operation. Quitting from the Required DWP Search";
}
else
{
reply = $"Oops! Something went wrong :( Technical Details: {ex.InnerException.Message}";
}
await context.PostAsync(reply);
}
}
private async Task ResumeAfterFollowUpDialog(IDialogContext context, IAwaitable<OrderFollowUp> result)
{
await context.PostAsync($"Order complete.");
context.Done<object>(null);
}
}
}
The difference here is that I removed the OnCompletion call and handled that logic on the resumption handler, after the first dialog completes. Based on the search result, you can call the second dialog, much like you did the first to extract additional information.

MS Bot with Multiple Form Dialog

I have 4 Dialogs in my Project the First one is a
RootDialog
with some enum.
This is my root dialog so If user select the first option then I want to send him to the
CreateContact
Dialog which I added below and if user selcts
GetContact
or
SendMail
then I will send him to the appropriate Dialog but I am unable to call anothe form dialog on completion or selection of first Dialog.
[Serializable]
public enum RootOptions
{
CreateContact = 1, GetContact = 2, SendMail
};
[Serializable]
public class RootForm
{
public RootOptions? Option;
public static IForm<RootForm> BuildForm()
{
return new FormBuilder<RootForm>()
.Message("Welcome to TEST BOT!")
.OnCompletion(async (context, rootoption) =>
{
switch(rootoption.Option.Value.ToString()) "")
{
case "CreateContact":
//How to call Contact Dialog
break;
case "GetContact":
//Call Get Contact Dialog
break;
case "SendMail":
//Call Send Mail Dialog
break;
}
})
.Build();
}
}
Create Contact Dialog
[Serializable]
public class ContactForm
{
[Prompt("What is the name? {||}")]
public string Name;
[Prompt("What is the Mobile No? {||}")]
public string Mobile;
[Prompt("What is the Email? {||}")]
public string Email;
public static IForm<ContactForm> BuildForm()
{
return new FormBuilder<ContactForm>()
.Message("Let's create a contact")
.OnCompletion(async (context, profileForm) =>
{
await context.PostAsync("Your contact has been created.");
})
.Build();
}
}
Send Mail Dialog
[Serializable]
public class MailForm
{
[Prompt("What is the Email Id? {||}")]
public string Email;
[Prompt("What is the Subject? {||}")]
public string subject;
[Prompt("What is the Message? {||}")]
public string Message;
public static IForm<MailForm> BuildForm()
{
return new FormBuilder<MailForm>()
.Message("Let's Send a Mail")
.OnCompletion(async (context, mailForm) =>
{
await context.PostAsync("Mail Sent.");
})
.Build();
}
}

To call a dialog, you need to use context.Call as explained in this post. However, I'm not fully sure if this will work in the OnCompletion event of a Form.
If it doesn't work, my recommendation would be to encapsulate the RootForm into a IDialog<object> dialog and use that dialog as the starting point for the Conversation.SendAsync of the controller.

You can use below line of code to call multiple dialogs:
await Task.Run(() => context.Call(new OptionDialog(),
this.OptionDialogResumeAfter)).ConfigureAwait(false);