We Keep Coding

Botframework v4 resume conversation outside of the waterfall dialog flow

Botframework V4 webchat
My bot displays a SigninCard to the user, which when clicked user is redirected to an external website where the user will enter login credentials. This external website will perform a service call to a separate endpoint in my bot controller with a token. Now I want to make the bot to display a message to the user, and let the user follow through the rest of the conversation.
For all intents and purposes, it needs to be as same as displaying a proactive message to the user, and jumping to the next waterfall step afterwards.
This is my code so far.
1st approach
Conversation reference information is passed from here.
private async Task<DialogTurnResult> SigninStepStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
var convRef = stepContext.Context.Activity.GetConversationReference();
var stateCol = System.Web.HttpUtility.ParseQueryString(string.Empty);
stateCol["userId"] = convRef.User.Id;
stateCol["botId"] = convRef.Bot.Id;
stateCol["conversationId"] = convRef.Conversation.Id;
stateCol["serviceUrl"] = convRef.ServiceUrl;
stateCol["channelId"] = convRef.ChannelId;
SigninCard signinCard = new SigninCard
{
Text = "Please sign-in to continue",
Buttons = new List<CardAction> {
new CardAction(ActionTypes.Signin, "Sign-in",
value: "http://some-external-url?state="+ System.Web.HttpUtility.UrlEncode(stateCol.ToString())) },
};
var reply = stepContext.Context.Activity.CreateReply();
reply.Attachments = new List<Attachment>
{
signinCard.ToAttachment(),
};
return await stepContext.PromptAsync(
"testprompt",
new PromptOptions
{
Prompt = reply,
},
cancellationToken);
}
Conversation reference information is received here. This endpoint is invoked by an external website.
[HttpGet]
public async Task<HttpResponseMessage> Callback(string state)
{
// Resume conversation
var stateCol = System.Web.HttpUtility.ParseQueryString(state);
ConnectorClient connector = new ConnectorClient(new Uri(stateCol["serviceUrl"]));
IMessageActivity newMessage = Activity.CreateMessageActivity();
newMessage.From = new ChannelAccount(stateCol["userId"]);
newMessage.Conversation = new ConversationAccount(id: stateCol["conversationId"]);
newMessage.Recipient = new ChannelAccount(stateCol["botId"]);
newMessage.ChannelId = stateCol["channelId"];
newMessage.Text = "hello user";
newMessage.ReplyToId = stateCol["botId"];
await connector.Conversations.SendToConversationAsync((Activity)newMessage);
//...
}
This piece of code correctly posts "Hello user" message to conversation flow, as a message coming from the bot. I tried connector.Conversations.ReplyTo method, also tried swapping botid and userid values to make it the user replying to the bot. But none of the cases made the waterfall step to jump to the next step in the flow like manually entering a reply text through emulator.
I followed the Idea 2 scenario of this question : https://stackoverflow.com/a/48977832 which is on botfraemwork v3. In botframework v3, it has a resume method as indicated in this answer. But I cannot find this method or anything that makes the conversation to resume, in botframework v4.
Conversation.ResumeAsync(reference, message);
Wondering if there is any special kind of activity that I can send, end the current prompt and make it start the next waterfall step.
2nd approach
On the side, I also tried to follow the ProactiveBot sample code in Botframework 4, to see if that approach will cause the waterfall flow to jump to the next step in the line.
[HttpGet]
public async Task<HttpResponseMessage> Callback(string code, string state, string session_state = null)
{
// Resume conversation
var stateCol = System.Web.HttpUtility.ParseQueryString(state);
MicrosoftAppCredentials.TrustServiceUrl(stateCol["serviceUrl"]);
ConversationReference cr = new ConversationReference();
cr.Bot = new ChannelAccount(stateCol["botId"]);
cr.User = new ChannelAccount(stateCol["userId"]);
cr.ChannelId = stateCol["channelId"];
cr.Conversation = new ConversationAccount(id: stateCol["conversationId"]);
cr.ServiceUrl = stateCol["serviceUrl"];
var msg = cr.GetContinuationActivity();
msg.Text = "login_succeed";
await ((BotAdapter)_adapter).ContinueConversationAsync(_appId, cr, BotCallback, default(CancellationToken));
//..
}
But at ContinueConversationAsync it threw below error, even though I am not using any clientSecret parameter anywhere. Not sure if this is because of using the Signin card to invoke the external web site url in the first code block above.
An unhandled exception occurred while processing the request.
ArgumentNullException: Value cannot be null. (Parameter 'clientSecret')
Microsoft.IdentityModel.Clients.ActiveDirectory.ClientCredential..ctor(string clientId, string clientSecret)

Microsoft Botframework Send Proactive Message to Bot (and not to the User)

we are currently developing some automation with the botframework.
At some point in the conversation, we sent some data through a service bus for processing and wait for a response and then want to continue with the conversation. We already implemented the part where we wait for an response entry in the service bus subscription and then we want to send an Activity from type Event to the bot.
We did the same steps with the proactive message as described in other posts.
We are able to recreate the botclient and conversation reference and all, but in the end when we send the activity, we always send it to the user and not to the bot. But this doesn't trigger the "EventActivityPrompt".
The only way where we achieved the desired outcome was when we made a post to api/messages, but this is too complicated for our taste, and we are looking for an easier way over the botClient (or similar technology)
Has anyone some good ideas? :)
ServiceBusReceiver Message Processing:
private static async Task ProcessMessagesAsync(Message message, CancellationToken token)
{
// Process the message.
Console.WriteLine($"Received message: SequenceNumber:{message.SystemProperties.SequenceNumber} Body:{Encoding.UTF8.GetString(message.Body)}");
_logger?.LogInformation("Received message '{id}' with label '{label}' from queue.", message.MessageId, message.Label);
var data = JsonSerializer.Deserialize<BotCarLicensingOrderRpaRequest>(message.Body);
data.AdditionalData.TryGetValue("ServiceUrl", out var serviceUrl);
data.AdditionalData.TryGetValue("ChannelId", out var channelId);
data.AdditionalData.TryGetValue("BotId", out var botId);
data.AdditionalData.TryGetValue("UserId", out var userId);
data.AdditionalData.TryGetValue("ReplyToId", out var replyToId);
var conversationReference = _offTurnConversationService.CreateSyntheticConversationReference(
channelId?.ToString(),
data.ConversationId,
serviceUrl?.ToString());
conversationReference.User = new ChannelAccount()
{
Id = userId?.ToString(),
Role = "user"
};
conversationReference.Bot = new ChannelAccount
{
Id = botId?.ToString(),
Role = "bot"
};
var activity = (Activity)Activity.CreateEventActivity();
activity.Text = "success";
activity.ChannelId = channelId?.ToString();
activity.ServiceUrl = serviceUrl?.ToString();
activity.RelatesTo = conversationReference;
activity.Conversation = new ConversationAccount
{
Id = data.ConversationId
};
activity.ReplyToId = replyToId?.ToString();
activity.ApplyConversationReference(conversationReference, true);
// Complete the message so that it is not received again.
// This can be done only if the subscriptionClient is created in ReceiveMode.PeekLock mode (which is the default).
await _messageReceiver.CompleteAsync(message.SystemProperties.LockToken);
// This "works" but is complicated, as we have to set up a whole HTTP call
await _offTurnConversationService.SendActivityToBotAsync(activity);
// This just sends the Event to the user, no matter how I set up the conversation
// reference regarding From/Recipient
// And it doesn't help in continuing the conversation
await _offTurnConversationService.SendToConversationThroughPipelineAsync(
async (turnContext, cancellationToken) =>
{
await turnContext.SendActivityAsync(activity, cancellationToken: cancellationToken);
},
conversationReference);
// Note: Use the cancellationToken passed as necessary to determine if the subscriptionClient has already been closed.
// If subscriptionClient has already been closed, you can choose to not call CompleteAsync() or AbandonAsync() etc.
// to avoid unnecessary exceptions.
}
OffTurnConversationService:
public ConversationReference CreateSyntheticConversationReference(string channelId, string conversationId, string serviceUrl)
{
ArgumentGuard.NotNull(channelId, nameof(channelId));
ArgumentGuard.NotNull(conversationId, nameof(conversationId));
ArgumentGuard.NotNull(serviceUrl, nameof(serviceUrl));
if (string.IsNullOrEmpty(_botOptions.CurrentValue.BotId))
{
throw new InvalidOperationException("A valid bot id must be configured in your bot options in order to create a synthetic conversation reference.");
}
// WARNING: This implementation works for directline and webchat.
// Changes could be necessary for other channels.
var supportedChannels = new List<string>()
{
Channels.Directline,
Channels.Webchat
};
if (supportedChannels.Any(c => c.Equals(channelId, StringComparison.OrdinalIgnoreCase)))
{
_logger.LogWarning(
"The synthetic conversation reference created for channel {UsedChannel} might not work properly, " +
"because it's not supported and tested. Supported channels are {SupportedChannel}.",
channelId,
string.Join(",", supportedChannels));
}
var conversationReference = new ConversationReference()
{
Conversation = new ConversationAccount()
{
Id = conversationId
},
Bot = new ChannelAccount()
{
Id = _botOptions.CurrentValue.BotId,
Name = _botOptions.CurrentValue.BotId
},
ChannelId = channelId,
ServiceUrl = serviceUrl
};
return conversationReference;
}
public virtual async Task SendActivityToBotAsync(IActivity activity)
{
// Create the new request to POST to the client
var forwardRequest = new HttpRequestMessage()
{
RequestUri = new Uri(_botOptions.CurrentValue.ReplyServiceUrl),
Method = HttpMethod.Post,
};
// Change the host for the request to be the forwarding URL.
forwardRequest.Headers.Host = forwardRequest.RequestUri.Host;
// If the child bot is not running on local mode (no app-id/password),
// we're going send an authentication header.
OAuthResponse authToken = await GetTokenAsync(_botOptions.CurrentValue.MicrosoftAppId, _botOptions.CurrentValue.MicrosoftAppPassword);
forwardRequest.Headers.Authorization = new AuthenticationHeaderValue("Bearer", authToken.AccessToken);
// Altered activity to JSON content
var json = JsonConvert.SerializeObject(activity);
var content = new StringContent(json, Encoding.UTF8, "application/json");
forwardRequest.Content = content;
using var client = new HttpClient();
var response = await client.SendAsync(forwardRequest);
if (!response.IsSuccessStatusCode)
{
string message = $"Failed to send activity '{activity.Id}' to client bot. {response.ReasonPhrase}";
throw new Exception(message);
}
}
public virtual async Task SendToConversationThroughPipelineAsync(
BotCallbackHandler callback,
ConversationReference conversationReference)
{
ArgumentGuard.NotNull(callback, nameof(callback));
ArgumentGuard.NotNull(conversationReference, nameof(conversationReference));
// Avoiding 401 "Unauthorized" errors
TrustServiceUrl(conversationReference.ServiceUrl);
// Reuse adapter with its pipeline to send responses back to the user (like pro-active messages)
await ((BotAdapter)_botFrameworkHttpAdapter).ContinueConversationAsync(
_botOptions.CurrentValue.MicrosoftAppId,
conversationReference,
callback,
default);
}

Sending message to a group chat using bot framework from teams

I want to proactively send a message to a group that the bot was added to (not a team, a simple group chat)
What I'm doing so far is saving (in memory) the conversation Id in a singleton dictionary
and then issue the notification to all conversations.
This is the API endpoint code..
public CINotificationsController(
IBotFrameworkHttpAdapter adapter,
IBot bot,
IConfiguration config,
ConcurrentDictionary<string, ConversationReference> conversationReferences,
ILogger<CINotificationsController> logger)
{
_Adapter = adapter;
_Bot = bot;
_ConversationReferences = conversationReferences;
_Logger = logger;
_AppId = config.GetSection("MicrosoftAppId").Value;
}
[HttpPost]
public async Task<IActionResult> PostAsync(CINotification notificationData)
{
_Logger.LogInformation($"Got CI notification, {JsonConvert.SerializeObject(notificationData)}");
var jobName = notificationData.JobName;
var culpritsEmails = notificationData.Commiter;
foreach (var conv in _ConversationReferences.Values)
{
await ((BotAdapter)_Adapter).ContinueConversationAsync(_AppId, conv, GetBotCallBack(notificationData, conv), default);
}
return Ok(culpritsEmails);
}
private BotCallbackHandler GetBotCallBack(CINotification notificationData, ConversationReference conv)
{
return async (ctx, cts) =>
{
_Logger.LogDebug($"conversationId:[{conv.ActivityId}], serviceUrl:[{conv.ServiceUrl}]");
var mention = new Mention();
if (!string.IsNullOrEmpty(notificationData.Commiter))
{
var membersByEmail = (await TeamsInfo.GetMembersAsync(ctx, cts)).ToDictionary(k => k.Email, v => v, StringComparer.OrdinalIgnoreCase);
_Logger.LogDebug($"members:[{string.Join(",", membersByEmail.Keys)}]");
if (membersByEmail.TryGetValue(notificationData.Commiter, out var teamMemeber))
{
mention.Mentioned = teamMemeber;
mention.Text = $"<at>{teamMemeber.Name}</at>";
}
_Logger.LogDebug($"got mentions: {mention.Text}");
}
var msgText = $"{mention.Text} {notificationData.Message}";
_Logger.LogDebug($"Sending message text: {msgText}");
var replyActivity = MessageFactory.Text(msgText);
replyActivity.Entities.Add(mention);
await ctx.SendActivityAsync(replyActivity);
};
}
Of course an in memory dictionary _ConversationReferences is a bad idea and I am looking for some way (API, something form the SDK ) to be able to list all groups the bot was added to or all conversation that were started ..
Do I have to store it in some DB or do i have a different option ?

I don't think such an API call exists. You can get from the Graph API, for instance, a list of all apps installed into a team, but there's no such thing for group chats or 1-1 chats.
However, you -have- all that information already - simply store it your side, with so many good storage options these days (cloud and otherwise). In that case, it's fully under your control anyway.

You can do it with this code
// Create or get existing chat conversation with user
var response = client.Conversations.CreateOrGetDirectConversation(activity.Recipient, activity.From, activity.GetTenantId());
// Construct the message to post to conversation
Activity newActivity = new Activity()
{
Text = "Hello",
Type = ActivityTypes.Message,
Conversation = new ConversationAccount
{
Id = response.Id
},
};
// Post the message to chat conversation with user
await client.Conversations.SendToConversationAsync(newActivity, response.Id);
Got it here
https://learn.microsoft.com/en-us/microsoftteams/platform/bots/how-to/conversations/send-proactive-messages?tabs=dotnet

How to start new Dialog to NEW user? (Microsoft Bot Builder - Proactive Messages)

I'm using the official example https://github.com/Microsoft/BotBuilder-Samples/tree/master/CSharp/core-proactiveMessages
There is 3 separate examples: one for sending a simple message and two others for starting new dialogs.
In the first example they create new activity then define: receiver, sender, message's text and so on. But in both the 2nd and the 3rd examples they define nothing. They just use data from an original incoming activity, which is in fact just a delayed in time reply to original message. That's even NOT a true proactive message. But how to send a true proactive message from child dialog?
So I need something like mixed one from these examples.
My ConversationStarter:
public class ConversationStarter
{
public static async Task Resume()
{
string ServiceUrl = "http://localhost:50891";
ChannelAccount BotAccount = new ChannelAccount(
name: "Bot",
id: "default-bot"
);
ChannelAccount UserAccount = new ChannelAccount(
name: "User",
id: "default-user"
);
ConnectorClient ForwardClient = new ConnectorClient(new Uri(ServiceUrl));
ConversationResourceResponse ForwardConversation = ForwardClient.Conversations.CreateDirectConversation(BotAccount, UserAccount);
IMessageActivity ForwardMessage = Activity.CreateMessageActivity();
ForwardMessage.From = BotAccount;
ForwardMessage.Recipient = UserAccount;
ForwardMessage.Conversation = new ConversationAccount(id: ForwardConversation.Id);
using (var scope = DialogModule.BeginLifetimeScope(Conversation.Container, ForwardMessage))
{
var botData = scope.Resolve<IBotData>();
await botData.LoadAsync(CancellationToken.None);
var task = scope.Resolve<IDialogTask>();
//interrupt the stack
var dialog =new ForwardDialog();
task.Call(dialog.Void<object, IMessageActivity>(), null);
await task.PollAsync(CancellationToken.None);
//flush dialog stack
await botData.FlushAsync(CancellationToken.None);
}
}
}
My dialog:
public class ForwardDialog : IDialog<object>
{
public async Task StartAsync(IDialogContext context)
{
await context.PostAsync("Hello, I'm ForwardDialog. I'm interrupting your conversation.");
context.Wait(this.MessageReceivedAsync);
}
public virtual async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> result)
{
await context.PostAsync("Great, back to the original conversation!");
context.Done(String.Empty); //Finish this dialog
}
}
context.PostAsync() here returns an error: "ReplyToId" cannot be null
But it is NOT a reply to any message, so that it just can't has any ReplyToId parameters.

Microsoft DirectLine API to bot does not work

So to explain my problem, I have to give you the context.
I got a Bot built with microsoft bot framework deployed on slack. Now it can happen these "events" on my backend that the bot communicates with. When a event occurs, I want to notify my bot of it and then let it send a message to all of it's conversations that something has happend. So basicly:
Backend>Microserivce>Bot>users
To do this I have to store all conversations in my backend, which I do in a database there. When a event happends, the backend will post an activity to the bot with all the conversations(basicly their id's) and the event it should show them.
So in essence my backend need to post a message to my bot.
For doing this I found the microsoft directline api which acts as a middleman here, a more abstract way to talk with the bot. The problem is that I don't know how to do it. I followed microsofts own tutorial but it doesn't seem to work for me:
This is the endpoint that my backend uses to notify the bot. "content" contains conversations and events as a json formated string.
[HttpPost]
[Route("conversationsEvents")]
public HttpResponseMessage PostConversationsEvents([FromBody]string content)
{
NotifyBot.Notify(content);
return Request.CreateResponse(HttpStatusCode.NoContent );
}
NotifyBot.Notify(content) looks like this:
private static async Task StartBotConversation( string contents)
{
string directLineSecret = "secret";
string fromUser = "microserviceNotifyEndpoint";
Activity activity = new Activity
{
From = new ChannelAccount(fromUser),
Text = contents,
Type = ActivityTypes.Event
};
DirectLineClient client = new DirectLineClient(directLineSecret);
var conversation = await client.Conversations.StartConversationAsync();
await client.Conversations.PostActivityAsync(conversation.ConversationId, activity);
}
Basicly the execution get's stuck at var conversation = await client.Conversations.StartConversationAsync(); , it just waits forever.
I tried changing it to var conversation = await client.Conversations.StartConversationAsync().ConfigureAwait(continueOnCapturedContext: false);´the execution goes on but the activity doesn't seem to get posted.

I'm not sure why the call to .StartConversationAsync() would freeze in your case. Maybe you haven't enabled the Direct Line channel on dev.botframework.com/bots? Nonetheless, as pointed out by Sergey, the Direct Line is a Channel and not a means for communicating with your bot on other channels.
Check out the Connector Client: bot-builder-dotnet-connector
Here is a static example of using it to proactively send a message to a user from a bot: MicrosoftDX/botFramework-proactiveMessages - sample: ConversationStarter.cs
pertinent code from sample:
public static async Task Resume(string conversationId,string channelId)
{
var userAccount = new ChannelAccount(toId,toName);
var botAccount = new ChannelAccount(fromId, fromName);
var connector = new ConnectorClient(new Uri(serviceUrl));
IMessageActivity message = Activity.CreateMessageActivity();
if (!string.IsNullOrEmpty(conversationId) && !string.IsNullOrEmpty(channelId))
{
message.ChannelId = channelId;
}
else
{
conversationId = (await connector.Conversations.CreateDirectConversationAsync( botAccount, userAccount)).Id;
}
message.From = botAccount;
message.Recipient = userAccount;
message.Conversation = new ConversationAccount(id: conversationId);
message.Text = "Hello, this is a notification";
message.Locale = "en-Us";
await connector.Conversations.SendToConversationAsync((Activity)message);
}
The serviceUrl, the channelId, conversationId, toId, fromId, etc are cached from previous communication by the user to the bot (these are statically stored in this example, so only work for one user). This example shows how it is possible to proactively send a message to a user from a bot. The Direct Line api is not required.

You don't need to use DirectLine, it is designed for creating alternative bot UIs.
To implementing what your want, you may try the following:
First, you need to store users addresses to whom you want to send the messages. It my be done by storing the ResumptionCookie of a user last message in your backend database.
var state = new ResumptionCookie(message).GZipSerialize();
When your PostConversationsEvents is called, you may resume the conversation at the latest point with each users.
var resumptionCookie = ResumptionCookie.GZipDeserialize(state);
var message = resumptionCookie.GetMessage();
message.Text = content;
await Conversation.ResumeAsync(resumptionCookie, message);
It is not the only solution. As I said, in this case you just resumed the conversation with the user at the latest point. Another solution is to save the user address (user the same ResumptionCookie class) but start the conversation when you need to:
var resumptionCookie = ResumptionCookie.GZipDeserialize(state);
var message = cookie.GetMessage();
ConnectorClient client = new ConnectorClient(new Uri(message.ServiceUrl));
var conversation = await
client.Conversations.CreateDirectConversationAsync(message.Recipient, message.From);
message.Conversation.Id = conversation.Id;
var newMessage = message.CreateReply();
newMessage.Text = content;
await client.Conversations.SendToConversationAsync(newMessage);
See more details on BotFramework documentation.