We Keep Coding

Making DialogFlow v2 DetectIntent Calls w/ C# (including input context)

So I finally figured out a way to successfully make detect intent calls and provide an input context. My question is whether or not this is the CORRECT (or best) way to do it:
(And yes, I know you can just call DetectIntent(agent, session, query) but I have to provide a input context(s) depending on the request)
var query = new QueryInput
{
Text = new TextInput
{
Text = model.Content,
LanguageCode = string.IsNullOrEmpty(model.Language) ? "en-us" : model.Language,
}
};
var commonContext = new global::Google.Cloud.Dialogflow.V2.Context
{
ContextName = new ContextName(agent, model.sessionId, "my-input-context-data"),
LifespanCount = 3,
Parameters = new Struct
{
Fields = {
{ "Source", Value.ForString(model.Source) },
{ "UserId" , Value.ForString(model.UserId.ToString())},
{ "Name" , Value.ForString(model.FirstName)}
}
}
};
var request = new DetectIntentRequest
{
SessionAsSessionName = new SessionName(agent, model.sessionId),
QueryParams = new QueryParameters
{
GeoLocation = new LatLng {Latitude = model.Latitude, Longitude = model.Longitude},
TimeZone = model.TimeZone ?? "MST"
},
QueryInput = query
};
request.QueryParams.Contexts.Add(commonContext);
// ------------
var creds = GetGoogleCredentials("myCredentials.json");
var channel = new Grpc.Core.Channel(SessionsClient.DefaultEndpoint.Host, creds.ToChannelCredentials());
var client = SessionsClient.Create(channel);
var response = client.DetectIntent(request);
channel.ShutdownAsync();
return response;
Note: I included the explicit ShutDownAsync (it's not in an async call) because I was getting some file locking issues when attempting to re-deploy the WebAPI project (and only after having executed this code).
Thanks
Chris
Updated 4/25: The most basic way I use this is to integrate the user's name into intent responses:
It can also be read from within the webhook/inline fulfillment index.js:
const name = request.body.queryResult && request.body.queryResult.outputContexts && request.body.queryResult.outputContexts[0].parameters.Name

How to store & retrieve Bot Data in Azure Table storage with directLine channel?

I'm using Microsoft Bot Framework with directLine channel. My Bot is a part of company's customer portal from where I fetch some user information and store it in BotState using stateClient as shown below
public ActionResult Index()
{
var userId = ClaimsPrincipal.Current.FindFirst(ClaimTypes.NameIdentifier).Value;
GetTokenViaBootStrap().Wait();
var botCred = new MicrosoftAppCredentials(
ConfigurationManager.AppSettings["MicrosoftAppId"],
ConfigurationManager.AppSettings["MicrosoftAppPassword"]);
var stateClient = new StateClient(botCred);
BotState botState = new BotState(stateClient);
BotData botData = new BotData(eTag: "*");
botData.SetProperty<string>("UserName", result.UserInfo.GivenName + " " + result.UserInfo.FamilyName);
botData.SetProperty<string>("Email", result.UserInfo.DisplayableId);
botData.SetProperty<string>("GraphAccessToken", UserAccessToken);
botData.SetProperty<string>("TokenExpiryTime", result.ExpiresOn.ToString());
stateClient.BotState.SetUserDataAsync("directline", userId, botData).Wait();
var UserData = new UserInformationModel
{
UserId = userId,
UserName = result.UserInfo.GivenName + " " + result.UserInfo.FamilyName
};
return View(UserData);
}
As its a directLine channel, I'm connecting my bot using secret in javascript as shown below:
BotChat.App({
bot: { id: 'my_bot_id', name: 'my_bot_id' },
resize: 'detect',
sendTyping: true, // defaults to false. set to true to send 'typing' activities to bot (and other users) when user is typing
user: { id: UserData.UserId},
directLine: {
secret: "my_bot_secret"
}
}, document.getElementById('my_bot_id'));
I'm accessing user information data in Node js Bot captured in MVC site as shown below:
function sessionUserCapture(session) {
switch (session.message.address.channelId) {
case 'skypeforbusiness':
// some code
break;
case 'directline':
userName= session.userData.UserName;
userEmail= session.userData.Email;
//some code
break;
case 'slack':
// some code
}
}
I referred Microsoft's Save state data from Manage state data for above code and then I used userData available in the session to access this data in my Node.JS Bot.
As the StateClient is Deprecated, I referred this to replace stateclient with Azure Table storage. However, I'm not able to understand how can I store the above data in the Table Storage.
Can anyone suggest any article which I can refer to solve this issue?
My Bot is in NodeJs and the I'm using directLine channel in a C# MVC application.

One option is to use the Microsoft Azure Storage Client Library for .NET, as explained in the answer here: How to retrieve Saved Conversation Data in Azure (Tablelogger) Just make sure to follow the exact same PartitionKey strategy as is followed by the TableBotDataStore class, and serialize the data field correctly.
--
Edit:
I tested this out, and it does in fact work as expected.
public class WebChatController : Controller
{
public ActionResult Index()
{
var connectionString = ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString;
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("BotStore");
string userId = Guid.NewGuid().ToString();
TableQuery<BotDataRow> query = new TableQuery<BotDataRow>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, userId));
var dataRow = table.ExecuteQuery(query).FirstOrDefault();
if(dataRow != null)
{
dataRow.Data = Newtonsoft.Json.JsonConvert.SerializeObject(new
{
UserName = "This user's name",
Email = "whatever#email.com",
GraphAccessToken = "token",
TokenExpiryTime = DateTime.Now.AddHours(1)
});
dataRow.Timestamp = DateTimeOffset.UtcNow;
table.Execute(TableOperation.Replace(dataRow));
}
else
{
var row = new BotDataRow(userId, "userData");
row.Data = Newtonsoft.Json.JsonConvert.SerializeObject(new
{
UserName = "This user's name",
Email = "whatever#email.com",
GraphAccessToken = "token",
TokenExpiryTime = DateTime.Now.AddHours(1)
});
row.Timestamp = DateTimeOffset.UtcNow;
table.Execute(TableOperation.Insert(row));
}
var vm = new WebChatModel();
vm.UserId = userId;
return View(vm);
}
public class BotDataRow : TableEntity
{
public BotDataRow(string partitionKey, string rowKey)
{
this.PartitionKey = partitionKey;
this.RowKey = rowKey;
}
public BotDataRow() { }
public bool IsCompressed { get; set; }
public string Data { get; set; }
}
}
In the node bot:
'use strict';
const builder = require('botbuilder');
const restify = require('restify');
var azure = require('botbuilder-azure');
var tableName = 'BotStore';
var azureTableClient = new azure.AzureTableClient(tableName,'accountname','accountkey');
var tableStorage = new azure.AzureBotStorage({ gzipData: false }, azureTableClient);
const connector = new builder.ChatConnector({
appId: process.env.MicrosoftAppId,
appPassword: process.env.MicrosoftAppPassword
});
const server = restify.createServer();
server.listen(process.env.port || process.env.PORT || 3979, () => {
console.log(`${server.name} listening to ${server.url}`);
});
server.post('/api/messages', connector.listen());
var bot = new builder.UniversalBot(connector)
.set('storage', tableStorage);;
bot.dialog('/',
[
function (session){
var data = session.userData;
}
]);

The code you are using is using the deprecated default state and will not work. In order to accomplish what you would like it depends on where you are in your code. The deciding factor is if you have access to the context object or not.
For example if you are in the MessagesController you will not have access to the context object and your code might look like this:
public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
{
if (activity.Type == ActivityTypes.Message)
{
var message = activity as IMessageActivity;
using (var scope = DialogModule.BeginLifetimeScope(Conversation.Container, message))
{
var botDataStore = scope.Resolve<IBotDataStore<BotData>>();
var key = Address.FromActivity(message);
var userData = await botDataStore.LoadAsync(key, BotStoreType.BotUserData, CancellationToken.None);
userData.SetProperty("key 1", "value1");
userData.SetProperty("key 2", "value2");
await botDataStore.SaveAsync(key, BotStoreType.BotUserData, userData, CancellationToken.None);
await botDataStore.FlushAsync(key, CancellationToken.None);
}
await Conversation.SendAsync(activity, () => new Dialogs.RootDialog());
}
}
then to get the data:
userData.GetProperty<string>("key 1");
The other situation would be if you did have access to the context object like in a Dialog for example, your code might look like this:
context.UserData.SetValue("key 1", "value1");
context.UserData.SetValue("key 2", "value2");
then to get the data:
context.UserData.GetValue<string>("key 1");

Have you configured your bot to connect to Azure Table Storage in Global.asax.cs, instead of the deprecated default state?
I have written a blog post with full details on how to move your bot's state to Azure Table Storage which you should review
here
protected void Application_Start()
{
Conversation.UpdateContainer(
builder =>
{
builder.RegisterModule(new AzureModule(Assembly.GetExecutingAssembly()));
// Using Azure Table for storage
var store = new TableBotDataStore(ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString);
builder.Register(c => store)
.Keyed<IBotDataStore<BotData>>(AzureModule.Key_DataStore)
.AsSelf()
.SingleInstance();
});
GlobalConfiguration.Configure(WebApiConfig.Register);
}

C# Unit tests - Unable to extract Content from the IHttpActionResult responce

I have this controller
[Route("GeocacheAddressObjectList")]
[HttpPost]
public async Task<IHttpActionResult> GeocacheAddressObjectList([FromBody] List<GeocacheAddress> addresses)
{
//check valid addresses
if(addresses == null)
{
return BadRequest("Invalid addresses. The address list object is null!") as IHttpActionResult;
}
ElasticHelper searchHelper = new ElasticHelper(ConfigurationManager.AppSettings["ElasticSearchUri"]);
List<GeocacheAddress> geocodedAddresses = new List<GeocacheAddress>();
// check each address in the addresses list against geocache db
foreach (GeocacheAddress address in addresses)
{
var elasticSearchResult = SearchGeocacheIndex(address);
// found a match
if (elasticSearchResult.Total != 0)
{
SearchProperties standardizedAddressSearch = new SearchProperties();
standardizedAddressSearch.Size = 1;
standardizedAddressSearch.From = 0;
Address elasticSearchResultAddress = elasticSearchResult.Hits.ElementAt(0).Source;
// query the standardized key in geocache db
standardizedAddressSearch.ElasticAddressId = elasticSearchResultAddress.Standardized.ToString();
// the address is already standardized, return the standardized address with its geocode
if (standardizedAddressSearch.ElasticAddressId == "00000000-0000-0000-0000-000000000000")
{
geocodedAddresses.Add(new GeocacheAddress
{
Id = address.Id,
Street = elasticSearchResultAddress.AddressString,
City = elasticSearchResultAddress.City,
State = elasticSearchResultAddress.State,
ZipCode = elasticSearchResultAddress.Zipcode,
Plus4Code = elasticSearchResultAddress.Plus4Code,
Country = elasticSearchResultAddress.Country,
Latitude = elasticSearchResultAddress.Coordinates.Lat,
Longitude = elasticSearchResultAddress.Coordinates.Lon
});
}
else // perform another query using the standardized key
{
Address standardizedAddress = StandardAddressSearch(standardizedAddressSearch).Hits.ElementAt(0).Source;
if (standardizedAddress == null)
{
return BadRequest("No standardized address found in geocache database") as IHttpActionResult;
}
geocodedAddresses.Add(new GeocacheAddress()
{
Id = address.Id,
Street = standardizedAddress.AddressString,
City = standardizedAddress.City,
State = standardizedAddress.State,
ZipCode = standardizedAddress.Zipcode,
Plus4Code = standardizedAddress.Plus4Code,
Country = standardizedAddress.Country,
Latitude = standardizedAddress.Coordinates.Lat,
Longitude = standardizedAddress.Coordinates.Lon
});
}
}
else // not found in geocache db, call SmartStreets API
{
List<Address> address_list = new List<Address>();
using (HttpClient httpClient = new HttpClient())
{
//Send the request and get the response
httpClient.BaseAddress = new System.Uri(ConfigurationManager.AppSettings["GeocodingServiceUri"]);
httpClient.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
//Lookup object to perform Geocoding service call
var postBody = JsonConvert.SerializeObject(new Lookup()
{
MaxCandidates = 1,
Street = address.Street,
City = address.City,
State = address.State,
ZipCode = address.ZipCode
});
var requestContent = new StringContent(postBody, Encoding.UTF8, "application/json");
// Send the request and get the response
var response = await httpClient.PostAsync("GeocodeAddressObject", requestContent);
if (!response.IsSuccessStatusCode) //error handling
{
geocodedAddresses.Add(new GeocacheAddress()
{
Id = address.Id,
Error = response.ReasonPhrase
});
}
Geocode geocodeFromGeocoder = JsonConvert.DeserializeObject<List<Geocode>>(response.Content.ReadAsStringAsync().Result).ElementAt(0);
GeocacheAddress geocodedAddress = new GeocacheAddress()
{
Id = address.Id,
Street = geocodeFromGeocoder.CorrectedAddress,
City = geocodeFromGeocoder.City,
State = geocodeFromGeocoder.State,
ZipCode = geocodeFromGeocoder.Zipcode,
Plus4Code = geocodeFromGeocoder.Plus4Code,
Country = geocodeFromGeocoder.Country,
Latitude = geocodeFromGeocoder.Latitude,
Longitude = geocodeFromGeocoder.Longitude
};
geocodedAddresses.Add(geocodedAddress);
// check each geocoded address against geocache db
Guid standardized_key;
var geocodedAddressResult = SearchGeocacheIndex(geocodedAddress);
// found a match
if (geocodedAddressResult.Total != 0)
{
Address standardizedAddress = geocodedAddressResult.Hits.ElementAt(0).Source;
standardized_key = standardizedAddress.AddressID;
}
else // not found, insert geocode into geocache db
{
Address new_standardized_address = createStandardizedAddress(geocodeFromGeocoder);
standardized_key = new_standardized_address.AddressID;
address_list.Add(new_standardized_address);
}
// insert non-standardized address into geocache db
Address new_nonstandardized_address = createNonStandardizedAddress(address, standardized_key);
address_list.Add(new_nonstandardized_address);
}
searchHelper.BulkIndex<Address>(address_list, "xxx", "xxx");
}
}
return Json(geocodedAddresses, new Newtonsoft.Json.JsonSerializerSettings()) as IHttpActionResult;
}
I am writing a unit test to test some part of this controller.
I want to compare the response received from the controller with the expected value. When i debug the result, it shows the content for the response but I am unable to use content like (result.Content) in the code.
When i try to use this line, then it returns null response.
var result = await controller.GeocacheAddressObjectList(testGeocacheAddress) as OkNegotiatedContentResult<GeocacheAddress>;
Actual unit test code. I would appreciate any help.
[TestMethod]
public async Task TestMethod1()
{
var controller = new GeocachingController();
var testGeocacheAddress = new List<GeocacheAddress>();
testGeocacheAddress.Add(new GeocacheAddress
{
City = "Renton",
});
var result = await controller.GeocacheAddressObjectList(testGeocacheAddress);
var expected = GetGeocacheAddress();
Assert.AreEqual(result.Content.City, expected[0].City);
}
private List<GeocacheAddress> GetGeocacheAddress()
{
var testGeocacheAddress = new List<GeocacheAddress>();
testGeocacheAddress.Add(new GeocacheAddress
{
Id = Guid.Empty,
Street = "365 Renton Center Way SW",
City = "Renton",
State = "WA",
ZipCode = "98057",
Plus4Code = "2324",
Country = "USA",
Latitude = 47.47753,
Longitude = -122.21851,
Error = null
});
return testGeocacheAddress;
}

In your unit test you need to cast the result to JsonResult<T>, more specifically JsonResult<List<GeocacheAddress>> as that is what you are returning.
var result = await controller.GeocacheAddressObjectList(testGeocacheAddress) as JsonResult<List<GeocacheAddress>>;
If you were to have used return Ok(geocodedAddresses) in your controller return (where you now return the call from Json) then you could have cast to OkNegotiatedContentResult<List<GeocacheAddress>>.
Also in your Controller code you do not need to cast the return to IHttpActionResult because JsonResult<T> already implements that. The cast is redundant.

That's quite simple to achieve, all you have to do is cast the content you're expecting in your unit test method.
Example:
Controller:
public class FooController : ApiController
{
public IHttpActionResult Get()
{
var foo = "foo";
return Ok(foo);
}
}
Unit test:
[TestMethod]
public void Get_Foo_From_Controller()
{
var fooController = new FooController();
var result = fooController.Get();
//Here we are casting the expected type
var values = (OkNegotiatedContentResult<string>)result;
Assert.AreEqual("Foo", values.Content);
}
By the way, i noticed you're using the async keyword in your controller action but i don't see the await keyword.
Using the async keyword without an await will give you a warning and result in a synchronous operation.
Also, you dont have to cast your response as an IHttpActionResult, you could do something like i showed in my example, wrap your content inside an Ok(your content here) and you're good to go.

Dispatching Index() from NEST into to Elasticsearch.NET failed

I am having error when calling client.Index when using Nest and Elasticsearch.NET
var httpConnection = new AwsHttpConnection(new AwsSettings
{
AccessKey = "AKIAIFIV4LN6XZAFVX7Q",
SecretKey = "MySecretKey",
Region = "us-east-1",
});
var pool = new SingleNodeConnectionPool(new Uri(sSearchURI));
var config = new ConnectionSettings(pool, httpConnection);
var client = new ElasticClient(config);
var person = new Person
{
Id = "1",
Firstname = "Martijn",
Lastname = "Laarman"
};
var index = client.Index(person);
Dispatching Index() from NEST into to Elasticsearch.NET failed
Received a request marked as PUT This endpoint accepts POST,PUT The
request might not have enough information provided to make any of
these endpoints:
- /{index}/{type}
- /{index}/{type}/{id}
Any idea?

You need to set the default index:
settings.DefaultIndex("indexName");

How to use Identity user managament with Cordova and OAuth.io?

I want to make a Cordova phone app and a web application. Both the application and the app share the same database.
On the mobile app, the user actions send requests to a web service ( over https ) that writes to the database. On the mobile app, I use https://oauth.io to let the user register and log in with multiple open auth providers. Trying to make it work for facebook, for now.
I just can't figure how to use the Identity user management in that context. Most of the examples I find are in the context of a web app where the user clicks and it calls the account controller. In my case, the oauth.io lib calls facebook, returns an access token, which I pass to my service.
The cordova app passes the accessToken to this method to my server side web service.
var client = new FacebookClient(accessToken);
if (client != null)
{
dynamic fbresult = client.Get("me");
if (fbresult["id"] != null)
{
var fbid = fbresult["id"].ToString();
and where do we go from now ?
how do I insert a new user
I tried this:
var user = new ApplicationUser() { UserName = fbresult["id"] };
Backend.Controllers.AccountController ac = new Controllers.AccountController();
ac.UserManager.CreateAsync(user);
Doesn't work because the usermanagement object inside the account controller is null.
There is an overload of the AccountController constructor but I have a feeling I'm doing this whole thing the wrong way.
Let's say the server side receives a facebook access token. How do use OWIN and Identity user management system from there?

Ok.
As suggested by a friend, I replaced the controllers etc from the original web api template for the ones in the Identity Sample Project
Here is the method called by the mobile app, with a angular jsonp
[OperationContract]
[WebGet(ResponseFormat = WebMessageFormat.Json)]
public string StartSession(string accessToken)
{
if (!HttpContext.Current.Request.IsAuthenticated)
{
var client = new FacebookClient(accessToken);
if (client != null)
{
dynamic fbresult = client.Get("me");
if (fbresult["id"] != null)
{
string fbid = fbresult["id"].ToString();
ApplicationUser user = null;
using (var context = new ApplicationDbContext())
{
user = context.Users.FirstOrDefault(u => u.UserName.ToString() == fbid);
}
if (user == null)
{
CreateUserAsync(fbid);
return "user created. ";
}
else
{
HttpContext.Current.Session["user"] = "holy fuck";
return "user logged in. ";
}
}
}
return "ok";
}
else
{
return "already auth !";
}
}
here is the CreateUserAsync I made
public async System.Threading.Tasks.Task<bool> CreateUserAsync(string fbid)
{
using (var context = new ApplicationDbContext())
{
var newUser = new ApplicationUser() { UserName = fbid, Email = "xxx#gmail.com" };
var userManager = new ApplicationUserManager(new UserStore<ApplicationUser>(context));
try
{
var result = await userManager.CreateAsync(newUser, "Admin#123456");
var test = await context.SaveChangesAsync();
return result.Succeeded;
}
catch (Exception ex)
{
throw ex;
}
}
}
And then, when the mobile app calls back my web service, I can check if the session exists like so:
[OperationContract]
[WebGet(ResponseFormat = WebMessageFormat.Json)]
public async Task<string> TestLogin(int id, string callback)
{
if (HttpContext.Current.Session["user"] != null)
{
return new JavaScriptSerializer().Serialize(new word() { Name = "woot" });
}
else
return new JavaScriptSerializer().Serialize(new word() { Name = "not logged" });
}
Yea, that's right. A if and a session. Just like I was doin' 13 years ago.
Also, while doing this abomination, I stumbled upon a hangin' problem in the IdentityConfig.cs file.
Apparantly, the problem is known by Microsoft and I guess it is probably fixed in the version 3 of Owin ? But I didn't know about that version 3 at that time, so I followed Program hangs during database initialization .
For some reason, some of the method posted in his solution didn't exist for me. I ended up fixing the code the best I could:
public static void InitializeIdentityForEF(ApplicationDbContext db)
{
//ApplicationUserManager userManager = HttpContext.Current.GetOwinContext().GetUserManager<ApplicationUserManager>();
RoleManager<IdentityRole> roleManager = new RoleManager<IdentityRole>(new RoleStore<IdentityRole>(db));
const string name = "admin#example.com";
const string password = "Admin#123456";
const string roleName = "Admin";
IdentityRole adminRole = new IdentityRole(roleName);
//Create Role Admin if it does not exist
if (!roleManager.RoleExists(roleName))
{
roleManager.Create(adminRole);
PasswordHasher hasher = new PasswordHasher();
ApplicationUser adminUser = new ApplicationUser { UserName = name, Email = name, PasswordHash = hasher.HashPassword(password), LockoutEnabled = false };
db.Users.Add(adminUser);
IdentityUserRole userRole = new IdentityUserRole() { RoleId = adminRole.Id, UserId = adminUser.Id };
adminUser.Roles.Add(userRole);
var x = db.SaveChanges();
}
}
Also just in case someone is wondering how to call the svc service from the mobile, here is the code.
(it's a little bit messy, but the important parts are there.)
(keep in mind I am using https://oauth.io/ )
$scope.refresh = function () {
$http.jsonp("https://10.0.100.38:6443/Service1.svc/helloworld?id=1&callback=JSON_CALLBACK").success(function JSON_CALLBACK(result) {
OAuth.popup('facebook')
.done(function (oauthResult) {
oauthResult.me() // standardise lesfield firstname, first-name etc
.done(function (response) {
alert("3");
$http.jsonp("https://10.0.100.38:6443/Service1.svc/StartSession?accessToken=" +oauthResult.access_token + "&callback=JSON_CALLBACK").success(function JSON_CALLBACK(result) {
alert("done " +result); // StartSession serverside success ");
}).error(function (data, status, headers, config) {
alert("icierror2" +data + " " +status + " " +headers + " " + config);
$scope.status = status;
});
}).fail(function (error) {
alert("icierror3 " +error);
});
})
.fail(function (error) {
console.log(error);
});
alert(result.Name); // result de la svc request over https
}).error(function (data, status, headers, config) {
alert("icierror" +data + " " +status + " " + headers + " " +config);
$scope.status = status;
});
Problems
As of now, I am not creating a Login, only a user is created.
Also, the project's OWIN version is 2.0, and apparantly, a 3.0 exists.
To be honest, the more I read online the more I feel like everything I've done is a big hack around the right way to do it. I just couldn't figure it out. This is thing is incredibly huge, confusing, chaothic and broken. Yea, I've added an opinion to my answer.