Is it possible to develop a chatbot with a possible voice integration (maybe Cortana somehow) with the Microsoft Bot Framework and integrate it in Outlook (mail, calendar, tasks)? I started looking at some resources
https://dev.outlook.com/Videos#outlook-addins
but cannot decide where I should look in particular. Can someone provide me with a good start in terms of reading material?
If you want to create bots the botframework is the place to start, https://dev.botframework.com. Here you can create bots that interact with a chatclient. To understand text LUIS.AI (https://luis.ai) is startpoint. LUIS translates utterances into intents (what does user want) and entities (with what parameters).
LUIS.AI has a number of pre-created application that act about Outlook (move appointment of 3 o'clock to 4)
For cortana you can now only build UWP App that integrate your App with Cortana using Speech. Sample can be found at: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CortanaVoiceCommand
If you want to have speech and transform it to text you can use cognitive services of Microsoft: https://www.microsoft.com/cognitive-services/en-us/speech-api
Hope this helps
Sander
Related
I have a need to connect to Office 365 in order to get some scheduling appointment for a room, make a book on a room if is available, modify and detele room's booking.
Log time ago I used Exchange 2007/10 for synchronisation, but now I'm a little lose in the Office 365 environment ...
I try to use Microsfoft Graph API and my personnal (just a mail on outlook.live.com) or company Outlook account in order to play with the use case.
But I don't understand how to connect a C# program to the new Office 365 access, because I Exchange 07/10 it was with a IIS actived EWS service with a certification. Now It seem to be with a Azure ID, but I don't understand if is really reccured even if you only use a simple C# program with you personnal account, you don't have a Azure App IP ...
Someony can give me some clues to create my connection plz ?
Thank a lot
I try to use Microsfoft Graph API and my personnal (just a mail on outlook.live.com) or company Outlook account in order to play with the use case.
You can get a proper Office365 to learn with by joining the Office365 developer program which is free https://developer.microsoft.com/en-us/microsoft-365/dev-program. You will also get an Access to the Azure Portal with this and be able to create Application registrations for oAuth
I would also suggest you look at using the Graph explorer https://developer.microsoft.com/en-us/graph/graph-explorer this allows you to test and learn how the Graph works
When you ready to code you can use the QuickStart for the Graph https://developer.microsoft.com/en-us/graph/quick-start which is quite good and step you through what you need to do.
The last few days, I checked dozens of samples, docs, blog posts, youtube videos (courses, that I couldn't fine) and I can say I'm still confused and I still without a Teams bot.
Every example I've checked is different in approach, and now I'm pretty sure there are many others like me.
Frequently problems: required input/parameters like service Url, conversation Id, etc. 1. You can't find a explanation for them. 2. You can't find a place where it's explained how to get them.
What I want to achieve
I want a bot on Teams, with a API, with 2 parameters: user email, message. The bot has to send the message to the specified user. That's it.
I'm looking how to:
Create/Setup the Application
Create the actual Bot
Install the application to Teams
I used to create a Teams conversation bot. Hope my experience will help you.
First, download sample provided by microsoft. I uses c# to create this bot, so I copy the '57.teams-conversation-bot' folder and opened it with vs studio. In this sample, I need to modify the configuration in appsettings.json. I need to enter the appid and apppassword.
So the next step is creating an Azure ad application in Azure portal. But if you never create Bot Framework registration resource before, you can create the azure ad application when you create a Bot Framework.By the way, You can leave 'Messaging endpoint' empty now. Please note, when you creating Azure ad application, you can choose to set this application as a multitenant app if your account creating the app is in a different tenant with your teams account.
Now I have a bot framework and an Azure ad app, I can modify appsettings.json, enter the appId and app password. I can get appId in app overview page and get app password when creating client secrets in Certificates & secrets tag.
Now, I need to debug my sample with web chat in the azure portal(Enter the bot framework created just now, you can see Tag 'Test in web chat' in the left panel). So I need to use ngrok to make the bot visit my code in local environment. Running the sample by pressing F5 in vs studio. Opening ngrok.exe and running the command ngrok http -host-header=rewrite 3978. Then I can get a https url, copy that and paste in the 'Messaging endpoint'. Don't forget to add '/api/messages'.
Now sending 'hello' to the bot and you will get default result. When you get the picture below, it means you have made a correct configuration.
Next step is to add this bot to Teams client. First, I add the Teams Channel in the bot I created in the azure portal.
Then I need to modify the manifest file. Replacing the "YOUR-MICROSOFT-APP-ID" with appId creating just now. And creating manifest.zip file which contains the three file in folder TeamsAppManifest.
Next step is uploading the zip file to Teams client. Click the 'Apps' in the lower-left corner and click the 'Upload the custom app' in the bottom of the left panel, then click add. Ok, now I get a team conversation bot.
There -are- a ton of approaches, partly because Teams development is still relatively new, partly because the Bot Framework (the set of tools for buildings Bots in the Microsoft world, both for Teams and otherwise) has also undergone some major changes. That's aside from the various language options (dotnet, node, python, etc.).
What's looking to be a good place to get started though, is the new Teams Toolkit, a plugin for Visual Studio and Visual Studio Code, for building a new Teams project (Tab, Bot, etc.). Have a look here for a link. It's very new though, so some rough edges, but it's aiming to be very comprehensive to get up and running quickly (does a lot of Azure registrations behind the scenes, for example). Check out the latest monthly Teams Developer call, from earlier this week, for a visual run-through: https://www.youtube.com/playlist?list=PLWZJrkeLOrbbTKzV1sQOFAXd_s0BHJDiL (this week's call isn't up yet, but it should be very soon).
Below is my code basics planning for implementation in chat bot:
develop teams core bot by enabling calling feature in teams channel settings.
upload bot as custom app and enable for Private, team, group options in teams app settings.
enable API graph permissions in bot channel registration for group calling in
teams.
Azure Speech Service (speech recognition) speech to text will implement in bot.
Azure Speaker Recognition Service to know which
user is speaking will implement in bot.
For Speech and Speaker services, I already tried sample POC in console.
So my queries:
Is it possible to add my bot in group meeting/channel group? or vice versa is possible(Adding users to bot)?
If #1 is yes, say the group call is connected to all members along with bot, is it possible by using speech to text API from bot to convert whatever user speaks and display in chat automatically?
Ex: 3 users joined in channel group , I added bot in that channel group or consider vice versa like in the bot app I added 3 users.
When 3 users are speaking in call, I want to convert speech to text and display in group chat using bot.
Is this scenario possible for implementation in Teams?
In short: just like live captions are displaying in teams audio chat pane, I want it to display same on chat panel.
Better understanding with example on Group audio call with bot in Teams:
Ex:(speaking/spoke) User1: Hi Everyone
(Displaying in chat panel) :Bot: User1-Hi everyone
(speaking/spoke) User2: Hello Everyone
(Displaying in chat panel) : Bot: User2- Hello everyone
I already explored some part of this, but it happens in UWP and window forms. so not sure it can be possible in teams with extension messaging.
Reference Links:
https://github.com/Azure-Samples/cognitive-services-speech-sdk
https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/tutorial-voice-enable-your-bot-speech-sdk
https://bisser.io/bot-framework-teams-messaging-extensions-walkthrough/
https://microsoftgraph.github.io/microsoft-graph-comms-samples/docs/articles/calls/register-calling-bot.html
I have created a bot and deployed it to Azure. In the Azure portal, I have configured Skype for business and Microsoft Teams as channels for my bot.
The bot is using Adaptive Cards to show details of entities, the user has entered (to create a meeting).
I have tested the bot with Bot Framework Emulator using the Azure URL of the bot. Everyting works as expected.
- Entities are properly resolved with LUIS
- Details are filled in the Adaptive Card
- The Adaptive Card is showing in the chat window.
Not so for Skype for Business and Microsoft Teams.
Nothing at all is showing in Skype for Business.
In Microsoft Teams, I receive an error message "The specified card version is not supported.".
I am using the latest NuGet Packages for BotBuilder - Version 3.16 at this point of time. Anyway, the problem exists.
I'd expect, Adaptive Cards code would be compatible with all Microsoft products.
What can I do, to get my Adaptive Cards working in Teams, Skype for Business... ?
What version of the AdaptiveCards package are you using? I had this same issue after support was announced for Microsoft Teams, but I found this StackOverflow post which directed me to update my AdaptiveCards package. Since updating, my AdaptiveCards work in Teams.
Franz,
Team's support for adaptive cards is still in Preview and Skype is still in development according to the Adaptive Card partner status page located here.
The announcement of Adaptive Cards for Teams was recently made but was premature according to comments on this site.
I'm eagerly awaiting this feature to go live myself. It does appear, at least for Teams, to be coming very, very soon.
David's reply below is correct! If you update to the latest Nuget package (1.0.3) Adaptive Cards will work in Teams. Hooray! No Skype support yet.
We have a chat application which works on ASP.Net web applications. We want to allow users to make calls to other users through our chat application. To provide that we want to make use of Microsoft Lync 2010 and use their SDK to leverage its capabilities and make calls. Any pointers on how to do that will be of great help.
You have a couple of options here - you could use the NameCtrl control to integrate presence and a pop-up menu into your aspx pages - see my answer here for more details, and an example.
Assuming you're not using Silverlight, you could create a .NET assembly that uses the SDK to initiate a call (see the walkthrough here), and then expose this to COM using .NET's COM Interop features - example here, if you haven't done this before
Also, Sevki's answer is a great way to get up and running quickly, although you won't be able to get the presence of the user that way.
All of this assumes that the users have Lync running on their machine, i.e. assumes you won't be opening chat up to users outside your organisation who may not have Lync installed.
Edit: Not having Lync installed changes things a lot. If Lync is installed, then Lync will handle setting up the audio call, and will handle the transfer of Audio between the participants in the conversation. Without Lync, you'd have to do that yourself, which won't be straightforward.
Your best bet might be to have your users use the Lync Web App. This is a web based client that allows external users to join conferences. You'd need to build a UCMA 3.0 application which would sit inside your Lync infrastructure. The app would:
Accept a request from your web app to start a new conference, inviting the desired user - see here and here
When the conference is created, get the id and return it to the web user
In the web client, use the returned id to build a URL that opens the Lync Web App.
I can't remember the syntax of the url off the top of my head, but to get it, you could create a new conference using the Lync Client, then select Invite By Email (from the icon of two "pawns") - this will generate an email containing a URL - clicking the URL take you to a page that gives you conference join options, including installing the web app (you'd need to try the URL on a machine that doesn't have Lync installed to get the option to install the web app)
Edit: My bad - the Lync Web App doesn't support Audio. You could actually use the same concept as above, but with the Lync Attendee Client instead. However, this forces the user to install some components client side, so may not be acceptable in your scenario.
Edit: See also the answer here for more details, this refers to getting video to the client, but is relevant for audio too
You could create a hyperlink like so
<a href=”tel:{blah phone number}″>{blah phone number}</a>
or
<a href=”sip:{blah sip blah}″>{blah sip blah}</a>
Which will launch lync from the browser, however if you want to do it inside the browser it seems that you will have to go in to Silverlight. You can find Silverlight samples in Lync Client Side API sdk.
There is an hour-long intro to integrating Lync recorded at Tech-Ed Europe at the end of last year. I've not watched it yet, but it looks quite good.
http://channel9.msdn.com/Events/TechEd/Europe/2010/UNC306