I have an order as an XDocument and I simply want to stick it in the body of a message and send it to an MSMQ queue. I've effectively serialized the order object already and now I just want to send it. Is this possible?
I'm using WCF here but I'd happy with a plain old msmq solution. I'm getting an error here indicating that an XDocument can't be serialized ... obviously can't do that, but how do I get my XDocument into the message body? Do I need to roll my own Serializer?
public void SendOrder(XDocument order)
{
var address = new EndpointAddress(#"msmq.formatname:DIRECT=OS:myServer\private$\myQueue");
var binding = new MsmqIntegrationBinding();
binding.Security.Mode = MsmqIntegrationSecurityMode.None;
binding.ExactlyOnce = false;
binding.Durable = false;
var channelFactory = new ChannelFactory<IOrderSubmitter>(binding, address);
var channel = channelFactory.CreateChannel();
var message = new MsmqMessage<XDocument>(order);
message.Label = "Really Big Order with lots of profit";
message.BodyType = (int)System.Runtime.InteropServices.VarEnum.VT_ARRAY;
using (var scope = new TransactionScope(TransactionScopeOption.Required))
{
channel.SubmitOrder(message);
scope.Complete();
}
}
[ServiceContractAttribute(Namespace = "http://my.namespace.com", Name = "Hello")]
public interface IOrderSubmitter
{
[OperationContract(IsOneWay = true)]
void SubmitOrder(MsmqMessage<XDocument> message);
}
An XDocument is a convenient wrapper over XML data. There is no need to serialize the XDocument, just send the XML data as a string, using XDocument.ToString()
I'm having the same problem developing on a Windows 7 box. It's putting my XML string inside another xml. Everything works just fine in server 2003.
I was finally able to fix this. There seem to be two ways to do this. Both involve setting the Formatter to XmlMessageFormatter. You can either set the Formatter on the MessageQueue or you can set it on the message before you send and after you peek/receive.
messageQueue.Formatter = new System.Messaging.XmlMessageFormatter(new Type[] { typeof(System.String) });
Related
I hope someone can help me out with this since when I drop a JSON test file in my rcv-folder the only result is that in the BizTalk console (in 'Running instances'), the message only states 'Queued (Awaiting processing)'. I am not sure where my issue is in the code.
I am supposed to receive a JSON which will contain some info and possibly multiple attachments (in Base64 format) and then send the mail out (with attachments in correct format, i.e. PDF, txt, xls) to a certain email-address. One of the requirements is to not use an orchestration. But I am stuck and have no idea what I am doing anymore. What makes this question different from others is that I have no orchestration in my solution. Everything will be processed in a custom send pipeline component.
The sendpipeline (in encode stage) does contain my custom component and also the MIME/SMIME encoder. I am using the SMTP-adapter.
I have created the custom pipeline component with a wizard and my initial plan was in the form below:
Receive the file (in JSON-format. It will be transformed into XML for further processing). This will all be taken care of in my rcv-pipeline and this step already works for me.
Pick out all the necessary variables from XML needed to send in the e-mail. These variables are the ones I want to show in the E-mail text. Not sure how to explain it better.
Pick out all the attachments in base64, loop through them, convert to 'regular files' and then attach them to the mail (with correct filenames, extension etc.)
The XML looks like below:
ArchiveobjectsListErrands
- ArchiveobjectErrand
* UUID (Here are the variables I need to show in the E-mail. The plain text, so to say)
- ArchiveobjectListPaper
- Attachments
* Name
* Extension
* Size
* Base64String (Base64 string which will be needed to be fetched (in GetAttachments) and then processed in (ProcessAttachments))
The code I have is below:
public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pContext, Microsoft.BizTalk.Message.Interop.IBaseMessage pInMsg)
{
// 1) Read file with XPathNavigator (https://learn.microsoft.com/en-us/dotnet/standard/data/xml/extract-xml-data-using-xpathnavigator)
XPathNavigator nav = ReadXmlFromMsgBox(pInMsg);
var outMsg = pContext.GetMessageFactory().CreateMessage();
outMsg.Context = PipelineUtil.CloneMessageContext(pInMsg.Context);
// 2) Pick out the necessary vars that the registrator requires
GetRegistratorProperties(nav, pContext, outMsg);
// 3) Read attachments
var attachments = GetAttachments(pInMsg, nav);
// 4) Processa attachments
ProcessAttachments(pContext, outMsg, attachments);
// 5) Send message along for further processing in the send pipeline
return outMsg;
}
private void GetRegistratorProperties(XPathNavigator _nav, IPipelineContext _pContext, IBaseMessage _msg)
{
var bodyPart = _pContext.GetMessageFactory().CreateMessagePart();
bodyPart.ContentType = "text/application";
bodyPart.PartProperties.Write("EmailBodyText", "http://schemas.microsoft.com/BizTalk/2003/smtp-properties", "EmailBodyText.");
bodyPart.PartProperties.Write("Subject", "http://schemas.microsoft.com/BizTalk/2003/smtp-properties", "Registratorsubject - Create errand");
_msg.AddPart("Body", bodyPart, true); // True for body but false for attachments
}
private void ProcessAttachments(IPipelineContext _pContext, IBaseMessage _msg, IList<Attachment> _attachments)
{
var msgPart = _pContext.GetMessageFactory().CreateMessagePart();
//outMsg.Context = PipelineUtil.CloneMessageContext(_msg.Context);
int i = 0;
foreach (var item in _attachments)
{
msgPart.PartProperties.Write("FileName", "http://schemas.microsoft.com/BizTalk/2003/mime-properties", item.filnamn+item.extension);
msgPart.PartProperties.Write("ContentDescription", "http://schemas.microsoft.com/BizTalk/2003/mime-properties", item.contentType);
msgPart.Data = new MemoryStream(BytesFromBase64String(item.base64));
//bodyPart.Charset = "utf-8";
msgPart.ContentType = item.contentType;
//_pInMsg.AddPart("Attachment part " + i.ToString(), bodyPart, false);
_msg.AddPart("Attachment part " + i.ToString(), msgPart, false);
i++;
}
}
private IList<Attachment> GetAttachments(IBaseMessage pInMsg, XPathNavigator _nav)
{
XPathNodeIterator iterator = _nav.Select("Path to attachments in xml");
IList<Attachment> myList = new List<Attachment>();
while (iterator.MoveNext())
{
XPathNavigator node = iterator.Current;
Attachment atttachments = new Attachment();
atttachments.filenamne = node.SelectSingleNode("Name").Value;
atttachments.extension = node.SelectSingleNode("Extension").Value;
atttachments.contentType = node.SelectSingleNode("Mimetype").Value;
atttachments.base64 = node.SelectSingleNode("Base64String").Value;
myList.Add(atttachments);
}
return myList;
}
private XPathNavigator ReadXmlFromMsgBox(IBaseMessage pInMsg)
{
// Using XPathNavigator to avoid creating a XMLDoc in memory
Stream originalMessage = pInMsg.BodyPart.GetOriginalDataStream();
XPathNavigator _navigator = new XPathDocument(originalMessage).CreateNavigator();
return _navigator;
}
[Serializable]
private class FileStreamFactory : IStreamFactory
{
byte[] _data;
public FileStreamFactory(byte[] data)
{
_data = data;
}
public Stream CreateStream()
{
return new MemoryStream(_data);
}
}
private static byte[] BytesFromBase64String(string msg)
{
return Convert.FromBase64String(msg);
}
#endregion
}
I can show some example file of the XML if deemed necessary. I avoided it due to brevity and also since it is quite large.
I would greatly appreciate if anyone could help out with how the code is supposed to look to achieve what is needed, a mail with some text and attachments named correctly regarding filename and extension.
Body text and Attachments are just different segments in a MIME encoded email, usually a plain text one is the first one.
If it has a status of 'Queued (Awaiting processing)', then it sounds like either
the host instance that is expecting to process it is not in a running state. Fix: Start the host instance.
the send port is not in a Started sate. Fix: set the send port to Started
or the send port has a service window set on it. Fix: disable the service window.
I am using KsqlDb a table with the following form:
KSQL-DB Query
create table currency (id integer,name varchar) with (kafka_topic='currency',partitions=1,value_format='avro');
C# model
public class Currency
{
public int Id{get;set;}
public string Name{get;set;}
}
Now i want to know how should i write/read data from this topic in C# using the Confluent library:
Writing
IProducer<int, Currency> producer=....
Currency cur=new Currency();
Message<int,Currency> message = new Message<int, Currency>
{
Key = msg.Id,
Timestamp = new Timestamp(DateTime.UtcNow, TimestampType.CreateTime),
Value = msg
};
DeliveryResult<int,Currency> delivery = await this.producer.ProduceAsync(topic,message);
Reading
IConsumer<int,Currency> iconsumer = new ConsumerBuilder<int, Currency>(config)
.SetKeyDeserializer(Deserializers.Int32) //i assume i need to use the id from my dto
.SetValueDeserializer(...) //what deserializer
.Build();
ConsumeResult<int,Currency> result = consumer.Consume();
Currency message = // what deserializer JsonSerializer.Deserialize<Currency>(result.Message.Value);
I am not sure how to go about this so i tried looking for serializer. I found this library AvroSerializer , but i do not get where the author fetches the schema.
Any help on how to read/write to a specific topic that would match with my ksqldb models ?
Update
After some research and some answers here i have started using the schemaRegistry
var config = new ConsumerConfig
{
GroupId = kafkaConfig.ConsumerGroup,
BootstrapServers = kafkaConfig.ServerUrl,
AutoOffsetReset = AutoOffsetReset.Earliest
};
var schemaRegistryConfig = new SchemaRegistryConfig
{
Url = kafkaConfig.SchemaRegistryUrl
};
var schemaRegistry = new CachedSchemaRegistryClient(schemaRegistryConfig);
IConsumer<int,Currency> consumer = new ConsumerBuilder<int, Currency>(config)
.SetKeyDeserializer(new AvroDeserializer<int>(schemaRegistry).AsSyncOverAsync())
.SetValueDeserializer(new AvroDeserializer<Currency>(schemaRegistry).AsSyncOverAsync())
.Build();
ConsumeResult<int, Currency> result = consumer.Consume();
Now i am getting another error:
Expecting data framing of length 5 bytes or more but total data size
is 4 bytes
As someone kindly pointed out it seems i retrieving only the id from the schema registry.
How can i just : insert into currency (id,name) values (1,3) and retrieve it in C# as a POCO (listed above) ?
Update 2
After i have found this source program it seems i am not able to publish messages to tables for some reason.
There is no error when sending the message but it is not published to Kafka.
I found this library AvroSerializer , but i do not get where the author fetches the schema.
Unclear why you need to use a library other than the Confluent one, but they get it from the Schema Registry. You can use CachedSchemaRegistryClient to get the schema string easily, however you shouldn't need this in the code as the deserializer will download from the registry on its own.
If you refer to the examples/ in the confluent-kafka-dotnet repo for Specific Avro consumption, you can see they generate the User class from User.avsc file, which seems to be exactly what you want to do here for Currency rather than write it yourself
I have solved the problem by defining my custom serializer , thus implementing the ISerializer<T> and IDeserializer<T> interfaces which in their belly are just wrappers over System.Text.Json.JsonSerializer or NewtonsoftJson.
Serializer
public class MySerializer:ISerializer<T>
{
byte[] Serialize(T data, SerializationContext context)
{
var str=System.Text.Json.JsonSerializer.Serialize(data); //you can also use Newtonsoft here
var bytes=Encoding.UTF8.GetBytes(str);
return bytes;
}
}
Usage
var config = new ConsumerConfig
{
GroupId = kafkaConfig.ConsumerGroup,
BootstrapServers = kafkaConfig.ServerUrl,
AutoOffsetReset = AutoOffsetReset.Earliest
};
IConsumer<int,Currency> consumer = new ConsumerBuilder<int, Currency>(config)
.SetValueDeserializer(new MySerializer<Currency>())
.Build();
ConsumeResult<int, Currency> result = consumer.Consume();
P.S
I am not even using the schema registry here afteri implemented the interface
I am new to Google APIs. I want to know how to call Google Dialogflow API in C# to get intent form the input text. But I can't find any example to call Dialogflow using C#.
Please provide some example to call Dialogflow from C#.
If I understand your question correctly you want to call the DialogFlow API from within a C# application (rather than writing fulfillment endpoint(s) that are called from DialogFlow. If that's the case here's a sample for making that call:
using Google.Cloud.Dialogflow.V2;
...
...
var query = new QueryInput
{
Text = new TextInput
{
Text = "Something you want to ask a DF agent",
LanguageCode = "en-us"
}
};
var sessionId = "SomeUniqueId";
var agent = "MyAgentName";
var creds = GoogleCredential.FromJson("{ json google credentials file)");
var channel = new Grpc.Core.Channel(SessionsClient.DefaultEndpoint.Host,
creds.ToChannelCredentials());
var client = SessionsClient.Create(channel);
var dialogFlow = client.DetectIntent(
new SessionName(agent, sessionId),
query
);
channel.ShutdownAsync();
In an earlier version of the DialogFlowAPI I was running into file locking issues when trying to re-deploy a web api project which the channel.ShutDownAsync() seemed to solve. I think this has been fixed in a recent release.
This is the simplest version of a DF request I've used. There is a more complicated version that passes in an input context in this post:
Making DialogFlow v2 DetectIntent Calls w/ C# (including input context)
(Nitpicking: I assume you know DialogFlow will call your code as specified/registered in the action at DialogFlow? So your code can only respond to DialogFlow, and not call it.)
Short answer/redirect:
Don't use Google.Apis.Dialogflow.v2 (with GoogleCloudDialogflowV2WebhookRequest and GoogleCloudDialogflowV2WebhookResponse) but use Google.Cloud.Dialogflow.v2 (with WebhookRequest and WebhookResponse) - see this eTag-error. I will also mention some other alternatives underneath.
Google.Cloud.Dialogflow.v2
Using Google.Cloud.Dialogflow.v2 NuGet (Edit: FWIW: this code was written for the beta-preview):
[HttpPost]
public dynamic PostWithCloudResponse([FromBody] WebhookRequest dialogflowRequest)
{
var intentName = dialogflowRequest.QueryResult.Intent.DisplayName;
var actualQuestion = dialogflowRequest.QueryResult.QueryText;
var testAnswer = $"Dialogflow Request for intent '{intentName}' and question '{actualQuestion}'";
var dialogflowResponse = new WebhookResponse
{
FulfillmentText = testAnswer,
FulfillmentMessages =
{ new Intent.Types.Message
{ SimpleResponses = new Intent.Types.Message.Types.SimpleResponses
{ SimpleResponses_ =
{ new Intent.Types.Message.Types.SimpleResponse
{
DisplayText = testAnswer,
TextToSpeech = testAnswer,
//Ssml = $"<speak>{testAnswer}</speak>"
}
}
}
}
}
};
var jsonResponse = dialogflowResponse.ToString();
return new ContentResult { Content = jsonResponse, ContentType = "application/json" }; ;
}
Edit: It turns out that the model binding may not bind all properties from the 'ProtoBuf-json' correctly (e.g. WebhookRequest.outputContexts[N].parameters),
so one should probably use the Google.Protobuf.JsonParser (e.g. see this documentation).
This parser may trip over unknown fields, so one probably also wants to ignore that. So now I use this code (I may one day make the generic method more generic and thus useful, by making HttpContext.Request.InputStream a parameter):
public ActionResult PostWithCloudResponse()
{
var dialogflowRequest = ParseProtobufRequest<WebhookRequest>();
...
var jsonResponse = dialogflowResponse.ToString();
return new ContentResult { Content = jsonResponse, ContentType = "application/json" }; ;
}
private T ParseProtobufRequest<T>() where T : Google.Protobuf.IMessage, new()
{
// parse ProtoBuf (not 'normal' json) with unknown fields, else it may not bind ProtoBuf correctly
// https://github.com/googleapis/google-cloud-dotnet/issues/2425 "ask the Protobuf code to parse the result"
string requestBody;
using (var reader = new StreamReader(HttpContext.Request.InputStream))
{
requestBody = reader.ReadToEnd();
}
var parser = new Google.Protobuf.JsonParser(JsonParser.Settings.Default.WithIgnoreUnknownFields(true));
var typedRequest = parser.Parse<T>(requestBody);
return typedRequest;
}
BTW: This 'ProtoBuf-json' is also the reason to use WebhookResponse.ToString() which in turn uses Google.Protobuf.JsonFormatter.ToDiagnosticString.
Microsoft's BotBuilder
Microsoft's BotBuilder packages and Visual Studio template.
I havent't used it yet, but expect approximately the same code?
Hand written proprietary code
A simple example of incoming request code (called an NLU-Response by Google) is provided by Madoka Chiyoda (Chomado) at Github. The incoming call is simply parsed to her DialogFlowResponseModel:
public static async Task<HttpResponseMessage> Run([...]HttpRequestMessage req, [...]CloudBlockBlob mp3Out, TraceWriter log)
...
var data = await req.Content.ReadAsAsync<Models.DialogFlowResponseModel>();
Gactions
If you plan to work without DialogFlow later on, please note that the interface for Gactions differs significantly from the interface with DialogFlow.
The json-parameters and return-values have some overlap, but nothing gaining you any programming time (probably loosing some time by starting 'over').
However, starting with DialogFlow may gain you some quick dialog-experience (e.g. question & answer design/prototyping).
And the DialogFlow-API does have a NuGet package, where the Gactions-interface does not have a NuGet-package just yet.
I am trying to deserialize messages from a service bus topic created as follows (.NET 4.6.1):
SubscriptionClient ServiceBusClient =
SubscriptionClient.CreateFromConnectionString
(ConnectionString, Topic, SubscriptionName);
I then create the on message method as follows:
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
options.AutoRenewTimeout = TimeSpan.FromMinutes(1);
ServiceBusClient.OnMessage((message) =>
{
try
{
Stream stream = message.GetBody<Stream>();
StreamReader reader = new StreamReader(stream);
string payload = reader.ReadToEnd();
dynamic deserialize = Newtonsoft.Json.Linq.JObject.Parse(payload);
var avl = AvlDatum.Parser.ParseFrom(Convert.FromBase64String(deserialize.body));
/* do other stuff here */
message.Complete();
}
catch (Exception)
{
message.Abandon();
}
}, options);
Using this code however I receive absolutely no messages. I thought something was wrong with the topic but if I comment out the line with the dynamic object it receives the messages fine. Why am I not able to use the Newtonsoft Parser/dynamic objects inside an OnMessage delegate? Any info would be appreciated.
I have the following Python script which I need to port to C#. This gets a JSON response from a URL and then pops it into a dictionary. Then it checks for the data next_page and if there is data (it's not empty) it then returns true. Underneath I'll paste the C# code I have but I'm really struggling to do the final part. I don't know and I certainly don't want to understand the data in the JSON response, I just want to know if the field next_page is there.
# Gets JSON response
response = requests.get(url, auth=(user, pwd))
if response.status_code != 200:
print('Status:', response.status_code, 'Problem with the request. Exiting.')
exit()
data = response.json()
if(data['next_page']):
return True
else:
return False
So this is the c# code I've got:
using Newtonsoft.Json;
string response = "";
using (WebClient client = new WebClient())
{
client.UseDefaultCredentials = true;
client.Credentials = new NetworkCredential(user, password);
try
{
response = client.DownloadString(url);
} catch (Exception e)
{
throw e;
}
}
XmlDocument xml = JsonConvert.DeserializeXmlNode(json, "RootObject");
XmlReader xr = new XmlNodeReader(xml);
DataSet ds = new DataSet("Json Data");
ds.ReadXml(xr);
From what I've seen on the web DataSets work best when you know what the data inside of it is. I just want to know if there is a field called next_page and if there is, is it empty or does it have data. I'm just struggling to get anything out of the DataSet.
You will want to include the JSON.net nuget package (http://james.newtonking.com/json) this lets you deserialize the JSON response into a dictionary (or preferably a new class) allowing you to access the response.
eg add this into your try catch after including the library
var dict = Newtonsoft.Json.JsonConvert.DeserializeObject<Dictionary<string, string>>(response);
Alternativly you could create a new class that represents the expected JSON and deserialize into that
public class ResponseObject
{
public string next_page { get; set; }
}
var responseResult = Newtonsoft.Json.JsonConvert.DeserializeObject<ResponseObject>(response);