We Keep Coding

Deserializing ServiceBus content in Azure Logic App

I'm trying to read the content body of a message in an Azure Logic App, but I'm not having much success. I have seen a lot of suggestions which say that the body is base64 encoded, and suggest using the following to decode:
#{json(base64ToString(triggerBody()?['ContentData']))}
The base64ToString(...) part is decoding the content into a string correctly, but the string appears to contain a prefix with some extra serialization information at the start:
#string3http://schemas.microsoft.com/2003/10/Serialization/�3{"Foo":"Bar"}
There are also some extra characters in that string that are not being displayed in my browser. So the json(...) function doesn't accept the input, and gives an error instead.
InvalidTemplate. Unable to process template language expressions in
action 'HTTP' inputs at line '1' and column '2451': 'The template
language function 'json' parameter is not valid. The provided value
#string3http://schemas.microsoft.com/2003/10/Serialization/�3{"Foo":"bar" }
cannot be parsed: Unexpected character encountered while parsing value: #. Path '', line 0, position 0.. Please see https://aka.ms/logicexpressions#json for usage details.'.
For reference, the messages are added to the topic using the .NET service bus client (the client shouldn't matter, but this looks rather C#-ish):
await TopicClient.SendAsync(new BrokeredMessage(JsonConvert.SerializeObject(item)));
How can I read this correctly as a JSON object in my Logic App?

This is caused by how the message is placed on the ServiceBus, specifically in the C# code. I was using the following code to add a new message:
var json = JsonConvert.SerializeObject(item);
var message = new BrokeredMessage(json);
await TopicClient.SendAsync(message);
This code looks fine, and works between different C# services no problem. The problem is caused by the way the BrokeredMessage(Object) constructor serializes the payload given to it:
Initializes a new instance of the BrokeredMessage class from a given object by using DataContractSerializer with a binary XmlDictionaryWriter.
That means the content is serialized as binary XML, which explains the prefix and the unrecognizable characters. This is hidden by the C# implementation when deserializing, and it returns the object you were expecting, but it becomes apparent when using a different library (such as the one used by Azure Logic Apps).
There are two alternatives to handle this problem:
Make sure the receiver can handle messages in binary XML format
Make sure the sender actually uses the format we want, e.g. JSON.
Paco de la Cruz's answer handles the first case, using substring, indexOf and lastIndexOf:
#json(substring(base64ToString(triggerBody()?['ContentData']), indexof(base64ToString(triggerBody()?['ContentData']), '{'), add(1, sub(lastindexof(base64ToString(triggerBody()?['ContentData']), '}'), indexof(base64ToString(triggerBody()?['ContentData']), '}')))))
As for the second case, fixing the problem at the source simply involves using the BrokeredMessage(Stream) constructor instead. That way, we have direct control over the content:
var json = JsonConvert.SerializeObject(item);
var bytes = Encoding.UTF8.GetBytes(json);
var stream = new MemoryStream(bytes);
var message = new BrokeredMessage(stream, true);
await TopicClient.SendAsync(message);

You can use the substring function together with indexOf and lastIndexOf to get only the JSON substring.
Unfortunately, it's rather complex, but it should look something like this:
#json(substring(base64ToString(triggerBody()?['ContentData']), indexof(base64ToString(triggerBody()?['ContentData']), '{'), add(1, sub(lastindexof(base64ToString(triggerBody()?['ContentData']), '}'), indexof(base64ToString(triggerBody()?['ContentData']), '}')))))
More info on how to use these functions here.
HTH

Paco de la Cruz solution worked for me, though I had to swap out the last '}' in the expression for a '{', otherwise it finds the wrong end of the data segment.
I also split it into two steps to make it a little more manageable.
First I get the decoded string out of the message into a variable (that I've called MC) using:
#{base64ToString(triggerBody()?['ContentData'])}
then in another logic app action do the substring extraction:
#{substring(variables('MC'),indexof(variables('MC'),'{'),add(1,sub(lastindexof(variables('MC'),'}'),indexof(variables('MC'),'{'))))}
Note that the last string literal '{' is reversed from Paco's solution.
This is working for my test cases, but I'm not sure how robust this is.
Also, I've left it as a String, I do the conversion to JSON later in my logic app.
UPDATE
We have found that just occasionally (2 in several hundred runs) the text that we want to discard can contain the '{' character.
I have modified our expression to explicitly locate the start of the data segment, which for me is:
'{"IntegrationRequest"'
so the substitution becomes:
#{substring(variables('MC'),indexof(variables('MC'),'{"IntegrationRequest"'),add(1,sub(lastindexof(variables('MC'),'}'),indexof(variables('MC'),'{"IntegrationRequest"'))))}

Finding specific parts of strings containing certain letters/symbols and then creating a reference to those parts

Currently I am storing data in form of jsons (strings) on a database. As jsons contain quotation marks though and the database I am using is unable to store quotation marks in this form: " it converts all quotation marks (like this one :") to "
Unity will therefor not allow me to deserialize the json anymore as it now looks somewhat like this:
{"coins":0,"level":0,"kills":0,"deaths":0,"xp":0.0}
instead of like this:
{"coins":0,"level":0,"kills":0,"deaths":0,"xp":0.0}
Obviously a possible solution to this would be to find all the parts of my json string containing ", storing a reference to these parts and then converting all of those parts to a simple "
Therefore I would ask you how I would go about doing this.

You can use String.replace(""","\"") and than String.split, but maybe you need to think about moving to a database that supports JSONs, like mongodb. Other direction to solve this: have you tried placing the " as \"?

The Database is doing a good job by encoding the text for you thereby preventing Hacks!! It is simply doing text encoding for you.
All you have to do is Decode the text before using it. If there are chances that double quote is part of the data then you should be careful while reverse converting the encoded text. Refer to this MSDN resource Anti-Cross Site Scripting Library to get better insight into topic

Using JSON with corrupt data in C#

I recently had to parse JSON data like
[
{"firstName":"John", "lastName":"Doe"},
{"firstName":"Anna", "lastName":"Smith"},
{"firstName":"Peter", "lastName":"Jones"}
]
like this:
var reqData = JsonConvert.DeserializeObject<Dictionary<string, object>>("{" + fileData + "}");
which I used in another project where the data was well formattted. Here, however, the data was somewhat corrupt. For instance "firstName" might appear as ".\"firstName" and so forth. Using JSON like above results in an exception thrown.
I tried various schemes to "purify" the data but as I cannot predict the state of other data, I stopped using JSON and just parsed it myself (with heavy use of substrings and counting to isolate the keys and values). That method works OK but of course using JSON would be much simplier.
Is there a way around this with JSON?

The main Problem is to define corrupt data. If you know that there is never a substring .\" so you can replace it with an empty string and parse it afterwards. That is no problem, but it can be dificult to do something like this if it is more complex.
It is sometimes no problem for an human to read corrupt data withut a valid format - but it is almost impossible for simple algorithms.
By the way, the formatting ".\"firstName" is a valid JSON element because the " is escaped by \. See this question too.

How to deserialize in C# an unknown JSON string to some Object

I need to parse in C# (key ,value wise) a string that is built in a JSON format (to be exact I need to parse the binding parameter of Knockout data-bind).
I go over the html file and I extract the bindings. I want to modify each and every binding (string-wise), but It's really hard for me to parse the string, since I can't really know where each binding stops and the other starts.
for example:
data-bind="text:'ggggg',event:{mouseover:x=function(){alert(1);return 'd,y'}}"
will result in the following string:
"text:'ggggg',event:{mouseover:x=function(){alert(1);return 'd,y'}}"
I want to modify the string in the following way:
newString= "text('gggg'),event(mouseover(x=function(){alert(1);return 'd,y'}))"
I figured out that the best way to do it is to deserialize the string by JSON and then it will be easier for me to get access to each and every binding element.
I write at C#, but since I go over the html file and each data-bind is different and can contain different amount and type of attributes I would like to have a general object that I can deserialize to.
I checked out DataContractJsonSerializer but I don't see how it solves my problem.
Can you please suggest me what's best for my case?
Mary

You can do it with something like this:
var obj = ko.bindingProvider.instance.getBindings(yourDomElement,
ko.contextFor(yourDomElement));
alert(JSON.stringify(obj));
And then do whatever you want with obj.
Fiddle
But... well... don't!

C# parse elements of an HTTP POST response object from a string

I have a set of files, each of which contain the full text of a series of HTTP POST responses. A number of these contain binary objects (e.g. images or PDFs). I've been trying to use regexes to extract the binary objects, but I can't seem to get it correctly. The HTTPListener class (and associated classes) all seem to require an active connection, i.e. parsing a real time request response pair, which I don't have. Is there a good library out there which can parse a file (or a string) as an HTTP response? If not, can anyone think of a better method for doing this than regex?
Thanks,
Rik

You can easily write your own Parser which does the following:
Reads the Response file line by line
Till the line Content Length, which specifies the number of bytes in Payload
Read the payload as binary
Image class has an overload which creates an image from a Stream. This way you can verify whether your result images matches the original image.
Regards