Our application uses temporary queues to direct service bus responses to the originating caller. We use the built-in ServiceStack.RabbitMq.RabbitMqServer to publish and handle messages.
Message<IReturn<ResponseDto>> message = BuildMessage(requestDto);
// get the temporary queue for the current IMessageQueueClient
string queueName = messageclient.GetTempQueueName();
message.ReplyTo = queueName;
// publish the message
messageclient.Publish(message);
However, capturing the response directly (below) will fail if the call throws an exception.
IMessage<ResponseDto> responseMessage = messageclient.Get<ResponseDto>(queueName, timeOut);
messageclient.Ack(responseMessage);
ResponseDto response = responseMessage.GetBody();
The body of the response message will be a ServiceStack.ErrorResponse causing responseMessage.GetBody() to return an empty object. And the error is not returned as responseMessage.Error.
We get around this by getting the body of the message as the raw JSV string and validating the result.
IMessage<string> responseMessage = messageclient.Get<string>(queueName, timeOut);
messageclient.Ack(responseMessage);
// get the body of the message as a string
string messageBody = responseMessage.GetBody();
// parse as error response
var error = messageBody.FromJsv<ErrorResponse>();
// if no response status, assume good
if (error?.ResponseStatus != null)
throw new Exception(error.ResponseStatus.Message);
// parse as response and return
return messageBody.FromJsv<ResponseDto>();
This works for most cases, however, if a string value in the returning RabbitMq message JSON contains a comma, the string is not contained in quotes in the message body JSV (this seems to be an issue in ServiceStack.Text when converting from JSON -> JSV using the JsonTypeSerializer)
which results in that field being incomplete, and the subsequent field being ignored.
Is there a better way to retrieve the exception? Or is there a workaround for the serialization issue?
There should be no converting JSON into JSV, they're different formats. JSV uses CSV-style encoding and escaping which is illegal in JSON which requires all string values to be quoted and strings escaped with \.
Essentially you should be deserializing the message using the exact Serializer and Type used to serialize it, if you do this there shouldn't be any issues.
ServiceStack's Rabbit MQ support serializes complex Type payloads as JSON so it's not clear where the JSV is coming from.
If you need to access the raw text payload you should use Rabbit MQ's low-level APIs instead of ServiceStack's RabbitMQ high-level APIs which automatically tries to deserialize complex types messages behind-the-scenes.
To get the raw text body you can do something like:
var rabbitMqClient = messageclient as RabbitMqProducer;
var mqResult = rabbitMqClient.GetMessage(queueName, noAck: false);
var props = msgResult.BasicProperties;
var bodyString = msgResult.Body.FromUtf8Bytes(); //normally JSON
Related
I have an AspNet.Core.Mvc REST API that is routed through a proxy gateway. The gateway has a limit of 10mb for the response body. If the body is over that threshold it returns 500 Internal Server Error.
My goal:
To check the size of the response body before returning from my API project and if it is over that threshold, return a BadRequest error instead, with a helpful error message in the response body content. Something like, "The response is too large. Try narrowing your request."
I've tried handling this in middleware, where the byte size of the body is known, but this is too late and the framework prevents this. I've tried working around this, something like this where you swap out the Body temporarily with a MemoryStream:
Modify middleware response
However, the status code is already set to 200 at that point and the framework throws an error if you try to change it. Here is the code snippet from the middleware where it copies the new response stream to the original body:
// Replace the body with a BadRequest error notifying user that the response
// was too large and they should narrow down their search parameters.
var errorStream = new MemoryStream();
var sw = new StreamWriter(errorStream);
sw.Write("Response exceeds maximum size. Try narrowing request parameters or setting a smaller page size if applicable.");
sw.Flush();
errorStream.Position = 0;
await errorStream.CopyToAsync(originalBody);
httpContext.Response.StatusCode = (int)HttpStatusCode.BadRequest; // throws exception!
I've also tried to handle it in the Controller method itself before returning, by serializing the response content to json, then converting the json string to bytes to check the size:
string json = JsonConvert.SerializeObject(value, jsonSerializerSettings);
byte[] bytes = Encoding.UTF8.GetBytes(json);
But that is all work that the framework will do for me, and seems wasteful (and frankly ugly). Is there a way to set this as the response and return it from the Controller? Maybe I can create a base class for the Controller that all API methods can use that will do this logic?
This all seems like a lot of trouble for something seemingly simple. I'm hoping someone here has a better solution.
I am calling an API from by C# Windows service. In some cases the following error is being raised.
The request body did not contain the specified number of bytes. Got 101,379, expected 102,044
In the RAW Request captured using fiddler content length as specified.
Content-Length: 102044
In the response from the API I am receiving the following message.
The request body did not contain the specified number of bytes. Got 101,379, expected 102,044
The strange thing for me is that it does not happen for each and every request, it is generated randomly and different points. Code which I am using to get the content length is specified below.
var data = Encoding.ASCII.GetBytes(requestBody); // requestBody is the JSON String
webReqeust.ContentLength = data.Length;
Is it mandatory to provide content length in REST API calls ?
Edit 1:
This is what my sample code looks like for web request
webReqeust = (HttpWebRequest)WebRequest.Create(string.Format("{0}{1}", requestURI, queryString));
webReqeust.Method = RequestMethod.ToString();
webReqeust.Headers.Add("Authorization", string.Format("{0} {1}", token_type, access_token));
webReqeust.Method = RequestMethod.ToString();
webReqeust.ContentType = "application/json";
var data = Encoding.ASCII.GetBytes(requestBody);
webReqeust.ContentLength = data.Length;
using (var streamWriter = new StreamWriter(webReqeust.GetRequestStream()))
{
streamWriter.Write(requestBody);
streamWriter.Flush();
streamWriter.Close();
}
I would suggest maybe instead try using HttpClient as done in the linked post from mjwills here. You don't have to use content length, but it sounds like that is being enforced by the API and ultimately you are trying to post too much.
Otherwise the way I see it is that something is making the request body too large. Is it serialized input data which gets encoded into a byte array? If that is what is happening then perhaps the correct length requirements are not being enforced on the data that composes the request body, and I would suggest inspecting what goes on in the composition of the request body object itself.
I have set the output of Azure stream analytics job to service bus queue which sends the data in JSON serialized format. When I receive the queue message in python script, along with the data in curly braces, I get #strin3http//schemas.microsoft.com/2003/10/Serialization/� appended in front. I am not able to trim it as the received message is not being recognized as either a string or a message. Because of this I cannot de-serialize the data.
This TechNet article suggests the following code:
// Get indices of actual message
var start = jsonString.IndexOf("{");
var end = jsonString.LastIndexOf("}") + 1;
var length = end - start;
// Get actual message
string cleandJsonString = jsonString.Substring(start, length);
Pretty primitive but whatever works I suppose...
The issue was similiar with the SO thread Interoperability Azure Service Bus Message Queue Messages.
Per my experience, the data from Azure Stream Analytics to Service Bus was sent via AMQP protocol, but the protocol of receiving the data in Python is HTTP. The excess content was generated by AMQP during transmission.
Assumption that receiving the message via the code below, please see https://azure.microsoft.com/en-us/documentation/articles/service-bus-python-how-to-use-queues/#receive-messages-from-a-queue. The function receive_queue_message with the False value of the argument peek_lock wrapped the REST API Receive and Delete Message (Destructive Read).
msg = bus_service.receive_queue_message('taskqueue', peek_lock=False)
print(msg.body)
According to the source code of Azure Service Bus SDK for Python include the functions receive_queue_message, read_delete_queue_message and _create_message, I think you can directly remove the excess content from the msg.body using the string common function lstrip or strip.
I ran into this issue as well. The previous answers are only workarounds and do not fix the root cause of this issue. The problem you are encountering is likely due to your Stream Analytics compatibility level. Compatibility level 1.0 uses an XML serializer producing the XML tag you are seeing. Compatibility level 1.1 "fixes" this issue.
See my previous answer here: https://stackoverflow.com/a/49307178/263139.
I had the same issue but in a .net solution. I was writing a service which sends data to a queue, and on the other hand, I was writing a service which gets that data from the queue. I've tried to send a JSON, like this:
var documentMessage = new DocumentMessage();
var json = JsonConvert.SerializeObject(documentMessage);
BrokeredMessage message = new BrokeredMessage(json);
await _client.SendAsync(message);
In this second service I was getting the JSON but with this prefix:
#strin3http//schemas.microsoft.com/2003/10/Serialization/�
I solved this problem by add DataContractJsonSerializer like that:
var documentMessage = new DocumentMessage();
var serializer = new DataContractJsonSerializer(typeof(DocumentMessage));
BrokeredMessage message = new BrokeredMessage(documentMessage , serializer);
await _client.SendAsync(message);
If you want to solve the problem in that way, you will have to add Data Attributes from System.Runtime.Serialization to the model:
[DataContract]
public class DocumentMessage
{
[DataMember]
public string Property1 { get; private set; }
[DataMember]
public string Property2 { get; private set; }
}
When using Microsoft.ServiceBus nuget package, replace
message.GetBody<Stream>();
with
message.GetBody<string>();
I am sending a message from a C# worker to the Queue, then I am getting it on another C# worker and call
string body = message.GetBody<string>();
This works and I later de-serialize the string/JSON message.
Now I am trying to send the same message from NodeJS in form of a JSON message. When I try to receive it and
string body = message.GetBody<string>();
call this I get an exception saying the input is in incorrect format.
My message object on NodeJS looks like this
{
body: JSON.stringify(message)
}
Any ideas?
Got it fixed!
By default the .NET Azure Queue library uses a DataContractSerializer and a binary XmlDictionaryWriter to serialize the string message when using
new BrokeredMessage("my message");
So instead you need to use this
new BrokeredMessage(new MemoryStream(Encoding.UTF8.GetBytes("my message")), true);
and to read the message in C# you need to use
string body = new StreamReader(message.GetBody<Stream>(), Encoding.UTF8).ReadToEnd();
I also stopped wrapping my JSON.stringify message in an object and pass it directly to the sendQueueMessage. The send message code looks like this now:
serviceBusService.sendQueueMessage('my_queue', JSON.stringify("my message"), function(error){});
JSON.stringify outputs a UTF8 string so it is fully compatible with the C# code.
I have a json object which I serialize and post. I found that if one of the string members of my json object has a few unicode characters such as é, ó, 6̄ (I have some other unicode characters that �犞ݮ do not return an HTTP 400 error) that the request returns HTTP 400 Bad Request.
Here is the simplified code for how the call is being made:
WebClient client;
Foo myObject = new Foo();
myObject.StringField1 = "é";
string serializedObjects = Newtonsoft.Json.JsonConvert.SerializeObject(myObject)
client.Headers[HttpRequestHeader.ContentType] = "application/json;charset=utf-8";
var response = client.UploadString(url, serializedObjects);
Here is the code for how the calls is being received on server side:
public async Task<IHttpActionResult> Post([FromBody] IList<Foo> myObjects){}
Here are some things that I have researched/tried and some assumptions I have made:
1) Get the string object as UTF8 bytes, then transform it back to a string
Encoding.UTF8.GetString(Encoding.UTF8.GetBytes(serializedObjects))
2) When I make the request and capture the traffic using Fiddler and inspect the request in Raw. For the unicode characters that don't return a 400 error they are replaced with a '?' however for the ones that do cause a 400 they are displayed as a blank. However when I look at JSON view it shows as a null character '�'
3) HTTP 400 is returned is ASP Web API can't deserialize the object correctly, so I tried using Newtonsoft library to serialize and deserialize the object and this works just fine. (The default serializer for ASP Web API is JSON.NET so it should be performing the same serialization http://www.asp.net/web-api/overview/formats-and-model-binding/json-and-xml-serialization)
string b = JsonConvert.SerializeObject(a);
Foo c = (Foo) JsonConvert.DeserializeObject(b, typeof(Foo));
// And I tried deserializing using the generic method
Foo d = JsonConvert.DeserializeObject<Foo>(b);
4) I was convinced that this was an issue with my server rejecting these special characters, but when I used Fiddler to retry the request (inserting the é back in the place where Fiddler had removed it from in the original request) the call succeeds.
5) I validated my Content-Type on http://www.iana.org/assignments/media-types/media-types.xhtml and also specify the utf8 charset explicitly.
So this seems to me to be an encoding issue, but é should be a valid UTF-8 character. I validated it in Jon Skeet's Unicode Explorer widget on his page. So I'm out of ideas for why this is causing issues...
As #roeland and #dbc pointed out there is an Encoding property on the WebClient. Setting this to Encoding.UTF8 resolved the issue.
client.Encoding = Encoding.UTF8;