I am calling an API from by C# Windows service. In some cases the following error is being raised.
The request body did not contain the specified number of bytes. Got 101,379, expected 102,044
In the RAW Request captured using fiddler content length as specified.
Content-Length: 102044
In the response from the API I am receiving the following message.
The request body did not contain the specified number of bytes. Got 101,379, expected 102,044
The strange thing for me is that it does not happen for each and every request, it is generated randomly and different points. Code which I am using to get the content length is specified below.
var data = Encoding.ASCII.GetBytes(requestBody); // requestBody is the JSON String
webReqeust.ContentLength = data.Length;
Is it mandatory to provide content length in REST API calls ?
Edit 1:
This is what my sample code looks like for web request
webReqeust = (HttpWebRequest)WebRequest.Create(string.Format("{0}{1}", requestURI, queryString));
webReqeust.Method = RequestMethod.ToString();
webReqeust.Headers.Add("Authorization", string.Format("{0} {1}", token_type, access_token));
webReqeust.Method = RequestMethod.ToString();
webReqeust.ContentType = "application/json";
var data = Encoding.ASCII.GetBytes(requestBody);
webReqeust.ContentLength = data.Length;
using (var streamWriter = new StreamWriter(webReqeust.GetRequestStream()))
{
streamWriter.Write(requestBody);
streamWriter.Flush();
streamWriter.Close();
}
I would suggest maybe instead try using HttpClient as done in the linked post from mjwills here. You don't have to use content length, but it sounds like that is being enforced by the API and ultimately you are trying to post too much.
Otherwise the way I see it is that something is making the request body too large. Is it serialized input data which gets encoded into a byte array? If that is what is happening then perhaps the correct length requirements are not being enforced on the data that composes the request body, and I would suggest inspecting what goes on in the composition of the request body object itself.
Related
I have an AspNet.Core.Mvc REST API that is routed through a proxy gateway. The gateway has a limit of 10mb for the response body. If the body is over that threshold it returns 500 Internal Server Error.
My goal:
To check the size of the response body before returning from my API project and if it is over that threshold, return a BadRequest error instead, with a helpful error message in the response body content. Something like, "The response is too large. Try narrowing your request."
I've tried handling this in middleware, where the byte size of the body is known, but this is too late and the framework prevents this. I've tried working around this, something like this where you swap out the Body temporarily with a MemoryStream:
Modify middleware response
However, the status code is already set to 200 at that point and the framework throws an error if you try to change it. Here is the code snippet from the middleware where it copies the new response stream to the original body:
// Replace the body with a BadRequest error notifying user that the response
// was too large and they should narrow down their search parameters.
var errorStream = new MemoryStream();
var sw = new StreamWriter(errorStream);
sw.Write("Response exceeds maximum size. Try narrowing request parameters or setting a smaller page size if applicable.");
sw.Flush();
errorStream.Position = 0;
await errorStream.CopyToAsync(originalBody);
httpContext.Response.StatusCode = (int)HttpStatusCode.BadRequest; // throws exception!
I've also tried to handle it in the Controller method itself before returning, by serializing the response content to json, then converting the json string to bytes to check the size:
string json = JsonConvert.SerializeObject(value, jsonSerializerSettings);
byte[] bytes = Encoding.UTF8.GetBytes(json);
But that is all work that the framework will do for me, and seems wasteful (and frankly ugly). Is there a way to set this as the response and return it from the Controller? Maybe I can create a base class for the Controller that all API methods can use that will do this logic?
This all seems like a lot of trouble for something seemingly simple. I'm hoping someone here has a better solution.
I have some C# code that does the following post request:
string postData = newFormUrlEncodedContent(params).ReadAsStringAsync().Result;
var postContent = new StringContent(postData, UTF8Encoding.UTF8, "application/x-www-form-urlencoded");
var responseMessage = httpClient.PostAsync(url, postContent).Result;
I would like to do the equivalent in Python using the Requests library. This is what I have:
headers = {'content-type':'application/x-www-form-urlencoded'}
postContent = requests.post(url, params=params, headers=headers, cookies=previousResponse.cookies)
But postContent.status_code for the Python code is 404, whereas the C# request returns 200. It's possible that there's something wrong with the params since I retrieve those via some Regex matching from a previous request, but that seems to be working.
Edit: I think setting the params parameter is for get requests, not post requests. Also I believe Requests takes care of form encoding:
Typically, you want to send some form-encoded data — much like an HTML form. To do this, simply pass a dictionary to the data argument. Your dictionary of data will automatically be form-encoded when the request is made
So now I have:
postContent = requests.post(url, data = params, cookies = previousResponse.cookies)
Now postContent.status_code == 500. The stack trace says the data is invalid at the root level. I will look into it.
I am making a portal for our team's Slack channel, and I'm wanting to pull a list of all of our current users using the Slack Web API - using the method api/users.list.
The code I am playing around with is:
var response = client.UploadValues("https://slack.com/api/users.list",
"POST", new NameValueCollection()
{
{ "token" ,"mySecretToken"}
});
I get an OK response back, but I'm having trouble actually finding out how to pull the data I want. When I look at the response object all I have is an array of bytes.
What am I missing to actually pull back an object with the user list information?
I needed to encode the response.
Just add this after:
var encoding = new UTF8Encoding();
var responseText = encoding.GetString(response);
I can then use the responseText to pull out the data I need.
I have a library that I wrote a while ago that allows me to post a new status to Twitter. So handling of the OAuth headers etc is all working.
However, I now have a requirement to upload an image using the Twitter REST API:
https://upload.twitter.com/1.1/media/upload.json
When posting a status I normally put the following in the request stream 'status=<my tweet here>'
Ideally I want to post the raw image data rather than a Base64 string, however, I am having issues with each of them working.
According to Twitter, the OAuth should only be build up from the keys starting 'oauth_' - I am only putting the following in:
parameters.Add("oauth_consumer_key", consumerKey);
parameters.Add("oauth_signature_method", "HMAC-SHA1");
parameters.Add("oauth_timestamp", Base.Methods.UNIXTimestamp);
parameters.Add("oauth_nonce", Guid.NewGuid().ToString().Replace("-", ""));
parameters.Add("oauth_version", "1.0");
parameters.Add("oauth_token", token);
Twitter says that when in doubt to use a content type of application/octet-stream - when doing this, I get a response of:
Code: 38
Message: Missing Parameter Media
In fact, I also get the same response when setting the content type to multipart/form-data as suggested in other pages from Twitter
I have tried various combinations of add the image data to the webrequest, and all seem to fail.
media=<my image byte data here>
Add header of 'media' with image data in the request
and as many other combinations I can think of. I even get the same issues when trying to send the Base64 version (which I'd rather not do).
Having read through lots of other questions I don't seem to be able to see what I am doing wrong.
Can anyone help?
I have found the problem, and it all comes down to the content being sent in the request stream.
As twitter says, only send in the oauth_xxxx parameters, the content based ones are not needed for this request.
The trick is the building up of the multipart form, I have done the following:
I am using an HttpWebrequest, so set it up to send the multipart form headers:
string boundary = "----" + DateTime.UtcNow.Ticks.ToString("x");
webRequest.ContentType = "multipart/form-data; boundary=" + boundary;
Build the content to be sent, which has both a prefix and a suffix to the actual image data
StringBuilder prefixData = new StringBuilder();
prefixData.Append("--");
prefixData.Append(boundary);
prefixData.Append(Environment.NewLine);
prefixData.Append("Content-Disposition: form-data; name=\"media\"");
prefixData.Append(Environment.NewLine);
prefixData.Append(Environment.NewLine);
byte[] prefix = Encoding.UTF8.GetBytes(prefixData.ToString());
StringBuilder suffixData = new StringBuilder();
suffixData.Append(Environment.NewLine);
suffixData.Append("--");
suffixData.Append(boundary);
suffixData.Append("--");
byte[] suffix = Encoding.UTF8.GetBytes(suffixData.ToString());
Join each of the data sections together to post to the stream:
byte[] data = new byte[prefix.Length + imageData.LongLength + suffix.Length];
Buffer.BlockCopy(prefix, 0, data, 0, prefix.Length);
Buffer.BlockCopy(imageData, 0, data, prefix.Length, imageData.Length);
Buffer.BlockCopy(suffix, 0, data, prefix.Length + imageData.Length, suffix.Length);
Write the data to the request stream as normal.
I am trying to upload some avi file to server. It works fine with HttpRequest but i need to continue uploading even if i suspend app so thats why i am trying to use BackgroundUploader. I am following this guideline on msdn http://msdn.microsoft.com/en-us/library/windows/apps/jj152727.aspx. So my code looks something like this.
StorageFile storageFile = KnownFolders.VideosLibrary.GetFileAsync("fileName");
BackgroundUploader uploader = new BackgroundUploader();
uploader.Method = "POST";
uploader.SetRequestHeader("Content-Type", "multipart/form-data; boundary=" + boundary);
var fs = await storageFile.OpenAsync(Windows.Storage.FileAccessMode.ReadWrite);
IInputStream aaaa = fs.GetInputStreamAt(0);
UploadOperation upload = uploader.CreateUploadFromStreamAsync(new Uri("uploadUri"), aaaa);
await HandleUploadAsync(upload, true);
the rest is same as on MSDN. And i am getting exception Unsupported media type (415) in method HandleUploadAsync on line
await upload.StartAsync().AsTask(cts.Token, progressCallback);
What am i doing wrong? Or what can cause this kind of exception?
EDIT : I solved my problem as i commented down here and in my answer. I think at the end i am basically just sending some data to server that are recognized and interpreted as i want to. So if i use BackgroundUploader i am not only uploading some file i am also sending information about how am i doing that(as i mentioned in my answer). So by the same idea i can also upload folder to server and by that i am not sending any actual content only some description about how to do that. And if i compare request that i am making by HttpRequest and BackgroundUploader they are equal and thats what i wanted.
So the problem part was the header of request. I have some header in my request that is recognized by server and i was trying to put it to BackgroundUploader through SetRequestheader method but it did not work. As Kieqic suggested i used Fiddler and by that i compare request made by HttpRequest and BackgroundUploader. I found out they are completely different. So through SetRequestheader i add some parts like expected Content-Type and for the rest parts of header to make them equal i add it before content of my file as array of bytes. And this works so conclusion is in my case using Fiddler that helped my how to construct request header.