I have a library that I wrote a while ago that allows me to post a new status to Twitter. So handling of the OAuth headers etc is all working.
However, I now have a requirement to upload an image using the Twitter REST API:
https://upload.twitter.com/1.1/media/upload.json
When posting a status I normally put the following in the request stream 'status=<my tweet here>'
Ideally I want to post the raw image data rather than a Base64 string, however, I am having issues with each of them working.
According to Twitter, the OAuth should only be build up from the keys starting 'oauth_' - I am only putting the following in:
parameters.Add("oauth_consumer_key", consumerKey);
parameters.Add("oauth_signature_method", "HMAC-SHA1");
parameters.Add("oauth_timestamp", Base.Methods.UNIXTimestamp);
parameters.Add("oauth_nonce", Guid.NewGuid().ToString().Replace("-", ""));
parameters.Add("oauth_version", "1.0");
parameters.Add("oauth_token", token);
Twitter says that when in doubt to use a content type of application/octet-stream - when doing this, I get a response of:
Code: 38
Message: Missing Parameter Media
In fact, I also get the same response when setting the content type to multipart/form-data as suggested in other pages from Twitter
I have tried various combinations of add the image data to the webrequest, and all seem to fail.
media=<my image byte data here>
Add header of 'media' with image data in the request
and as many other combinations I can think of. I even get the same issues when trying to send the Base64 version (which I'd rather not do).
Having read through lots of other questions I don't seem to be able to see what I am doing wrong.
Can anyone help?
I have found the problem, and it all comes down to the content being sent in the request stream.
As twitter says, only send in the oauth_xxxx parameters, the content based ones are not needed for this request.
The trick is the building up of the multipart form, I have done the following:
I am using an HttpWebrequest, so set it up to send the multipart form headers:
string boundary = "----" + DateTime.UtcNow.Ticks.ToString("x");
webRequest.ContentType = "multipart/form-data; boundary=" + boundary;
Build the content to be sent, which has both a prefix and a suffix to the actual image data
StringBuilder prefixData = new StringBuilder();
prefixData.Append("--");
prefixData.Append(boundary);
prefixData.Append(Environment.NewLine);
prefixData.Append("Content-Disposition: form-data; name=\"media\"");
prefixData.Append(Environment.NewLine);
prefixData.Append(Environment.NewLine);
byte[] prefix = Encoding.UTF8.GetBytes(prefixData.ToString());
StringBuilder suffixData = new StringBuilder();
suffixData.Append(Environment.NewLine);
suffixData.Append("--");
suffixData.Append(boundary);
suffixData.Append("--");
byte[] suffix = Encoding.UTF8.GetBytes(suffixData.ToString());
Join each of the data sections together to post to the stream:
byte[] data = new byte[prefix.Length + imageData.LongLength + suffix.Length];
Buffer.BlockCopy(prefix, 0, data, 0, prefix.Length);
Buffer.BlockCopy(imageData, 0, data, prefix.Length, imageData.Length);
Buffer.BlockCopy(suffix, 0, data, prefix.Length + imageData.Length, suffix.Length);
Write the data to the request stream as normal.
Related
I am calling an API from by C# Windows service. In some cases the following error is being raised.
The request body did not contain the specified number of bytes. Got 101,379, expected 102,044
In the RAW Request captured using fiddler content length as specified.
Content-Length: 102044
In the response from the API I am receiving the following message.
The request body did not contain the specified number of bytes. Got 101,379, expected 102,044
The strange thing for me is that it does not happen for each and every request, it is generated randomly and different points. Code which I am using to get the content length is specified below.
var data = Encoding.ASCII.GetBytes(requestBody); // requestBody is the JSON String
webReqeust.ContentLength = data.Length;
Is it mandatory to provide content length in REST API calls ?
Edit 1:
This is what my sample code looks like for web request
webReqeust = (HttpWebRequest)WebRequest.Create(string.Format("{0}{1}", requestURI, queryString));
webReqeust.Method = RequestMethod.ToString();
webReqeust.Headers.Add("Authorization", string.Format("{0} {1}", token_type, access_token));
webReqeust.Method = RequestMethod.ToString();
webReqeust.ContentType = "application/json";
var data = Encoding.ASCII.GetBytes(requestBody);
webReqeust.ContentLength = data.Length;
using (var streamWriter = new StreamWriter(webReqeust.GetRequestStream()))
{
streamWriter.Write(requestBody);
streamWriter.Flush();
streamWriter.Close();
}
I would suggest maybe instead try using HttpClient as done in the linked post from mjwills here. You don't have to use content length, but it sounds like that is being enforced by the API and ultimately you are trying to post too much.
Otherwise the way I see it is that something is making the request body too large. Is it serialized input data which gets encoded into a byte array? If that is what is happening then perhaps the correct length requirements are not being enforced on the data that composes the request body, and I would suggest inspecting what goes on in the composition of the request body object itself.
I'm using RestSharp to try to upload a video to Vimeo, but I keep getting an Http 413 RequestEntityTooLarge error. I think I'm sending just the bytes and not the encoded video, so I'm not sure what is wrong. This is my code:
//construct request
RestRequest request = new RestRequest(endpoint);
request.Method = Method.PUT;
//add headers
request.AddHeader("Authorization", string.Format("Bearer {0}", _accessToken));
request.AddHeader("Content-Length", fileSize.ToString());
request.AddHeader("Content-Type", mimeType);
request.AddParameter(mimeType, fileData, ParameterType.RequestBody);
//allow for the transfer of larger files (10min timeout)
request.Timeout = 2400000;
// Upload the file
IRestResponse uploadResponse = _client.Execute(request);
This has nothing to do with your code. Vimeo is telling you what's wrong; The file you're uploading is too large for them to accept.
Double check that your fileSize variable is correct. If incorrect, Vimeo may thing you're trying to upload a file that's much larger than it really is.
If the file is, in fact, too large, compress it using any number of file compressors and then upload that.
UPDATE: According to this forum post, you need to make sure you're not encoding your file in any way.
I am breaking my head trying to upload a feed to walmart, after many times trying i used postman to generate C# restsharp code for me, in postman it works, but when using the c# restsharp code it returns a mysterious error. like this:
"No message body writer has been found for response class FeedAcknowledgement"
what does that mean?
here is my code:
string requestUrl = "";
requestUrl = string.Format("https://marketplace.walmartapis.com/v2/feeds?feedType=inventory");
string method = "POST";
// string[] sig = getSig(method, requestUrl).Replace("\r", "").Split('\n');
var mySig = new Signature(ConsumerID, SecretKEY, requestUrl, method);
var s = mySig.TimeStamp;
var returendSigniture = mySig.GetSignature(s);
var client = new RestClient("https://marketplace.walmartapis.com/v2/feeds?feedType=inventory");
var request = new RestRequest(Method.POST);
//request.AddHeader("postman-token", "c325ba5f-813a-f990-7899-6bfc4b14aa1b");
request.AddHeader("cache-control", "no-cache");
request.AddHeader("content-type", "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW");
request.AddHeader("accept", "application/xml");
request.AddHeader("wm_consumer.id", "--");
request.AddHeader("wm_sec.auth_signature", returendSigniture);
request.AddHeader("wm_sec.timestamp", mySig.TimeStamp);
request.AddHeader("wm_qos.correlation_id", "123456abcdef");
request.AddHeader("wm_svc.name", "Walmart Marketplace");
request.AddParameter("multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW", "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"BOUNDERY\"\r\n\r\n<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<wm:inventory xmlns:wm=\"http://walmart.com/\">\n <wm:sku>PP00500-2PC</wm:sku>\n <wm:quantity>\n <wm:unit>EACH</wm:unit>\n <wm:amount>120</wm:amount>\n </wm:quantity>\n <wm:fulfillmentLagTime>1</wm:fulfillmentLagTime>\n</wm:inventory>\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW--", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
I spent all day in figuring out how to request Walmart v3. I propose you the following two steps:
Use Walmart signer in order to generate signed token.
You will need to use HttpWebRequest for getting response from Walmart in a way similar to what is described here.
I have not been able to get this to work natively in C#, but I do have a work around.
The Java SDK can successfully submit multi-part requests to Walmart. I wrote a wrapper around the SDK functions that can accept basic command line input to read a text file and send the appropriate call with attached files. From here, you can just call the .jar file (I do it via dynamically generated batch file) from your C# program and receive responses back via text file. This is a sub-optimal system, but it works reliably and when the choice was between updating inventory on 2000 items every day and using some dirty code, I went with the Java wrapper method. This will be replaced as soon as the C# SDK comes out, but I believe this is one of the reasons why the C# SDK may be being delayed.
This solution was used, only after spending about a week trying to get boundaries / streams / attachments to work in C# and having zero success. Cases were also submitted to walmart and I was able to work with some of their top tier engineering support staff and this problem completely stumped them. I was able to trace the Java SDK execution all the way down to a built in Maven / Java function that constructed the web request so there's something under the hood that Java is doing with a multi-part request that isn't immediately clear in C#.
I am trying to upload some avi file to server. It works fine with HttpRequest but i need to continue uploading even if i suspend app so thats why i am trying to use BackgroundUploader. I am following this guideline on msdn http://msdn.microsoft.com/en-us/library/windows/apps/jj152727.aspx. So my code looks something like this.
StorageFile storageFile = KnownFolders.VideosLibrary.GetFileAsync("fileName");
BackgroundUploader uploader = new BackgroundUploader();
uploader.Method = "POST";
uploader.SetRequestHeader("Content-Type", "multipart/form-data; boundary=" + boundary);
var fs = await storageFile.OpenAsync(Windows.Storage.FileAccessMode.ReadWrite);
IInputStream aaaa = fs.GetInputStreamAt(0);
UploadOperation upload = uploader.CreateUploadFromStreamAsync(new Uri("uploadUri"), aaaa);
await HandleUploadAsync(upload, true);
the rest is same as on MSDN. And i am getting exception Unsupported media type (415) in method HandleUploadAsync on line
await upload.StartAsync().AsTask(cts.Token, progressCallback);
What am i doing wrong? Or what can cause this kind of exception?
EDIT : I solved my problem as i commented down here and in my answer. I think at the end i am basically just sending some data to server that are recognized and interpreted as i want to. So if i use BackgroundUploader i am not only uploading some file i am also sending information about how am i doing that(as i mentioned in my answer). So by the same idea i can also upload folder to server and by that i am not sending any actual content only some description about how to do that. And if i compare request that i am making by HttpRequest and BackgroundUploader they are equal and thats what i wanted.
So the problem part was the header of request. I have some header in my request that is recognized by server and i was trying to put it to BackgroundUploader through SetRequestheader method but it did not work. As Kieqic suggested i used Fiddler and by that i compare request made by HttpRequest and BackgroundUploader. I found out they are completely different. So through SetRequestheader i add some parts like expected Content-Type and for the rest parts of header to make them equal i add it before content of my file as array of bytes. And this works so conclusion is in my case using Fiddler that helped my how to construct request header.
I have a project at work the requires me to be able to enter information into a web page, read the next page I get redirected to and then take further action. A simplified real-world example would be something like going to google.com, entering "Coding tricks" as search criteria, and reading the resulting page.
Small coding examples like the ones linked to at http://www.csharp-station.com/HowTo/HttpWebFetch.aspx tell how to read a web page, but not how to interact with it by submitting information into a form and continuing on to the next page.
For the record, I'm not building a malicious and/or spam related product.
So how do I go read web pages that require a few steps of normal browsing to reach first?
You can programmatically create an Http request and retrieve the response:
string uri = "http://www.google.com/search";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
// encode the data to POST:
string postData = "q=searchterm&hl=en";
byte[] encodedData = new ASCIIEncoding().GetBytes(postData);
request.ContentLength = encodedData.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(encodedData, 0, encodedData.Length);
// send the request and get the response
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
// Do something with the response stream. As an example, we'll
// stream the response to the console via a 256 character buffer
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
Char[] buffer = new Char[256];
int count = reader.Read(buffer, 0, 256);
while (count > 0)
{
Console.WriteLine(new String(buffer, 0, count));
count = reader.Read(buffer, 0, 256);
}
} // reader is disposed here
} // response is disposed here
Of course, this code will return an error since Google uses GET, not POST, for search queries.
This method will work if you are dealing with specific web pages, as the URLs and POST data are all basically hard-coded. If you needed something that was a little more dynamic, you'd have to:
Capture the page
Strip out the form
Create a POST string based on the form fields
FWIW, I think something like Perl or Python might be better suited to that sort of task.
edit: x-www-form-urlencoded
You might try Selenium. Record the actions in Firefox using Selenium IDE, save the script in C# format, then play them back using the Selenium RC C# wrapper. As others have mentioned you could also use System.Net.HttpWebRequest or System.Net.WebClient. If this is a desktop application see also System.Windows.Forms.WebBrowser.
Addendum: Similar to Selenium IDE and Selenium RC, which are Java-based, WatiN Test Recorder and WatiN are .NET-based.
What you need to do is keep retrieving and analyzing the html source for each page in the chain. For each page, you need to figure out what the form submission will look like and send a request that will match that to get the next page in the chain.
What I do is build a custom class the wraps System.Net.HttpWebRequest/HttpWebResponse, so retrieving pages is as simple as using System.Net.WebClient. However, my custom class also keeps the same cookie container across requests and makes it a little easier to send post data, customize the user agent, etc.
Depending on how the website works you can either manipulate the url to perform what you want. e.g to search for the word "beatles" you could just open a request to google.com?q=beetles and then just read the results.
Alternatively if the website does not use querystring values (url) to process page actions then you will need to work on a webrequest which posts the required values to the website instead. Search in Google for working with WebRequest and webresponse.