We Keep Coding

How to Submit String with 536000 Characters to API

I interpret a G code file (CNC language), serialize it into a class, and try to send in the http protocol to my API, which has a GET method.
However it is too long a string to be sent by Http.
Is there any solution to this problem? Something like compression?
Request URL Too Long
HTTP Error 414. The request URL is too long.
Using Asp.Net WebAPI

Try using POST on both sides. Doing so also makes more sense from a REST point of view, as you are submitting data, not doing a query.
Note that GET requests are usually very limited in size, see e.g. here: maximum length of HTTP GET request?

How safe or dangerous it is to use "UserAgent" of the request to identify the origin of the request?

I am developing an SMS service which sends SMS to the destination numbers using Twilio as an SMS provider. Twilio is suppose to send a POST request to my web service as and when the status of the message is updated (i.e., sent, delivered, etc).
In order to make sure that the POST request is not sent by anyone else than Twilio, I am validating UserAgent of the request as below.
If ((HttpRequest)request.OriginalRequest).UserAgent.StartsWith("TwilioProxy/"))
{
return true;
}
Currently I am getting "TwilioProxy/1.0" as User Agent in each of the POST action, where I believe the version number can be changed in future, so I have skipped it from validation.
Is it possible to receive a request with the same user agent (something starting to "TwilioProxy/") from any other origin than Twilio? Is it safe to rely on UserAgent for this type of verification?
Any inputs/ suggestions on this will be much helpful to me.
Thanks

Twilio developer evangelist here.
As the comments have mentioned, it is trivial to spoof a header and since the UserAgent header for Twilio is very simple, then it is unreliable to rely on it.
However, if you are interested in validating that requests are made by Twilio then you need to check out how we sign requests to ensure they are not malicious.
Here's how it works:
Turn on TLS on your server and configure your Twilio account to use HTTPS urls.
Twilio assembles its request to your application, including the final URL and any POST fields (if the request is a POST).
If your request is a POST, Twilio takes all the POST fields, sorts them by alphabetically by their name, and concatenates the parameter name and value to the end of the URL (with no delimiter).
Twilio takes the resulting string (the full URL with query string and all POST parameters) and signs it using HMAC-SHA1 and your AuthToken as the key.
Twilio sends this signature in an HTTP header called X-Twilio-Signature
Then to verify that this X-Twilio-Signature contains a valid signature, you need to do the following in your application:
Take the full URL of the request URL you specify for your phone number or app, from the protocol (https...) through the end of the query string (everything after the ?).
If the request is a POST, sort all of the POST parameters alphabetically (using Unix-style case-sensitive sorting order).
Iterate through the sorted list of POST parameters, and append the variable name and value (with no delimiters) to the end of the URL string.
Sign the resulting string with HMAC-SHA1 using your AuthToken as the key (remember, your AuthToken's case matters!).
Base64 encode the resulting hash value.
Compare your hash to ours, submitted in the X-Twilio-Signature header. If they match, then you're good to go.
Within our official libraries, we include a request validator that can do all of this for you. There is an example of doing this in C# in the documentation.
Let me know if this helps at all.

How to post data from C# to an ajax website?

I work in C# and so far I used WebRequest method to GET and POST data. I use Fiddler to check what is the browser doing and I got to a point where the data is retrieved from Ajax after posting some data.
I am not sure if I have to add to my project a javascript page or what and what code do I need in the javascript file and how to call it.
In essence, I have to post the data {"name":"ABCD"} to url www.example.com/Website.AJAX,Website.ashx.

Ajax is not so different from ordinary request, so you can just post it as usual. Most likely problem is how the backend treat that it is an ajax request (if it does at all).
As it looks like you are using the WebForms there on backend, you just need to add a special header most likely (X-Requested-With). Some frameworks add it, though it's not a real requirement of the ajax request.
All in all I would just post an ordinary request with WebRequest as you did before. If that does not work, you need to study the original request from web UI to see what is different. E.g. a special header or request Content-Type is JSON or something like that.
P.S. If you use JSON in the body it's better to explicitly set content type to application/json; charset=utf-8 unless there is something special with the server.

WebAPI get request parameters being UrlDecoded at the controller

I have an issue where I'm issuing a GET to a WebAPI controller, essentially:
$.getJSON('/api/feefo/getproductfeedback?id='+ encodeURIComponent(skuNum))
I'm using encodeURIComponent to url encode the skuNum parameter, viewing a request in dev tools I get the expected result for a skuNum that needs to be encoded:
The skuNum has gone from 1000EF+ to 1000EF%2B as expected.
However, when I view the id parameter in the WebAPI controller, it's coming through un-encoded:
It's as though the client side url encoding is being undone somehow, can anyone explain what's going on here? Obviously I can work around this by just doing the encoding in the controller, but I'd like to understand why this is happening.

That is by design. The API framework will decode the URL encoded parameters by default. the encoding should only be used for transporting the data. once on server developer shouldn't have to deal with having to decode it (cross cutting concern). Use the value as intended.

What is the correct encoding for querystrings?

I am trying to send a request to an url like this "http://mysite.dk/tværs?test=æ" from an asp.net application, and I am having trouble getting the querystring to encode correctly. Or maybe the querystring is encoded correctly, the service I am connecting to just doesn't understand it correctly.
I have tried to send the request with different browsers and logging how they encode the request with Wireshark, and I get these results:
Firefox: http://mysite.dk/tv%C3%A6rs?test=%E6
Ie8: http://mysite.dk/tv%C3%A6rs?test=\xe6
Curl: http://mysite.dk/tv\xe6rs?test=\xe6
Both Firefox, IE and Curl receive the correct results from the service. Note that they encode the danish special character 'æ' differently in the querystring.
When I send the request from my asp.net application using HttpWebRequest, the URL gets encoded this way:
http://mysite.dk/tv%C3%A6rs?test=%C3%A6
It encodes the querystring the same way as the path part of the url. The remote service does not understand this encoding, so I don't get a correct answer.
For the record, 'æ' (U+00E6) is %E6 in ISO-LATIN-1, and %C3%A6 in UTF-8.
I could change the remote service to accept the UTF-8 encoded querystring, but then the service would stop working in browsers and I am not really interested in that. Is there a way to specify to .NET that it shouldn't encode querystrings with UTF-8?
I am creating the webrequest like this:
var req = WebRequest.Create("http://mysite.dk/tværs?test=æ") as HttpWebRequest;
But the problem seems to originate from System.Uri which is apparently used inside WebRequest.Create:
var uri = new Uri("http://mysite.dk/tværs?test=æ");
// now uri.AbsolutePath == "http://mysite.dk/tv%C3%A6rs?test=%C3%A6"

It looks like you're applying UrlEncode over the entire URL - this isn't correct, paths and query strings are encoded differently as you've seen. What is doing the encoding of the URI, WebRequest?
You could manually build the various parts using a UriBuilder, or manually encode using UrlPathEncode for the path and UrlEncode for the query string names and values.
Edit:
If the problem lies in the path, rather than the query string you could try turning on IRI support, via web.config
<configuration>
<uri>
<iriParsing enabled="true" />
</uri>
</configuration>
That should then leave international characters alone in the path.

Have you tried the UrlEncode?
http://msdn.microsoft.com/en-us/library/zttxte6w.aspx

I ended up changing my remote webservice to expect the querystring to be UTF-8 encoded. It solves my immediate problem, the webservice can not be correctly called by both PHP and the .NET framework.
However, the behavior is now strange in browsers. Copy pasting an url like "http://mysite.dk/tv%C3%A6rs?test=%C3%A6" into the browser and then pressing return works, it even corrects the encoded characters and displays the location as "http://mysite.dk/tværs?test=æ". If then reload the page (F5) it still works. But if I click on the location bar and press return again, the querystring will become encoded with latin-1 and fail.
For anyone interested here is an old Firefox bugreport about the problem: https://bugzilla.mozilla.org/show_bug.cgi?id=284474 (thanks to #dtb)
So, it seems there is no good solution.
Thanks to everyone who helped though!