We Keep Coding

Posting request to Amazon MWS services

I have been wasting few days on this issue already...
The main problem is - I dont know how to POST data.
I would like to start by getting OrderReferenceDetails.
by looking at MWS Scratchpad (API Section: Off Amazon payments, Operation: GetOferenceDetails).
As I understand, first thing I should do, is to calculate signature. So by using this Amazon developer guide, and taking values in example in scratchpad, I created string, from which I calculated signature. This string looks like this
POST
mws.amazonservices.com
/offamazonpayments_sandbox/2013-01-01
Action=GetOrderReferenceDetails&AmazonOrderReferenceId=[_my order reference id_]&AWSAccessKeyId=[_my access key_]&SellerId=[_my seller id_]&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2014-11-04T12%3a15%3a38.1988397Z&Version=2013-01-01
So I hash this and get a signature, then I should add it to line of query parameters like &Signature=[_my clalculated signature]
Function I use to calculate signature:
private static String sign(String data, String secretKey)
{
String result = null;
KeyedHashAlgorithm algorithm = new HMACSHA256();
Encoding encoding = new UTF8Encoding();
algorithm.Key = encoding.GetBytes(secretKey);
result = Convert.ToBase64String(algorithm.ComputeHash(encoding.GetBytes(data.ToCharArray())));
return result;
}
Problem start here (I think). Scratchpad provides "data" to post, but I cant understand, how to use it (copy-paste from scratchpad):
POST /OffAmazonPayments_Sandbox/2013-01-01?AWSAccessKeyId=
&Action=GetOrderReferenceDetails
&SellerId=
&SignatureVersion=2
&Timestamp=2014-11-04T12%3A37%3A58Z
&Version=2013-01-01
&Signature=rEqRKP27Pklu%2BAmRLR%2ByRpUtfhNsVOWuGTQ7s%2FgkB2w%3D
&SignatureMethod=HmacSHA256
&AmazonOrderReferenceId= HTTP/1.1
Host: mws.amazonservices.com
x-amazon-user-agent: AmazonJavascriptScratchpad/1.0 (Language=Javascript)
Content-Type: text/xml
What is this? I am familiar of making request like this:
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.Method = "POST";
request.ContentType = "text/xml";
request.Host = "whatever host I need to use";
request.ContentLength = Encoding.UTF8.GetByteCount(dataAsString);
byte[] byteArray = Encoding.UTF8.GetBytes(dataAsString);
using (Stream dataStream = request.GetRequestStream())
{
dataStream.Write(byteArray, 0, byteArray.Length);
}
try
{
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
using (var streamReader = new StreamReader(response.GetResponseStream()))
{
string responseAsString = streamReader.ReadToEnd();
}
}
catch (WebException exception)
{
HttpWebResponse exceptionalResponse = exception.Response as HttpWebResponse;
using (var streamReader = new StreamReader(exceptionalResponse.GetResponseStream()))
{
String responseAsString = streamReader.ReadToEnd();
}
}
Questions are:
Am I calculating signature properly?
How to construct web request?
Thanks for reading and trying to help!
EDIT:
Well, I finally got a response from MWS Amazon, that is not an error!
Steps I did to make it work:
Construct string that will be signed by hand - order of query parameters is important.
Construct url where I will post data by hand - again, order is important, but different from string signed before. Keys in signed string should be sorted using lexicographic byte ordering. Well, I dont know what kind of animal is that, so thats why I did everything by hand, by example taking scratchpad.

I struggled with this for a few days and it came down to the order of the parameters. I tried to find a solution to order the parameters without doing it by hand and finally found the answer. I created a SortedDictionary() at first still did not work. Then I saw an example from Claytondus nuget package that works with MWS and he had SortedDictionary(StringComparer.Ordinal). Once I did that it sorted correctly and got a good response.

GetRequestStream() is throwing time out exception when posting data to HTTPS url

I'm calling an API hosted on Apache server to post data. I'm using HttpWebRequest to perform POST in C#.
API has both normal HTTP and secure layer (HTTPS) PORT on the server. When I call HTTP URL it works perfectly fine. However, when I call HTTPS it gives me time-out exception (at GetRequestStream() function). Any insights? I'm using VS 2010, .Net framework 3.5 and C#. Here is the code block:
string json_value = jsonSerializer.Serialize(data);
HttpWebRequest request = (HttpWebRequest)System.Net.WebRequest.Create("https://server-url-xxxx.com");
request.Method = "POST";
request.ProtocolVersion = System.Net.HttpVersion.Version10;
request.ContentType = "application/x-www-form-urlencoded";
byte[] buffer = Encoding.ASCII.GetBytes(json_value);
request.ContentLength = buffer.Length;
System.IO.Stream reqStream = request.GetRequestStream();
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
EDIT:
The console program suggested by Peter works fine. But when I add data (in JSON format) that needs to be posted to the API, it throws out operation timed out exception. Here is the code that I add to console based application and it throws error.
byte[] buffer = Encoding.ASCII.GetBytes(json_value);
request.ContentLength = buffer.Length;

I ran into the same issue. It seems like it is solved for me. I went through all my code making sure to invoke webResponse.Close() and/or responseStream.Close() for all my HttpWebResponse objects. The documentation indicates that you can close the stream or the HttpWebResponse object. Calling both is not harmful, so I did. Not closing the responses may cause the application to run out of connections for reuse, and this seems to affect the HttpWebRequest.GetRequestStream as far as I can observe in my code.

I don't know if this will help you with your specific problem but you should consider Disposing some of those objects when you are finished with them. I was doing something like this recently and wrapping stuff up in using statements seems to clean up a bunch of timeout exceptions for me.
using (var reqStream = request.GetRequestStream())
{
if (reqStream == null)
{
return;
}
//do whatever
}
also check these things
Is the server serving https in your local dev environment?
Have you set up your bindings *.443 (https) properly?
Do you need to set credentials on the request?
Is it your application pool account accessing the https resources or is it your account being passed through?
Have you thought about using WebClient instead?
using (WebClient client = new WebClient())
{
using (Stream stream = client.OpenRead("https://server-url-xxxx.com"))
using (StreamReader reader = new StreamReader(stream))
{
MessageBox.Show(reader.ReadToEnd());
}
}
EDIT:
make a request from console.
internal class Program
{
private static void Main(string[] args)
{
new Program().Run();
Console.ReadLine();
}
public void Run()
{
var request = (HttpWebRequest)System.Net.WebRequest.Create("https://server-url-xxxx.com");
request.Method = "POST";
request.ProtocolVersion = System.Net.HttpVersion.Version10;
request.ContentType = "application/x-www-form-urlencoded";
using (var reqStream = request.GetRequestStream())
{
using(var response = new StreamReader(reqStream )
{
Console.WriteLine(response.ReadToEnd());
}
}
}
}

Try this:
WebRequest req = WebRequest.Create("https://server-url-xxxx.com");
req.Method = "POST";
string json_value = jsonSerializer.Serialize(data); //Body data
ServicePointManager.Expect100Continue = false;
using (var streamWriter = new StreamWriter(req.GetRequestStream()))
{
streamWriter.Write(json_value);
streamWriter.Flush();
streamWriter.Close();
}
HttpWebResponse resp = req.GetResponse() as HttpWebResponse;
Stream GETResponseStream = resp.GetResponseStream();
StreamReader sr = new StreamReader(GETResponseStream);
var response = sr.ReadToEnd(); //Response
resp.Close(); //Close response
sr.Close(); //Close StreamReader
And review the URI:
Reserved characters. Send reserved characters by the URI can bring
problems ! * ' ( ) ; : # & = + $ , / ? # [ ]
URI Length: You should not exceed 2000 characters

I ran into this, too. I wanted to simulate hundreds of users with a Console app. When simulating only one user, everything was fine. But with more users came the Timeout exception all the time.
Timeout occurs because by default the ConnectionLimit=2 to a ServicePoint (aka website).
Very good article to read: https://venkateshnarayanan.wordpress.com/2013/04/17/httpwebrequest-reuse-of-tcp-connections/
What you can do is:
1) make more ConnectionGroups within a servicePoint, because ConnectionLimit is per ConnectionGroups.
2) or you just simply increase the connection limit.
See my solution:
private HttpWebRequest CreateHttpWebRequest<U>(string userSessionID, string method, string fullUrl, U uploadData)
{
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(fullUrl);
req.Method = method; // GET PUT POST DELETE
req.ConnectionGroupName = userSessionID; // We make separate connection-groups for each user session. Within a group connections can be reused.
req.ServicePoint.ConnectionLimit = 10; // The default value of 2 within a ConnectionGroup caused me always a "Timeout exception" because a user's 1-3 concurrent WebRequests within a second.
req.ServicePoint.MaxIdleTime = 5 * 1000; // (5 sec) default was 100000 (100 sec). Max idle time for a connection within a ConnectionGroup for reuse before closing
Log("Statistics: The sum of connections of all connectiongroups within the ServicePoint: " + req.ServicePoint.CurrentConnections; // just for statistics
if (uploadData != null)
{
req.ContentType = "application/json";
SerializeToJson(uploadData, req.GetRequestStream());
}
return req;
}
/// <summary>Serializes and writes obj to the requestStream and closes the stream. Uses JSON serialization from System.Runtime.Serialization.</summary>
public void SerializeToJson(object obj, Stream requestStream)
{
DataContractJsonSerializer json = new DataContractJsonSerializer(obj.GetType());
json.WriteObject(requestStream, obj);
requestStream.Close();
}

You may want to set timeout property, check it here http://www.codeproject.com/Tips/69637/Setting-timeout-property-for-System-Net-WebClient

pass string from C# Windows Form Application to php webpage

How can I pass some data to a webpage from C#.net? I'm currently using this:
ProcessStartInfo p1 = new ProcessStartInfo("http://www.example.com","key=123");
Process.Start(p1);
but how can I access it from PHP? I tried:
<?php echo($_GET['key']); ?>
but it prints nothing.

Try passing it with the url itself
ProcessStartInfo p1 = new ProcessStartInfo("http://timepass.comule.com?key=123","");
Process.Start(p1);

you should put the key parameter as a query string :
ProcessStartInfo p1 = new ProcessStartInfo("http://timepass.comule.com?key=123");

I would suggest using the HttpWebRequestClass.
http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.aspx
This way, you would also have the ability to post data to your page, add auth parameters, cookies etc - in case you might need it.
I'm not sure if this matters in your particular setup, passing data thru the query string is not secure. But if security is an issue as well, I would POST the data thru an SSL connection.
Update:
so if you POST'ed data to your php page like so:
string dataToSend = "data=" + HttpUtility.UrlEncode("this is your data string");
var dataBytes = System.Text.Encoding.UTF8.GetBytes(dataToSend);
HttpWebRequest req = (HttpWebRequest) WebRequest.Create("http://localhost/yourpage.php");
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = dataBytes.Length;
req.Method = "POST";
using (var stream = req.GetRequestStream())
{
stream.Write(dataBytes, 0, dataBytes.Length);
}
// -- execute request and get response
HttpWebResponse resp = (HttpWebResponse) req.GetResponse();
if (resp.StatusCode == HttpStatusCode.OK)
Console.WriteLine("Hooray!");
you can retrieve it by using the following code in your php page:
echo $_POST["data"])
Update 2:
AFAIK, ProcessStartInfo/Process.Start() actually starts a process - in this case, I think it will start your browser. The second parameter is the command line arguments. This information is used by programs so they know how to behave when started (hidden, open a default document etc). Its not related to the Query string in anyway. if you prefer to use Process.Start(), then try something like this:
ProcessStartInfo p1 = new ProcessStartInfo("iexplore","http://google.com?q=test");
Process.Start(p1);
If you run that, it will open internet explorer and open google with test on the search box. If that were you're page, you could access "q" by calling:
echo $_GET["q"])

In my applications i used different method i.e using webClient i done it
WebClient client1 = new WebClient();
string path = "dtscompleted.php";//your php path
NameValueCollection formData = new NameValueCollection();
byte[] responseBytes2=null;
formData.Add("key", "123");
try
{
responseBytes2 = client1.UploadValues(path, "POST", formData);
}
catch (WebException web)
{
//MessageBox.Show("Check network connection.\n"+web.Message);
}

getting this error: "The remote server returned an error: (422) Unprocessable Entity." when doing post from C# to RoR

This code is for an outlook plugin. We're trying to POST to a page and are getting this error:
The remote server returned an error: (422) Unprocessable Entity.
The C# code is here:
webClient.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
ASCIIEncoding asciiEncoding = new System.Text.ASCIIEncoding();
Byte[] postData = asciiEncoding.GetBytes("email=e2#email.com&password=hunter2");
char[] resultHTML = asciiEncoding.GetChars(webClient.UploadData("http://url", "POST", postData));
string convertedResultHTML = new string(resultHTML);
Any idea what could be causing this?

POST data must be encoded prior to be sent out on the wire as ASCII, if you are sending character not in the ASCII range. You should try something like:
Byte[] postData = asciiEncoding.GetBytes(HttpUtility.UrlEncode("email=e2#email.com&password=hunter2"));

Because of its limited functionality, I avoid using WebClient and use WebRequest instead. The code below:
does not expect an HTTP 100 status code to be returned,
creates a CookieContainer to store any cookies we pick up,
sets the Content Length header, and
UrlEncodes each value in the post data.
Give the following a try and see if it works for you.
System.Net.ServicePointManager.Expect100Continue = false;
System.Net.CookieContainer cookies = new System.Net.CookieContainer();
// this first request just ensures we have a session cookie, if one exists
System.Net.WebRequest req = System.Net.WebRequest.Create("http://localhost/test.aspx");
((System.Net.HttpWebRequest)req).CookieContainer = cookies;
req.GetResponse().Close();
// this request submits the data to the server
req = System.Net.WebRequest.Create("http://localhost/test.aspx");
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
((System.Net.HttpWebRequest)req).CookieContainer = cookies;
string parms = string.Format("email={0}&password={1}",
System.Web.HttpUtility.UrlEncode("e2#email.com"),
System.Web.HttpUtility.UrlEncode("hunter2"));
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(parms);
req.ContentLength = bytes.Length;
// perform the POST
using (System.IO.Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
// read the response
string response;
using (System.Net.WebResponse resp = req.GetResponse())
{
if (resp == null) return;
using (System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream()))
{
response = sr.ReadToEnd().Trim();
}
}
// the variable response holds the results of the request...
Credits: Hanselman, Simon (SO Question)

This is the RoR application telling you that you have not formed a request that it can handle; the destination script exists (otherwise you'd see a 404), the request is being handled (otherwise you'd get a 400 error) and it's been encoded correctly (or you'd get a 415 error) but the actual instruction can't be carried out.
Looking at it, you seem to be loading some email information. The RoR application could be telling you that the username and password is wrong, or that the user doesn't exist, or something else. It's up to the RoR application itself.
I think the code itself is good; it's just that the app at the other end isn't happy about doing what you ask it. Are you missing something else in the request information, like a command? (eg command=getnetemails&email=e2#email.com&password=hunter2) Are you sure the email/password combination you are passing is good?
see here for more on the 422 error.

Add the below line above your code.
System.Net.ServicePointManager.Expect100Continue = false;
Are you trying to access an authentication required page?

it was solved by returning xml instead of just unstructured text on the RoR side

How do you programmatically fill in a form and 'POST' a web page?

Using C# and ASP.NET I want to programmatically fill in some values (4 text boxes) on a web page (form) and then 'POST' those values. How do I do this?
Edit: Clarification: There is a service (www.stopforumspam.com) where you can submit ip, username and email address on their 'add' page. I want to be able to create a link/button on my site's page that will fill in those values and submit the info without having to copy/paste them across and click the submit button.
Further clarification: How do automated spam bots fill out forms and click the submit button if they were written in C#?

The code will look something like this:
WebRequest req = WebRequest.Create("http://mysite/myform.aspx");
string postData = "item1=11111&item2=22222&Item3=33333";
byte[] send = Encoding.Default.GetBytes(postData);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = send.Length;
Stream sout = req.GetRequestStream();
sout.Write(send, 0, send.Length);
sout.Flush();
sout.Close();
WebResponse res = req.GetResponse();
StreamReader sr = new StreamReader(res.GetResponseStream());
string returnvalue = sr.ReadToEnd();

You can use the UploadValues method on WebClient - all it requires is passing a URL and a NameValueCollection. It is the easiest approach that I have found, and the MS documentation has a nice example:
http://msdn.microsoft.com/en-us/library/9w7b4fz7.aspx
Here is a simple version with some error handling:
var webClient = new WebClient();
Debug.Info("PostingForm: " + url);
try
{
byte [] responseArray = webClient.UploadValues(url, nameValueCollection);
return new Response(responseArray, (int) HttpStatusCode.OK);
}
catch (WebException e)
{
var response = (HttpWebResponse)e.Response;
byte[] responseBytes = IOUtil.StreamToBytes(response.GetResponseStream());
return new Response(responseBytes, (int) response.StatusCode);
}
The Response class is a simple wrapper for the response body and status code.

View the source of the page and use the WebRequest class to do the posting. No need to drive IE. Just figure out what IE is sending to the server and replicate that. Using a tool like Fiddler will make it even easier.

I had a situation where I needed to post free text from a html textarea programmatically and I had issues where I was getting <br /> in my param list i was building.
My solution was a replace of the br tags with linebreak characters and htmlencoding just to be safe.
Regex.Replace( HttpUtility.HtmlDecode( test ), "(<br.*?>)", "\r\n" ,RegexOptions.IgnoreCase);

Where you encode the string:
Encoding.Default.GetBytes(postData);
Use Ascii instead for the google apis:
Encoding.ASCII.GetBytes(postData);
this makes your request the same as and equivalent "curl --data "..." [url]" request

you can send a post/get request with many ways. Different types of library is there to help.
I found it is confusing to choose which one I should use and what are the differences among them.
After surfing stack overflow this is the best answer I found. this thread explains all
https://stackoverflow.com/a/4015346/1999720