We Keep Coding

Issues using certificate for HTTP POST web request

I'm quite stuck trying to perform a POST request using a P12 certificate that was provided by the server. I've never dealt with HTTPS before, and I'm going in circles (I'm not sure I'm even using the right terminology). I know this must be quite basic, but this is the first time I need to code something like this.
I've even tried to replicate what others have done for exactly the same purpose, but for some reason it does not work (perhaps it has to do with how it works in Android, I'm not sure).
This is what I have right now:
// P12 certificate is placed in "assets" folder
AssetManager assets = Application.Context.Assets;
byte[] bytes;
using (StreamReader sr2 = new StreamReader(assets.Open("myCertificate.p12")))
{
using (var memstream = new MemoryStream())
{
sr2.BaseStream.CopyTo(memstream);
bytes = memstream.ToArray();
}
}
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://www.aidap.naimes.faa.gov/aidap/XmlNotamServlet");
X509Certificate2 clientCertificate = new X509Certificate2(bytes, "myPassword");
req.ClientCertificates.Add(clientCertificate);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
string postData = "uid=myUsername&password=myPassword&active=Y";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
req.ContentLength = byteArray.Length;
// **Next line triggers the Exception**
using (Stream dataStream = req.GetRequestStream())
{
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
}
// Show the first 10 lines
using (StreamReader sr = new StreamReader(req.GetResponse().GetResponseStream()))
{
for (int i = 0; i < 10; i++)
Console.WriteLine(sr.ReadLine());
}
When calling GetRequestStream() I get the following (I copy the first bit, as it's quite long):
{System.Net.WebException: Error: SecureChannelFailure (One or more errors occurred.) ---> System.AggregateException: One or more errors occurred. ---> System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> Mono.Btls.MonoBtlsException: Syscall
at Mono.Btls.MonoBtlsContext.ProcessHandshake () [0x00038] in <9624f2cab6ac4ffc9c31e9469d40591a>:0
at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake
The certificate and password seems to work correctly, as I can inspect in debug mode and verify that all details are loaded in "clientCertificate".
EDIT
So I tried following Ketan's suggestion but I'm not quite sure how to add a trace to an Android project. What I did, is to create a small console application but I didn't have to add a trace because it worked perfectly right away.
I also tried a different approach with RestSharp and it worked once again perfectly in the console application but not in Xamarin:
var client = new RestClient("https://www.aidap.naimes.faa.gov/aidap/XmlNotamServlet");
client.ClientCertificates = new X509CertificateCollection() { clientCertificate };
string postData = "uid=myUsername&password=myPassword&active=Y";
var request = new RestRequest(Method.POST);
request.AddParameter("application/x-www-form-urlencoded", postData, ParameterType.RequestBody);
var response = client.Execute(request);
So there must be something preventing the SSL from working in Android.

Debugging an HttpWebResponse

This is being done on a Windows Forms App. I've spent a ton of time stepping through this code with the debugger. What I've found are the following things and they all seem to be at this line:
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
1. If I include
request.SendChunked = true;
I get this error at the response line previously stated:
'System.Net.WebException: The remote server returned an error: (415) Unsupported Media Type.
2. If I comment out the code in #1, I receive this error at that main response line that I mentioned in the beginning:
'System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly.
3. If I go with route #1, the "Connection" of the request remains as "KeepAlive" all the way through. But if I go with route #2, the "Connection" of the request changes to "null" at the response line that I mentioned in the beginning.
private void HttpPost()
{
HttpWebRequest request = null;
Uri uri = new Uri("https://post.craigslist.org/bulk-rss/post");
request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
XmlDocument doc = new XmlDocument();
doc.Load("XMLFile1.xml");
//request.ContentLength = doc.InnerXml.Length;
request.SendChunked = true;
using (Stream writeStream = request.GetRequestStream())
{
UTF8Encoding encoding = new UTF8Encoding();
byte[] bytes = encoding.GetBytes(doc.InnerXml);
//request.ContentLength = bytes.Length;
writeStream.Write(bytes, 0, bytes.Length);
}
string result = string.Empty;
request.ProtocolVersion = System.Net.HttpVersion.Version11;
request.KeepAlive = false;
try
{
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (System.IO.StreamReader readStream = new System.IO.StreamReader(responseStream, Encoding.UTF8))
{
result = readStream.ReadToEnd();
}
}
}
}
catch (Exception e)
{
string innerException = String.Format("Inner exception: '{0}'", e.Data);
string exceptionCause = String.Format("An error occurred: '{0}'", e);
System.IO.File.WriteAllText(#"C:\Users\Nathan\Documents\DebugOutputFile\exception.txt", exceptionCause);
System.IO.File.WriteAllText(#"C:\Users\Nathan\Documents\DebugOutputFile\innerException.txt", innerException);
}
}
I feel like these things are adding up towards a solution, but I could really use some guidance.

Option 1: Change your content type to match the body encoding
request.ContentType = "application/xml";
Option 2: Change your body encoding to match the specified content-type
If your server expects only "application/x-www-form-urlencoded", then you need to change your body encoding to suit it, for example, like this:
using (Stream writeStream = request.GetRequestStream())
{
UTF8Encoding encoding = new UTF8Encoding();
string response = String.Concat("arg=", HttpUtility.UrlEncode(doc.InnerXml))
byte[] bytes = encoding.GetBytes(doc.InnerXml);
//request.ContentLength = bytes.Length;
writeStream.Write(bytes, 0, bytes.Length);
}
You need to know the parameter name (above was set to "arg") and add a reference to System.Web, if you haven't.
See following XML...
<?xml version="1.0" encoding="UTF-8"?><test></test>
and encoded string for reference (your request body should look similar to this):
arg=%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%3Ctest%3E%3C%2Ftest%3E
Explanation
If you look at the response you are getting with the first approach: 415 - Unsupported Media Type, you can notice that the content type you are specifying ("application/x-www-form-urlencoded") doesn't match what you are sending in the body (an XML document). Chunk encoding should be enabled when sending files.
Note
When you are having trouble with a request done in source code, try to test the request alone with a web debugging tool, like Fiddler. There you would compose and issue the request until you get the response you want. Then you can compare that with what you are sending from source code (again you should use the same tool for inspecting your request).

Simple REST request c# - error

I am trying write simple request to REST service.Follow to documentation from REST Service provider:
I should use in header Content-Type: application/json.
Response return in json format
For authorization proccess I have to send two headers one with APIKey and second with APISign
Controller PING in REST service to test code.
I use .net 2.0
string sha1String = APIKey + "/rest/ping" + APISecret;
string XRestApiSign = SHA1HashStringForUTF8String(sha1String);
string data = "";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(restServer);
request.Method = "POST";
request.ContentType = "application/json";
request.Headers.Add("X-Rest-ApiSign", XRestApiSign);
request.Headers.Add("X-Rest-ApiKey", APIKey);
request.ContentLength = data.Length;
StreamWriter requestWriter;
Stream webStream = request.GetRequestStream();
using (requestWriter = new StreamWriter(webStream, System.Text.Encoding.ASCII)) ;
{
requestWriter.Write(data);
}
request.BeginGetResponse((x) =>
{
using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(x))
{
List<string> list = Newtonsoft.Json.JsonConvert.DeserializeObject<List<string>>(response.GetResponseStream().ToString());
}
}, null);
I should get response string PONG but I get below message
Unexpected character encountered while parsing value: S. Path '', line 0, position 0
Is code OK ? Why I get this message?

This is mostly the case because the script that generates the JSON on the server side adds the byte order mark to the response.
In your case, however, you're trying to convert the stream to JSON, not the content of the stream. You need to read all text from the stream and deserialize the object from that. You need to call one of the methods on the stream that reads the content.

I modified my code with yours suggestion and it works. Now I have problem with send data :)
Subscription user1 = new Subscription
{
Email = "kubaIt#test.com.pl",
List = "xfct2bjcdv",
};
List<Subscription> user = new List<Subscription>();
user.Add(user1);
string json = JsonConvert.SerializeObject(user);
string data = json;
I added above. Other code is the same. I got error :The request was aborted: The request was canceled."
json = [{\"email\":\"kubaIt#test.com.pl\",\"list\":\"xfct2bjcdv\"}] //value from debuger

getting this error: "The remote server returned an error: (422) Unprocessable Entity." when doing post from C# to RoR

This code is for an outlook plugin. We're trying to POST to a page and are getting this error:
The remote server returned an error: (422) Unprocessable Entity.
The C# code is here:
webClient.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
ASCIIEncoding asciiEncoding = new System.Text.ASCIIEncoding();
Byte[] postData = asciiEncoding.GetBytes("email=e2#email.com&password=hunter2");
char[] resultHTML = asciiEncoding.GetChars(webClient.UploadData("http://url", "POST", postData));
string convertedResultHTML = new string(resultHTML);
Any idea what could be causing this?

POST data must be encoded prior to be sent out on the wire as ASCII, if you are sending character not in the ASCII range. You should try something like:
Byte[] postData = asciiEncoding.GetBytes(HttpUtility.UrlEncode("email=e2#email.com&password=hunter2"));

Because of its limited functionality, I avoid using WebClient and use WebRequest instead. The code below:
does not expect an HTTP 100 status code to be returned,
creates a CookieContainer to store any cookies we pick up,
sets the Content Length header, and
UrlEncodes each value in the post data.
Give the following a try and see if it works for you.
System.Net.ServicePointManager.Expect100Continue = false;
System.Net.CookieContainer cookies = new System.Net.CookieContainer();
// this first request just ensures we have a session cookie, if one exists
System.Net.WebRequest req = System.Net.WebRequest.Create("http://localhost/test.aspx");
((System.Net.HttpWebRequest)req).CookieContainer = cookies;
req.GetResponse().Close();
// this request submits the data to the server
req = System.Net.WebRequest.Create("http://localhost/test.aspx");
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
((System.Net.HttpWebRequest)req).CookieContainer = cookies;
string parms = string.Format("email={0}&password={1}",
System.Web.HttpUtility.UrlEncode("e2#email.com"),
System.Web.HttpUtility.UrlEncode("hunter2"));
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(parms);
req.ContentLength = bytes.Length;
// perform the POST
using (System.IO.Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
// read the response
string response;
using (System.Net.WebResponse resp = req.GetResponse())
{
if (resp == null) return;
using (System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream()))
{
response = sr.ReadToEnd().Trim();
}
}
// the variable response holds the results of the request...
Credits: Hanselman, Simon (SO Question)

This is the RoR application telling you that you have not formed a request that it can handle; the destination script exists (otherwise you'd see a 404), the request is being handled (otherwise you'd get a 400 error) and it's been encoded correctly (or you'd get a 415 error) but the actual instruction can't be carried out.
Looking at it, you seem to be loading some email information. The RoR application could be telling you that the username and password is wrong, or that the user doesn't exist, or something else. It's up to the RoR application itself.
I think the code itself is good; it's just that the app at the other end isn't happy about doing what you ask it. Are you missing something else in the request information, like a command? (eg command=getnetemails&email=e2#email.com&password=hunter2) Are you sure the email/password combination you are passing is good?
see here for more on the 422 error.

Add the below line above your code.
System.Net.ServicePointManager.Expect100Continue = false;
Are you trying to access an authentication required page?

it was solved by returning xml instead of just unstructured text on the RoR side

Getting error " The remote server returned an error: (400) Bad Request." on line WebResponse response = request.GetResponse();

//Create a request using a URL that can receive a post.
WebRequest request = WebRequest.Create("https://go.urbanairship.com/api/push/");
request.Credentials = new NetworkCredential("pvYMExk3QIO7p2YUs6BBkg", "rO3DsucETRadbbfxHkd6qw");
// Set the Method property of the request to POST.
request.Method = "POST";
// Create POST data and convert it to a byte array.
//WRITE JSON DATA TO VARIABLE D
string postData = "{\"aps\": {\"badge\": 1, \"alert\": \"Hello from Urban Airship!\"}, \"device_tokens\": [\"6334c016fc643baa340eca25bc661d15055a07b475e9a6108f3f644b15dd05ac\"]}";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
// Set the ContentType property of the WebRequest.
request.ContentType = "application/json";
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
// Get the request stream.
using (Stream dataStream = request.GetRequestStream())
{
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
}
// Get the response.
WebResponse response = request.GetResponse();
//Error "The remote server returned an error: (400) Bad Request"
// Display the status.
Console.WriteLine(((HttpWebResponse)response).StatusDescription);
// Get the stream containing content returned by the server.
using (Stream dataStream = response.GetResponseStream())
{
// Open the stream using a StreamReader for easy access.
using (var reader = new StreamReader(dataStream))
{
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Display the content.
Console.WriteLine(responseFromServer);
response.Close();
}
}

I have experienced a similar problem to what he is getting.
When the exception is thrown calling GetResponse(), it is a WebException. Cast it as such, then check out the response stream. Yes, the content length is -1, but ignore that.
catch (Exception ex)
{
//byte[] buffer = new byte[999999];
WebException wex = (WebException)ex;
var s = wex.Response.GetResponseStream();
string ss = "";
int lastNum = 0;
do
{
lastNum = s.ReadByte();
ss += (char)lastNum;
} while (lastNum != -1);
s.Close();
s = null;
ErrorHasOccurred(new Exception("An error has occurred sending the notification to Urban Airship. Please see the InnerException for details. Please note that, for sending messages, the master password is required (instead of the regular password). ERROR: " + ss, ex));
}
Then just breakpoint right where I have ErrorHasOccurred and read the contents of the ss variable. It will tell you the actual error that Urban Airship returns.

What is your question? The server is saying your request is bad. Use Fiddler if you're not sure what you are actually sending to the server, then fix your request. Otherwise fix your server code.
Either way, this is "not a real question" fodder without some clarification.

It is a valid question...
First. Not use hard code for build the json string, use JavaScriptSerializer
var json = new JavaScriptSerializer().Serialize(yourObject);
Second. For single parameter, use
...
BodyStyle = WebMessageBodyStyle.Bare,
...
insted of
BodyStyle = WebMessageBodyStyle.WrappedRequest,
(I spend a few hours with a similar problem)