We Keep Coding

Http.WinHttpException sent on SOAP request .NET CORE 2.2. The message received was unexpected or badly formatted

We have three IHostedService in our .NETCore2.0 webapp performing operations periodically. Two of them are in polling on an external system asking for new data; the third sends to the same external system some data gathered by our webapp. Every request is SOAP and it's done with the following code:
try
{
#region PFC Certificate
// Pfx certificate management
string certPath = GetCertPath();
string certPass = GetCertPassword();
X509Certificate2Collection X509collection = new X509Certificate2Collection();
X509collection.Import(certPath, certPass, X509KeyStorageFlags.PersistKeySet);
#endregion
if (X509collection.Count > 0)
{
X509Certificate2 x509 = X509collection[0];
var request = CreateSOAPWebRequest(url, x509);
byte[] bytes;
bytes = Encoding.ASCII.GetBytes(xmlRequestContent);
request.ContentType = "application/xml; encoding='utf-8'";
request.ContentLength = bytes.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(bytes, 0, bytes.Length);
requestStream.Close();
if (request == null) throw new Exception($"url:{url}: Request NULL - xml: {xmlRequestContent}");
try
{
using (HttpWebResponse response = (HttpWebResponse)await request.GetResponseAsync())
{
if (response.StatusCode == HttpStatusCode.OK)
{
using (Stream responseStream = response.GetResponseStream())
{
// Response deserialization
string responseStr = await new StreamReader(responseStream).ReadToEndAsync();
T result = new T();
XmlSerializer serializer = new XmlSerializer(typeof(T));
using (StringReader reader = new StringReader(responseStr))
{
result = (T)(serializer.Deserialize(reader));
return result;
}
}
}
}
}
catch (WebException ex)
{
_logger.LogError(ex);
throw;
}
}
return default(T);
}
catch(Exception ex)
{
_logger.LogError(ex);
throw;
}
The CreateSOAPWebRequest method is defined as:
private HttpWebRequest CreateSOAPWebRequest(string url, X509Certificate certificate)
{
Uri uri = new Uri(url);
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(uri);
webRequest.Proxy = null;
webRequest.Headers.Add("SOAP:Action");
webRequest.KeepAlive = true;
webRequest.ContentType = "text/xml;charset=\"utf-8\"";
webRequest.Accept = "text/xml";
webRequest.Method = "POST";
webRequest.AuthenticationLevel = AuthenticationLevel.MutualAuthRequired;
if (certificate != null)
webRequest.ClientCertificates.Add(certificate);
return webRequest;
}
The first two hosted services worked very well together for years since the third cames in: some requests go OK in the beginning, then this exception is thrown and no one of the services is able to send the SOAP request anymore (until we restart the webapp):
The SSL connection could not be established, see inner exception. Authentication failed, see inner exception.
---> The SSL connection could not be established, see inner exception.
---> Authentication failed, see inner exception.
---> The message received was unexpected or badly formatted
This is thrown on the line
HttpWebResponse response = (HttpWebResponse)await request.GetResponseAsync()
This seems to be a certificate/security/SSL problem. But the requests work well in the beginning and/or without the third hosted service, so we thought it could be a syncronization problem between the services and we detached the third one by running it on a separate cloned webapp, alone, but we got the same error on the second SOAP call anyway (while the first worked).
We were able to reproduce this error in debug only by disabling the service in the production environment and running the webapp locally in debug mode, reading and sending production data.
We have no idea on what is causing this, so thank you in advance for every advice.

I finally figured it out. The whole thing was a bit misleading while the exception was telling the truth: at some point in our code, after the SOAP request, the flow COULD go through this:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls
So everything was fine until this instruction and of course everything was broken after this instruction.

c# Sign a SOAP envelope webrequest with a 509 cert

I'm working with a third party that refuses to fix their scheme for a published WSDL. The issue is the SOAP service is expecting different name spaces than the WSDL is providing. So in my C# app I'm having a lot of trouble using the Proxy by .Net. In an attempt to work around this I want to just send a web-request and package up my data
I just can't figure out how to add my signature to the header of my request.
Microsoft.Web.Services3.SoapEnvelope soapEnvelopeXml = new Microsoft.Web.Services3.SoapEnvelope();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(#"https://Illinois-stage.tylerhost.net/efm/FilingReviewMDEPort.svc");
request.Headers.Add("SOAPAction:\"urn:oasis:names:tc:legalxml-courtfiling:wsdl:WebServicesProfile-Definitions-4.0/FilingReviewMDEPort/ReviewFilingRequest\"");
request.Headers.Add("Accept-Encoding: gzip, deflate");
request.KeepAlive = true;
request.Headers.Add("MIME-Version: 1.0");
request.ContentType = "multipart/related; type=\"application/xop+xml\";start=\"<http://tempuri.org/0>\";boundary=\"uuid:936f2c7e-590a-4f19-b154-ce8285adf18a+id=2\";start-info=\"text/xml\"";
request.Method = "POST";
soapEnvelopeXml.Load(#"c:\temp\ReviewFilingRequest.xml");
System.IdentityModel.Tokens.X509SecurityToken securityToken = new System.IdentityModel.Tokens.X509SecurityToken(X509);
Microsoft.Web.Services3.Security.Tokens.X509SecurityToken signatureToken = new Microsoft.Web.Services3.Security.Tokens.X509SecurityToken(X509);
MessageSignature sig = new MessageSignature(signatureToken);
////////// XmlDocumentFragment xfrag = soapEnvelopeXml.CreateDocumentFragment();
//////////xfrag.InnerXml = messageHeader;
//////////soapEnvelopeXml.DocumentElement.FirstChild.AppendChild(xfrag);
XmlDocument xmlDoc = new XmlDocument();
MemoryStream xmlStream = new MemoryStream();
xmlDoc.Save(xmlStream);
using (var writer = XmlDictionaryWriter.CreateMtomWriter(xmlStream, Encoding.UTF8, int.MaxValue, "text/xml", "uuid:936f2c7e-590a-4f19-b154-ce8285adf18a+id=2", "http://tempuri.org/0",false,false))
{
soapEnvelopeXml.WriteTo(writer);
using (Stream stream = request.GetRequestStream())
{
stream.Write(xmlStream.ToArray(),0, xmlStream.ToArray().Length );
}
}
using (WebResponse response = request.GetResponse())
{
using (StreamReader rd = new StreamReader(response.GetResponseStream()))
{
string soapResult = rd.ReadToEnd();
Console.WriteLine(soapResult);
}
}

A couple things that might be able to help.
What error message are you getting back? Access Denied (or something along those lines)?
Have you tried looking at which headers you are missing? The best way to do this is to download WireShark or Telerik's Fiddler which allows HTTPS decryption. Using one of these, you can access the website in an internet browser and view the headers that are being sent through a normal browser.

I ended up signing it in Node.js Making the switch to node gave me a lot more control over the message. As an added bonus I didn't have to pay the Microsoft Tax.
https://www.npmjs.com/search?q=soap&page=1&ranking=optimal

Google Drive insert file permission

I can't insert permission to a file with this code:
string URI = String.Format("https://www.googleapis.com/drive/v2/files/{0}/permissions&access_token={1}", fileId, "token");
var request = (HttpWebRequest)HttpWebRequest.Create(URI);
request.Method = "POST";
request.ContentType = "application/json";
string json = "{\"role\": \"reader\",\"type\": \"anyone\"}";
byte[] byteData = new System.Text.ASCIIEncoding().GetBytes(json);
request.ContentLength = byteData.Length;
using (var dataStream = request.GetRequestStream())
{
dataStream.Write(byteData, 0, byteData.Length);
}
var response = (HttpWebResponse)request.GetResponse();
using (var reader = new StreamReader(response.GetResponseStream()))
{
json = reader.ReadToEnd();
}
I al getting a 404 error. What's the problem?

string URI = String.Format("https://www.googleapis.com/drive/v2/files/{0}/permissions&access_token={1}", fileId, "token");
Access token is not a string "token" it must be a valid access token for the user who owns the file.
Update:
permissions?access_token={1}",
You should be using ? and not & to add a parameter to the url. Not even sure you can do it like that with a HTTP Post.
Added info:
If this is not simply a typo on your part you may want to read up on Authorization a little
I also recommend checking out the Google client library instead of writing this yourself. Google.Apis.Drive.v2 Client Library.
There is a newer version of the Google Drive API you might also be interested in looking at rather then writing new code for an older version of the API. Google Drive API v3.

Debugging an HttpWebResponse

This is being done on a Windows Forms App. I've spent a ton of time stepping through this code with the debugger. What I've found are the following things and they all seem to be at this line:
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
1. If I include
request.SendChunked = true;
I get this error at the response line previously stated:
'System.Net.WebException: The remote server returned an error: (415) Unsupported Media Type.
2. If I comment out the code in #1, I receive this error at that main response line that I mentioned in the beginning:
'System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly.
3. If I go with route #1, the "Connection" of the request remains as "KeepAlive" all the way through. But if I go with route #2, the "Connection" of the request changes to "null" at the response line that I mentioned in the beginning.
private void HttpPost()
{
HttpWebRequest request = null;
Uri uri = new Uri("https://post.craigslist.org/bulk-rss/post");
request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
XmlDocument doc = new XmlDocument();
doc.Load("XMLFile1.xml");
//request.ContentLength = doc.InnerXml.Length;
request.SendChunked = true;
using (Stream writeStream = request.GetRequestStream())
{
UTF8Encoding encoding = new UTF8Encoding();
byte[] bytes = encoding.GetBytes(doc.InnerXml);
//request.ContentLength = bytes.Length;
writeStream.Write(bytes, 0, bytes.Length);
}
string result = string.Empty;
request.ProtocolVersion = System.Net.HttpVersion.Version11;
request.KeepAlive = false;
try
{
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (System.IO.StreamReader readStream = new System.IO.StreamReader(responseStream, Encoding.UTF8))
{
result = readStream.ReadToEnd();
}
}
}
}
catch (Exception e)
{
string innerException = String.Format("Inner exception: '{0}'", e.Data);
string exceptionCause = String.Format("An error occurred: '{0}'", e);
System.IO.File.WriteAllText(#"C:\Users\Nathan\Documents\DebugOutputFile\exception.txt", exceptionCause);
System.IO.File.WriteAllText(#"C:\Users\Nathan\Documents\DebugOutputFile\innerException.txt", innerException);
}
}
I feel like these things are adding up towards a solution, but I could really use some guidance.

Option 1: Change your content type to match the body encoding
request.ContentType = "application/xml";
Option 2: Change your body encoding to match the specified content-type
If your server expects only "application/x-www-form-urlencoded", then you need to change your body encoding to suit it, for example, like this:
using (Stream writeStream = request.GetRequestStream())
{
UTF8Encoding encoding = new UTF8Encoding();
string response = String.Concat("arg=", HttpUtility.UrlEncode(doc.InnerXml))
byte[] bytes = encoding.GetBytes(doc.InnerXml);
//request.ContentLength = bytes.Length;
writeStream.Write(bytes, 0, bytes.Length);
}
You need to know the parameter name (above was set to "arg") and add a reference to System.Web, if you haven't.
See following XML...
<?xml version="1.0" encoding="UTF-8"?><test></test>
and encoded string for reference (your request body should look similar to this):
arg=%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%3Ctest%3E%3C%2Ftest%3E
Explanation
If you look at the response you are getting with the first approach: 415 - Unsupported Media Type, you can notice that the content type you are specifying ("application/x-www-form-urlencoded") doesn't match what you are sending in the body (an XML document). Chunk encoding should be enabled when sending files.
Note
When you are having trouble with a request done in source code, try to test the request alone with a web debugging tool, like Fiddler. There you would compose and issue the request until you get the response you want. Then you can compare that with what you are sending from source code (again you should use the same tool for inspecting your request).

Sending xml to SOAP web service returning error 500

I have a piece of code written in C#.Net which reads from a database, collects records, builds a SOAP envelope around them and sends them to the following URL:
https://nodeD1.production.webservices.amadeus.com/SECRETNUMBER
I am able to copy my SOAP envelope string outputted from my code and paste it into SOAPUI, and it works flawlessly. But when I try to send it to the URL through my code, I always get:
The remote server returned an error: (500) Internal Server Error.
My entire SOAP xml request (including the soap header and everything else) is a single string by the time my code is done building it, and I have tried sending to the URL using the following code:
ServicePointManager.ServerCertificateValidationCallback = ValidateServerCertficate;
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
string httpResponseString = "";
webRequest.Proxy = null; //This will result in a quicker post.
byte[] requestBytes = null;
byte[] responseBytes = null;
requestBytes = System.Text.Encoding.UTF8.GetBytes(xmlRequestString);
//Set HttpWebRequest properties
webRequest.Method = "POST";
webRequest.ContentLength = requestBytes.Length;
webRequest.ContentType = "application/soap+xml;charset=UTF-8;";
//webRequest.Headers.Add("SOAPAction: http://webservices.amadeus.com/Profile_PartialUpdateNGProfile_11.1");
//webRequest.ContentType = "text/xml";
using (Stream requestStream = webRequest.GetRequestStream())
{
requestStream.Write(requestBytes, 0, requestBytes.Length);
}
try
{
using (var response = (HttpWebResponse)webRequest.GetResponse())
{
Stream stream = response.GetResponseStream();
using (MemoryStream ms = new MemoryStream())
{
int count = 0;
do
{
byte[] buf = new byte[1024];
count = stream.Read(buf, 0, 1024);
ms.Write(buf, 0, count);
}
while (stream.CanRead && count > 0);
responseBytes = ms.ToArray();
}
}
httpResponseString = System.Text.Encoding.Default.GetString(responseBytes);
}
catch (Exception e)
{
return e.Message;
}
return httpResponseString;
Am I forgetting something obvious? I have been stuck on this problem for a couple days now and I'm not sure what it could be. I have tried commenting and uncommenting the line:
webRequest.Headers.Add("SOAPAction: http://webservices.amadeus.com/Profile_PartialUpdateNGProfile_11.1");
But no dice.
NOTE: My piece of code is an asp.net web service (so I am a web service trying to communicate with another web service), could it be something in my Web.config file that's causing this?

Looks like all I had to do in my case was change my soap action in my soap envelope from:
https://nodeD1.production.webservices.amadeus.com/SECRETNUMBER
to:
https://noded1.production.webservices.amadeus.com/SERCRETNUMBER
Thanks to Fiddler I was able to see the following message:
<faultstring>A header representing a Message Addressing Property is not valid and the message cannot be processed</faultstring>
I Googled the error message and I found a StackOverflow post about it:
Error in Amadeus Service: A header representing a Message Addressing Property is not valid and the message cannot be processed
Apparently .net doesn't like capital letters in URLs in SOAP envelopes. The thing is I thought this might be the case, but I tried with lowercase and it still failed because my SOAP envelope wasn't properly built.