We Keep Coding

No client certificate was presented during SSL Handshake

I'd like to ask for explanation. I am using X.509 certificate, and when I tried to post my data to a webservice which I want to communicate with, I am getting the following error: "
No client certificate was presented during SSL Handshake
can you please explain me what is the issue?
NB: I am using .NET Framework 3.5 / C#
What I did exactly is: First I imported the certificate into the store, then I used the code below in order to find it and then recieve the token (using AskForToken function). However, when I send with my data, I got handshake failure.
My first question is why I succeed to get token (if I am not mistaken, the client (which is my application) sent the certificate to the server and got the token, which means the connection has been done well)?
My second question, what do I have to change or check to get rid of this handshake failure.
private static string RequestSecurityToken()
{
WSTrustChannelFactory trustChannelFactory = new Microsoft.IdentityModel.Protocols.WSTrust.WSTrustChannelFactory(
new CertificateWSTrustBinding(SecurityMode.TransportWithMessageCredential),
new EndpointAddress(new Uri(stsAddress)));
trustChannelFactory.TrustVersion = TrustVersion.WSTrust13;
string thumb = "fe14593dd66b2406c5269d742d04b6e1ab03adb1";
trustChannelFactory.Credentials.ClientCertificate.SetCertificate(StoreLocation.CurrentUser, StoreName.My, X509FindType.FindByThumbprint, thumb);
trustChannelFactory.Credentials.ServiceCertificate.Authentication.CertificateValidationMode = X509CertificateValidationMode.PeerOrChainTrust;
cert = trustChannelFactory.Credentials.ClientCertificate.Certificate;
var tokenString = AskForToken(serviceURL, trustChannelFactory);
trustChannelFactory.Close();
return tokenString;
}

SSL has a possibility to demand client authentification. So the Client (your application) has to send a certificate that the Server trusts before the connection is established. It seems that this client authentification fails, because your application doesn´t send such a certificate.
Depending on the webservice you try to use it won´t be possible to create such a certificate, because the server only trusts application from e.g. a certain company.

During an SSL handshake, the client presents it's public key to the other party, which apparently is not happening.
I'm not a C# programmer so I can't present you with the code. But you need to create an SSL keypair (private + public key pair) and use it to define your SSL sockets etc.

"Call to SSPI failed" when connecting to OpenSSL Server

When calling AuthenticateAsClient(), I receive the error "Call to SSPI failed." with the inner exception "The message received was unexpected or badly formatted".
I've seen this to be a semi-popular problem, but I haven't been able to find a working solution. Here is where I'm at after scouring articles for a few hours:
Using my private key, the server issued me a signed client cert
Using open ssl, I combined the two into a pfx using:
openssl pkcs12 -in My-Client-Cert.pem -inkey ssl-client-privatekey.pem -export -out private-key-pair.pfx
I imported the cert to the personal folder on LocalMachine. I was able to see that cert recognized that a private key was present.
The private key is not password protected
I imported the CA Root for the service I'm talking to into both my Trusted Root Certification Authorities and Personal stores.
In app code, I pull the cert by thumbnail. I can see in the debugger that both private and public keys are present.
I initialize a TcpClient
I then initialize an SslStream using the the TcpClient
I call AuthenticateAsClient()
I receive the error message
For the hostname parameter of AuthenticateAsClient() I have also tried using the CN of the CA root and also the CN of the client cert with the same result.
I verified that the service I'm trying to talk to does work just fine when I connect to it using openssl s_client connect
Here is the code for the connection:
private void Setup(string hostname, int port, X509Certificate2Collection certs)
{
try
{
// create the socket
var clientSocket = new TcpClient(hostname, port);
_sslStream = new SslStream(
clientSocket.GetStream(),
false,
ValidateServerCertificate,
null);
_sslStream.AuthenticateAsClient(
"VirtuCrypt Root CA - Test", // hostname here must match the name on the server certificate
certs,
SslProtocols.Tls11,
false);
Debug.WriteLine("Connected!");
}
catch(Exception ex)
{
Debug.WriteLine(ex.Message);
if(ex.InnerException != null)
Debug.WriteLine(ex.InnerException.Message);
}
}
Pulling my hair out... any thoughts?
UPDATE:
I spun up Wireshark and noticed that all communication to and from the host was done over TCP, and none of it was done over TLS. Not sure if that's something I should be concerned with despite the fact that I am requesting TLS12
UPDATE2:
I'm looking at Wireshark again and I fixed the decode so that it shows the TLS communications in addition to the TCP connection. Now that I can see the handshake, I can see that no client cert is actually being presented. I'm providing that cert to SslStream, so I don't know why it's not being transmitted.

Ok after finally making friends with Wireshark and breaking about 4 coffee mugs chasing my tail, here's what I found out...
The CA root that was given to us was a pem file that actually included the CA root and 3 intermediate certs.
I had to break those into separate files and import the CA root into our trusted root CA store and the 3 intermediate certs into the intermediate CA store
I hadn't noticed before, but our client cert that was installed in the personal store was saying it didn't have enough info to be validated, but what was really going on is that it was failing validation because we didn't have the CA certs installed correctly as described above. Once I installed those separately, this message went away
It appears that .Net will not send your client cert unless it can be validated. In my 2nd edit in the OP, I mentioned that I could see the client cert was not being sent at all. This was the actual problem. Once I installed these CA certs, the cert started getting sent and everything else was good.

Calling a REST web service over SSL

I'm struggling to connect to a REST web service that's working only over HTTPS / SSL from my .NET application.
I received the certificate and private key to use as two separate files - a certificate.pem file which contains the certificate, and the webservice.key file which contains the private key. Those are both text files with BASE64 encoded binary data contained in them.
The provider also sent me a PDF showing how to call that web service using CURL and those two files, and that works just fine:
curl.exe -k -v "https://(URL)" --cert certificate.pem --key webservice.key
I need to use the -k option since there seems to be a self-signed certificate somewhere in the hierarchy of certs. Without this option, the call fails.
In order to call this web service from a .NET application (a console app for now), I used OpenSSL (on Windows) to combine these two files into a *.pfx file using this command:
openssl pkcs12 -export -out webservice.pfx -in certificate.pem -inkey webservice.key
This seems to have worked, too - no errors were reported, the file was created and is about 3K in size and it's a totally binary file.
Now, I tried to call that web service from my .NET code something like this:
try
{
// use the SSL protocol (instead of TLS)
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
// ignore any certificate complaints
ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => { return true; };
// create HTTP web request with proper content type
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.ContentType = "application/xml;charset=UTF8";
// grab the PFX as a X.509 certificate from disk
string certFileName = Path.Combine(certPath, "webservice.pfx");
// load the X.509 certificate and add to the web request
X509Certificate cert = new X509Certificate(certFileName, "(top-secret password)");
request.ClientCertificates.Add(cert);
request.PreAuthenticate = true;
// call the web service and get response
WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();
}
catch (Exception exc)
{
// log and print out error
}
However, I can try whatever I like (fiddling around with various settings, on the ServicePointManager and the HttpWebRequest, but I just keep getting these errors:
WebException: The underlying connection was closed: An unexpected error occurred on a send.
IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
SocketException: An existing connection was forcibly closed by the remote host
and no response - even though communicating with the service with CURL has worked just fine.....
What am I missing?? I'm a bit puzzled and mystified by all those certificates, private keys, service point manager options and so on - just waaaaay too many knob and switches to turn, set or turn off - what are the RIGHT settings here??
Update:
If I use
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls;
then the error just simply is:
WebException: The request was aborted: Could not create SSL/TLS secure channel.
S O L U T I O N :
In the end, with looking at the output from curl and a lot of help from #Alexandru and #JurajMajer, I was able to get this to work with this code:
try
{
// use the TLS protocol
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls;
// create HTTP web request with proper content type
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.ContentType = "application/xml;charset=UTF8";
// grab the PFX as a X.509 certificate from disk
string certFileName = Path.Combine(certPath, "webservice.pfx");
// load the X.509 certificate and add to the web request
X509Certificate2 cert = new X509Certificate2(certFileName, "(top-secret password)");
request.ClientCertificates.Add(cert);
request.PreAuthenticate = true;
// call the web service and get response
WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();
string xmlContents = new StreamReader(responseStream).ReadToEnd();
}
catch (Exception exc)
{
// log and print out error
}

You've used the X509Certificate(String, String) constructor with a PKCS#12 certificate, but that constructor only works for PKCS#7 certificates, as MSDN says it...
Initializes a new instance of the X509Certificate class using the name
of a PKCS7 signed file and a password to access the certificate.
PKCS#7 does not include the private (key) part of a certificate/private-key pair, which you will need. This means you will need to use your PKCS#12 certificate given the nature of your certificate.
You may want to try the X509Certificate2(String, String) constructor with your existing PKCS#12 certificate, as this constructor is used with PKCS#12 (PFX) files that contain the certificate's private key, as MSDN says...
This constructor creates a new X509Certificate2 object using a
certificate file name and a password needed to access the certificate.
It is used with PKCS12 (PFX) files that contain the certificate's
private key. Calling this constructor with the correct password
decrypts the private key and saves it to a key container.

Try to enable Network Tracing in App.config on the client - instructions here. That should create network.log with more debug info. In my test environment I have one pfx which works and one which doesn't.
network.log for working pfx:
SecureChannel#9343812 - We have user-provided certificates. The server has specified 34 issuer(s). Looking for certificates that match any of the issuers.
SecureChannel#9343812 - Left with 1 client certificates to choose from.
SecureChannel#9343812 - Trying to find a matching certificate in the certificate store.
SecureChannel#9343812 - Locating the private key for the certificate:
SecureChannel#9343812 - Certificate is of type X509Certificate2 and contains the private key.
network log for non-working pfx:
SecureChannel#26756241 - We have user-provided certificates. The server has specified 34 issuer(s). Looking for certificates that match any of the issuers.
SecureChannel#26756241 - Left with 0 client certificates to choose from.
So for me the problem is my non-working certificate was issued by CA not in list.
Interesting points (possible problems):
1.) Server sends the list of known issuers for client certificate.
2.) Client code is looking for certificate and private key in certificate store event though both are in pfx.

understanding server/proxy/client certificates with .NET sslstream

I'm creating a TCP proxy with C# using TcpListener for the proxy server and TcpCLient for the communication between client and proxy and between proxy and target server. This works really nice.
I also have to support SSL and TLS encrypted communication.
This works almost well. I create a SslStream from the proxy to the target server with this Code:
var sslStream = new SslStream(remoteStream, false);
sslStream.AuthenticateAsClient(state.RemoteHost);
And I create a SslStream from the proxy to the Client with the following code:
var sslStream = new SslStream(state.ClientStream, false);
sslStream.AuthenticateAsServer(certificate, false, SslProtocols.Tls | SslProtocols.Ssl3 | SslProtocols.Ssl2, true);
The certificate is loaded from the X509Store:
X509Certificate2 certificate;
var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadOnly);
var certificates = store.Certificates.Find(X509FindType.FindBySubjectDistinguishedName, "CN=localhost", false);
store.Close();
if (certificates.Count == 0)
{
Console.WriteLine("Server certificate not found...");
return;
}
else
{
certificate = certificates[0];
}
This also works well if i force the clients to trust the certificate manually.
My questions are:
How can I force the (all) clients to trust the certificate?
Which kind of certificate which is valid for all clients do I need on the proxy?
If needed, what kind of client certificate do I have to install to force the clients to trust the proxy?
How can I create the needed kinds of proxy with openssl or makecert?
I don't want to tunnel the SSL communication threw the proxy because I need to read and manipulate the streams.
[UPDATE]
Yes I used Google and the search in StackOverflow and I tried some different solution without any success.
I also tried the solutions in the following threads:
SSLStream example - how do I get certificates that work?
How do I identify my server name for server authentication by client in c#
[UPDATE2]
This is a very good tutorial to create a CA and a server certificate with openssl, but it doesn't work for me:
http://webserver.codeplex.com/wikipage?title=HTTPS&referringTitle=Home

There is no single certificate which is valid for all requests. So my idea doesn't work.
Because it is not possible to generate a single license for every domain name.
But the Answer is easier than expected: To solve my problem I have to create a certificate for every single request.
All I need to do is:
to create a self-signed root certificate as a certificate authority
to install this in the clients "Trusted Root Certification Authorities" store.
to create the server certificates on-the-fly for every incoming request
sign this certificates with the root certificates tree to set the issuer
I can cache the certificates in files or in a system certificate store if needed.
(this is completely the same in fiddler)

Calling HTTPS JBoss service from C# code throws an authentication error

I'm trying to call a JBoss service from a C# program and I'm getting an annoyingly vague error.
JbossService proxy = new JbossService();
proxy.Credentials = new NetworkCredential("ME", "thepwd");
proxy.Url = //https url snipped
proxy.CookieContainer = new CookieContainer();
proxy.PreAuthenticate = true;
Console.WriteLine("Calling service...");
queryResponse qr = proxy.query();
Console.WriteLine("Done.");
The exception and inner exception thrown are as follows:
exception : The underlying connection was closed: An unexpected error occurred on a send.
inner exception : Authentication failed because the remote party has closed the transport stream.
I'm not quite sure what this means, other than perhaps that JBoss likes me even less than I like it. I'm calling from the local machine so I don't think it's a networking issue. Has anyone seen this before?

This usually happens when your client cannot verify trust over https with the server (usually because the server certificate is self signed or if it is signed by a root authority not installed on your client machine.
Easy fix (although there are security consequences)....somewhere in your initialization code add the following:
System.Net.ServicePointManager.ServerCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) => { return true;};
Basically this replaces the application wide handling of server certificate validation and causes your application to accept any certificate. If you want to get finer grained, you can examine the certificate and put some logic in the method.
This works for anything based on System.Net, so it should work for Web Services and any thing based on WebRequest.

I haven't used JBOSS. This is how I troubleshoot similar problems, when using Microsoft technologies -- the same issues may be affecting your program:
Firewall settings or network issue (try connecting manually, to rule this out)
Self-service certificate issues:
Check the following certificate values:
Ensure the server's certificate issuer has a valid, matching issuing trusted root Certificate Authority (CA), on the same machine
The server certificate subject name matches the machine name exactly
The machine name the client is accessing matches that defined in the server certificate
An administrator account set (server) certificate thumbprint
Try recreating the SSL Certificate on both servers)
Try creating your own CA cert, add to trusted publishers, and then create an SSL sert based on that