I have a fully operational system where openssl based clients interact with an openssl server. Each client have its own certificate that is validated by the server. Certificates have been generated with openssl (X509, pem). They are self-signed.
I now want to write a test client based on SslStream. I used the client example from the SslStream class documentation.
My SslStream client is unable to complete the handshake. stunnel complains about the client not sending its certificate. This is confirmed in Wireshark (Certificates Length: 0 in handshake messages).
My client displays the following exception:
Inner exception: The message received was unexpected or badly
formatted
This is how I load my certificate:
X509Certificate cert = new X509Certificate2(filename, password);
X509CertificateCollection certColl = new X509CertificateCollection();
certColl.Add(cert);
I tried retrieving certificate various properties (ex: GetSerialNumberString()). It works. The Verify method returns false. This is the next thing I am going to investigate.
How I setup my SslStream does not seem to matter (same result):
sslStream.AuthenticateAsClient(serverName);
SslStream sslStream = new SslStream(
client.GetStream(),
false,
new RemoteCertificateValidationCallback(ValidateServerCertificate),
new LocalCertificateSelectionCallback(SelectLocalCertificate));
Same with authentication:
sslStream.AuthenticateAsClient(serverName);
sslStream.AuthenticateAsClient(serverName,
certColl,
SslProtocols.Tls,
true);
SelectLocalCertificate gets called (twice) and returns my certificate. ValidateServerCertificate currently never gets called (to my surprise).
How can I debug that? Even better if you can nail my problem.
UPDATE
I have added a function to perform chain validation based on the X509Chain example from the documentation. It displays all sorts of information on the certificate, including two intriguing messages:
Element certificate is valid: False
Element error status length: 1
In the end, I don't really have more details than when I call verify.
The output of openssl verify cert.pem does not report anything unusual.
error 18 at 0 depth lookup:self signed certificate
OK
UPDATE
I extracted the private key and certificate from the pem, and I generated a cert.pfx (pkcs12). I had no issues importing cert.pfx in my personal key store. In my certificate details, I can see a small icon indicating an attached private key.
I modified my client to retrieve the certificate from my personal store. But I am getting the same exact failure.
The solution was to import my CA root certificate on my Windows machine. My client is now able to complete the handshake!
Found the solution by searching for a more complete SslStream example. Thanks to http://geekswithblogs.net/luskan/archive/2007/10/01/115758.aspx.
Related
When calling AuthenticateAsClient(), I receive the error "Call to SSPI failed." with the inner exception "The message received was unexpected or badly formatted".
I've seen this to be a semi-popular problem, but I haven't been able to find a working solution. Here is where I'm at after scouring articles for a few hours:
Using my private key, the server issued me a signed client cert
Using open ssl, I combined the two into a pfx using:
openssl pkcs12 -in My-Client-Cert.pem -inkey ssl-client-privatekey.pem -export -out private-key-pair.pfx
I imported the cert to the personal folder on LocalMachine. I was able to see that cert recognized that a private key was present.
The private key is not password protected
I imported the CA Root for the service I'm talking to into both my Trusted Root Certification Authorities and Personal stores.
In app code, I pull the cert by thumbnail. I can see in the debugger that both private and public keys are present.
I initialize a TcpClient
I then initialize an SslStream using the the TcpClient
I call AuthenticateAsClient()
I receive the error message
For the hostname parameter of AuthenticateAsClient() I have also tried using the CN of the CA root and also the CN of the client cert with the same result.
I verified that the service I'm trying to talk to does work just fine when I connect to it using openssl s_client connect
Here is the code for the connection:
private void Setup(string hostname, int port, X509Certificate2Collection certs)
{
try
{
// create the socket
var clientSocket = new TcpClient(hostname, port);
_sslStream = new SslStream(
clientSocket.GetStream(),
false,
ValidateServerCertificate,
null);
_sslStream.AuthenticateAsClient(
"VirtuCrypt Root CA - Test", // hostname here must match the name on the server certificate
certs,
SslProtocols.Tls11,
false);
Debug.WriteLine("Connected!");
}
catch(Exception ex)
{
Debug.WriteLine(ex.Message);
if(ex.InnerException != null)
Debug.WriteLine(ex.InnerException.Message);
}
}
Pulling my hair out... any thoughts?
UPDATE:
I spun up Wireshark and noticed that all communication to and from the host was done over TCP, and none of it was done over TLS. Not sure if that's something I should be concerned with despite the fact that I am requesting TLS12
UPDATE2:
I'm looking at Wireshark again and I fixed the decode so that it shows the TLS communications in addition to the TCP connection. Now that I can see the handshake, I can see that no client cert is actually being presented. I'm providing that cert to SslStream, so I don't know why it's not being transmitted.
Ok after finally making friends with Wireshark and breaking about 4 coffee mugs chasing my tail, here's what I found out...
The CA root that was given to us was a pem file that actually included the CA root and 3 intermediate certs.
I had to break those into separate files and import the CA root into our trusted root CA store and the 3 intermediate certs into the intermediate CA store
I hadn't noticed before, but our client cert that was installed in the personal store was saying it didn't have enough info to be validated, but what was really going on is that it was failing validation because we didn't have the CA certs installed correctly as described above. Once I installed those separately, this message went away
It appears that .Net will not send your client cert unless it can be validated. In my 2nd edit in the OP, I mentioned that I could see the client cert was not being sent at all. This was the actual problem. Once I installed these CA certs, the cert started getting sent and everything else was good.
The RemoteCertificateValidationCallback Delegate, shown below, is used to verify the remote Secure Sockets Layer (SSL) certificate. The certificate parameter is the end entity server certificate returned by the remote server. What I'm not sure about is how the chain parameter is constructed. Is it built from the the list of certificates returned by the remote server (typically the server cert and intermediate CA certs) or does it go to the certificate local store and try to build a chain for the end entity server certificate returned in the certificate parameter?
// The following method is invoked by the RemoteCertificateValidationDelegate.
public static bool ValidateServerCertificate(
object sender,
X509Certificate certificate,
X509Chain chain,
SslPolicyErrors sslPolicyErrors)
{
...
}
UPDATE
I've got a suspicion it's using X509Chain.Build(server_cert), but would like to know what's really happening.
This person sees something similar: c# Retrieving a Certificate from an SSL stream shows different chain results vs other external tools
a-certificate-from-an-ssl-stream-shows-different-chain-result
It is received from the server as part of the TLS handshake:
7.4.2. Server Certificate
When this message will be sent:
The server MUST send a Certificate message whenever the agreed-
upon key exchange method uses certificates for authentication
(this includes all key exchange methods defined in this document
except DH_anon). This message will always immediately follow the
ServerHello message.
Meaning of this message:
This message conveys the server's certificate chain to the client.
The certificate MUST be appropriate for the negotiated cipher
suite's key exchange algorithm and any negotiated extensions.
Structure of this message:
opaque ASN.1Cert<1..2^24-1>;
struct {
ASN.1Cert certificate_list<0..2^24-1>;
} Certificate;
certificate_list
This is a sequence (chain) of certificates. The sender's
certificate MUST come first in the list. Each following
certificate MUST directly certify the one preceding it. Because
certificate validation requires that root keys be distributed
independently, the self-signed certificate that specifies the root
certificate authority MAY be omitted from the chain, under the
assumption that the remote end must already possess it in order to
validate it in any case.
Well, according to X509Chain.BuildChain() it uses CAPI CertGetCertificateChain, which means is taken from local cert store, built up from a given cert. You can see how the validation callback is invoked in TransportSecurityHelpers.cs. The chain is built in _SecureChannel.VerifyRemoteCertificate:
chain = new X509Chain();
chain.ChainPolicy.RevocationMode = m_CheckCertRevocation? X509RevocationMode.Online : X509RevocationMode.NoCheck;
chain.ChainPolicy.RevocationFlag = X509RevocationFlag.ExcludeRoot;
if (remoteCertificateStore != null)
chain.ChainPolicy.ExtraStore.AddRange(remoteCertificateStore);
if (!chain.Build(remoteCertificateEx) // Build failed on handle or on policy
&& chain.ChainContext == IntPtr.Zero) // Build failed to generate a valid handle
{
throw new CryptographicException(Marshal.GetLastWin32Error());
}
So it looks like the .Net takes the server cert from the SSL context and builds the chain using the CAPI chain building functions.
I'm creating a TCP proxy with C# using TcpListener for the proxy server and TcpCLient for the communication between client and proxy and between proxy and target server. This works really nice.
I also have to support SSL and TLS encrypted communication.
This works almost well. I create a SslStream from the proxy to the target server with this Code:
var sslStream = new SslStream(remoteStream, false);
sslStream.AuthenticateAsClient(state.RemoteHost);
And I create a SslStream from the proxy to the Client with the following code:
var sslStream = new SslStream(state.ClientStream, false);
sslStream.AuthenticateAsServer(certificate, false, SslProtocols.Tls | SslProtocols.Ssl3 | SslProtocols.Ssl2, true);
The certificate is loaded from the X509Store:
X509Certificate2 certificate;
var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadOnly);
var certificates = store.Certificates.Find(X509FindType.FindBySubjectDistinguishedName, "CN=localhost", false);
store.Close();
if (certificates.Count == 0)
{
Console.WriteLine("Server certificate not found...");
return;
}
else
{
certificate = certificates[0];
}
This also works well if i force the clients to trust the certificate manually.
My questions are:
How can I force the (all) clients to trust the certificate?
Which kind of certificate which is valid for all clients do I need on the proxy?
If needed, what kind of client certificate do I have to install to force the clients to trust the proxy?
How can I create the needed kinds of proxy with openssl or makecert?
I don't want to tunnel the SSL communication threw the proxy because I need to read and manipulate the streams.
[UPDATE]
Yes I used Google and the search in StackOverflow and I tried some different solution without any success.
I also tried the solutions in the following threads:
SSLStream example - how do I get certificates that work?
How do I identify my server name for server authentication by client in c#
[UPDATE2]
This is a very good tutorial to create a CA and a server certificate with openssl, but it doesn't work for me:
http://webserver.codeplex.com/wikipage?title=HTTPS&referringTitle=Home
There is no single certificate which is valid for all requests. So my idea doesn't work.
Because it is not possible to generate a single license for every domain name.
But the Answer is easier than expected: To solve my problem I have to create a certificate for every single request.
All I need to do is:
to create a self-signed root certificate as a certificate authority
to install this in the clients "Trusted Root Certification Authorities" store.
to create the server certificates on-the-fly for every incoming request
sign this certificates with the root certificates tree to set the issuer
I can cache the certificates in files or in a system certificate store if needed.
(this is completely the same in fiddler)
I am attempting to write a client-server library for letting my LAN computers talk to each other. As this is largely a self-education attempt, I am trying to make these connections occur over SSL. My plan is to maintain a single host machine, and have each client maintain a mutually authenticated System.Net.SslStream instance to the host. The host code looks (roughly speaking) like this (error handling omitted for brevity):
SslStream newStream = null;
newStream = new SslStream(this.client.GetStream(), false);
newStream.AuthenticateAsServer(
this.Certificate as X509Certificate,
true,
System.Security.Authentication.SslProtocols.Tls,
false);
if (!(
newStream.IsMutuallyAuthenticated &&
newStream.IsEncrypted &&
newStream.IsSigned))
{
throw new AuthenticationException("Authentication results unsatisfactory.");
}
In this, this.Certificate is an X509Certificate2. The client code looks like this:
SslStream connectionStream = new SslStream(this.client.GetStream(), false);
connectionStream.AuthenticateAsClient(
this.ServerHostname,
new X509CertificateCollection(new X509Certificate[] { (X509Certificate)this.Certificate }),
SslProtocols.Tls,
false);
if (!(
connectionStream.IsEncrypted &&
connectionStream.IsMutuallyAuthenticated &&
connectionStream.IsSigned))
{
throw new AuthenticationException("Authentication results unsatisfactory");
}
I have generated client and server certificates by following this post's instructions, and installed the root authority into Trusted Authorities for the local machine on both the client and server. I then generated certificates for both client and server, put copies of the .cer file in places accessible to the client/server (loaded to get the X509Certificate objects) and installed the with private keys in the Personal stores for the respective machines. Now, everything works when the client and server are on the same machine. But when I generate a new cert and put it on a second machine, the client and server now fail to mutually authenticate. The connection gets set up, encrypted and signed, but IsMutuallyAuthenticated is false. How can I fix this? What am I doing wrong?
Edit: LocalCertificateSelectionCallback and the other callback tell me that the client finds one local certificate, which is mine, no remote certificates, and no acceptable issuers. My certificate is selected and returned from the local certificate selection. There are no SSL policy errors on the client end, and on the host end, I still get RemoteCertificateNotAvailable. Sniffing this transaction with Wireshark, I see a standard TCP handshake, followed by a few hundred bytes of gibberish which includes the name of the host cert, and then the FIN, ACK packets that mark the end of the transaction. So the client certificate indeed isn't being sent, even though it's selected by the LocalCertificateSelectionCallback routine. What might cause this? How can I fix it?
There is a constructor of SSLStream which have RemoteCertificateVAlidationCallback parameter. It should help.
I have a weird requirement. I am trying to communicate with a server written in C#. It looks like this basically:
SslStream sslStream = new SslStream(client.GetStream(), true,
ValidateServerCertificate,
SelectLocalCertificate);
sslStream.AuthenticateAsServer(_pushCert);
I also have example code in C# that uses a X509 certificate and connects to the server. I have the password for the cert.pfx file as well.
What I would like to do is setup some kind of shell script that can connect to the socket, transmit a few bytes and receive the response. (any language really, although I was looking at Python or Ruby or Perl)
I tried using the SSL wrapper from Python, but I get an error stating their is no known algorithm for the server/client to talk.
Example of my Python code:
ss = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s = ssl.wrap_socket(ss, ca_certs=CERT, ssl_version=ssl.PROTOCOL_SSLv23 )
#Attempt connection to our server
try:
s.connect((HOST, PORT))
print s
except:
print 'ERROR Connecting'
sys.exit(0)
For CERT I tried a few different filee: the .pfx, and some extracted from the .pfx using openssl.
I tried many different examples as well (Arguments for the ssl.wrap_socket). I am not really familiar with these connections either.
Perhaps someone here could lend a hand?
Thanks!
You can simplify your SslStream constructor call:
SslStream sslStream = new SslStream(client.GetStream());
sslStream.AuthenticateAsServer(_pushCert);
This server sends _pushCert and does not expect the client to send a certificate back. The server needs the private key for the certificate to make the SSL connection.
The client only needs the CA root certificate that signed the server certificate (or, an option to accept an untrusted certificate.) This needs to be in the "trusted root certificate store" or otherwise identified as trusted to the client wrapper.
If the server certificate is signed by an intermediate CA certificate that is itself signed by the root CA certificate, the client needs that intermediate certificate too. That can be sent by the server, or can already be at the client. Either way, the entire chain of signing certificates has to be in hand at the client to verify all of the signatures along the chain. The intermediate CA certificate does not need to be in the trusted root store.
Neither side needs a private key for the CA root, or for an intermediate signing certificate.
However, if your server expects the client to send a client certificate, then you have to call AuthenticateAsServer with more arguments (clientCertificateRequired == true). In that case, the client needs both its own certificate and the private key for its certificate. The server needs the CA root that signs the client certificate in its trusted store. The client wrapper will take a pfx file, for example, containing the client certificate and private key. The server does not need the client's private key.