We Keep Coding

Verifying the chain of a self-signed certificate when using SslStream

I have a chain.pem
-----BEGIN CERTIFICATE-----
// My server cert signed by intemediate CA
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
// My intermediate cert signed by root CA
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
// My self signed root cert
-----END CERTIFICATE-----
as well as a server.key.pem
-----BEGIN RSA PRIVATE KEY-----
// Private key for server cert
-----END RSA PRIVATE KEY-----
From there, I generate a pfx file - which has the server cert with its private key along with the rest of the chain.
openssl pkcs12 -export -out certificate.pfx -inkey server.key.pem -in chain.pem
I leave the export password blank
Next I host an TcpListener with an SslStream
namespace fun_with_ssl
{
internal class Program
{
public static int Main(string[] args)
{
var serverCertificate = new X509Certificate2("certificate.pfx");
var listener = new TcpListener(IPAddress.Any, 1443);
listener.Start();
while (true)
{
using (var client = listener.AcceptTcpClient())
using (var sslStream = new SslStream(client.GetStream(), false))
{
sslStream.AuthenticateAsServer(serverCertificate, false, SslProtocols.Tls12, false);
//send/receive from the sslStream
}
}
}
}
}
But when I try to check the chain from openssl, it fails
openssl s_client -connect 127.0.0.1:1443 -CAfile ca.cert.pem
CONNECTED(00000005)
depth=0 CN = SERVER
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = SERVER
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
0 s:CN = SERVER
i:CN = Intermediate
---
Server certificate
-----BEGIN CERTIFICATE-----
// My Server certificate
-----END CERTIFICATE-----
subject=CN = SERVER
issuer=CN = Intermediate
---
No client certificate CA names sent
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:RSA+SHA384:RSA+SHA1:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA1:DSA+SHA1:RSA+SHA512:ECDSA+SHA512
Shared Requested Signature Algorithms: RSA+SHA256:RSA+SHA384:RSA+SHA1:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA1:DSA+SHA1:RSA+SHA512:ECDSA+SHA512
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, P-384, 384 bits
---
SSL handshake has read 1439 bytes and written 481 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: E82C0000B86186D0051CFE6290C12F0D62C4D376B7E40437029B8B85687C4B18
Session-ID-ctx:
Master-Key: 13681EAE940F241726072A4586A96A9FEEEF29B8309B9122FA2F07AC7C9F949128CB66D0F9C430E1D2480E61E287C578
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1566533377
Timeout : 7200 (sec)
Verify return code: 21 (unable to verify the first certificate)
Extended master secret: yes
---
140266287337920:error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown:../ssl/record/rec_layer_s3.c:1528:SSL alert number 46
I doesn't seem as though it is presenting the intermediate or the root certificate so that it can verify the chain. What am I missing here?
In my scenario, the client would have the root certificate public key.

Even if the PFX contained the entire chain, using the single-certificate constructor makes it only load the cert which had a private key, and the rest are discarded.
Even if the load-the-PFX-as-a-collection method is used, SslStream only uses a collection to find an acceptable server certificate (has a private key and the proper EKU value), then ignores the rest.
The intermediate certificate is only sent when it can be found, via system ambient context, by X509Chain. If your certificate is not public-trust then the best answer for your scenario is to add the intermediate (and, optionally, the root) to the CurrentUser\CA (X509StoreName.CertificateAuthority) certificate store. The "CA" store doesn't provide trust, it's just a grab bag of all the intermediate issuer CAs the system has seen, and the system uses it as a cache when building new chains.
You can do this programmatically at startup by
X509Certificate2 serverCertificate = null;
using (X509Store store = new X509Store(StoreName.CertificateAuthority, StoreLocation.CurrentUser))
{
store.Open(OpenFlags.ReadWrite);
X509Certificate2Collection coll = new X509Certificate2Collection();
coll.Import("certificate.pfx");
foreach (X509Certificate2 cert in coll)
{
if (cert.HasPrivateKey)
{
// Maybe apply more complex logic if you really expect multiple private-key certs.
if (serverCertificate == null)
{
serverCertificate = cert;
}
else
{
cert.Dispose();
}
}
else
{
// This handles duplicates (as long as no custom properties have been applied using MMC)
store.Add(cert);
cert.Dispose();
}
}
}
// tcpListener, et al.
Other options: Feed the whole collection into X509Chain.ChainPolicy.ExtraStore, call X509Chain.Build on serverCert, only add the certs after the first one (and optionally not the last one)... just depends on how much extra stuff is expected to be in the PFX.

Why might the server not receive a certificate attached to a request using HttpClient?

We're trying to connect to an endpoint that does nothing more than verify whether a certificate has been correctly attached.
Our code is straightforward:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
using (var handler = new HttpClientHandler())
{
var rawcert = File.ReadAllText(#"C:\OpenSSL\bin\cert.pem");
var rawkey = File.ReadAllText(#"C:\OpenSSL\bin\private.key");
var provider = new CertificateFromFileProvider(rawcert, rawkey);
var certificate = provider.Certificate;
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.ClientCertificates.Add(certificate);
handler.ServerCertificateCustomValidationCallback +=
(HttpRequestMessage req, X509Certificate2 cert2, X509Chain chain, SslPolicyErrors err) =>
{
Trace.WriteLine($"Sender: {req}");
Trace.WriteLine($"cert: {cert2}");
Trace.WriteLine($"chain: {chain}");
Trace.WriteLine($"sslPolicyErrors: {err}");
return true;
};
using (var client = new HttpClient(handler))
{
var response = await client.GetStringAsync("https://{uri}/mtlsTest");
return response;
}
}
(This is a sandbox environment for a proof of concept, so we're comfortable shortcutting the ValidationCallback for now.)
We can see the certificate is attached appears well-formed in the callback (origin and subject are as-expected), but we're getting a response from the server that indicates no certificate was attached. Why might that be?
We have also tried exporting the certificate as a pfx and attaching that instead of the original pem file, with an identical result.
Update
Thanks to useful comments below from #bartonjs and #Crypt32, we've tried adding the private key (using the handy Nuget Package from #stef-heyenrath), but although this results in the certificate that we add to the handler showing HasPrivateKey=true:
... when the ValidationCallback fires, the X509Certificate2 traces HasPrivateKey=false:
... and we get the same error as before - the server never acknowledges receiving the certificate.
If we package the .pem and the private key into a .pfx using open ssl:
pkcs12 -inkey private.key -in cert.pem -export -out cert.pfx
... then we get the same result again, and the server can find no certificate. Likewise if we install the certificate in the personal store and load it from there (same result). An inspection with Wireshark suggests the certificate is simply never being attached.
Why would this be? Full revised code above.

The initial problem here was a badly-formed certificate provided by our test host.
A further problem (once we had a good certificate to work with) was with the output of the CertificateFromFileProvider method of the OpenSSL.X509Certificate2Provider NuGet package.
For whatever reason, the certificate created would not work correctly. Taking the same input files and generating a .pfx with OpenSSL resulted in an X509Certificate2 that worked fine.

SslStream authentication failure

everyone, I'm trying to write something about SSL and here's the question:
I've built things below:
CA certs (a self-made CA)
Server pfx, Server cert, Server key (signed by the self-made CA to "localhost")
Now I'm using .Net SslStream to test the connection:(Client and Server are in different thread, and the TCP connection was built already)
Client:
sslStream.AuthenticateAsClient("localhost");
Server:
sslStream.AuthenticateAsServer(serverCert);
//serverCert is X509Certificate2 built from "server.pfx"
the client's AuthenticateAsClient Method will throw a Exception
"The remote certificate is invalid according to the validation procedure."
I guess the reason is that the Server's certificate is signed by a untrusted CA, so the authentication failed, then how could I add the CA certificate to my trust list?
I tried to add code below in client code, but it won't work
X509Store store = new X509Store(StoreName.TrustedPublisher, StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadWrite);
store.Add(new X509Certificate2(Resources.CACertPath));
store.Close();
sslStream.AuthenticateAsClient("localhost");

The following code will avoid the Windows certificate stores and validate the chain.
I see no reason to add the CA certificate needed to verify a chain to the hundreds in the certificate store already. That means Windows will try to verify the chain with "hundreds + 1" certificates rather than the one certificate truly required.
I have not figured out how to use this chain (chain2 below) by default such that there's no need for the callback. That is, install it on the ssl socket and the connection will "just work". And I have not figured out how install it such that its passed into the callback. That is, I have to build the chain for each invocation of the callback. I think these are architectural defects in .Net, but I might be missing something obvious.
The name of the function does not matter. Below, VerifyServerCertificate is the same callback as RemoteCertificateValidationCallback in SslStream class. You can also use it for the ServerCertificateValidationCallback in ServicePointManager.
static bool VerifyServerCertificate(object sender, X509Certificate certificate,
X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
try
{
String CA_FILE = "ca-cert.der";
X509Certificate2 ca = new X509Certificate2(CA_FILE);
X509Chain chain2 = new X509Chain();
chain2.ChainPolicy.ExtraStore.Add(ca);
// Check all properties
chain2.ChainPolicy.VerificationFlags = X509VerificationFlags.NoFlag;
// This setup does not have revocation information
chain2.ChainPolicy.RevocationMode = X509RevocationMode.NoCheck;
// Build the chain
chain2.Build(new X509Certificate2(certificate));
// Are there any failures from building the chain?
if (chain2.ChainStatus.Length == 0)
return true;
// If there is a status, verify the status is NoError
bool result = chain2.ChainStatus[0].Status == X509ChainStatusFlags.NoError;
Debug.Assert(result == true);
return result;
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
return false;
}

SslStream.AuthenticateAsClient(string, certCollection, protocol, bool) won't send the certificate

I have a PEM certificate (the ones with -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----) that should be sent to server for server requires client certification.
I know that it is said X509Certificate.CreateFromCertFile(string) does not support PEM and it only does DER but I actually get the information including public key, CN, Issuer, etc.
Anyway, my RemoteCertificationValidationCallBack does not get the certificate (it's null) even when I use t he DER (converted by openssl)
code on server
SslStream sslStream = new SslStream(
client.GetStream(), false, (sender, certificate, chain, errors) =>
{
return true;
});
Can any of you guys help me understand why?
Thanks
Peyman

Verify Remote Server X509Certificate using CA Certificate File

I've generated a CA and multiple certificates (signed by CA) using OpenSSL and I have a .NET/C# client and server both using SslStream which each have their own certificates/keys, mutual authentication is enabled and revocation is disabled.
I'm using RemoteCertificateValidationCallback for SslStream to validate the remote server's certificate and I was hoping I could just load the CA's public certificate (as a file) in the program and use it to verify the remote certificate rather then actually installing the CA in the Windows Certificate Store. The problem is the X509Chain won't show anything else unless I install the CA into the store, either will the Windows CryptoAPI shell when I open a PEM version of one of the certificates.
My question is, how can I verify a certificate has been signed by my specific CA just by using the CA's public certificate file without using Windows certificate store or WCF when RemoteCertificateValidationCallback, X509Certificate and X509Chain don't seem to give me anything to work with?

Because the CA certificate is NOT in the root certificate store, you will have within the RemoteCertificateValidationCallback() an error flag of SslPolicyErrors.RemoteCertificateChainErrors ; a possibility is to validate explicitely the certificate chain against your own X509Certificate2Collection, since you are not using the local store.
if (sslPolicyErrors == SslPolicyErrors.RemoteCertificateChainErrors)
{
X509Chain chain0 = new X509Chain();
chain0.ChainPolicy.RevocationMode = X509RevocationMode.NoCheck;
// add all your extra certificate chain
chain0.ChainPolicy.ExtraStore.Add(new X509Certificate2(PublicResource.my_ca));
chain0.ChainPolicy.VerificationFlags = X509VerificationFlags.AllowUnknownCertificateAuthority;
isValid = chain0.Build((X509Certificate2)certificate);
}
You can also re-use the chain passed in the callback, add your extra certificate(s) in the ExtraStore collection, and validate with the AllowUnknownCertificateAuthority flag which is needed since you add untrusted certificate(s) to the chain.
You could also prevent the original error by adding programmatically the CA certificate in the trusted root store (of course it opens a popup, for it is a major security problem to globally add a new trusted CA root) :
var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadWrite);
X509Certificate2 ca_cert = new X509Certificate2(PublicResource.my_ca);
store.Add(ca_cert);
store.Close();
EDIT: For those who want to clearly test the chain with your CA :
Another possibility is to use the library BouncyCastle to build the certificate chain and validate the trust. The options are clear and errors are easy to understand. In cas of success it will build the chain, otherwise an exception is returned. Sample below :
// rootCerts : collection of CA
// currentCertificate : the one you want to test
var builderParams = new PkixBuilderParameters(rootCerts,
new X509CertStoreSelector { Certificate = currentCertificate });
// crls : The certificate revocation list
builderParams.IsRevocationEnabled = crls.Count != 0;
// validationDate : probably "now"
builderParams.Date = new DateTimeObject(validationDate);
// The indermediate certs are items necessary to create the certificate chain
builderParams.AddStore(X509StoreFactory.Create("Certificate/Collection", new X509CollectionStoreParameters(intermediateCerts)));
builderParams.AddStore(X509StoreFactory.Create("CRL/Collection", new X509CollectionStoreParameters(crls)));
try
{
PkixCertPathBuilderResult result = builder.Build(builderParams);
return result.CertPath.Certificates.Cast<X509Certificate>();
...

How can I verify a certificate has been signed by my specific CA just by using the CA's public certificate file without using Windows certificate store or WCF when RemoteCertificateValidationCallback, X509Certificate and X509Chain don't seem to give me anything to work with?
The following code will avoid the Windows certificate stores and validate the chain. Its a little different than JB's code, especially in the use of flags. The code below does not require AllowUnknownCertificateAuthority (but it does use X509RevocationMode.NoCheck since I don't have a CRL).
The name of the function does not matter. Below, VerifyServerCertificate is the same callback as RemoteCertificateValidationCallback in SslStream class. You can also use it for the ServerCertificateValidationCallback in ServicePointManager.
static bool VerifyServerCertificate(object sender, X509Certificate certificate,
X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
try
{
String CA_FILE = "ca-cert.der";
X509Certificate2 ca = new X509Certificate2(CA_FILE);
X509Chain chain2 = new X509Chain();
chain2.ChainPolicy.ExtraStore.Add(ca);
// Check all properties
chain2.ChainPolicy.VerificationFlags = X509VerificationFlags.NoFlag;
// This setup does not have revocation information
chain2.ChainPolicy.RevocationMode = X509RevocationMode.NoCheck;
// Build the chain
chain2.Build(new X509Certificate2(certificate));
// Are there any failures from building the chain?
if (chain2.ChainStatus.Length == 0)
return true;
// If there is a status, verify the status is NoError
bool result = chain2.ChainStatus[0].Status == X509ChainStatusFlags.NoError;
Debug.Assert(result == true);
return result;
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
return false;
}
I have not figured out how to use this chain (chain2 below) by default such that there's no need for the callback. That is, install it on the ssl socket and the connection will "just work". And I have not figured out how install it such that its passed into the callback. That is, I have to build the chain for each invocation of the callback. I think these are architectural defects in .Net, but I might be missing something obvious.