We Keep Coding

Mailkit IMapClient not hitting ServerCertificateValidationCallback & SslHandshakeException

I'm attempting to connect to an IMAP server using the following code on a Windows Server 2019 machine:
using (var client = new ImapClient(new ProtocolLogger("protocol.log")))
{
var address = EnvReader.GetStringValue("EMAIL_ADDRESS");
var password = EnvReader.GetStringValue("EMAIL_PASSWORD");
var creds = new NetworkCredential(address, password);
client.CheckCertificateRevocation = false;
client.ServerCertificateValidationCallback = (s, c, h, e) =>
{
Console.WriteLine("ALL UP IN THIS CALLBACK" + e.ToString());
return true;
};
client.Connect("outlook.office365.com", 993, SecureSocketOptions.SslOnConnect);
client.Authenticate(address, password);
}
On my Mac, this code runs perfectly fine, I can connect and subsequently authenticate just fine.
On the Windows machine I receive the following exception:
MailKit.Security.SslHandshakeException: An error occurred while attempting to establish an SSL or TLS connection.
This usually means that the SSL certificate presented by the server is not trusted by the system for one or more of
the following reasons:
1. The server is using a self-signed certificate which cannot be verified.
2. The local system is missing a Root or Intermediate certificate needed to verify the server's certificate.
3. A Certificate Authority CRL server for one or more of the certificates in the chain is temporarily unavailable.
4. The certificate presented by the server is expired or invalid.
5. The set of SSL/TLS protocols supported by the client and server do not match.
6. You are trying to connect to a port which does not support SSL/TLS.
See https://github.com/jstedfast/MailKit/blob/master/FAQ.md#SslHandshakeException for possible solutions
Based on the info in the linked FAQ, I added the ServerCertificateValidationCallback, however the callback is never hit (The previously mentioned exception is still thrown, the relevant console logging never occurs, and a breakpoint inside the callback is never hit while debugging).
From my reading, the ServerCertificateValidationCallback should handle cases #1-4 that the exception message mentions. The fact that I can connect on the specified port on my Mac would seem to rule out case #6 (I also tried port 143 + SecureSocketOptions.StartTls). That leaves case #5, however, I can't find any information suggesting that Windows Server 2019 can't handle SSL/TSL protocols.
Any ideas for a) dealing with this exception and/or b) figuring out why the ServerCertificateValidationCallback is not firing would be greatly appreciated.
Edit: My project is referencing .NET 5.0

Let's go through each of the possibilities:
The server is using a self-signed certificate which cannot be verified.
outlook.office365.com would not be using a self-signed certificate, so that wouldn't be an issue in this case.
The local system is missing a Root or Intermediate certificate needed to verify the server's certificate.
This one is very possible, but the ServerCertificateValidationCallback override should be overriding this failure. However, it's not getting hit... so it's not actually bypassing this potential error.
A Certificate Authority CRL server for one or more of the certificates in the chain is temporarily unavailable.
This would be negated by client.CheckCertificateRevocation = false;
The certificate presented by the server is expired or invalid.
This is not the case because the certificate does not expire until 1/21/2022.
The set of SSL/TLS protocols supported by the client and server do not match.
The server supports at least TLSv1.2 which is a default TLS protocol version supported by MailKit in all target framework versions (.NET 4.5 -> 5.0 + netstandard2.x's).
You are trying to connect to a port which does not support SSL/TLS.
Port 993 is the correct port and SslOnConnect is the correct option, so this is not the issue.
Assuming there isn't a bug in MailKit's SslStream.AuthenticateAsClientAsync() call that passes in the validation callback method (.NET 5.0 is different than other versions), what is the InnerException? Maybe that will provide some insight.

C# target host name for SSL/TLS (HTTPS) is always empty

I have created a C# Web Server and I have some problems with the SSL/TLS certificates.
For the setup I have changed my /etc/hosts file for that any request to example.com will direct
to localhost. After that I have created a TCP server which works quite as expected. For the
HTTPS I added a System.Net.Security.SslStream layer with a default self-signed certificate which
also works as expected.
But for now I want to send a certificate that depends on the location the user requests. My server
code is like this:
var stream = new SslStream(
innerStream: client.GetStream(),
leaveInnerStreamOpen: false,
userCertificateValidationCallback: null,
userCertificateSelectionCallback: LocalCertificateSelection
);
stream.AuthenticateAsServer(
// I am forced to provide a default certificate. null will throw an error
serverCertificate: Settings.Certificate,
clientCertificateRequired: false,
enabledSslProtocols: SslProtocols.None,
checkCertificateRevocation: true
);
private X509Certificate LocalCertificateSelection(
object sender,
string targetHost,
X509CertificateCollection localCertificates,
X509Certificate remoteCertificate,
string[] acceptableIssuers
)
{
// at this point: targetHost == null && acceptableIssuers.Length == 0
return Settings.Certificate;
};
But if I open the page in Firefox or in curl the targetHost and acceptableIssuers from my
certificate selection callback are always empty.
How am I supposed to get the host name to provide the correct certificate?
Edit 1:
My library is targeted for .Net Standard 2.1. Maybe this will help.
Edit 2:
I have added the certificate to the key store of the server (as it was mentioned in a comment), but this wont resolve the problem:
using var store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadWrite);
store.Add(Settings.Certificate);
After that I have captured some packages with Wireshark and I have noticed the following:
After TCP connection is finished:
Client sends a TLS Hello request with a valid SNI extension (here is the host name I want to get)
Debugger break point in my code is called. The SslStream tries to get a valid certificate: targetHost and acceptableIssuers are empty.
Server sends the default certificate with possible invalid host name

understanding server/proxy/client certificates with .NET sslstream

I'm creating a TCP proxy with C# using TcpListener for the proxy server and TcpCLient for the communication between client and proxy and between proxy and target server. This works really nice.
I also have to support SSL and TLS encrypted communication.
This works almost well. I create a SslStream from the proxy to the target server with this Code:
var sslStream = new SslStream(remoteStream, false);
sslStream.AuthenticateAsClient(state.RemoteHost);
And I create a SslStream from the proxy to the Client with the following code:
var sslStream = new SslStream(state.ClientStream, false);
sslStream.AuthenticateAsServer(certificate, false, SslProtocols.Tls | SslProtocols.Ssl3 | SslProtocols.Ssl2, true);
The certificate is loaded from the X509Store:
X509Certificate2 certificate;
var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadOnly);
var certificates = store.Certificates.Find(X509FindType.FindBySubjectDistinguishedName, "CN=localhost", false);
store.Close();
if (certificates.Count == 0)
{
Console.WriteLine("Server certificate not found...");
return;
}
else
{
certificate = certificates[0];
}
This also works well if i force the clients to trust the certificate manually.
My questions are:
How can I force the (all) clients to trust the certificate?
Which kind of certificate which is valid for all clients do I need on the proxy?
If needed, what kind of client certificate do I have to install to force the clients to trust the proxy?
How can I create the needed kinds of proxy with openssl or makecert?
I don't want to tunnel the SSL communication threw the proxy because I need to read and manipulate the streams.
[UPDATE]
Yes I used Google and the search in StackOverflow and I tried some different solution without any success.
I also tried the solutions in the following threads:
SSLStream example - how do I get certificates that work?
How do I identify my server name for server authentication by client in c#
[UPDATE2]
This is a very good tutorial to create a CA and a server certificate with openssl, but it doesn't work for me:
http://webserver.codeplex.com/wikipage?title=HTTPS&referringTitle=Home

There is no single certificate which is valid for all requests. So my idea doesn't work.
Because it is not possible to generate a single license for every domain name.
But the Answer is easier than expected: To solve my problem I have to create a certificate for every single request.
All I need to do is:
to create a self-signed root certificate as a certificate authority
to install this in the clients "Trusted Root Certification Authorities" store.
to create the server certificates on-the-fly for every incoming request
sign this certificates with the root certificates tree to set the issuer
I can cache the certificates in files or in a system certificate store if needed.
(this is completely the same in fiddler)

SSLStream Mutual Authentication

I am attempting to write a client-server library for letting my LAN computers talk to each other. As this is largely a self-education attempt, I am trying to make these connections occur over SSL. My plan is to maintain a single host machine, and have each client maintain a mutually authenticated System.Net.SslStream instance to the host. The host code looks (roughly speaking) like this (error handling omitted for brevity):
SslStream newStream = null;
newStream = new SslStream(this.client.GetStream(), false);
newStream.AuthenticateAsServer(
this.Certificate as X509Certificate,
true,
System.Security.Authentication.SslProtocols.Tls,
false);
if (!(
newStream.IsMutuallyAuthenticated &&
newStream.IsEncrypted &&
newStream.IsSigned))
{
throw new AuthenticationException("Authentication results unsatisfactory.");
}
In this, this.Certificate is an X509Certificate2. The client code looks like this:
SslStream connectionStream = new SslStream(this.client.GetStream(), false);
connectionStream.AuthenticateAsClient(
this.ServerHostname,
new X509CertificateCollection(new X509Certificate[] { (X509Certificate)this.Certificate }),
SslProtocols.Tls,
false);
if (!(
connectionStream.IsEncrypted &&
connectionStream.IsMutuallyAuthenticated &&
connectionStream.IsSigned))
{
throw new AuthenticationException("Authentication results unsatisfactory");
}
I have generated client and server certificates by following this post's instructions, and installed the root authority into Trusted Authorities for the local machine on both the client and server. I then generated certificates for both client and server, put copies of the .cer file in places accessible to the client/server (loaded to get the X509Certificate objects) and installed the with private keys in the Personal stores for the respective machines. Now, everything works when the client and server are on the same machine. But when I generate a new cert and put it on a second machine, the client and server now fail to mutually authenticate. The connection gets set up, encrypted and signed, but IsMutuallyAuthenticated is false. How can I fix this? What am I doing wrong?
Edit: LocalCertificateSelectionCallback and the other callback tell me that the client finds one local certificate, which is mine, no remote certificates, and no acceptable issuers. My certificate is selected and returned from the local certificate selection. There are no SSL policy errors on the client end, and on the host end, I still get RemoteCertificateNotAvailable. Sniffing this transaction with Wireshark, I see a standard TCP handshake, followed by a few hundred bytes of gibberish which includes the name of the host cert, and then the FIN, ACK packets that mark the end of the transaction. So the client certificate indeed isn't being sent, even though it's selected by the LocalCertificateSelectionCallback routine. What might cause this? How can I fix it?

There is a constructor of SSLStream which have RemoteCertificateVAlidationCallback parameter. It should help.

Calling HTTPS JBoss service from C# code throws an authentication error

I'm trying to call a JBoss service from a C# program and I'm getting an annoyingly vague error.
JbossService proxy = new JbossService();
proxy.Credentials = new NetworkCredential("ME", "thepwd");
proxy.Url = //https url snipped
proxy.CookieContainer = new CookieContainer();
proxy.PreAuthenticate = true;
Console.WriteLine("Calling service...");
queryResponse qr = proxy.query();
Console.WriteLine("Done.");
The exception and inner exception thrown are as follows:
exception : The underlying connection was closed: An unexpected error occurred on a send.
inner exception : Authentication failed because the remote party has closed the transport stream.
I'm not quite sure what this means, other than perhaps that JBoss likes me even less than I like it. I'm calling from the local machine so I don't think it's a networking issue. Has anyone seen this before?

This usually happens when your client cannot verify trust over https with the server (usually because the server certificate is self signed or if it is signed by a root authority not installed on your client machine.
Easy fix (although there are security consequences)....somewhere in your initialization code add the following:
System.Net.ServicePointManager.ServerCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) => { return true;};
Basically this replaces the application wide handling of server certificate validation and causes your application to accept any certificate. If you want to get finer grained, you can examine the certificate and put some logic in the method.
This works for anything based on System.Net, so it should work for Web Services and any thing based on WebRequest.

I haven't used JBOSS. This is how I troubleshoot similar problems, when using Microsoft technologies -- the same issues may be affecting your program:
Firewall settings or network issue (try connecting manually, to rule this out)
Self-service certificate issues:
Check the following certificate values:
Ensure the server's certificate issuer has a valid, matching issuing trusted root Certificate Authority (CA), on the same machine
The server certificate subject name matches the machine name exactly
The machine name the client is accessing matches that defined in the server certificate
An administrator account set (server) certificate thumbprint
Try recreating the SSL Certificate on both servers)
Try creating your own CA cert, add to trusted publishers, and then create an SSL sert based on that