I am new with MQTT protocol. When I searched for an MQTT server I found that Mosquitto broker is one of the most used one and therefore I have started using it. I have to develop an MQTT client on C#/.NET and I found only the M2Mqtt project and this C# MQTT client example.
I managed to install Mosquitto broker on Windows 10 and change the access control list using topics. Using MqttClient I am able to connect to the broker with an username, subscribe to topics and publish them with the following code.
Connect:
byte result = this.mqttClient.Connect(Guid.NewGuid().ToString(), username, string.Empty);
Subscribe:
this.mqttClient.Subscribe(new string[] { topic }, new byte[] { 2 });
Publish:
ushort result = this.mqttClient.Publish(topic, message, MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, true);
Now I want to add security in communications. I followed these steps to create the CA certificate, the server key and server certificate (I created the certificates twice). I changed the mosquito.conf file as explained in the link:
port 8883
cafile C:\mosquitto\certs\ca.crt
certfile C:\mosquitto\certs\server.crt
keyfile C:\mosquitto\certs\server.key
I don't know if it is necessary, but I added the ca.crt to the Trusted Root Certificates following these steps.
I changed the client to use the CA certificate in the connection:
//this.mqttClient = new MqttClient(brokerAddress);
X509Certificate caCertificate = new X509Certificate("ca.crt");
this.mqttClient = new MqttClient(brokerAddress, 8883, true, caCertificate, null, MqttSslProtocols.TLSv1_0);
A copy of the ca.crt file is in the same folder of the .exe file. When I run the application I always get the same exception:
uPLibrary.Networking.M2Mqtt.Exceptions.MqttConnectionException: Exception connecting to the broker
[Inner exception] System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
Do you know if I am missing something?
managed to get this working.
Instead of using a ca.crt I exported the certificate to a ca.pfx, installed the .pfx certificate in the client computer's Trusted Root Certification Authorities cache.
To install, Simply right click on the file, choose Local machine, and complete prompts as required. Critical to select local machine and select the proper certificate store ("Trusted Root Certification Authorities").
Related
I'm attempting to connect to an IMAP server using the following code on a Windows Server 2019 machine:
using (var client = new ImapClient(new ProtocolLogger("protocol.log")))
{
var address = EnvReader.GetStringValue("EMAIL_ADDRESS");
var password = EnvReader.GetStringValue("EMAIL_PASSWORD");
var creds = new NetworkCredential(address, password);
client.CheckCertificateRevocation = false;
client.ServerCertificateValidationCallback = (s, c, h, e) =>
{
Console.WriteLine("ALL UP IN THIS CALLBACK" + e.ToString());
return true;
};
client.Connect("outlook.office365.com", 993, SecureSocketOptions.SslOnConnect);
client.Authenticate(address, password);
}
On my Mac, this code runs perfectly fine, I can connect and subsequently authenticate just fine.
On the Windows machine I receive the following exception:
MailKit.Security.SslHandshakeException: An error occurred while attempting to establish an SSL or TLS connection.
This usually means that the SSL certificate presented by the server is not trusted by the system for one or more of
the following reasons:
1. The server is using a self-signed certificate which cannot be verified.
2. The local system is missing a Root or Intermediate certificate needed to verify the server's certificate.
3. A Certificate Authority CRL server for one or more of the certificates in the chain is temporarily unavailable.
4. The certificate presented by the server is expired or invalid.
5. The set of SSL/TLS protocols supported by the client and server do not match.
6. You are trying to connect to a port which does not support SSL/TLS.
See https://github.com/jstedfast/MailKit/blob/master/FAQ.md#SslHandshakeException for possible solutions
Based on the info in the linked FAQ, I added the ServerCertificateValidationCallback, however the callback is never hit (The previously mentioned exception is still thrown, the relevant console logging never occurs, and a breakpoint inside the callback is never hit while debugging).
From my reading, the ServerCertificateValidationCallback should handle cases #1-4 that the exception message mentions. The fact that I can connect on the specified port on my Mac would seem to rule out case #6 (I also tried port 143 + SecureSocketOptions.StartTls). That leaves case #5, however, I can't find any information suggesting that Windows Server 2019 can't handle SSL/TSL protocols.
Any ideas for a) dealing with this exception and/or b) figuring out why the ServerCertificateValidationCallback is not firing would be greatly appreciated.
Edit: My project is referencing .NET 5.0
Let's go through each of the possibilities:
The server is using a self-signed certificate which cannot be verified.
outlook.office365.com would not be using a self-signed certificate, so that wouldn't be an issue in this case.
The local system is missing a Root or Intermediate certificate needed to verify the server's certificate.
This one is very possible, but the ServerCertificateValidationCallback override should be overriding this failure. However, it's not getting hit... so it's not actually bypassing this potential error.
A Certificate Authority CRL server for one or more of the certificates in the chain is temporarily unavailable.
This would be negated by client.CheckCertificateRevocation = false;
The certificate presented by the server is expired or invalid.
This is not the case because the certificate does not expire until 1/21/2022.
The set of SSL/TLS protocols supported by the client and server do not match.
The server supports at least TLSv1.2 which is a default TLS protocol version supported by MailKit in all target framework versions (.NET 4.5 -> 5.0 + netstandard2.x's).
You are trying to connect to a port which does not support SSL/TLS.
Port 993 is the correct port and SslOnConnect is the correct option, so this is not the issue.
Assuming there isn't a bug in MailKit's SslStream.AuthenticateAsClientAsync() call that passes in the validation callback method (.NET 5.0 is different than other versions), what is the InnerException? Maybe that will provide some insight.
The Java Azure Iot Device Client has the "SetCertificatePath" option to add the root ca certificate in java code so that devices with symmetric key attestation can validate the gateway device.
import com.microsoft.azure.sdk.iot.device.DeviceClient;
...
client = new DeviceClient("HostName=xxx-dev-westeurope.azure-devices.net;DeviceId=demo-source-1;SharedAccessKey=xxx;GatewayHostName=demo-edge.dev.example.com",IotHubClientProtocol.MQTT);
client.setOption("SetCertificatePath", "C:\\certs\\device_ca_aliasazxVrrdEVxd7kvKvne1pOEyuSHF8EXSowNDhMzl30jI_.cert.pem");
client.open();
Is there a similar option in the C# Device Client, like client.setOption("SetCertificatePath",...)?
using Microsoft.Azure.Devices.Client;
...
var deviceClient = DeviceClient.CreateFromConnectionString("HostName=xxx-dev-westeurope.azure-devices.net;DeviceId=demo-source-1;SharedAccessKey=xxx;GatewayHostName=demo-edge.dev.example.com", TransportType.Mqtt);
await deviceClient.OpenAsync();
With this code I get an invalid certifacte error:
AuthenticationException: TLS authentication error.
AggregateException: One or more errors occurred. (The remote certificate is invalid according to the validation procedure.)
Does the C# Azure Iot Device Client have a similar option?
The C# client picks up CA certificates from the local system's Trusted Root Certificate Authority store.
Be sure to use the root certificate, something like: edge_owner_azxVrrdEVxd7kvKvne1pOEyuSHF8EXSowNDhMzl30jI_.cert.pem
You may also have to transform the .pem file into .crt file to be able to add it to your local store.
openssl x509 -outform der -in .\edge_owner_azxVrrdEVxd7kvKvne1pOEyuSHF8EXSowNDhMzl30jI_.cert.pem -out .\edge_owner_azxVrrdEVxd7kvKvne1pOEyuSHF8EXSowNDhMzl30jI_.cert.crt
Links:
How to add a certificate to Windows certificate store
Azure IoT Hub certificates - important to know
Not quite, I believe. As far as I know, the C# SDK relies on the certificate being in the Windows or Linux certificate store. It's mentioned in this doc as well.
There is a sample where the path to the certificate is provided to the application, which then opens the certificate store and saves the cert to it. I hope this is sufficient for your case!
When calling AuthenticateAsClient(), I receive the error "Call to SSPI failed." with the inner exception "The message received was unexpected or badly formatted".
I've seen this to be a semi-popular problem, but I haven't been able to find a working solution. Here is where I'm at after scouring articles for a few hours:
Using my private key, the server issued me a signed client cert
Using open ssl, I combined the two into a pfx using:
openssl pkcs12 -in My-Client-Cert.pem -inkey ssl-client-privatekey.pem -export -out private-key-pair.pfx
I imported the cert to the personal folder on LocalMachine. I was able to see that cert recognized that a private key was present.
The private key is not password protected
I imported the CA Root for the service I'm talking to into both my Trusted Root Certification Authorities and Personal stores.
In app code, I pull the cert by thumbnail. I can see in the debugger that both private and public keys are present.
I initialize a TcpClient
I then initialize an SslStream using the the TcpClient
I call AuthenticateAsClient()
I receive the error message
For the hostname parameter of AuthenticateAsClient() I have also tried using the CN of the CA root and also the CN of the client cert with the same result.
I verified that the service I'm trying to talk to does work just fine when I connect to it using openssl s_client connect
Here is the code for the connection:
private void Setup(string hostname, int port, X509Certificate2Collection certs)
{
try
{
// create the socket
var clientSocket = new TcpClient(hostname, port);
_sslStream = new SslStream(
clientSocket.GetStream(),
false,
ValidateServerCertificate,
null);
_sslStream.AuthenticateAsClient(
"VirtuCrypt Root CA - Test", // hostname here must match the name on the server certificate
certs,
SslProtocols.Tls11,
false);
Debug.WriteLine("Connected!");
}
catch(Exception ex)
{
Debug.WriteLine(ex.Message);
if(ex.InnerException != null)
Debug.WriteLine(ex.InnerException.Message);
}
}
Pulling my hair out... any thoughts?
UPDATE:
I spun up Wireshark and noticed that all communication to and from the host was done over TCP, and none of it was done over TLS. Not sure if that's something I should be concerned with despite the fact that I am requesting TLS12
UPDATE2:
I'm looking at Wireshark again and I fixed the decode so that it shows the TLS communications in addition to the TCP connection. Now that I can see the handshake, I can see that no client cert is actually being presented. I'm providing that cert to SslStream, so I don't know why it's not being transmitted.
Ok after finally making friends with Wireshark and breaking about 4 coffee mugs chasing my tail, here's what I found out...
The CA root that was given to us was a pem file that actually included the CA root and 3 intermediate certs.
I had to break those into separate files and import the CA root into our trusted root CA store and the 3 intermediate certs into the intermediate CA store
I hadn't noticed before, but our client cert that was installed in the personal store was saying it didn't have enough info to be validated, but what was really going on is that it was failing validation because we didn't have the CA certs installed correctly as described above. Once I installed those separately, this message went away
It appears that .Net will not send your client cert unless it can be validated. In my 2nd edit in the OP, I mentioned that I could see the client cert was not being sent at all. This was the actual problem. Once I installed these CA certs, the cert started getting sent and everything else was good.
I'm creating a TCP proxy with C# using TcpListener for the proxy server and TcpCLient for the communication between client and proxy and between proxy and target server. This works really nice.
I also have to support SSL and TLS encrypted communication.
This works almost well. I create a SslStream from the proxy to the target server with this Code:
var sslStream = new SslStream(remoteStream, false);
sslStream.AuthenticateAsClient(state.RemoteHost);
And I create a SslStream from the proxy to the Client with the following code:
var sslStream = new SslStream(state.ClientStream, false);
sslStream.AuthenticateAsServer(certificate, false, SslProtocols.Tls | SslProtocols.Ssl3 | SslProtocols.Ssl2, true);
The certificate is loaded from the X509Store:
X509Certificate2 certificate;
var store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadOnly);
var certificates = store.Certificates.Find(X509FindType.FindBySubjectDistinguishedName, "CN=localhost", false);
store.Close();
if (certificates.Count == 0)
{
Console.WriteLine("Server certificate not found...");
return;
}
else
{
certificate = certificates[0];
}
This also works well if i force the clients to trust the certificate manually.
My questions are:
How can I force the (all) clients to trust the certificate?
Which kind of certificate which is valid for all clients do I need on the proxy?
If needed, what kind of client certificate do I have to install to force the clients to trust the proxy?
How can I create the needed kinds of proxy with openssl or makecert?
I don't want to tunnel the SSL communication threw the proxy because I need to read and manipulate the streams.
[UPDATE]
Yes I used Google and the search in StackOverflow and I tried some different solution without any success.
I also tried the solutions in the following threads:
SSLStream example - how do I get certificates that work?
How do I identify my server name for server authentication by client in c#
[UPDATE2]
This is a very good tutorial to create a CA and a server certificate with openssl, but it doesn't work for me:
http://webserver.codeplex.com/wikipage?title=HTTPS&referringTitle=Home
There is no single certificate which is valid for all requests. So my idea doesn't work.
Because it is not possible to generate a single license for every domain name.
But the Answer is easier than expected: To solve my problem I have to create a certificate for every single request.
All I need to do is:
to create a self-signed root certificate as a certificate authority
to install this in the clients "Trusted Root Certification Authorities" store.
to create the server certificates on-the-fly for every incoming request
sign this certificates with the root certificates tree to set the issuer
I can cache the certificates in files or in a system certificate store if needed.
(this is completely the same in fiddler)
I have a weird requirement. I am trying to communicate with a server written in C#. It looks like this basically:
SslStream sslStream = new SslStream(client.GetStream(), true,
ValidateServerCertificate,
SelectLocalCertificate);
sslStream.AuthenticateAsServer(_pushCert);
I also have example code in C# that uses a X509 certificate and connects to the server. I have the password for the cert.pfx file as well.
What I would like to do is setup some kind of shell script that can connect to the socket, transmit a few bytes and receive the response. (any language really, although I was looking at Python or Ruby or Perl)
I tried using the SSL wrapper from Python, but I get an error stating their is no known algorithm for the server/client to talk.
Example of my Python code:
ss = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s = ssl.wrap_socket(ss, ca_certs=CERT, ssl_version=ssl.PROTOCOL_SSLv23 )
#Attempt connection to our server
try:
s.connect((HOST, PORT))
print s
except:
print 'ERROR Connecting'
sys.exit(0)
For CERT I tried a few different filee: the .pfx, and some extracted from the .pfx using openssl.
I tried many different examples as well (Arguments for the ssl.wrap_socket). I am not really familiar with these connections either.
Perhaps someone here could lend a hand?
Thanks!
You can simplify your SslStream constructor call:
SslStream sslStream = new SslStream(client.GetStream());
sslStream.AuthenticateAsServer(_pushCert);
This server sends _pushCert and does not expect the client to send a certificate back. The server needs the private key for the certificate to make the SSL connection.
The client only needs the CA root certificate that signed the server certificate (or, an option to accept an untrusted certificate.) This needs to be in the "trusted root certificate store" or otherwise identified as trusted to the client wrapper.
If the server certificate is signed by an intermediate CA certificate that is itself signed by the root CA certificate, the client needs that intermediate certificate too. That can be sent by the server, or can already be at the client. Either way, the entire chain of signing certificates has to be in hand at the client to verify all of the signatures along the chain. The intermediate CA certificate does not need to be in the trusted root store.
Neither side needs a private key for the CA root, or for an intermediate signing certificate.
However, if your server expects the client to send a client certificate, then you have to call AuthenticateAsServer with more arguments (clientCertificateRequired == true). In that case, the client needs both its own certificate and the private key for its certificate. The server needs the CA root that signs the client certificate in its trusted store. The client wrapper will take a pfx file, for example, containing the client certificate and private key. The server does not need the client's private key.