We Keep Coding

Kafka Producer in .Net - SSL Handshake Failed

This is the first time I am trying to connect to Kafka server using Asp.Net console app and I ran into a bunch of issues. I solved most but can't shake this one off.
I am testing the producer part and keep getting "SSL Handshake Failed" error. I followed some suggestions here, including setting SslEndpointIdentificationAlgorithm to blank.
This is my producer config section:
_producerConfig = new ProducerConfig
{
BootstrapServers = bootstrapServer,
EnableDeliveryReports = true,
ClientId = Dns.GetHostAddresses(),
//SecurityProtocol = SecurityProtocol.SaslSsl,
SecurityProtocol = SecurityProtocol.Ssl,
SslCaLocation = #"D:\Something\Certificate\myCert.pem",
SslKeyPassword = "blahblah",
SslEndpointIdentificationAlgorithm=SslEndpointIdentificationAlgorithm.None
};
...
public async Task StartSendingMessages(string topicName)
{
using (var producer = new ProducerBuilder<long, string>(_producerConfig)
.SetKeySerializer(Serializers.Int64)
.SetValueSerializer(Serializers.Utf8)
.SetLogHandler((_, message) => Console.WriteLine($"Facility: {message.Facility}-{message.Level} Message: {message.Message}"))
.SetErrorHandler((_, e) => Console.WriteLine($"Error: {e.Reason}. Is Fatal: {e.IsFatal}"))
.Build())
try
{
Console.WriteLine("\nProducer loop started...\n\n");
for (var character = 'A'; character <= 'C'; character++)
{
var message = $"Character #{character} sent at {DateTime.Now:yyyy-MM-dd_HH:mm:ss}";
var deliveryReport = await producer.ProduceAsync(topicName,
new Message<long, string>
{
Key = DateTime.UtcNow.Ticks,
Value = message
});
When I run the application, I get:
Facility: FAIL-Error Message: [thrd:ssl://1.2.3.4:9093/bootstrap]: ssl://1.2.3.4:9093/bootstrap: SSL handshake failed: ssl\statem\statem_clnt.c:1890:tls_post_process_server_certificate error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 46ms in state SSL_HANDSHAKE)
Error: ssl://1.2.3.4:9093/bootstrap: SSL handshake failed: ssl\statem\statem_clnt.c:1890:tls_post_process_server_certificate error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 46ms in state SSL_HANDSHAKE). Is Fatal: False
What am I doing wrong, or missing?

c# How to create X509 Certificate from certificate and key or pfx or p7b

I've created a C# SSL TCP webserver primarily using the example from the SslStream docs, and I've created and downloaded an SSL Certificate (with sslforfree) and created pkcs#7/p7b and pkcs#12 (with a blank password) certs using openssl through powershell. I have found no way to use pkcs# 12 or 7 yet.
(using OpenSSL.X509Certificate2Provider and System.Security.Cryptography.X509Certificates)
The most success was using this code:
string certfiletext = File.ReadAllText(Path.Combine(workingpath, "certificate.crt"));
string privatekeytext = File.ReadAllText(Path.Combine(workingpath, "private.key"));
ICertificateProvider provider = new
CertificateFromFileProvider(certfiletext, privatekeytext);
serverCertificate = provider.Certificate;
This code gets past the sslStream.AuthenticateAsServer requirement, but fails at bytes = sslStream.Read(buffer, 0, buffer.Length);
The stack trace associated with this failure expresses that The decryption operation failed, see inner exception, with inner exception being: Win32Exception: An unknown error occurred while processing the certificate.
This leads me to believe the certificate isn't created properly. The other method I've gotten anywhere with is this:
serverCertificate =
new X509Certificate2(Path.Combine(workingpath, "certificate.pfx"), "");
but that causes the same issue as the prior code.(Decryption error due to processing certificate error). This error appears whether the certificate has been installed on my machine or not. There is also no issue with the server finding the correct files.
This is the code I use for a JS client side embedded in html:
wsUri = "wss://thisismyip:11000",
websocket = new WebSocket(wsUri);
websocket.onopen = function (e) {
websocket.send(window.location.hostname + window.location.pathname.substring(1));
};
websocket.onclose = function (e) {};
websocket.onmessage = function (e) {
//server response, do stuff when it responds
};
websocket.onerror = function (e) {};
While I doubt the issue is related to the JS code, I might have missed something important.
Summary: I can't create an SSL connection between a javascript websocket and a C# TCP server due to a server-side certificate issue.

I've created such code for this certificate issue. Maybe it will help you:
private const string _password = "123";
private const string _certificatePath = certificate.pfx";
public static HttpClient Build()
{
HttpClientHandler clientHandler = new HttpClientHandler();
clientHandler.SslProtocols = SslProtocols.Tls12;
clientHandler.ClientCertificateOptions =
ClientCertificateOption.Manual;
// to hide any ssl errors
clientHandler.ServerCertificateCustomValidationCallback = (sender,
cert, chain, sslPolicyErrors) => { return true; };
var cert = new X509Certificate2(_certificatePath, _password);
clientHandler.ClientCertificates.Add(cert);
return new HttpClient(clientHandler);
}

Same x509certificate in server and client

I have the same certificate (use x509certificate2) in the application and in the server (both in C#),
I have to extend the validity of the certificate file in the server,
I use in SSL3 with the certificate to connect to soap web service from client (with set or get)
My question Is it mandatory for the same certificate to be in the app?
try
{
// Initiate a new request to the WS
System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls;
MyServiceSSL.MyWebServiceClient clClient = new MyServiceSSL.MyWebServiceClient();
clClient.Endpoint.Address = new System.ServiceModel.EndpointAddress(sConnStr);
ServicePointManager.ServerCertificateValidationCallback += new RemoteCertificateValidationCallback(ValidateRemoteCertificate);
try
{
clClient.ClientCredentials.ClientCertificate.SetCertificate(
StoreLocation.LocalMachine,
StoreName.TrustedPeople,
X509FindType.FindBySubjectName,
"myDomain.com");
}
catch (Exception e)
{
...
}
MyServiceSSL.myDataRequest clRequest = new MyServiceSSL.myDataRequest();
// Set up request parameters
clRequest = new myDataRequest();
clResult = clClient.myFunc(clRequest.MyRequest_1);

I have to extend the validity of the certificate file in the server,
Does that mean the validity date? If so, of course you need to renew both otherwise one would be expired and therefore not valid any more.
BTW: You should not use SSL3

How to scan and verify the version of TLS for a website?

How to scan and verify the version of TLS for a website?
I was hunting for solutions which can automatically scan all the endpoint website domain in our company. SSL Labs had a manual online solution
https://www.ssllabs.com/ssltest/index.html
How can this feature be achieved programmatically?

I had same issue. Created free and open-source project on GitHub https://github.com/JocysCom/SslScanner, which contains:
Portable standalone SSL, TLS, STARTTLS Scanner Tool.
/Tool/Common/Test_SSL_Support.cs - C# code which will check SSL/TLS version
/Tool/Common/Test_SSL_Support.bat - simple batch script which runs C# like a script on Windows Command Line. Note: I use PowerShell and Batch on Windows to run C# like a scripting language in order to automate tons of stuff.
Test_SSL_Support.cs have 'TestTCP' method which returns 'true' if connection supports specified protocol version:
static bool TestTCP(string host, int port, SslProtocols protocol, out bool connected)
{
var success = false;
var client = new TcpClient();
var asyncResult = client.BeginConnect(host, port, null, null);
// 5 seconds timeout.
connected = asyncResult.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(5));
// Connected.
if (connected)
{
var stream = client.GetStream();
// Don't dispose underlying stream.
using (var sslStream = new SslStream(stream, true, ValidateServerCertificate))
{
sslStream.ReadTimeout = 15000;
sslStream.WriteTimeout = 15000;
sslStream.AuthenticateAsClient(host, null, protocol, false);
result.UpdateFromSslStream(sslStream);
success = true;
}
client.EndConnect(asyncResult);
}
return success;
}
Test_SSL_Support.cs supports StartTLS protocol too. Look for method:
static bool TestStarTLS(string host, int port, SslProtocols protocol, out bool connected)
You can add multiple hosts and ports to Test_SSL_Support.bat for scan:
:: Test SSL/TLS.
CALL:PS www.google.com 443
:: Test StartTLS.
CALL:PS mail.jocys.com 110
Command line results are represented as:
172.217.169.4 www.google.com:443
Ssl2 = False | The client and server cannot communicate, because they do not possess a common algorithm
Ssl3 = False | The client and server cannot communicate, because they do not possess a common algorithm
Tls = True | Exchange = ECC-256 | Cipher = AES128 | Hash = SHA1
Tls11 = True | Exchange = ECC-256 | Cipher = AES128 | Hash = SHA1
Tls12 = True | Exchange = ECC-256 | Cipher = AES128 | Hash = SHA256
Tls13 = True | Exchange = ECC-256 | Cipher = AES256 | Hash = SHA384
62.30.149.144 mail.jocys.com:110
Ssl2 = False | The client and server cannot communicate, because they do not possess a common algorithm
Ssl3 = False | The client and server cannot communicate, because they do not possess a common algorithm
Tls = False | Authentication failed because the remote party has closed the transport stream.
Tls11 = False | Authentication failed because the remote party has closed the transport stream.
Tls12 = True | Exchange = ECC-384 | Cipher = AES256 | Hash = SHA384
Tls13 = False | Authentication failed because the remote party has closed the transport stream.

SSLLabsAPI seems to work
using SslLabsLib.Enums;
using SslLabsLib;
public static void TLSChecker(string urlString)
{
SslLabsClient client = new SslLabsClient();
// Get ipaddress and host
var analysis = client.GetAnalysis(urlString, 24, AnalyzeOptions.Publish);
Console.WriteLine(analysis.Host);
foreach (var item in analysis.Endpoints)
{
Console.WriteLine(item.IpAddress);
var endpointanalysis = client.GetCachedEndpointAnalysis(analysis.Host, IPAddress.Parse(item.IpAddress));
// get protocol list
foreach (var protocol in endpointanalysis.Details.Protocols)
{
Console.WriteLine(protocol.Id);
Console.WriteLine(protocol.Name);
Console.WriteLine(protocol.Version);
}
}

Use HttpClient... force it to use TLS 1.2... see if it fails... profit.
public async Task<bool> SupportsTls12(string url)
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var client = new HttpClient();
try
{
var response = await client.GetAsync(url);
return true;
}
catch(HttpRequestException)
{
return false;
}
}
If you find this gives you false positive/negatives, you may have to modify the exception handler to inspect the specific error that is returned. But this is the general idea.

C# - TLS / SSL Websockets using fleck

Im trying to make a TLS/SSL websocket connection using Fleck lib.
https://github.com/statianzo/Fleck (0.9.8.25)
now i got the server startet .
but when a client connects i get the following message.
28-02-2014 19:16:15 [Info] Server started at wss://localhost:8081
28-02-2014 19:18:51 [Debug] Client connected from 127.0.0.1:62543
28-02-2014 19:18:51 [Debug] Authenticating Secure Connection
28-02-2014 19:18:52 [Debug] 0 bytes read. Closing.
anybody got an idea of what im doing wrong ?
Browser: Chrome, version : 33.0.1750.117
// sample code.
FleckLog.Level = LogLevel.Debug;
var allSockets = new List<IWebSocketConnection>();
var server = new WebSocketServer("wss://localhost:8081");
server.Certificate = new X509Certificate2(#"CRT.pfx", "Pwd");
server.Start(socket =>
{
socket.OnOpen = () =>
{
Console.WriteLine("Open!");
allSockets.Add(socket);
};
socket.OnClose = () =>
{
Console.WriteLine("Close!");
allSockets.Remove(socket);
};
socket.OnMessage = message =>
{
foreach (var user in allSockets.ToList())
{
if(user.ConnectionInfo.Id != socket.ConnectionInfo.Id){
user.Send(message);
}
}
};
});
var input = Console.ReadLine();
while (input != "exit")
{
foreach (var socket in allSockets.ToList())
{
socket.Send(input);
}
input = Console.ReadLine();
}

Is certificate signed by a browser trusted CA? If not, the web page you are opening with Chrome has to be under HTTPS so you can be prompted to accept the certificate, otherwise the connection will fail.
If even doing that does not work, please try with WebSocketListener and tell me which error you get if any.
Some issues I found debugging WSS:
Remember to change the port number to a one different to the one you used for not secure connections. Some browsers get confused if suddenly a port becomes secure or viceversa.
Remember to use the hostname indicated in the certificate to connect and not the IP.
If you are using a self-signed certificate, use it for HTTPS so you can see the dialog for accepting that certificate. When accessing via WSS:// there is not certificate acceptance dialog, it will just fail to connect.
Try with a self-signed certificate as well and see if it works.