We Keep Coding

Kafka Producer in .Net - SSL Handshake Failed

This is the first time I am trying to connect to Kafka server using Asp.Net console app and I ran into a bunch of issues. I solved most but can't shake this one off.
I am testing the producer part and keep getting "SSL Handshake Failed" error. I followed some suggestions here, including setting SslEndpointIdentificationAlgorithm to blank.
This is my producer config section:
_producerConfig = new ProducerConfig
{
BootstrapServers = bootstrapServer,
EnableDeliveryReports = true,
ClientId = Dns.GetHostAddresses(),
//SecurityProtocol = SecurityProtocol.SaslSsl,
SecurityProtocol = SecurityProtocol.Ssl,
SslCaLocation = #"D:\Something\Certificate\myCert.pem",
SslKeyPassword = "blahblah",
SslEndpointIdentificationAlgorithm=SslEndpointIdentificationAlgorithm.None
};
...
public async Task StartSendingMessages(string topicName)
{
using (var producer = new ProducerBuilder<long, string>(_producerConfig)
.SetKeySerializer(Serializers.Int64)
.SetValueSerializer(Serializers.Utf8)
.SetLogHandler((_, message) => Console.WriteLine($"Facility: {message.Facility}-{message.Level} Message: {message.Message}"))
.SetErrorHandler((_, e) => Console.WriteLine($"Error: {e.Reason}. Is Fatal: {e.IsFatal}"))
.Build())
try
{
Console.WriteLine("\nProducer loop started...\n\n");
for (var character = 'A'; character <= 'C'; character++)
{
var message = $"Character #{character} sent at {DateTime.Now:yyyy-MM-dd_HH:mm:ss}";
var deliveryReport = await producer.ProduceAsync(topicName,
new Message<long, string>
{
Key = DateTime.UtcNow.Ticks,
Value = message
});
When I run the application, I get:
Facility: FAIL-Error Message: [thrd:ssl://1.2.3.4:9093/bootstrap]: ssl://1.2.3.4:9093/bootstrap: SSL handshake failed: ssl\statem\statem_clnt.c:1890:tls_post_process_server_certificate error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 46ms in state SSL_HANDSHAKE)
Error: ssl://1.2.3.4:9093/bootstrap: SSL handshake failed: ssl\statem\statem_clnt.c:1890:tls_post_process_server_certificate error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 46ms in state SSL_HANDSHAKE). Is Fatal: False
What am I doing wrong, or missing?

Advantage or disadvantage when enabling SkipNegotation in SignalR (.net core) when using with aws eks

We are using AWS kubernetes cluster (EKS) with an ALB/Ingress and redis-cache for signalr connections. If we have a replicaset of 3, connecting to the services throws randomly a 404 error. When setting the options:
SkipNegotiation = true
Transports = Microsoft.AspNetCore.Http.Connections.HttpTransportType.WebSockets
Everything seems to work. I could not find any disadvantages of setting SkipNegotation to true. But could not find whether this has any side effects (there must be a reason it is set to false by default).
Could setting SkipNegotation to true cause any further problems?
Sample code that is working:
Console.WriteLine("Start");
HubConnection connection = new HubConnectionBuilder()
.WithUrl("https://<url>/api/v1/taskboard", o =>
{
o.Transports = Microsoft.AspNetCore.Http.Connections.HttpTransportType.WebSockets;
o.SkipNegotiation = true;
})
.WithAutomaticReconnect()
.Build();
await connection.StartAsync();
connection.On<string>("ReceiveMessage", message =>
{
Console.WriteLine(message);
});
for (int i = 0; i < 20; i++)
{
await connection.InvokeAsync("SendMessage", $"Hello {i}");
await Task.Delay(55);
}
await connection.StopAsync();
await connection.DisposeAsync();
EDIT
Those settings has been added to the ingress:
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:......
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
alb.ingress.kubernetes.io/ssl-redirect: '443'
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/subnets: subnet-..., subnet-...
alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=600
alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=600
alb.ingress.kubernetes.io/healthcheck-path: /HealthCheck
alb.ingress.kubernetes.io/healthcheck-port: '80'

The main reason to use SkipNegotiation is to avoid using sticky sessions.
When you want to establish a connection with the SignalR server, the client sends a post request to the server, and the connection wont be established until the client receives a response from the server:
Client request:
{
"connectionId":"807809a5-31bf-470d-9e23-afaee35d8a0d",
"availableTransports":[
{
"transport": "WebSockets",
"transferFormats": [ "Text", "Binary" ]
},
{
"transport": "ServerSentEvents",
"transferFormats": [ "Text" ]
},
{
"transport": "LongPolling",
"transferFormats": [ "Text", "Binary" ]
}
]
}
Server response:
{
"url": "https://myapp.com/chat",
"accessToken": "accessToken"
}
To avoid using sticky sessions, the client needs to skip negotiation but is restricted to only using websockets.
Another thing to note is that the method WithAutomaticReconnect has [0, 2000, 10000, 30000, null] as the default values. Meaning that, if the client gets disconnected from the server, it will wait 0, 2, 10 and 30 seconds respectively before trying to reconnect again. If it doesnt get reconnected at that point it will stop trying. If you want to have your client trying to reconnect infinitely you have to create your own retry policy:
public class RetryPolicyLoop : IRetryPolicy
{
private const int ReconnectionWaitSeconds = 5;
public TimeSpan? NextRetryDelay(RetryContext retryContext)
{
return TimeSpan.FromSeconds(ReconnectionWaitSeconds);
}
}
And in your HubConnectionBuilder:
.WithAutomaticReconnect(new RetryPolicyLoop())
That way your client will try to reconnect infinitely (waiting 5 seconds after each failed attempt).
EDIT:
Source:
https://github.com/aspnet/SignalR/blob/release/2.2/specs/TransportProtocols.md

Smart home report state SYNC new device | Requested entity was not found. [404]

I've implemented the HomeGraph API with the help of the package Google.Apis.HomeGraphService.v1 (1.50.0.2260)
It seems to work fine as well, the ReportStateAndNotification function works fine on the query, execute, and some sync requests.
But when I add a new device to my system through our app and a SYNC request is sent to Google and comes in our backend, the HomeGraph API will return an exception when sending this sync request..
-> The sync request does not throw an exception when I modify a device name in our app. It only occurs when new devices are added.
I've searched through google and multiple StackOverflow posts.. But I'm probably missing something. Most posts say check the API key etc but then the ReportStateAndNotification function should always fail, not only when the sync request comes from Google to our backend.
Could anyone point me in the right direction?
Function that is used for sync requests:
public static void Send(Dictionary<string, object> deviceStateList, string requestId, string googleCustomerId)
{
string deviceIdList = String.Format("({0})", string.Join(", ", deviceStateList.Keys));
try
{
var jsonFilePath = _appSettingsRetriever.PrivateGoogleAuthenticationFile;
string scope = "https://www.googleapis.com/auth/homegraph";
using (var stream = new FileStream(jsonFilePath, FileMode.Open, FileAccess.Read))
{
GoogleCredential credentials = GoogleCredential.FromStream(stream);
if (credentials.IsCreateScopedRequired)
credentials = credentials.CreateScoped(scope);
HomeGraphServiceService service = new HomeGraphServiceService(new BaseClientService.Initializer()
{
HttpClientInitializer = credentials
});
var request = new ReportStateAndNotificationRequest
{
AgentUserId = googleCustomerId,
RequestId = requestId,
Payload = new StateAndNotificationPayload
{
Devices = new ReportStateAndNotificationDevice
{
States = deviceStateList
}
}
};
_log.Debug($"Sending to HomeGraph for devices: {deviceIdList} customer: {googleCustomerId} requestId: {requestId}");
DevicesResource.ReportStateAndNotificationRequest rp = service.Devices.ReportStateAndNotification(request);
ReportStateAndNotificationResponse resop = rp.Execute();
}
}
catch (Exception ex)
{
_log.Error($"Exception in ReportToHomeGraph for Customer: {googleCustomerId}. DeviceList: {deviceIdList}. JsonPath: {_appSettingsRetriever.PrivateGoogleAuthenticationFile} Exception: {ex}.");
}
}
Exception:
2021-09-24 14:16:13,547 [110] ERROR ReportToHomeGraph
Exception in ReportToHomeGraph for Customer: 05. DeviceList: (
fe965e6a-21ad-425f-b594-914bf63510a9,
1cc0ee97-a87f-44c5-a3e3-a39d159ee193,
618cdf94-2b31-434f-b91e-00837d155d4a
).
JsonPath: C:/myfile.json Exception: The service homegraph has thrown an exception:
Google.GoogleApiException: Google.Apis.Requests.RequestError
Requested entity was not found. [404]
Errors [
Message[Requested entity was not found.] Location[ - ] Reason[notFound] Domain[global]
]
at Google.Apis.Requests.ClientServiceRequest`1.<ParseResponse>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Google.Apis.Requests.ClientServiceRequest`1.Execute()
at BusinessLogic.GoogleAssistant.TokenService.HomeGraph.ReportToHomeGraph.Send(Dictionary`2 deviceStateList,
String requestId, String googleCustomerId) in C:\Repos\GoogleAssistant
.TokenService\HomeGraph\ReportToHomeGraph.cs:line 57.

When users add a new device, the first step you need to do is to issue a Request Sync to Google. This indicates the set of devices for that user has changed, and you need a new Sync request to update the data in homegraph. Google will follow this by delivering a Sync intent to your fulfillment endpoint, which you can respond with the updated set of devices.
Getting a 404 when calling Request Sync might indicate your Service Account Key might be invalid, or the agent user id you target might be wrong. Otherwise getting an error for your Sync Response might indicate it’s structured incorrectly. You can find out more about how to structure it in our examples.

I get "Remote computer is not available" (MessageQueueException) when trying to read queue of remote server

I am trying to reach into and read the private queues of a remote server.
I have tried this locally with success:
MessageQueue[] QueueList = MessageQueue.GetPrivateQueuesByMachine(".");
foreach(MessageQueue queueItem in QueueList)
{
Console.WriteLine(queueItem.Path);
}
And i have tried this pointing at the remote server with no success:
MessageQueue[] QueueList = MessageQueue.GetPrivateQueuesByMachine(#"FormatName:DIRECT=OS:XXX.XX.X.XXX");
foreach(MessageQueue queueItem in QueueList)
{
Console.WriteLine(queueItem.Path);
}
also tried with no success
var queue = new MessageQueue(#"FormatName:DIRECT=OS:XXX.XX.X.XXX\Private$\test-queue");
Console.WriteLine(queue.GetAllMessages().Count());
I want to get the list of private queues on the remote server.
(and count the messages in the queues)

SignalR: The remote server returned an error: (500) Inter nal Server Error

I have installed both client and server of signalR in my MVC project; and I am able to call my hub from the web client;
var testHub;
$(function () {
// Setup SignalR
testHub = $.connection.myhub;
testHub.msg = function (ref) {
alert(ref);
};
$.connection.hub.start(function () {
testHub.addToQueue("TESTMSG");
});
});
And on a controller, I need to call my hub after processing some request; so I have added references to SignalR.Client.dll -Version(0.5.2.0) and Newtonsoft.Json.dll(4.5.0.0) runtime 4.
My controller call is as below
var connection = new HubConnection("http://localhost:50439/");
IHubProxy myHub = connection.CreateProxy("myhub");
connection.Start().Wait();
myHub.Invoke("addToQueue", new { message = "Hello world" }).ContinueWith(task =>
{
}, TaskContinuationOptions.OnlyOnFaulted);
And my hub looks as below:
[HubName("myhub")]
public class QueueHub:Hub
{
public void addToQueue(string message)
{
Clients.msg(message);
}
}
But when the call from controller happens, the system gets below error:
Error reading string. Unexpected token: StartObject. Path '', line 1, position 46., JsonSerializer
And In details:
---> (Inner Exception #0) System.AggregateException: One or more errors occurred
. ---> System.Net.WebException: The remote server returned an error: (500) Inter
nal Server Error.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at SignalR.Client.Http.HttpHelper.<>c__DisplayClass2.<GetHttpResponseAsync>b_
_0(IAsyncResult ar) in C:\Documents and Settings\mrafeeq\My Documents\Downloads\
SignalR-SignalR-0.5.0-231-g7808c90\SignalR-SignalR-7808c90\SignalR.Client\Http\H
ttpHelper.cs:line 19

You're calling the server side method incorrectly. You need to do this:
myHub.Invoke("addToQueue", "Hello world").ContinueWith(task =>
{
}, TaskContinuationOptions.OnlyOnFaulted);