We Keep Coding

Not able to connect with broker using MQTTNet library .net core

I need an help, I can't connect with the broker.
I'm using MQTTNet library into my api project .net core
this is my code:
// GET: api/<SendCommandController>
[HttpGet]
public void Get()
{
var options = new MqttClientOptionsBuilder()
.WithTcpServer("broker.hivemq.com", 1883)
.Build();
var factory = new MqttFactory();
var mqttClient = factory.CreateMqttClient();
mqttClient.ConnectAsync(options, CancellationToken.None);
var message = new MqttApplicationMessageBuilder()
.WithTopic("Test/Mqtt")
.WithPayload("Hello World")
.WithExactlyOnceQoS()
.WithRetainFlag()
.Build();
mqttClient.PublishAsync(message, CancellationToken.None);
}
so I follow the tutorial but can't connect to broker hivemq and I can't connect to my personal broker.
So, I tested hivemq broker with mqtt.fx and works.
Only in the code the return is connected = false.
Any ideas? the error is "the client is not connected"

C# is not a language I've done much with, but I assume you are missing an await before mqttClient.ConnectAsync(options, CancellationToken.None); so the rest of the code waits for the connection to complete before trying to send the message

ZeroMQ architecture for cache-like service

I made a service which accepts requests on a 0MQ router socket in NetMQ, and acts mostly like a cache for other apps, i.e. it can receive a "GET" request and send a response, or it can receive a "PUT" request which doesn't need a response.
I have two issues though:
I tried using a RequestSocket in client code, but it still fails if I don't wait for a response after a sent message (I guess this is how REQ/REP works in 0MQ). What is the appropriate 0MQ socket type if I only want to get a response for some message types?
Currently my client code creates a new RequestSocket instance whenever it needs to communicate with the service. Is there a thread-safe way in NetMQ where I could have a single socket that can be reused over the client code when I need to send a request and get a response, without the need to create a new connection each time?
The service code is currently basically:
var socket = new RouterSocket(endpoint);
socket.ReceiveReady += (sender, e) =>
{
var msg = e.Socket.ReceiveMultipartMessage();
var action = msg[3].ConvertToString();
switch (action)
{
case "GET":
{
var key = msg[4].ConvertToString();
var data = _cache.Get(key);
var response = new NetMQMessage();
response.Append(msg[0]); // msg id
response.Append(NetMQFrame.Empty);
response.Append(msg[4]);
response.Append(data);
e.Socket.SendMultipartMessage(response);
}
break;
case "SET":
{
var key = msg[4].ConvertToString();
var data = msg[5].Buffer;
_cache.Set(key, data);
// no response needed here, but needed for RequestSocket clients
}
break;
default: throw new NotImplementedException();
}
};

"An operation on a socket could not be performed" with HttpClientFactory

I'm having a long-running triggered webjob on Azure, that uses multiple typed http clients to access remote services. The amount of requests per launch goes as far as millions, but after ~30k I'm getting the following exception:
System.Net.Http.HttpRequestException: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.
I'm using the Microsoft.Extensions.Http 2.2.0 to inject typed http clients with Http Client Factory.
By now I've tried running this locally with fake data and receiver; and monitor the connections with
netstat -b
...and
var ipProperties = IPGlobalProperties.GetIPGlobalProperties();
var tcpConnections = ipProperties.GetActiveTcpConnections();
...but I couldn't reproduce the error; and amount of connections stayed around the same the whole time. It makes me think something other than connection limit causes this error.
I'd try the same in Azure; but this does not seem to work as Kudu doesn't allow netstat or similar command.
The clients are added to services like this:
collection.AddHttpClient<ISenderClient, SenderClient>()
.ConfigureHttpClient((provider, client) =>
{
//... configuring base url and headers
})
.SetHandlerLifetime(TimeSpan.FromMinutes(1));
Then injected into SenderClient:
public sealed class SenderClient : ISenderClient, IDisposable
{
private readonly HttpClient _httpClient;
public SenderClient(HttpClient httpClient)
{
_httpClient = httpClient ?? throw new ArgumentNullException(nameof(httpClient));
}
public void Dispose()
{
_httpClient?.Dispose();
}
}
SenderClient itself is retrieved from service provider:
_client = _provider.GetService<ISenderClient>();
...and later used in Parallel.ForEach loop to distribute http requests:
Parallel.ForEach(batches, new ParallelOptions { MaxDegreeOfParallelism = 8 }, (batch, state, i) =>
{
using (var scope = _provider.CreateScope())
{
var provider = scope.ServiceProvider;
//... dto mapping
var response = _client.SendAsync<Request, Response>(request).GetAwaiter().GetResult();
}
}
where SendAsync is
public async Task<TResponse> SendAsync<TRequest, TResponse>(TRequest request)
{
var json = JsonConvert.SerializeObject(request);
using (var content = new StringContent(json, Encoding.UTF8, "application/json"))
using (var response = await _client.PostAsync(_url, content))
{
var responseString = await response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<TResponse>(responseString);
}
}
Could anybody please explain what could cause this exception, as I'm having hard times trying to fix this issue for a long time by now..?
EDIT: I believe that my question is not a duplicate of this post, because I'm using HttpClientFactory rather than handling clients manually. This should take care of managing, refreshing and disposing of connections; and is considered a preferred way of handling http clients.
I should only have one connection of a type at a certain period of time with this approach.

We transferred the job to app service with a superior pricing tier, and it kind of fixed the issue; though I'm not entirely satisfied with this solution.

Validate multiple endpoints using HTML status codes before failing when consuming a asmx web service C#

I am currently using a web service, which offers 2 endpoints, as backups for fall over. I need to test all 2 endpoints before my code completely fails and then will need to log the exception. My thoughts were to be to return the status code of the HTML response using this:
Function1:
public string ValidateHttpRequest(string endpointUrl)
{
try
{
var url = endpointUrl;
HttpClient httpClient = new HttpClient();
var reponse = httpClient.GetAsync(endpointUrl);
return reponse.Result.StatusCode.ToString();
}
catch (Exception ex)
{
Log.Log("exception thrown in ValidateHttpRequest()! " + ex.ToString());
Log.Log(ex);
return null;
}
}
This is called from another function, say function2().
Function 2:
private bool function2()
{
//Specify the binding to be used for the client.
BasicHttpsBinding binding = new BasicHttpsBinding();
var epA = "https://www1.endpoint1.com/endpointService.asmx";
var epB = "https://www2.endpoint1.com/endpointService.asmx";
if (ValidateHttpRequest(epA)== "OK")
{
EndpointAddress address = new EndpointAddress("https://www1.enpoint1.com/endpointService.asmx");
_Client = new WebService.SoapClient(binding, address);
return true;
}
else if ((ValidateHttpRequest(epB))== "OK")
{
EndpointAddress address2 = new EndpointAddress(("https://www2.enpoint2.com/endpointService.asmx"));
else
{
// Now Log error here completely, and only fail here if both above checks return anything apart from 200 status code
LogException(“Only log exception if all endpoints fail”);
return false;
}
}
This is all well and good, however I need this to not fail on the first call, as I will need to check if the other endpoint is valid/active. The issue is that if the response is null, the exception is handled and I will not check the rest of my endpoints, how can I correctly ensure my code is safe with i.e. exceptions are handled correctly, but continuing my code to check all endpoints before completely failing and halting execution. it should fail if i receive any other response apart from 200 OK I have researched about how to check the HTTP response and all that I can come up with is this but it doesn’t completely suit my needs .If anyone could point me in the right direction or help with a solution I would be very grateful.
Thanks in advance

Mvc Application Async Methods Are Hanging

We have SOA for our solution. We are using .net framework 4.5.1, asp.net mvc 4.6, sql server, windows server and thinktecture identity server 3 ( for token based webapi calls. )
Solution structure looks like;
Our mvc frontend application talks with our webapi application via a httpClient wrapper. Here is the generic http client wrapper code;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;
namespace Cheetah.HttpClientWrapper
{
public class ResourceServerRestClient : IResourceServerRestClient
{
private readonly ITokenProvider _tokenProvider;
public ResourceServerRestClient(ITokenProvider tokenProvider)
{
_tokenProvider = tokenProvider;
}
public string BaseAddress { get; set; }
public Task<T> GetAsync<T>(string uri, string clientId)
{
return CheckAndInvokeAsync(async token =>
{
using (var client = new HttpClient())
{
ConfigurateHttpClient(client, token, clientId);
HttpResponseMessage response = await client.GetAsync(uri);
if (response.IsSuccessStatusCode)
{
return await response.Content.ReadAsAsync<T>();
}
var exception = new Exception($"Resource server returned an error. StatusCode : {response.StatusCode}");
exception.Data.Add("StatusCode", response.StatusCode);
throw exception;
}
});
}
private void ConfigurateHttpClient(HttpClient client, string bearerToken, string resourceServiceClientName)
{
if (!string.IsNullOrEmpty(resourceServiceClientName))
{
client.DefaultRequestHeaders.Add("CN", resourceServiceClientName);
}
if (string.IsNullOrEmpty(BaseAddress))
{
throw new Exception("BaseAddress is required!");
}
client.BaseAddress = new Uri(BaseAddress);
client.Timeout = new TimeSpan(0, 0, 0, 10);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", bearerToken);
}
private async Task<T> CheckAndInvokeAsync<T>(Func<string, Task<T>> method)
{
try
{
string token = await _tokenProvider.IsTokenNullOrExpired();
if (!string.IsNullOrEmpty(token))
{
return await method(token);
}
var exception = new Exception();
exception.Data.Add("StatusCode", HttpStatusCode.Unauthorized);
throw exception;
}
catch (Exception ex)
{
if (ex.Data.Contains("StatusCode") && ((HttpStatusCode)ex.Data["StatusCode"]) == HttpStatusCode.Unauthorized)
{
string token = await _tokenProvider.GetTokenAsync();
if (!string.IsNullOrEmpty(token))
{
return await method(token);
}
}
throw;
}
}
public void ThrowResourceServerException(List<string> messages)
{
string message = messages.Aggregate((p, q) => q + " - " + p);
var exception = new Exception(message);
exception.Data.Add("ServiceOperationException", message);
throw exception;
}
}
}
Also, sometimes this http client wrapper using with NitoAsync manager ( Call async methods as sync. ), and sometimes we are using this generic method directly with await - async task wait like;
var result = await _resourceServerRestClient.GetAsync<ServiceOperation<DailyAgendaModel>>("dailyAgenda/" + id);
So here is our problem:
When we test our mvc application with jmeter (for making some-kind-of load test / 10 threads per 1 sec), after a couple of minutes, mvc application stops working [ exception is task canceled due to timeout ] ( maybe only 1-2 requests timeouts ) on this line: HttpResponseMessage response = await client.GetAsync(uri);. But after that request, all requests will be failed like they are in row. So mvc application is hanging for 2-15 minutes ( randomly ) but in that time I can send new requests from postman to webapi. They are ok, I mean webapi is responding well. After a couple of minutes mvc application turnback to normal.
Note: We have load-balancer for mvc-ui and webapi. Because sometimes we get 120K requests in a minute in a busy day. But it gives same error if there is no load balancer in front of webapi or mvc application. So it's not LB problem.
Note2: We tried to use RestSharp for mvc-ui and webapi communication. We got same error here. When a reuqest is failing, all requests will be failed in a row. It looks like it's a network error but we can't find a proof for it.
Can you see any error on my httpClient wrapper ? or better question is;
In your solution, how is your mvc application communicating with your webapi application ? What are the best practices here ?
Update1: We moved projects .net 4.5.1 to 4.6.1. Same deadlock happened again. And than we temporary moved all source codes of the layer: "Business & Repository" as dll level. There is no webapi between business & presentation level now. Dead lock solved. We are still searching why httpClientWrapper codes are not working properly when we called webapi methods from our webapplication controllers.

better question is;
In your solution, how is your mvc application communicating with your webapi application ? What are the best practices here ?
A best practice here is for the client (browser in your case) to directly retrieve data from the Web API Controllers and for the MVC controllers to only serve pure HTML views which include layout, styles (css), visual structure, scripts (ie. javascript) etc and not the data.
Image credit: Ode to Code. Incidentally the author on that site also does not recommend your approach although it is listed as an option.
This servers as a good SOC between your views and your data allowing you more easily to make changes to either part.
It allows for the client (browser) to retrieve data asynchronously which creates for a better user experience.
By not doing this and adding a network request step in the call stack you have created an unnecessary expensive step in the flow of data (call from MVC Controller(s) to Web API deployment). The more boundaries are crossed during executing the slower the execution.
The fast solution, as you have already figured out, is to call your business code library directly from your MVC project. This will avoid the additional and unnecessary network step. There is nothing wrong with doing this and many more traditional sites serve both the view (html) and data in the same call. It makes for a more tightly coupled design but it is better than what you had.
The best long term solution is to change the MVC views so they call your Web API deployment directly. This can be done using frameworks like Angular, React, Backbone, etc. If the Web API method calls are limited and are not expected to grow you can also use JQuery or pure javascript BUT I would not try to build a complex application on this, there is a reason why frameworks like Angular have become so popular.
As to the actual underlying technical problem in this case we can't be sure without a memory dump to see what resources are causing the deadlock. It might be something as simple as making sure your MVC Action Methods are also returning async Task<ActionResult> (instead of just ActionResult which, I am guessing, is how you have them structured now) so they can call the HttpClient using an actual async/await pattern. Honestly, because its a bad design, I would not spend any time into trying to get this to work.

I'm not exactly sure whu, but I'll start by refactoring the GetAsync() method
public async Task<T> GetAsync<T>(string uri, string clientId)
{
try
{
string token = await _tokenProvider.IsTokenNullOrExpired();
if (!string.IsNullOrEmpty(token))
{
using (var client = new HttpClient())
{
ConfigurateHttpClient(client, token, clientId);
HttpResponseMessage response = await client.GetAsync(uri);
if (response.IsSuccessStatusCode)
{
return await response.Content.ReadAsAsync<T>();
}
var exception = new Exception($"Resource server returned an error. StatusCode : {response.StatusCode}");
exception.Data.Add("StatusCode", response.StatusCode);
throw exception;
}
}
else
{
var exception = new Exception();
exception.Data.Add("StatusCode", HttpStatusCode.Unauthorized);
throw exception;
}
}
catch (Exception ex)
{
throw;
}
}

You should put .ConfigureAwait(false) to your inner awaits statements:
HttpResponseMessage response = await client.GetAsync(uri).ConfigureAwait(false);
(...)
return await response.Content.ReadAsAsync<T>().ConfigureAwait(false);
(...)
string token = await _tokenProvider.IsTokenNullOrExpired().ConfigureAwait(false);
(...)
return await method(token).ConfigureAwait(false);;
(...)
string token = await _tokenProvider.GetTokenAsync().ConfigureAwait(false);;
(...)
return await method(token).ConfigureAwait(false);
This way you will avoid to capture the synchronization context before the await is done. Otherwise the continuation will be done in this context, which might result in a lock if this one is in use by other threads.
Doing so will allow the continuation to be done whithin the context of the task which is awaited.