We are using masstransit with Request/Response pattern as described here: http://masstransit-project.com/MassTransit/usage/request-response.html
For some reason when the bus is initialized with localhost I receive a MassTransit.EndpointNotFoundException on the respond side, using context.RespondAsync method.
When the bus is initialized with the actual IP it works good without any exception.
We are using the bus also with other patterns of masstransit initialized with localhost without any problems.
We tried using both, Request and GetResponse patterns on the request side thinking maybe the problem source is there, but with no success.
Following is the consumer code the Response side, which generate the exception on the RespondAsync:
public override async Task Consume(ConsumeContext<IImageRequest> context)
{
var msg = context.Message;
try
{
var image = GetImage(msg.Id);
await context.RespondAsync<IImageResult>(new
{
msg.Id,
AImage = image
});
}
catch (Exception e)
{
Logger.Instance.Error($"{e}");
}
}
Following is the request side:
Task.Run(async () =>
{
var reqResClient =
BusControl.GetBusControl.CreateRequestClient<IImageRequest, IImageResult>("requestimage");
var req = new ImageRequest(msg.Id);
var response = await reqResClient.Request(req);
}).Wait();
Bus initialization code:
_bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Durable = true;
_host = cfg.Host(HostName, port, VirtualHost, h =>
{
h.Username(UserName);
h.Password(Password);
});
cfg.ReceiveEndpoint(host, subscriber.QueueName, ep =>
{
ep.Durable = !subscriber.AutoDelete;
ep.AutoDelete = subscriber.AutoDelete;
ep.Instance(subscriber.Consumer);
if (subscriber.PrefetchCount.HasValue)
ep.PrefetchCount = subscriber.PrefetchCount.Value;
if (subscriber.ConcurrencyLimit.HasValue)
ep.UseConcurrencyLimit(subscriber.ConcurrencyLimit.Value);
});
cfg.Message<T>(x =>
{
x.SetEntityName(entityName);
});
});
public void Start()
{
if (_state == BusState.NotInitialized)
throw new InvalidOperationException("Cannot start bus. Bus is not initialized");
_bus?.Start();
_state = BusState.Started;
}
I wanted to know if there's a solution for it? or is it a known problem?
Thanks,
Related
I'm trying to use Polly as retry policy handler for grpc in my .net core 6 project. I noticed that the retryFunc is never invoked. I started from this project gRPC & ASP.NET Core 3.1: Resiliency with Polly
class Program
{
static async Task Main(string[] args)
{
// DI
var services = new ServiceCollection();
var loggerFactory = LoggerFactory.Create(logging =>
{
logging.AddConsole();
logging.SetMinimumLevel(LogLevel.Debug);
});
var serverErrors = new HttpStatusCode[] {
HttpStatusCode.BadGateway,
HttpStatusCode.GatewayTimeout,
HttpStatusCode.ServiceUnavailable,
HttpStatusCode.InternalServerError,
HttpStatusCode.TooManyRequests,
HttpStatusCode.RequestTimeout
};
var gRpcErrors = new StatusCode[] {
StatusCode.DeadlineExceeded,
StatusCode.Internal,
StatusCode.NotFound,
StatusCode.ResourceExhausted,
StatusCode.Unavailable,
StatusCode.Unknown
};
Func<HttpRequestMessage, IAsyncPolicy<HttpResponseMessage>> retryFunc = (request) =>
{
return Policy.HandleResult<HttpResponseMessage>(r => {
var grpcStatus = StatusManager.GetStatusCode(r);
var httpStatusCode = r.StatusCode;
return (grpcStatus == null && serverErrors.Contains(httpStatusCode)) || // if the server send an error before gRPC pipeline
(httpStatusCode == HttpStatusCode.OK && gRpcErrors.Contains(grpcStatus.Value)); // if gRPC pipeline handled the request (gRPC always answers OK)
})
.WaitAndRetryAsync(3, (input) => TimeSpan.FromSeconds(3 + input), (result, timeSpan, retryCount, context) =>
{
var grpcStatus = StatusManager.GetStatusCode(result.Result);
Console.WriteLine($"Request failed with {grpcStatus}. Retry");
});
};
services.AddGrpcClient<CountryServiceClient>(o =>
{
o.Address = new Uri("https://localhost:5001");
}).AddPolicyHandler(retryFunc);
var provider = services.BuildServiceProvider();
var client = provider.GetRequiredService<CountryServiceClient>();
try
{
var countries = (await client.GetAllAsync(new EmptyRequest())).Countries.Select(x => new Country
{
CountryId = x.Id,
Description = x.Description,
CountryName = x.Name
}).ToList();
Console.WriteLine("Found countries");
countries.ForEach(x => Console.WriteLine($"Found country {x.CountryName} ({x.CountryId}) {x.Description}"));
}
catch (RpcException e)
{
Console.WriteLine(e.Message);
}
}
}
but at the end WaitAndRetryAsync is never called.
I created a small project available on github in order to reproduce it.
My test is fairly simple. I start the client without a listening back-end, expecting to read 3 times the output from Console.WriteLine($"Request failed with {grpcStatus}. Retry"); on the console. But the policy handler in never fired. I have the following exception instead
Status(StatusCode="Unavailable", Detail="Error connecting to
subchannel.", DebugException="System.Net.Sockets.SocketException
(10061): No connection could be made because the target machine
actively refused it.
without any retry.
This is not working for you because Retry is now built into Grpc. In order to make this work, register your service as follows:
var defaultMethodConfig = new MethodConfig
{
Names = { MethodName.Default },
RetryPolicy = new RetryPolicy
{
MaxAttempts = 3,
InitialBackoff = TimeSpan.FromSeconds(3),
MaxBackoff = TimeSpan.FromSeconds(3),
BackoffMultiplier = 1,
RetryableStatusCodes =
{
// Whatever status codes you want to look for
StatusCode.Unauthenticated, StatusCode.NotFound, StatusCode.Unavailable,
}
}
};
var services = new ServiceCollection();
services.AddGrpcClient<TestServiceClient>(o => {
o.Address = new Uri("https://localhost:5001");
o.ChannelOptionsActions.Add(options =>
{
options.ServiceConfig = new ServiceConfig {MethodConfigs = {defaultMethodConfig}};
});
});
That will add the retry policy to your client. One other thing that you might run into. I didn't realize this at the time, but in my service implementation, I was setting up errors something like this:
var response = new MyServiceResponse()
// something bad happens
context.Status = new Status(StatusCode.Internal, "Something went wrong");
return response;
The retry logic will not kick in if you implement your service like that, you actually have to do something more like this:
// something bad happens
throw new RpcException(new Status(StatusCode.Internal, "Something went wrong"));
The retry logic you configured when registering your client will then work. Hope that helps.
With the help of #PeterCsala I tried some fix.
As a first attempt I tried without DependencyInjection, registering the policy as follows
var policy = Policy
.Handle<Exception>()
.RetryAsync(3, (exception, count) =>
{
Console.WriteLine($"Request {count}, {exception.Message}. Retry");
});
var channel = GrpcChannel.ForAddress("https://localhost:5001");
TestServiceClient client = new TestServiceClient(channel);
await policy.ExecuteAsync(async () => await client.TestAsync(new Empty()));
This way it's working.
Then I came back to DI and used to register the policy as follows
IAsyncPolicy<HttpResponseMessage> policy =
Policy<HttpResponseMessage>.Handle<Exception>().RetryAsync(3, (exception, count) =>
{
Console.WriteLine($"Request {count}, {exception.Exception.Message}. Retry");
});
var services = new ServiceCollection();
services.AddGrpcClient<TestServiceClient>(o => {
o.Address = new Uri("https://localhost:5001");
}).AddPolicyHandler(policy);
var provider = services.BuildServiceProvider();
var client = provider.GetRequiredService<TestServiceClient>();
var testClient = (await client.TestAsync(new Empty()));
And still not working.
At the end it seems AddPolicyHandler is not suitable for grpc clients?
I am writing sample applications in .Net core to interact with Kafka.
I have downloaded Kafka and Zookeeper official docker images to my machine.
I am using Confluent.Kafka nuget package for both producer and consumer. I am able to produce the message to Kafka. But my consumer part is not working.
Below is my producer and consumer code snippet. I am not sure what mistake I'm doing here.
Do we need to Explicitly create a consumer group?
Consumer Code (This code not working. Thread waiting at consumer.Consume(cToken);)
var config = new ConsumerConfig
{
BootstrapServers = "localhost:9092",
GroupId = "myroupidd",
AutoOffsetReset = AutoOffsetReset.Earliest,
};
var cToken = ctokenSource.Token;
var consumerBuilder = new ConsumerBuilder<Null, string>(config);
consumerBuilder.SetPartitionsAssignedHandler((consumer, partitionlist) =>
{
consumer.Assign(new TopicPartition("myactual-toppics", 0) { });
Console.WriteLine("inside SetPartitionsAssignedHandler action");
});
using var consumer = consumerBuilder.Build();
consumer.Subscribe("myactual-toppics");
while (!cToken.IsCancellationRequested)
{
var consumeResult = consumer.Consume(cToken);
if (consumeResult.Message != null)
Console.WriteLine(consumeResult.Message.Value);
}
Producer Code (This is working fine. I am able to see the messages using Conduckto Tool)
var config = new ProducerConfig
{
BootstrapServers = "localhost:9092",
ClientId = Dns.GetHostName(),
};
using (var producer = new ProducerBuilder<Null, string>(config).Build())
{
while (!stoppingToken.IsCancellationRequested)
{
var top = new TopicPartition("myactual-toppics", 0);
var result = await producer.ProduceAsync(top, new Message<Null, string> { Value = "My First Message" });
Console.WriteLine($"Publishedss1234" );
await Task.Delay(5000, stoppingToken);
I struggle with understanding how does LSP client-side works. I mean I think I understand the theory of communication (JSON-RPC/LSP Protocol basics) but I struggle with existing libraries that are used for this for VS Code and I think trying to rewrite it is kinda pointless, especially client-side where I do not feel proficient at all
All examples I see provide a path to the server, so the LSP client can start it
it makes sense, but I'd rather avoid it during development, I'd want to have the server aopen in debugging mode and just start VS Code
I tried to start with basic of basic server implementation (C#)
public class Server
{
private JsonRpc RPC { get; set; }
public async Task Start()
{
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.WriteTo.Console()
.CreateLogger();
var pipeName = "LSP_Pipe";
var writerPipe = new NamedPipeClientStream(pipeName);
var readerPipe = new NamedPipeClientStream(pipeName);
await writerPipe.ConnectAsync(10_000);
await readerPipe.ConnectAsync(10_000);
Log.Information("RPC Listening");
RPC = new JsonRpc(writerPipe, readerPipe, this);
RPC.StartListening();
this.RPC.Disconnected += RPC_Disconnected;
await Task.Delay(-1);
}
private void RPC_Disconnected(object sender, JsonRpcDisconnectedEventArgs e)
{
Log.Information("Disconnected");
}
[JsonRpcMethod(RPCMethods.InitializeName)]
public object Initialize(JToken arg)
{
Log.Information("Initialization");
var serializer = new JsonSerializer()
{
ContractResolver = new ResourceOperationKindContractResolver()
};
var param = arg.ToObject<InitializeParams>();
var clientCapabilities = param?.Capabilities;
var capabilities = new ServerCapabilities
{
TextDocumentSync = new TextDocumentSyncOptions(),
CompletionProvider = new CompletionOptions(),
SignatureHelpProvider = new SignatureHelpOptions(),
ExecuteCommandProvider = new ExecuteCommandOptions(),
DocumentRangeFormattingProvider = false,
};
capabilities.TextDocumentSync.Change = TextDocumentSyncKind.Incremental;
capabilities.TextDocumentSync.OpenClose = true;
capabilities.TextDocumentSync.Save = new SaveOptions { IncludeText = true };
capabilities.CodeActionProvider = clientCapabilities?.Workspace?.ApplyEdit ?? true;
capabilities.DefinitionProvider = true;
capabilities.ReferencesProvider = true;
capabilities.DocumentSymbolProvider = true;
capabilities.WorkspaceSymbolProvider = false;
capabilities.RenameProvider = true;
capabilities.HoverProvider = true;
capabilities.DocumentHighlightProvider = true;
return new InitializeResult { Capabilities = capabilities };
}
}
but I'm unable to setup client with those vscode-languageclient/node libraries even to get Log.Information("Initialization"); part
How can I provide the way they communicate - e.g name of named pipe? or just HTTP posts?
I'm not proficent in js / node development at all, so sorry for every foolish question
I've seen mature/production grade C# Language Server implementations but I'm overwhelmed just by their builders, there's sooo much stuff happening, sop that's why I'd want to write server from scratch, but for client use existing libs
var server = await LanguageServer.From(
options =>
options
.WithInput(Console.OpenStandardInput())
.WithOutput(Console.OpenStandardOutput())
.ConfigureLogging(
x => x
.AddSerilog(Log.Logger)
.AddLanguageProtocolLogging()
.SetMinimumLevel(LogLevel.Debug)
)
.WithHandler<TextDocumentHandler>()
.WithHandler<DidChangeWatchedFilesHandler>()
.WithHandler<FoldingRangeHandler>()
.WithHandler<MyWorkspaceSymbolsHandler>()
.WithHandler<MyDocumentSymbolHandler>()
.WithHandler<SemanticTokensHandler>()
.WithServices(x => x.AddLogging(b => b.SetMinimumLevel(LogLevel.Trace)))
.WithServices(
services => {
services.AddSingleton(
provider => {
var loggerFactory = provider.GetService<ILoggerFactory>();
var logger = loggerFactory.CreateLogger<Foo>();
logger.LogInformation("Configuring");
return new Foo(logger);
}
);
services.AddSingleton(
new ConfigurationItem {
Section = "typescript",
}
).AddSingleton(
new ConfigurationItem {
Section = "terminal",
}
);
}
)
.OnInitialize(
async (server, request, token) => {
var manager = server.WorkDoneManager.For(
request, new WorkDoneProgressBegin {
Title = "Server is starting...",
Percentage = 10,
}
);
workDone = manager;
await Task.Delay(2000);
manager.OnNext(
new WorkDoneProgressReport {
Percentage = 20,
Message = "loading in progress"
}
);
}
)
.OnInitialized(
async (server, request, response, token) => {
workDone.OnNext(
new WorkDoneProgressReport {
Percentage = 40,
Message = "loading almost done",
}
);
await Task.Delay(2000);
workDone.OnNext(
new WorkDoneProgressReport {
Message = "loading done",
Percentage = 100,
}
);
workDone.OnCompleted();
}
)
.OnStarted(
async (languageServer, token) => {
using var manager = await languageServer.WorkDoneManager.Create(new WorkDoneProgressBegin { Title = "Doing some work..." });
manager.OnNext(new WorkDoneProgressReport { Message = "doing things..." });
await Task.Delay(10000);
manager.OnNext(new WorkDoneProgressReport { Message = "doing things... 1234" });
await Task.Delay(10000);
manager.OnNext(new WorkDoneProgressReport { Message = "doing things... 56789" });
var logger = languageServer.Services.GetService<ILogger<Foo>>();
var configuration = await languageServer.Configuration.GetConfiguration(
new ConfigurationItem {
Section = "typescript",
}, new ConfigurationItem {
Section = "terminal",
}
);
var baseConfig = new JObject();
foreach (var config in languageServer.Configuration.AsEnumerable())
{
baseConfig.Add(config.Key, config.Value);
}
logger.LogInformation("Base Config: {Config}", baseConfig);
var scopedConfig = new JObject();
foreach (var config in configuration.AsEnumerable())
{
scopedConfig.Add(config.Key, config.Value);
}
logger.LogInformation("Scoped Config: {Config}", scopedConfig);
}
)
);
Thanks in advance
I'm trying to create an API that consumes various topics.
For this, I'm trying to multi-thread things, so that the whole thing can be scalable into multiple APIs, later on, but that's very besides the point.
I'm using ASP.net Core 4.0, if that's got anything to do with it. Entity Framework as well.
My problem is based on my connection to my Mosquitto server being broken without throwing an exception or anything of the like, after a minute or so. It doesn't matter how big the messages are, or how many are exchanged. I have no idea of how I can create a callback or anything of the kind to know what's going on with my connection. Can anyone help?
I'll link the code I use to establish a connection and subscribe to a connection below. Using the Subscribe method or doing it manually also changes nothing. I'm at a loss, here.
Thanks in advance!
Main.cs:
Task.Factory.StartNew(() => DataflowController.ResumeQueuesAsync());
BuildWebHost(args).Run();
DataflowController.cs:
public static Boolean Subscribe(String topic)
{
Console.WriteLine("Hello from " + topic);
MqttClient mqttClient = new MqttClient(brokerAddress);
byte code = mqttClient.Connect(Guid.NewGuid().ToString());
// Register to message received
mqttClient.MqttMsgPublishReceived += client_recievedMessageAsync;
string clientId = Guid.NewGuid().ToString();
mqttClient.Connect(clientId);
// Subscribe to topic
mqttClient.Subscribe(new String[] { topic }, new byte[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
System.Console.ReadLine();
return true;
}
public static async Task ResumeQueuesAsync()
{
var mongoClient = new MongoClient(connectionString);
var db = mongoClient.GetDatabase(databaseName);
var topics = db.GetCollection<BsonDocument>(topicCollection);
var filter = new BsonDocument();
List<BsonDocument> result = topics.Find(filter).ToList();
var resultSize = result.Count;
Task[] subscriptions = new Task[resultSize];
MqttClient mqttClient = new MqttClient(brokerAddress);
byte code = mqttClient.Connect(Guid.NewGuid().ToString());
// Register to message received
mqttClient.MqttMsgPublishReceived += client_recievedMessageAsync;
string clientId = Guid.NewGuid().ToString();
mqttClient.Connect(clientId);
int counter = 0;
foreach(var doc in result)
{
subscriptions[counter] = new Task(() =>
{
Console.WriteLine("Hello from " + doc["topic"].ToString());
// Subscribe to topic
mqttClient.Subscribe(new String[] { doc["topic"].ToString() }, new byte[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
System.Console.ReadLine();
});
counter++;
}
foreach(Task task in subscriptions)
{
task.Start();
}
}
static async void client_recievedMessageAsync(object sender, MqttMsgPublishEventArgs e)
{
// Handle message received
var message = System.Text.Encoding.Default.GetString(e.Message);
var topic = e.Topic;
var id = topic.Split("/")[2];
BsonDocument doc = new BsonDocument {
{"Plug ID", id },
{"Consumption", message }
};
await Save(doc, "smartPDM_consumption");
System.Console.WriteLine("Message received from " + topic + " : " + message);
}
This line was the issue:
byte code = mqttClient.Connect(Guid.NewGuid().ToString());
Deleted it, and it just worked.
I want to retrieve ticks from Poloniex in real time. They use wamp for that. I installed via nugget WampSharp and found this code :
static async void MainAsync(string[] args)
{
var channelFactory = new DefaultWampChannelFactory();
var channel = channelFactory.CreateMsgpackChannel("wss://api.poloniex.com", "realm1");
await channel.Open();
var realmProxy = channel.RealmProxy;
Console.WriteLine("Connection established");
int received = 0;
IDisposable subscription = null;
subscription =
realmProxy.Services.GetSubject("ticker")
.Subscribe(x =>
{
Console.WriteLine("Got Event: " + x);
received++;
if (received > 5)
{
Console.WriteLine("Closing ..");
subscription.Dispose();
}
});
Console.ReadLine();
}
but no matter at the await channel.open() I have the following error : HHTP 502 bad gateway
Do you have an idea where is the problem
thank you in advance
The Poloniex service seems not to be able to handle so many connections. That's why you get the HTTP 502 bad gateway error. You can try to use the reconnector mechanism in order to try connecting periodically.
static void Main(string[] args)
{
var channelFactory = new DefaultWampChannelFactory();
var channel = channelFactory.CreateJsonChannel("wss://api.poloniex.com", "realm1");
Func<Task> connect = async () =>
{
await Task.Delay(30000);
await channel.Open();
var tickerSubject = channel.RealmProxy.Services.GetSubject("ticker");
var subscription = tickerSubject.Subscribe(evt =>
{
var currencyPair = evt.Arguments[0].Deserialize<string>();
var last = evt.Arguments[1].Deserialize<decimal>();
Console.WriteLine($"Currencypair: {currencyPair}, Last: {last}");
},
ex => {
Console.WriteLine($"Oh no! {ex}");
});
};
WampChannelReconnector reconnector =
new WampChannelReconnector(channel, connect);
reconnector.Start();
Console.WriteLine("Press a key to exit");
Console.ReadKey();
}
This is based on this code sample.
Get rid of the Console.WriteLine it is interfering with your code.