We Keep Coding

Azure function: Entity Framework Core: A second operation started on this context before a previous operation completed

In my azure functions I am using EfCore and I am getting this error
Entity Framework Core: A second operation started on this context before a previous operation completed
My code in program.cs is as follows
public static void Main()
{
System.Text.Encoding.RegisterProvider(System.Text.CodePagesEncodingProvider.Instance);
var host = new HostBuilder()
.ConfigureFunctionsWorkerDefaults()
.ConfigureServices(s =>
{
s.AddDbContext<HumanRisksDbContext>(o =>
o.UseSqlServer(Environment.GetEnvironmentVariable("ConnectionStrings:HumanRisksDbContext")),
ServiceLifetime.Transient);
s.AddSingleton<IAuditApiService, AuditApiService>();
})
.Build();
host.Run();
}
Now I have three azure functions which can overlap when running.
Each of these functions access the same service class which is like this
public class AuditApiService : IAuditApiService
{
private readonly IConfiguration _configuration;
private readonly ILogHandler _logHandler;
private readonly HumanRisksDbContext context;
public AuditApiService(IConfiguration configuration,
ILogHandler logHandler,
HumanRisksDbContext context)
{
_configuration = configuration;
_logHandler = logHandler;
this.context = context;
}
}
This class has three public functions which uses the context.
Here is function one
public async Task Process(string appServer)
{
// Go through active controls
foreach (var control in ActiveControls(context).Where(c => c.TaskType.Equals(Controls.TASK_TYPE.Interval)))
{
try
{
// some processing
AuditRepository auditRepository = new AuditRepository(context);
await auditRepository.Create(upcomming, _logHandler);
await context.SaveChangesAsync();
}
catch (Exception err)
{
}
}
}
function two is like this
public async Task ProcessLate(string appServer)
{
// Go through all audits older than now that is not completed
foreach (var audit in NotCompletedAuditsBeforeToday(context))
{
try
{
// processing
await context.SaveChangesAsync();
}
catch (Exception err)
{
LogHandlerService.Log("Error process late audits id=" + (audit?.Id) + ": " + err.Message);
}
}
}
and last function is like this
public void ProcessReport(string appServer)
{
UserRepository userrep = new UserRepository(context);
var administratorRole = context.Roles.Where(w => w.Name.Equals(Roles.Administrator.ToString())).SingleOrDefault();
var managerRole = context.Roles.Where(w => w.Name.Equals(Roles.Manager.ToString())).SingleOrDefault();
var employeeRole = context.Roles.Where(w => w.Name.Equals(Roles.Employee.ToString())).SingleOrDefault();
//todo: should we have is null check here on administratorRole?
// Find users in either an administrator or manager role
var administrators = userrep.ListByRole(context, administratorRole.Id);
//var administrators = context.Users.Include(w => w.ro.Where(u => u.Roles.Select(r => r.RoleId).Contains(administratorRole.Id)).ToList();
var managers = userrep.ListByRole(context, managerRole.Id);
//context.Users.Where(u => u.Roles.Select(r => r.RoleId).Contains(managerRole.Id)).ToList();
administrators.AddRange(managers);
// processing
var employees = userrep.ListByRole(context, employeeRole.Id);
// processing
}
Am I missing something? If these functions are called at the same time would it be an issue?

How to use added scheduler using DI in Quartz.net

I have added scheduler in Startup of service like:
services.AddQuartz(q =>
{
q.SchedulerId = "S1";
q.SchedulerName = "S1";
q.UseMicrosoftDependencyInjectionJobFactory();
q.UsePersistentStore(s =>
{
s.UseProperties = true;
s.UsePostgres("ConnectionString");
s.UseJsonSerializer();
});
})
Now I am tring to use this created Scheduler via DI like:
public SchedulerStartup(ISchedulerFactory schedulerFactory)
{
this.schedulerFactory = schedulerFactory;
}
public async Task StartAsync(CancellationToken cancellationToken)
{
Scheduler = await schedulerFactory.GetScheduler("S1", cancellationToken);
await Scheduler.Start(cancellationToken);
}
But somehow Scheduler is null. I wont able to access created Scheduler in startup configuration (S1).
Link: https://www.quartz-scheduler.net/documentation/quartz-3.x/packages/microsoft-di-integration.html#di-aware-job-factories

Here I have missed services.AddQuartzHostedService() to start scheduler using Hosting Service. Additional startup class not required.
This should be like this:
services.AddQuartz(q =>
{
q.SchedulerName = "S1";
q.UseMicrosoftDependencyInjectionJobFactory();
q.UsePersistentStore(s =>
{
s.UseProperties = true;
s.UsePostgres(DbConnectionString);
s.UseJsonSerializer();
});
});
services.AddQuartzHostedService(options =>
{
options.WaitForJobsToComplete = true;
});
Later created instance of this Scheduler can be used as (S1):
public MyRuntimeScheduler(ISchedulerFactory schedulerFactory)
{
Scheduler = schedulerFactory.GetScheduler("S1").GetAwaiter().GetResult();
}

gRPC DNS resolution failed when using machine name

I am new to gRPC and trying to learn it by using the chat server/client sample from cactuaroid here. I’ve modified the code to show progress in a WPF app from a long running task. All code is running on .NET 5 and I’m using the latest versions of the gRPC packages.
The process is working fine when using the computer's IP address but when using computer name for the gRPC client, I’m getting a “DNS resolution failed” exception (computer name is “skylake”):
RpcException: Status(StatusCode="Unavailable", Detail="DNS resolution
failed for service: skylake:6001",
DebugException="Grpc.Core.Internal.CoreErrorDetailException:
{"created":"#1615312867.300000000","description":"Resolver transient
failure","file":"......\src\core\ext\filters\client_channel\client_channel.cc","file_line":2138,"referenced_errors":[{"created":"#1615312867.300000000","description":"DNS
resolution failed for service:
skylake:6001","file":"......\src\core\ext\filters\client_channel\resolver\dns\c_ares\dns_resolver_ares.cc","file_line":362,"grpc_status":14,"referenced_errors":[{"created":"#1615312867.300000000","description":"C-ares
status is not ARES_SUCCESS qtype=AAAA name=skylake is_balancer=0:
Could not contact DNS
servers","file":"......\src\core\ext\filters\client_channel\resolver\dns\c_ares\grpc_ares_wrapper.cc","file_line":716,"referenced_errors":[{"created":"#1615312866.142000000","description":"C-ares
status is not ARES_SUCCESS qtype=A name=skylake is_balancer=0: Could
not contact DNS
servers","file":"......\src\core\ext\filters\client_channel\resolver\dns\c_ares\grpc_ares_wrapper.cc","file_line":716}]}]}]}")
I verified that I could reach the port with telnet skylake 6001.
I am testing locally, client and server both on the same machine. Oddly enough, the gRPC server seems to be just fine with the computer name. Its just the client that has an issue with it.
Server code:
[Export(typeof(IService))]
public class ProgressServiceGrpcServer : Progress.ProgressBase, IService
{
[Import]
private Logger m_logger = null;
[Import]
private ProgressService m_progressService = null;
private readonly Empty m_empty = new Empty();
private const int Port = 6001;
private readonly Grpc.Core.Server m_server;
public ProgressServiceGrpcServer()
{
m_server = new Grpc.Core.Server
{
Services =
{
Progress.BindService(this)
.Intercept(new IpAddressAuthenticator())
},
Ports =
{
new ServerPort("skylake", Port, ServerCredentials.Insecure)
}
};
}
public void Start()
{
m_server.Start();
m_logger.Info("Started.");
}
public override async Task Subscribe(ChannelName channelName, IServerStreamWriter<ProgressReport> responseStream, ServerCallContext context)
{
context.CancellationToken.Register(() => m_logger.Info($"{context.Host} cancels subscription."));
try
{
await m_progressService.GetProgressReportsAsObservable(channelName)
.ToAsyncEnumerable()
.ForEachAwaitAsync(async (x) => await responseStream.WriteAsync(x), context.CancellationToken)
.ConfigureAwait(false);
}
catch (TaskCanceledException)
{
m_logger.Info($"{context.Host} unsubscribed.");
}
}
public override Task<Empty> Write(ProgressReport request, ServerCallContext context)
{
m_logger.Info($"{context.Host} {request}");
m_progressService.Add(request);
return Task.FromResult(m_empty);
}
}
Client code:
public class ProgressServiceClient
{
private readonly Progress.ProgressClient m_client =
new Progress.ProgressClient(
new Channel("skylake”, 6001, ChannelCredentials.Insecure));
public async Task Write(ProgressReport progressReport)
{
await m_client.WriteAsync(progressReport);
}
public IAsyncEnumerable<ProgressReport> ProgressReports(ChannelName channelName)
{
var call = m_client.Subscribe(channelName);
return call.ResponseStream
.ToAsyncEnumerable()
.Finally(() => call.Dispose());
}
}
Progress write method:
while (inProgress)
{
progressServiceClient.Write(new GrpcServer.ProgressReport
{
Id = Task.Id.ToString(),
PercentDone = percentDone,
TimeRemain = timeRemain
}).Wait();
Thread.Sleep(500);
}
Progress read method:
m_progressService = new ProgressServiceClient();
ChannelName channelName = new ChannelName() { Id = id };
var cts = new CancellationTokenSource();
_ = m_progressService.ProgressReports(channelName)
.ForEachAsync((x) =>
{
Log.Debug($"id: {x.Id} progress: {x.PercentDone}");
}, cts.Token);
this.Dispatcher.Invoke(() =>
{
Application.Current.Exit += (_, __) => cts.Cancel();
this.Unloaded += (_, __) => cts.Cancel();
});

Thanks to #jdweng for pointing me in the right direction, this was solved by adding the DNS suffix to the hostname (skylake.lan in my case).
We can get the DNS suffix via IPInterfaceProperties.DnsSuffix.
Alternatively, (which might be safer) we can use the correct IP address instead of the host name by using something like that.

How to specify which Azure Service Bus Topic to use with MassTransit

I've tried using MassTransit to publish a message to a topic named events in an Azure Service Bus. I have problems configuring MassTransit to use my predefined topic events, instead it creates a new topic named by the namespace/classname for the message type. So I wonder how to specify which topic to use instead of creating a new one.
This is the code I've tested with:
using System;
using System.Threading.Tasks;
using MassTransit;
using MassTransit.AzureServiceBusTransport;
using Microsoft.ServiceBus;
namespace PublisherNameSpace
{
public class Publisher
{
public static async Task PublishMessage()
{
var topic = "events";
var bus = Bus.Factory.CreateUsingAzureServiceBus(
cfg =>
{
var azureServiceBusHost = cfg.Host(new Uri("sb://<busname>.servicebus.windows.net"), host =>
{
host.OperationTimeout = TimeSpan.FromSeconds(5);
host.TokenProvider =
TokenProvider.CreateSharedAccessSignatureTokenProvider(
"RootManageSharedAccessKey",
"<key>"
);
});
cfg.ReceiveEndpoint(azureServiceBusHost, topic, e =>
{
e.Consumer<TestConsumer>();
});
});
await bus.Publish<TestConsumer>(new TestMessage { TestString = "testing" });
}
}
public class TestConsumer : IConsumer<TestMessage>
{
public Task Consume(ConsumeContext<TestMessage> context)
{
return Console.Out.WriteAsync("Consuming message");
}
}
public class TestMessage
{
public string TestString { get; set; }
}
}

The accepted answer clears up the subscription side:
cfg.SubscriptionEndpoint(
host,
"sub-1",
"my-topic-1",
e =>
{
e.ConfigureConsumer<TestConsumer>(provider);
});
For those wondering how to get the bus configuration right on the publish side, it should look like:
cfg.Message<TestMessage>(x =>
{
x.SetEntityName("my-topic-1");
});
You can then call publish on the bus:
await bus.Publish<TestMessage>(message);
Thanks to #ChrisPatterson for pointing this out to me!

If you want to consume from a specific topic, create a subscription endpoint instead of a receive endpoint, and specify the topic and subscription name in the configuration.
The simplest form is shown in the unit tests:
https://github.com/MassTransit/MassTransit/blob/develop/tests/MassTransit.Azure.ServiceBus.Core.Tests/Subscription_Specs.cs

I was able to send to an Azure Service Bus Topic using the _sendEndpointProvider.GetSendEndpoint(new Uri("topic:shape")); where... "shape" is the topic name.
public class MassTransitController : ControllerBase
{
private readonly ILogger<MassTransitController> _logger;
private readonly ISendEndpointProvider _sendEndpointProvider;
public MassTransitController(ILogger<MassTransitController> logger, ISendEndpointProvider sendEndpointProvider)
{
_logger = logger;
_sendEndpointProvider = sendEndpointProvider;
}
[HttpGet]
public async Task<IActionResult> Get()
{
try
{
var randomType = new Random();
var randomColor = new Random();
var shape = new Shape();
shape.ShapeId = Guid.NewGuid();
shape.Color = ShapeType.ShapeColors[randomColor.Next(ShapeType.ShapeColors.Count)];
shape.Type = ShapeType.ShapeTypes[randomType.Next(ShapeType.ShapeTypes.Count)];
var endpoint = await _sendEndpointProvider.GetSendEndpoint(new Uri("topic:shape"));
await endpoint.Send(shape);
return Ok(shape);
}
catch (Exception ex)
{
throw ex;
}
}
}
I also was able to get a .NET 5 Worker Consumer working with code like this... where the subscription "sub-all" would catch all shapes.. I'm going to make a blog post / git repo of this.
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureServices((hostContext, services) =>
{
services.AddMassTransit(x =>
{
x.UsingAzureServiceBus((context, cfg) =>
{
cfg.Host("Endpoint=sb://******");
cfg.SubscriptionEndpoint(
"sub-all",
"shape",
e =>
{
e.Handler<Shape>(async context =>
{
await Console.Out.WriteLineAsync($"Shape Received: {context.Message.Type}");
});
e.MaxDeliveryCount = 15;
});
});
});
services.AddMassTransitHostedService();
});

Issues Publishing From Subsequent Rebus Instances

I have several services that are essentially console applications hosted using TopShelf, and communiate using Rebus 0.99.50. One of these services (StepManager) loops through a collection of objects (of type Step), each of which contains a Bus instance, which it uses to send a message, and a handler used to handle a reply. The following Step(s) used for this example, in this order, are:
ReceiveFile
LogFileMetrics
ArchiveIncomingFile
In my actual scenario, I have a total of 7 Step(s)...When looping through these Step(s), ReceiveFile and LogFileMetrics behave as expected, however when ArchiveIncomingFile runs, .Send(req) is called, but the message never reaches its destination, leaving the process waiting for the reply that never returns. Regardless of what type of Step object or order of the objects in the list, this happens consistently at second instance of type Step (which does a .Send(req) in the Run() method) in the list. BUT, when I comment out the while (!Completed) { await Task.Delay(25); } statements, the messages appear to get sent, however without those statements, the Step(s) will all run with no specific execution order, which is a problem.
Why is this happening? What am I missing/doing wrong here? And is there a better alternative to accomplish what I am trying to do?
Here are the relevant portions of the classes in question:
public class StepManager
{
...
public string ProcessName { get; set; }
public List<Step> Steps { get; set; }
public BuiltinHandlerActivator ServiceBus { get; set; }
...
public async Task Init()
{
...
Steps = new List<Step>();
var process = Db.Processes.Include("Steps")
.Where(p => p.Name == ProcessName)
.FirstOrDefault();
...
foreach (var s in process.Steps)
{
var step = container.Resolve<Step>(s.Name);
...
Steps.Add(step);
}
}
public async Task Run()
{
foreach (var step in Steps)
{
await step.Run();
}
}
}
public class Step
{
public BuiltinHandlerActivator ServiceBus { get; set; }
public Step()
{
Db = new ClearStoneConfigContext();
Timer = new Stopwatch();
StepId = Guid.NewGuid().ToString();
Completed = false;
}
public virtual async Task Run() { }
}
public class ReceiveFile : Step
{
public ReceiveFile()
{
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<ProcessLog>("stepmanager"))
.Transport(t => t.UseMsmq("receivefile"))
.Start();
}
public override async Task Run()
{
...
LogEntry.Message = "File " + FileEvent.Name + " received.";
await ServiceBus.Bus.Advanced.Routing.Send("stepmanager", LogEntry);
Completed = true;
}
}
public class LogFileMetrics : Step
{
public LogFileMetrics()
{
SubscriptionTable = "SandboxServiceBusSubscriptions";
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<LogFileMetricsRequest>("metrics"))
.Transport(t => t.UseMsmq("logfilemetrics"))
.Start();
ServiceBus.Handle<FileMetricsLogged>(async msg=> await FileMetricsLogged(msg));;
}
public override async Task Run()
{
...
await ServiceBus.Bus.Send(new LogFileMetricsRequest { ProcessId = ProcessId, FileEvent = FileEvent }).ConfigureAwait(false);
while (!Completed) { await Task.Delay(25); }
}
private async Task FileMetricsLogged(FileMetricsLogged msg)
{
...
await ServiceBus.Bus.Advanced.Routing.Send("stepmanager", LogEntry);
Completed = true;
}
}
public class ArchiveIncomingFile : Step
{
public ArchiveIncomingFile()
{
SubscriptionTable = "SandboxServiceBusSubscriptions";
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<ArchiveIncomingFileRequest>("incomingarchivefilerouter"))
.Transport(t => t.UseMsmq("archiveincomingfile"))
.Start();
ServiceBus.Handle<IncomingFileArchived>(async msg => await IncomingFileArchived(msg));
}
public override async Task Run()
{
...
ServiceBus.Bus.Send(req);
while (!Completed) { await Task.Delay(25); }
}
private async Task IncomingFileArchived(IncomingFileArchived msg)
{
...
await ServiceBus.Bus.Advanced.Routing.Send("stepmanager", LogEntry);
Completed = true;
}
}

I can see several issues with your code, although it is not clear to me what is causing the funny behavior you are experiencing.
First off, it seems like you are creating new bus instances every time you are creating steps. Are you aware that Rebus' bus instance is supposed to be created once at startup in your application, kept as a singleton, and must be properly disposed when your application shuts down?
You can of course perform this create-dispose cycle as many times as you like, it's not like Rebus will leave anything behind in any way, but the fact that you are NOT disposing the bus anywhere tells me that your application probably forgets to do this.
You can read more on the Rebus wiki, especially in the section about Rebus' bus instance.
Another issue is the subtle potential race condition in the ArchiveIncomingFile class whose ctor looks like this:
public ArchiveIncomingFile()
{
SubscriptionTable = "SandboxServiceBusSubscriptions";
ServiceBus = new BuiltinHandlerActivator();
Configure.With(ServiceBus)
.Logging(l => l.ColoredConsole(LogLevel.Info))
.Routing(r => r.TypeBased().Map<ArchiveIncomingFileRequest>("incomingarchivefilerouter"))
.Transport(t => t.UseMsmq("archiveincomingfile"))
.Start();
//<<< bus is receiving messages at this point, but there's no handler!!
ServiceBus.Handle<IncomingFileArchived>(async msg => await IncomingFileArchived(msg));
}
As you can see, there is a (very very very short, admittedly) time (marked by //<<<) in which the bus has been started (and thus will start to pull messages out of its input queue) where no handlers yet have been configured.
You should be sure to configure handlers BEFORE you start the bus.
Finally, you are asking
And is there a better alternative to accomplish what I am trying to do?
but I am unable to answer that question because I simply cannot figure out what you are trying to do ;)
(but if you explain to me at a slightly higher level what problem you are trying to solve, I might have some hints for you :))