I'm trying to figure out the best way to wait for some number of I/O Completion ports to complete.
For this scenario, let's say that I'm in a MVC3 web app. (My understanding is the use of I/O Completion ports here is recommended so I can return the original thread back to IIS to service other requests)
Lets say I have an array of IDs and I want to fetch an object for each ID from some network call.
What is the best way to parallelize this synchronous method?
public class MyController: Controller
{
public ActionResult Index(IEnumerable<int> ids)
{
ids.Select(id => _context.CreateQuery<Order>("Orders")
.First(o => o.id == id));
DataServiceQuery<Order> query = _context.CreateQuery<Order>("Orders");
return Json(query);
}
private DataServiceContext _context; //let's ignore how this would be populated
}
I know it would start like this:
public class MyController: AsyncController
{
public void IndexAsync(IEnumerable<int> ids)
{
// magic here...
AsyncManager.Sync(() => AsyncManager.Parameters["orders"] = orders);
}
public ActionResult IndexCompleted(IEnumerable<Order> orders)
{
return Json(orders);
}
private DataServiceContext _context; //let's ignore how this would be populated
}
Should I be using DataServiceContext.BeginExecute Method? DataServiceContext.BeginExecuteBatch ? The data service I'm consuming can only get one record at a time (this is beyond my control) and I want these individual queries to run in parallel.
This is the pattern I've ended up using for running a batch of async operations inside MVC3:
public class MyController: AsyncController
{
public void IndexAsync(int[] ids)
{
var orders = new Orders[ids.Length];
AsyncManager.Parameters["orders"] = orders;
// tell the async manager there are X operations it needs to wait for
AsyncManager.OutstandingOperations.Increment(ids.Length);
for (int i = 0; i < ids.Length; i++){
var index = i; //<-- make sure we capture the value of i for the closure
// create the query
var query = _context.CreateQuery<Order>("Orders");
// run the operation async, supplying a completion routine
query.BeginExecute(ar => {
try {
orders[index] = query.EndExecute(ar).First(o => o.id == ids[index]);
}
catch (Exception ex){
// make sure we send the exception to the controller (in case we want to handle it)
AsyncManager.Sync(() => AsyncManager.Parameters["exception"] = ex);
}
// one more query has completed
AsyncManager.OutstandingOperations.Decrement();
}, null);
}
}
public ActionResult IndexCompleted(Order[] orders, Exception exception)
{
if (exception != null){
throw exception; // or whatever else you might like to do (log, etc)
}
return Json(orders);
}
private DataServiceContext _context; //let's ignore how this would be populated
}
Using TPL is easy.
public ActionResult Index(IEnumerable<int> ids)
{
var result = ids.AsParallel()
.Select(id => GetOrder(id))
.ToList();
return Json(result);
}
Order GetOrder(int id) { ... }
Related
I'm working on an ASP.NET Core MVC application and I have a view who do a post request as:
$.ajax({
url:'/Advertisers/ActiveAdvertiser?id='+id+'&isActive='+!isActive,
method: 'POST',
success: function(r){
Swal.fire("Inactivated!", "Advertiser inactivated successfully", "success");
},
error: function (request) {
console.log(request.responseText)
Swal.fire("Error!", "Something went wrong, please try again`", "warning");
}
});
Controller:
[HttpPost]
public async Task<JsonResult> ActiveAdvertiser(int id, bool isActive)
{
var advertiser = await _advertisersService.GetAdvertiserByAdvertiserIdAsync(id);
if (advertiser != null)
{
var model = AssingAdvertiserViewModel(advertiser, id);
model.IsActive = isActive;
var result = await _advertisersService.UpdateAdvertiserAsync(model, GetCurrentUserAsync().Id);
if (result != null)
{
return Json(new { result = "OK" });
}
}
return Json(new { result = "BadRequest" });
}
Post method services:
public Task<Advertiser?> GetAdvertiserByAdvertiserIdAsync(int advertiserId)
{
return _db.Advertisers
.Include(a => a.Address)
.Include(pc => pc.PrimaryContact)
.Include(ac => ac.AlternateContact)
.FirstOrDefaultAsync(x => x.AdvertiserId == advertiserId);
}
private AdvertiserViewModel AssingAdvertiserViewModel(Advertiser advertiser, int id)
{
var model = new AdvertiserViewModel()
{
//Fill model here
};
return model;
}
public async Task<Advertiser?> UpdateAdvertiserAsync(AdvertiserViewModel model, int updatedById)
{
var advertiser = await GetAdvertiserByAdvertiserIdAsync(model.AdvertiserId);
if (advertiser is null)
return null;
advertiser.Name = model.Name;
// fill model here
await _db.SaveChangesAsync();
return advertiser;
}
The problem is I do the first request, and it returns Success with any issues, but if I try to do a second one, it throws an exception:
System.InvalidOperationException: A second operation was started on
this context instance before a previous operation completed. This is
usually caused by different threads concurrently using the same
instance of DbContext.
If I stop the project and run it again it works one time again and in the second time get the error again
I read about this issue in other questions, and apparently is because you don't use the await services, I check my code and almost everything uses await. Can someone see something that I don't see? Regards
You could check this document for how to handle this error:
Therefore, always await async calls immediately, or use separate DbContext instances for operations that execute in parallel.
So you could check if misssing the await keyword on async operation
and use separate Dbcontext instances with DbcontextFactory as below:
regist the factory in service collection:
builder.Services.AddDbContextFactory<SomeContext>();
inject it into controller/Service/Somewhereelse:
public class SomeEntitiesController : Controller
{
private readonly IDbContextFactory<SomeContext> _factory;
public SomeEntitiesController(IDbContextFactory<SomeContext> factory)
{
_factory = factory;
}
}
create a new dbcontext:
_factory.CreateDbContext()
I solve this by adding ServiceLifetime.Transient into my services as:
services.AddDbContext<ApplicationDbContext>(
options =>
options.UseSqlServer(Configuration.GetConnectionString("ApplicationDbConnection")),
ServiceLifetime.Transient
);
I'm building an n-tier web app in ASP.NET Core 3.1 that uses Entity Framework to interact with a SQL Server database. Everything works fine except for updating an entity twice without refreshing in between both requests.
So the first time I read and entity from the database and update it, it works perfectly. When I try to update the entity immediately after, I get the following exception:
Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: Database operation expected to affect 1 row(s) but actually affected 0 row(s).
Data may have been modified or deleted since entities were loaded...
I'm using the generic repository pattern in combination with asynchronous code.
An overview of my code:
GenericRepository.cs:
public class Repository<T> : IRepository<T> where T : class
{
protected readonly DataContext _context;
protected DbSet<T> _entities;
public Repository(DataContext context)
{
this._context = context;
_entities = context.Set<T>();
}
}
MemberRepository.cs:
public class MemberRepository : Repository<Member>, IMemberRepository
{
public MemberRepository(DataContext context) : base(context)
{
//
}
public async override Task<Member> SelectByIdAsync(int id)
{
return await this._entities
.Where(m => m.Id == id)
.Include(m => m.Addresses)
.Include(m => m.MemberHobbies)
.Include(m => m.Car)
.Include(m => m.EmailAddresses)
.FirstOrDefaultAsync();
//return await this._context.Members.Where(m => m.Id == id).Include(m => m.Addresses).Include(m => m.MemberHobbies).Include(m => m.Car).Include(m => m.EmailAddresses).FirstOrDefaultAsync();
}
public async override Task UpdateAsync(Member member)
{
this._context.Members.Update(member);
//this._context.Set<Member>().Update(member);
await this._context.SaveChangesAsync();
}
}
startup.cs:
services.AddScoped(typeof(IRepository<>), typeof(Repository<>));
services.AddScoped<IMemberRepository, MemberRepository>();
I have tried to change the way I read or update my entity (see the commented out lines), but those give the same result. Also: I have other entities which work perfectly, the only difference is that the Member class has multiple foreign keys (see .Include(...))
Might it have anything to do with improper use of asynchronous functions? Maybe because I'm not returning the updated Member and thus next time I execute an update I'm sending a now out of date member?
Also: I found this answer, but it makes no sense to me at all
Thanks!
I use the generic code like this for update:
public virtual async Task<T> UpdateAsync(T t)
{
if (t == null) return null;
T exist;
try
{
exist = await Context.Set<T>().FindAsync(t.Id);
if (exist == null)
{
t.ErrorMessage = "Can't find item to update";
return t;
}
Context.Entry(exist).CurrentValues.SetValues(t);
var result = await Context.SaveChangesAsync();
if (result == 0) t.ErrorMessage = "Can't saved item";
}
catch (Exception ex)
{
t.ErrorMessage = ex.Message;
return t;
}
return exist;
}
Or you can try this code for Member class:
public async Task UpdateAsync(Member member)
{
try
{
var exist = await _context.Members.Where(i.Id=member.Id).FirstOrDefaultAsync();
if (exist == null) ....errorMessage = "Can't find item to update";
Context.Entry(exist).CurrentValues.SetValues(member);
var result = await Context.SaveChangesAsync();
if (result == 0) ....errorMessage = "Can't save item";
}
catch (Exception ex)
{
...errorMessage = ex.Message;
}
}
EF keeps track of your data. Here's what you do:
Get two copies of the same entry;
Change the former copy;
Update the change for former copy;
Change the latter copy, which kept the track of the entry's state prior the 3rd step;
Update the change made by the latter copy, but since this change reflects to the former state of an entry - you get that error;
Here's what you should do instead:
Add an extra layer to your business objects that will carry the states around;
Get a current state of your database entry upon receiving the update request;
Apply the changes to your DB entry;
Save;
Example:
// DB entry
public class Member
{
public int Id { get; set; }
public string Name { get; set; }
}
// DTO for carrying around
public class MemberForNameChangeDto
{
public int Id { get; set; }
public string Name { get; set; }
}
public class Repo
{
public Task<bool> UpdateMemberName(MemberForNameChangeDto memberForNameChange)
{
// _context.Members is DbSet<Member>
var memberEntry = _context.Members.Find(memberForNameChange.Id);
// null checks, verifications, etc
memberEntry.Name = memberForNameChange.Name;
var result = await _context.SaveChangesAsync();
return result > 0;
}
}
A second operation started on this context before a previous asynchronous operation completed. Use 'await' to ensure that any asynchronous operations have completed before calling another method on this context. Any instance members are not guaranteed to be thread safe.
My unitofwork code
public class UnitOfWork : IUnitOfWork
{
private readonly CAMSDbEntities _context;
private bool _disposed;
public Dictionary<Type, object> repositories = new Dictionary<Type, object>();
private Guid _objectId;
public UnitOfWork(IContextFactory contextFactory)
{
_context = contextFactory.DbContext as CAMSDbEntities;
_objectId = Guid.NewGuid();
}
public IGenericRepository<T> Repository<T>() where T : class
{
if (repositories.Keys.Contains(typeof(T)) == true)
{
return repositories[typeof(T)] as GenericRepository<T>;
}
GenericRepository<T> repo = new GenericRepository<T>(_context);
repositories.Add(typeof(T), repo);
return repo;
}
My unity config
container.RegisterType<IHttpContext, HttpContextObject>();
container.RegisterType<IDataBaseManager, DataBaseManager>();
container.RegisterType<IContextFactory, ContextFactory>();
container.RegisterType(typeof(IGenericRepository<>), typeof(GenericRepository<>));
container.RegisterType<IUnitOfWork, UnitOfWork>();
container.RegisterType<IAnalytics, DashbordService>();
GlobalConfiguration.Configuration.DependencyResolver = new UnityDependencyResolver(container);
webApi Controller
public class DashbordController : ApiController
{
private static IAnalytics _analytics;
public DashbordController(IAnalytics dashbordService)
{
_analytics = dashbordService;
}
[HttpGet]
[Route("GetStudentAssessmentHistory")]
public IHttpActionResult GetStudentAssessmentHistory(int studentID)
{
var result = _analytics.GetStudentAssessmentHistoryGraphData(studentID);
return Ok(result);
}
[HttpGet]
[Route("GetStudentFeePaymentHistory")]
public async Task<IHttpActionResult> GetStudentFeePaymentData(int studentID)
{
var result = await _analytics.GetStudentFeePaymentData(studentID);
return Ok(result);
}
[HttpGet]
[Route("GetLedgerHitoryByDepartment")]
public async Task<IHttpActionResult> GetLedgerHitoryByDepartment(int schoolID, int departmentId)
{
var result = await _analytics.GetLedgerHitory(schoolID, departmentId);
return Ok(result);
}
[HttpGet]
[Route("GetLedgerExpenseTrendByDepartment")]
public async Task<IHttpActionResult> GetLedgerExpenseTrendByDepartment(int schoolID)
{
var result = await _analytics.GetLedgerExpenseTrend(schoolID);
return Ok(result);
}
dashboardservice Code
public async Task<List<LedgerExpense>> GetLedgerExpenseTrend(int schoolId)
{
try
{
var ledgerExpenses = new List<LedgerExpense>();
var currentDate = TimeZoneInfo.ConvertTimeFromUtc(DateTime.UtcNow, INDIAN_ZONE);
DateTime previoYearDate = currentDate.AddYears(-1);
var ledgerPayments = await _unitOfWork.Repository<LedgerDetail>().GetManyAsync(x => x.SchoolID == schoolId && x.PaymentDate <= currentDate
&& x.PaymentDate >= previoYearDate);
foreach (var ledgerPayment in ledgerPayments.OrderBy(x => x.PaymentDate).GroupBy(y => y.DepartmentID))
{
var department = await _unitOfWork.Repository<DeptartmentType>().GetAsync(x => x.ID == ledgerPayment.Key);
var ledgerData = new LedgerExpense
{
Department = department.DepartmentName,
TotalLedgerExpense = 0
};
foreach (var departmentPayment in ledgerPayment)
{
ledgerData.TotalLedgerExpense += departmentPayment.TotalPaidAmount;
}
ledgerExpenses.Add(ledgerData);
}
return ledgerExpenses;
}
catch (Exception ex)
{
logger.Log("An error occurred while fetching ledger expenses");
return null;
}
}
I have similar type of asynchronous metods implemented in my dashboardservice code. whenever I request a dashboard UI all request comes to the same controller at the same time and creates the new object for unitofwork and dbcontext for each request one by one. it works perfectly sometimes but Sometimes I think unitofwork and dbcontext object flows with the wrong thread and throws this error. I think somehow its picking wrong dbcontext which is already busy with someother api request from dashboard service.
Please remove the static keyword in your controller from this code:
private static IAnalytics _analytics;`
Once that has been created, it will never be created again unless the application pool is recycled (manual or IIS restart etc.) Since you are using the same instance for all requests, you are getting that error at random. If a request finishes before the next one arrives, it will NOT result in an error. Otherwise it will. Hence the reason for not always getting the error (as you mention in your question).
Please read about how static affects the design in a web scenario (or server).
Try and think of web requests as a single transaction, all classes are created for each request and then thrown away after the request has been served. That means if you have static or any other mechanism which is for sharing, it will be shared between requests.
I have a list lstSubscriptionRequests on which indivisual items i am doing some sort of processing asynchronously. Then after All items are processed i have to return the updated list items. My current implementation is like
List<SubscriptionRequest> lstSubscriptionRequests = FromSomeResource();
var tsk = new List<Task>();
foreach (var oTsk in lstSubscriptionRequests.Select(objSubscriptionRequest => new Task(
() => ProcessSubscriptionForASingleRecord(objSubscriptionRequest))))
{
oTsk.Start();
lock (tsk)
{
tsk.Add(oTsk);
}
}
Task.WaitAll(tsk.ToArray());
It's looks like some of the items after all the tasks completed are not updated.
Please let me know what correction i needed
You could accomplish this much easier. Note that using the Task constructor is not typically necessary or recommended, also mutating the state of a particular object can be difficult to follow or debug. Returning a new object that represents your desired state will allow you to enforce a minimum valid state. The following code will processes all your items and return the completed items to your client code.
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
namespace Processing {
public class MyProcessor {
public async Task<IEnumerable<Subscription>> ProcessSubscriptionRequestsAsync(IEnumerable<SubscriptionRequest> subscriptionRequests) {
var subscriptionProcessingTasks = subscriptionRequests.Select(request => ProcessSubscriptionForASingleRecord(request)).ToArray();
return await Task.WhenAll(subscriptionProcessingTasks);
}
public async Task<Subscription> ProcessSubscriptionForASingleRecord(SubscriptionRequest request) {
//process the request
try {
var subscription = await Context.ProcessRequest(request);
return subscription;
} catch {
//something went wrong with the request
}
}
}
public class SubscriptionRequest {
//A subcription request
}
public class Subscription {
//A completed subscription request
}
}
Update
It could help if you could exclude new Class Subscription and add the solution in your answer. i will then try it out
Hopefully the simplified before and after view will be easier to integrate. The primary difference is replacing the Parallel.ForEach with a Select to create your collection of tasks without needing to take a lock on the task list for every SubscriptionRequest, also spooling up Tasks in parallel isn't typically necessary as each one will be executing asynchronously you only gain reaching a point where all are waiting sooner, not finishing. Next each tasks is allowed to start and all are awaited at await Task.WhenAll(tasks). It will be important for you to determine what type of processing each SubscriptionRequest undergoes. For the sake of example i"ve made the assumption that each request is somehow linked to database access, i.e. storing the request, updating a user profile of sorts etc..
public class OriginalSynchronous {
public void ProcessSubscriptionRequest() {
List<SubscriptionRequest> lstSubscriptionRequests = FromSomeResource();
List<Task> tsk = new List<Task>();
Parallel.ForEach(lstSubscriptionRequests, objSubscriptionRequest => {
var oTsk =
new Task(
() => ProcessSubscriptionForASingleRecord(objSubscriptionRequest));// update some properties after processing SubscriptionRequest
oTsk.Start();
lock (tsk) {
tsk.Add(oTsk);
}
});
Task.WaitAll(tsk.ToArray());
}
private void ProcessSubscriptionForASingleRecord(SubscriptionRequest request) {
//modify SubscriptionRequest
}
}
public class ModifiedAsync {
public async Task ProcessSubscriptionRequest() {
var subscriptionRequests = await FromSomeResourceAsync();
var tasks = subscriptionRequests.Select(request => {
return ProcessSubscriptionForASingleRecord(request);
}).ToArray();
await Task.WhenAll(tasks);
}
public async Task ProcessSubscriptionForASingleRecord(SubscriptionRequest request) {
//modify SubscriptionRequest
}
}
I'm trying to learn how to use AsyncController in MVC2, but there is very little documentation/tutorials out there. I'm looking to take one normal controller method that has a very slow export to a 3rd party service and convert that to an async method.
The original controller method:
public JsonResult SaveSalesInvoice(SalesInvoice invoice)
{
SaveInvoiceToDatabase(invoice); // this is very quick
ExportTo3rdParty(invoice); // this is very slow and should be async
}
So I created a new controller that inherits from AsyncController:
public class BackgroundController : AsyncController
{
public void ExportAysnc(int id)
{
SalesInvoice invoice = _salesService.GetById(id);
ExportTo3rdParty(invoice);
}
public void ExportCompleted(int id)
{
// I dont care about the return value right now,
// because the ExportTo3rdParty() method
// logs the result to a table
}
public void Hello(int id)
{
}
}
And then call the Export method from jQuery:
function Export() {
$.post("Background/Export", { id: $("#Id").val() }, function (data) {
// nothing to do yet
});
}
BUT the result is a 404 not found error (Background/Export is not found). If I try to call Background/Hello or Background/ExportAysnc they are found.
What am I doing wrong?
There are indeed two use cases
You care about the result of the lengthy operation
You don't care about the result (fire and forget)
Let's start with the first case:
public class BackgroundController : AsyncController
{
public void ExportAysnc(int id)
{
AsyncManager.OutstandingOperations.Increment();
Task.Factory.StartNew(() => DoLengthyOperation(id));
// Remark: if you don't use .NET 4.0 and the TPL
// you could manually start a new thread to do the job
}
public ActionResult ExportCompleted(SomeResult result)
{
return Json(result, JsonRequestBehavior.AllowGet);
}
private void DoLengthyOperation(int id)
{
// TODO: Make sure you handle exceptions here
// and ensure that you always call the AsyncManager.OutstandingOperations.Decrement()
// method at the end
SalesInvoice invoice = _salesService.GetById(id);
AsyncManager.Parameters["result"] = ExportTo3rdParty(invoice);
AsyncManager.OutstandingOperations.Decrement();
}
}
Now you could invoke it like this:
$.getJSON(
'<%= Url.Action("Export", "Background") %>',
{ id: $("#Id").val() },
function (data) {
// do something with the results
}
);
Now because you have mentioned a web service call, this means when you generated the client proxy of your web service you had the chance to emit async methods (XXXCompleted and XXXAsync):
public class BackgroundController : AsyncController
{
public void ExportAysnc(int id)
{
AsyncManager.OutstandingOperations.Increment();
// that's the web service client proxy that should
// contain the async versions of the methods
var someService = new SomeService();
someService.ExportTo3rdPartyCompleted += (sender, e) =>
{
// TODO: Make sure you handle exceptions here
// and ensure that you always call the AsyncManager.OutstandingOperations.Decrement()
// method at the end
AsyncManager.Parameters["result"] = e.Value;
AsyncManager.OutstandingOperations.Decrement();
};
var invoice = _salesService.GetById(id);
someService.ExportTo3rdPartyAsync(invoice);
}
public ActionResult ExportCompleted(SomeResult result)
{
return Json(result, JsonRequestBehavior.AllowGet);
}
}
This is the best possible usage of an async controller as it relies on I/O Completion Ports and doesn't monopolize any threads on the server during the execution of the lengthy operation.
The second case is easier (don't really need an async controller):
public class BackgroundController : Controller
{
public ActionResult Export(int id)
{
// Fire and forget some lengthy operation
Task.Factory.StartNew(() => DoLengthyOperation(id));
// return immediately
return Json(new { success = true }, JsonRequestBehavior.AllowGet);
}
}
Here's a nice article on MSDN on Async controllers.