We Keep Coding

Entity Framework locking database

The query below is very slow and inefficient. Depending on the user supplied parameters, it could take a minute to run. We can obviously make the query more efficient, but our issue is that when it's run, it seems to lock the database.
On our clients site, when it's run, our application will hang on every pc on our clients site until it completes. If we run the same query in management studio, it still takes about a minute to run, but it doesn't affect any other pc terminals..i.e. the database doesn't lock.
Does anybody have any ideas as to why this is happening through EF? (we're using EF6)
Edit: Just to reiterate - Our issue isn't about improving the query (we can do that)..It's why this query locks the database when run through the entity framework (affecting every instance of our application running on our clients site), and doesn't affect any when run through management studio.
var details =
(from stock in
entities.StockTransactions.Where(x => x.Deleted == null
&& x.NouTransactionTypeID == NouvemGlobal.TransactionTypeGoodsReceiptId
&& x.IsBox == true)
join attribute in entities.Attributes.Where(x => DbFunctions.TruncateTime(x.GradingDate) >= start &&
DbFunctions.TruncateTime(x.GradingDate) <= end && x.NouDocStatusID != status) on stock.AttributeID equals attribute.AttributeID
join intakeDetail in entities.APGoodsReceiptDetails on stock.MasterTableID equals intakeDetail.APGoodsReceiptDetailID
join intake in entities.APGoodsReceipts on intakeDetail.APGoodsReceiptID equals intake.APGoodsReceiptID
select new StockDetail
{
LoadingStockDetail = true,
Paid = stock.KillPaymentItems.Any(pay => pay.Deleted == null),
StockTransactionID = stock.StockTransactionID,
APGoodsReceiptDetailID = stock.MasterTableID,
AttributeID = attribute.AttributeID,
TransactionWeight = stock.TransactionWeight,
Serial = stock.Serial,
TransactionDate = stock.TransactionDate,
Detained = attribute.Detained,
Condemned = attribute.Condemned,
Eartag = attribute.Eartag,
KillType = attribute.KillType,
NouDocStatusID = attribute.NouDocStatusID,
Breed = attribute.NouBreed,
Grade = attribute.Grade,
DOB = attribute.DOB,
Identigen = attribute.Identigen,
HoldingNumber = attribute.HoldingNumber,
CountryOfOrigin = attribute.CountryOfOrigin,
KillNumber = attribute.KillNumber,
Generic1 = attribute.Generic1,
CarcassNumber = attribute.CarcassNumber,
CarcassSide = attribute.CarcassSide == 1 ? StockDetail.Side.Side1 : StockDetail.Side.Side2,
AgeInMonths = attribute.AgeInMonths,
AgeInDays = attribute.AgeInDays,
Cleanliness = attribute.NouCleanliness,
SequencedDate = attribute.SequencedDate,
Category = attribute.NouCategory,
Sex = attribute.Sex,
GradingDate = attribute.GradingDate,
CustomerID = attribute.Customer,
FarmAssured = attribute.FarmAssured,
Clipped = attribute.Clipped,
Casualty = attribute.Casualty,
Lame = attribute.Lame,
IsFullCarcass = stock.IsBox,
NumberOfMoves = attribute.NoOfMoves,
PreviousResidency = attribute.PreviousResidency,
TotalResidency = attribute.TotalResidency,
CurrentResidency = attribute.CurrentResidency,
DateOfLastMove = attribute.DateOfLastMove,
Imported = attribute.Imported,
PaidWeight = attribute.ColdWeight,
UTM = attribute.AgeInMonths != null && attribute.AgeInMonths <= ApplicationSettings.GraderLabelUOMAge ? 1 : 0,
OTM = attribute.AgeInMonths != null && attribute.AgeInMonths > ApplicationSettings.GraderLabelUOMAge ? 1 : 0,
SupplierID = intake.BPMasterID_Supplier
})
.ToList();

Linq calling a method in select statement very slow

I am trying to find if there is a way to call a method from within my linq select statement to build a list of objects that doesn't dramatically slow it down. The reason behind this is that I also want to call the same method when trying to get only one of the objects and do not want to have to maintain both versions (i.e. if I have another field added to the object or want to render one of the fields differently I will not have to change it in multiple places).
In the example below, TEST 1 runs over 100 times faster than TEST 2:
// Start timer
var timer = new Stopwatch();
timer.Start();
var test = (from job in dc.Jobs
where !job.archived
select new JobExtended()
{
JobId = job.jobId,
NodeId = job.nodeId,
JobName = job.name != string.Empty ? job.name : "TBC",
Salary = job.salary,
RecruiterId = job.fkRecruiterId,
RecruiterNodeId = job.JobRecruiter != null ? job.JobRecruiter.recruiterNodeId : null,
RecruiterName = job.JobRecruiter != null ? job.JobRecruiter.name : string.Empty,
LocationId = job.fkLocationId,
Location = job.refJobLocation != null ? job.refJobLocation.jobLocation : "",
ContractTypeId = job.fkContractTypeId,
ContractType = job.refJobContractType != null ? job.refJobContractType.contractType : "",
CategoryId = job.fkCategoryId,
Category = job.refJobCategory != null ? job.refJobCategory.category : "",
ClosingDate = job.closingDate,
Featured = job.featured,
JobOfTheWeek = job.jobOfTheWeek,
PublishedDate = job.publishedDate,
Url = "/jobs/" + job.name.Replace(" ", "-").Replace("&", "and").Replace("'", "") + (job.fkLocationId.HasValue ? "-in-" + job.refJobLocation.jobLocation.Replace(" ", "-").Replace("&", "and").Replace("'", "") : "") + "-jn" + job.jobId,
CreatedOn = job.createdOnDate,
PrintWidth = job.printWidth,
PrintHeight = job.printHeight,
UntilFilled = (job.untilFilled != null && job.untilFilled.Value),
AdvertCost = job.advertCost,
DatesToShow = job.relJobDates.Where(x => x.fkJobId == job.jobId).Select(x => x.date).ToList(),
IsParentJob = job.relLinkedJobs != null && job.relLinkedJobs.Any(x => x.fkParentJobId == job.jobId),
IsAlternateWeekJob = job.alternateWeek != null && job.alternateWeek.Value,
Archived = job.archived,
LastModifiedDate = job.lastModifiedDate,
RecruiterContactDetails = job.recruiterContactDetails
}).ToList();
// Stop timer
timer.Stop();
// Output info
litTest.Text = "TEST 1 in " + timer.Elapsed.TotalSeconds + " seconds<br/>";
//Start timer
timer = new Stopwatch();
timer.Start();
var test2 = (from job in dc.Jobs
where !job.archived
select GetJobDetails(job)).ToList();
//Stop timer
timer.Stop();
//Output info
litTest.Text += "TEST 2 in " + timer.Elapsed.TotalSeconds + " seconds<br/>";
This is the method that TEST 2 is calling which should be creating the same object that is being returned in TEST 1:
public static JobExtended GetJobDetails(Data.Job job)
{
return new JobExtended()
{
JobId = job.jobId,
NodeId = job.nodeId,
JobName = job.name != string.Empty ? job.name : "TBC",
Salary = job.salary,
RecruiterId = job.fkRecruiterId,
RecruiterNodeId = job.JobRecruiter != null ? job.JobRecruiter.recruiterNodeId : null,
RecruiterName = job.JobRecruiter != null ? job.JobRecruiter.name : string.Empty,
LocationId = job.fkLocationId,
Location = job.refJobLocation != null ? job.refJobLocation.jobLocation : "",
ContractTypeId = job.fkContractTypeId,
ContractType = job.refJobContractType != null ? job.refJobContractType.contractType : "",
CategoryId = job.fkCategoryId,
Category = job.refJobCategory != null ? job.refJobCategory.category : "",
ClosingDate = job.closingDate,
Featured = job.featured,
JobOfTheWeek = job.jobOfTheWeek,
PublishedDate = job.publishedDate,
Url = "/jobs/" + job.name.Replace(" ", "-").Replace("&", "and").Replace("'", "") + (job.fkLocationId.HasValue ? "-in-" + job.refJobLocation.jobLocation.Replace(" ", "-").Replace("&", "and").Replace("'", "") : "") + "-jn" + job.jobId,
CreatedOn = job.createdOnDate,
PrintWidth = job.printWidth,
PrintHeight = job.printHeight,
UntilFilled = (job.untilFilled != null && job.untilFilled.Value),
AdvertCost = job.advertCost,
DatesToShow = job.relJobDates.Where(x => x.fkJobId == job.jobId).Select(x => x.date).ToList(),
IsParentJob = job.relLinkedJobs != null && job.relLinkedJobs.Any(x => x.fkParentJobId == job.jobId),
IsAlternateWeekJob = job.alternateWeek != null && job.alternateWeek.Value,
Archived = job.archived,
LastModifiedDate = job.lastModifiedDate,
RecruiterContactDetails = job.recruiterContactDetails
};
}
The reason for this is because I want to be able to call "GetJobDetails" for returning a single job for example:
public JobExtended GetJobDetails(int jobId)
{
using (DataContext dc = new DataContext())
{
return dc.Jobs.Where(x => x.jobId == jobId).Select(j => GetJobDetails(j)).FirstOrDefault();
}
}
Doing it like this would allow me to only ever have to update the "GetJobDetails" method if for example I decided to add a new field of change how the "Url" value was generated but doing it this way is a lot slower. Is there a way around this, I have already tried the following which do not seem to help:
var test3 = (from job in dc.Jobs
where !job.archived
select job).AsEnumerable()
.Select(GetJobDetails).ToList();
var test4 = (from job in dc.Jobs
where !job.archived
select GetJobDetails(job));
var test4a = test4.ToList();

The reason TEST 1 is faster because the query is executed once on the server and only returns the selected fields.
var test = (from job in dc.Jobs
where !job.archived
select new JobExtended()
{
JobId = job.jobId,
NodeId = job.nodeId,
...
}).ToList();
When you call GetJobDetails in TEST 2 the parameter j needs to be materialized first before it can be send as parameter to GetJobDetails. And so there are multiple calls of the full objects.
return dc.Jobs.Where(x => x.jobId == jobId).Select(j => GetJobDetails(j)).FirstOrDefault();
In order to achieve something like you want you should use extension methods. This one extends IQueryable.
public static IEnumerable<JobExtended> SelectJobExtended(this IQueryable<Data.Job> query)
{
return query
.Select(o => new JobExtended()
{
JobId = job.jobId,
NodeId = job.nodeId,
...
}
}
Then you can call:
dc.Jobs.Where(x => x.jobId == jobId).SelectJobExtended().FirstOrDefault();

I've seen this kind of issue before. If I recall, what we did was "stacked" the queries.
public IEnumerable<JobExtended> ConvertToJobExtended(IEnumerable<Job> jobs)
{
return
from job in jobs
select new JobExtended()
{
MyString = job.MyInt.ToString(),
...
};
}
Then what you can do is call it the following way:
var query = (from job in dc.Jobs
where !job.archived
select job;
var test2 = ConvertToJobExtended(query).ToList();
There are plenty of alternatives that can go from here... I hope this goes in the right direction of what you're looking to do.

LINQ to Entities Method Not Recognized - Extension Method Style

Here's yet another "LINQ to entities does not recognize the method question"... however, isn't the code below doing fundamentally the exact same thing?
Works:
var returnData = from x in MyEntities.MyDBSet
where x.MyDBSetPrimaryKey == id
select new Models.MyModelDTO
{
MyPropOne = (int)x.MyModel.MyOtherPropOne,
MyPropTwo = x.MyOtherPropTwo ?? 0,
MyPropThree = x.MyModel.MyOtherPropThree,
MyPropFour = x.MyModel.MyOtherPropFour,
MyPropFive = x.MyModel.Entity.MyOtherPropFive,
MyPropSix = x.MyModel.MyOtherPropSix == null ? 0 : (decimal)x.MyModel.MyOtherPropSix,
MyPropSeven = x.MyModel.SomeType.MyOtherPropSeven,
MyPropEight = (int)x.MyModel.MyOtherPropEight,
MyPropNine = x.MyModel.MyPropNine == null ? 0 : (int)x.MyModel.MyOtherPropNine,
MyPropTen = x.MyModel.MyOtherPropTen == null ? 0 : (int)x.MyModel.MyOtherPropTen,
MyPropEleven = x.OtherEntity.MyOtherPropEleven,
MyPropTwelve = x.MyOtherpropTwelve
};
Doesn't Work:
The same exact assignments wrapped in an extension method:
public static MyModelDTO ToModelDTO(this MyModel x)
{
return new MyModelDTO()
{
MyPropOne = (int) x.MyModel.MyOtherPropOne,
MyPropTwo = x.MyOtherPropTwo ?? 0,
MyPropThree = x.MyModel.MyOtherPropThree,
MyPropFour = x.MyModel.MyOtherPropFour,
MyPropFive = x.MyModel.Entity.MyOtherPropFive,
MyPropSix = x.MyModel.MyOtherPropSix == null ? 0 : (decimal) x.MyModel.MyOtherPropSix,
MyPropSeven = x.MyModel.SomeType.MyOtherPropSeven,
MyPropEight = (int) x.MyModel.MyOtherPropEight,
MyPropNine = x.MyModel.MyPropNine == null ? 0 : (int) x.MyModel.MyOtherPropNine,
MyPropTen = x.MyModel.MyOtherPropTen == null ? 0 : (int) x.MyModel.MyOtherPropTen,
MyPropEleven = x.OtherEntity.MyOtherPropEleven,
MyPropTwelve = x.MyOtherpropTwelve
};
}
And later called:
var returnData = from x in MyEntities.MyDBSet
where x.MyDBSetPrimaryKey == id
select x.ToModelDto();
Resulting in:
LINQ to Entities does not recognize the method 'MyExtensionMethods.MyModels.MyModelDTO ToModelDTO(API.Models.MyModel)' method, and this method cannot be translated into a store expression.

When the query provider sees that method it doesn't know what to do with it. It can't go in and see the source code of the method, the way it can look into Expression objects and see what is done. It can't evaluate it on the client side as it doesn't have the items yet, and it can't think of any SQL to translate that method call into.
Instead you should write a method that takes an IQueryable and returns another IQueryable, like so:
public static IQueryable<MyModelDTO> ToModelDTO(this IQueryable<MyModel> query)
{
return query.Select(x => new MyModelDTO()
{
MyPropOne = (int)x.MyModel.MyOtherPropOne,
MyPropTwo = x.MyOtherPropTwo ?? 0,
MyPropThree = x.MyModel.MyOtherPropThree,
MyPropFour = x.MyModel.MyOtherPropFour,
MyPropFive = x.MyModel.Entity.MyOtherPropFive,
MyPropSix = x.MyModel.MyOtherPropSix == null ? 0 : (decimal)x.MyModel.MyOtherPropSix,
MyPropSeven = x.MyModel.SomeType.MyOtherPropSeven,
MyPropEight = (int)x.MyModel.MyOtherPropEight,
MyPropNine = x.MyModel.MyPropNine == null ? 0 : (int)x.MyModel.MyOtherPropNine,
MyPropTen = x.MyModel.MyOtherPropTen == null ? 0 : (int)x.MyModel.MyOtherPropTen,
MyPropEleven = x.OtherEntity.MyOtherPropEleven,
MyPropTwelve = x.MyOtherpropTwelve
});
}
Here the mapping is still compiled down into an Expression that the query provider can parse. Now you can do:
var returnData = (from x in MyEntities.MyDBSet
where x.MyDBSetPrimaryKey == id
select x)
.ToModelDTO();

Your problem and possible solutions are not really any different than many of the other "LINQ to entities does not recognize the method" questions.
For examples, see https://stackoverflow.com/a/7259649/120955 and https://stackoverflow.com/a/18901609/120955

c# linq to sql two threads visit the same db, but one is very slow

I've created two threads in my application and the first one is the main UI thread, which will do the normal job like quering below
var order = db.Orders.Where(x=>x.systemId==id).ToList();
the other thread is checking if there is any warning situation to handle, the code is as below:
TransactionOptions transOptions = new TransactionOptions()
{ IsolationLevel = System.Transactions.IsolationLevel.ReadUncommitted };
using (new TransactionScope(TransactionScopeOption.Required, transOptions))
{
BathDBDataContext dc = new BathDBDataContext(connectionString);
var all_menus = dc.Menu.Where(x => x.addAutomatic);
var menus = all_menus.Select(x => x.name).ToList();
var orders = dc.Orders.Where(x => menus.Contains(x.menu) && !x.paid && x.deleteEmployee == null);
var add_menus = orders.Select(x => x.menu).Distinct();
var ids = orders.Select(x => x.systemId).Distinct();
foreach (var systemId in ids)
{
var seat_orders = orders.Where(x => x.systemId == systemId);
foreach (var add_menu in add_menus)
{
var add_orders = seat_orders.Where(x => x.menu == add_menu && (x.priceType == null || x.priceType != "stop counting"));
if (add_orders.Count() == 0)
continue;
var max_time = add_orders.Max(x => x.inputTime);
var max_order = add_orders.OrderByDescending(x => x.inputTime).FirstOrDefault();
//var max_order = add_orders.FirstOrDefault(x => x.inputTime == max_time);
if (max_order == null || max_order.priceType == "per hour")
continue;
var the_menu = all_menus.FirstOrDefault(x => x.name == add_menu);
string menu_time = the_menu.timeLimitHour.ToString() +
":" + the_menu.timeLimitMiniute.ToString() +
":" + the_menu.timeLimitSecond.ToString();
TimeSpan tsm = TimeSpan.Parse(menu_time);
if (DateTime.Now - max_order.inputTime < tsm)
continue;
if (the_menu.addType == "by unit")
{
Orders new_order = new Orders();
new_order.menu = max_order.menu;
new_order.text = max_order.text;
new_order.systemId = systemId;
new_order.number = 1;
new_order.money = the_menu.price;
new_order.technician = max_order.technician;
new_order.techType = max_order.techType;
new_order.inputTime = DateTime.Now;
new_order.inputEmployee = "computer";
new_order.paid = false;
dc.Orders.InsertOnSubmit(new_order);
}
else if (the_menu.addType == "by time")
{
Orders new_order = new Orders();
new_order.menu = max_order.menu;
new_order.text = max_order.text;
new_order.systemId = systemId;
new_order.number = 1;
new_order.priceType = "per hour";
new_order.money = Convert.ToDouble(the_menu.addMoney);
new_order.technician = max_order.technician;
new_order.techType = max_order.techType;
new_order.inputTime = DateTime.Now;
new_order.inputEmployee = "computer";
new_order.paid = false;
dc.Orders.InsertOnSubmit(new_order);
}
}
dc.SubmitChanges();
}
in this situation, the query in the main UI thread will be very slow or sometimes doesn't even work.
can anyone help me with that?

Correct me if I am wrong but you seem to insert values to Orders in the loop and also trying to use an expression in your main UI.
Besides 5 seconds interval might be short since your LINQ expressions get more complex so that time needed for the calculations increases.
Loading that "BathDBDataContext" might take more time as your database gets new records.
I'd suggest using Cache and increase 5 seconds refresh timer.

finding a member property of an item in an array

I've passed in the whole sale object into this function. In that sale object, there are many childs e.g. saleOrderItem, Payment, User...
sale.payment is an array
Question:
1) How do I get the CodeNo from the `sale.payment' object? (as payment is an array)
2) How to get the sale.payment.CourseByTutor.TutorId (as payment is an array)
public static object GetSalesDataBySaleOrderID(SaleOrder sale)
{
return sale.saleOrderItem
.Where(s => s != null)
.Select(s => new {
Id = s.Id,
Username = sale.User.GetSingleUserById(s.UserId).Name,
Amount = s.Amount,
CodeNo = s.SaleOrder.payment.First().CodeNo,
TutorName = sale.User.GetSingleUserById(s.SaleOrder.payment.FirstOrDefault().CourseByTutor.TutorId).Name,
})
.ToList();
}
Here is how i bind the value to the object
private void SaveValueToObject()
{
saleOrder.UserId = UserInfo.Id;
saleOrder.Date = DateTime.Now;
for (int i = 0; i < dgSale.Rows.Count; i++)
{
SaleOrderItem SaleItem = new SaleOrderItem();
SaleItem.InventoryTypeId = Convert.ToInt32(dgSale.Rows[i].Cells["inventoryTypeId"].Value);
SaleItem.InventoryOrCourseId = Convert.ToInt32(dgSale.Rows[i].Cells["inventoryId"].Value);
SaleItem.Qty = Convert.ToInt32(dgSale.Rows[i].Cells["qty"].Value);
SaleItem.Amount = Convert.ToDecimal(dgSale.Rows[i].Cells["total"].Value);
SaleItem.UserId = Convert.ToInt32(dgSale.Rows[i].Cells["userId"].Value);
SaleItem.Discount = Convert.ToDecimal(dgSale.Rows[i].Cells["discount"].Value);
SaleItem.Remarks = (dgSale.Rows[i].Cells["remark"].Value == null) ? "" : dgSale.Rows[i].Cells["remark"].Value.ToString();
SaleItem.Status = (int)SaleOrderStatus.Active;
saleOrder.saleOrderItem[i] = SaleItem;
Payment p = new Payment();
p.PayType = Convert.ToInt32(dgSale.Rows[i].Cells["payType"].Value);
p.Code = (int)PaymentCode.RCT; // For Receipt prefix
p.CodeNo = p.GetNewReceiptNo(p.Code); //Check if it is receipt, if yes, Get last Receipt No + 1
p.UserId = (txtUserId.Text == string.Empty) ? 0 : Convert.ToInt32(txtUserId.Text);
p.Amount = Convert.ToDecimal(dgSale.Rows[i].Cells["total"].Value);
p.Remark = (dgSale.Rows[i].Cells["remark"].Value == null) ? "" : dgSale.Rows[i].Cells["remark"].Value.ToString();
p.PaymentMethodId = saleOrder.PaymentMethodId;
p.DateIssue = saleOrder.Date;
p.CourseByTutor.TutorId = Convert.ToInt32(dgSale.Rows[i].Cells["tutorId"].Value);
saleOrder.payment[i] = p;
}
}

I have a workaround and solved this question by adding the desired properties e.g. TutorId in the sale.saleOrderItem class. Thus making the direct access to the sale.SaleOrderItem.TutorId.
This sound stupid but it works.