Entity Framework 5+ is supposed to precompile all queries. However, for queries such as
List<Guid> ids;
var entities = context.MyEntities.Where(x => ids.Contains(x.Id)).ToArray();
Entity Framework cannot precompile the query, and depending on the complexity of the overall query, the parsing of the expression tree to SQL can consume several seconds. Has anyone found a workaround to get a precompiled query anyway? I do not really understand why it would be so hard; of course it is difficult to do with paramters, since the number of elements can differ, but it would be good enough to have SQL like
SELECT a, b, c from MyEntities
WHERE c in __PLACEHOLDER__
and then to substitute the placeholder with the actual list elements. Of course, it is not as nice as passing parameters, but it would be by far better than waiting for seconds for parsing the entire expression tree over and over.
You have to first understand how "IN" operator works in parameterized SQL query.
SELECT A FOM B WHERE C IN #p
does not work, SQL command parameter does not accept ARRAY as a parameter value, instead the query is translated to
SELECT A FROM B WHERE C IN (#p1, #p2, #p3 ... etc)
This query has variable number of parameters and this the reason, there is no way to precompile this query with IEnumerable.Contains.
The only other alternative (long long way) is to use Xml or Json (Coming up in Sql 2016).
Save your IEnumerable as xml.
[10,20,20,50] can be translated to
<data>
<int value="10"/>
<int value="20"/>
<int value="20"/>
<int value="50"/>
</data>
And you can then define a VIEW with parameters as
SELECT A FROM B WHERE C IN (SELECT INT FROM Xml(#P1))
And you can use this View, however there are more challenges in EF to how to fire this query, but this query can be precompiled as it has only one parameter.
Custom SQL for Performance Hack
For pretty simple query like,
List<Guid> ids;
var entities = context.MyEntities.Where(x => ids.Contains(x.Id)).ToArray();
I could simply use a custom SQL and fire,
var parameterList = ids.Select(
(x,i)=> new SqlCommandParameter(
"#p"+i, x));
var pnames = String.Join(",", parameterList.Select(x=> x.ParameterName));
var entities =
context.SqlQuery<MyEntity>(
"SELECT * FROM TABLE WHERE Id in (" + pnames + ")",
parameterList.ToArray());
Temporary Table
You can also use a temporary table, but this increases number of active transactions in your database.
Guid sid = Guid.NewGuid();
foreach(var p in ids){
db.TempIDs.Add(new TempID{ SID = sid, Value = p });
}
db.SaveChanges();
var qIDs = db.TempIDs.Where( x=> x.SID == sid );
var myEntities db.MyEntities.Where( x => qIDs.Any( q.Value == x.Id) );
// delete all TempIDs...
db.SqlQuery("DELETE FROM TempIDs WHERE SID=#sid,
new SqlCommandParameter("#sid", sid));
Related
I tried the Internet and the SOF but couldn't locate a helpful resource. Perhaps I may not be using correct wording to search. If there are any previous questions I have missed due to this reason please let me know and I will take this question down.
I am dealing with a busy database so I am required to send less queries to the database.
If I access different columns of the same Linq query from different levels of the code then is Entity Framework smart enough to foresee the required columns and bring them all or does it call the db twice?
eg.
var query = from t1 in table_1
join t2 in table_2 on t1.col1 equals t2.col1
where t1.EmployeeId == EmployeeId
group new { t1, t2 } by t1.col2 into grouped
orderby grouped.Count() descending
select new { Column1 = grouped.Key, Column2 = grouped.Sum(g=>g.t2.col4) };
var records = query.Take(10);
// point x
var x = records.Select(a => a.Column1).ToArray();
var y = records.Select(a => a.Column2).ToArray();
Does EF generate query the database twice to faciliate x and y (send a query first to get Column1, and then send another to get Column2) or is it smart enough to know it needs both Columns to be materialised and bring them both at point x?
Added to clarify the intention of the question:
I understand I can simply add a greedy method to the end of query.Take(10) and get it done but I am trying to understand if the approach I try (and in my opinion, more elegant) does work of if not what makes EF to make two queries please.
Yes currently your code will generate 2 queries that will be executed to the database. Reason being is because you have 2 different sqls generated:
First is the top query, taking only 10 records and then only Column1
Second is the top query, taking only 10 records and then only Column2
The reason these are 2 queries is because you have a ToArray over different Select statements -> generating different sql. Most of linq queries are differed executed and will be executed only when you use something like ToArray()/ToList()/FirstOrDefault() and so on - those that actually give you the concrete data. In your original query you have 2 different ToArray on data that has not yet been retrieved - meaning 2 queries (once for the first field and then for the second).
The following code will result in a single query to the database
var records = (from t1 in table_1
join t2 in table_2 on t1.col1 equals t2.col1
where t1.EmployeeId == EmployeeId
group new { t1, t2 } by t1.col2 into grouped
orderby grouped.Count() descending
select new { Column1 = grouped.Key, Column2 = grouped.Sum(g=>g.t2.col4) })
.Take(10).ToList();
var x = records.Select(a => a.Column1).ToArray();
var y = records.Select(a => a.Column2).ToArray();
In my solution above I added a ToList() after filtering out only that data you need (Take(10)) and then at that point it will execute to the database. Then you have all the data in memory and you can do any other linq operation over it without it going again to the database.
Add to your code ToString() so you can check the generated sql at different points. Then you will understand when and what is being executed:
var query = from t1 in table_1
join t2 in table_2 on t1.col1 equals t2.col1
where t1.EmployeeId == EmployeeId
group new { t1, t2 } by t1.col2 into grouped
orderby grouped.Count() descending
select new { Column1 = grouped.Key, Column2 = grouped.Sum(g=>g.t2.col4) };
var generatedSql = query.ToString(); // Here you will see a query that brings all records
var records = query.Take(10);
generatedSql = query.ToString(); // Here you will see it taking only 10 records
// point x
var xQuery = records.Select(a => a.Column1);
generatedSql = xQuery.ToString(); // Here you will see only 1 column in query
// Still nothing has been executed to DB at this point
var x = xQuery.ToArray(); // And that is what will be executed here
// Now you are before second execution
var yQuery = records.Select(a => a.Column2);
generatedSql = yQuery.ToString(); // Here you will see only the second column in query
// Finally, second execution, now with the other column
var y = yQuery.ToArray();
When you are running linq statement on an entity in EF if only prepares the Select statement (thats why the type is IQueryable). The data is loaded lazily. When you try to use a value from that query then only the result gets evaluated using a enumerator.
So when you turn it to a collection (.toList() etc.) explicitly it tries to get data to populate the list and hence the sql command is fired.
It is designed so to enhance the performance. So if a particular property of an entity is to be used EF doesn't get the value for all the columns from that table
If I have a summary class of some objects I need ( for example its primary key, etc..) is there a way to use a list of that class object when I am writing a joining to other tables? so all the things in my LINQ query are real table like this.Context.MyTable but one of them be my List<MyClass> ?
or is there any LINQ related Nuget project that makes this possible?
The EF LINQ queries aren't actually code that is run in C#, they are converted to SQL and run on the database server; so you can't join them with an in-memory array (or List<T>). What you can do, is use Contains like so:
public IEnumerable<Table1> GetThingsFromDatabse(DataContext db, IList<MyObject> objects)
{
var ids = objects.Select(x => x.Id).ToList();
var results = Enumerable.ToList(
from x in db.Table1s
where ids.Contains(x.Id)
select x
);
return results;
}
This gets translated into SQL that looks like this:
SELECT *
FROM [Table1] x
WHERE x.Id IN (#Id1, #Id2, ...)
I have the following linq queries:
Var q1 = (from t1 in context.Table1
where column = value
select t1).FirstOrDefault();
Var q2 = (from t2 in context.Table2
where column = value
select t2).FirstOrDefault();
As far as I understand, the above linq statements will call the database two times to get the table data but I want to write the linq query in such a way to get both the tables data in a single database call. How can I achieve this?
You can achieve this by selecting to anonymous type:
Var q = (from t1 in context.Table1
where t1.column == value
select t1
)
.Select(t1 => new {
t1 = t1,
t2 = context.Table2
.FirstOrDefault(t2 => t2.column == value);
})
.FirstOrDefault();
var t1 = q.t1;
var t2 = q.t2;
This way it will make one query from this all. I simplified a bit the query part to obtain t2 item, but there is not obstacle to use the one you wrote.
Short Answer: This is not possible
Although both queries hitting the same DB but they are working on different tables,
think about it as you are creating normal SQL statement, you will not be able to combine this tow queries in only one
Correct.
You can however use Entity Framework.Extended to do bulk queries though if you don't mind adding an extension.
See: github Link
It is possible to load related entities using Include() method of EF.
// Load all blogs, all related posts, and all related comments
var blogs1 = context.Blogs
.Include(b => b.Posts.Select(p => p.Comments))
.ToList();
i have the following query running 10-20 times / page in my project. i have tried to run this query with linq to sql, linq to entities but this is far more faster then them.
The question is if i could pass in the external list(sContentIds) into query with a join statement, would it make the query faster then using SQL IN statement? If so how can i achieve this. sContentIds.Count may vary from 1-40 most of the times.
List<filterContentsPCDTO> cContents = unitOfWork.ExecuteQuery<filterContentsPCDTO>(#"SELECT c.ContentId, c.ContentPageId, c.CreatedById, p.PCA, p.PCC, p.PCD, c.AlbumId, a.AlbumTypeId
FROM Contents c
INNER JOIN Privatizations p ON c.ContentId = p.ContentId
LEFT JOIN Albums a ON c.AlbumId = a.AlbumId
WHERE c.ContentId IN (" + string.Join(",", sContentIds) + ")").ToList();
We are working on ASP.NET MVC4 framework and using unit of work pattern for database interactions. Normally i had built this query like follows but it was 5 times slower then raw sql query.
var cContents = unitOfWork.ContentRepository
.GetFiltered(x => contentIds.Contains(x.ContentId)).Select(x => new filterContentsPCDTO()
{
ContentId = x.ContentId,
ContentPageId = x.ContentPageId,
CreatedById = x.CreatedById,
PCA = x.Privatization.PCA,
PCC = x.Privatization.PCC,
PCD = x.Privatization.PCD,
PrivatizationModifiedById = x.Privatization.ModifiedById,
AlbumId = x.AlbumId,
albumTypeId = x.AlbumId == null ? -1 : x.Album.AlbumTypeId
}).ToList();
Implementation of GetFiltered Method
public IEnumerable<T> GetFiltered(
Expression<Func<T, bool>> filter = null,
Func<IQueryable<T>, IOrderedQueryable<T>> orderBy = null,
string includeProperties = "")
{
IQueryable<T> query = _dbSet;
if (filter != null)
{
query = query.Where(filter);
}
foreach (var includeProperty in includeProperties.Split
(new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
{
query = query.Include(includeProperty);
}
if (orderBy != null)
{
return orderBy(query);
}
else
{
return query;
}
}
If you're using SQL Server 2008 (or newer) and increasing performance is the main objective here (and you're maybe willing to abandon LINQ to SQL for this scenario), I would recommend writing this query as a stored procedure that takes a user-defined table type as a parameter. This will allow you to pass your entire sContentIds collection to the database and still benefit from advantages of a stored procedures over an ad-hoc query.
First, define the table type as something like:
CREATE TYPE [dbo].[ContentList] AS TABLE(
[ContentId] [int]
)
Then create the procedure as something like:
CREATE PROCEDURE [dbo].[usp_GetContents]
#contentIds ContentList READONLY
AS
SELECT c.ContentId
,c.ContentPageId
,c.CreatedById
,p.PCA
,p.PCC
,p.PCD
,c.AlbumId
, a.AlbumTypeId
FROM Contents c
INNER JOIN Privatizations p
ON c.ContentId = p.ContentId
LEFT JOIN Albums a
ON c.AlbumId = a.AlbumId
WHERE c.ContentId IN (SELECT ContentId FROM #contentIds)
Then you should be able to call it from C# using the technique described in this answer (basically, create a DataTable from your list then add it like a regular parameter). Unfortunately it looks like this is tough to do with LINQ to SQL, but, as I said, if increasing performance is the main goal, this could be an option.
I would like to query a DataTable that produces a DataTable that requires a subquery. I am having trouble finding an appropriate example.
This is the subquery in SQL that I would like to create:
SELECT *
FROM SectionDataTable
WHERE SectionDataTable.CourseID = (SELECT SectionDataTable.CourseID
FROM SectionDataTable
WHERE SectionDataTable.SectionID = iSectionID)
I have the SectionID, iSectionID and I would like to return all of the records in the Section table that has the CourseID of the iSectionID.
I can do this using 2 separate queries as shown below, but I think a subquery would be better.
string tstrFilter = createEqualFilterExpression("SectionID", strCriteria);
tdtFiltered = TableInfo.Select(tstrFilter).CopyToDataTable();
iSelectedCourseID = tdtFiltered.AsEnumerable().Select(id => id.Field<int>("CourseID")).FirstOrDefault();
tdtFiltered.Clear();
tstrFilter = createEqualFilterExpression("CourseID", iSelectedCourseID.ToString());
tdtFiltered = TableInfo.Select(tstrFilter).CopyToDataTable();
Although it doesn't answer your question directly, what you are trying to do is much better suited for an inner join:
SELECT *
FROM SectionDataTable S1
INNER JOIN SectionDataTable S2 ON S1.CourseID = S2.CourseID
WHERE S2.SectionID = iSectionID
This then could be modeled very similarily using linq:
var query = from s1 in SectionDataTable
join s2 in SectionDataTable
on s1.CourseID equals s2.CourseID
where s2.SectionID == iSectionID
select s1;
When working in LINQ you have to think of the things a bit differently. Though you can go as per the Miky's suggestion. But personally I would prefer to use the Navigational properties.
For example in your given example I can understand that you have at-least 2 tables,
Course Master
Section Master
One Section must contain a Course reference
Which means
One Course can be in multiple Sections
Now if I see these tables as entities in my model I would see navigational properties as,
Course.Sections //<- Sections is actually a collection
Section.Course //<- Course is an object
So the same query can be written as,
var lstSections = context.Sections.Where(s => s.Course.Sections.Any(c => c.SectionID == iSectionID)).ToList();
I think you main goal is, you are trying extract all the Sections where Courses are same as given Section's Courses.