I have a query that returns a resultset. And I want to apply filter and sorting on the resultset.
Can someone help me understand if I use the query in LINQ (I'm using EF 4.0), will I be able to get deferred executuin so that when i apply filter/sort in entity model, the execution happens only one time (deffered)
Thanks in advance!
Regards,
Bhavik
If the query takes no parameters, then yes, as you could make a view that calls that sproc, expose the view in your model, then query it.
If it takes parameters, then if you need the sort/filter done server-side, then I think you'd have to add a wrapper sproc (or modify the existing one) to pass in the sort and filter to perform (basically, do it manually, but at least server-side).
Alternatively, you could write the sql to do it server side (sproc results into temp table, then select from that temp table and apply filtering, still manually) and then ExecuteStoreQuery
No, you can't defer the execution of a linq filtering to the stored proc in sql. The stored proc will be executed first, a resultset will be returned, you can then cast it to a list of your object types, once done that you can filter using Linq.
You can easily cast the resultset to a list of your objects using context.Translate<>
Have a look to these links :
enter link description here
List item
Of course the query (in your code) will not be evaluated until you cast it to a list, so you can concatenate all the filtering you want to your resultset and then call the ToList() to get the results.
Related
I am new to Dapper though I am aware about ORMs and DAL and have implemented DAL with NHibernate earlier.
Example Query: -
string sql = "SELECT * FROM MyTable";
public int GetCount()
{
var result = Connection.Query<MyTablePoco>(sql).Count();
return result;
}
Will Dapper convert this query (internally) to SELECT COUNT(*) FROM MyTable looking at .Count() at the end?
Similarly, will it convert to SELECT TOP 1 * FROM MyTable in case of SingleOrDefault()?
I came from NHibernate world where it generates query accordingly. I am not sure about Dapper though. As I am working with MS Access, I do not see a way to check the query generated.
No, dapper will not adjust your query. The immediate way to tell this is simply: does the method return IEnumerable... vs IQueryable...? If it is the first, then it can only use local in-memory mechanisms.
Specifically, by default, Query will actually return a fully populated List<>. LINQ's Count() method recognises that and just accesses the .Count property of the list. So all the data is fetched from the database.
If you want to ask the database for the count, ask the database for the count.
As for mechanisms to view what is actually sent to the database: we use mini-profiler for this. It works great.
Note: when you are querying exactly one row: QueryFirstOrDefault (and the other variants you would expect) exist and have optimizations internally (including hints to ADO.NET, although not all providers can act on those things) to do things as efficiently as possible, but it does not adjust your query. In some cases the provider itself (not dapper) can help, but ultimately: if you only want the first row, ask the database for the first row (TOP or similar).
I use the PreprocessQuery method to extend a query in lightswitch.
Something like this:
query = (from item in query
where (validIDs.Contains(item.tableIDs.myID)) &&
elementCount[item.ID] <= maxEleCount)
select item);
Where validIDs is a HashSet(int) and elementCount is a Dictionary(int, int).
the first where clause is working fine, but the second -> elementCount[item.ID] <= maxEleCount
is not working.
What i want to do is to filter a table by some IDs (validIDs) and check also if in another table the number of entries for every of this IDs does not exceed a limit.
Any ideas?
EDIT
I found a solution. Instead of a Dictionary I also used a HashSet for the second where clause. It seems it is not possible to do the Dictionary lookup inside the LINQ statement for some reason (?)
First, although being a bit pedantic, what you're doing in a PreProcessQuery method is "restricting" records in the query, not "extending" the query.
What you put in a LING query has to be able to be processed by the Entity Framework data provider (in the case of LS, the SQL Server Data Provider).
Sometimes you'll find that while your LINQ query compiles, it fails at runtime. This is because the data provider is unable to express it to the data store (again in this case SQL Server).
You're normally restricted to "primitive" values, so if you hadn't said that using a Dictionary actually worked, I would have said that it wouldn't.
Any time you have a static (as in non-changing) value, I'd suggest that you create a variable outside of your LINQ query, then use the variable in the LINQ query. By doing this, you're simply passing a value, the data provider doesn't have to try to figure out how to pass it to the data store.
Reading your code again, this might not be what you're doing, but hopefully this explanation will still be helpful.
I have currently in a table about 90k rows. And it's will grow up about 1kk ~ 5kk before i execute a clean up and put all rows in a "historical table". So, when i run this following query (MyEntities is a ObjectSet):
MyEntities.Skip(amount * page).Take(amount).ToList();
This query takes about 1.2s... but when i run this following query with OrderBy and ThenBy:
MyEntities.OrderBy(b => b.Day).ThenBy(b => b.InitialHour).Skip(amount * page).Take(amount).ToList();
This query takes about 5.7s. There is a way to optimize the second query?
A few suggestions:
Check that it really is happening in the database (instead of fetching all entities, then sorting)
Make sure that both Day and InitialHour are indexed.
Check the generated SQL isn't doing anything crazy (check the query plan)
EDIT: Okay, so it looks like MyEntities is actually declared as IEnumerable<MyEntity>, which means everything will be done in-process... all your LINQ calls will be via Enumerable.Select etc, rather than Queryable.Select etc. Just change the declared type of MyEntities to IQueryable<MyEntity> and watch it fly...
For reading data from your DB, it's usually a good idea to create custom SQL Views, say one View per grid and one View per Form that you want to populate.
In this example, you would create a view that does the sorting for you, then map that View to an Entity in Entity Framework, then query that Entity using LINQ.
This is nice, clean, readable, maintainable and as optimal as you can make it.
Good luck!
I have an xml file which contains query details like
<queries>
<query>
<name>GetStudentById</name>
<statement>select student_name from student where student_id=#id</statement>
</query>
</queries>
In C# code, the user will call a method ExecuteQuery(string queryname, hashtable params)
I will retrieve the query statement from the xml file using the query name.
The params hash table will contain values like
params['#id']=5
I will then use ADO .NET to construct the query and execute it at runtime using ExecuteReader after constructing the query statement and passing it the parameter list.
I want to know if it is possible to do it by using LinQ. Can someone show me how to execute the statement as a LinQ query?
Thanks,
David
Have a look at this SO post. Look at the answer from john skeet where he explains how to use the CSharpCodeProvider to execute the statement at runtime.
You will however have to convert your statement from sql to linq yourself. If there is no particular reason for it having to be linq, I would suggest keeping it in sql because at the end of the day you are going to parse sql to linq and then back to sql.
Even with dynamic linq you will have a problem because you will always have to specify the from class in your query.
Another option is that if you want to use this with your entity framework model, it would be way easier to use entity sql, have a look here and here for more info.
It's a considerable leap to go from using string query statements via ADO.NET to Linq to SQL. In the first instance you need to build your Linq to SQL entity model/DataContext. How you go about parsing a sql statement from text to Linq I'm not entirely sure, but if I wanted to look into how to achieve this I would be reading up on expression trees with a view to building a parser that takes your string query and referers to your entity model in order to produce a linq query.
Found something called LinqTextQueryBuilder at codeplex and another MS Library.
Links below
LinqTextQueryBuilder
http://msdn.microsoft.com/en-us/vstudio/bb894665.aspx
How can I return first 100 records using Linq?
I have a table with 40million records.
This code works, but it's slow, because will return all values before filter:
var values = (from e in dataContext.table_sample
where e.x == 1
select e)
.Take(100);
Is there a way to return filtered? Like T-SQL TOP clause?
No, that doesn't return all the values before filtering. The Take(100) will end up being part of the SQL sent up - quite possibly using TOP.
Of course, it makes more sense to do that when you've specified an orderby clause.
LINQ doesn't execute the query when it reaches the end of your query expression. It only sends up any SQL when either you call an aggregation operator (e.g. Count or Any) or you start iterating through the results. Even calling Take doesn't actually execute the query - you might want to put more filtering on it afterwards, for instance, which could end up being part of the query.
When you start iterating over the results (typically with foreach) - that's when the SQL will actually be sent to the database.
(I think your where clause is a bit broken, by the way. If you've got problems with your real code it would help to see code as close to reality as possible.)
I don't think you are right about it returning all records before taking the top 100. I think Linq decides what the SQL string is going to be at the time the query is executed (aka Lazy Loading), and your database server will optimize it out.
Have you compared standard SQL query with your linq query? Which one is faster and how significant is the difference?
I do agree with above comments that your linq query is generally correct, but...
in your 'where' clause should probably be x==1 not x=1 (comparison instead of assignment)
'select e' will return all columns where you probably need only some of them - be more precise with select clause (type only required columns); 'select *' is a vaste of resources
make sure your database is well indexed and try to make use of indexed data
Anyway, 40milions records database is quite huge - do you need all that data all the time? Maybe some kind of partitioning can reduce it to the most commonly used records.
I agree with Jon Skeet, but just wanted to add:
The generated SQL will use TOP to implement Take().
If you're able to run SQL-Profiler and step through your code in debug mode, you will be able to see exactly what SQL is generated and when it gets executed. If you find the time to do this, you will learn a lot about what happens underneath.
There is also a DataContext.Log property that you can assign a TextWriter to view the SQL generated, for example:
dbContext.Log = Console.Out;
Another option is to experiment with LINQPad. LINQPad allows you to connect to your datasource and easily try different LINQ expressions. In the results panel, you can switch to see the SQL generated the LINQ expression.
I'm going to go out on a limb and guess that you don't have an index on the column used in your where clause. If that's the case then it's undoubtedly doing a table scan when the query is materialized and that's why it's taking so long.