I guess it should be really simple, but i cannot find how to do it.
I have a linq query, that selects one column, of type int, and i need it sorted.
var values = (from p in context.Products
where p.LockedSince == null
select Convert.ToInt32(p.SearchColumn3)).Distinct();
values = values.OrderBy(x => x);
SearchColumn3 is op type string, but i only contains integers. So i thought, converting to Int32 and ordering would definitely give me a nice 1,2,3 sorted list of values. But instead, the list stays ordered like it were strings.
199 20 201
Update:
I've done some tests with C# code and LinqPad.
LinqPad generates the following SQL:
SELECT [t2].[value]
FROM (
SELECT DISTINCT [t1].[value]
FROM (
SELECT CONVERT(Int,[t0].[SearchColumn3]) AS [value], [t0].[LockedSince], [t0].[SearchColumn3]
FROM [Product] AS [t0]
) AS [t1]
WHERE ([t1].[LockedSince] IS NULL)
) AS [t2]
ORDER BY [t2].[value]
And my SQL profiler says that my C# code generates this piece of SQL:
SELECT DISTINCT a.[SearchColumn3] AS COL1
FROM [Product] a
WHERE a.[LockedSince] IS NULL
ORDER BY a.[SearchColumn3]
So it look like C# Linq code just omits the Convert.ToInt32.
Can anyone say something useful about this?
[Disclaimer - I work at Telerik]
You can solve this problem with Telerik OpenAccess ORM too. Here is what i would suggest in this case.
var values = (from p in context.Products
where p.LockedSince == null
orderby "cast({0} as integer)".SQL<int>(p.SearchColumn3)
select "cast({0} as integer)".SQL<int>(p.SearchColumn3)).ToList().Distinct();
OpenAccess provides the SQL extension method, which gives you the ability to add some specific sql code to the generated sql statement.
We have started working on improving this behavior.
Thank you for pointing this out.
Regards
Ralph
Same answer as one my other questions, it turns out that the Linq provider i'm using, the one that comes with Telerik OpenAccess ORM does things different than the standard Linq to SQL provider! See the SQL i've posted in my opening post! I totally wasn't expecting something like this, but i seem that the Telerik OpenAccess thing still needs a lot of improvement. So be careful before you start using it. It looks nice, but it has some serious shortcomings.
I can't replicate this problem. But just make sure you're enumerating the collection when you inspect it. How are you checking the result?
values = values.OrderBy(x => x);
foreach (var v in values)
{
Console.WriteLine(v.ToString());
}
Remember, this won't change the order of the records in the database or anywhere else - only the order that you can retrieve them from the values enumeration.
Because your values variable is a result of a Linq expression, so that it doest not really have values until you calling a method such as ToList, ToArray, etc.
Get back to your example, the variable x in OrderBy method, will be treated as p.SearchColumn3 and therefore, it's a string.
To avoid that, you need to let p.SearchColumn3 become integer before OrderBy method.
You should add a let statement in to your code as below:
var values = (from p in context.Products
where p.LockedSince == null
let val = Convert.ToInt32(p.SearchColumn3)
select val).Distinct();
values = values.OrderBy(x => x);
In addition, you can combine order by statement with the first, it will be fine.
Related
var ids = IdsList.Select(pID => pID.ID).ToArray();
var response= await MyService.GetByMyQuery(
new QueryDefinition(
"SELECT * FROM p WHERE p.id NOT IN(#ids)"
)
.WithParameter("#ids", string.Join(",", ids)),
);
So this is not working. The operator returns all the items, instead of just the ones not in the list. In the cosmos DB SQL query editor i can easily do
SELECT * FROM p WHERE p.id NOT IN("id1","id2")
and it returns the expected results without any problems. SO i guess that the problem is in the code layer, the way I'm passing the ids to the WithParameter() menthod.
Any insights is greatly appreciated.
The problem
Your C# code is not sending multiple values as the #ids parameter, but a single string value, effectively like this:
SELECT * FROM p
WHERE p.id NOT IN("id1, id2")
Since this compound id does not exist then it returns all items like you observed.
Solution
It may be possible with IN keyword as well, but I do know for sure that this pattern will work:
SELECT * FROM p
WHERE NOT ARRAY_CONTAINS(#ids, p.id)
NB! Correct me if I'm mistaken, but most likely this condition will NOT be served by index. So, you may want to reconsider your design, unless your real case would have an additional good indexable predicate, or else it will be slow and costly.
I am still in the midst of converting our web app from vb to C# and this complex query, that I just finished converting and have tested and it works properly, needs refactoring.
This section could probably perform a little better if lambdas were used (I've read in numerous places that lambda expressions perform faster)
foreach (var datapersymbol in (from symId in symblist
select (
from q in ctx.HistoricalPriceData.Where(q => q.SymbolId == symId)
join x in ctx.Symbols on q.SymbolId equals x.SymbolId
orderby q.Date descending
select new {q.Date, q.LastPrice, x.Symbol1})).ToList())
As you can see I've added lambdas where I thought appropriate but Im certain they can be in other parts of this expressions as well. I'm still getting up to speed in C# so Im not an expert here.
Assuming you have a navigation property on HistoricalPriceData to Symbols called "Symbol":
foreach(var datapersymbol in ctx
.HistoricalPriceData
.Where(h=>symblist.Contains(h.SymbolId))
.OrderByDescending(h=>h.Date)
.Select(h=>new {h.Date,h.LastPrice,h.Symbol.Symbol1}))
As Rahul pointed out, there is no performance difference between lamda/method syntax and query syntax, however, this will perform slightly better because it drops the .ToList(), which will allow the loop to begin processing as soon as it gets the first record from the database rather than waiting for the entire result (and it doesn't need to create a List). I've also simplified it somewhat by using .Contains instead of a subquery as well, and switched to using navigation properties rather than a explicit join which makes it easier to read/more maintainable.
For the clarity of the code I will rather refactor it like below;
var datapersymbols = from symId in symblist
select (
from q in ctx.HistoricalPriceData.Where(q => q.SymbolId == symId)
join x in ctx.Symbols on q.SymbolId equals x.SymbolId
orderby q.Date descending
select new {q.Date, q.LastPrice, x.Symbol1});
foreach (var datapersymbol in datapersymbols)
{
}
As far as I can see, in this case, there is not much that you can gain from the C# side. You have to check the query plan for the SQL query and then add necessary indexes. The SymbolId field should be an indexed field.
I am currently working on a project leveraging EF and I am wondering if there is a more efficient or cleaner way to handle what I have below.
In SQL Server I could get the data I want by doing something like this:
SELECT tbl2.* FROM
dbo.Table1 tbl
INNER JOIN dbo.Table2 tbl2 ON tbl.Column = tbls2.Colunm
WHERE tbl.Column2 IS NULL
UNION
SELECT * FROM
dbo.Table2
WHERE Column2 = value
Very straight forward. However in LINQ I have something that looks like this:
var results1 = Repository.Select<Table>()
.Include(t => t.Table2)
.Where(t => t.Column == null);
var table2Results = results1.Select(t => t.Table2);
var results2 = Repository.Select<Table2>().Where(t => t.Column2 == "VALUE");
table2Results = table2Results.Concat(results2);
return results2.ToList();
First and foremost the return type of the method that contains this code is of type IEnumerable< Table2 > so first I get back all of the Table2 associations where a column in Table1 is null. I then have to select out my Table2 records so that I have a variable that is of type IEnumerable. The rest of the code is fairly straightforward in what it does.
This seems awfully chatty to me and, I think, there is a better way to do what I am trying to achieve. The produced SQL isn't terrible (I've omitted the column list for readability)
SELECT
[UnionAll1].*
FROM (SELECT
[Extent2].*
FROM [dbo].[Table1] AS [Extent1]
INNER JOIN [dbo].[Table2] AS [Extent2] ON [Extent1].[Column] = [Extent2].[Column]
WHERE [Extent1].[Column2] IS NULL
UNION ALL
SELECT
[Extent3].*
FROM [dbo].[Table2] AS [Extent3]
WHERE VALUE = [Extent3].[Column]) AS [UnionAll1]
So is there a cleaner / more efficient way to do what I have described? Thanks!
Well, one problem is that your results may not return the same data as your original SQL query. Union will select distinct values, Union All will select all values. First, I think your code could be made a lot clearer like so:
// Notice the lack of "Include". "Include" only states what should be returned
// *with* the original type, and is not necessary if you only need to select the
// individual property.
var firstResults = Repository.Select<Table>()
.Where(t => t.Column == null)
.Select(t => t.Table2);
var secondResults = Repository.Select<Table2>()
.Where(t => t.Column2 == "Value");
return firstResults.Union(secondResults);
If you know that it's impossible to have duplicates in this query, use Concat instead on the last line (which will produce the UNION ALL that you see in your current code) for reasons described in more detail here. If you want something similar to the original query, continue to use Union like in the example above.
It's important to remember that LINQ-to-Entities is not always going to be able to produce the SQL that you desire, since it has to handle so many cases in a generic fashion. The benefit of using EF is that it makes your code a lot more expressive, clearer, strongly typed, etc. so you should favor readability first. Then, if you actually see a performance problem when profiling, then you might want to consider alternate ways to query for the data. If you profile the two queries first, then you might not even care about the answer to this question.
This works in LINQ-to-SQL:
var customersTest = from c in db.Customers
select new
{
Id = c.Id,
Addresses = from a in db.Addresses where c.Id.ToString() ==
a.ReferenzId select a
};
foreach (var item in customersTest)
{
Console.WriteLine(item.Id);
}
But a similar example in Entity Framework gets an error message that says basically that it can't "translate it to SQL", here is the original error message in German:
"'LINQ to Entities' erkennt die
Methode 'System.String ToString()'
nicht, und diese Methode kann nicht in
einen Speicherausdruck übersetzt
werden."
Translation:
"'LINQ to Entities' does not recognize
Method 'System.String ToString()',
this method can not be translated into
a memory expression.
Can anyone shed any light on how we could get this kind of statement to work in Entity Framework or explain why it gets this error?
Simply put: LINQ to Entities doesn't know about the conversion from your ID type to a string.
What is the type of c.ID? Is there any reason why it's one type for ID, but another for ReferenzId? If at all possible, make them the same type, at which point you won't have a problem any more. I don't know if there are other ways of performing conversions in LINQ to Entities - there may be - but aligning the types would be cleaner.
By the way, this really looks like it's a join:
var query = from c in db.Customers
join a in db.Addresses on c.Id equals a.ReferenzId into addresses
select new { Id = c.Id, Addresses = addresses };
EDIT: To respond to your comment - ToString appears in IntelliSense because the compiler has no real idea what your query is going to mean or how it will be translated. It's perfectly valid C#, and can generate a valid expression tree - it's just that EF doesn't know how to convert that expression tree into SQL.
You could try using Convert.ToString(c.Id) instead of just calling c.Id.ToString()...
Updated answer:
If you followed the link I gave you at the beginning of my answer, this missing feature has meanwhile received 75 votes and is now (finally!) implemented by Microsoft in EF 6.1. To all who participated: Thank you for voting! Your voice was heard.
For example:
var query = from e in context.Employees where e.EmployeeID.ToString() == "1" select e;
will now be translated to:
DECLARE #p0 NVarChar(1000) = '1'
SELECT [t0].[EmployeeID], [t0].[LastName], [t0].[FirstName], [t0].[Title],
[t0].[TitleOfCourtesy], [t0].[BirthDate], [t0].[HireDate], [t0].[Address],[t0].[City],
[t0].[Region], [t0].[PostalCode], [t0].[Country], [t0].[HomePhone], [t0].[Extension],
[t0].[Photo], [t0].[Notes], [t0].[ReportsTo], [t0].[PhotoPath]
FROM [Employees] AS [t0]
WHERE (CONVERT(NVarChar,[t0].[EmployeeID])) = #p0
i.e. e.EmployeeID.ToString() translates to (CONVERT(NVarChar,[t0].[EmployeeID])).
Original answer:
It makes no sense to me why Linq2EF does not translate .ToString() into a proper SQL statement, as Linq2SQL does, only the Microsoft dev team knows the reason why they did not implement it yet. :-(
But you can raise the priority to implement it if you vote for this feature by following this link.
Luckily there are also 2 workarounds available, both of them I used recently in EF queries:
I) What helped me to get around this limitation was to change the query into a list, like so:
var customersList = (from c in db.Customers
select c).ToList(); // converts to IEnumerable<T> ...
var customersTest = (from c in customersList
select new {Id=c.ID.ToString()}); // ... which allows to use .ToString()
The statement .ToList() converts to IEnumerable<T>, where .ToString() is available. Note that, Depending on the requirements, you can use .AsEnumerable() as well, which has the advantage that deferred execution is supported which is better if you have multiple linq queries depending on each other or if you're using different parameter values (many thanks to Divega for this hint!).
Afterwards you can use this query as you wish , e.g.:
var customersTest2 = from c in customersTest
select new
{
Id = c.Id,
Addresses = from a in db.Addresses where c.Id == a.ReferenzId select a
};
Of course if you need you can add more properties to the objects of customersTest as required. You can also optimize the query above, I have only used 3 steps for readability of this example.
II) For simple conversions, and if you have to reuse the generated query in further subqueries (and it needs to remain IQueryable), use SqlFunctions from System.Data.Objects.SqlClient, they will be translated into SQL queries correctly.
Example 1: Date conversion (you have to use dateparts as below shown)
var customersTest = from c in db.Customers
select new {
strDate=SqlFunctions.DateName("dd", c.EndDate)
+"."+SqlFunctions.DateName("mm", c.EndDate)
+"."+SqlFunctions.DateName("yyyy", c.EndDate)
}
Example 2: Numeric to string conversion
var customersTest = from c in db.Customers
select new {
strID=SqlFunctions.StringConvert((double)c.ID)
}
This should help you out of most situations where conversions into strings are required.
Entity Framework 6.1 RTM which was just released now support .ToString()
LINQ to Entities as far as I understand it (for v1) is very primative. In otherwords it doesn't know how to take the extension method "ToString()" and generate the SQL for it.
In LINQ to SQL, it executes the extension method "ToString()" before generating the SQL. The difference is that LINQ to Entities uses IQueryable instead of IEnumerable.
BUT, from what I remember casting should work (because casting is a data type and SQL knows about CAST()).
So
c.Id.ToString() should really be (string)c.Id
(also, make sure it is (string) and not (String)).
One of the downfalls I would say about using Lambda (in Entity Framework) to generate the SQL expression instead of pure LINQ.
Keep in mind too, that using CAST on the left side of the equals sign in SQL is a bit ill performing :-)
I have some linq that returns the correct data.
var numEmails = (from row in EmailBatchProposal
where row.EmailBatchId == emailBatchId
select row.EmailBatchProposalId).Count();
However, if I understand linq correctly, this does not perform optimally. It grabs all the data and then walks through the list and counts the rows. What I'd really like is for linq (in the background) to use like:
Select count(*) from ...
I trust the performance reasons are obvious.
Does anyone know the proper way to do this?
Actually, if the linq query is used with a collection that implements IQueryable and supports translation into underlying SQL variant, it is quite a basic functionality to translate the Count function from your example correctly.
People generally learn best by practicing. I would suggest you get a copy of LinqPad (free), enter in your Linq query, and see what SQL it generates. Then you can modify the Linq query until you get exactly what you want.
Actually, the LINQ-to-SQL is smart enough to know that it should do a count... For example, if I have the following query:
var v = (from u in TblUsers
select u).Count();
the SQL that actually executes when I run this is:
SELECT COUNT(*) AS [value]
FROM [tblUsers] AS [t0]
Amazing, huh? Another poster made the really good suggestion of getting LinqPad - it is a WONDERFUL tool. It will show you the exact SQL that gets executed. You can always confirm with SQL profiler too.
Check the Log of the SQL used in the query.
using (var dbc = new siteDataContext())
{
dbc.Log = Console.Out;
var bd = (from b in dbc.birthdays
select b).Count();
Console.WriteLine("{0}", bd);
}
gives the query:
SELECT COUNT(*) AS [value]
FROM [dbo].[birthdays] AS [t0]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 3.5.30729.1
You can just use the argument of .Count().
int numEmails = EmailBatchProposal.Count(x => x.EmailBatchId == emailBatchId)
As noted below, this doesn't actually resolve to any different SQL, but I think it's at least a cleaner looking alternative.