So I was thinking about creating a dynamic sql question, meaning that i want the amount of parameters to be dynamic.
Having looked at this: Parameterize an SQL IN clause i was thinking that using like '%x%' is SLOW and not good.
What i actually would like to do is using the IN keyword and have it somewhat like this:
select UserId from NameTable where Name IN (#List_of_names)
Where the #List_of_names could contain i.e.
Filip Ekberg
Filip
Ekberg Filip
Ekberg
Johan Ekberg
Johan
( my second name is johan, thats why it's in there ,) )
So all these should match up with Johan Filip Ekberg.
I want to be using either LINQ to SQL or LINQ to SQL ( Stored Procedure ) using C#.
Suggestions and thoughts please!
----------------- Clearification of scenario and tables -------------------------
Imagine i have the following: A table with Users, A table with Names and a Table that connects the Names to a certain user.
So by having
[User Table]
User ID Full Name
1 Johan Filip Ekberg
2 Anders Ekberg
[Names]
Name ID Name
1 Filip
2 Ekberg
3 Johan
4 Anders
[Connect Names]
Name ID User ID
1 1
2 1
3 1
2 4
2 2
So if i want to look for: Ekberg
The return should be:
Johan Filip Ekberg
Anders Ekberg
If i want to search for Johan Ekberg
Johan Filip Ekberg
If i want to search for Anders
Anders Ekberg
The amount of "search names" can be infinite, if i have the name: Johan Anders Carl Filip Ekberg ( This is just 1 person with many names ) and i just pass "Carl Filip" in, i want to find that User.
Does this make everything clearer?
It looks like SQL 2008 is an option, so passing a table parameter is probably your best bet. Another possible solution for those who can't move to 2008 yet, is to write a table-valued UDF that generates a table from a delimited string. Then you can do something like this:
SELECT DISTINCT
CN.user_id
FROM
dbo.Names N
INNER JOIN dbo.Connect_Names CN ON CN.name_id = N.name_id
INNER JOIN dbo.GetTableFromNameList(#names) T ON T.name = N.name
This will give you user IDs where ANY of the passed names match ANY of the user's names. If you want to change it so that it only gives a match when ALL of the passed names match one of the user's names then you could do something like this:
SELECT
CN.user_id
FROM
dbo.Names N
INNER JOIN dbo.Connect_Names CN ON CN.name_id = N.name_id
INNER JOIN dbo.GetTableFromNameList(#names) T ON T.name = N.name
GROUP BY CN.user_id
HAVING COUNT(*) = (SELECT COUNT(*) FROM dbo.GetTableFromNameList(#names))
I haven't tested that second query, so you may need to fiddle with it. Also, you might want to create a local table and fill it with the function so that you don't have to run the function multiple times.
I can post my own version of the string to table function if you need it. Also, if you want to use LIKE rather than exact matches, you can do that.
I hear that SQL Server 2008 has a feature called Table Parameters, so you can pass a Table as a parameter to a Function or Stored Procedure.
Until then, you could use the classic:
SELECT UserId FROM NameTable WHERE CHARINDEX( '|' + Name + '|', '|Filip Ekberg|Filip|Ekberg Filip|') > 0
This means that column name can be any of the values you have in the list.
You can also pass an XML parameter into the stored procedure and then use it as a table in your code, via the OPENXML command.
You say you want to use IN ? IN looks for exact matches so if you use IN('Ekberg')
You will only find Ekberg and not Anders Ekberg.
If you want the above to find Anders Ekberg you need Like '%Ekberg'
In fact in your second example where you have name A B C and you pass A C you want it to find A B C.
Really you want a Like on every word you pass. There are other ways to achieve what it sounds like you require. Mind if I ask the database you are running against first of all ? :) Thanks
From the similar SQL-only question here: SQL query: Simulating an "AND" over several rows instead of sub-querying
Try this.
List<string> targets = new List<string>() {"Johan", "Ekberg"};
int targetCount = targets.Count;
//
List<Users> result =
dc.Names
.Where(n => targets.Contains(n.Name))
.SelectMany(n =>
dc.ConnectNames.Where(cn => cn.NameId == n.NameId)
)
.GroupBy(cn => cn.UserId)
.Where(g => g.Count() == targetCount)
.SelectMany(g =>
dc.Users.Where(u => u.UserId == g.Key)
)
.ToList();
This is freehand code, so there's probably some syntax errors in there.
Related
I came across the following in some code that I was refactoring today.
context.Entities.Where(x => x.ForeignKeyId == id)
.OrderBy(x => x.FirstSortField)
.OrderBy(x => x.SecondSortField);
Initially, I took out the .OrderBy(x => x.FirstSortField) thinking that the first OrderBy statement would just be replaced by the second OrderBy. After testing, I realized that it was generating the SQL as ORDER BY SecondSortField, FirstSortField.
Therefore, the equivalent is actually:
context.Entities.Where(x => x.ForeignKeyId == id)
.OrderBy(x => x.SecondSortField)
.ThenBy(x => x.FirstSortField);
Can anyone explain the reasoning behind EF6 doing this? It seems to me that it would be more intuitive to replace the first sort field with the second.
I can only conclude that we're actually looking at LINQ-to-SQL here. In Linqpad until v. 5, it's very easy to make this mistake, because the selection of a EF6 DbContext driver is easily overlooked when creating a new connection. (In Linqpad v6 this choice is more conspicuous).
I have tested the reported behavior in EF6, EF-core 3 and 5, and in LINQ-to-SQL. Only in the latter do I see a generated SQL statement with two columns in the ORDER BY.
The statement...
Products.OrderBy(p => p.Description).OrderBy(p => p.LastSale)
...is translated by LINQ-to-SQL as:
SELECT [t0].[ID], [t0].[Description], [t0].[Discontinued], [t0].[LastSale]
FROM [Product] AS [t0]
ORDER BY [t0].[LastSale], [t0].[Description]
The reasoning is explained in this answer, which boils down to: LastSale is the dominant ordering field as it, sort of, overrides the first OrderBy.
All EF queries only have ORDER BY LastSale.
I must say agree with EF's implementation. As this answer explains, the result of two consecutive orderings depends on the sorting algorithm. Which means that all we can say for sure is that the results from any LINQ query will be ordered by LastSale and the ordering within LastSale groups isn't certain. Then, IMO, it's a better choice for SQL translation to handle a second OrderBy statement as a full override of the first one so it's visible that no expectations can be based on the first statement. To me it's more intuitive.
The message is: be explicit when ordering by multiple fields. Use OrderBy - ThenBy. Don't rely on a database provider's handling of consecutive OrderBy statements.
This is all in terms of local data, but EF wants to do the logical equivalent when building the expression tree and writing your query.
You should research the concept of stable sorting. When you use a stable sorting algorithm, the original order of equal items is preserved.
So let's say you have data like this with obvious first name/last name fields:
Brad Jones
Tom Smith
Sam Jones
Jim Doe
James Smith
Ryan Smith
If you order initially by only by first name, you get this:
Brad Jones
James Smith
Jim Doe
Ryan Smith
Sam Jones
Tom Smith
If you now take this sorted list, and again sort by last name, you get a result sorted by both fields, where later sorts have precedence over earlier sorts... but you only guarantee the exact order if the sort is stable:
Jim Doe
Brad Jones
Sam Jones
James Smith
Ryan Smith
Tom Smith
This brings us to the question of what algorithm .Net uses, and whether it's stable. To the documentation we go, where we find this in the Remarks section:
This method performs a stable sort
The specific algorithm is not documented here. I believe it's a Quicksort, but leaving that out of the documentation is probably intentional, to allow the maintainers to update for the best available option that meets the stability requirement should something better be discovered.
But, again, that's for local data. Databases will do what the SQL tells them.
Actually, it is quite simple and it makes total sense:
The first part of the query
context.Entities.Where(x => x.ForeignKeyId == id)
would be translated to SQL more or less like this
select * from Entities
Adding the first order by
.OrderBy(x => x.FirstSortField)
it would be translated to this
select * from (
select * from Entities
)
order by FirstSortField
and then adding the second order by
.OrderBy(x => x.FirstSortField)
.OrderBy(x => x.SecondSortField)
would be translated to:
select * from (
select * from (
select * from Entities
)
order by FirstSortField
)
order by SecondSortField
Entity framework is smart enough to simplify to something like
select * from Entities
order by SecondSortField, FirstSortField
Background
I have a table that contains only one column: Name.
There are only four rows in it, say
| Name |
| test1.com |
| test2.com |
| test3.com |
| test4.com |
Problem
If I query
var email = "a#test2.com";
Table.Where(x => email.EndsWith(x.Name));
I'll get an empty list. but If I query all rows first and calculate Where in memory like this
var email = "a#test2.com";
Table.ToList().Where(x => email.EndsWith(x.Name));
I'll get a list contains only test2.com which is correct.
The generated SQL for the first query is
SELECT "Extent1"."Name" AS "Name"
FROM "USER"."Table" "Extent1"
WHERE (( NVL(INSTR(REVERSE(:p__linq__0), REVERSE("Extent1"."Name")), 0) ) = 1)
I've tried replacing :p__linq__0 with 'a#test2.com' and running the query in the SQLDeveloper, the result is correct.
More Information
If I change EndsWith() to Contains(), the problem will be gone. Here is the generated SQL for Contains()
SELECT "Extent1"."Name" AS "Name"
FROM "USER"."Table" "Extent1"
WHERE (( NVL(INSTR(:p__linq__0, "Extent1"."Name"), 0) ) > 0)
Do you have any idea what's wrong with EndsWith or REVERSE method?
Environment
EF5.0
.NET4.5
Oracle11g
ODP.NET11.2 Release 3
This line concerns me and is a common pitfall with people using EF:
Table.ToList().Where(x => email.EndsWith(x.Name));
The part Table.ToList() is the worst part because this will actually materialise the entire table into memory and then perform the EndsWith in C#.
This line:
Table.Where(x => email.EndsWith(x.Name));
I would caution this approach just on general principle as it will be horrendously slow when the table grows to reasonable size. You can do the heavy lifting before the query hits the database by splitting out the domain from the the email as you construct the query:
var email = "a#test2.com";
/* You should null check this of course and not just assume a match was found */
var domain = Regex.Match(email , "#(.*)").Groups[1].Value;
/* Note: ToList() materialisation happens at the end */
var result = Table.Where(x => x.Name == domain).ToList();
Furthermore, if you need to match on the domain names of a column storing emails, then my preferred approach would be to split the email and store the domain name in a separate column that you index and just match on, this will scale and be a heck of a lot easier to manage. Remember that these days data is cheap... especially compared to non-indexable table scans.
Also remember (for both scenarios) that your database is set to CI (case insensitive)
The backend is PostgreSQL server 9.1.
I am trying to build AdHoc XML reports. The report files will contain SQL queries, all of which must start with a SELECT statement. The SQL queries will have parameters. Depending upon the data type of the associated columns, these parameters will be accordingly presented to the user to provide value(s).
A rought SQL query:
SELECT * FROM customers
WHERE
(
customers.customer_code=#customer_code AND customers.location=#location
AND customers.type=
(
SELECT type from types
WHERE types.code=#type_code
AND types.is_active = #type_is_active
)
AND customers.account_open_date BETWEEN #start_date AND #end_date
)
OR customers.flagged = #flagged;
I want to get list of the column names and parameters from the query string and put them into a string array and process later.
I am able to match only the parameters using the following regular expression:
#(?)(?<parameter>\w+)
Expected Matches:
customers.customer_code=#customer_code
customers.location=#location
types.code=#type_code
types.is_active = #type_is_active
customers.account_open_date BETWEEN #start_date AND #end_date
customers.flagged = #flagged
How to to match "#Parameter", "=", and "BETWEEN" right away?
I know it's a little late, but for future researches' sake:
I think this Regex serves your purpose:
(\w+\.\w+(?:\s?\=\s?\#\w+|\sBETWEEN\s\#\w+\sAND\s\#\w+))
Check this Regex101 fiddle here, and read carefully the explanation for each part of it.
Basically, it first looks for your customer.xxx_yyy columns, and then either for = #variable or BETWEEN #variable1 AND #variable2.
Captured groups:
MATCH 1
1. [37-75]
`customers.customer_code=#customer_code`
MATCH 2
1. [80-108]
`customers.location=#location`
MATCH 3
1. [184-205]
`types.code=#type_code`
MATCH 4
1. [218-251]
`types.is_active = #type_is_active`
MATCH 5
1. [266-327]
`customers.account_open_date BETWEEN #start_date AND #end_date`
MATCH 6
1. [333-361]
`customers.flagged = #flagged`
How can I do a LINQ to SQL join without an exact match? For example, say I have a table form with the data John Smith (2) and I want to join it to the field Smith in table name. Something like this
var query =
from f in db.form
join n in db.name
on f.nameField like '%' + n.firstName + '%'
Although the like keyword doesn't seem to be available to me.
You can't use like in a Linq join. In fact, you can't use like in Linq at all, only conventional string methods like StartsWith, EndsWith, or Contains.
You'd have to do something like this:
var query =
from f in db.form
from n in db.name.Where(x => f.nameField.Contains(x.firstName))
...
Actually, there is a way of doing it, but it's not as neat as using the standard linq stuff:
from c in dc.Organization
where SqlMethods.Like(c.Hierarchy, "%/12/%")
select *;
(borrowed from the user L P's answer in the linked question)
In SQL, it'd be done as such:
SELECT * FROM Student
WHERE SchoolId IN
(SELECT id FROM School WHERE Name LIKE '%elementary%')
How do I implement this with LINQ? I've tried the following:
var list = context.Students.Where(x => context.Schools.Where(r => r.Name.Contains("elementary").Select(r => r.Id).Contains(x.SchoolId))
but it's not giving me what I want, unfortunately...
I know it's possible to retrieve all the Ids from the School table first, but I think it'd take a heavy toll on the performance. Preferably I'd like LINQ to SQL to handle everything; I can't do this using vanilla SQL because I need stuff to be dynamic and currently LINQ is the best solution for me.
The code above is all for illustration purposes; what I'm doing is a tad different (but more or less the same). I really do need some help on this; if you need any more information just feel free to ask.
EDIT: My bad, I missed out a field. It works, but the results didn't show up because I was missing that field... So sorry...
Try this:
var result = from st in context.Student
from sc in context.Schools
where sc.Name.Contains("elementary") && sc.SchoolId == st.SchoolId
select st;
I am a bit hazy on the syntax, pardon me. But this should point you to the right direction.
Something like this should work. The first use of Contains is on a string object to see if the string contains the substring "elementary". The second use of Contains is on a list and checks to see if the first result list contains SchoolId.
var sublist = from s in context.Schools
where s.Name.Contains("elementary")
select id;
var list = from s in context.Students
where sublist.Contains(s.SchoolId)
select s;