We Keep Coding

Can I write a query that returns all the values from a fixed list that are not in the database?

Let's say I have a list of integers.
int[] keys = { 14, 27, 32, 34 };
Is there a way to construct an Entity Framework query such that I find which, if any, of these integers does not equal the Id column in any row of my Users table? And I'd like to do this without actually loading any of the Users records.
So, for example, if my Users table contains the following data:
Id
Name
8
Bob Harrison
14
Sam Shephard
32
Bill Robinson
33
Susan Wilson
34
Gary Wright
I would want the results to be { 27 }, because only one key value was not in the list of user IDs.
Is this possible without loading any rows from the Users table?

You will possibly need to query some data from the DB, not entire User records, but their IDs. Provided the list of IDs and # of users being checked against is relatively manageable, something like this may be an option:
var minKey = keys.Min();
var maxKey = keys.Max();
var existingIds = _context.Users
.Where(x => x.Id >= minKey && x.Id <= maxKey)
.Select(x => x.Id)
.ToList();
var IdsNotInDb = keys.Except(existingIds);
This would fetch only the IDs within the range of Keys from the DB, then you can use that to compare against the list of Keys to find items in your list that don't have corresponding records. Since this is only loading the IDs, this would be just hitting the Index rather than the User table itself.
If you are instead expecting to compare very large numbers of keys against very large numbers of users, then you would probably want to take a Temporary Table approach inserting the Keys values into an indexed temporary table, then using EF to query using a join between the user table and this new temp table. An approach to consider for something like that can be found here: (https://www.thinktecture.com/en/entity-framework-core/temp-tables-in-3-1/)

I don't know how it can be done in a single line, but here's what i suggest you do :
You get an array the Ids of the users that do exist in the database, then you compare it to your first array 'keys'
Then you find the Ids that are not in the database.
But for some reason, using Linq expressions doesn't work for me, so here's how i did it :
//First your prepare a query that will return users which had their Id in
the keys array
var query = from user in Database.Users
where keys.Contains(user.id)
select user.id;
//Then you execute the query
var users = query.ToArray();
Then, you execute a query on the original list of keys, the same way you do it on your database (using Linq expressions), to find the items that do not exit in your returned array (your database) :
var notInUsers = keys.Where(k => !users.Contains(k)).ToArray();
Now, you got a new array called notInUsers which contains the Ids that were present in your 'keys' array, but which are not in your database.
Here you go, you have an optimised solution [it doesn't store your entire table (that may contain millions of records) in-memory], an you can do whatever you want with that array.
Note, if you want to work with lists instead of arrays, replace 'ToArray()' with 'ToList()'.

Yes, you can construct a query expression for the above case without loading the users.
var excludedIds = userContext
.Where(user => !keys.Contains(user.Id))
.Select(user => user.Id);
Nothing has executed at this point, and no user records have been loaded.

LINQ - how to remove duplicate rows in table

After certain proccess, I wan to remove duplicates from the table and commit the changes, so only single values remain.
I have three criteria for removal:
Name
date
status (is always 1)
So if there are records with same Name, and same date and same status... remove one. Does not matter which one.
I have:
dbContext.tbl_mytable

Since you are talking about deleting records, you need to test this first.
So if there are records with same Name, and same date and same status... remove one. Does not matter which one.
I'm assuming you want to remove all but one, ie, if you have three records with the same details, you remove two and leave one.
If so, you should be able to identify the duplicates by grouping by { Name, date, status} and then selecting all except the first record in each group.
ie something like
var duplicates = (from r in dbContext.tbl_mytable
group r by new { r.Name, r.date, r.status} into results
select results.Skip(1)
).SelectMany(a=>a);
dbContext.tbl_mytable.DeleteAllOnSubmit(duplicates);
dbContext.SubmitChanges();

Which is faster between Linq to Sql And SQl Query

I have List of object like this
List<Product> _products;
Then I get productId input and search in this list like this
var target = _peoducts.Where(o => o.productid == input).FirstOrDefault();
my Question is
If This list have 100 Products (productId from 1 to 100) and an
input I get productId = 100. that mean this Method must loop for 100
time Right ? (If I ORDER BY productId ASC in Query)
Between use this Method and Query on Database with where clause like
this WHERE productId = #param
Thank you.

No. If there is an index with key productId it finds the correct row with O(log n) operations
Just implement both methods and take the time. (hint: use StopWatch() class)
Edit
To get the full performance you should not create an intermediate (unsorted) List<T> but put all your logic in a LINQ query which operates on the SQL Server.

#might be helpful to get your answer.
https://www.linqpad.net/WhyLINQBeatsSQL.aspx

If you execute that Where on a List<Product>, then:
you got all 100 rows from the database
and then looped through all products in memory until you found the one that matches or until you went through the entire list and found nothing.
If, on the other hand, you used an IQueryable<Product> that was connected to the database table, then:
You wouldn't have read anything from the database yet
When you apply the Where, you still wouldn't read anything
When you apply the FirstOrDefault a sql query is constructed to find just the one row you need. Given correct indexes on the table, this would be quite fast.

select where comma-separted-string field contains a value from a list of strings

Table B has a field called 'tagged'. It is a string of comma-separated numbers, e.g.
454545,324512,234,345234
idList is a List of long numbers.
I want to get (a list of) all items in B, that one of the numbers in their 'tagged' field is contained within idLIST.
Trying:
context.B.Where(b => b.tagged.Split(',').ToList().Any(t => idList.ConvertAll<string>(f => f.ToString()).Contains(t))).ToList();
Gives me an exception with:
System.String[] Split(Char[])' has no supported translation to SQL
:(
(Note that I'm aware I can convert the long list to a string list beforehand, outside of the query, but it's not relevant to the question :) ).

You can use SqlMethods.Like, but it's really better to add table for ids.

group by first letter of the string

I have a Person table, with huge number of records, and want to group by duplicate persons in it,
by one of requirements persons are duplicates, if they have the same family name, and the first letter of first names are equal, so I want to group by first name, and first letter of family name, is there a way to group in sql like this? I need this in C#, so some code processing could be done, but the number of persons is huge, so it should be fast algorithm.

If I understand you correctly, from SqlServer you can do something like
SELECT DISTINCT
Surname,
LEFT(FirstName,1) FirstNameLetter
FROM Persons
Other than that, we will need a little bit more detail. Table schema, expected result set, etc...

SELECT MEMBER.MEMBER_FIRSTNAME, COUNT(MEMBER.MEMBER_LASTNAME)
FROM dbo.MEMBER
GROUP BY MEMBER.MEMBER_FIRSTNAME, SUBSTRING(MEMBER.MEMBER_LASTNAME, 1,1)
HAVING COUNT(MEMBER.MEMBER_LASTNAME) > 1
This query will give you all (members in this case) where the first name is the same and the last name's first letter is the same for more than one member. In other words duplicates as you've defined it.