I am trying to use compiled query for one of my linq to sql queries. This query contains 5 to 6 joins. I was able to create the compiled query but the issue I am facing is my query needs to check if the key is within a collection of keys passed as input. But compiled queries do not allow passing of collection (since collection can have varying number of items hence not allowed).
For instance
input to the function is a collection of keys. Say: List<Guid> InputKeys
List<SomeClass> output = null;
var compiledQuery = CompiledQuery.Compile<DataContext, List<Guid>, IQueryable<SomeClass>>(
(context, inputKeys) => from a in context.GetTable<A>()
where inputKeys.Contains(a.Key)
select a);
using(var dataContext = new DataContext())
{
output = compiledQuery(dataContext, InputKeys).ToList();
}
return output;
The above query does not compile since it is taking list as one of the inputs. Is there any work around or better way to do the above?
I'm not sure this is possible using only Linq to SQL. I think you'll need to have a stored procedure or function written on the server that lets you pass in a delimited string representing your list, and parses that returning a table, which you can then compare against.
I think the easiest way to accomplish this would be to write (or have your DBA write) the entire thing as a stored procedure, which will still need to take your list as a string for its argument, calling the aforementioned splitter function. The stored procedure will have its execution plan precompiled by the server.
You can easily make your list into a string using Linq with something like
string[] strings = new string[4] { "1", "2", "3", "4" };
string listOfStrings = strings.Aggregate((acc, s) => acc = acc + ", " + s);
You can turn a list of anything that can be cast to string into an IEnumerable of strings with
IEnumerable<string> strings = list.Cast<string>();
You can then add your stored procedure to your dbml file and call it using Linq to SQL.
I seem to recall that Linq to SQL, in order to remain general, doesn't deal with lists of things, and converts all lists you pass into a parameter for each entry in the list.
Related
I have a local List with entities, some hundreds, and I have a SQL Server table where I store the ID of the successful processed entities, some millions. I would like to know, which entities form my local set are not yet processed i.e. are not in the SQL Table.
The first approach is to iterate through the local list with the following Linq statement:
Entity entity = db.Entities.FirstOrDefault(m => m.ID == ID);
if (entity == null) { NewList.Add(ID) }
the NewList would then contain all the new entities. However this is very slow.
In LINQ, how would you send the entire local list to the SQL Server with one call and then return the ones not in the SQL table?
Do you really have to create a temporary table with my local list, then left-join on the already processed table and return the ones with a null?
Use .Contains method to retrieve already processed ids
and Except to create list of not yet processed ids.
var localList = new List<int> { 1, 2, 3 };
var processed = db.Entities
.Where(entity => localList.Contains(entity.Id))
.Select(entity => entity.Id)
.ToList();
var notProcessed = localList.Except(processed).ToList();
It will depend on provider, but .Contains should generate sql like:
SELECT Id FROM Entity WHERE Id IN (1, 2, 3)
suggestion:
create a temp table and insert your IDs
select your result on the SQL side
EDIT:
"Can you do that in LINQ?"
TL;DR:
yes* but that's an ugly piece of work, write the SQL yourself
*)depends on what you mean with "in" LINQ, because that is not in the scope of LINQ. In other words: a LINQ expression is one layer too abstract, but if you happen to have an LINQ accessible implementation for this, you can use this in your LINQ statements
on the LINQ expression side you have something like:
List<int> lst = new List<int>() { 1,2,3 };
List<int> result = someQueryable.Where(x=>lst.Contains(x.ID)).Select(x=>x.ID).ToList();
the question now is: what happens on the SQL side (assuming the queryable leads us to a SQL database)?
the queryable provider (e.g. Entity Framework) somehow has to translate that into SQL, execute it and come back with the result
here would be the place to modify the translation...
for example examine the expression tree with regard to the object that is the target for the Contains(...) call and if it is more than just a few elements, go for the temp table approach...
the very same LINQ expression can be translated into different SQL commands. The provider decides how the translation has to be done.
if your provider lacks support for large Contains(...) cases, you will probably experience poor performance... good thing is usually nobody forces you to use it this way ... you can skip linq for performance optimized queries, or you could write a provider extension yourself but then you are not on the "doing something with LINQ"-side but extending the functionality of your LINQ provider
if you are not developing a large scalable product that will be deployed to work with different DB-Backends, it is usually not worth the effort... the easier way to go is to write the sql yourself and just use the raw sql option of your db connection
Suppose I have a List containing one string value. Suppose I also have an IQueryable that contains several strings from a database. I want to be able to concatenate these two containers into one list and then be able to call methods such as .Skip or .Take on the list. I want to be able to do this in such a way that when I combine the two containers I don't load all of the DB data into memory (only after I call .Skip and .Take). Basically, I want to do something like this (pseudocode):
IQueryable someQuery = myEntities.GetDBQuery(); // Gets "test2", "test3"
IList inMemoryList = new List();
inMemoryList.Add("test");
IList finalList = inMemoryList.Union(someQuery) // Can I do something like this without loading DB data into memory? finalList should contain all 3 strings.
// At this point it is fine to load the filtered query into memory.
foreach (string myString in finalList.Skip(100).Take(200))
{
// Do work...
}
How can I achieve this?
If I didn't misunderstand, you are trying to query the data, part of which comes from memory and others from database, like this:
//the following code will not compile, just for example
var dbQuery = BuildDbQuery();
var list = BuildListInMemory();
var myQuery = (dbQuery + list).OrderBy(aa).Skip(bb).Take(cc).Select(dd);
//and you don't want to load all records into memory by dbQuery
//because you only need some of them
The short answer is NO, you can't. Consider the .OrderBy method, all data have to be in a same "place", otherwise the code can't sort them. So the code loads all records in database by dbQuery into memory(now they are in a same place) and then sorts all of them including those in list. That probably causes a memory issue when dbQuery gives thousands of rows.
HOW TO RESOLVE
Pass the data in list into database (as parameters of dbQuery) so that the query happens in database. This is easy if your list has only a few items.
If list also has lots of records that will makes dbQuery too complex, you can try to query twice, one for dbQuery and one for list. For example, you have 10,000 users in database and 1,000 users in your memory list, and you want to get the top 10 youngest users. You don't need to load 10,000 users into memory and then find the youngest 10. Instead, you find 10 youngest (ResultA) in dbQuery and load into memory, and 10 youngest (ResultB) in memory list, and then compare between ResultA and ResultB.
I entirely agree with Danny's answer when he says you need to somehow find a way to include in memory user list into db so that you achieve what you want. As for the example which you sought in your comment, without knowing data structure of your User object, seems difficult. However assuming you would be able to connect the dots. Here is my suggested approach:
Create temporary table with identical structure that of your regular user table in your db and insert all your inmemory users into it
Write a query to Union temporary and regular table both identical in structure so that should be easy.
Return the result in your application and use it performing standard Linq operations
If you want exact code which you can use as it is then you will have to provide your User object structure - fields type etc in db to enable me to write the code.
You specify that your query and your list are both sequences of strings. someQuery can be performed completely on the database side (not in-memory)
Let's make your sequences less generic:
IQueryable<string> someQuery = ...
IList<string> myList = ...
You also specify that myList contains only one element.
string myOneAndOnlyString = myList.Single();
As your list is in-memory, this has to be performed in-memory. But because the list has only one element, this won't take any time.
The query that you request:
IQueryable<string> correctQuery = someQuery
.Where(item => item.Equals(myOneandOnlyString)
.Skip(skipCount)
.Take(takeCount)
Use your SQL server profiler to check the used SQL and see that the request is completely performed in one SQL statement.
Let's Assume I have Table in SQL server that represents employee information for example
I want to do the Textual Mining on the Degree column that remove the Duplicates based on "##" notation.
LINQ to SQL
I am using Linq to SQL , so I am planning to get this data in C# variable context.And Perform operation on string and store again to the location!
Rules: i need to update the data or generate new table!
Is this right way of doing whether its possible ? need some suggestion on this approach or any alternative suggestions are welcome
So it looks like you need to break up the string based on the "##" delimiters, take the distinct items, and put them back in -- comma-delimited this time? The String.Split method to break up the string and then LINQ's Distinct extension method should get you just the unique ones.
Assuming you've got the text of the degree in a variable somewhere:
var uniques = degree
.Split(new String[] { "##" }, StringSplitOptions.None)
.Distinct();
String.Split usually works with a single character delimiter, but there's an overload that allows splitting on a larger string, so you'll have to use that one.
Then you can use String.Join to comma-delimit the unique items, or whatever else you need to do.
Edit: Apologies, I thought your original question was more about how to eliminate the duplicates than how to use LINQ to SQL.
Assuming you've got your DataContext and object model set up, you just need to select your object(s) out of the database using LINQ to SQL, make the changes you need to them, and then and then call SubmitChanges() on them.
For example:
var degrees = from d in context.GetTable<Employee>() select d;
foreach (var d in degrees)
{
d.Degree = String.Join(",", d.Degree
.Split(new String[] { "##" }, StringSplitOptions.None)
.Distinct());
}
context.SubmitChanges();
If you're new to LINQ to SQL, it may be worthwhile to run through a tutorial or two first. Here's part 1 of a pretty good series:
Lastly, you mentioned in your edit that you have the option of creating a new table after making your changes -- if that's the case, I'd consider storing the individual degrees in a table that links back to the employee record, rather than storing them as comma-separated values. It depends on your needs, of course, but SQL is designed to work in tables and sets, so the less string parsing/processing you can do the better.
Good luck!
This is my code:
var query = context.SomeEntities.Select(e => new SomeDto
{
Name = e.Title,
})
The variable query is IQueryable.
var list = new List<SomeDto>
{
new SomeDto
{
Name = "SomeName1"
},
new SomeDto
{
Name = "SomeName2"
}
};
Now I need union these collections,I write
query = query.Union(list)
But in this place I get exception
Unable to create a constant value of type 'SomeDto'. Only primitive types or enumeration types are supported in this context
I know that I can do ToList() for first collection,but I need union collections without call to database.How can I do this?
As user3825493 pointed, there are answers regarding similar issues. The first link pretty much explains the whole thing.
Your problem is that due to the fact that Union is mapped to SQL UNION, arguments of the operation are to be converted to data types acceptable by Entity Framework. However, the only types acceptable by it for items inside IEnumerable input parameters are, as specified in the error you get, primitive types and enum's.
You should either:
var unionOfQueryWithList = query.AsEnumerable().Union(list)
or, if you like to use the SQL facilities, write a stored procedure which performs the union as part of its logic and find the way to pass your data into SQL, such as this.
The former loads all the results of the query into a memory before performing the union. It is convenient for the cases when you have all the pre-filtered data returned from EF and only want to use the output for later processing in your code.
The later takes you out of the EF zone and required for the cases when you have operations which will benefit from running on SQL engine, such as filtering or joining DB data based on a complex data you have access to in your code.
either you convert the query to IEnumerable or you can try this :
list.Union(query); // use Union() of list not query :)
it will work and you still don't need to get data from DB you can reverse the order if you want :)
I have a webpage built with a dropdown that has a list of books. These books are stored on the sql sever. Using the MVC and aspx pages i am trying to figure out how to retrieve information about about a book such that when the user selects a book it passes the price of the book. I am new to sql sever and var statements.
I am able to retrieve the books name from the webpage and send it to my controller
In my model i am trying to get that data here is my thought. I want to get the price and store into a string. But the only way i have seen to pull information is using the var statement.
such that
var price = from p in BooksDB.Price
where p.Book_Name==bookName
select new {p.Book_Price}
but how do i get that value and store it into a string based on the Books_Name that I have retrieved from the dropdown box
by the way my table looks like this
Id_Num Book_Name Book_Price
1 Pro C# 29.99
2 Beg C++ 10.99
First a terminology correction.
var is just a keyword the compiler lets you use to subsitute for the type. During compiletime, the compiler will figure out what the type is based on usage.
var myString = "hihihi";
string myString = "hihihi";
The var statement has nothing to do with accessing a database, although it was added to make using LINQ easier on us lazy developers.
Instead what you are doing is creating a LINQ to SQL query. I've modified it slightly (you don't need to create an anonymous object). After you create the statement, you need to execute it by calling "ToList(), First(), or FirstOrDefault() etc"
LINQ typically employs lazy or deferred evaluation for queries, and isn't executed until you trigger execution.
var price = from p in BooksDB.Price
where p.Book_Name==bookName
select p.Book_Price;
//assuming Book_Price is stored as a string datatype.
string bookPrice = price.FirstOrDefault();
//otherwise
string bookPrice = (price.FirstOrDefault() ?? "").ToString();
if(!String.IsNullOrEmpty(bookPrice))
{
//do something with the string.
}
If I understand correctly something like below should work.
var record = BooksDB.Price.FirstOrDefault(r => r.Book_Name == bookName);
If record is not null at this point then record.Book_Price should contain the data you are looking for (not accounting for ambiguity in the database.)
Instead of select new {p.Book_Price}, use something like select p.Book_Price. You could add .ToString() to the end to force it to come out as a String rather than the data type from the table.
You'll also want to wrap the entire LINQ statement in () and append .FirstOrDefault() to get just one value. The "OrDefault" part protects you from an exception when the result set us empty.