I have a task to construct a dynamic query (or algorithm) based on existing query with user choosed fields. Let me explain:
Lets say I have a function
ConstructQuery(string inputQuery, string[] mandatoryTables, string[] userFields) with 2 input parameters:
inputQuery: string query with many fields and tables, joins and where conditions
mandatoryTables: a list of mandatory tables
userFields: a list of fields that user choose in some web or desktop app
Function would have to return optimized query with tables and joins that are only needed for query to succeed.
inputQuery is for example constructed like this:
SELECT
Table1.SomeFieldA,
Table2.SomeFieldB,
Table2.SomeFieldC,
Table3.SomeFieldD
FROM Table1
JOIN Table2 ON Table1.Code = Table2.Code
JOIN Table3 ON Table2.Code = Table3.Code
WHERE Table1.SomeConditionField = "xyz"
userFields are: SomeFieldB, SomeFieldC
mandatoryTables: Table1
So the expected query is:
SELECT
Table2.SomeFieldB,
Table2.SomeFieldC
FROM Table1
JOIN Table2 ON Table1.Code = Table2.Code
WHERE Table1.SomeConditionField = "xyz"
My question is: is there a tool of some sort for solving this kind of problems or how you guys would solve it? I'm thinking of binary trees…
Regards,
Jani
This is something called join removal. This is (very) hard. Just parsing the query is nontrivial, then You'd have to analyze semantics, consider what are the unique keys, what are foreeign keys to have a chance to remove some tables. In Your example: the algorithm would have to know that table3.code is unique, and a foreign key to table2.code, otherwise the queries are not equivalent.
It could be easier to generate the right query in the first place. This is what some ORMs do.
Related
I was practicing today when I realized that there are two ways linq to sql can retrieve data from db, I created two datagrid and used the two different ways to populate each of these datagrids and they produced the same result.
The first method is using joins to get data from related tables, and the other methods is using linq query like an object to access related tables. The code is shown below:
NorthWindDataContext dbContext = new NorthWindDataContext();
var orders = from ord in dbContext.Orders
select new { ord.ShipCountry , ord.Customer.ContactName};
var orders2 = from ord in dbContext.Orders
join cust in dbContext.Customers on ord.CustomerID equals cust.CustomerID
select new
{
ord.ShipCountry, cust.ContactName
};
var data = orders2;
DataGrid.ItemsSource= orders;
DataGrid2.ItemsSource = orders2;
My question like the title is if it is entirely necessary to use joins, because I find them really cumbersome to use sometimes.
You need to use something that gets you from the order to the customer.
Join can do this. This is how the second query works.
Having the order "know" about the customer can do this. This is how the first query works.
If your data provider is aware of the connection between order and customer then these will amount to the same thing.
If your data provider is not aware of the connection, then the approach in the first example would result in N + 1 look ups instead of 1.
A linq-friendly ORM will generally be aware of these connections as long as the appropriate relationship-marking attributes are present (just what that is differs between Linq2SQL, EF, NHibernate, etc.).
It's still important to know the join approach for cases where either the relationship isn't known about by the provider, or you have a reason to join on something other than a foreign-key relationship.
The answer is "sort of". Since you're using an ORM such as Linq-to-Sql, no you don't directly need to call join within your linq queries to accomplish what you're trying to do.
However, when the ORM activates the query it will generate actual SQL code that'll have a join statement in it to get the results you're querying. Since you're using an ORM though, the data returned is mapped to objects, and since Customer has a relationship between the objects, the relationship will also be translated to from the database INTO the objects.
ord.Customer.ContactName
The above statement is most likely translated to a JOIN statement performing an INNER JOIN between Customer & Orders.
Due to this, both of your LINQ queries most likely generating similar SQL queries. Both of which has a JOIN statement in them. Because the relationships between your objects also exists within the database (and everything is mapped together showing this relationship) you don't directly need to use join within a LINQ statement.
I've three tables
Investigators: id, name, first name
Articles: id, name, date
ArticlesInvestigators: id_Investigator, id_Articles, order
I want to get only all the investigators from the Investigators table to have one article published:
In SQL it would look like:
SELECT * FROM Investigators i, ArticlesInvestigators a
WHERE i.id = a.id_Articles
But how shall be in LINQ and lambda? Without join because only want the results of investigators table.
I don't know if i understand your question or not but try this way and tell us if it work or not :
1) you need to make FOREIGN KEY between your tables
2) do you query in C#
3) then you can get you data using the relation between your tables
In our current application we have some performance issues with some of our queries. Usually we have something like:
List<int> idList = some data here…;
var query = (from a in someTable where idList.Contains(a.Id));
while for simple queries this is acceptable, it becomes a bottleneck when we have more items in idList (in some queries we have about 700 id’s to check, for example).
Is there any way to use something other then Contains? We are thinking of using some temporary tables to first insert the Ids, and then to execute join instead of Contains, but it would seem EntityFramework does not support such operations (creating temporary tables in code) :(
What else can we try?
I Suggest using LINQ PAD it offers a "Transform to SQL" option which allows you to see your query in SQL syntax.
there is a chance that this is the optimal solution (if youre not into messy stuff).
might try holding the idList as a sorted array and have the contains method replaced with a binary search. (you can implement your own extension).
You can try this:
var query = someTable.Where(a => idList.Any(b => b.Id == a.Id));
If you don't mind having a physical table you could use a semi-temporary table. The basic idea is:
Create a physical table with a "query id" column
Generate a unique ID (not random, but unique)
Insert data into the table tagging the records with the query ID
Pass the query id to the main query, using it to join to the link table
Once the query is complete, delete the temporary records
At worst if something goes wrong you will have orphaned records in the link table (which is why you use a unique query ID).
It's not the cleanest solution but it will be faster than using Contains if you have a lot of values to check against.
When Entity Framework starts being a performance bottleneck, generally it's time to write actual SQL.
So what you could do for example is build a table-valued function that takes a table-valued parameter (your list of IDs) as parameter. The function would just return the result of your JOIN.
Table valued function feature requires EF5, so it might be not an option if you're really stuck with EF4.
The idea is to refactor your queries to get rid of idList.
For example you should return the list of orders of male users 18-25 year, from France. If you filter users table by age, sex and country to get idList of users you end up with 700+ id's. Instead you make Orders table join with Users and apply filters to the Users table. So you don't have 2 requests (one for ids and one for orders) and it works much faster cause it can use indexes while joining the table.
Makes sense?
I am just wondering about how we can determine whether to use join or not in linq to sql.
Eg. let say if we have two tables like this
Table 1 Customer
id
name
Table 2 addresstype
id
address1
customerid
and
var address = from cu in Customer
from ad in addresstype
where cu.id == ad.customerid
select ad;
or
var address = from cu in Customer
join ad in addresstype on cu.id equals ad.customerid
select de;
Is both way are the same. Is there any difference in performance?
Also the second method, will it come up with an error if there isn’t any matching?
Are you using linq to entities or linq to SQL? If its the former then you can avoid both of these by defining your relationships in the model and using navigation properties. This would be the clearest way of doing things
Basically, these two LINQ queries are equivalent to the following SQL queries:
select ad.*
from Customer cu, AddressType ad
where cu.ID == ad.CustomerID -- I assume this was meant by the OP
and
select ad.*
from Customer cu
inner join AddressType ad on cu.id = ad.CustomerID;
The difference between these two queries is mostly semantic, since the database will do the same thing in both cases and return a same result set for both queries.
I would prefer the join syntax in both SQL and LINQ since it defines an explicit relationship between the two tables/entities, that is only implied in the join-less version.
These are seems same query, they return same result but I don't know which one can be a faster, it should be bench marked.
But, In the case of linq2sql I prefer correlated subquery over join, because currently if you want t check the equation two element you should use syntax of:
new {X,Y} equals new {X',Y'}
in join and if you have more than this equations you should convert it to nested query. So I Prefer to have a more readable code which uses minimum differences in difference actions.
To throw a third and more prefered method into the mix with LINQ to SQL, use associations between the tables (even if you don't have them set up in your database). With that in place, you can navigate the object graph rather than using joins:
var query = from cu in Customer
from ad in cu.Addresses
select ad;
Note: when querying the object graphs, LINQ to SQL translates the join into a left outer join where-as the join/where syntax by default is an inner join.
Joins in LINQ should be used when there isn't a natural relationship between the objects. For example, use a join if you want to see the the listing of stores that are in the same city as your customers. (Join Customer.Address.City with Store.Address.City).
There should not be a difference between these two queries. I actually wondered this question myself a few months ago. I verified this through LINQPad. It's a free tool that you can download and actually see the generated SQL of any LINQ query (this is the query that is sent to the database).
The generated SQL should be the same for these two queries.
If you're doing this through Visual Studio, there is also a way you can see the generated SQL as well.
I keep tables on different .sdf files because it's easy to manage them, ie; back up only changed db file, etc, plus in future db size might bigger and there is -4GB limit-
I need to join the tables and this will be my first -possibly LINQ- attempt. I know there are tons of examples/documents but a simple example would be nice to start.
This is the query for MS SQL Server:
SELECT personID, personPin, personName, seenTime
FROM db1.personList
LEFT JOIN db2.personAttendances on personID = seenPersonID
ORDER BY seenTime DESC
I think LINQ will be the way to go as you're querying across 2 different contexts. LINQ joins are quite easy: http://msdn.microsoft.com/en-gb/vcsharp/ee908647
Something like:
var q = from c in db1Context.personList
join p in db2Context.personAttendances on c.personID equals p.seenPersonID
select new { Category = c, p.ProductName };
I don't think SqlCE supports linking at the Db (SQL) level.
That means you'll have to use Linq-to-Objects. The example query has no WHERE clause so you can simply load the entire tables into Lists. But when the datasets get bigger that may not be acceptable.