i have about 1 million data in my database(MySQL)
and there's a advancedSearch function which is very slow(more than 30 sec), because the SQL EntityFramework generated is not very good, SQL:
SELECT
`Project1`.*
FROM
(
SELECT
`Extent1`.*
FROM `tnews` AS `Extent1`
WHERE `Extent1`.`Region` = 'Americas(2)'
) AS `Project1`
ORDER BY
`Project1`.`PnetDT` DESC LIMIT 0,20
C# function:
private List<CNNews> AdvancedSearchAndPage(int pagenum, int pagesize,
AdvSearchArgs advArgs)
{
IQueryable<CNNews> result = _dbRawDataContext.CNNews.
OrderByDescending(n => n.PnetDT);
if (!string.IsNullOrWhiteSpace(advArgs.Feed))
{
result = result.Where(news => news.Feed == advArgs.Feed);
}
if (!string.IsNullOrWhiteSpace(advArgs.PNET))
{
result = result.Where(news=>news.PNET == advArgs.PNET);
}
if (!string.IsNullOrWhiteSpace(advArgs.ProdCode))
{
result = (from news in result
where news.ProdCode == advArgs.ProdCode
select news);
}
if (!string.IsNullOrWhiteSpace(advArgs.Code))
{
result = (from news in result
where news.Code == advArgs.Code
select news);
}
if (!string.IsNullOrWhiteSpace(advArgs.BegineDate))
{
var begin = Convertion.ToDate(advArgs.BegineDate);
var end = Convertion.ToDate(advArgs.EndDate);
result = (from news in result
where news.PnetDT >= begin && news.PnetDT < end
select news);
}
if (!string.IsNullOrWhiteSpace(advArgs.Region))
{
result = result.Where(x => x.Region == advArgs.RegionName);
}
var pagedList = result.
Skip(pagenum * pagesize).
Take(pagesize);
return pagedList.ToList();
}
if the SQL format like this, it will very fast:
SELECT
*
FROM `tnews` AS `Extent1`
WHERE `Extent1`.`Region` = 'Americas(2)'
ORDER BY
`PnetDT` DESC LIMIT 0,20
You can execute your own SQL directly off the DbSet and get all the benefits of EF, see
http://msdn.microsoft.com/en-us/library/system.data.entity.dbset.sqlquery(v=vs.103).aspx
Also other ways, see these answers for more details
Is it possible to run native sql with entity framework?
The LINQ that generated your query looks something like this:
IQueryable<CNNews> result = _dbRawDataContext.CNNews
.OrderByDescending(n => n.PnetDT)
.Where(x => x.Region == advArgs.RegionName)
.Skip(pagenum * pagesize)
.Take(pagesize);
You tell LINQ to select all items and order them. Then you tell it to take a subset of that. The SQL looks exactly like what you have specified, I would say.
If you rearrange your code somewhat so that the Where() call is before the OrderByDescending() call I think you might get better SQL:
IQueryable<CNNews> result = _dbRawDataContext.CNNews
.Where(x => x.Region == advArgs.RegionName)
.OrderByDescending(n => n.PnetDT)
.Skip(pagenum * pagesize)
.Take(pagesize);
Also, I don't know if changing order of OrderByDescending() and Skip()/Take() would give different results.
(Disclaimer: I haven't tested it)
Related
if (rowCount == 1)
{
query =
(from x in partJoinTableRepository.GetPartJoinQuery()
join y in partRepository.GetPartsQuery() on x.PartId equals y.Id
join z in partProductTypeReposiotry.GetPartProductTypesQuery() on x.PartId equals z.PartId
where y.IsSkipped == 0 && (y.IsDisabled != "Y" || y.IsDisabled == null) && z.CreatedDate == x.CreatedDate
&& x.CreatedDate == Convert.ToDateTime(fromDate) && cpaclassids.Contains(x.ProductTypeId.ToString())
select x).Cast<PartJoinTable>().AsQueryable();
predicate = PredicateBuilder.True(query);
}
else
{
query = query.Join(partJoinTableRepository.GetPartJoinQuery(), "PartID", "PartID", "inner", "row1", null).Cast<PartJoinTable>().AsQueryable();
// predicate = PredicateBuilder.True(query);
} //query contains multiple dynamic inner joins
//repids contains the list ,I used the predicate builder for the linq to create AND Queries
foreach(var item in repids)
{
predicate = PredicateBuilder.True(query);
if (typeid == "3")
{
predicate = predicate.And(z => ids.Contains(z.ProductTypeId.ToString()) &&
z.CreatedDate == Convert.ToDateTime(fromDate));
}
}
var count = query.Where(predicate).Distinct().Count();
the above line is taking long time to execute,ids contains the lists and query contains the linq query.basically I need to form a multiple "AND" conditions
//The Query is taking lot of time to execute and multiple and conditions are not working
Remove ToList to improve performance. Because ToList execute your query and retrieve object list to memory. But you need only count. you don't need objects.
var count = query.Where(predicate).Distinct().Count();
If I understood you right, your problem is that this query has a long running time. Let's see your code in the last line:
var count = query.Where(predicate).Distinct().ToList().Count();
In LINQ to SQL (and to entities), your query doesn't execute thought you use ToList(), ToArray() etc.. For example, consider the following query:
var strings = Db.Table
.Where((string s) => s.Contains("A")) // Will convert to something like WHERE s LIKE '%A%'
.Select(s => s.ToUpper()) // Will convert to something like SELECT upper(s)
.ToList(); // Here the query sends to the DB and executes
The final query is SELECT upper(s) FROM [Table] WHERE s LIKE '%A%'.
In you case, first you send the query to the DB and get all the objects corresponding to the condition (.Where()), and then get their count inside your app.
Instead, if you'll get from the DB only the count, the query will be faster:
var count = query.Where(predicate).Distinct().Count(); // No .ToList()! Here, .Count() executes the query.
I'm completely new to LINQ, i want to rewrite some of mine SQL querys into LINQ (just to learn) and i'v already stuck at the beginning. Probably solution is very simple but as i'v said I'm completely new and i didn't find solution to this.
I have one query :
string typMoneta = textBox1.Text;
var moneta = from x in db.grupyTowarowes
where x.typ == typMoneta
select new
{
x.grupa
};
Which works ok and when i set
dataGridView1.DataSource = moneta;
Then i got output
And i want to use this output in my second query :
var query = from c in dbContext.Picking
where c.Number == 1000 && c.Group == moneta
select new
{
c.id
};
Problem is with c.Group == moneta. I don't know the correct syntax. Could someone help me?
I think you meant to use moneta.Contains(c.Group). In first query, make sure you use ToList() to load data into memory.
IList<string> moneta = (from x in db.grupyTowarowes
where x.typ == typMoneta
select x.grupa).ToList();
var query = (from c in dbContext.Picking
where c.Number == 1000 && moneta.Contains(c.Group)
select c.id).ToList();
The moneta is an IEnumerable<T> where T in your case is the type of grupa
That being said you should write your query like below:
var query = from c in dbContext.Picking
where c.Number == 1000
&& moneta.Contais(c.Group)
select new
{
c.id
};
or in fluent syntax like below:
var query = dbContext.Picking
.Where(pick => pick.Number == 1000
&& moneta.Contains(pick.Group))
.Select(pick => pick.id);
Note that moneta is not a collection of strings. It's a collection of objects that have a string property named "grupa".
Does this work for you?
var query =
from c in dbContext.Picking
where c.Number == 1000
&& moneta.Any(m => m.grupa == c.Group)
select new { c.id };
You could also do this:
// Get list of strings
var groups = moneta.Select(m => m.grupa).ToList();
// Get items where "Group" value is one of the strings in groups list, above.
var query =
from c in dbContext.Picking
where c.Number == 1000
&& groups.Contains(c.Group)
select new { c.id };
I am new to the entity framework and am trying to convert the following query into the correct function calls.
Select Distinct a.nodeId FROM
(SELECT *
FROM reportContents
Where fitId = '29' and reportId =
(select max(reportId)
from reportContents
where fitId = '29')
) a Where (a.nodeId IS NOT NULL)
I know this query does what i want, however i'm not sure how to translate that into the entitiy framework!
Here was my attempt.
var prevSelectedNodes = db.reportContents.Where(
f => f.fitId == id).Select(
f => f.nodeId).Distinct().ToList();
I need to somehow put a .Select() in the where call. However that kind of thing dosen't seem possible
Thank you in advance!
As you can't make two LINQ nested lambda expression. You can do it with two requests :
var maxReportId = db.reportContents.Where(r => r.fitId = "29").Max(r => r.RepordId);
var result = db.reportContents.Where(r => r.fitId == "29" && r.reportId == maxReportId && r.nodeId != null).Select(a => a.nodeId).Distinct().ToList() ;
Hello I have the following linq statement:
IEnumerable<TabTransaktion> allTransactions
= TabTransaktions1.Union(TabTransaktions2)
.Where(trans => trans.TabVorgang != null).
.OrderBy(tran => tran.TabVorgang.Wirkdatum)
.OrderByDescending(trans2 => trans2.TabVorgang.ID);
But I want the second order by descending when only trans2.TabVorgang.ID equals to 0. So I need a "case" in "order by clause" for LinQ. A LinQ equivalent of something like this:
SELECT BusinessEntityID, SalariedFlag
FROM HumanResources.Employee
ORDER BY CASE SalariedFlag WHEN 1 THEN BusinessEntityID END DESC
,CASE WHEN SalariedFlag = 0 THEN BusinessEntityID END;
GO
I would appreciate any help.
Assuming SalariedFlag is a bool (in SQL of type bit), the two ordering expressions are exclusively mutual. In other words, the main query can be separated into two disjunctive queries and the final result is the union of them:
IEnumerable<TabTransaktion> mainQuery
= TabTransaktions1.Union(TabTransaktions2)
.Where(trans => trans.TabVorgang != null);
var queryOne = mainQuery.Where(p=>p.SalariedFlag ==1)
.OrderByDescending(tran => tran.BusinessEntityID );
var queryTwo = mainQuery.Where(p=>p.SalariedFlag ==0)
.OrderBy(tran => tran.BusinessEntityID);
var finalResult = queryOne.Union(queryTwo);
I have ths function to query a set of records from the DB:
public IQueryable<PointTransactionViewModel> GetPointTransactions(int UserID)
{
return
(
from PointTransaction p in entities.PointTransaction
join ActivityLog a in entities.ActivityLog
on p.TransactionID equals a.TransactionID
where p.UserID == UserID
select new PointTransactionViewModel
{
ID = p.TransactionID,
Balance = p.Balance,
Points = p.Amount,
RelatedActivityID = a.ID,
When = p.When,
Sender = p.SenderUserInfo.CompleteName
}
);
}
I wish to add an additional cause, like this
var entries = GetPointTransaction(1);
return entries.OrderbyDescending.Where( x => x.When >= start && w.When <= end).
( x => x.When);
However, I seem to need to create a new query from the existing one for this to work. But, I have seem this work before without creating a new query, in the code snippet before:
public PaginatedList(IQueryable<T> source, int pageIndex, int pageSize)
{
PageIndex = pageIndex;
PageSize = pageSize;
TotalCount = source.Count();
TotalPages = (int)Math.Ceiling(TotalCount / (double)PageSize);
this.AddRange(source.Skip(PageIndex * PageSize).Take(PageSize));
}
Does the code above somehow doesn't need a new query to be created for the IQueryable source object? Was a temporary object created?
Edit
It's strange, but to get it to work I have to do the following:
IQueryable<ActivityLogEntry> log = activityRepo.GetPointTransaction(userID).
Where(x => x.PointsEarned == 50);
return log.ToList();
The following will not work:
var log = = activityRepo.GetPointTransaction(userID);
log.Where( x => x.PointsEarned == 50);
return log.ToList();
There is no error message, just that the where clause seems to be ignored (it is also returning all data which PointsEarned is not 50)
Your entries is of IQueryable type, that's enough and you can add any number of clauses before fetching the data, e.g. before calling the ToList() function.
It doesn't execute the SQL code, just an expression tree will be created until you fetch the whole data with one of the existing methods (again, e.g. the ToList() function).
var query = context.Where(x=>x.id == test);
query = query.Where(anotherCondition1);
query = query.Where(anotherCondition2);
...
var result = query.ToList();
it's equal to
var result = context.Where(x=>x.id == test)
.Where(anotherCondition1)
.Where(anotherCondition2)
....
.ToList()
This is called deferred execution, for more details see the MSDN blog post on LINQ and Deferred Execution.
You do need to create a new object. IQueryable is immutable. Don't worry this is how you are supposed to do it. This is how the queries are formed internally. All the extension methods like "Where" don't actually change the object. They just return a new one.
The code that you claim works should not work. The method doesn't even have a type.
i mean you can write this sample :
opportunites = from opp in oppDC.Opportunities
join org in oppDC.Organizations on opp.OrganizationID equals org.OrgnizationID
select new
{
opp.OpportunityID,
opp.Title,
opp.PostedBy,
opp.Address1,
opp.CreatedDate,
org.OrganizationName
};
if(condition)
{
opportunites = opportunites.Where(opp => opp.Title.StartsWith(title));
}
//------Other Condition you need
if(!String.IsNullOrEmpty(title))
{
opportunites = opportunites.Where(.....);
}
if(!String.IsNullOrEmpty(name))
{
opportunites = opportunites.Where(.....);
}
As others have pointed out, you do not need a new object. Your syntax for OrderByDescending is wrong though, you need to specify the key selector.
var entries = GetPointTransaction(1);
return entries.Where(x => x.When >= start && w.When <= end).OrderbyDescending(x => x.When);