I have ths function to query a set of records from the DB:
public IQueryable<PointTransactionViewModel> GetPointTransactions(int UserID)
{
return
(
from PointTransaction p in entities.PointTransaction
join ActivityLog a in entities.ActivityLog
on p.TransactionID equals a.TransactionID
where p.UserID == UserID
select new PointTransactionViewModel
{
ID = p.TransactionID,
Balance = p.Balance,
Points = p.Amount,
RelatedActivityID = a.ID,
When = p.When,
Sender = p.SenderUserInfo.CompleteName
}
);
}
I wish to add an additional cause, like this
var entries = GetPointTransaction(1);
return entries.OrderbyDescending.Where( x => x.When >= start && w.When <= end).
( x => x.When);
However, I seem to need to create a new query from the existing one for this to work. But, I have seem this work before without creating a new query, in the code snippet before:
public PaginatedList(IQueryable<T> source, int pageIndex, int pageSize)
{
PageIndex = pageIndex;
PageSize = pageSize;
TotalCount = source.Count();
TotalPages = (int)Math.Ceiling(TotalCount / (double)PageSize);
this.AddRange(source.Skip(PageIndex * PageSize).Take(PageSize));
}
Does the code above somehow doesn't need a new query to be created for the IQueryable source object? Was a temporary object created?
Edit
It's strange, but to get it to work I have to do the following:
IQueryable<ActivityLogEntry> log = activityRepo.GetPointTransaction(userID).
Where(x => x.PointsEarned == 50);
return log.ToList();
The following will not work:
var log = = activityRepo.GetPointTransaction(userID);
log.Where( x => x.PointsEarned == 50);
return log.ToList();
There is no error message, just that the where clause seems to be ignored (it is also returning all data which PointsEarned is not 50)
Your entries is of IQueryable type, that's enough and you can add any number of clauses before fetching the data, e.g. before calling the ToList() function.
It doesn't execute the SQL code, just an expression tree will be created until you fetch the whole data with one of the existing methods (again, e.g. the ToList() function).
var query = context.Where(x=>x.id == test);
query = query.Where(anotherCondition1);
query = query.Where(anotherCondition2);
...
var result = query.ToList();
it's equal to
var result = context.Where(x=>x.id == test)
.Where(anotherCondition1)
.Where(anotherCondition2)
....
.ToList()
This is called deferred execution, for more details see the MSDN blog post on LINQ and Deferred Execution.
You do need to create a new object. IQueryable is immutable. Don't worry this is how you are supposed to do it. This is how the queries are formed internally. All the extension methods like "Where" don't actually change the object. They just return a new one.
The code that you claim works should not work. The method doesn't even have a type.
i mean you can write this sample :
opportunites = from opp in oppDC.Opportunities
join org in oppDC.Organizations on opp.OrganizationID equals org.OrgnizationID
select new
{
opp.OpportunityID,
opp.Title,
opp.PostedBy,
opp.Address1,
opp.CreatedDate,
org.OrganizationName
};
if(condition)
{
opportunites = opportunites.Where(opp => opp.Title.StartsWith(title));
}
//------Other Condition you need
if(!String.IsNullOrEmpty(title))
{
opportunites = opportunites.Where(.....);
}
if(!String.IsNullOrEmpty(name))
{
opportunites = opportunites.Where(.....);
}
As others have pointed out, you do not need a new object. Your syntax for OrderByDescending is wrong though, you need to specify the key selector.
var entries = GetPointTransaction(1);
return entries.Where(x => x.When >= start && w.When <= end).OrderbyDescending(x => x.When);
Related
(In theory) We have the following query:
var variableDate = DateTime.Parse("Nov 2, 2021")
var results = (from x in db.FooBar
where x.Date == variableDate
select x).ToList();
We can simply modify the variableDate to 11/3/21, 11/4/21, etc. to return results for different dates.
The question is can we use this same query to return all results after variableDate with some modification to the operator(s)/variable(s)? For example including the operator (> or ==) within variableDate (let's call it variableDateFormula):
> 11/2/2021
or
== 11/2/2021
This way we can simply call the same query with the same variable to return results for different operators:
var results = (from x in db.FooBar
where x.Date variableDateFormula
select x).ToList();
I know the suggestion above will not work, it is just for visualization. I have a query that takes up 9 lines and has 8 different iterations depending on 3 values passed into the function. I would love to avoid writing basically the same query 8 times and taking up ~70 lines, and instead dynamically change the operators/criteria to have only 1 query.
Yes, the operator is just syntactic sugar for a function call and you can change that function, for example with a lambda:
var equalFunc = ((d1,d2) => d1 == d2);
var greaterFunc = ((d1,d2) => DateTime.Compare(d1, d2) > 0);
var actualFunc = greaterFunc; // Insert logic here to choose the appropriate function
Then your select becomes
from x in db.FooBar
where actualFunc(x.Date, variableDate)
select x
Yes. The LINQ is just building an ExpressionTree that Entity Framework translates in to SQL, so you can use other expressions or lambdas as parameters that get inserted into the tree, like in this LINQpad example:
void Main()
{
var aDate = new DateTime(2002, 1, 1);
GetResults(d => d.TheDate == aDate).Dump();
GetResults(d => d.TheDate <= aDate).Dump();
GetResults(d => d.TheDate > aDate).Dump();
}
IEnumerable<Data> GetResults(Func<Data, bool> op)
{
var data = new List<Data>
{
new Data{TheDate = new DateTime(2000,1,1)},
new Data{TheDate = new DateTime(2001,1,1)},
new Data{TheDate = new DateTime(2002,1,1)},
new Data{TheDate = new DateTime(2003,1,1)},
};
return data.Where(d => op(d));
}
public class Data
{
public DateTime TheDate { get; set; }
}
Which produces the following results:
If you are doing more complex logic in your expressions, the EF Database Provider may not be able to translate the expression to SQL though. There will be limitations. You couldn't for example, do this:
GetResults(d => d.TheDate.ToString().Reverse() == "1234");
because it wouldn't understand the custom Reverse extension method.
I would suggest to use LINQKit for such task. EF Core cannot translate local variable as expression function.
var variableDate = DateTime.Parse("Nov 2, 2021");
Expression<Func<DateTime, DateTime, bool>> compareFunc = (d1, d2) => d1 > d2;
var results = (from x in db.FooBar
where compareFunc.Invoke(x.Date, variableDate)
select x).ToList();
For enabling LINQKit for EF Core add the following to the options:
builder
.UseSqlServer(connectionString)
.WithExpressionExpanding(); // enabling LINQKit extension
Or if you use other LINQ provider, add AsExpandable() at the top of the query:
var results = (from x in db.FooBar.AsExpandable()
where compareFunc.Invoke(x.Date, variableDate)
select x).ToList();
I have a query in EF where there is a List of string value that it checks for existence in another table.
Please consider the below query for more details.
Code
List<string> ItmsStock = item.Select(ds => ds.ItemNum).ToList(); // Currently, This List items count is 80,000 records.
this.Db.Database.CommandTimeout = 180;
var existsStckList = Db.Stocktakes.Where(ds => ItmsStock.Contains(ds.ItemNo)).Select(ds => ds.ItemNo).ToList();
item.RemoveAll(ds => existsStckList.Contains(ds.ItemNum));
var ItmsExists = Db.Items.Where(ds => ItmsStock.Contains(ds.ItemNo)).Select(ds => ds.ItemNo).ToList();
ItmsExists = Db.Stocktakes.Where(ds => !ItmsExists.Contains(ds.ItemNo)).Select(ds => ds.ItemNo).ToList();
I searched on the internet and found the converted sql uses IN to check for existence. so, the limit for the IN makes the problem. My question here is, How can I efficiently perform the above actions without using for loop.
I ll be appreciating you, If anybody can help me out.
Edit
Previously, I had the below code. After facing the performance issue with the below code, I wrote the above one.
foreach (var stockitems in item)
{
if (Db.Stocktakes.Any(a => a.ItemNo == stockitems.ItemNum))
{
StockResult ss = new StockResult();
ss.ItemNumber = stockitems.ItemNum;
ss.FileName = stockitems.FileName;
Stockres.Add(ss);
}
else if (!Db.Stocktakes.Any(a => a.ItemNo == stockitems.ItemNum) && Db.Items.Any(a => a.ItemNo == stockitems.ItemNum))
{
var ItemNo = stockitems.ItemNum;
var AdminId = Convert.ToInt32(Session["AccId"]);
var CreatedOn = System.DateTime.Now;
int dbres = Db.Database.ExecuteSqlCommand("insert into Stocktake values({0},{1},{2})", ItemNo, AdminId, CreatedOn);
Db.SaveChanges();
totalcount = totalcount + 1;
}
else
{
StockResult sss = new StockResult();
sss.ItemNumber = stockitems.ItemNum;
sss.FileName = stockitems.FileName;
Stockitemsdup.Add(sss);
}
}
Thanks.
Issue batches of 1000 item IDs to the database, or use native SQL and submit a table-valued parameter, or a temp table filled with SqlBulkCopy.
I'm surprised you got htis particular message. The parameter limit is about 2000 parameters. Your query should have been rejected.
I have 2 tables: POHeader and PODetail. I want to return all POHeaders that have an associated PODetail.ItemId = intItemId. How can I do this in LINQ?
This is what I've tried.
First I have a method in my Repository that uses the Include parameter to include the PODetails:
public IQueryable<POHeader> SearchForWithDetails(int intFacilityId)
{
return DbSet.Include("PODetails").Where(x => x.FacilityId == intFacilityId);
}
Then the result of that gets passed to:
public IQueryable<POHeader> SearchForPODetailsByItemId(IQueryable<POHeader> poHeaders, int intItemId)
{
//This returns a type of PODetail not POHeader
var q = poHeaders.SelectMany(c => c.PODetails).Where(c => c.ItemId == intItemId);
//In this case, I can't figure out the syntax :(
var p = from poHeader in poHeaders
let filteredPOs = from poDetail in poHeader.PODetails
where poDetail.ItemId == intItemId
select ????
return p;
}
What is the correct way to do this?
Also, I can foresee needing 2 results of this:
just return a IQueryable
return a joined table result.
Try this;
var result = poHeaders.Where(e => e.PODetails.Any(a => a.ItemId == intItemId));
Assuming your a Header->Detail is a 1-to-many relationship, and Detail has a navigation back to it's header called .Header:
public IQueryable<POHeader> SearchForPODetailsByItemId(IQueryable<POHeader> poHeaders, int intItemId)
{
var headersForThisItem = poHeaders.SelectMany(pod => pod.PODetails).Where(pod => pod.ItemId == intItemId)
.Select(pod=> pod.Header).Distinct();//.Distinct to eliminate duplicates when 2 Details have the same header. Not necessary if ItemId filter naturally provides distinct results.
return headersForThisItem ;
}
Untested, but I think that will give you what you want.
i have about 1 million data in my database(MySQL)
and there's a advancedSearch function which is very slow(more than 30 sec), because the SQL EntityFramework generated is not very good, SQL:
SELECT
`Project1`.*
FROM
(
SELECT
`Extent1`.*
FROM `tnews` AS `Extent1`
WHERE `Extent1`.`Region` = 'Americas(2)'
) AS `Project1`
ORDER BY
`Project1`.`PnetDT` DESC LIMIT 0,20
C# function:
private List<CNNews> AdvancedSearchAndPage(int pagenum, int pagesize,
AdvSearchArgs advArgs)
{
IQueryable<CNNews> result = _dbRawDataContext.CNNews.
OrderByDescending(n => n.PnetDT);
if (!string.IsNullOrWhiteSpace(advArgs.Feed))
{
result = result.Where(news => news.Feed == advArgs.Feed);
}
if (!string.IsNullOrWhiteSpace(advArgs.PNET))
{
result = result.Where(news=>news.PNET == advArgs.PNET);
}
if (!string.IsNullOrWhiteSpace(advArgs.ProdCode))
{
result = (from news in result
where news.ProdCode == advArgs.ProdCode
select news);
}
if (!string.IsNullOrWhiteSpace(advArgs.Code))
{
result = (from news in result
where news.Code == advArgs.Code
select news);
}
if (!string.IsNullOrWhiteSpace(advArgs.BegineDate))
{
var begin = Convertion.ToDate(advArgs.BegineDate);
var end = Convertion.ToDate(advArgs.EndDate);
result = (from news in result
where news.PnetDT >= begin && news.PnetDT < end
select news);
}
if (!string.IsNullOrWhiteSpace(advArgs.Region))
{
result = result.Where(x => x.Region == advArgs.RegionName);
}
var pagedList = result.
Skip(pagenum * pagesize).
Take(pagesize);
return pagedList.ToList();
}
if the SQL format like this, it will very fast:
SELECT
*
FROM `tnews` AS `Extent1`
WHERE `Extent1`.`Region` = 'Americas(2)'
ORDER BY
`PnetDT` DESC LIMIT 0,20
You can execute your own SQL directly off the DbSet and get all the benefits of EF, see
http://msdn.microsoft.com/en-us/library/system.data.entity.dbset.sqlquery(v=vs.103).aspx
Also other ways, see these answers for more details
Is it possible to run native sql with entity framework?
The LINQ that generated your query looks something like this:
IQueryable<CNNews> result = _dbRawDataContext.CNNews
.OrderByDescending(n => n.PnetDT)
.Where(x => x.Region == advArgs.RegionName)
.Skip(pagenum * pagesize)
.Take(pagesize);
You tell LINQ to select all items and order them. Then you tell it to take a subset of that. The SQL looks exactly like what you have specified, I would say.
If you rearrange your code somewhat so that the Where() call is before the OrderByDescending() call I think you might get better SQL:
IQueryable<CNNews> result = _dbRawDataContext.CNNews
.Where(x => x.Region == advArgs.RegionName)
.OrderByDescending(n => n.PnetDT)
.Skip(pagenum * pagesize)
.Take(pagesize);
Also, I don't know if changing order of OrderByDescending() and Skip()/Take() would give different results.
(Disclaimer: I haven't tested it)
I have a linq query from two database, however, each time the program stops at the query point. I don't know how to debug linq using VS. Can someone help me figure it out what's wrong here? Thank you.
public List<Promotion> GetBroder(string source)
{
string _connString = ConfigurationManager.AppSettings["DB1"];
PromotionDataContext dc = new PromotionDataContext(_connString);
string connString = ConfigurationManager.AppSettings["DB2"];
ReachDirectDataContext RDdc = new ReachDirectDataContext(connString);
return (from b in RDdc.BrokerNos
from p in dc.Promotions
where p.Source == source && p.Broker == b.BrokerNo1
select new Promotion() {Code=p.Code,BrokerName=b.Name}).ToList<Promotion>();
}
Your linq statement looks fine. To aid in debugging, I find it helpful to assign the linq query to a local variable, then return the local variable. You can then set a breakpoint on the return statement, and when the debugger stops at the breakpoint, you can inspect the query local variable to see what's in it, interactively. You can use the Locals window in VS, or the Immediate Window to surf around inside your app's variables and see what's going on.
In particular, double check that the inputs into your linq query are actually providing data. Verify that RDdc.Brokernos is non-empty, and dc.Promotions, etc. If these are empty, the result will be empty. Track your bug "upstream".
Minor point: You don't need to specify the type parameter on the .ToList() call in the select. The compiler will infer the type automagically.
You can use the following to display the generated SQL for the Linq statement.
ReachDirectDataContext RDdc = new ReachDirectDataContext(connString);
RDdc.Log = Console.Out;
return (from b in RDdc.BrokerNos
from p in dc.Promotions
where p.Source == source && p.Broker == b.BrokerNo1
select new Promotion() {Code=p.Code,BrokerName=b.Name}).ToList<Promotion>();
You can try the following to separate out the queries.
var promotions = from p in dc.Promotions
where p.Source == source
select p;
var brokers = from o in promotions
join b in RDdc.BrokerNos on o.Broker equals b.BrokerNo1
select new Promotion
{
Code = o.Code,
BrokerName = b.Name
};
return brokers.ToList();
The double from looks suspicious to me.
(This is not the equivalent to SQL's JOIN statement, if that is what you were aiming for.)
If you can combine the contexts:
Try creating relationship between BrokerNos and Promotions in edmx and using navigation property in query.
For example:
var result = dc.Promotions.Where(p => p.Source == source).
Select(p => new Promotion() {
Code = p.Code,
BrokerName = p.Broker.Name, // use new navigation property here
});
If not (intersection will be done in memory, not on DB!!!:
var result1 = dc.Promotions.Where(p => p.Source == source).
Select(p => new Promotion() {
Code = p.Code,
BrokerId = p.BrokerId, // add id property for intermediate results
}).ToList();
var result2 = RDdc.Brokers.ToList();
var finalResult = result1.Where(p => result2.Contains(b => b.BrokerId == p.BrokerId)).Select(p => new Promotion{
Code = p.Code,
BrokerName = result2.Single(b => b.BrokerId == p.BrokerId).Name,
});