i have the following self-referencing table
public partial class products_category
{
public long id { get; set; }
public string category_name { get; set; }
public string category_description { get; set; }
//self referencing to table id
public Nullable<long> Parent_Id { get; set; }
public string navPath {get; set; }
}
here string navpath contains all the leading parents for a child categroy, say:
"Clothes" = 1 Parent_id=null, navpath=""
"Silk" = 2 Parent_id=1 navpath="1"
"Silk Suit"=3 parent_id=2 navpath="1-2"
"Saree" =4 parent_id=3 navpath="1-2-3"
"Dress Material"=5 parent_id=1 navpath="1" and so on....
now as per this scenario i want to access the flattend tree for frther processing for a certain depth only say to level 2 or until level 4 depth of children associated with navpath.
my idea regarding this issue was to approach using linq to ef this way:
var catTrees = db.products_category.Where(pc => pc.navpath.Split('-').Length < 4).ToList();
i am using the following link to do further traversing and tree generation:
https://bitlush.com/blog/recursive-hierarchical-joins-in-c-sharp-and-linq
and it is doing a great work so far, the only issue is i dont want to pre select whole table for processing. i want to achieve paging and a certain level of depth for first iteration, so i can maintain performance in case of thousand of records. [think of this as a category hierarchy or blog/youtube comments hierarchy].
but using the above ef linq command is giving the following error:
The LINQ expression node type 'ArrayLength' is not supported in LINQ to Entities.
i checked with ef docs and other places in SO to know that string.split doesn't work with EF implicitly. but can we apply it using extension methods or can this tree selection have alternate approach without using string.split and hitting DB only ones?
please advice.
This looks like an issues with building SQL code out of your LINQ mpre specifically SQL which takes a string splits it on dash and counts the elements.
if you dont hate the idea of loading into memory then you can force anything :)
var catTrees = db.products_category.ToList().Where(pc => pc.navpath.Split('-').Length < 4).ToList();
The trick here is to force the execution of the SQL by adding the .ToList() when we want the data from the database. This is called realizing the data.
Even with that realization trick the count is faster
var catTrees = db.products_category.ToList().Where(pc => pc.navpath.Count(a => a == '-') < 3).ToList();
these solutions are essentially the same as
List<Result> filter() {
List<Result> r = new List<Result>();
foreach(var a in db.products_category) {
if(a.navpath.Count(a => a == '-') < 3) {
r.add(a);
}
}
return r;
}
When thinking about it the filter method is somewhat less memory intensive as it reads one and one and never stores everything in memory. (in theory at least, only a few really knows what the .NET compiler does in the shadows)
I would advice you against using the navpath for checking depth.
If you can change your model, you could add an additional numeric Depth field for each category and populate it according its navpath, then you could select them from your code in this way:
var catTrees = db.products_category.Where(pc => pc.Depth < 3).ToList();
There are many ways to populate that new column, but the bottom line is that you will have to do it just once (given that you keep track of it every time you modify the navpath of a category).
One possible way of populating it would be looping through all categories, something like:
var allCategories = db.products_category.ToList();
foreach(var category in allCategories)
{
var depth = category.navpath == "" ? 0 : category.navpath.Split('-').Length + 1;
category.Depth = depth;
}
allCategories.SubmitChanges();
Related
I am currently preparing my Application to transition over to MVC and in doing so have replaced all SQL statements with LINQ (EF) and removing all Datasets/DataTables, replacing them with strongly typed Lists.
I am stuck on one scenario where I need to pivot a strongly typed List<T> , after I pivot (number of columns produced vary) I am attempting to re-assign the results back to the GridView, keeping in mind that I don't want to use a DataTable.
I have looked at various examples where people are attempting to use ExpandoObject but I can't get it to work and continue to get this error:
The data source for GridView with id 'GridReport' did not have any properties or attributes from which to generate columns. Ensure that your data source has content.
The alternative would be to create some kind of class dynamically with properties getter and setter, would this be the right approach?
Given that eventually I will discard GridView too in MVC (controls not supported) I am now just thinking to maybe create an output just using HTML table? Since all I am doing is outputting the display and not using the GridView for any other purpose.
Some guidance and code example would help for the right scenario.
My List <T> looks like this (shortened for simplicity) and I pivot on ticker_id using a GroupBy. Am I able to return the property names from the Linq query too? if so how?:
public class CorporationCompare
{
public int ticker_id { get; set; }
public string tickerSymbol { get; set; }
public decimal? price { get; set; }
}
//pivot
var query = (from item in lstCompareCorp
let key = new { ticker_id = item.ticker_id }
group new { tickerSymbol = item.tickerSymbol, price = item.price } by key)
.ToList();
Before Pivot:
ticker_id tickerSymbol price
1 GOOG 123.45
208 AAPL 543.21
After Pivot:
ticker_id 1 208
tickerSymbol GOOG AAPL
price 123.45 543.21
I want to iterate through my table, check to see if the Quantity received is higher than the quantity expected for each row, and if so execute some code.
The way I've done it below feels amateurish.
bool allRecieved = true;
foreach (DataRow row in SubVwr.Tables[0].Tbl.Rows)
{
if(row["QtyRcvd"] < row["QtyPer"]) allRecieved = false;
}
if(allRecieved) // execute code
You can use LINQ, for better readability (presuming those are int-columns):
bool allRecieved = SubVwr.Tables[0].Tbl.AsEnumerable()
.All(row => row.Field<int>("QtyRcvd") >= row.Field<int>("QtyPer"));
An advantage over your current loop is that this will stop as soon as one record doesn't match this condition. Your loop will continue until end without break.
This is a bit radical, but I'd start by not using DataTable, and instead use a simple model:
public class SomeType {
// I'd probably name these QuantityReceived etc, but... meh
public int QtyRcvd {get;set;}
public int QtyPer {get;set;}
// ... etc
}
Then I can very conveniently check properties etc. For example, to mirror Tim Schmelter's LINQ answer:
List<SomeType> someData = ...
var allReceived = someData.All(x => x.QtyRcvd >= x.QtyPer);
so now all we need is to load SomeType from the DB, which is where ORMs and micro-ORMs excel. For example, with "Dapper", this would be:
string region = "North"; // just to show parameter usage
var someData = connection.Query<SomeType>(
#"select * from SomeTable where Region=#region", new { region }).AsList();
I have an ASP.NET MVC web application.
The SQL table has one column ProdNum and it contains data such as 4892-34-456-2311.
The user needs a form to search the database that includes this field.
The problem is that the user wants to have 4 separate fields in the UI razor view whereas each field should match with the 4 parts of data above between -.
For example ProdNum1, ProdNum2, ProdNum3 and ProdNum4 field should match with 4892, 34, 456, 2311.
Since the entire search form contains many fields including these 4 fields, the search logic is based on a predicate which is inherited from the PredicateBuilder class.
Something like this:
...other field to be filtered
if (!string.IsNullOrEmpty(ProdNum1) {
predicate = predicate.And(
t => t.ProdNum.toString().Split('-')[0].Contains(ProdNum1).ToList();
...other fields to be filtered
But the above code has run-time error:
The LINQ expression node type 'ArrayIndex' is not supported in LINQ to Entities`
Does anybody know how to resolve this issue?
Thanks a lot for all responses, finally, I found an easy way to resolve it.
instead of rebuilding models and change the database tables, I just add extra space in the search strings to match the search criteria. since the data format always is: 4892-34-456-2311, so I use Startwith(PODNum1) to search first field, and use Contains("-" + PODNum2 + "-") to search second and third strings (replace PODNum1 to PODNum3), and use EndWith("-" + PODNum4) to search 4th string. This way, I don't need to change anything else, it is simple.
Again, thanks a lot for all responses, much appreciated.
If i understand this correct,you have one column which u want to act like 4 different column ? This isn't worth it...For that,you need to Split each rows column data,create a class to handle the splitted data and finally use a `List .Thats a useless workaround.I rather suggest u to use 4 columns instead.
But if you still want to go with your existing applied method,you first need to Split as i mentioned earlier.For that,here's an example :
public void test()
{
SqlDataReader datareader = new SqlDataReader;
while (datareader.read)
{
string part1 = datareader(1).toString.Split("-")(0);///the 1st part of your column data
string part2 = datareader(1).toString.Split("-")(1);///the 2nd part of your column data
}
}
Now,as mentioned in the comments,you can rather a class to handle all the data.For example,let's call it mydata
public class mydata {
public string part1;
public string part2;
public string part3;
public string part4;
}
Now,within the While loop of the SqlDatareader,declare a new instance of this class and pass the values to it.An example :
public void test()
{
SqlDataReader datareader = new SqlDataReader;
while (datareader.read)
{
Mydata alldata = new Mydata;
alldata.Part1 = datareader(1).toString.Split("-")(0);
alldata.Part2 = datareader(1).toString.Split("-")(1);
}
}
Create a list of the class in class-level
public class MyForm
{
List<MyData> storedData = new List<MyData>;
}
Within the while loop of the SqlDatareader,add this at the end :
storedData.Add(allData);
So finally, u have a list of all the splitted data..So write your filtering logic easily :)
As already mentioned in a comment, the error means that accessing data via index (see [0]) is not supported when translating your expression to SQL. Split('-') is also not supported hence you have to resort to the supported functions Substring() and IndexOf(startIndex).
You could do something like the following to first transform the string into 4 number strings ...
.Select(t => new {
t.ProdNum,
FirstNumber = t.ProdNum.Substring(0, t.ProdNum.IndexOf("-")),
Remainder = t.ProdNum.Substring(t.ProdNum.IndexOf("-") + 1)
})
.Select(t => new {
t.ProdNum,
t.FirstNumber,
SecondNumber = t.Remainder.Substring(0, t.Remainder.IndexOf("-")),
Remainder = t.Remainder.Substring(t.Remainder.IndexOf("-") + 1)
})
.Select(t => new {
t.ProdNum,
t.FirstNumber,
t.SecondNumber,
ThirdNumber = t.Remainder.Substring(0, t.Remainder.IndexOf("-")),
FourthNumber = t.Remainder.Substring(t.Remainder.IndexOf("-") + 1)
})
... and then you could simply write something like
if (!string.IsNullOrEmpty(ProdNum3) {
predicate = predicate.And(
t => t.ThirdNumber.Contains(ProdNum3)
I have a scenario where I need to update few items based on the data from another list. I have already gone through various questions over here but none helped.
Scenario
listA: Total Count around 88000
public class CDRs
{
public string cld { get; set; }
public string prefix2 { get; set; }
public string country { get; set; }
public string city { get; set; }
}
listB: Total Count : 3000.
public class RatesVM
{
public string prefix { get; set; }
public string Country { get; set; }
public string City { get; set; }
}
Now in listB there can be multiple matches of listA field that is cld
for eg.
listA.cld = "8801123232";
Matched prefixes from ListB I get is
880 BGD Proper
8801 BGD Mobile
88011 BGD Dhaka Mobile
88017 BGD Dhaka Mobile
88018 BGD Dhaka Mobile
88019 BGD Dhaka Mobile
Now I want the closest match in this case it would be
88011 BGD Dhaka Mobile
Approach I am following right now.
foreach (var x in listA)
{
var tempObj = listB.FirstOrDefault(y => x.cld.StartsWith(y.prefix));
if (tempObj != null)
{
x.prefix2 = tempObj.prefix;
x.country = tempObj.Country;
x.city = tempObj.City;
}
else
{
x.prefix2 = "InBound";
x.country = "Unknown";
x.city = "Unknown";
}
}
It works fine but takes a lot of time. Around 2-3 minutes for this case.
There are few scenarios where ListA will have around 1 million records. I am worried it will take forever.
Many Thanks in advance
I would suggest the below code. The key difference is using orderedListB to ensure that you get the most specific match possible (i.e. start with the longest prefixes first), as well as a Dictionary to cache results. *
Dictionary<string, RatesVM> cache = new Dictionary<string, RatesVM>();
var orderedListB = listB.OrderByDescending(z => z.prefix.Length).ToList();
foreach (var x in listA)
{
RatesVM cached;
cache.TryGetValue(x.cld, out cached);
var tempObj = cached ?? orderedListB.FirstOrDefault(z => x.cld.StartsWith(z.prefix));
if (tempObj != null)
{
if (cached == null)
{
cache.Add(x.cld, tempObj);
}
x.prefix2 = tempObj.prefix;
x.country = tempObj.Country;
x.city = tempObj.City;
}
else
{
x.prefix2 = "InBound";
x.country = "Unknown";
x.city = "Unknown";
}
}
You may also want to consider using Parallel.ForEach rather than just foreach.
Your problem is hard, because you need the 'closest' solution rather than any solution at all. This forces you to iterate of every record in listB, for each element in listA.
Since you need an answer for every element in listA you are forced to check every element in it.
You can however preprocess listB by creating a tree structure. You create a node for each different first number of all strings in B. Then that node will be the parent of all records in listB that start with that number. The nodes below that node will hold the second number in the string, and so on.
Went ahead and drew you a visual idea of what such a tree might look like:
Now if you search in listB, you don't have to iterate over the entire listB but can just traverse down the list which will increase your time per iteration from O(n) to O(log n).
You would take the first letter in a record in listA and compare it to the tree, and traverse in that branch (instantly eliminating a huge amount of records you otherwise would need to compare against, increasing your performance). Then compare the second letter, etc untill no more letters can be found in the tree. When you stop, you have found the longest matching record in listB guaranteeing the 'closest' match, which FirstOrDefault(\x -> x.StartsWith()) doesn't do at all! (It finds the first match only, which is almost always just the first letter!).
You only have to create this tree once for all searches in listA, and if there are changes in listB you can easily update the tree as well.
If you're running this on a decent machine with more than one core, you can also parallelize this search. It increases the complexity of the program you're writing because you need to manage which thread searches which record in listA though it will help out greatly with the performance and will greatly lower the amount of time needed.
I'm using Realm + Xamarin Forms to do what I figure is about the most basic operation possible: a list view shows a collection of items, with a search bar filtering the results as the user types.
I have a get only collection property used as the list view's items source, initially populated from a Realm query, and this gets updated automatically with any changes to data, but I can't figure out how to update the search text without adding a set and literally replacing the entire collection.
This is very inefficient--I assume this is triggering re-registration of a bunch of notify-changed event listeners for the collection and each item in it and generally causing mass chaos with each letter tapped.
In the past I've created my own wrapping observable collection with a search method to handle this and I suppose that is an option here as well, but is there any way to do this with Realm? That is, to update the query without recreating the entire collection, some way to re-run the original query?
Update: This technique not longer works.
https://github.com/realm/realm-dotnet/issues/1569
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...it also differs from the behavior of LINQ to Objects, where every iteration will reevaluate expressions, meaning that changes to both sides of a condition will affect the result. A Realm query will evaluate the right-hand sides of the conditions on the first run.
When you construct a query that contains Where parameters that are based upon non-Realm based conditions the query results do not update when those variable/parameters are changed unless you update/execute the query again.
Realm queries are live, in the sense that they will continue to represent the current state of the database.
So what I do is to create a filter class (RealmObject), then if you instance a "filter object", save it to Realm, you can base your Linq's Where parameters upon one or more of the "filter" properties. Updating this RealmObject filter via Realm.Add(filterObject, true) your queries based upon that object are also updated
Realm queries are live, in the sense that they will continue to represent the current state of the database.
The results are lighting fast filtering that works great in any UI Search routine.
Example Model:
public class ARealmClass : RealmObject
{
public int Key { get; set; }
public string KeyString { get; set; }
}
public class ARealmClassFilter : RealmObject
{
[PrimaryKey]
public int Key { get; set; }
public int FilterKeyBy { get; set; }
}
Populate a Realm with some test data
var realm = Realm.GetInstance();
var all = realm.All<ARealmClass>();
if (all.Count() == 0)
{
realm.Write(() =>
{
for (int i = 0; i < 1000; i++)
{
var obj = new ARealmClass { Key = i, KeyString = i.ToString() };
realm.Add(obj);
}
});
}
Dynamic Live Query Example:
var realm = Realm.GetInstance();
var all = realm.All<ARealmClass>();
Console.WriteLine(all.Count());
var filterItem = new ARealmClassFilter { Key = 1, FilterKeyBy = 500 };
realm.Write(() =>
{
realm.Add(filterItem);
});
var filtered = all.Where(_ => _.Key > filterItem.FilterKeyBy);
Console.WriteLine(filtered.Count());
realm.Write(() =>
{
filterItem.FilterKeyBy = 750;
realm.Add(filterItem, true);
});
Console.WriteLine(filtered.Count());
Output:
2017-04-24 11:53:20.376 ios_build_foo[24496:3239020] 1000
2017-04-24 11:53:20.423 ios_build_foo[24496:3239020] 499
2017-04-24 11:53:20.425 ios_build_foo[24496:3239020] 249
Note: Quoted text # https://realm.io/docs/xamarin/latest/api/linqsupport.html