We Keep Coding

using string.split in entity to traverse tree depth

i have the following self-referencing table
public partial class products_category
{
public long id { get; set; }
public string category_name { get; set; }
public string category_description { get; set; }
//self referencing to table id
public Nullable<long> Parent_Id { get; set; }
public string navPath {get; set; }
}
here string navpath contains all the leading parents for a child categroy, say:
"Clothes" = 1 Parent_id=null, navpath=""
"Silk" = 2 Parent_id=1 navpath="1"
"Silk Suit"=3 parent_id=2 navpath="1-2"
"Saree" =4 parent_id=3 navpath="1-2-3"
"Dress Material"=5 parent_id=1 navpath="1" and so on....
now as per this scenario i want to access the flattend tree for frther processing for a certain depth only say to level 2 or until level 4 depth of children associated with navpath.
my idea regarding this issue was to approach using linq to ef this way:
var catTrees = db.products_category.Where(pc => pc.navpath.Split('-').Length < 4).ToList();
i am using the following link to do further traversing and tree generation:
https://bitlush.com/blog/recursive-hierarchical-joins-in-c-sharp-and-linq
and it is doing a great work so far, the only issue is i dont want to pre select whole table for processing. i want to achieve paging and a certain level of depth for first iteration, so i can maintain performance in case of thousand of records. [think of this as a category hierarchy or blog/youtube comments hierarchy].
but using the above ef linq command is giving the following error:
The LINQ expression node type 'ArrayLength' is not supported in LINQ to Entities.
i checked with ef docs and other places in SO to know that string.split doesn't work with EF implicitly. but can we apply it using extension methods or can this tree selection have alternate approach without using string.split and hitting DB only ones?
please advice.

This looks like an issues with building SQL code out of your LINQ mpre specifically SQL which takes a string splits it on dash and counts the elements.
if you dont hate the idea of loading into memory then you can force anything :)
var catTrees = db.products_category.ToList().Where(pc => pc.navpath.Split('-').Length < 4).ToList();
The trick here is to force the execution of the SQL by adding the .ToList() when we want the data from the database. This is called realizing the data.
Even with that realization trick the count is faster
var catTrees = db.products_category.ToList().Where(pc => pc.navpath.Count(a => a == '-') < 3).ToList();
these solutions are essentially the same as
List<Result> filter() {
List<Result> r = new List<Result>();
foreach(var a in db.products_category) {
if(a.navpath.Count(a => a == '-') < 3) {
r.add(a);
}
}
return r;
}
When thinking about it the filter method is somewhat less memory intensive as it reads one and one and never stores everything in memory. (in theory at least, only a few really knows what the .NET compiler does in the shadows)

I would advice you against using the navpath for checking depth.
If you can change your model, you could add an additional numeric Depth field for each category and populate it according its navpath, then you could select them from your code in this way:
var catTrees = db.products_category.Where(pc => pc.Depth < 3).ToList();
There are many ways to populate that new column, but the bottom line is that you will have to do it just once (given that you keep track of it every time you modify the navpath of a category).
One possible way of populating it would be looping through all categories, something like:
var allCategories = db.products_category.ToList();
foreach(var category in allCategories)
{
var depth = category.navpath == "" ? 0 : category.navpath.Split('-').Length + 1;
category.Depth = depth;
}
allCategories.SubmitChanges();

Redis key partitioning practices with linked items

I'm using a Redis database and ServiceStack client for it. I have a class called "Post" which has a property GroupId. Now when I'm storing this class the key is "urn:post:2:groupid:123". Now if I want to find all posts related to one group i need to use SearchKeys("urn:*groupid:123") method to retrieve all posts related to one group. Is this best practice to use Redis DB or should I convert my post key into form of "urn:groupid:123"post:2" ? If so how I can achieve this?
Post class:
public class Post
{
public const string POST_INCREMENT_KEY = "POST_INCREMENT";
public string Id { get; set; }
public string Message { get; set; }
public string GroupId { get; set; }
public void BuildId(long uniqueId)
{
Id = uniqueId + ":groupid:" + GroupId;
}
}
Code for storing post:
var post = new Post
{
GroupId = groupId,
Message = Request.Form["message"]
};
post.BuildId(_client.Increment(Post.POST_INCREMENT_KEY, 1));
_client.Store(post);

The best practice in redis is to maintain an index of the relationship you want to query.
Manually maintaining an index in Redis
An index is just a redis SET containing the related Ids you want to maintain, given that you want to "retrieve all posts related to one group" I would maintain the following index:
const string GroupPostIndex = "idx:group>post:{0}";
So that everytime you store a post, you also want to update the index, e.g:
client.Store(post);
client.AddItemToSet(GroupPostIndex.Fmt(groupId), post.Id);
Note: Redis SET operations are idempotent in that adding an item/id multiple times to a SET will always result in there being only one occurrence of that item in the SET, so its always safe to add an item to the set whenever storing a POST without needing to check if it already exists.
Now when I want to retrieve all posts in a group I just need to get all the ids from the SET with:
var postIds = client.GetAllItemsFromSet(GroupPostIndex.Fmt(groupId));
Then fetch all the posts with those ids:
var posts = redis.As<Post>().GetByIds(postIds);
Using ServiceStack.Redis Related Entities API's
The above shows what's required to maintain an index in Redis yourself, but as this is a common use-case, ServiceStack.Redis also offers a high-level typed API that you can use instead.
Which lets you store related entities with:
client.As<Group>().StoreRelatedEntities(groupId, post);
Note: this also takes care of storing the Post
and retrieve them with:
var posts = client.As<Group>().GetRelatedEntities<Post>(groupId);
It also offers other convenience API's like quickly finding out how many posts there are within a given group:
var postsCount = client.As<Group>().GetRelatedEntitiesCount<Post>(groupId);
and deleting either 1 or all entities in the group:
client.As<Group>().DeleteRelatedEntity<Post>(groupId, postId);
client.As<Group>().DeleteRelatedEntities<Post>(groupId); //all group posts

Delete junction table record in Entity Framework Code First (M:M)

How do I delete a record in a junction table within Entity Framework 5?
When reverse engineering my DataContext, Entity Framework seems to have recognized my junction table and automatically added Collections to my Models to represent the M:M relationship. This is great when adding items, as I can simply build my entire Entity and everything gets inserted properly. Perfect.
However, I'm stumped on removing a relationship. For example, an Activity can have multiple Contacts associated to it, and this is linked using a junction table (dbo.ActivityContacts) that consists of the columns:
ActivityID
ContactID
Both my Activity and Contact models have been updated by EF with Collections to represent the other. For example, my Activity model looks like this:
public class Activity
{
public int ActivityID { get; set; }
public string Subject { get; set; }
public virtual ICollection<Contacts> Contacts { get; set; }
}
In a non-EF environment, I would simply delete the record from the junction table and move on with my day. However, it seems I cannot access the junction table directly using EF, so I'm a tad confused on how to remove the record (relationship).
How can I properly remove a record from a junction table in Entity Framework?

Agree with #Chris.
Another solution is to do:
context.Entry(activity).State = EntityState.Deleted;

Entity Framework should remove the record for you, if you remove the associated object from either side of the relationship.
Assuming you've obtained this Activity instance from your context and want to remove a specific Contact with a known ID:
unwantedContact = context.Contacts.Find(contactID);
myActivity.Contacts.Remove(unwantedContact);
context.SaveChanges();
Should delete the record in your junction table, unless I'm being daft.

ali golshani did a good job providing a solution. Let me try to expand on it a little more. In my scenario I have two list boxes where you can move items left or right (selected or not selected)
The 'dto' object below is sent from the client. It's checking the selected state for each item in the list. If anyone knows of any way to improve this any more please leave feedback.
file_appender selectedAppender = context.file_appender.Find(dto.Id);
int[] ids = dto.Loggers.Where(x => !x.Selected).Select(x => x.Id).ToArray();
var loggers_to_delete = selectedAppender.logger.Where(x => ids.Contains(x.id));
loggers_to_delete.ToList().ForEach(x =>
{
selectedAppender.logger.Remove(x);
});
ids = dto.Loggers.Where(x => x.Selected).Select(x => x.Id).ToArray();
var loggers_to_add = context.logger.Where(x => ids.Contains(x.id));
loggers_to_add.ToList().ForEach(x =>
{
selectedAppender.logger.Add(x);
});
Lets look at another example....This one is for a list box with embedded check boxes (a little simpler). Honestly this could probably be applied to the solution above to make easier to read code.
protected void saveRelatedConnectors(test_engine testEngine, List<int> connectorTypes)
var stepConnectorsToDelete = testEngine.step_connector.Where(x => (connectorTypes.Count == 0) ||
(connectorTypes.Count != 0 && !connectorTypes.Contains(x.id)));
stepConnectorsToDelete.ToList().ForEach(x =>
{
testEngine.step_connector.Remove(x);
});
var stepConnectorsToAdd = entities.step_connector.Where(x => connectorTypes.Contains(x.id));
stepConnectorsToAdd.ToList().ForEach(x =>
{
testEngine.step_connector.Add(x);
});
entities.SaveChanges();

contact_to_delete = context.Contacts.Find(contactID);
selected_activity = context.Activity.Find(ActivityID);
context.Entry(selected_activity).Collection("Activity").Load();
selected_activity.Contacts.Remove(contact_to_delete);
db.SaveChanges();

LINQ - convert group to array

I have a problem displaying most popular tags from database in view. I'm not sure about this question title so if someone has a better one please do rename it.
Scenario
I need to show recent posts, recent galleries and most popular tags on my index page. I decided to use tuples for this and it worked fine until I tried to show most popular tags.
Error
The model item passed into the dictionary is of type
'System.Tuple3[photoBlog.Models.Gallery[],photoBlog.Models.Post[],System.Linq.IGrouping2[System.Int32,photoBlog.Models.PostTag][]]',
but this dictionary requires a model item of type
'System.Tuple`3[photoBlog.Models.Gallery[],photoBlog.Models.Post[],photoBlog.Models.PostTag[]]'.
As you can see it gives my view wrong object type. I want expect it to be an array, but it gives me some kind of System.Linq.Grouping item.
Code
As you can see I convert it into array in my controller.
public ActionResult Index()
{
photoBlogModelDataContext _db = new photoBlogModelDataContext();
var posts = _db.Posts.OrderByDescending(x => x.DateTime).Take(4).ToArray();
var galleries = _db.Galleries.OrderByDescending(x => x.ID).Take(4).ToArray();
var posttags = _db.PostTags.GroupBy(x => x.TagID).OrderBy(x => x.Count()).Take(4).ToArray();
return View(Tuple.Create(galleries, posts, posttags));
}
My view is straight forward, note that this did work until I tried to add most popular tags.
#model Tuple<photoBlog.Models.Gallery[], photoBlog.Models.Post[], photoBlog.Models.PostTag[]>
#foreach (var tag in Model.Item3)
{
#tag.Tag.Name
}

You probably want
var posttags = _db.PostTags
.GroupBy(x => x.TagID)
.OrderBy(x => x.Count())
// Take each group and pass the first tag of the group
.Select(g => g.First())
.Take(4)
.ToArray();
At the moment you're passing e.g. all 8 instances of the most popular tag, all 5 instances of the second most popular, etc., each in their own group. I imagine you just want to pass an "example" of each tag.

I don't mean to completely dodge the casting issue, but how about simplifying a little:
public class PopularStatsViewModel{
public Gallery[] Galleries { get; set; }
public Post[] Posts { get; set; }
public PostTags[] Tags { get; set; }
}
New up a PopularStats instance and set the properties to the results of your db calls. Then in your view:
#model photoblog.Models.PopularStatsViewModel
This gets you past the casting problem, sure, but more importantly it makes it a little easier to test, gives you a little more freedom in refactoring/extending that stats object, and it's pretty clear what you're working with on the view when you're accessing the VM properties.

Try changing photoBlog.Models.PostTag[] to IGrouping<TagIDType, photoBlog.Models.PostTag>[]

How to create view model without sorting collections in memory

I have a view model (below).
public class TopicsViewModel
{
public Topic Topic { get; set; }
public Reply LastReply { get; set; }
}
I want to populate an IQueryable<TopicsViewModel> with values from my IQueryable<Topic> collection and IQueryable<Reply> collection. I do not want to use the attached entity collection (i.e. Topic.Replies) because I only want the last reply for that topic and doing Topic.Replies.Last() loads the entire entity collection in memory and then grabs the last one in the list. I am trying to stay in IQueryable so that the query is executed in the database.
I also don't want to foreach through topics and query replyRepository.Replies because looping through IQueryable<Topic> will start the lazy loading. I'd prefer to build one expression and have all the leg work done in the lower layers.
I have the following:
IQueryable<TopicsViewModel> topicsViewModel = from x in topicRepository.Topics
from y in replyRepository.Replies
where y.TopicID == x.TopicID
orderby y.PostedDate ascending
select new TopicsViewModel { Topic = x, LastReply = y };
But this isn't working. Any ideas how I can populate an IQueryable or IEnumerable of TopicsViewModel so that it queries the database and grabs topics and that topic's last reply? I am trying really hard to avoid grabbing all replies related to that topic. I only want to grab the last reply.
Thank you for any insight you have to offer.

Since nobody is answering I am going with a foreach solution for now. I figure foreaching through the topics that are eventually going to be lazy loaded anyway is far better than populating a collection of replies just so I can access the last object in the collection.
Here is what I did for now:
List<TopicsViewModel> topicsViewModelList = new List<TopicsViewModel>();
foreach (Topic topic in topics)
{
Reply lastReply = replyRepository.GetRepliesBy_TopicID(topic.TopicID).OrderBy(x => x.PostedDate).LastOrDefault();
topicsViewModelList.Add(new TopicsViewModel
{
Topic = topic,
LastReply = lastReply
});
}
I'm just loading my IQueryable<Topics> first, then looping through the final results (so as to ensure that proper paging of the data is done before looping) and loading in the last reply. This seems to avoid ever populating a collection of replies and instead grabs only the last reply for each topic.

from r in replies
group r by new { r.TopicId } into g
select new
{
TopicId = g.Key.TopicId,
LastReply = g.Max(p => p.PostedDate)
}