Map reduce in RavenDb over 2 collections with child collection - c#

I have 2 different object types stored in RavenDb, which are a parent/child type relationship, like this in JSON:
Account/1
{
"Name": "Acc1",
}
Items/1
{
"Account": "Account/1",
"Value" : "100",
"Tags": [
"tag1",
"tag2"]
}
Items/2
{
"Account": "Account/1",
"Value" : "50",
"Tags": [
"tag2"]
}
Note that I don't want to store these in the same document, as an account may have thousands of items.
I am trying to write a map/reduce index that will return me something like:
{
"Account": "Acc1",
"TagInfo": [
{ "TagName" : "tag1",
"Count" : "1", //Count of all the "tag1" occurrences for acc1
"Value" : "100" //Sum of all the Values for acc1 which are tagged 'tag1'
},
{ "TagName" : "tag2",
"Count" : "2", //Two items are tagged "tag2"
"Value" : "150"
}]
}
i.e. a list of all the distinct tag names along with the number of each and their value.
I think I need to use a multi-map to map the Account and Items collections together, but I can't figure out the reduce part to create the "TagInfo" part of the result.
Is this possible, or am I modelling this all wrong in Raven?
EDIT:
The class I want to retrieve from this query would look something like this:
public class QueryResult
{
public string AccountId {get;set;}
public TagInfo Tags {get;set;}
}
public class TagInfo
{
public string TagName {get;set;}
public int Count {get;set;}
public int TotalSum {get;set;}
}

You can't use a Multi Map/Reduce index for that because you want one map on the tags and the other on the account. They don't have a common property, so you can't have a multi maps/reduce here.
However, you can use TransformResult instead. Here's how to do it:
public class Account
{
public string Id { get; set; }
public string Name { get; set; }
}
public class Item
{
public string Id { get; set; }
public string AccountId { get; set; }
public int Value { get; set; }
public List<string> Tags { get; set; }
}
public class TagsWithCountAndValues : AbstractIndexCreationTask<Item, TagsWithCountAndValues.ReduceResult>
{
public class ReduceResult
{
public string AccountId { get; set; }
public string AccountName { get; set; }
public string Tag { get; set; }
public int Count { get; set; }
public int TotalSum { get; set; }
}
public TagsWithCountAndValues()
{
Map = items => from item in items
from tag in item.Tags
select new
{
AccountId = item.AccountId,
Tag = tag,
Count = 1,
TotalSum = item.Value
};
Reduce = results => from result in results
group result by result.Tag
into g
select new
{
AccountId = g.Select(x => x.AccountId).FirstOrDefault(),
Tag = g.Key,
Count = g.Sum(x => x.Count),
TotalSum = g.Sum(x => x.TotalSum)
};
TransformResults = (database, results) => from result in results
let account = database.Load<Account>(result.AccountId)
select new
{
AccountId = result.AccountId,
AccountName = account.Name,
Tag = result.Tag,
Count = result.Count,
TotalSum = result.TotalSum
};
}
}
Later then, you can query like this:
var results = session.Query<TagsWithCountAndValues.ReduceResult, TagsWithCountAndValues>()
.Where(x => x.AccountId == "accounts/1")
.ToList();

OK, so I figured out a way to do this in an acceptable manner that builds on Daniel's answer, so I'll record it here for any future travellers (probably myself!).
I changed from trying to return one result per account, to one result per account/tag combination, so the index had to change as follows (note the group by in the reduce is on 2 properties):
public class TagsWithCountAndValues : AbstractIndexCreationTask<Item, TagsWithCountAndValues.ReduceResult>
{
public class ReduceResult
{
public string AccountId { get; set; }
public string AccountName { get; set; }
public string TagName { get; set; }
public int TagCount { get; set; }
public int TagValue { get; set; }
}
public TagsWithCountAndValues()
{
Map = items => from item in items
from tag in item.Tags
select new ReduceResult
{
AccountId = item.AccountId,
TagName = tag,
TagCount = 1,
TagValue = item.Value
};
Reduce = results => from result in results
where result.TagName != null
group result by new {result.AccountId, result.TagName}
into g
select new ReduceResult
{
AccountId = g.Key.AccountId,
TagName = g.Key.TagName,
TagCount = g.Sum(x => x.TagCount),
TagValue = g.Sum(x => x.TagValue),
};
TransformResults = (database, results) => from result in results
let account = database.Load<Account>(result.AccountId)
select new ReduceResult
{
AccountId = result.AccountId,
AccountName = account.Name,
TagName = result.TagName,
TagCount = result.TagCount,
TagValue = result.TagValue,
};
}
}
As before, querying this is just:
var results = session
.Query<TagsWithCountAndValues.ReduceResult, TagsWithCountAndValues>()
.ToList();
The result of this can then be transformed into the object I originally wanted by an in-memory LINQ query. At this point the number of results that could be returned would be relatively small, so performing this at the client end is easily acceptable. The LINQ statement is:
var hierachicalResult = from result in results
group new {result.TagName, result.TagValue} by result.AccountName
into g
select new
{
Account = g.Key,
TagInfo = g.Select(x => new { x.TagName, x.TagValue, x.TagCount })
};
Which gives us one object per account, with a child list of TagInfo objects - one for each unique tag.

Related

C# group by a column and form hierarchical data with other columns

I am trying to group a column and form the the rest of the columns as child, hierarchical data:
I am trying to group by Code and form the parent and child relationship from a flat list, below is the hierarchical data I am trying to form:
source list:
public class ItemAssignmentFlatList
{
public int Code { get; set; }
public string Type { get; set; }
public string Description { get; set; }
public int ItemCode{ get; set; }
public DateTime EffectiveDate{ get; set; }
public string Area{ get; set; }
public string TaxCode{ get; set; }
public string LocationId { get; set; }
}
Need to convert above flat list into below List of hierarchical data:
public class ItemInfo
{
public int Code { get; set; }
public string Type { get; set; }
public string Description { get; set; }
public List<TaxInfo> TaxPlan { get; set; }
}
public class TaxPlan
{
public int ItemCode{ get; set; }
public DateTime EffectiveDate{ get; set; }
public string Area{ get; set; }
public string TaxCode{ get; set; }
public string LocationId { get; set; }
}
Need hierarchical list with above flat data list with C# extension methods.
I have below code, but looking for clean code to reduce number of lines:
var items= results.GroupBy(x => new { x.Code, x.Type });
List<ItemInfo> result = new List<ItemInfo>();
foreach (var group in items)
{
var taxPlans = group.
Select(y => new TaxPlan
{
TaxArea = y.TaxArea,
ItemCode = y.ItemCode
});
var itemInfo= new ItemInfo
{
Code = group.FirstOrDefault().Code,
Type = group.FirstOrDefault().Type,
Description = group.FirstOrDefault().Description,
TaxPlan = taxPlans.ToList()
};
result.Add(itemInfo);
}
Something like this?:
var input = new List<ItemAssignmentFlatList>(){
new ItemAssignmentFlatList{
Code = 1,
Area = "a"
},
new ItemAssignmentFlatList{
Code = 1,
Area = "b"
},
new ItemAssignmentFlatList{
Code = 2,
Area = "c"
}
};
input
.GroupBy(
x => x.Code,
(int code, IEnumerable<ItemAssignmentFlatList> items) =>
{
var first = items.FirstOrDefault();
var key = new ItemInfo
{
Code = first.Code
//, ...
};
var plan = items.
Select(y => new TaxPlan
{
Area = y.Area
//, ...
});
return new
{
key = key,
items = plan
};
}
).Dump();
Whenever you have a sequence of similar object, and you want to make "Items with their SubItems", based on common properties in your source sequence, consider to use one of the overloads of Enumerable.GroupBy
Because you don't just want "Groups of source items" but you want to specify your output, consider to use the overload that has a parameter resultSelector.
parameter keySelector: what should all elements in a group have in common
parameter resultSelector: use the common thing, and all elements that have this common thing to make one output element.
.
IEnumerable<ItemAssignmentFlatList> flatItemAssignments = ...
IEnumerable<ItemInfo> items = flatItemAssignments
// make groups with same {Code, Type, Description}
.GroupBy(flatItemAssignment => new {Code, Type, Description},
// parameter resultSelector: take the common CodeTypeDescription,
// and all flatItemAssignments that have this common value
// to make one new ItemInfo
(codeTypeDescription, flatItemAssignmentsWithThisCodeTypeDescription) => new ItemInfo
{
Code = codeTypeDescription.Code,
Type = codeTypeDescription.Type,
Description = codeTypeDescription.Description,
TaxPlans = flatItemAssignmentsWithThisCodeTypeDescription
.Select(flatItemAssignment => new TaxPlan
{
ItemCode = flatItemAssignment.ItemCode,
EffectiveDate = flatItemAssignment.EffectiveDate,
Area = flatItemAssignment.Area,
...
})
.ToList(),
});

Query separate collection in RavenDB Index (WHERE IN)

Using RavenDB v4.2 or higher, I want to setup an index that queries another collection. Basically, reproduce a WHERE IN clause in the mapping part of the index.
The models below represent two collections. Here each User has a collection of Device ID's:
class Device {
public string Id { get; set; }
public string Name { get; set; }
}
class User {
public string Id { get; set; }
public string BlogPostId { get; set; }
public List<string> DeviceIds { get; set; }
}
Now consider the following index as an example on what I'm trying to achieve:
public class DeviceIndex : AbstractIndexCreationTask<Device, DeviceIndex.Result>
{
public class Result
{
public string Id { get; set; }
public string DeviceName { get; set; }
public bool HasUser { get; set; }
public int UserCount { get; set; }
}
public DeviceIndex()
{
Map = devices => from d in devices
select new Result
{
Id = d.Id,
DeviceName = d.Name,
HasUser = ... ?, // How to get this from Users collection?
UserCount = ... ? // same...
};
}
How do I fill the HasUser true/false and UserCount properties in this index? E.g. how can I query the 'User' collection here?
Please note that this example is seriously simplified for brevity. I'm not so much interested in workarounds, or changing the logic behind it.
As #Danielle mentioned you need to use a mutli-map-index and reduce the result.
Here is a working example
public class DeviceIndex : AbstractMultiMapIndexCreationTask<DeviceIndex.Result>
{
public class Result
{
public string Id { get; set; }
public string DeviceName { get; set; }
public bool HasUser { get; set; }
public int UserCount { get; set; }
}
public DeviceIndex()
{
AddMap<User>(users => from u in users
from deviceId in u.DeviceIds
let d = LoadDocument<Device>(deviceId)
select new Result
{
Id = d.Id,
HasUser = true,
UserCount = 1,
DeviceName = d.Name,
});
AddMap<Device>(devices => from d in devices
select new Result
{
Id = d.Id,
HasUser = false,
UserCount = 0,
DeviceName = d.Name,
});
Reduce = results => from result in results
group result by new { result.Id } into g
select new Result
{
Id = g.First().Id,
DeviceName = g.First().DeviceName,
HasUser = g.Any(e => e.HasUser),
UserCount = g.Sum(e => e.UserCount),
};
}
}
and you can call it like this
var result = await _session.Query<DeviceIndex.Result, DeviceIndex>().ToListAsync();
If you would have a Users List in the Device class List<string> Users
a list that contains the document ids from the Users collection then you could Index these Related documents.
See:
https://demo.ravendb.net/demos/csharp/related-documents/index-related-documents
Or do the opposite,
Create an index on the Users collection, and index the related Device info
Without changing current models,
You can create a Multi-Map Index to index data from different collections.
https://ravendb.net/docs/article-page/4.2/csharp/indexes/multi-map-indexes
https://ravendb.net/docs/article-page/4.2/csharp/studio/database/indexes/create-multi-map-index
https://ravendb.net/learn/inside-ravendb-book/reader/4.0/10-static-indexes-and-other-advanced-options#querying-many-sources-at-once-with-multimap-indexes

LINQ Where clause from array of strings

I have a list of class Products:
class Products
{
public string Name { get; set; }
public string Size { get; set; }
public string ProductId { get; set; }
public string Category { get; set; }
}
I would like to use one TextBox to search through any matching products utilizing a wildcard value. This would return me a list of items where all values in the search string are found somewhere in the four properties listed above.
As of now, I'm using string[] values = searchText.Split("*".ToCharArray) to seperate the values of the search string into an array of strings (based on an asterisk wildcard). From there, I get stumped, since I want to search for all values of the search string in all properties of the class.
I tried to figure it out using a complex LINQ statement, but I have not been able to figure it out how to make this work. I don't know how to build a Where statement when I don't know how many values I'm going need to test against my four properties.
So, if you're breaking search up into separate keywords, using * as the delimiter, which you've described in the comments, then this is how you do it:
var products = new List<Products>()
{
new Products()
{
Name = "theo frederick smith",
Size = "",
ProductId = "",
Category = "brown",
}
};
var searchText = "fred*brown";
var splits = searchText.Split("*".ToCharArray(), StringSplitOptions.RemoveEmptyEntries);
var results =
products
.Where(p => splits.All(s =>
p.Name.Contains(s)
|| p.Size.Contains(s)
|| p.ProductId.Contains(s)
|| p.Category.Contains(s)));
That matches the input.
Alternatively, if you really want a wildcard search, such as "fred*smith" (meaning that any one field must contain "fred" followed by zero or more characters and followed by "smith"), then this works:
var products = new List<Products>()
{
new Products()
{
Name = "theo frederick smith",
Size = "",
ProductId = "",
Category = "brown",
}
};
var searchText = "fred*smith";
var wildcard =
new Regex(
String.Join(".*",
searchText
.Split('*')
.Select(x => Regex.Escape(x))));
var results =
products
.Where(p => new []
{
p.Name, p.Size, p.ProductId, p.Category
}.Any(x => wildcard.IsMatch(x)));
Naively, you could write
products.Where(x=>x.Name.Contains(search)
|| x.Size.Contains(search)
|| x.ProductId.Contains(search)
|| x.Category.Contains(search))
You would be better off putting that logic in your Product class.
So you would have:
class Products
{
public bool Contains(string term) {
return Name.Contains(search) || Size.Contains(search) ||
ProductId.Contains(search) || Category.Contains(search)
}
public string Name { get; set; }
public string Size { get; set; }
public string ProductId { get; set; }
public string Category { get; set; }
}
And then simply products.Where(x=>x.Contains(search))
You could also use reflection to get all the property names and do a for each on each string and check for Contains.

LINQ: Getting item from dictionary by key and also its values then assign them to variable

In Parts class we have Data dictionary that contains key "Number" and value "1" for example. The key is always called "Number" and the value is always string of some number 1,2,3 etc. I want to assign to one variable (List) all items that has the key "number" with their values and then to group them by the id in Parts. So in the end the result should be the Id from Parts, Number and its value.
public class People
{
public List<Parts> Parts { get; set; }
}
public class Parts
{
public string Name {get;set;}
public string Id {get;set;}
public Dictionary<string,string> Data {get;set}
}
var msf = new People();
Currently my example that does not work properly with linq :
var temp = msf
.Parts
.Select(s => s.Data.Keys.Where(key => key.Contains("Number"))
.ToList()
.Select(s = > s.Value));
Can someone give me better solution for this scenario code with linq?
"People":[
"id":"1234567"
"Parts":[
"id":"234567",
"name":"Lqlq"
"Data":{
"number" : "1"
}
"id":"3424242",
"name":"Lqlq2"
"Data":{
"number" : "2"
}
]
]
This should give you a Dictionary<string, List<string>> containing a list of ID strings for each "Number" value:
var idsByNumber = msf.Parts.Where(p => p.Data.ContainsKey("number")) // filter for all that have a number
.Select(p => new { ID = p.ID, Number = p.Data["number"] }) // select ID and the number value
.GroupBy(x => x.Number) // group by number
.ToDictionary(g => g.Key, g => g.ToList()); // create dictionary number -> id list
Here's an alternative syntax.
var temp = from part in msf.Parts
where part.Data["Number"] == "2"
select part;
Usually is a good idea to ask your questions using an MCVE - here's some code that can be pasted in Linqpad:
void Main()
{
var msf = new People() {
Parts = new List<Parts> {
new Parts { Name = "Lqlq", Id = "234567", Data = new Dictionary<string, string> { { "Number", "1"} } },
new Parts { Name = "Lqlq2", Id = "3424242", Data = new Dictionary<string, string> { { "Number", "2"} } },
}
};
var temp = from part in msf.Parts
where part.Data["Number"] == "2"
select part
;
temp.Dump();
}
public class People
{
public List<Parts> Parts { get; set; }
}
public class Parts
{
public string Name { get; set; }
public string Id { get; set; }
public Dictionary<string, string> Data { get; set; }
}

How can I select from an included entity in LINQ and get a flat list similar to a SQL Join would give?

I have two classes:
public class Topic
{
public Topic()
{
this.SubTopics = new HashSet<SubTopic>();
}
public int TopicId { get; set; }
public string Name { get; set; }
public virtual ICollection<SubTopic> SubTopics { get; set; }
}
public class SubTopic
public int SubTopicId { get; set; }
public int Number { get; set; }
public int TopicId { get; set; }
public string Name { get; set; }
public virtual Topic Topic { get; set; }
}
What I would like to do is to get a Data Transfer Object output from LINQ that will show me. I do want to see the TopicId repeated if there is more than one SubTopic inside that topic:
TopicId Name SubTopicId Name
1 Topic1 1 SubTopic1
1 Topic1 2 SubTopic2
1 Topic1 3 SubTopic3
2 Topic2 4 SubTopic4
I tried to code a Linq statement like this:
var r = context.Topics
.Select ( s => new {
id = s.TopicId,
name = s.Name,
sid = s.SubTopics.Select( st => st.SubTopicId),
sidname = s.SubTopics.Select ( st => st.Name)
}).
ToList();
But this does not really work as it returns sid and sidname as lists.
How will it be possible for me to get a flat output showing what I need?
You need SelectMany to expand a nested collection, along these lines
var r = context.Topics.SelectMany(t => t.SubTopics
.Select(st => new
{
TopicID = t.TopicId,
TopicName = t.Name,
SubTopicID = st.SubTopicId,
SubTopicName = st.Name
}));
try this :
var r = context.Topics
.Select ( s => new {
id = s.TopicId,
name = s.Name,
sid = s.SubTopics.Where(st=>st.TopicId==s.TopicId).Select( st => st.SubTopicId ),
sidname = s.SubTopics..Where(st=>st.TopicId==s.TopicId).Select ( st => st.Name)
}).
ToList();
Hope it will help
#Sweko provided an answer that satisfies the exact output that you requested. However, this can be even simpler if you just return the subtopic intact. It may run a bit quicker as well, since you don't need to create a new object for each element in the result.
Lastly, it looks like you wanted your result set ordered. For completeness, I've added those clauses as well.
var r = context.Topics
.SelectMany( topic => topic.SubTopics )
.OrderBy(sub => sub.TopicId)
.ThenBy(sub => sub.SubTopicId);

Categories