We Keep Coding

Dynamic linq filter children using pivot

I have business objects that look like the following:
class Project
{
public int ID
{
get;set;
}
public string ProjectName
{
get;set;
}
public IList<ProjectTag> ProjectTags
{
get;set;
}
}
class ProjectTag
{
public int ID
{
get;set;
}
public int ProjectID
{
get;set;
}
public string Name
{
get;set;
}
public string Value
{
get;set;
}
}
Example Data:
Project:
ID ProjectName
1 MyProject
ProjectTags:
ID ProjectID Name Value
1 1 Name 1 Value 1
2 1 Name 2 Value 2
3 1 Name 3 Value 3
Basically it's a way for our users to define their own columns on the Project. As a result, it's important to remember that I don't know the names of the ProjectTag entries at design time.
What I'm trying to accomplish is to give our users the ability to select projects based on search criteria using System.Linq.Dynamic. For instance, to select just the project in my example above, our users could enter this:
ProjectName == "MyProject"
The more complicated aspect is applying a filter to the ProjectTags. Our application currently allow users to do this in order to filter Projects by their ProjectTags:
ProjectTags.Any(Name == "Name 1" and Value == "Value 1")
That works, but starts to get a bit messy for end users to use. Ideally I'd like to write something that would let them do the following:
Name 1 == "Value 1"
Or if necessary (due to white space in the name), something like the following...
[Name 1] == "Value 1"
"Name 1" == "Value 1"
For lack of a better explanation, it seems like I want to do the equivalent of a SQL pivot on the ProjectTags, and then still be able to execute a where clause against that. I've looked at some of the questions on StackOverflow about pivots and dynamic pivoting, but I haven't found anything too useful.
I've also been thinking about looping through all the ProjectTag Names and building a dynamic query using a left join on each. I guess something like this:
select
Project.*,
Name1Table.Value [Name 1],
Name2Table.Value [Name 2],
Name3Table.Value [Name 3]
from
Project
left join ProjectTag Name1Table on Name = 'Name 1'
left join ProjectTag Name2Table on Name = 'Name 2'
left join ProjectTag Name3Table on Name = 'Name 3'
And then take that query and apply a where clause to it. But I'm not really sure how to do that in Linq as well as dealing with the white space in the name.
I also came across ExpandoObject. I thought possibly I could convert Project to an ExpandoObject. Then loop through all known ProjectTag names, adding each name to the ExpandoObject and, if that Project had a ProjectTag for that name, use that ProjectTag value as the value, else empty string. For example...
private static object Expand(
Project project,
List<string> projectTagNames)
{
var expando = new ExpandoObject();
var dictionary = (IDictionary<string, object>) expando;
foreach (var property in project.GetType()
.GetProperties())
{
dictionary.Add(property.Name, property.GetValue(project));
}
foreach (var tagName in projectTagNames)
{
var tagValue = project.ProjectTags.SingleOrDefault(p => p.Name.Equals(tagName));
dictionary.Add(tagName, tagValue?.Value ?? "");
}
return expando;
}
The exciting thing about this solution is I have an object that looks exactly like I think it should prior to filtering with a where clause. It even seems to accommodate spaces in the property name.
Then of course I found out that dynamic linq doesn't work nicely with ExpandoObject, and so it can't find the dynamic properties. I guess that's because it essentially has a type of Object which isn't going to define any of the dynamic properties. Maybe it's possible to generate a type at run time that matches? Even if that works, I don't think it can account for spaces in the Name.
Am I trying to accomplish too much with this functionality? Should I just tell the users to use syntax like ProjectTags.Any(Name == "Name1" and Value == "Value1")? Or is there some way to trick dynamic linq into understanding ExpandoObject? Seems like having a way to override the way dynamic linq resolves property names would be very handy.

How about using a translator to convert tag references?
I assume that tag names containing spaces will be surrounded by brackets ([]) and that Project field names are a known list.
public static class TagTranslator {
public static string Replace(this string s, Regex re, string news) => re.Replace(s, news);
public static string Surround(this string src, string beforeandafter) => $"{beforeandafter}{src}{beforeandafter}";
public static string SurroundIfMissing(this string src, string beforeandafter) => (src.StartsWith(beforeandafter) && src.EndsWith(beforeandafter)) ? src : src.Surround(beforeandafter);
public static string Translate(string q) {
var projectFields = new[] { "ID", "ProjectName", "ProjectTags" }.ToHashSet();
var opREStr = #"(?<op>==|!=|<>|<=|>=|<|>)";
var revOps = new[] {
new { Fwd = "==", Rev = "==" },
new { Fwd = "!=", Rev = "!=" },
new { Fwd = "<>", Rev = "<>" },
new { Fwd = "<=", Rev = ">=" },
new { Fwd = ">=", Rev = "<=" },
new { Fwd = "<", Rev = ">" },
new { Fwd = ">", Rev = "<" }
}.ToDictionary(p => p.Fwd, p => p.Rev);
var openRE = new Regex(#"^\[", RegexOptions.Compiled);
var closeRE = new Regex(#"\]$", RegexOptions.Compiled);
var termREStr = #"""[^""]+""|(?:\w|\.)+|\[[^]]+\]";
var term1REStr = $"(?<term1>{termREStr})";
var term2REStr = $"(?<term2>{termREStr})";
var wsREStr = #"\s?";
var exprRE = new Regex($"{term1REStr}{wsREStr}{opREStr}{wsREStr}{term2REStr}", RegexOptions.Compiled);
var tq = exprRE.Replace(q, m => {
var term1 = m.Groups["term1"].Captures[0].Value.Replace(openRE, "").Replace(closeRE, "");
var term1q = term1.SurroundIfMissing("\"");
var term2 = m.Groups["term2"].Captures[0].Value.Replace(openRE, "").Replace(closeRE, "");
var term2q = term2.SurroundIfMissing("\"");
var op = m.Groups["op"].Captures[0].Value;
if (!projectFields.Contains(term1) && !term1.StartsWith("\"")) { // term1 is Name, term2 is Value
return $"ProjectTags.Any(Name == {term1q} && Value {op} {term2})";
}
else if (!projectFields.Contains(term2) && !term2.StartsWith("\"")) { // term2 is Name, term1 is Value
return $"ProjectTags.Any(Name == {term2q} && Value {revOps[op]} {term1})";
}
else
return m.Value;
});
return tq;
}
}
Now you just translate your query:
var q = "ProjectName == \"Project1\" && [Name 1] == \"Value 1\" && [Name 3] == \"Value 3\"";
var tq = TagTranslator.Translate(q);

Passing multiple values to same column in the lambda query C#

Hello I am trying to create a query where in the where clause i need to pass multiple values to the same column and i have built a query using string builder.
My actual Lambda query was
crentitiesbkp.Filter = obj =>
{
SPFetchCREntity entity = obj as SPFetchCREntity;
return obj.ToString() != null && entity.SW_Version==getmuid;
};
Here i changed the entity.SW_Version==getmuid like this
crentitiesbkp.Filter = obj =>
{
SPFetchCREntity entity = obj as SPFetchCREntity;
return obj.ToString() != null + cquery.ToString();
};
so the cquery is built and looks like this
&& entity.SW_Version=="TEST" && entity.SW_Version=="Result"
But its not filtering as per the query. Is this because this is passed as string ?
The cquery is built like this
foreach (var objj in storemuid)
{
cquery.Append(" && entity.SW_Version ==" +"\""+ objj.Value + "\"");
}

I resolved this scenario by using dictionary
Dictionary<string, List<SPFetchCREntity>> objDic = new Dictionary<string, List<SPFetchCREntity>>();
List<SPFetchCREntity> sp = new List<SPFetchCREntity>();
foreach (string str in objDic.Keys)
{
sp.AddRange(objDic[str]);
}
CRmappings2 = new ObservableCollection<SPFetchCREntity>(sp.ToList());
crentitiesbkp = CollectionViewSource.GetDefaultView(CRmappings2);
}
So now based on my checkbox selection i remove and add values from dictionary which works perfect for me now. This can be of any help to starter programmers like me .

Serializing a list of dynamic objects to a CSV with ServiceStack.Text

All of my EF classes have a Projection() method that helps me choose what I want to project from the class to the SQL queries:
Example:
public static Expression<Func<Something, dynamic>> Projection()
{
return e => new
{
something = e.SomethingId,
name = e.Name,
requiredThingId = e.RequiredThingId,
requiredThing = new
{
requiredThingId = e.RequiredThing.RequiredThingId,
name = e.RequiredThing.Name,
...
},
optionalThingId = e.OptionalThingId,
optionalThing = e.OptionalThingId == null ? null : new
{
optionalThingId = e.OptionalThing.OptionalThingId,
name = e.OptionalThing.Name
}
...
};
}
I use these projection methods like this:
List<dynamic> projected = dbContext.Something.Select(Something.Projection()).ToList();
This way lets me reuse my projections all around my project.
I want to use ServiceStack.Text to serialize these lists to CSV.
I'm doing this:
string csvString = CsvSerializer.SerializeToCsv<dynamic>(Something.Select(Something.Projection()).ToList());
byte[] csvBytes = System.Text.Encoding.Unicode.GetBytes(csvString);
return File(csvBytes, "text/csv", "foo.csv");
But the result is an empty csv (csvString is full of "\r\n"'s and nothing more)
Questions:
Am I using correctly the SerializeToCsv() method?
Can I use <dynamic> in this library?
Suggestions for achieving my purpose?
For the example above the desired CSV would flatten all the properties, something like:
"somethingId";"name";"requiredThingId";"requiredThing.requiredThingId";"requiredThing.name";"optionalThingId";"optionalThing.optionalThingId";"optionalThing.name"

I will answer my own questions, but will not mark as accepted in hope of a new greater answer..
Am I using correctly the SerializeToCsv() method? Can I use dynamic
in this library?
Answer: Yes to both but ServiceStack.Text v4.0.36 is needed (which is available at MyGet, not Nuget)
Suggestions for achieving my purpose?
Answer: With ServiceStack.Text it is possible to serialize to CSV a List<dynamic>, but all the nested properties will be rendered as JSON and they will not be flattened out, eg:
List<dynamic> list = new List<dynamic>();
list.Add(new
{
name = "john",
pet = new
{
name = "doggy"
}
});
string csv = CsvSerializer.SerializeToCsv(list);
This list will be serialized to this csv:
name, pet
"john", { name = "doggy" }
And not to this csv, as I was expecting:
name, pet_name
"john", "doggy"
So... I finally ended up writing this code:
public class CsvHelper
{
public static string GetCSVString(List<dynamic> inputList)
{
var outputList = new List<Dictionary<string, object>>();
foreach (var item in inputList)
{
Dictionary<string, object> outputItem = new Dictionary<string, object>();
flatten(item, outputItem, "");
outputList.Add(outputItem);
}
List<string> headers = outputList.SelectMany(t => t.Keys).Distinct().ToList();
string csvString = ";" + string.Join(";", headers.ToArray()) + "\r\n";
foreach (var item in outputList)
{
foreach (string header in headers)
{
if (item.ContainsKey(header) && item[header] != null)
csvString = csvString + ";" + item[header].ToString();
else
csvString = csvString + ";";
}
csvString = csvString + "\r\n";
}
return csvString;
}
private static void flatten(dynamic item, Dictionary<string, object> outputItem, string prefix)
{
if (item == null)
return;
foreach (PropertyInfo propertyInfo in item.GetType().GetProperties())
{
if (!propertyInfo.PropertyType.Name.Contains("AnonymousType"))
outputItem.Add(prefix + "__" + propertyInfo.Name, propertyInfo.GetValue(item));
else
flatten(propertyInfo.GetValue(item), outputItem, (prefix.Equals("") ? propertyInfo.Name : prefix + "__" + propertyInfo.Name));
}
}
}
What this does is:
It flattens the List, so that all the properties of the objects in the list are primitives (eg: no nested properties)
It creates a CSV from that flattened list.
This algorithm is O(n*m), being
n: number of items in the list
m: number of properties inside each item (including nested properties)

Whilst the recommendation is to use clean POCO's for Serialization, you can serialize in-line anonymous types, e.g:
var somethings = dbContext.Something.Select(e => new {
something = e.SomethingId,
name = e.Name
});
somethings.ToCsv().Print();
Otherwise I've just added support for serializing IEnumerable<object> and IEnumerable<dynamic> in this commit.
This change is available from v4.0.36+ that's now available on MyGet.

I believe that part of what is going on is that the library is using properties in the object to determine what to serialize. I believe that a dynamic object does not construct properties like it's a POCO. If you don't setup a getter and setter on your object, it certainly won't work. Just for reference, I using version 4.5.6.0 of this library.

Perform a linq expression for 'contains' with searching through a list for 'like' not exact matches

Okay so I am stumped and have looked around for this and I know I am doing the implementation of something very simple more complex than it needs to be. Basically I have a POCO object that will have a member that contains a string of other members. This is labeled as 'st' and it may have strings that are comma seperated series in one string. Thus I may have two members of strings be 'images, reports' and another 'cms, crm'. I have a list of objects that I want to match for PART OF those strings but not necessarily all as a DISTINCT LIST. So a member of 'cms' would return the value of anything that contained 'cms' thus 'cms, crm' would be returned.
I want to hook this up so a generic List can be queried but I cannot get it to work and was looking at other threads but there methods do not work in my case. I keep thinking it is something simple but I am missing it completely. Please let me know if anyone has better ideas. I was looking here but could not get the logic to apply correctly:
Linq query list contains a list
I keep trying methods of 'Select', 'SelectMany', 'Contains', 'Any', 'All' at different levels of scope of the continuations to no avail. Here is a simple excerpt of where I am at with a simple Console app example:
public class Program
{
public class StringModel
{
public string name { get; set; }
public string str { get; set; }
}
static void Main(string[] args)
{
string s = "";
List<StringModel> sm = new List<StringModel>
{
new StringModel
{
name = "Set1",
str = "images, reports"
},
new StringModel
{
name = "Set2",
str = "cms, crm"
},
new StringModel
{
name = "Set3",
str = "holiday, pto, cms"
}
};
sm.ForEach(x => s += x.name + "\t" + x.str + "\n");
var selected = new List<object> {"cms", "crm"};
s += "\n\nITEMS TO SELECT: \n\n";
selected.ForEach(x => s += x + "\n");
s += "\n\nSELECTED ITEMS: \n\n";
// works on a single item just fine
var result = sm.Where(p => p.str.Contains("cms")).Select(x => new { x.name, x.str}).ToList();
// I am not using select to get POCO on other methods till I can get base logic to work.
// Does not return anything
var result2 = sm.Where(p => selected.Any(x => x == p.str)).ToList();
// Does not return anything
var result3 = sm.Where(p => selected.Any(x => selected.Contains(p.str))).ToList();
result.ForEach(y => s += y + "\n");
s += "\n\n2nd SET SELECTED: \n\n";
result2.ForEach(y => s += y + "\n");
s += "\n\n3rd SET SELECTED: \n\n";
result3.ForEach(y => s += y + "\n");
Console.WriteLine(s);
Console.ReadLine();
}
}

result2 is empty because you're comparing an object (x) with a string (StringModel.str). This will be a reference comparison. Even if you convert x to a string, you'll be comparing each value in selected ("cms", "crm") with your comma-separated string values ("images, reports", "cms, crm", "holiday, pto, cms").
result3 is empty because selected ("cms", "crm") does not contain any of the string values ("images, reports", "cms, crm", "holiday, pto, cms"), although in this case at least the comparisons are value comparisons.
I think you're looking for something like:
var result = sm.Where(p => selected.Any(x => p.str.Contains((string)x)));

Inline if Statement not working

For some reason, when I add the ternary if statement to this bit of code, a NullPointerException is thrown. I'm not sure quite why...any ideas? This is the method for jqGrid - returning the Json data.
var gridModel = from entity in vendorList.AsQueryable()
select new
{
VendorId = "<a href='/Admin/DetailsPlan/" + entity.VendorId + "'><img src='/Images/next_icon_sm.png' class='icon' alt='View Vendor' /></a>",
VendorNm = entity.VendorNm,
Phone = (entity.Phone.Length < 5) ? String.Format("{0:(###) ###-####}", Convert.ToInt64(entity.Phone)) : entity.Phone,
City = entity.City,
State = entity.LkState.StateAbbr
};
Can you not have a ternary if statement in that location?

var gridModel = from entity in vendorList.AsQueryable()
let unformattedPhone = entity.Phone??string.Empty
select new
{
VendorId = "<a href='/Admin/DetailsPlan/" + entity.VendorId + "'><img src='/Images/next_icon_sm.png' class='icon' alt='View Vendor' /></a>",
VendorNm = entity.VendorNm,
Phone = (unformattedPhone.Length < 5) ? String.Format("{0:(###) ###-####}", Convert.ToInt64(unformattedPhone)) : unformattedPhone,
City = entity.City,
State = entity.LkState.StateAbbr
};
This may solve your problem.

One question, is entity.Phone null? If so, that would be the cause.
Side note: I have to say, that is an odd way of storing a phone number..
UPDATE
The problem is with the "entity.Phone.Length" part. If Phone is null, then you can't access it's length property... hence the error. So you need to add a null test. Something like:
Phone = ((entity.Phone != null) && (entity.Phone.Length < 5)) ? String.Format("{0:(###) ###-####}", Convert.ToInt64(entity.Phone)) : entity.Phone
That way, if it is null you are just emitting a null value.