Let me preface by saying I'm very new to C# development so if the solution seems obvious I apologize.
I'm getting a JSON string back from a user and I need to filter a list of C# objects based on what the JSON string contains. The JSON can only have fields that my C# model has but I don't know what fields the JSON string will contain. My C# model looks something like this:
public class Enrollment {
public int Year { get; set; }
public int NumEnrolls { get; set; }
public int DaysIntoEnrollment { get; set; }
public string Name { get; set; }
}
The JSON will have one or more of these properties with values to filter out. It could look like this:
{
"Year": ["2020", "2019"],
"Name": ["CourseA", "CourseB"],
"DaysIntoEnrollment": "20"
}
I need to filter my list of Enrollment objects based on the above JSON. So I would want the end result to have all Enrollment objects that don't contain a Year of 2020 or 2019 for example.
I've gotten a filter to work with linq on a single property but my real model has much more properties that can be filtered and I'm looking for a compact solution that will work regardless of which properties are included in the JSON. This is what I have working
public void GetFilteredData(string filters) {
var enrollList = new List<Enrollments>(); // Pretend this contains a list of valid Enrollment data
var json = JObject.Parse(filters); // filters string is in the json format from above
var propsToFilter =
from p in json["Year"]
select p;
var filtered = enrollList.Where(e => !propsToFilter.Contains(e.Year.ToString())));
}
Is there a simple way to do this without manually going through each property like I did above?
Related
I am using an api for a shopping cart that has some complex json (very complicated to me) data structured like in my screenshot below. In this scenario in my code I am trying to fix an error which I am going to explain by illustrating the data and how its structured as I am very new to JSON and arrays.
This is from the Visual Studio json reader of the data that belongs to an order placed by a customer. This item at the index of [0] has a customFields which has a value.
When a customer completes a purchase, some items they bought can have custom fields, like the size of a shirt (Large) or (Medium) or (Small) etc... In the JSON these customFields have a value which in this case is the size of the shirt for me to display at the thank you page so the customer knows what size he bought. Essentially I am trying to have the data ready to pass to the thank you page view.
When I am calling for these items in my controller, the code only works if ALL the items that were purchased have a customFields. If the customer buys something like a coffee mug that has NO custom fields, then the application breaks because I guess my code is only accounting for items that actually have customFields.
This is the code that I have so far that only works when ALL items that were purchased have a custom field. This is inside my controller.
public ActionResult Thankyou(string token)
{
int itemsCountAddedToCart = (int)obj["items"].Count();
var items = obj["items"].Select(o =>
new Item
{
name = o["name"].ToString(),
quantity = int.Parse(o["quantity"].ToString()),
price = double.Parse(o["price"].ToString()),
image = o["image"].ToString(),
url = o["url"].ToString(),
//This customFields is what works, but only if all items had custom fields.
customFields = o["customFields"][0]["value"].ToString(),
});
thankYouViewModel.OrderItems = items;
}
//ThankYou View Model that loads hold the data to be able to show in the view.
public class ThankYouViewModel
{
public IEnumerable<Item> OrderItems { get; set; }
}
public class Item
{
public string name { get; set; }
public double price { get; set; }
public int quantity { get; set; }
public string image { get; set; }
public string url { get; set; }
//customFields
public string customFields { get; set; }
}
So that code above works, but breaks when I have items that do not have customFields. This is the error that I get:
System.ArgumentOutOfRangeException: 'Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index'
So how should my code look where its currently breaking so that it can account for situations where one of the items from the JSON does not have a customFields attribute? I am very stuck and have tried to add some conditional statements but did not work because I am dealing with some complex json I do not understand very well yet.
If you want to forget the possibility of more than one element in the customFields array, and only cast the first element value to a string, then use this:
customFields = (o["customFields"] == null || o["customFields"].Count() == 0)?null:o["customFields"][0]["value"].ToString(),
With customFields = o["customFields"][0]["value"].ToString(), you directly receive the value from the customFields Array. If there is no Array in your case then there is nothing to get.
I would recommend you to check if your customFields exists:
var item = new Item ();
item.name = o["name"].ToString();
item.quantity = int.Parse(o["quantity"].ToString());
item.price = double.Parse(o["price"].ToString());
item.image = o["image"].ToString();
item.url = o["image"].ToString();
if(o["customFields"] != null)
{
item.customFields = o["customFields"][0]["value"].ToString();
}
I asked a question a couple of days ago to collect data from MongoDB as a tree.
MongoDB create an array within an array
I am a newbie to MongoDB, but have used JSON quite substantially. I thought using a MongoDB to store my JSON would be a great benefit, but I am just experiencing immense frustration.
I am using .NET 4.5.2
I have tried a number of ways to return the output from my aggregate query to my page.
public JsonResult GetFolders()
{
IMongoCollection<BsonDocument> collection = database.GetCollection<BsonDocument>("DataStore");
PipelineDefinition<BsonDocument, BsonDocument> treeDocs = new BsonDocument[]
{
// my query, which is a series of new BsonDocument
}
var documentGroup = collection.Aggregate(treeDocs).ToList();
// Here, I have tried to add it to a JsonResult Data,
// as both documentGroup alone and documentGroup.ToJson()
// Also, loop through and add it to a List and return as a JsonResult
// Also, attempted to serialise, and even change the JsonWriterSettings.
}
When I look in the Immediate Window at documentGroup, it looks exactly like Json, but when I send to browser, it is an escaped string, with \" surrounding all my keys and values.
I have attempted to create a model...
public class FolderTree
{
public string id { get; set; }
public string text { get; set; }
public List<FolderTree> children { get; set; }
}
then loop through the documentGroup
foreach(var docItem in documentGroup)
{
myDocs.Add(BsonSerializer.Deserialize<FolderTree>(docItem));
}
but Bson complains that it cannot convert int to string. (I have to have text and id as a string, as some of the items are strings)
How do I get my MongoDB data output as Json, and delivered to my browser as Json?
Thanks for your assistance.
========= EDIT ===========
I have attempted to follow this answer as suggested by Yong Shun below, https://stackoverflow.com/a/43220477/4541217 but this failed.
I had issues, that the "id" was not all the way through the tree, so I changed the folder tree to be...
public class FolderTree
{
//[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
//public string id { get; set; }
[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
public string text { get; set; }
public List<FolderTreeChildren> children { get; set; }
}
public class FolderTreeChildren
{
[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
public string text { get; set; }
public List<FolderTreeChildren> children { get; set; }
}
Now, when I look at documentGroup, I see...
[0]: {Plugins.Models.FolderTree}
[1]: {Plugins.Models.FolderTree}
To be fair to sbc in the comments, I have made so many changes to get this to work, that I can't remember the code I had that generated it.
Because I could not send direct, my json result was handled as...
JsonResult json = new JsonResult();
json.Data = documentGroup;
//json.Data = JsonConvert.SerializeObject(documentGroup);
json.JsonRequestBehavior = JsonRequestBehavior.AllowGet;
return json;
Note, that I also tried to send it as...
json.Data = documentGroup.ToJson();
json.Data = documentGroup.ToList();
json.Data = documentGroup.ToString();
all with varying failures.
If I leave as documentGroup, I get {Current: null, WasFirstBatchEmpty: false, PostBatchResumeToken: null}
If I do .ToJson(), I get "{ \"_t\" : \"AsyncCursor`1\" }"
If I do .ToList(), I get what looks like Json in json.Data, but get an error of Unable to cast object of type 'MongoDB.Bson.BsonInt32' to type 'MongoDB.Bson.BsonBoolean'.
If I do .ToString(), I get "MongoDB.Driver.Core.Operations.AsyncCursor`1[MongoDB.Bson.BsonDocument]"
=========== EDIT 2 =================
As this way of extracting the data from MongoDB doesn't want to work, how else can I make it work?
I am using C# MVC4. (.NET 4.5.2)
I need to deliver json to the browser, hence why I am using a JsonResult return type.
I need to use an aggregate to collect from MongoDB in the format I need it.
My Newtonsoft.Json version is 11.0.2
My MongoDB.Driver is version 2.11.1
My method is the simplest it can be.
What am I missing?
I've been wanting to deserialize automatically from a Cypher Result into their corresponding objects via neo4jclient.
These are the classes I want it to deserialize into
public class QuestionHub
{
public string Id { get; set; }
public string Title { get; set; }
public ICollection<Question> Questions {get;set;}
}
public class Question
{
public string Id { get; set; }
public string Title { get; set; }
public ICollection<Answer> Answers { get;set; }
}
public class Answer
{
public string Id { get; set; }
public string Value { get; set; }
}
I know for a fact that this code will put the corresponding QuestionHubs into a list of QuestionHubs. This is exactly what I want, but the problem is that those property that navigate to other classes, are not included.
var hubQuery = graphClient.Cypher
.Match("(qh:QuestionHub)-[:PART_OF]-(q:Question)-[:ANSWER]-(a:Answer)")
.ReturnDistinct<QuestionHub>("qh");
This is the
result
As you can see, the questions are not included.
Whenever I do this
var hubQuery = graphClient.Cypher
.Match("(qh:QuestionHub)-[:PART_OF]-(q:Question)-[:ANSWER]-(a:Answer)")
.ReturnDistinct<QuestionHub>("*");
I get the error
System.ArgumentException: 'Neo4j returned a valid response, however Neo4jClient was unable to deserialize into the object structure you supplied.
First, try and review the exception below to work out what broke.
If it's not obvious, you can ask for help at http://stackoverflow.com/questions/tagged/neo4jclient
Include the full text of this exception, including this message, the stack trace, and all of the inner exception details.
Include the full type definition of Neo4JTests.QuestionHub.
Include this raw JSON, with any sensitive values replaced with non-sensitive equivalents:
{ "columns":["a","q","qh"], "data":[[ {"data":{ "Value":"answer9aaf5134-9e73-4681-ba2f-e8224242ff19","Id":"9aaf5134-9e73-4681-ba2f-e8224242ff19" }},{"data":{ "Title":"questiond287a365-364a-4de0-b9f2-574893c1eaaa","Id":"d287a365-364a-4de0-b9f2-574893c1eaaa" }},{"data":{ "Title":"questionHub222a2fbe-6644-491a-b0a1-66df59f05f11","Id":"222a2fbe-6644-491a-b0a1-66df59f05f11" }} ]] } Arg_ParamName_Nam'
Inner Exception
InvalidOperationException: The deserializer is running in single column mode, but the response included multiple columns which indicates a projection instead. If using the fluent Cypher interface, use the overload of Return that takes a lambda or object instead of single string. (The overload with a single string is for an identity, not raw query text: we can't map the columns back out if you just supply raw query text.)
This error is probably because the cypher result gives multiple columns instead of one.
This is what I want to get
If I do this
var hubQuery = graphClient.Cypher
.Match("(u:User)-[:CREATOR_HUB]-(qh:QuestionHub)-[:PART_OF]-(q:Question)-[:ANSWER]-(a:Answer)")
.With("u, qh, q, COLLECT({Id: a.Id, Value: a.Value}) as answers")
.With("u, qh, COLLECT({Id: q.Id, Title: q.Title, Answers:answers}) as questions")
.With("{Creator: {Id:u.Id, FirstName: u.FirstName, LastName: u.LastName}," +
"Id: qh.Id, Title: qh.Title, Questions: questions} as result")
.ReturnDistinct<string>("result");
var hubQueryRes = await hubQuery .ResultsAsync;
List<QuestionHub> hubList = new List<QuestionHub>();
foreach (var hub in hubQueryRes )
{
hubList .Add(JsonConvert.DeserializeObject<QuestionHub>(hub));
}
I get what I want, but I needed to write all those .With
I want a way to automatically do that without all the .With writing.
I'm looking for a way to automatically deserialize the Cypher result into their corresponding objects with nested objects included.
Is there a way to do this?
Thanks in advance!
You're 100% correct that the reason it didn't create your QuestionHub instance is because the return was entirely in the wrong format for the client to cope with.
Unfortunately - your with workaround is about the only way - As you're using it to return the output of Cypher into a format that the Json Deserializer can handle.
The best I can see to do would be this:
var query = gc.Cypher
.Match("(qh:QuestionHub)-[:PART_OF]-(q:Question)-[:ANSWER]-(a:Answer)")
.With("qh, q{.*, Answers: COLLECT(a)} AS qAndA")
.With("qh{.*, Questions: COLLECT(qAndA)} AS result")
.Return(result => result.As<QuestionHub>());
Bear in mind, you would also need your ICollection to be either List or IEnumerable to deserialize properly - deserializing into ICollection isn't supported.
Let's say we have a realm results taken with
RealmDb.All<Entry>();
Then I want to do some search over those results using not yet supported techniques, like StartsWith on a function return or on a property which is not mapped in realm etc, so I get a subset
IEnumerable<Entry> subset = bgHaystack;
var results = subset.Where(entry => entry.Content.ToLower().StartsWith(needle));
To get somehow these as part of RealmResults, I extract the entry ids like this:
List<int> Ids = new List<int>();
foreach (Entry entry in entries)
{
Ids.Add(entry.Id);
}
return Ids;
and finally I want to return a subset of RealmResults (not IEnumerable) of only those Entries that contain those ids, how can I do that? IDE says the Contains method is not supported.
Can I use some kind of predicate or a comparer for that?
Entry is my model class
using System.ComponentModel.DataAnnotations.Schema;
using Realms;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System;
namespace Data.Models
{
[Table("entry")]
public class Entry : RealmObject
{
public class EntryType
{
public const byte Word = 1;
public const byte Phrase = 2;
public const byte Text = 3;
};
[Key]
[PrimaryKey]
[Column("entry_id")]
public int Id { get; set; }
[Column("user_id")]
public int UserId { get; set; }
[Column("source_id")]
public int SourceId { get; set; }
[Indexed]
[Column("type")]
public byte Type { get; set; }
[Column("rate")]
public int Rate { get; set; }
[Column("created_at")]
public string CreatedAt { get; set; }
[Column("updated_at")]
public string UpdatedAt { get; set; }
[NotMapped]
public Phrase Phrase { get; set; }
[NotMapped]
public Word Word { get; set; }
[NotMapped]
public Text Text { get; set; }
[NotMapped]
public IList<Translation> Translations { get; }
[NotMapped]
public string Content
{
get {
switch (Type)
{
case EntryType.Phrase:
return Phrase?.Content;
case EntryType.Word:
return Word?.Content;
case EntryType.Text:
return Text?.Content;
}
return "";
}
}
}
}
According to the documentation, Realm .NET supports LINQ, so that's promising. In your specific example, you indicate that StartsWith isn't supported, but I see that on the above page, specifically here.
Now, your example makes clear that Entry is a RealmObject, so it's not clear where you'd possibly get a RealmResult from (nor does their documentation on that page mention a RealmResult). Specifically, the home page indicates that you're really only going to ever work with Realm, RealmObject and Transaction, so I'm going to just assume that you meant that you'll need a resulting RealmObject per their examples.
The way you presently have your data object set up, you're rather stuck calling it like you are (though if I could make a recommendation to simplify it a little bit:
var entries = RealmDb.All<Entry>().ToList();
var results = entries.Where(entry => entry.Content.ToLower().StartsWith(needle));
var ids = results.Select(a => a.Id).ToList();
Now, your big issue with just combining the filter predicate in line 2 with the end of line 1: Content itself is marked with a [NotMapped] attribute. Per the documentation again:
As a general rule, you can only create predicates with conditions that
rely on data in Realm. Imagine a class
class Person : RealmObject
{
// Persisted properties
public string FirstName { get; set; }
public string LastName { get; set; }
// Non-persisted property
public string FullName => FirstName + " " + LastName;
}
Given this class, you can create queries with conditions that apply to
the FirstName and LastName properties but not to the FullName
property. Likewise, properties with the [Ignored] attribute cannot be
used.
Because you're using [NotMapped], I've got to believe that's going to behave similarly to [Ignored] and further, because it's just a computed value, it's not something that Realm is going to be able to process as part of the query - it simply doesn't know it because you didn't map it to the information Realm is storing. Rather, you'll have to compute the Content property when you've actually got the instances of your Entry objects to enumerate through.
Similarly, I expect you'll have issues pulling values from Phrase, Word and Text since they're also not mapped, and thus not stored in the record within Realm (unless you're populating those in code you didn't post before executing your Where filter).
As such, you might instead consider storing separate records as a PhraseEntry, WordEntry, and TextEntry so you can indeed perform exactly that filter and execute it on Realm. What if you instead used the following?
public class Entry : RealmObject
{
[Key]
[PrimaryKey]
[Column("entry_id")]
public int Id { get; set; }
[Column("user_id")]
public int UserId { get; set; }
[Column("source_id")]
public int SourceId { get; set; }
[Column("rate")]
public int Rate { get; set; }
[Column("created_at")]
public string CreatedAt { get; set; }
[Column("updated_at")]
public string UpdatedAt { get; set; }
[Column("content")]
public string Content { get; set; }
[NotMapped]
public IList<Translation> Translations { get; }
}
[Table("wordEntry")]
public class WordEntry : Entry
{
}
[Table("phraseEntry")]
public class PhraseEntry : Entry
{
}
[Table("textEntry")]
public class TextEntry : Entry
{
}
And now, you can offload the filtering to Realm:
var wordEntries = RealmDb.All<WordEntry>.Where(entry =>
entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var phraseEntries = RealmDb.All<PhraseEntry>.Where(entry => entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var textEntries = RealmDb.All<TextEntry>.Where(entry => entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var entries = new List<Entry>();
entries.AddRange(wordEntries);
entries.AddRange(phraseEntries);
entries.AddRange(textEntries);
var ids = entries.Select(entry => entry.Id).ToList();
It's not quite as brief as storing it all in one table, but I'm not immediately seeing any Realm documentation that indicates support for executing the same query against multiple tables simultaneously, so at least this would allow you to leave the filtering to the database and work against a more limited subset of values locally.
Finally, so we have all that and I missed your final question up top. You indicate that you want to return a subset of your entries based on some collection of ids you create. In the logic you provide, you're retrieving all the Id properties in all your results, so there's really no further subset to pull.
That said, let's assume you have a separate list of ids that for whatever complicated reason, you were only able to derive after retrieving the list of Entry types from above (themselves all PhraseEntry, WordEntry or TextEntry objects).
At this point, since you've already pulled all the values from Realm and have them locally, just execute another Where statement against them. Because a List implements IEnumerable, you can thus execute the LINQ locally without any of the Realm restrictions:
var myLimitedIdSet = new List<int>()
{
10, 15, 20, 25 //Really complicated logic to narrow these down locally
};
var resultingEntries = entries.Where(entry => myLimitedIdSet.Contains(entry.Id)).ToList();
And you're set. You'll have only those entries that match the IDs listed in myLimitedIdSet.
Edit to address comment
You see this error because of the detail provided at the top of this page in the documentation. Specifically (and adapting to your code):
The first statement gives you a new instance of Entry of a class that implements IQueryable... This is standard LINQ implementation - you get an object representing the query. The query doesn't do anything until you made a further call that needs to iterate or count the results.
Your error is then derived by taking the result from RealmDb.All<Entry>() and trying to cast it to an IEnumerable<Entry> to operate against it as though you have local data. Until you call ToList() onRealmDb.All` you simply have a LINQ representation of what the call will be, not the data itself. As such, when you further refine your results with a Where statement, you're actually adding that to a narrowed version of the IQueryable statement, which will also fail because you lack the appropriate mapping in the Realm dataset.
To skip the optimization I provided above, the following should resolve your issue here:
var bgHaystack = realm.All<Entry>().ToList(); //Now you have local data
var results = bgHaystack.Where(entry => entry.Content.ToLower().StartsWith(needle));
Unfortunately, given your provided code, I don't expect that you'll see any matches here unless needle is an empty string. Not only is your Content property not part of the Realm data and you thus cannot filter on it within Realm, but neither are your Phrase, Word or Text properties mapped either. As a result, you will only ever see an empty string when getting your Content value.
You can further refine the results variable above to yield only those instances with a provided ID as you see fit with normal LINQ (as again, you'll have pulled the data from Realm in the first line).
var limitedIds = new List<int>{10, 20, 30};
var resultsLimitedById = results.Select(a => limitedIds.Contains(a.Id)).ToList();
I've updated my examples above to reflect the use of ToList() in the appropriate places as well.
I am trying to use the fanart.tv webservice API but have a couple of issues.
I am using Json.Net (Newtonsoft.Json) and with other web-services I have de-serialized the JSON reponses into C# objects directly.
The issue here is that the element names are changing. Eg. if I search for artist thumbs for Metallica you get
{"Metallica":{"mbid_id":"65f4f0c5-ef9e-490c-aee3-909e7ae6b2ab","artistthumb": [{"id":"36181","url":"http://assets.fanart.tv/fanart/music/65f4f0c5-ef9e-490c-aee3-909e7ae6b2ab/artistthumb/metallica-4fd83b0129f83.jpg","likes":"1"},...]}}
So the root element is called Metallica. Obviously search for another artist and you get a different element name. After attempting to deserialize this to an object I gave up and as in reality all I needed was a list of strings (the urls) I tried to process the JSON
var obj = _downloader.Download<JObject>(url);
if (obj != null)
{
if (obj.HasValues)
{
var fanartArtist = (JProperty)obj.First;
if (fanartArtist.HasValues)
{
var thumbs = fanartArtist.Value[SearchSubTypeToString(subType)];
if (thumbs.HasValues)
{
thumbUrls.AddRange(thumbs.Select(thumb => thumb["url"].ToString()));
}
}
}
}
which works fine when there is a response but if there are no thumbs the web-service returns null and this code fails with
Unable to cast object of type 'Newtonsoft.Json.Linq.JValue' to type 'Newtonsoft.Json.Linq.JObject'.
To complicate matters slightly I am sort of limited by the application and ideally I need to use
JsonConvert.DeserializeObject<TE>(json);
So the question is what what is the best approach to solve both of these issues?
Try this:
First, define objects to hold the data deserialized from the JSON:
class Artist
{
public Guid mb_id { get; set; }
public List<Thumb> artistthumb { get; set; }
}
class Thumb
{
public int id { get; set; }
public string url { get; set; }
public int likes { get; set; }
}
You can then deserialize it like this (where json is a string containing the JSON data from the web service):
Dictionary<string, Artist> artists =
JsonConvert.DeserializeObject<Dictionary<string, Artist>>(json);
Once deserialized, you can access the data like this:
foreach (KeyValuePair<string, Artist> kvp in artists)
{
Console.WriteLine("Urls for " + kvp.Key + ":");
foreach (Thumb thumb in kvp.Value.artistthumb)
{
Console.WriteLine(thumb.url);
}
}
Assuming the data you showed in your question, the output would look like this:
Urls for Metallica:
http://assets.fanart.tv/fanart/music/65f4f0c5-ef9e-490c-aee3-909e7ae6b2ab909e7ae6b2ab/artistthumb/metallica-4fd83b0129f83.jpg
.