We Keep Coding

C# getting MongoDB BsonDocument to return as JSON

I asked a question a couple of days ago to collect data from MongoDB as a tree.
MongoDB create an array within an array
I am a newbie to MongoDB, but have used JSON quite substantially. I thought using a MongoDB to store my JSON would be a great benefit, but I am just experiencing immense frustration.
I am using .NET 4.5.2
I have tried a number of ways to return the output from my aggregate query to my page.
public JsonResult GetFolders()
{
IMongoCollection<BsonDocument> collection = database.GetCollection<BsonDocument>("DataStore");
PipelineDefinition<BsonDocument, BsonDocument> treeDocs = new BsonDocument[]
{
// my query, which is a series of new BsonDocument
}
var documentGroup = collection.Aggregate(treeDocs).ToList();
// Here, I have tried to add it to a JsonResult Data,
// as both documentGroup alone and documentGroup.ToJson()
// Also, loop through and add it to a List and return as a JsonResult
// Also, attempted to serialise, and even change the JsonWriterSettings.
}
When I look in the Immediate Window at documentGroup, it looks exactly like Json, but when I send to browser, it is an escaped string, with \" surrounding all my keys and values.
I have attempted to create a model...
public class FolderTree
{
public string id { get; set; }
public string text { get; set; }
public List<FolderTree> children { get; set; }
}
then loop through the documentGroup
foreach(var docItem in documentGroup)
{
myDocs.Add(BsonSerializer.Deserialize<FolderTree>(docItem));
}
but Bson complains that it cannot convert int to string. (I have to have text and id as a string, as some of the items are strings)
How do I get my MongoDB data output as Json, and delivered to my browser as Json?
Thanks for your assistance.
========= EDIT ===========
I have attempted to follow this answer as suggested by Yong Shun below, https://stackoverflow.com/a/43220477/4541217 but this failed.
I had issues, that the "id" was not all the way through the tree, so I changed the folder tree to be...
public class FolderTree
{
//[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
//public string id { get; set; }
[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
public string text { get; set; }
public List<FolderTreeChildren> children { get; set; }
}
public class FolderTreeChildren
{
[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
public string text { get; set; }
public List<FolderTreeChildren> children { get; set; }
}
Now, when I look at documentGroup, I see...
[0]: {Plugins.Models.FolderTree}
[1]: {Plugins.Models.FolderTree}
To be fair to sbc in the comments, I have made so many changes to get this to work, that I can't remember the code I had that generated it.
Because I could not send direct, my json result was handled as...
JsonResult json = new JsonResult();
json.Data = documentGroup;
//json.Data = JsonConvert.SerializeObject(documentGroup);
json.JsonRequestBehavior = JsonRequestBehavior.AllowGet;
return json;
Note, that I also tried to send it as...
json.Data = documentGroup.ToJson();
json.Data = documentGroup.ToList();
json.Data = documentGroup.ToString();
all with varying failures.
If I leave as documentGroup, I get {Current: null, WasFirstBatchEmpty: false, PostBatchResumeToken: null}
If I do .ToJson(), I get "{ \"_t\" : \"AsyncCursor`1\" }"
If I do .ToList(), I get what looks like Json in json.Data, but get an error of Unable to cast object of type 'MongoDB.Bson.BsonInt32' to type 'MongoDB.Bson.BsonBoolean'.
If I do .ToString(), I get "MongoDB.Driver.Core.Operations.AsyncCursor`1[MongoDB.Bson.BsonDocument]"
=========== EDIT 2 =================
As this way of extracting the data from MongoDB doesn't want to work, how else can I make it work?
I am using C# MVC4. (.NET 4.5.2)
I need to deliver json to the browser, hence why I am using a JsonResult return type.
I need to use an aggregate to collect from MongoDB in the format I need it.
My Newtonsoft.Json version is 11.0.2
My MongoDB.Driver is version 2.11.1
My method is the simplest it can be.
What am I missing?

How to save/pass MongoDB UpdateDefinition for logging and later use?

I am stumped on how to save/pass MongoDB UpdateDefinition for logging and later use
I have created general functions for MongoDB in Azure use on a collection for get, insert, delete, update that work well.
The purpose is to be able to have a standard, pre-configured way to interact with the collection. For update especially, the goal is to be able to flexibly pass in an appropriate UpdateDefinition where that business logic is done elsewhere and passed in.
I can create/update/set/combine the UpdateDefinition itself, but when i try to log it by serializing it, it shows null:
JsonConvert.SerializeObject(updateDef)
When I try to log it, save it to another a class or pass it to another function it displays null:
public class Account
{
[BsonElement("AccountId")]
public int AccountId { get; set; }
[BsonElement("Email")]
public string Email { get; set; }
}
var updateBuilder = Builders<Account>.Update;
var updates = new List<UpdateDefinition<Account>>();
//just using one update here for brevity - purpose is there could be 1:many depending on fields updated
updates.Add(updateBuilder.Set(a => a.Email, email));
//Once all the logic and field update determinations are made
var updateDef = updateBuilder.Combine(updates);
//The updateDef does not serialize to string, it displays null when logging.
_logger.LogInformation("{0} - Update Definition: {1}", actionName, JsonConvert.SerializeObject(updateDef));
//Class Created for passing the Account Update Information for Use by update function
public class AccountUpdateInfo
{
[BsonElement("AccountId")]
public int AccountId { get; set; }
[BsonElement("Update")]
public UpdateDefinition<Account> UpdateDef { get; set; }
}
var acct = new AccountUpdateInfo();
acctInfo.UpdateDef = updateDef
//This also logs a null value for the Update Definition field when the class is serialized.
_logger.LogInformation("{0} - AccountUpdateInfo: {1}", actionName, JsonConvert.SerializeObject(acct));
Any thoughts or ideas on what is happening? I am stumped on why I cannot serialize for logging or pass the value in a class around like I would expect

give this a try:
var json = updateDef.Render(
BsonSerializer.SerializerRegistry.GetSerializer<Account>(),
BsonSerializer.SerializerRegistry)
.AsBsonDocument
.ToString();
and to turn a json string back to an update definition (using implicit operator), you can do:
UpdateDefinition<Account> updateDef = json;
this is off the top of my head and untested. the only thing i'm unsure of (without an IDE) is the .Document.ToString() part above.

Using Contains() in a Realm query

Let's say we have a realm results taken with
RealmDb.All<Entry>();
Then I want to do some search over those results using not yet supported techniques, like StartsWith on a function return or on a property which is not mapped in realm etc, so I get a subset
IEnumerable<Entry> subset = bgHaystack;
var results = subset.Where(entry => entry.Content.ToLower().StartsWith(needle));
To get somehow these as part of RealmResults, I extract the entry ids like this:
List<int> Ids = new List<int>();
foreach (Entry entry in entries)
{
Ids.Add(entry.Id);
}
return Ids;
and finally I want to return a subset of RealmResults (not IEnumerable) of only those Entries that contain those ids, how can I do that? IDE says the Contains method is not supported.
Can I use some kind of predicate or a comparer for that?
Entry is my model class
using System.ComponentModel.DataAnnotations.Schema;
using Realms;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System;
namespace Data.Models
{
[Table("entry")]
public class Entry : RealmObject
{
public class EntryType
{
public const byte Word = 1;
public const byte Phrase = 2;
public const byte Text = 3;
};
[Key]
[PrimaryKey]
[Column("entry_id")]
public int Id { get; set; }
[Column("user_id")]
public int UserId { get; set; }
[Column("source_id")]
public int SourceId { get; set; }
[Indexed]
[Column("type")]
public byte Type { get; set; }
[Column("rate")]
public int Rate { get; set; }
[Column("created_at")]
public string CreatedAt { get; set; }
[Column("updated_at")]
public string UpdatedAt { get; set; }
[NotMapped]
public Phrase Phrase { get; set; }
[NotMapped]
public Word Word { get; set; }
[NotMapped]
public Text Text { get; set; }
[NotMapped]
public IList<Translation> Translations { get; }
[NotMapped]
public string Content
{
get {
switch (Type)
{
case EntryType.Phrase:
return Phrase?.Content;
case EntryType.Word:
return Word?.Content;
case EntryType.Text:
return Text?.Content;
}
return "";
}
}
}
}

According to the documentation, Realm .NET supports LINQ, so that's promising. In your specific example, you indicate that StartsWith isn't supported, but I see that on the above page, specifically here.
Now, your example makes clear that Entry is a RealmObject, so it's not clear where you'd possibly get a RealmResult from (nor does their documentation on that page mention a RealmResult). Specifically, the home page indicates that you're really only going to ever work with Realm, RealmObject and Transaction, so I'm going to just assume that you meant that you'll need a resulting RealmObject per their examples.
The way you presently have your data object set up, you're rather stuck calling it like you are (though if I could make a recommendation to simplify it a little bit:
var entries = RealmDb.All<Entry>().ToList();
var results = entries.Where(entry => entry.Content.ToLower().StartsWith(needle));
var ids = results.Select(a => a.Id).ToList();
Now, your big issue with just combining the filter predicate in line 2 with the end of line 1: Content itself is marked with a [NotMapped] attribute. Per the documentation again:
As a general rule, you can only create predicates with conditions that
rely on data in Realm. Imagine a class
class Person : RealmObject
{
// Persisted properties
public string FirstName { get; set; }
public string LastName { get; set; }
// Non-persisted property
public string FullName => FirstName + " " + LastName;
}
Given this class, you can create queries with conditions that apply to
the FirstName and LastName properties but not to the FullName
property. Likewise, properties with the [Ignored] attribute cannot be
used.
Because you're using [NotMapped], I've got to believe that's going to behave similarly to [Ignored] and further, because it's just a computed value, it's not something that Realm is going to be able to process as part of the query - it simply doesn't know it because you didn't map it to the information Realm is storing. Rather, you'll have to compute the Content property when you've actually got the instances of your Entry objects to enumerate through.
Similarly, I expect you'll have issues pulling values from Phrase, Word and Text since they're also not mapped, and thus not stored in the record within Realm (unless you're populating those in code you didn't post before executing your Where filter).
As such, you might instead consider storing separate records as a PhraseEntry, WordEntry, and TextEntry so you can indeed perform exactly that filter and execute it on Realm. What if you instead used the following?
public class Entry : RealmObject
{
[Key]
[PrimaryKey]
[Column("entry_id")]
public int Id { get; set; }
[Column("user_id")]
public int UserId { get; set; }
[Column("source_id")]
public int SourceId { get; set; }
[Column("rate")]
public int Rate { get; set; }
[Column("created_at")]
public string CreatedAt { get; set; }
[Column("updated_at")]
public string UpdatedAt { get; set; }
[Column("content")]
public string Content { get; set; }
[NotMapped]
public IList<Translation> Translations { get; }
}
[Table("wordEntry")]
public class WordEntry : Entry
{
}
[Table("phraseEntry")]
public class PhraseEntry : Entry
{
}
[Table("textEntry")]
public class TextEntry : Entry
{
}
And now, you can offload the filtering to Realm:
var wordEntries = RealmDb.All<WordEntry>.Where(entry =>
entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var phraseEntries = RealmDb.All<PhraseEntry>.Where(entry => entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var textEntries = RealmDb.All<TextEntry>.Where(entry => entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var entries = new List<Entry>();
entries.AddRange(wordEntries);
entries.AddRange(phraseEntries);
entries.AddRange(textEntries);
var ids = entries.Select(entry => entry.Id).ToList();
It's not quite as brief as storing it all in one table, but I'm not immediately seeing any Realm documentation that indicates support for executing the same query against multiple tables simultaneously, so at least this would allow you to leave the filtering to the database and work against a more limited subset of values locally.
Finally, so we have all that and I missed your final question up top. You indicate that you want to return a subset of your entries based on some collection of ids you create. In the logic you provide, you're retrieving all the Id properties in all your results, so there's really no further subset to pull.
That said, let's assume you have a separate list of ids that for whatever complicated reason, you were only able to derive after retrieving the list of Entry types from above (themselves all PhraseEntry, WordEntry or TextEntry objects).
At this point, since you've already pulled all the values from Realm and have them locally, just execute another Where statement against them. Because a List implements IEnumerable, you can thus execute the LINQ locally without any of the Realm restrictions:
var myLimitedIdSet = new List<int>()
{
10, 15, 20, 25 //Really complicated logic to narrow these down locally
};
var resultingEntries = entries.Where(entry => myLimitedIdSet.Contains(entry.Id)).ToList();
And you're set. You'll have only those entries that match the IDs listed in myLimitedIdSet.
Edit to address comment
You see this error because of the detail provided at the top of this page in the documentation. Specifically (and adapting to your code):
The first statement gives you a new instance of Entry of a class that implements IQueryable... This is standard LINQ implementation - you get an object representing the query. The query doesn't do anything until you made a further call that needs to iterate or count the results.
Your error is then derived by taking the result from RealmDb.All<Entry>() and trying to cast it to an IEnumerable<Entry> to operate against it as though you have local data. Until you call ToList() onRealmDb.All` you simply have a LINQ representation of what the call will be, not the data itself. As such, when you further refine your results with a Where statement, you're actually adding that to a narrowed version of the IQueryable statement, which will also fail because you lack the appropriate mapping in the Realm dataset.
To skip the optimization I provided above, the following should resolve your issue here:
var bgHaystack = realm.All<Entry>().ToList(); //Now you have local data
var results = bgHaystack.Where(entry => entry.Content.ToLower().StartsWith(needle));
Unfortunately, given your provided code, I don't expect that you'll see any matches here unless needle is an empty string. Not only is your Content property not part of the Realm data and you thus cannot filter on it within Realm, but neither are your Phrase, Word or Text properties mapped either. As a result, you will only ever see an empty string when getting your Content value.
You can further refine the results variable above to yield only those instances with a provided ID as you see fit with normal LINQ (as again, you'll have pulled the data from Realm in the first line).
var limitedIds = new List<int>{10, 20, 30};
var resultsLimitedById = results.Select(a => limitedIds.Contains(a.Id)).ToList();
I've updated my examples above to reflect the use of ToList() in the appropriate places as well.

How to use the Serialize method from the ElasticClient class from the NEST client in c#?

I've created a successful connection to ES, and then written my json query. Now, I would like to send that query via the Serialize method.
The Serialize method requires two parameters:
1. object and
2. Stream writableStream
My question is, with the second one. When I create a stream with the following code line:
Stream wstream;
And use it to initialize my json2 variable with the following code:
var json2 = highLevelclient.Serializer.Serialize(query, wstream).Utf8String();
I get the following error on the wstream variable:
Use of unassigned local variable 'wstream'.
Am I missing something? Is it the way I create the wstream variable that is wrong? thank you.
/* \\\ edit: ///// */
There is another issue now, I use Searchblox to index and search my files, which itself calls ES 2.x to do the job. Searchblox uses a "mapping.json" file to initialize a mapping upon the creation of an index. Here's the link to that file.
As "#Russ Cam" suggested, I created my own class content with the following code (just like he did with the "questions" index and "Question" class):
public class Content
{
public string type { get; set; }
public Fields fields { get; set; }
}
public class Fields
{
public Content1 content { get; set; }
public Autocomplete autocomplete { get; set; }
}
public class Content1
{
public string type { get; set; }
public string store { get; set; }
public string index { get; set; }
public string analyzer { get; set; }
public string include_in_all { get; set; }
public string boost { get; set; }
} //got this with paste special->json class
These fields from the content class (type,store etc.) come from the mapping.json file attached above.
Now, when I (just like you showed me) execute the following code:
var searchResponse = highLevelclient.Search<Content>(s => s.Query(q => q
.Match(m => m.Field(f => f.fields.content)
.Query("service")
All I get as a response on the searchResponse variable is:
Valid NEST response built from a successful low level call on POST: /idx014/content/_search
Audit trail of this API call:
-HealthyResponse: Node: http://localhost:9200/ Took: 00:00:00.7180404
Request:
{"query":{"match":{"fields.content":{"query":"service"}}}}
Response:
{"took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}
And no documents in searchResponse.Documents.
Contradictorily, when I search for the "service" query on Searchblox or make an API call to localhost:9200 with the Sense extension of Google Chrome, I get 2 documents. (the documents that I was looking for)
In brief, all I want is to be able to :
get all the documents (no criteria)
get all the documents within a time range and based upon keywords.. such as "service"
What am I doing wrong? I can provide with more information if needed..
Thank you all for your detailed answers.

It's actually much simpler than this with NEST :) The client will serialize your requests for you and send them to Elasticsearch, you don't need to take the step to serialize them yourself and then pass them to the client to send to Elasticsearch.
Take a search request for example. Given the following POCO
public class Question
{
public string Body { get; set; }
}
We can search for questions that contain the phrase "this should never happen" in the body with
var settings = new ConnectionSettings(new Uri("http://localhost:9200"))
.InferMappingFor<Question>(m => m
.IndexName("questions")
);
var client = new ElasticClient(settings);
var searchResponse = client.Search<Question>(s => s
.Query(q => q
.MatchPhrase(m => m
.Field(f => f.Body)
.Query("this should never happen")
)
)
);
// do something with the response
foreach (var question in searchResponse.Documents)
{
Console.WriteLine(question.Body);
}

this line
My question is, with the second one. When I create a stream with the following code line:
Stream wstream;
does not create na object. It nearly declares it. You need to new-ed it.
Stream wstream = new MemoryStream(); //doesn't have to be MemoryStream here - check what does Serialize expects
Just remember to close it later or use using statement.
using(Stream stream = new MemoryStream())
{
//do operations on wstream
} //closes automatically here

You just declared wstream but never assigned an actual stream to it. Depending on how Serialize method works it could be:
you need to create a stream and pass it to the Serialize method
or you need to pass stream parameter with out prefix

pass array of an object to webapi

I have a .net mvc 4 webapi project that I'm trying to pass an array of an object to a method on my controller.
I've found some examples here on SO that talk about needing to set my object's properties with: param1=whatever&param2=bling&param3=blah.
But I don't see how I can pass in a collection using that.
Here is my method signature. Notice I've decorated the argument with the [FromUri] attribute.
public List<PhoneResult> GetPhoneNumbersByNumbers([FromUri] PhoneRequest[] id)
{
List<PhoneResult> prs = new List<PhoneResult>();
foreach (PhoneRequest pr in id)
{
prs.Add(PhoneNumberBL.GetSinglePhoneResult(pr.PhoneNumber, pr.RfiDate, pr.FinDate, pr.State));
}
return prs;
}
here is my simple PhoneRequest object:
public class PhoneRequest
{
public string PhoneNumber { get; set; }
public string RfiDate { get; set; }
public string FinDate { get; set; }
public string State { get; set; }
}
and here's a sample of what I'm using to pass in:
http://localhost:3610/api/phonenumber/getphonenumbersbynumbers/
[{"PhoneNumber":"8016667777","RfiDate":"","FinDate":"2012-02-11","State":"UT"},
{"PhoneNumber":"8018889999","RfiDate":"2012-12-01","FinDate":"","State":"UT"}]
using this comes back with "bad request"
I also tried this
http://localhost:3610/api/phonenumber/getphonenumbersbynumbers?
id=[{"PhoneNumber":"8016667777","RfiDate":"","FinDate":"2012-02-11","State":"UT"},
{"PhoneNumber":"8018889999","RfiDate":"2012-12-01","FinDate":"","State":"UT"}]
which does reach the method, but the array is null.
how can I pass in an array of my PhoneRequest object to my Web API method?

Try passing the PhoneRequest[] from the uri in this format:
http://localhost:3610/api/phonenumber/getphonenumbersbynumbers?
id[0][PhoneNumber]=8016667777&id[0][FinDate]=2012-02-11&id[0][State]=UT&
id[1][PhoneNumber]=8018889999&id[1][RfiDate]=2012-12-01&id[1][State]=UT

I suggest you use POST for this.
As you query string grows, you will run into problems with the maximum length of the URL, which is browser dependent.
If you have a lot of parameters to pass, a POST is perfectly acceptable even if you are really only GETting data. What you will lose, however, is the ability for the user to bookmark a particular page with the query string.

I created a custom model binder, the FieldValueModelBinder class, which can effectively pass any object containing nested array or generic list types of data with query strings having field-name pairs without imbedding any JSON and XML structures. The model binder can resolve all issues discussed above. Since this question was extended by the question ID 19302078, you can see details of my answer in that thread.