We Keep Coding

Mongo .Net Driver PipelineStageDefinitionBuilder.Project automatically ignores all Id values with a facet

When using the PipelineStageDefinitionBuilder when creating projection stages for an aggregation pipeline it appears to be always ignoring any Id values in the dataset. I'm using the Mongo .Net driver 2.8 in a .Net Core app. Below are the steps for reproduction.
The same projection worked when using the IAggregateFluent syntax on Aggregate() however I needed to use the builders for a facet. When running the builder against Aggregate it also works, however within a facet it fails to bind any Id values.
Just empty classes with id for testing (Added Type to show normal mapping works):
public class DatabaseModel
{
public Guid Id { get; set; }
public string Type { get; set; }
}
public class ProjectionClass
{
public Guid Id { get; set; }
public string Type { get; set; }
}
When I create the projection with the below, it produces a query sucessfully, however within all models returned the Id value is set to null. The query seems to have a Id_ : 0 value but the same also seems to be produced in normal aggregation so I don't think this is related?
var typeFilter = Builders<DatabaseModel>.Filter.Eq(x => x.Type, "Full");
var aggregationPipeline = new EmptyPipelineDefinition<DatabaseModel>()
.AppendStage(PipelineStageDefinitionBuilder.Match(typeFilter))
.AppendStage(PipelineStageDefinitionBuilder.Project<DatabaseModel, ProjectionClass>(x => new ProjectionClass
{
Id = x.Id,
Type = x.Type,
}));
var normalAggregationResult = await db.Aggregate(aggregationPipeline).ToListAsync();//The id's appear here
var databaseModelsFacet = AggregateFacet.Create("DatabaseModels", aggregationPipeline);
var faucetResult = db.Aggregate().Facet(databaseModelsFacet).SingleOrDefault().Facets;
var projectionModels = faucetResult.
Single(x => x.Name == "DatabaseModels")
.Output<ProjectionClass>();// This results in missing Id's (Including in nested objects with anything named Id)
Resulting mongo query
{[{
"$match" : { "Type" : "Full" } },
{ "$project" : { "Id" : "$_id", "Type" : "$Type", "_id" : 0 }
}]}
Is there any way to be able to run a projection using the pipeline builders with a facet while not ignoring the Id? I have seen examples using similar queries but haven't seen this as an issue. It could be an issue with facet as it only appears to happen when using this.
Thanks!
UPDATE 6/1/2020: Updated question after finding it only seems to occur with facet

It seems to be an Driver issue, (or other issue is when the structure does not match the fields), as Id cant simply be serialized to Id , but if you choose any other value it will work for example
[BsonNoId]
public class DatabaseModel
{
[BsonRepresentation(BsonType.ObjectId)]
public string Identifier { get; set; }
public string Type { get; set; }
}

Elasticsearch NEST and basic Search query

I'm new to Elasticsearch and I'm using NEST. When I run my query in the browser (host/logstash-2019.03.17/_search?pretty) I get the following result:
{
"took" : 138,
"timed_out" : false,
"shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 10,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "logstash-2019.03.17",
"_type" : "logevent",
"_id" : "aa7djGkB1zvCMljS8jPd",
"_score" : 1.0,
"_source" : {
"#timestamp" : "2019-03-17T18:15:43.9506399Z",
"level" : "Info",
"message" : "Attempting to get results from ElasticSearch",
"logger" : "App.Api.Controllers.MyController"
}
}, OTHER HITS IN THE SAME FORMAT
However, when I'm trying to query the same index using ElasticClient i get the following exception:
Elasticsearch.Net.UnexpectedElasticsearchClientException: „Cannot
deserialize the current JSON object (e.g. {"name":"value"}) into type
'System.Int64' because the type requires a JSON primitive value (e.g.
string, number, boolean, null) to deserialize correctly. To fix this
error either change the JSON to a JSON primitive value (e.g. string,
number, boolean, null) or change the deserialized type so that it is a
normal .NET type (e.g. not a primitive type like integer, not a
collection type like an array or List) that can be deserialized
from a JSON object. JsonObjectAttribute can also be added to the type
to force it to deserialize from a JSON object. Path
'hits.total.value', line 1, position 115.”
I thought NEST is capable of autodeserializing JSON correctly on condition that it is provided with a class whose properties correspond to "_source" object fields. At least this is what you can infer from this tutorial.
Here is my POCO class follwed by the query which throws the exception:
public class Logevent
{
public string Id { get; set; }
public DateTime Timestamp { get; set; }
public string Level { get; set; }
public string Message { get; set; }
public string Logger { get; set; }
}
var client = new ElasticClient();
var searchResponse = client.Search<Logevent>(s => s.Index("logstash-2019.03.17").Query(q => q.Match(m => m.Field(f => f.Level).Query("message"))));
Could anyone explain what I'm doing wrong?

It looks like you're using a newer version of Elasticsearch (maybe one of the 7.0.0 prereleases?) where the total field is no longer just an Int64 value
"total" : {
"value" : 10,
"relation" : "eq"
}
NEST 6.x does not handle this, NEST 7.x will however, when it's released. For now, I would recommend using the latest Elasticsearch 6.x, currently 6.6.2.
Major versions of NEST are tied to major versions of Elasticsearch, so
NEST 5.x -> Elasticsearch 5.x
NEST 6.x -> Elasticsearch 6.x
etc.
But I would recommend keeping NEST up to date on minors within a major; we maintain backwards binary compatibility within a major to help with this.

Serializing "string list" to JSON in C#

(I'v restated my question here: Creating class instances based on dynamic item lists)
I'm currently working on a program in Visual Studio 2015 with C#.
I have 5 list strings that contain data that I wish to serialize to a json file.
public List<string> name { get; private set; }
public List<string> userImageURL { get; private set; }
public List<string> nickname { get; private set; }
public List<string> info { get; private set; }
public List<string> available { get; private set; }
An example of the desired json file format is the fallowing:
{
"users" :
[
{
"name" : "name1",
"userImageURL" : "userImageURL1",
"nickname" : "nickname1",
"info" : "info1",
"available" : false,
},
{
"name" : "name2",
"userImageURL" : "userImageURL2",
"nickname" : "nickname2",
"info" : "info2",
"available" : false,
},
{
"name" : "name3",
"userImageURL" : "userImageURL3",
"nickname" : "nickname3",
"info" : "info3",
"available" : false,
},
{
"name" : "name4",
"userImageURL" : "userImageURL4",
"nickname" : "nickname4",
"info" : "info4",
"available" : false,
}
]
}
Note that there might be errors in the json example above.
I've tried combining the 5 lists to create 1 list to serialize it using the following code:
users = new List<string>(name.Count + userImageURL.Count + nickname.Count + info.Count + available.Count);
allPlayers.AddRange(name);
allPlayers.AddRange(userImageURL);
allPlayers.AddRange(nickname);
allPlayers.AddRange(info);
allPlayers.AddRange(available);
Then I serialize the list with the fallowing code:
string data = JsonConvert.SerializeObject(users);
File.WriteAllText("data.json", data);
This just creates an array of unorganized objects. I wish to know how can I organize them as expressed in the format above.
PS: I'm pretty new to coding as you can tell. Sorry if I'm not expressing the question correctly or using the right terminology. Also, this is not the original code. The code creates this lists which I wish to serialize into a json file.
PSS: This data is collected using HtmlAgilityPack. I asked a question yesterday asking how could I parse an html file and serialize it's data to a json file. Using HtmlAgilityPack to get specific data in C# and serialize it to json . As nobody answered, I decided to try and do it myself. The method that I used may not be the best, but it is what I could do with the knowledge that I have.

I would suggest refactoring your code to start with - instead of having 5 "parallel collections", have a single collection of a new type, User:
public class User
{
public string Name { get; set; }
public string ImageUrl { get; set; }
public string NickName { get; set; }
public string Info { get; set; }
public bool Available { get; set; }
}
...
// In your containing type
public List<User> Users { get; set; }
This is likely to make life simpler not just for your JSON, but for the rest of the code too - because you no longer have the possibility of having more nicknames than image URLs, etc. In general, having multiple collections that must be kept in sync with each other is an antipattern. There are times where it's appropriate - typically providing different efficient ways of retrieving the same data - but for something like this it's best avoided.

It is actually what Jon says, but to move from your parallel lists to Jon's single list you need something like (assuming all lists have the same number of elements in the same order):
Users = new List<User>();
for (var i = 0; i < name.Count; i++)
{
Users.Add(new User
{
Available = available[i],
ImageUrl = userImageURL[i],
Info = info[i],
Name = name[i],
NickName = nickname[i]
});
}
And then serialise the Users list.

MongoDB - return document references as nested document

imagine this structure:
collection companies:
{
"company" : "foo",
"city" : "1234"
}
{
"company" : "bar",
"city" : "1234"
}
collection cities:
{
"_id" : "1234",
"cityname" : "Berlin",
"zipcode" : "09123"
}
now i want to get as result when i query for company = "foo":
{
"company" : "foo",
"city" : {
"_id" : "1234",
"cityname" : "Berlin",
"zipcode" : "09123"
}
}
How to query this from the C# driver? i have the result structure as C# class and i would like to have it strong-typed.
like this:
MongoCollection<Company> mc = mongodb.GetCollection<Company>("companies")
And the Class company looks like:
public class Company {
public string company {get;set;}
public City city {get;set;}
}
You should get the idea.
The Mongo ducomentation didn't say much about this.

Your are frequently asking the city data by reference, embedding this document makes it faster to query.
Restructure your document schema and embed the city (address) document into the company document.
{
company : "Deutsche Bank",
address : {
street : "Müllerstraße",
number : "34a",
zipcode : "13353",
city : "Berlin"
}
}
If you have no other option because of any business logic is relying on the company document, you could choose to create aggregation queries and project a new document as result.
[additional info]
You have to run an extra query to get the referenced document, foreach every company and embed the city info.

Assign JSON value to property in C#

From an API I receive a JSON-object that looks like this:
{
"wind" : {
"speed" : 7.31,
"deg" : 187.002
},
"rain" : {
"3h" : 0
},
"clouds" : {
"all" : 92
},
"coord" : {
"lon" : 139,
"lat" : 35
},
"dt" : 1369824698,
"id" : 1851632,
"cod" : 200,
"weather" : [
{
"id" : 804,
"main" : "clouds",
"icon" : "04n",
"description" : "overcast clouds"
}
],
"main" : {
"humidity" : 89,
"temp_max" : 292.04,
"temp_min" : 287.04,
"temp" : 289.5,
"pressure" : 1013
},
"sys" : {
"country" : "JP",
"sunrise" : 1369769524,
"sunset" : 1369821049
},
"name" : "Shuzenji"
}
I would like to assign two of these values to my class:
public class Weather {
public string Name { get; set; }
public string Temp { get; set; }
}
The name I can assign like this:
weather.Name = TheJSON.name.ToString();
But the temp is trickier, because it's nested inside the "main"-array. I´ve seen many examples on how to do this in Javascript but not so much in C#. Thanks!

Main is not an array. It is an object, so
TheJSON.main.temp.ToString()

The easiest way to work with JSON data is to deserialize them as C# objects and directly use them in your application. You can use a tool like JSON C# Class Generator to automatically generate the C# class from the JSON data. Once you have your C# classes generated, you can deserialize the JSON string using the JsonConvert.DeserializeObject(jsonText); The generated code requires Newtonsoft Json.NET which you can easily add as a NuGet package.
If you save your JSON content in D:\test.json, you can use the following code to access the values using the C# objects generated. The example below is to just give you an idea on the usage.
var json = File.ReadAllText(#"D:\test.json");
var weather = JsonConvert.DeserializeObject<Weather>(json);
Console.WriteLine(weather.Name);
Console.WriteLine(weather.Sys.Country);