We have documents saving to MongoDb. The problem is that one of our sub-documents has an Id property that is getting returned as _id, which is causing serialize/deserialize issues with the C# driver due to how it interprets Id fields (see http://mongodb.github.io/mongo-csharp-driver/2.0/reference/bson/mapping/)
I would like to rename the property from Id to SetId, but our data is fairly dynamic and simple field rename solutions that I've seen elsewhere do not apply. Here's an example of some heavily edited simple data:
{
"Id": "5a6238dbccf20b38b0db6cf2",
"Title": "Simple Document",
"Layout": {
"Name": "Simple Document Layout",
"Tabs": [
{
"Name": "Tab1",
"Sections": [
{
"Name": "Tab1-Section1",
"Sets": [
{
"Id": 1
}
]
}
]
}
]
}
}
Compare with more complex data:
{
"Id": "5a6238dbccf20b38b0db6abc",
"Title": "Complex Document",
"Layout": {
"Name": "Complex Document Layout",
"Tabs": [
{
"Name": "Tab1",
"Sections": [
{
"Name": "Tab1-Section1",
"Sets": [
{
"Id": 1
}
]
},
{
"Name": "Tab1-Section2",
"Sets": [
{
"Id": 1
}
]
}
]
},
{
"Name": "Tab2",
"Sections": [
{
"Name": "Tab2-Section1",
"Sets": [
{
"Id": 1
}
]
}
]
},
{
"Name": "Tab3",
"Sections": [
{
"Name": "Tab3-Section1",
"Sets": [
{
"Id": 1
},
{
"Id": 2
}
]
}
]
}
]
}
}
Note that the Set.Id field can be on multiple tabs on multiple sections with multiple sets. I just don't know how to approach a query to handle renaming data at all these levels.
I took #Veerum's advice and did a manual iteration over the collection with something like this:
myCol = db.getCollection('myCol');
myCol.find({ "Layout.Tabs.Sections.Sets._id": {$exists: true} }).forEach(function(note) {
for(tab = 0; tab != note.Layout.Tabs.length; ++tab) {
for(section = 0; section != note.Layout.Tabs[tab].Sections.length; ++section) {
for(set = 0; set != note.Layout.Tabs[tab].Sections[section].Sets.length; ++set) {
note.Layout.Tabs[tab].Sections[section].Sets[set].SetId = NumberInt(note.Layout.Tabs[tab].Sections[section].Sets[set]._id);
delete note.Layout.Tabs[tab].Sections[section].Sets[set]._id
}
}
}
myCol.update({ _id: note._id }, note);
});
Perhaps there is a more efficient way, but we are still on Mongo v3.2 and it seems to work well.
Related
I'm looking for a configurable value masking/sanitizer for both JSON and XML library on c#.
JSON Configuration:
{
"DefaultMaskValue": "#",
"ConfigurationValue": [
{
"id": 1,
"XmlMask": {
"MaskValue": "#",
"IsFullMasking": false,
"Sensitivity": [
{
"TargetProperties": [
"number",
],
"Positions": {
"Left": 3,
"Center": 3,
"Right": 3
}
}
]
},
"JsonMask": {
"MaskValue": "#",
"IsFullMasking": false,
"Sensitivity": [
{
"TargetProperties": [
"number",
],
"Positions": {
"Left": 3,
"Center": 3,
"Right": 3
}
}
]
}
}
]
}
Result:
XXX456789012345 , 12345678901XXX , 12345XXX9012345
NuGet Library: JSanitizer
Configurable masking/sanitizer for XML and JSON through extension method.
With JSON Configuration:
Through Class Configuration:
Medium
GitHub
I have documents like this:
[
// 1
{
"_id": ObjectId("573f3944a75c951d4d6aa65e"),
"Source": "IGN",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "Japan",
}
]
}
]
},
// 2
{
"_id": ObjectId("573f3d41a75c951d4d6aa65f"),
"Source": "VG",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "USA"
}
]
}
]
},
// 3
{
"_id": ObjectId("573f4367a75c951d4d6aa660"),
"Source": "NRK",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "Germany"
}
]
}
]
},
// 4
{
"_id": ObjectId("573f4571a75c951d4d6aa661"),
"Source": "VG",
"Family": [
{
"Countries": [
{
"uid": 10,
"name": "France"
}
]
}
]
},
// 5
{
"_id": ObjectId("573f468da75c951d4d6aa662"),
"Source": "IGN",
"Family": [
{
"Countries": [
{
"uid": 14,
"name": "England"
}
]
}
]
}
]
I want to return only the documents with source equals 'Countries.uid' equal 17
so I have in the end :
[
{
"_id": ObjectId("573f3944a75c951d4d6aa65e"),
"Source": "IGN",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "Japan",
}
]
}
]
},
{
"_id": ObjectId("573f3d41a75c951d4d6aa65f"),
"Source": "VG",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "USA"
}
]
}
]
},
{
"_id": ObjectId("573f4367a75c951d4d6aa660"),
"Source": "NRK",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "Germany"
}
]
}
]
}
]
How can I do this with the official C# MongoDB driver?
Tried this :
public List<Example> getLinkedCountry(string porduitId)
{
var filter = Builders<Example>.Filter.AnyIn("Family.Countries.uid", porduitId);
var cursor = await _certificats.FindAsync(filter);
var docs = cursor.ToList();
return docs;
}
Unfortunately, I think my filter is wrong.
Is there a way to find all the documents by accessing the nested list by id and retrieving it?
Solution 1
Use ElemMatch instead of AnyIn.
var filter = Builders<Example>.Filter.ElemMatch(
x => x.Family,
y => y.Countries.Any(z => z.uid == porduitId));
Output
Solution 2
If you are unconfident with MongoDB .Net Driver syntax, you can convert the query as BsonDocument via MongoDB Compass (Export to language feature).
var filter = new BsonDocument("Family.Countries.uid", porduitId);
Just to expand on #Yong Shun 's answer,
if you just want to return the list of nested documents and not all of it, you have a few options.
Using project
var filter = Builders<Example>.Filter.ElemMatch(
x => x.Family,
y => y.Countries.Any(z => z.uid == porduitId));
var project = Builders<Example>.Project.ElemMatch(
x => x.Family,
y => y.Countries.Any(z => z.uid == porduitId)
);
var examples = await collection.filter(filter).Project<Example>(project).toListAsync();
Using the aggregate pipeline
var filter = Builders<Example>.Filter.ElemMatch(
x => x.Family,
y => y.Countries.Any(z => z.uid == porduitId));
var project = Builders<ServiceProvider>.Projection.Expression(
x => x.Faimily.Where(y => y.uid == porduitId)
);
var result = await collection
.Aggregate()
.Match(filter)
.Project(project)
.ToListAsync(); //Here result is a list of Iterable<Countries>
Here is my json Array, i want to add a new field in array but i dont know how to loop it
{
"data": {
"pasca": [
{
"code": "PDI1231",
"name": "Water Bottle",
"status": 1,
"price": 2500,
"type": "plastic"
},
{
"code": "PDI9999",
"name": "Soccel Ball",
"status": 1,
"price": 99123,
"type": "plastic"
}
]
}
}
i want to add a new field in pasca array like this
"pasca": [
{
"code": "PDI1231",
"name": "Water Bottle",
"status": 1,
"price": 2500,
"type": "plastic",
"new_field_1": "new value_1",
"new_field_2": "new value_2"
}
]
If you are using Newtosoft's Json.NET it can be done as simple as that:
var jObj = JsonConvert.DeserializeObject<JObject>(json);
foreach(var el in jObj["data"]["pasca"])
{
el["new_field_1"] = "new value_1";
}
I have a nested json object which looks like this
{{
"id": "99ed8a1a-68fa-4464-b5cb-f116ede0a520",
"title": "NUnitTestDemo",
"has_children": true,
"level": 0,
"children": [
{
"id": "c41764b1-a59a-420b-b06e-9f97f3876e9b",
"title": "TestScripts",
"has_children": true,
"level": 1,
"children": [
{
"id": "cfba3d9e-d305-464d-9154-cdd2efcb5436",
"title": "SmokeTest",
"has_children": true,
"level": 2,
"children": [
{
"id": "b58596fc-aeab-4f20-8a91-85599c08c0fc",
"title": "TestAdd",
"has_children": false,
"level": 3,
"tag": "mytag,add",
"property": "Testing Addition of Two numbers",
"children": []
},
{
"id": "c819746e-25b9-4c84-8fb4-28794c3b2fe4",
"title": "TestSubtract",
"has_children": false,
"level": 3,
"tag": "mytag,subtract",
"property": "Testing Subtraction of Two numbers",
"children": []
}
]
},
{
"id": "7dce76a3-f318-4d49-920c-1d94b3ec9519",
"title": "SmokeTest1",
"has_children": true,
"level": 2,
"children": [
{
"id": "64bf771d-313b-44ce-b910-27945505dada",
"title": "TestMultiply",
"has_children": false,
"level": 3,
"tag": "mytag,add",
"property": "Testing Addition of Two numbers",
"children": []
},
{
"id": "5d810bc0-a9af-4838-b8eb-5fc7c47e910a",
"title": "TestDivide",
"has_children": false,
"level": 3,
"tag": "mytag,subtract",
"property": "Testing Subtraction of Two numbers",
"children": []
}
]
},
{
"id": "8af09935-93fa-4379-9aa4-9f809055d1ea",
"title": "Sample",
"has_children": true,
"level": 2,
"children": [
{
"id": "407944bd-437e-48cd-8af0-bbd2eb376e72",
"title": "Tests",
"has_children": true,
"level": 3,
"children": [
{
"id": "01223a10-2dda-4e3c-9287-d49b0c08870d",
"title": "SmokeTest2",
"has_children": true,
"level": 4,
"children": [
{
"id": "6e3df5f1-bcca-40a8-9ed5-3eaa74558488",
"title": "TestA",
"has_children": false,
"level": 5,
"tag": "mytag,add",
"property": "Testing Addition of Two numbers",
"children": []
},
{
"id": "0414d2c3-e4c8-4e52-9584-6a9e8516a3e2",
"title": "TestB",
"has_children": false,
"level": 5,
"tag": "mytag,subtract",
"property": "Testing Subtraction of Two numbers",
"children": []
}
]
}
]
}
]
},
{
"id": "7dbfcdfe-f6cb-4942-bcc6-3ec899aec674",
"title": "MyTestFolder",
"has_children": true,
"level": 2,
"children": [
{
"id": "16c3a197-824a-4309-bb97-24d454d448f5",
"title": "MyTestClass",
"has_children": true,
"level": 3,
"children": [
{
"id": "c37f2d67-0db0-49bf-80e0-9e99d7e9d767",
"title": "TestC",
"has_children": false,
"level": 4,
"tag": "mytag,add",
"property": "Testing Addition of Two numbers",
"children": []
},
{
"id": "a91c8c04-8a60-4872-b990-db9f993ddbe5",
"title": "TestD",
"has_children": false,
"level": 4,
"tag": "mytag,subtract",
"property": "Testing Subtraction of Two numbers",
"children": []
}
]
}
]
}
]
}
]
}}
If you notice at every level there is a "children" array. Now, when I am searching for "MyTestFolder" inside the above mentioned json, it is returning me "null".
My function looks like this. It is written in C# and I am using Newtonsoft.json library.
I am using .NET Core 3.1
public JObject RetrieveSpecifiedJsonObject(string propertyName, JObject jsonObject)
{
//propertyName is the property to be retrieved
string title = jsonObject.SelectToken("title").ToString();
if(title != propertyName)
{
JArray childArray = jsonObject.SelectToken("children") as JArray;
for(int i=0; i<childArray.Count; i++)
{
JObject childArrElem = childArray[i] as JObject;
string arrElemTitle = childArrElem.SelectToken("title").ToString();
if(arrElemTitle != propertyName)
{
RetrieveSpecifiedJsonObject(propertyName, childArrElem);
}
else
{
return childArrElem;
}
}//FOR ENDS
return null;
}//IF title != propertyName ENDS
else
{
return jsonObject;
}
}
I guess it has to be a recursive function. But, not getting what to do. When I am searching.
FYI, I can't search like jsonobject["children"][0]["MyTestFolder"]. I may have to search for any node at any point of time. For that, I need to write a generic function.
it returns null because this block doesn't do any action with the result from RetrieveSpecifiedJsonObject
if(arrElemTitle != propertyName)
{
RetrieveSpecifiedJsonObject(propertyName, childArrElem);
}
maybe you should do that:
if(arrElemTitle != propertyName)
{
var result = RetrieveSpecifiedJsonObject(propertyName, childArrElem);
if (result != null)
return result;
}
In addition to what JimmyN wrote in his answer, I'd say that you could simplify your recursive function as such (you already check the title property at the top of the function, so you don't need to check it again inside the for loop):
public JObject RetrieveSpecifiedJsonObject(string propertyName, JObject jsonObject)
{
//propertyName is the property to be retrieved
string title = jsonObject.SelectToken("title").ToString();
if(title != propertyName)
{
JArray childArray = jsonObject.SelectToken("children") as JArray;
for(int i=0; i<childArray.Count; i++)
{
JObject childArrElem = childArray[i] as JObject;
// the following will already check childArray and all of its children
JObject result = RetrieveSpecifiedJsonObject(propertyName, childArrElem);
if(result != null)
return result;
}//FOR ENDS
return null;
}//IF title != propertyName ENDS
else
{
return jsonObject;
}
}
I have two arrays variables and values like below
arraydata1 =
[
{
"id": "1",
"name": "aaa"
},
{
"id": "2",
"name": "bbb"
},
{
"id": "3",
"name": "ccc"
},
{
"id": "4",
"name": "ddd"
},
{
"id": "12",
"name": "aaa"
}
]
and
arraydata2 =
[
{
"id": "111",
"tablename": "aaa"
},
{
"id": "222",
"tablename": "bbb"
}
]
I want to compare arraydata1.name == arraydata2.tablename and if matching then form new array from arraydata1 .
output is -
[
{
"id": "1",
"name": "aaa"
},
{
"id": "2",
"name": "bbb"
},
{
"id": "12",
"name": "aaa"
}
]
I have more than 2000+ records to compare in arraydata1 how to reduce time as well. I can use normal foreach but it will take too much time to compare.
I was doing inside logic app using 2 foreach so it is taking time. so i thought better to use c# code.
One Linq solution could look like this:
var tableNameKeys = arraydata2.Select(t => t.tablename).ToHashSet();
var resultArray = arraydata1.Where(x => tableNameKeys.Contains(x.name)).ToArray();
The advantage of this approach is that HashSet.Contains
... is an O(1) operation.
Result: