We Keep Coding

Auto-Delete Certain type of Document in Mongodb in collection

I have a collection called PeopleDocument. This collection contains three different types of files: IDCardCopy, taxCopy, PermitCopy. Users can upload any of these files. I want to autodelete IDCardCopy one year after it was uploaded. I am looking at MongoDB TTL, however I have some questions:
db.PeopleDocument.createIndex( { "lastModifiedDate": 1 }, { expireAfterSeconds: 31622400} )
If I create an index like the one above, I think it will delete all files in PeopleDocument after 1 year, is it possible to only delete IDCardCopy?
More detail:
This is a C# code I use to insert Document:
var collInternDocuments = _database.GetCollection<BsonDocument>("PPLDocuments");
var doc = new BsonDocument{
{"Documentid", strDocumentID},
{ "FileType", typeOfDocument},
{ "FileContent", file.ContentType},
{ "FileName", file.FileName},
{ "UploadTime", DateTime.Now},
{ "UploadedBy", uploadedBy},
{ "GridFSFileID", strGridFSFileID}
};
collInternDocuments.UpdateOne( Builders<BsonDocument>.Filter.Eq("_id", internUserID), Builders<BsonDocument>.Update.AddToSet("Documents", doc));
This is how I created Index:
db.PeopleDocument.createIndex( {UploadTime:1},
{
expireAfterSeconds:900,
partialFilterExpression:{
FileType:{$eq:"IDCardCopy"}
}
})
This is the results for db.PeopleDocument.getIndexes():
{
"v":2,
"key":{
"_id":1
},
"name" :"_id_",
"ns" : "people.PeopleDocument"
},
{
"v":2,
"key":{
"UploadTime":1
},
"name" :"UploadTime_1",
"ns" : "people.PeopleDocument",
"expireAfterSeconds":900,
"partialFilterExpression":{
"FileType": { "$eq":"IDCardCopy"
}
}
}
This didn't delete the file after 900 sec, could this be a date issue?

this works for me. after 10secods (not exactly due to the deletion thread running every 60seconds), the IDCardCopy document is gone.
db.PeopleDocument.createIndex(
{ "uploadTime": 1 },
{
expireAfterSeconds: 10,
partialFilterExpression: { "FileType": { $eq: "IDCardCopy" } }
}
)
db.PeopleDocument.insertMany(
[
{
uploadTime: new ISODate(),
FileType: "IDCardCopy"
},
{
uploadTime: new ISODate(),
FileType: "taxCopy"
},
]
)
make sure you're setting the uploadTime field to the correct UTC now time in your application.

I think you should add in PeopleDocument the column expiresAt and fill this field on creating concrete record depended on your file type. And then simply create index like that:
db.PeopleDocument.createIndex( { "expiresAt": 1 }, { expireAfterSeconds: 0} )

I solved this by creating a C# console app that will delete documents created one year ago, I created a Task Scheduler that will run everyday to run this app.

How do I convert a Json file (that contains a array) to a List (of a class object) C# Unity

i'm using a NewtonSoft Json API to read my dialogue texts and i'm having problem to populate a List of dialogues information.
i would like to populate my list (of this class below) using one of this 2 Json formats.
[System.Serializable]
public class DialogueList
{
public string dialogueName;
public bool isDialogueOption;
public string[] dialogueText;
public string option1;
public string option2;
}
Ex: I would like if the list be like this.
//(Slot 1)
dialogueList[0].dialogueName = "Nyma";
dialogueList[0].isDialogueOption = true;
dialogueList[0].dialogueText[0] = "Hi Xire! how are you?";
dialogueList[0].dialogueText[1] = "Hi Nyma! i'm fine and you?";
dialogueList[0].option1 = "Fine!";
dialogueList[0].option2 = "I'm not fine!";
//(Slot2)
dialogueList[1].dialogueName = "Xire";
dialogueList[1].isDialogueOption = false;
dialogueList[1].dialogueText[0] = "Run Nyma";
dialogueList[1].dialogueText[1] = "I'm Running Xire";
dialogueList[1].option1 = Null;
dialogueList[1].option2 = Null;
Json Format 1:
{
"Dialogue_Nyma": [
{
"dialogueName": "Nyma",
"isDialogueOption": true,
"dialogueText": [
"Hi Xire! how are you?",
"Hi Nyma! i'm fine and you?"
],
"Option1": "Fine!",
"Option2": "i'm not fine!"
}
],
"Dialogue_Xire": [
{
"dialogueName": "Xire",
"isDialogueOption": false,
"dialogueText": [
"Run Nyma!",
"i'm Running Xire."
],
"Option1": null,
"Option2": null
}
]
}
Json format 2:
[
{
"dialogueName": "Nyma",
"isDialogueOption": true,
"dialogueText": [
"Hi Xire! how are you?",
"Hi Nyma! i'm fine and you?"
],
"Option1": "Fine!",
"Option2": "i'm not fine!"
},
{
"dialogueName": "Xire",
"isDialogueOption": false,
"dialogueText": [
"Run Nyma!",
"i'm Running Xire."
],
"Option1": null,
"Option2": null
}
]
If someone could help me to find a way to deserialize one of these json formats to populate my list i'll be really thankful!
i also tried to create a class that contains a array of Dialogue List
[System.Serializable]
public class DialogueListCollection
{
public DialogueList[] dialogueList;
}
and tried to parse like this
string path = "DialogueJson/Textos";
var contents = Resources.Load<TextAsset>(path);
dialogueList = JsonConvert.DeserializeObject<DialogueListCollection>(contents.text);
but didn't work.

Your class should represent the dialog:
public class Dialogue
{
public string dialogueName;
public bool isDialogueOption;
public string[] dialogueText;
public string option1;
public string option2;
}
(Notice the "List" is gone from the class name)
You can then use Newtonsoft to deserialize it to an array:
var json = #"[
{
""dialogueName"": ""Nyma"",
""isDialogueOption"": true,
""dialogueText"": [
""Hi Xire! how are you?"",
""Hi Nyma! i'm fine and you?""
],
""Option1"": ""Fine!"",
""Option2"": ""i'm not fine!""
},
{
""dialogueName"": ""Xire"",
""isDialogueOption"": false,
""dialogueText"": [
""Run Nyma!"",
""i'm Running Xire.""
],
""Option1"": null,
""Option2"": null
}
]";
Dialogue[] list = JsonConvert.DeserializeObject<Dialogue[]>(json);
Your array list now contains two entries.
The second format you provided is correctly formatted and used in this sample.

Add class that will contain only array of DialogueList and parse json as this class. You'll need to add one field in json to put list in it.

Parsing/Iterating over Json

I'm fairly new to parsing Json with C# and i'm having a little issue i can't work my head around.
My data looks something like this:
{
"languages": {
"ja_lang": {
"data": {
"name": "Japanese"
},
"files": [["ja",
"Japanese File",
"lang_ja.txt"]]
},
"en_lang": {
"data": {
"name": "English"
},
"files": [["en",
"English File",
"lang_en.txt"]]
}
}
}
Now i want to iterate over the items in languages and only work with the one where the object-name starts with "ja_" (in this case it would only work with "ja_lang" and ignore "en_lang"), then extract the name inside data and the "lang_ja.txt" in files.
To Parse the Json in C# i downloaded the Newtonsoft.Json library and came up with this:
dynamic json_obj = JsonConvert.DeserializeObject("json string");
// when debugging language holds { "ja_lang": { "data": { "name": "Japanese" }, "files": [["ja", "Japanese File", "lang_ja.txt"]] } }
foreach (var language in json_obj.languages)
{
// not sure how i can access the object-name
/*if(!language.StartsWith("ja_"))
continue;*/
// Exception: 'Newtonsoft.Json.Linq.JProperty' does not contain a definition for 'data' - Not sure why it is treated as a property?
var name = language.data.name;
var file = language.files[2];
}
I'm sorry for this probably dumb question, but i've been trying to cast it to different types and searched the web for solutions, but i just couldn't figure it out. So if someone could help me out with this i would be really greatful.
Thanks in advance!

Since you're stating in a comment (on an answer that has been deleted) that the data changes so a fixed model won't work, you can still fix what is known:
Here's a LINQPad program that demonstrates:
void Main()
{
var collection = JsonConvert.DeserializeObject<LanguagesCollection>(File.ReadAllText(#"c:\temp\test.json"));
foreach (var keyValuePair in collection.Languages)
if (keyValuePair.Key.StartsWith("ja_"))
keyValuePair.Value.Dump();
}
public class LanguagesCollection
{
public Dictionary<string, JObject> Languages { get; } = new Dictionary<string, JObject>();
}
This will deserialize the outer object, with the "languages" key, and inside you have a dictionary with the keys, "ja_lang", "en_lang", and you can just process the values as you see fit. These are left as JObject which means they will contain whatever json was present as a value for that key in the dictionary.

Using a site like json2sharp you can just pass your json data in and get a ready to use c# model out.
Then you can easily deserialize your json data into that c# model and use the properties for much easier handling:
string jsonData = #"{
'languages': {
'ja_lang': {
'data': {
'name': 'Japanese'
},
'files': [['ja',
'Japanese File',
'lang_ja.txt']]
},
'en_lang': {
'data': {
'name': 'English'
},
'files': [['en',
'English File',
'lang_en.txt']]
}
}
}";
RootObject data = JsonConvert.DeserializeObject<RootObject>(jsonData);
foreach(Languages lang in data.languages) //would work if Languages was a listing
{
}
Although I admit that your Json is a bit strange and that Languages most likly should be a listing and not a property for each language.

json foreach loop (without json.net)

I'm currently trying to create a small launcher to solve some problems using the existing launcher from minecraft.
I'm trying to read a .json file to get all the informations that i need.
If you need to take a look at the .json file here.
I got it working if i just need a single information like
string clienturl = readJson("//downloads/client/url");
with this:
private string readJson(string element)
{
string json = File.ReadAllText(Path.Combine(appPath + "1.10.2.json"));
var jsonReader = JsonReaderWriterFactory.CreateJsonReader(Encoding.UTF8.GetBytes(json), new System.Xml.XmlDictionaryReaderQuotas());
var root = XElement.Load(jsonReader);
return root.XPathSelectElement(element).Value;
}
The problem now is that i need to get informations for all the other files.
The "element" would be:
libraries/downloads/artifact/path
libraries/downloads/artifact/url
but obviously there is more then one entry for "path" and "url" so i need a foreach loop.
What do i need to change in my code above to make it working with a foreach loop?
Sorry for my bad english, i hope its not to hard to understand.
Small preview of the .json in case you dont want to download the file:
"libraries": [
{
"name": "com.mojang:netty:1.6",
"downloads": {
"artifact": {
"size": 7877,
"sha1": "4b75825a06139752bd800d9e29c5fd55b8b1b1e4",
"path": "com/mojang/netty/1.6/netty-1.6.jar",
"url": "https://libraries.minecraft.net/com/mojang/netty/1.6/netty-1.6.jar"
}
}
},
{
"name": "oshi-project:oshi-core:1.1",
"downloads": {
"artifact": {
"size": 30973,
"sha1": "9ddf7b048a8d701be231c0f4f95fd986198fd2d8",
"path": "oshi-project/oshi-core/1.1/oshi-core-1.1.jar",
"url": "https://libraries.minecraft.net/oshi-project/oshi-core/1.1/oshi-core-1.1.jar"
}
}
},
{
"name": "net.java.dev.jna:jna:3.4.0",
"downloads": {
"artifact": {
"size": 1008730,
"sha1": "803ff252fedbd395baffd43b37341dc4a150a554",
"path": "net/java/dev/jna/jna/3.4.0/jna-3.4.0.jar",
"url": "https://libraries.minecraft.net/net/java/dev/jna/jna/3.4.0/jna-3.4.0.jar"
}
}
}
]

The issue is that your JSON contains an array, and it's not immediately obvious how JsonReaderWriterFactory maps an array to XML elements, given that XML doesn't have the concept of an array.
One way to determine this is to read through the documentation Mapping Between JSON and XML which describes this mapping. But another is simply to determine for ourselves, by using one of the answers from Get the XPath to an XElement? to find out the actual paths for each element. Using this answer, the following code:
var paths = root.DescendantsAndSelf().Select(e => e.GetAbsoluteXPath()).ToList();
Debug.WriteLine(String.Join("\n", paths));
Produces output like:
/root/libraries
/root/libraries/item[1]
/root/libraries/item[1]/name
/root/libraries/item[1]/downloads
/root/libraries/item[1]/downloads/artifact
/root/libraries/item[1]/downloads/artifact/size
/root/libraries/item[1]/downloads/artifact/sha1
/root/libraries/item[1]/downloads/artifact/path
/root/libraries/item[1]/downloads/artifact/url
/root/libraries/item[2]
/root/libraries/item[2]/name
/root/libraries/item[2]/downloads
/root/libraries/item[2]/downloads/artifact
/root/libraries/item[2]/downloads/artifact/size
/root/libraries/item[2]/downloads/artifact/sha1
/root/libraries/item[2]/downloads/artifact/path
/root/libraries/item[2]/downloads/artifact/url
So, as you can see, each array item is placed in a synthetic <item> node.
Thus you can query your paths and urls as follows:
var files = root
.XPathSelectElements("libraries/item/downloads/artifact")
.Select(e => new PathAndUrl { Path = (string)e.Element("path"), Url = (string)e.Element("url") })
.ToList();
Placing the result into a list of the following class:
public class PathAndUrl
{
public string Path { get; set; }
public string Url { get; set; }
}

How to reformat datetime

in my client side i get an object with a datetime property so when i send the object to the server side i send it in json format so when i try to use that property appears in a different format.
the original datetime property is "2016-03-09T17:16:33" and when i try to use that property appears like this: "09/03/2016 05:16:33 p. m."
what i want is to avoid that cast, or reformat the property to the original format.
im using Newtonsoft.Json to handle the json in the server side.
this is the client side´s object (angularjs)
var factura = {
comprobante:{
fecha:"2016-03-09T17:16:33",
version:"3.2",
tipoDeComprobante:"comprobante",
formaDePago:"Efectivo",
subTotal:"123123.123",
total:"4234234",
metodoDePago:"deposito",
LugarExpedicion:"guadalajara"
},
cliente:{
token:"69113116100"
},
receptor:{
rfc:"234dfdg3gfdgsg42",
domicilio:{
pais:"Mexico"
}
},
emisor:{
rfc:"234dfdg3gfdgsg42",
domicilioFiscal:{
calle:"san cristobalito",
municipio:"zapopan",
estado:"jalisco",
codigoPostal:"3434",
pais:"mexico"
},
regimenFiscal:{
Regimen:"regimen"
}
},
conceptos:
[
{
cantidad:"234",
unidad:"1.0",
descripcion:"descripcion",
valorUnitario:"decimal",
importe:"importe"
},
{
cantidad:"234",
unidad:"1.0",
descripcion:"descripcion",
valorUnitario:"decimal",
importe:"importe"
}
],
impuestos:{
traslado:[
{
impuesto:"1.3",
tasa:"2.5",
importe:"importe"
},
{
impuesto:"1.3",
tasa:"2.5",
importe:"importe"
}
],
retencion:[
{
impuesto:"1.3",
tasa:"2.5",
importe:"importe"
},
{
impuesto:"1.3",
tasa:"2.5",
importe:"importe"
}
]
},
tieneComplementoDonat:"0"
};
and this is my server side´s code as you can see in the comprobante.SetAttributeValue(DatosComprobante[y].NombreDato, objeto["factura"]["comprobante"][DatosComprobante[y].NombreDato]); method i insert the properties in a XElement node so when the property "fecha" comes in the for loop, it appears different as i sent it from client's side.
i think is something about to disable the deserialization to get only the raw string because when i see the "objeto" variable in the debugger the date appears correctly.
public XElement retornaNodoXml(JObject objeto)
{
Domicilio domicilio = new Domicilio();
Dato dato = new Dato();
XElement comprobante = new XElement("cfdi" + dato.ComodinCfdi + "Comprobante");
for (int y = 0; y < DatosComprobante.Count; y++)
{
if (objeto["factura"]["comprobante"][DatosComprobante[y].NombreDato] != null)
{
comprobante.SetAttributeValue(DatosComprobante[y].NombreDato, objeto["factura"]["comprobante"][DatosComprobante[y].NombreDato]);
}
}
return comprobante;
}