We Keep Coding

C# Lists - do I use Class Methods (Get/ Set etc) again once the data is in a list?

A quick question on OOP. I am using a list together with a class and class constructor. So I use the class constructor to define the data set and then add each record to my list as the user creates them.
My questions is once the data is in the list and say I want to alter something is it good practice to find the record, create an instance using that record and then use my class methods to do whatever needs doing - and then put it back in the list?
For example below I have my class with constructor. Lets say I only want the system to release strCode if the Privacy field is set to public. Now just using Instances I would use for example Console.WriteLine(whateverproduct.ProductCode) but if the record is already in a list do i take it out of the list - create an instance and then use this method?
class Product
{
private String strCode;
private Double dblCost;
private Double dblNet;
private String strPrivacy;
public Product(String _strCode, Double _dblCost, Double _dblNet, String _strPrivacy)
{
strCode = _strCode;
dblCost = _dblCost;
dblNet = _dblNet;
strPrivacy = _strPrivacy;
}
public string ProductCode
{
get
{
if (strPrivacy == "Public")
{
return strCode;
}
else
{
return "Product Private Can't release code";
}
}
}

Lets say we have the following:
public class Test
{
public int Id { get; set; }
public string Name { get; set; }
public decimal Amount { get; set; }
private string _test = "Some constant value at this point";
public string GetTest()
{
return _test;
}
public void SetTest()
{
//Nothing happens, you aren't allow to alter it.
//_test = "some constant 2";
}
}
public class Program
{
public static void Main(string[] args)
{
List<Test> listOfTest = new List<Test>()
{
new Test() {Id = 0, Name = "NumberOne", Amount = 1.0M},
new Test() {Id = 1, Name = "NumberTwo", Amount = 2.0M}
};
Test target = listOfTest.First(x => x.Id == 0);
Console.WriteLine(target.Name);
target.Name = "NumberOneUpdated";
Console.WriteLine(listOfTest.First(x => x.Id == 0).Name);
Console.WriteLine(listOfTest.First(x => x.Id == 0).GetTest());//This will alsways be "Some constant value at this point";
Console.ReadLine();
}
}
Technically you could do away with the SetTest method entirely. However, I included it to demonstrate, what it would look like, if you wanted to alter _test.
You don't want to ever create a new instance of a class, you already have an instance of. you can just alter the class where it is allowed by the author of the class, where you need to. And keep that class reference for as long as you need it. Once you are done, the reference will be garbage collected, once the program finds no active reference to your object(instance).

MongoDB Concurrent writing/fetching from multiple processes causes bulk write operation error

I'm currently implementing a MongoDB database for caching.
I've made a very generic client, with the save method working like this:
public virtual void SaveAndOverwriteExistingCollection<T>(string collectionKey, T[] data)
{
if (data == null || !data.Any())
return;
var collection = Connector.MongoDatabase.GetCollection<T>(collectionKey.ToString());
var filter = new FilterDefinitionBuilder<T>().Empty;
var operations = new List<WriteModel<T>>
{
new DeleteManyModel<T>(filter),
};
operations.AddRange(data.Select(t => new InsertOneModel<T>(t)));
try
{
collection.BulkWrite(operations, new BulkWriteOptions { IsOrdered = true});
}
catch (MongoBulkWriteException mongoBulkWriteException)
{
throw mongoBulkWriteException;
}
}
With our other clients, calling this method looking similar to this:
public Person[] Get(bool bypassCache = false)
{
Person[] people = null;
if (!bypassCache)
people = base.Get<Person>(DefaultCollectionKeys.People.CreateCollectionKey());
if (people.SafeAny())
return people;
people = Client<IPeopleService>.Invoke(s => s.Get());
base.SaveAndOverwriteExistingCollection(DefaultCollectionKeys.People.CreateCollectionKey(), people);
return people;
}
After we've persisted data to the backend we reload the cache from MongoDB by calling our Get methods, passing the argument true. So we reload all of the data.
This works fine for most use cases. But considering how we are using a Web-garden solution (multiple processes) for the same application this leads to concurrency issues. If I save and reload the cache while another user is reloading the page, sometimes it throws a E11000 duplicate key error collection.
Command createIndexes failed: E11000 duplicate key error collection:
cache.Person index: Id_1_Name_1_Email_1 dup
key: { : 1, : "John Doe", : "foo#bar.com" }.
Considering how this is a web garden with multiple IIS processes running, locking won't do much good. Considering how bulkwrites should be threadsafe I'm a bit puzzled. I've looked into Upserting the data, but changing our clients to be type specific and updating each field will take too long and feels like unnecessary work. Therefore I'm looking for a very generic solution.
UPDATE
Removed the Insert and Delete. Changed it to a collection of ReplaceOneModel. Currently experiencing issues with only the last element in a collection being persisted.
public virtual void SaveAndOverwriteExistingCollection<T>(string collectionKey, T[] data)
{
if (data == null || !data.Any())
return;
var collection = Connector.MongoDatabase.GetCollection<T>(collectionKey.ToString());
var filter = new FilterDefinitionBuilder<T>().Empty;
var operations = new List<WriteModel<T>>();
operations.AddRange(data.Select(t => new ReplaceOneModel<T>(filter, t) { IsUpsert = true }));
try
{
collection.BulkWrite(operations, new BulkWriteOptions { IsOrdered = true });
}
catch (MongoBulkWriteException mongoBulkWriteException)
{
throw mongoBulkWriteException;
}
}
Just passed in a collection of 811 items and only the last one can be found in the collection after this method has been executed.
Example of a DTO being persisted:
public class TranslationSetting
{
[BsonId(IdGenerator = typeof(GuidGenerator))]
public object ObjectId { get; set; }
public string LanguageCode { get; set; }
public string SettingKey { get; set; }
public string Text { get; set; }
}
With this index:
string TranslationSettings()
{
var indexBuilder = new IndexKeysDefinitionBuilder<TranslationSetting>()
.Ascending(_ => _.SettingKey)
.Ascending(_ => _.LanguageCode);
return MongoDBClient.CreateIndex(DefaultCollectionKeys.TranslationSettings, indexBuilder);
}

MongoDB Text Search with projection

Using MongoDB with C# and driver 2.0, I am trying to do the following:
Text search
Sort the hits by text search score
Project BigClass to SmallClass
Here is a (simplified version of) the classes:
class BigClass
{
[BsonIgnoreIfDefault]
public ObjectId _id { get; set; }
public string Guid { get; set; }
public string Title { get; set; }
public DateTime CreationTime { get; set; }
// lots of other stuff
[BsonIgnoreIfNull]
public double? TextMatchScore { get; set; } // Temporary place for the text match score, for sorting
}
class SmallClass
{
[BsonIgnoreIfDefault]
public ObjectId _id { get; set; }
public string Title { get; set; }
[BsonIgnoreIfNull]
public double? TextMatchScore { get; set; } // Temporary place for the text match score, for sorting
}
If I do a text search, it is pretty straightforward:
var F = Builders<BigClass>.Filter.Text("text I am looking for");
var Result = MongoDriver.Find(F).ToListAsync().Result;
If I want to sort by the score of the text search, it's a bit more messy (and very POORLY documented):
var F = Builders<BigClass>.Filter.Text("text I am looking for");
var P = Builders<BigClass>.Projection.MetaTextScore("TextMatchScore");
var S = Builders<BigClass>.Sort.MetaTextScore("TextMatchScore");
var Result = MongoDriver.Find(F).Project<BigClass>.Sort(S).ToListAsync().Result;
Essentially it requires me to add a field in the class (TextMatchScore) to hold the result.
If I want to get the data, without sorting and project it to SmallClass, it is straightforward:
var F = Builders<BigClass>.Filter.Text("text I am looking for");
var P = Builders<BigClass>.Projection.Include(_ => _.id).Include(_ => _.Title);
var Result = MongoDriver.Find(F).Project<SmallClass>(P).ToListAsync().Result;
Now if "I want it all", that's where problem arises:
var F = Builders<BigClass>.Filter.Text("text I am looking for");
var P = Builders<BigClass>.Projection.MetaTextScore("TextMatchScore").Include(_ => _.id).Include(_ => _.Title).Include(_ => _.TextMatchScore);
var S = Builders<BigClass>.Sort.MetaTextScore("TextMatchScore");
var Result = MongoDriver.Find(F).Project<SmallClass>.Sort(S).ToListAsync().Result;
I get an exception:
Message = "QueryFailure flag was true (response was { \"$err\" : \"Can't canonicalize query: BadValue must have $meta projection for all $meta sort keys\", \"code\" : 17287 })."
As expected, the error is not documented anywhere as the Mongo guys expect users to self-document everything.
If I make the projection to 'BigClass', there is no problem, the code runs and just fills in the right fields.
If you google that text with C#, the posts you find are mine when I was trying to figure out the text search, which is also poorly documented.
So when we combine projection, text search and sorting, there doesn't seem to be any example anywhere and I just can't get it to work.
Does anyone know the reason for that problem?

This works for me:
var client = new MongoClient();
var db = client.GetDatabase("test");
var col = db.GetCollection<BigClass>("big");
await db.DropCollectionAsync(col.CollectionNamespace.CollectionName);
await col.Indexes.CreateOneAsync(Builders<BigClass>.IndexKeys.Text(x => x.Title));
await col.InsertManyAsync(new[]
{
new BigClass { Title = "One Jumped Over The Moon" },
new BigClass { Title = "Two went Jumping Over The Sun" }
});
var filter = Builders<BigClass>.Filter.Text("Jump Over");
// don't need to Include(x => x.TextMatchScore) because it's already been included with MetaTextScore.
var projection = Builders<BigClass>.Projection.MetaTextScore("TextMatchScore").Include(x => x._id).Include(x => x.Title);
var sort = Builders<BigClass>.Sort.MetaTextScore("TextMatchScore");
var result = await col.Find(filter).Project<SmallClass>(projection).Sort(sort).ToListAsync();
I removed the include of the TextMatchScore. It still comes back, because it was included by the MetaTextScore("TextMatchScore").
Documentation is a work in progress. We tackle the major use cases first as those hit the most people. This use case isn't that common and hasn't been documented. We certainly accept pull requests, both for code and documentation. Also, feel free to file a documentation ticket at jira.mongodb.org under the CSHARP project.

Solution which works in MongoDB.Driver 2.x is as follows. What is important is to not do Include in Projection, as it will erase default one, (or remember to add proper projection)
Query:
{
"find":"SoceCollection",
"filter":{
"$text":{
"$search":"some text to search"
}
},
"sort":{
"TextScore":{
"$meta":"textScore"
}
},
"projection":{
"TextScore":{
"$meta":"textScore"
},
"_id":0,
"CreatedDate":0
},
"limit":20,
"collation":{
"locale":"en",
"strength":1
} ...
CODE
var sort = Builders<BigModel>.Sort.MetaTextScore(nameof(LightModel.TextScore));
var projection = Builders<BigModel>.Projection
.MetaTextScore(nameof(LightModel.TextScore))
.Exclude(x => x.Id)
.Exclude(x => x.CreatedDate);
return await Collection()
.Find(filter, new FindOptions { Collation = new Collation("en", strength: CollationStrength.Primary) })
.Project<LightModel>(projection)
.Sort(sort)
.Limit(20)
.ToListAsync();

Element Metrics with Custom collection in C#

I am trying to figure out the best way to organise a bunch of my data classes, given I need to be able to access some metrics on them all at some point.
Here's a snippet of my OR class:
public enum status { CLOSED, OPEN }
public class OR
{
public string reference { get; set; }
public string title { get; set; }
public status status { get; set; }
}
Not every OR I initialise will have values for all properties. I want to be able to 'collect' thousands of these together in such a way that I can easily obtain a count of how many OR objects had a value set. For example:
OR a = new OR() { reference = "a" }
OR b = new OR() { reference = "b", title = "test" }
OR c = new OR() { reference = "c", title = "test", status = status.CLOSED }
Now these are somehow collected in such a way I can do (pseudo):
int titleCount = ORCollection.titleCount;
titleCount = 2
I would also want to be able gather metrics for the enum type properties, for example retrieve a Dictionary from the collection that looks like:
Dictionary<string, int> statusCounts = { "CLOSED", 1 }
The reason for wanting access to these metrics is that I am building two collections of ORs and comparing them side-by-side for any differences (they should be identical). I want to be able to compare their metrics at this higher level first, then break-down where precisely they differ.
Thanks for any light that can be shed on how to accomplish this. :-)

... to 'collect' thousands of these
Thousands is not a huge number. Just use a List<OR> and you can get all your metrics with Linq queries.
For example:
List<OR> orList = ...;
int titleCount = orList
.Where(o => ! string.IsNullOrEmpty(o.title))
.Count();
Dictionary<status, int> statusCounts = orList
.GroupBy(o => o.status)
.ToDictionary(g => g.Key, g => g.Count());

The existing answers using Linq are absolutely great and really elegant, so the idea presented below is just for posterity.
Here is a (very rough) reflection-based program that will alow you to count the "valid" properties in any collection of objects.
The validators are defined by you in the Validators dictionary so that you can easily change what is a valid/invalid value for each property. You may find it useful as a concept if you end up with objects having tons of properties and don't want to have to write inline linq metrics on the actual collection itself for every single property.
You could weaponise this as a function and then run it against both collections, giving you a basis to report on the exact differences between both since it records the references to the individual objects in the final dictionary.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Reflection;
namespace reftest1
{
public enum status { CLOSED, OPEN }
public class OR
{
public string reference { get; set; }
public string title { get; set; }
public status status { get; set; }
public int foo { get; set; }
}
//creates a dictionary by property of objects whereby that property is a valid value
class Program
{
//create dictionary containing what constitues an invalid value here
static Dictionary<string,Func<object,bool>> Validators = new Dictionary<string, Func<object,bool>>
{
{"reference",
(r)=> { if (r ==null) return false;
return !String.IsNullOrEmpty(r.ToString());}
},
{"title",
(t)=> { if (t ==null) return false;
return !String.IsNullOrEmpty(t.ToString());}
},
{"status", (s) =>
{
if (s == null) return false;
return !String.IsNullOrEmpty(s.ToString());
}},
{"foo",
(f) =>{if (f == null) return false;
return !(Convert.ToInt32(f.ToString()) == 0);}
}
};
static void Main(string[] args)
{
var collection = new List<OR>();
collection.Add(new OR() {reference = "a",foo=1,});
collection.Add(new OR(){reference = "b", title = "test"});
collection.Add(new OR(){reference = "c", title = "test", status = status.CLOSED});
Type T = typeof (OR);
var PropertyMetrics = new Dictionary<string, List<OR>>();
foreach (var pi in GetProperties(T))
{
PropertyMetrics.Add(pi.Name,new List<OR>());
foreach (var item in collection)
{
//execute validator if defined
if (Validators.ContainsKey(pi.Name))
{
//get actual property value and compare to valid value
var value = pi.GetValue(item, null);
//if the value is valid, record the object into the dictionary
if (Validators[pi.Name](value))
{
var lookup = PropertyMetrics[pi.Name];
lookup.Add(item);
}
}//end trygetvalue
}
}//end foreach pi
foreach (var metric in PropertyMetrics)
{
Console.WriteLine("Property '{0}' is set in {1} objects in collection",metric.Key,metric.Value.Count);
}
Console.ReadLine();
}
private static List<PropertyInfo> GetProperties(Type T)
{
return T.GetProperties(BindingFlags.Public | BindingFlags.Instance).ToList();
}
}
}

You can get the title count using this linq query:
int titleCount = ORCollection
.Where(x => !string.IsNullOrWhiteSpace(x.title))
.Count();
You could get the count of closed like this:
int closedCount = ORCollection
.Where(x => x.status == status.CLOSED)
.Count();
If you were going to have larger collections or you access the values a lot it might be worth creating a custom collection implementation that stores the field counts, it could then increment/decrement these values as you add and remove items. You could also store a dictionary of status counts in this custom collection that gets updated as you add and remove items.

How to convert from 'System.Linq.IQueryable<System.Collections.Generic.List<Model.Record> to List<Record>

How does one retrieve the list in a model?
This is what I'm trying:
private void cbxPlayers_SelectedValueChanged(object sender, EventArgs e)
{
List<Record> records = new List<Record>();
string selectedPlayer = cbxPlayers.SelectedItem.ToString();
using (ProgressRecordContext context = new ProgressRecordContext())
{
records = (from Player in context.Players
where Player.Name == selectedPlayer
select Player.Records).ToList<Record>();
}
}
That doesn't work however, what am I missing?
These are the models in case they're needed:
public class Player
{
[Key][DatabaseGenerated(DatabaseGeneratedOption.None)]
public int AccountNumberId { get; set; }
public string Name { get; set; }
public virtual List<Record> Records { get; set; }
}
public class Record
{
public int RecordId { get; set; }
public int AccountNumberId { get; set; }
public double Level { get; set; }
public int Economy { get; set; }
public int Fleet { get; set; }
public int Technology { get; set; }
public int Experience { get; set; }
public DateTime TimeStamp { get; set; }
public virtual Player Player { get; set; }
}
EDIT: Here's the error messages:
Error 1 'System.Linq.IQueryable>' does not contain a definition for 'ToList' and the best extension method overload 'System.Linq.ParallelEnumerable.ToList(System.Linq.ParallelQuery)' has some invalid arguments
Error 2 Instance argument: cannot convert from 'System.Linq.IQueryable>' to 'System.Linq.ParallelQuery'
EDIT:
I see that I probably wasn't very clear with what I was trying to do. I eventually worked out a way to do what I wanted and here it is:
private void cbxPlayers_SelectedValueChanged(object sender, EventArgs e)
{
lstvRecords.Items.Clear();
if(cbxPlayers.SelectedIndex == -1)
{
return;
}
string selectedPlayer = cbxPlayers.SelectedItem.ToString();
using (ProgressRecordContext context = new ProgressRecordContext())
{
var records = from Player in context.Players
from Record in context.Records
where Player.Name == selectedPlayer &&
Player.AccountNumberId == Record.AccountNumberId
select new
{
Level = Record.Level,
Economy = Record.Economy,
Fleet = Record.Fleet,
Technology = Record.Technology,
Experience = Record.Experience,
TimeStamp = Record.TimeStamp
};
foreach (var element in records)
{
string[] elements = {element.Level.ToString(),
element.Economy.ToString(),
element.Fleet.ToString(),
element.Technology.ToString(),
element.Experience.ToString(),
element.TimeStamp.ToString()
};
ListViewItem lvi = new ListViewItem(elements);
lstvRecords.Items.Add(lvi);
}
}
}
Is there a better way to write that query or is the way that I've done it correct?

No idea why you're getting ParallelQuery - unless you've got some wacky usings in your source file.
In any case, you appear to have an enumerable of enumerables - try SelectMany (note you need using System.Linq; for this to work as an extension method, too):
records = (from Player in context.Players
where Player.Name == selectedPlayer
select Player.Records).SelectMany(r => r).ToList();
Also - unless you intend to add/remove to/from that list, you should just use an array, i.e. use .ToArray().
As pointed out by #Tim S (+1) - if you expect only a single player here then you should be using SingleOrDefault() to get the single player - whose Records you then turn into an array/list.

Your problem is that Player.Records is a List<Record>, and you are getting an IEnumerable<List<Record>> (i.e. 0 to many player's records) from your query, so .ToList() gets you a List<List<Record>>. If there are multiple players with the same name and you want it to collect the records from all of them, use Andras Zoltan's solution. If you want to ensure (via throwing an exception if there are 0 or more than 1 results) that exactly one player has the given name, and only his records are returned, use one of these solutions: (key change being .Single() - also take a look at SingleOrDefault to see if it fits your needs better)
//I prefer this solution for its conciseness and clarity.
records = context.Players.Single(Player => Player.Name == selectedPlayer).Records;
//if you'd like to use the LINQ query format, I'd recommend this.
records = (from Player in context.Players
where Player.Name == selectedPlayer
select Player).Single().Records;
//this is more similar to your original query.
records = (from Player in context.Players
where Player.Name == selectedPlayer
select Player.Records).Single().ToList();

If you change
List<Record> records = new List<Record>();
to
var records = new List<List<Record>>();
Does it work? If a Player has a list of Records, it looks like your query is returning a List of a List of Records.
Edit:
There, fixed the return list... either way this is probably not the solution you're looking for, just highlighting what the problem is.

You could try refactoring your query
records = context.Players.First(player => player.Name == selectedPlayer).Records.ToList();