Improve efficiency of modifying object properties in a list - c#

I have a list of custom objects that I am working with. I need to find matching objects, and save two attributes to the object, and move on. I can't help but think that my method of working with these objects is sub-optimal. Given I am working with large volumes of data (in this instance a list with ~ 10000 objects, but in other instances significantly larger), I would appreciate any information that might help me optimize the process.
List<WebListingVerification> listings = new List<WebListingVerification>(); //This list is fully populated, and is actually passed into the function.
string sku = reader["vsr_sku"].ToString();
string vendorName = reader["v_name"].ToString();
string vendorSku = reader["vsr_vendor_sku"].ToString();
WebListingVerification listing = listings.Find(x => x.SKU == sku);
if(listing != null)
{
listings.Remove(listing);
listing.Vendor = vendorName;
listing.VendorSKU = vendorSku;
listings.Add(listing);
}
As you can see above, I first remove the listing, then edit it, and then re-add it. I imagine there is a way to safely edit the object in the list without running Remove / Add which would help a great deal, but I can't seem to find how to do it. I'm not sure if you could do a compound function off of the listings.Find call (listings.Find(x => x.SKU == sku).Vendor = "vendor") but it would be unsafe, as there will be null returns in this circumstance anyways so..
Any help optimizing this would be greatly appreciated.
Edit
Thank you for the comments, I did not understand the fact that the result of the List.Find function call is in fact a pointer to the object in the list, and not a copy of the object. This clears up my issue!
In addition, thank you for the additional answers. I was looking for a simple improvement, predominantly to remove the Add / Remove routines, but the additional answers give me some good ideas on how to write these routines in the future which may net some significant performance improvements. I've been focused on reporting tasks in the past few months, so this example snippet is very similar to probably 100 different routines where I am gathering data from various source databases. Again, I very much appreciate the input.

public class WebListingVerification
{
public string Sku { get; set; }
public string VendorName { get; set; }
public string VendorSku { get; set; }
}
public class ListingManager : IEnumerable <WebListingVerification>
{
private Dictionary<string, WebListingVerification> _webListDictionary;
public ListingManager(IEnumerable <WebListingVerification> existingListings)
{
if (existingListings == null)
_webListDictionary = new Dictionary<string, WebListingVerification>();
else
_webListDictionary = existingListings.ToDictionary(a => a.Sku);
}
public void AddOrUpdate (string sku, string vendorName, string vendorSku)
{
WebListingVerification verification;
if (false == _webListDictionary.TryGetValue (sku, out verification))
_webListDictionary[sku] = verification = new WebListingVerification();
verification.VendorName = vendorName;
verification.VendorSku = vendorSku;
}
public IEnumerator<WebListingVerification> GetEnumerator()
{
foreach (var item in _webListDictionary)
yield return item.Value;
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
}

If your items are unique, might I suggest a HashSet<T>?
HashSet<WebListingVerification> listings = new HashSet<WebListingVerification>();
string sku = reader["vsr_sku"].ToString();
string vendorName = reader["v_name"].ToString();
string vendorSku = reader["vsr_vendor_sku"].ToString();
if(listings.Contains(listing))
{
listings.Remove(listing);
listing.Vendor = vendorName;
listing.VendorSKU = vendorSku;
listings.Add(listing);
}
You'd have to roll your own IEqualityComparer<T> interface on the WebListingVerification object and match on the SKU, which I assume is unique.
public class WebListingVerification : IEqualityComparer<WeblistingVerification>
{
public string Sku { get; set; }
public bool Equals(WebListingVerification obj, WebListingVerification obj2)
{
if (obj == null && obj2 == null)
return true;
else if (obj == null | obj2 == null)
return false;
else if (obj.Sku == obj2.Sku)
return true;
else
return false;
}
public int GetHashCode(WebListingVerification obj)
{
return Sku.GetHashCode();
}
}
HashSet.Contains() performance is phenomenal on large datasets like this.

To speed up the lookup you could first convert your list into a dictionary. Note though if your update method is a method, you should not do the conversion inside the method, but outside the update loop.
var dictionary = listings.ToDictionary(l => l.SKU);
And get the item from the dictionary with the sku value.
WebListingVerification listing;
if (dictionary.TryGetValue(sku, out listing))
{
listing.Vendor = vendorName;
listing.VendorSKU = vendorSku;
}

No need to remove and add back the object into the list. Just;
if(listing != null)
{
listing.Vendor = vendorName;
listing.VendorSKU = vendorSku;
}

Related

c# SetIfChanged-Method with Expressions

I have to following C#-code quite a lot:
public void UpdateDB(Model.ResultContext db)
{
Model.Planning.Part dbPart = db.Parts.Where(/*someClause*/).FirstOrDefault();
//The awkward part
if (dbPart.Number != Number)
{
dbPart.Number = Number;
}
if (dbPart.NumberShort != NumberShort)
{
dbPart.NumberShort = NumberShort;
}
if (dbPart.Designation != Designation)
{
dbPart.Designation = Designation;
}
}
It is obviously kind of awkward to check every field and wrap it in if != then set
Yes, the check is needed because otherwise the database sees everything as changed columns.
The fields to set are auto-Properties:
public class Part
{
[MaxLength(36), MinLength(1)]
public string Number { get; set; } = null!;
[MaxLength(80)]
public string Designation { get; set; } = null!;
}
and I don't want to write an explicit setter for every field which of course could do the checking before setting.
So what I thought of is some Method ´SetIfChanged´ which is called like this to make the code more readable and less error-prone:
//Options
dbPart.SetIfChanged(dbPart.Number, this.Number);
dbPart.SetIfChanged(dbPart.Number = this.Number);
dbPart.SetIfChanged(Number, this.Number);
I think something like that is possible with expressions or lambdas but to be honest... I'm stuck with the syntax of declaring and calling such a method
Anybody can help me out?
Unfortunately, C# is lacking a number of things to help you with this (e.g. property refs or extension methods on reference objects), but you can use Reflection to help with this. It is likely to be quite slow, however.
With a method that takes a lambda, you can write a set method:
public static void SetIfDifferent<T>(Expression<Func<T>> getterFnE, T newVal) {
var me = (MemberExpression)getterFnE.Body;
var target = me.Expression;
var targetLambda = Expression.Lambda(target);
var prop = me.Member;
var oldVal = getterFnE.Compile().Invoke();
if ((oldVal == null && newVal != null) || !oldVal.Equals(newVal)) {
var obj = targetLambda.Compile().DynamicInvoke();
prop.SetValue(obj, newVal);
}
}
This would be used like:
SetIfDifferent(() => dbPart.Number, Number);
SetIfDifferent(() => dbPart.NumberShort, NumberShort);
SetIfDifferent(() => dbPart.Designation, Designation);
This would be slow because of the need to compile the Expression trees and use DynamicInvoke. One way to speed it up would be to pass in a setter and getter lambda instead, but that leads to as much duplication as your original code.
If you would be willing to pass the object and name of the property instead, you could use:
public static T GetValue<T>(this MemberInfo member, object srcObject) => (T)member.GetValue(srcObject);
public static void SetIfDifferent2<TObj, TField>(this TObj obj, string fieldName, TField newVal) {
var prop = typeof(TObj).GetProperty(fieldName);
var oldVal = prop.GetValue<TField>(fieldName);
if ((oldVal == null && newVal != null) || !oldVal.Equals(newVal))
prop.SetValue(obj, newVal);
}
Which you could use like:
dbPart.SetIfDifferent2(nameof(dbPart.Number), Number);
dbPart.SetIfDifferent2(nameof(dbPart.NumberShort), NumberShort);
dbPart.SetIfDifferent2(nameof(dbPart.Designation), Designation);
Unfortunately, it requires repeating dbPart unless you are willing to just put in the field name (e.g. "Number") but that will cause runtime errors if the field changes.
You could also cache the PropertyInfo instead of looking it up with GetProperty, but that is generally pretty fast and caching probably isn't worth it.
Well if you really need checking (lets say at the end you want to know if anything has been changed or not) you can use Reflection and loop through properties. but in your case no check is needed.
take this for instance:
if (dbPart.Number != Number)
{
dbPart.Number = Number;
}
true) if the value is different you are setting the new one
false) means that the new value and the old value are the same, so doesn't hurt to set it again
If you want to know if anything has changed at the end:
bool changed = false;
var type = dbPart.GetType();
foreach(var (PropertyInfo)pi in type.GetProperties()
{
if(pi.GetValue(dbPart) != newValue)
{
changed = true;
pi.SetValue(dbPart, newValue);
}
}
or you can do something like:
bool changed = dbPart.Number != Number || dbPart.Designation != Designation;
dbPart.Number = Number;
dbPart.Designation = Designation;

C# List<[ThreeDimensionalObject]>.Find("ValueOfFirstElement") and Return "ValueOfSecondElement"

I've searched in vain mainly because I don't think I know what to look for. I think I need to use the Find method for Lists in C# to do what I want, but I can't seem to get it right. This leads me to think one of two obvious scenarios is happening:
1) I'm going about this the wrong way (my money is on this one).
2) I'm going about this the right way, but I just don't understand the syntax for finding something.
Also, just to answer some of the "Why are you..." or "Why haven't you..." questions: If you can't tell, I'm figuring this out as I go along - and this is about the extent of what I've been able to teach myself thus far.
Here is what I'm trying to do:
First, I have a list<FitHeaderCard> object called HeaderBlock. and it is populated by this custom type:
public class FitHeaderCard
{
public string keyword;
public string value;
public string comment;
//Constructor
public FitHeaderCard()
{}
public FitHeaderCard(string fitHeaderCard)
{ // input: 80 byte string containing 3 fixed width delimited values.
keyword = fitHeaderCard.Substring(0, 8).Trim();
value = fitHeaderCard.Substring(10, 20).Trim();
comment = fitHeaderCard.(33,47).Trim();
}
public FitHeaderCard(FitHeaderCard fitHeaderCard)
{ //clone a FitHeaderCard
keyword = fitHeaderCard.keyword;
value = fitHeaderCard.value;
comment = fitHeaderCard.comment;
}
}
The header cards are fed into a list when read from a file on my hard drive (using a BinaryReader).
The FitHeader object code is as follows:
public class FitHeader
{
public string headerCard;
public string headerSize;
public FitHeaderCard fitHeaderCard = new FitHeaderCard();
public List<FitHeaderCard> HeaderBlock = new List<FitHeaderCard>();
//Constructor
public FitHeader()
{
}
public FitHeader(string _headerCard)
{
headerCard = _headerCard;
}
public FitHeaderCard FitHeaderCard
{
get;
private set;
}
//Methods
public void AddHeaderCard(string _headerCard)
{
FitHeaderCard = new FitHeaderCard(_headerCard);
HeaderBlock.Add(FitHeaderCard);
}
public List<FitHeaderCard> GetHeader()
{
return HeaderBlock;
}
public int GetHeaderSize();
{
return headerSize = //some convoluted math but it works!
}
//This is what I'm having trouble with
public FitHeaderCard GetFitHeaderCard(string _keyword)
{
HeaderBlock.Find("BITPIX");
fitHeaderCard = new FitHeaderCard(fitHeaderCard); // clone constructor
return fitHeaderCard;
}
}
So the GetFitHeaderCard method is supposed to take in a keyword, and return the the entire FitHeaderCard object found in the HeaderBlock list.
Optionally, I'd love to just get the Value back, possibly in a separate method, but for now, I can live with just referencing the value of FitHeaderCard.Value
Thanks in advance for your help! I look forward to face palming myself when an elegant and obvious answer is revealed to this newbie .Net coder!
The Find method of List requires a predicate(i.e. a function that takes in an element, does some calculation on it, and returns a boolean). Right now you're passing a string. That's not a predicate. What you might do instead is create either a separate method, anonymous function, or lambda that matches what I just specified:
(separate method; could also be local function)
...
//This is what I'm having trouble with
public FitHeaderCard GetFitHeaderCard(string _keyword)
{
fitHeaderCard = HeaderBlock.Find(FilterByBitPix);
fitHeaderCard = new FitHeaderCard(fitHeaderCard); // clone constructor
return fitHeaderCard;
}
public FitHeaderCard FilterByBitPix(FitHeaderCard item)
{
return item.keyword == "BITPIX";
}
}
OR (anonymous function)
...
//This is what I'm having trouble with
public FitHeaderCard GetFitHeaderCard(string _keyword)
{
fitHeaderCard = HeaderBlock.Find(delegate (FitHeaderCard item) { return item.keyword == "BITPIX"); };
fitHeaderCard = new FitHeaderCard(fitHeaderCard); // clone constructor
return fitHeaderCard;
}
}
OR (lambda)
...
//This is what I'm having trouble with
public FitHeaderCard GetFitHeaderCard(string _keyword)
{
fitHeaderCard = HeaderBlock.Find(item => item.keyword == "BITPIX");
fitHeaderCard = new FitHeaderCard(fitHeaderCard); // clone constructor
return fitHeaderCard;
}
}
List<FitHeaderCard> matches = HeaderBlock.Where(p=>p.SomePublicStringPropertyInYourObject.Contains(searchString));
You want to code with C#, my advice: learn linq queries ASAP. It is very powerful for extracting what you need from lists.
Given that you have a List<>, there are a number of LINQ-based queries you can use to extract this information. In your particular example, this is probably the simplest:
var returnValue = HeaderBlock.FirstOrDefault(hb => hb.keyword.Equals("BITPIX"))?.value;
As the name suggests FirstOrDefault, this will return either the first item it finds in the list that matches, or the default value (which in this case is NULL because it is a list of objects).
There are other LINQ extensions you could use, such as:
First, which throws an exception if it can't find the keword;
var first = HeaderBlock.FirstOrDefault(hb => hb.keyword.Equals("BITPIX")).value;
Where, which returns a collection;
var foundItems = HeaderBlock.Where(hb => hb.keyword.Equals("BITPIX"));
Single, which is similar to First, but will throw an exception if there is more than one;
var single = HeaderBlock.Single(hb => hb.keyword.Equals("BITPIX")).value;
and SingleOrDefault, which again is similar to Single but will return NULL if it can't find anything.
var singleOrDefault = HeaderBlock.SingleOrDefault(hb => hb.keyword.Equals("BITPIX"))?.value;
LINQ is really important and incredibly powerful, and there is a lot of literature available to help you (admittedly, I personally found the official documentation quite hard to read, and hard to grasp the actual syntax in usage).

converting entities to viewmodels for use with web api

I'm currently struggling with finding a better way to populate my ViewModel objects with my Entitiy objects. I have the following Web Api controller method:
[HttpGet]
public IEnumerable<ClientSearchViewModel> FindClients(string query)
{
var clients = _clientService.SearchClient(query).ToList();
var returnClients = new List<ClientSearchViewModel>();
foreach (var client in clients)
{
returnClients.Add(new ClientSearchViewModel(client));
}
return returnClients;
}
And I'm doing this in my ClientSearchViewModel constructor:
public ClientSearchViewModel(Client client)
{
this.Id = client.Id;
this.FirstName = client.PersonName.FirstName;
this.LastName = client.PersonName.LastName;
}
Is there another way other than going through the list of returned objects and creating a new ViewModel list?
I strongly suggest use of a mapping plugin for this, such as:
AutoMapper
or
ValueInjector
Plugins like this will allow you to map between the objects being used internally or in your data layer, with your external objects (DTOs/ViewModels). They handle a number of things out of the box such as automatic mapping of any like named properties with the same type, but also allow for a lot of control in the specific mapping of properties or types, for those times when you need something more custom.
For a brief comparison of the two, there isn't much better than hearing the authors themselves respond: AutoMapper vs ValueInjecter
Personally, I find ValueInjector to be quicker to use, while having more control overall, but I also find it to be much less readable/inuitive than AutoMapper, which can require a bit more code to accomplish similar goals. As such, I'd pick the one that you find you and/or your team will prefer the syntax of and how easily you can grasp the concepts vs how much power you really need.
So I had the same miff... I can't say that I've benchmarked my solution, but it does seem to run reasonably fast...
3 bits:
public static T Transform<T>(this object convertFrom) where T : class, new()
{
return (T) (new ServiceExtension().Transform(convertFrom, typeof (T)));
}
private class ServiceExtension
{
public object Transform(object convertFrom, Type convertTo)
{
object _t = Activator.CreateInstance(convertTo);
if (convertFrom == null) return _t;
var convertType = convertFrom.GetType();
foreach (
var property in _t.GetType().GetProperties().Where(f => f.CanWrite && f.GetSetMethod(true).IsPublic)
)
{
if (property.GetCustomAttributes(typeof (TransformAttribute), true).Any())
{
var transform =
(property.GetCustomAttributes(typeof (TransformAttribute), true).FirstOrDefault() as
TransformAttribute);
var transformname = transform.RelatedField ?? property.Name;
if (convertType.GetProperty(transformname) == null)
throw new ArgumentException(
string.Format(
"We were unable to find property:\"{0}\" on {1}. Please check the RelativeField value on the {2} for \"{0}\"",
transformname, convertFrom.GetType().Name, convertTo.Name));
var theValue = convertType.GetProperty(transformname).GetValue(convertFrom);
if (isCollection(theValue))
{
foreach (var item in (theValue as ICollection))
{
var someVal = new object();
var newToType = property.PropertyType.GetGenericArguments().FirstOrDefault();
if (!String.IsNullOrEmpty(transform.FullyQualifiedName))
someVal =
Transform(
item.GetType().GetProperty(transform.FullyQualifiedName).GetValue(item),
newToType);
else
someVal = Transform(item, newToType);
if (property.GetValue(_t) == null)
throw new NullReferenceException(
string.Format(
"The following property:{0} is null on {1}. Likely this needs to be initialized inside of {1}'s empty constructor",
property.Name, _t.GetType().Name));
property.PropertyType.GetMethod("Add")
.Invoke(property.GetValue(_t), new[] {someVal});
//property.SetValue(_t, theValue.Transform(theValue.GetType()));
}
}
else
property.SetValue(_t, theValue);
}
//property.SetValue(_t, property.GetValue(convertFrom, null), null);
}
return _t;
}
public bool isCollection(object o)
{
return o is ICollection
|| typeof (ICollection<>).IsInstanceOfType(o);
}
}
public class TransformAttribute : Attribute
{
public string RelatedField { get; private set; }
public string FullyQualifiedName { get; set; }
public TransformAttribute()
{
}
public TransformAttribute(string relatedField)
{
RelatedField = relatedField;
}
}
such that the end result is: myObject.Transform()
But the decorations let you account for differences between your POCO and your ViewModel

List custom class members and type

This seems like the most basic thing ever but somehow I couldnt find the answer and couldnt figure it out.
Lets say I have a custom class:
public class WineCellar
{
public string year;
public string wine;
public double nrbottles;
}
Now I would like a function:
WineCellar ex = new WineCellar();
ex.members();
This should return: year, wine, nrbootles.
And:
ex.members().types();
Should return: string, string, double
I guess on the same note, lets say you have one instance {2010, Rioja, 6}. Is there syntax that returns these by indexing? i.e.
ex[1]
or
ex.{1}
that returns 2010?
Sorry for the basic question.
As Michelle said in the comments, this sounds like a wrong approach to a bigger problem.
However, if you do need this kind of things, you can get the using reflection:
//returns a list of propertyInfo objects for the class
// with all kinds of usefull information
public List<PropertyInfo> GetMemberInfos()
{
return this.GetType().GetProperties().ToList();
}
//returns a list of property names
public List<string> GetMemberNames
{
return this.GetType().GetProperties().Select(pi => pi.Name).ToList();
}
//returns a list of names of the property types
public List<string> GetMemberTypeNames
{
return this.GetType().GetProperties().Select(pi => pi.PropertyType.Name).ToList();
}
//indexer that uses the property name to get the value
//since you are mixing types, you can't get more specific than object
public object this[string property]
{
get { return this.GetType().GetProperty(property).GetValue(this); }
set { this.GetType().GetProperty(property).SetValue(this, value); }
}
//indexer that uses the property index in the properties array to get the value
public object this[int index]
{
get { return this.GetType().GetProperties()[index].GetValue(this); }
set { this.GetType().GetProperties()[index].SetValue(this, value); }
}
Note that all of these methods are very slow, because in general, reflection is slow. You can try to cache some thing to speed it up.
Also, the last method is downright dangerous. It will (try to) read and write to an array that does not have a guaranteed order. In fact, the documentation specifies:
The GetProperties method does not return properties in a particular
order, such as alphabetical or declaration order. Your code must not
depend on the order in which properties are returned, because that
order varies.
For example, if you change your class to:
public class WineCellar
{
public string year;
public string region;
public string wine;
public double nrbottles;
}
and you were used to using winecellar[1] = "Pinot Noir" that will most likely now update the region property, instead of the wine property.
This is how you would implement Members method (In case if you wanted property names as strings)
public List<string> Members()
{
List<string> propNames = new List<string>();
foreach (var prop in typeof(WineCellar).GetProperties())
{
propNames.Add(prop.Name);
}
return propNames;
}
And this is how you would implement Types (In same case)
public List<string> Types()
{
List<string> propTypes = new List<string>();
foreach (var prop in typeof(WineCellar).GetProperties())
{
propTypes.Add(prop.PropertyType.ToString());
}
return propTypes ;
}
And the last thing if you want to get values of the parameters like this ex[n] you can just make a simple indexer in you class like this
public string this[int n]
{
get
{
int current = 0;
foreach (var prop in typeof(WineCellar).GetProperties())
{
if (current == n)
return prop.GetValue(this, null).ToString();
current++;
}
return null;
}
}
but for these methods to work you should change your variables into properties like this
public class WineCellar
{
public string Year { get; set; }
public string Wine { get; set; }
public double Nrbottles { get; set; }
}
You can use reflection
foreach (var prop in typeof(WineCellar).GetProperties())
{
if (prop.PropertyType == typeof(double) || prop.PropertyType == typeof(double?))
{
}
}
to get the value, you can do:
prop.GetValue(obj);

How should I use properties and what should be structure of my class for using indexers across multiple classes

I need help as to how do I go about the structure of classes. How do I use Indexers? I want to have something like
Company.Employees[empId].Employee["Designation"].Salary
To be more specific something like
Grid.Rows[rowIndex].Columns["CurrentColumnName"].Width
Add a method like
public string this[string s]
{
get{
if(s == ...)
return this.property;
}
}
Yet, this seems to be more a Situation for Collections, but
see here for a complete example.
Actually indexers are used to get element by index, and your EmpId is not a good candidate for indexing as these may be compost or non sequential.
If you still want to use it here is the code. It will mimic as Indexer but its modified version.
class Employee
{
public int EmpId { get; set; }
public float Salary { get; set; }
public string Designation { get; set; }
}
class Employees
{
List<Employee> EmpList = new List<Employee>();
public Employee this[int empId]
{
get
{
return EmpList.Find(x => x.EmpId == empId);
}
}
}
I would rather have a method because I can make it generic.
public T GetPropertyValue<T>(string property)
{
var propertyInfo = GetType().GetProperty(property);
return (T)propertyInfo.GetValue(this, null);
}
var emp = employee.GetPropertyValue<Employee>("Designation");
var salary = emp.Salary;
That said... Be careful for having so many dot notations. When you get that NullReferenceException on your line in a log file, it is very difficult to find out what exactly was null. So rather break things up a bit and have more lines then you have less trouble of resolving bugs.

Categories