I am fairly new to programming and C#, and I am creating a game using C# 9.0 in which all instances of Entity have certain stats. I want to be able to change their private data fields using properties, though I'm not entirely sure how properties work. I know they are useful in encapsulation as getters and setters.
Context:
I am trying to optimize code and decrease memory usage where possible
The byte field str should be variable (through events, training, etc.), but have a "ceiling" and "floor"
If dog.str = 253, then dog.Str += 5; should result in dog.str being 255
If dog.str = 2, then dog.Str -= 5; should result in dog.str being 0
private byte str;
public short Str
{
get => str;
set
{
if (value > byte.MaxValue) str = byte.MaxValue; //Pos Overflow
else if (value < byte.MinValue) str = byte.MinValue; //Neg Overflow
else str = (byte)value;
}
}
Questions:
Since the property is of datatype Short, does it create a new private backing field that consumes memory? Or is value/Str{set;} just a local variable that later disappears?
Does the property public float StrMod {get => (float)(str*Effects.Power);} create a backing field? Would it be better to just create a method like public float getStrMod() instead?
Is this code optimal for what I'm trying to achieve? Is there some better way to do this, considering the following?
If for some reason the Short overflowed (unlikely in this scenario, but there may be a similar situation), then I would end up with the same problem. However, if extra memory allocation isn't an issue, then I could use an int.
The {get;} will return a Short, which may or may not be an issue.
Question 1:
No it doesn't, its backing field is str.
Question 2:
Profile your code first instead of making random changes in hope to reduce memory usage.
"Premature optimization is the root of all evil", do you really have such issues at this point ?
Personally I'd use int and use same type for property and backing field for simplicity.
This would avoid wrapping such as assigning 32768 which would then result as -32768 for short.
Side note, don't think that using byte necessarily results in 1 byte, if you have tight packing requirements then you need to look at StructLayoutAttribute.Pack.
Other than that I see nothing wrong with your code, just get it to work first then optimize it!
Here's how I'd write your code, maybe you'll get some ideas from it:
class Testing
{
private int _value;
public int Value
{
get => _value;
set => _value = Clamp(value, byte.MinValue, byte.MaxValue);
}
private static int Clamp(int value, int min, int max)
{
return Math.Max(min, Math.Min(max, value));
}
}
EDIT:
Different scenarios:
class Testing
{
private int _value1;
public int Value1 // backing field is _value1
{
get => _value1;
set => _value1 = value;
}
public int Value2 { get; set; } // adds a backing field
public int Value3 { get; } // adds a backing field
public int Value4 => 42; // no backing field
}
As you might have guessed, properties are syntactic sugar for methods, they can do 'whatever' under the hood compared to a field which can only be assigned a value to.
Also, one difference with a method is that you can browse its value in the debugger, that's handy.
Suggested reading:
https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/properties
Finally, properties are expected to return quickly, else write a method, and possibly async if it's going to take a while (advantage to method in this case as properties can't be async).
#aybe answer covers main thing about you question. I would like to add additional info to your 2nd question. You should consider on which platform you write application. There is a word term:
In computing, a word is the natural unit of data used by a particular
processor design. A word is a fixed-sized piece of data handled as a
unit by the instruction set or the hardware of the processor. The
number of bits in a word (the word size, word width, or word length)
is an important characteristic of any specific processor design or
computer architecture.
If processor has 64 bit word, then every variable which type is less than 64 bits will still occupy 64 bits in memory. Keep in mind that variable of given type will be handled as given type and size in memory doesn't impact range, overflow or underflow - arithmetic will be processed for given type.
In short - if you have 64-bit desktop processor and you will use only short variables, then you will not observe any memory savings in comparison to declaring int variables.
I am running a simulation part of which requires sort of array of pairs of values.
When I used Array.Sort(v1,v2) it sorts 2 arrays based on first and all the simulation takes roughly 9 ms.
But I need to sort based on first then second so I created array of structs. See my code below.
private struct ValueWithWeight : IComparable<ValueWithWeight>
{
public double Value;
public double Weight;
public int CompareTo(ValueWithWeight other)
{
int cmp = this.Value.CompareTo(other.Value);
if (cmp != 0)
return cmp;
else
return this.Weight.CompareTo(other.Weight);
}
}
void Usage()
{
ValueWithWeight[] data = FillData();
Array.Sort(data);
}
Now it takes roughly 27ms. Is there any better way to sort ?
Since you're going to extremely optimize it please consider following:
Array.Sort runs over your array and performs comparison. In your case, there will not be unboxing since you implemented an interface on structure.
Array.Sort performs swap of elements while sorting. Swapping is internally memmove. Your structure takes at least 16 bytes. You can try to reduce impact by allocating your double values in class. Class will always occupy IntPtr.Size bytes (because you will store pointers) so it should copy less bytes.
Background
One of the most used data-structures in our application is a custom Point struct. Recently we have been running into memory issues, mostly caused by an excessive number of instances of this struct.
Many of these instances contain the same data. Sharing a single instance would significantly help to reduce memory usage. However, since we are using structs, instances cannot be shared. It is also not possible to change it to a class, because the struct semantics are important.
Our workaround for this is to have a struct containing a single reference to a backing class, which contains the actual data. These flyweight dataclasses are stored in and retrieved from a factory to ensure no duplicates exist.
A narrowed down version of the code looks something like this:
public struct PointD
{
//Factory
private static class PointDatabase
{
private static readonly Dictionary<PointData, PointData> _data = new Dictionary<PointData, PointData>();
public static PointData Get(double x, double y)
{
var key = new PointData(x, y);
if (!_data.ContainsKey(key))
_data.Add(key, key);
return _data[key];
}
}
//Flyweight data
private class PointData
{
private double pX;
private double pY;
public PointData(double x, double y)
{
pX = x;
pY = y;
}
public double X
{
get { return pX; }
}
public double Y
{
get { return pY; }
}
public override bool Equals(object obj)
{
var other = obj as PointData;
if (other == null)
return false;
return other.X == this.X && other.Y == this.Y;
}
public override int GetHashCode()
{
return X.GetHashCode() * Y.GetHashCode();
}
}
//Public struct
public Point(double x, double y)
{
_data = Point3DDatabase.Get(x, y);
}
public double X
{
get { return _data == null ? 0 : _data.X; }
set { _data = PointDatabase.Get(value, Y); }
}
public double Y
{
get { return _data == null ? 0 : _data.Y; }
set { _data = PointDatabase.Get(X, value); }
}
}
This implementation ensures that the struct semantics are maintained, while ensuring only one instance of the same data is kept around.
(Please don't mention memory leaks or such, this is simplified example code)
The Problem
Although above approach works to lower our memory usage, the performance is horrendous. A project in our application can easily contain a million different points or more. As a result, the lookup of a PointData instance is very costly. And this lookup has to be done whenever a Point is manipulated, which, as you can probably guess, is what our application is all about. As a result, this approach is not suitable for us.
As an alternative, we could make two versions of the Point class: one with backing flyweight as above, and one containing its own data (with possible duplicates). All (short-lived) calculations could be done in the second class, while when storing the Point for longer durations they could be converted to the first, memory-efficient class. However, this means that all the users of the Point class have to be inspected and adjusted to this scheme, something which is not feasible for us.
What we are looking for is an approach which meets below criteria:
When there are multiple Points with the same data, the memory usage should be lower than having a different struct instance for each of these.
Performance should not be much worse than working directly on primitive data in the struct.
Struct semantics should be maintained.
The 'Point' interface should remain the same (i.e. classes that use 'Point' should not have to be changed).
Is there any way we can improve our approach towards these criteria? Or can anyone suggest a different approach we can attempt?
Rather than re-work an entire data structure and programming model, my go-to solution for performance and memory issues is to cache, pre-fetch and most importantly cull you data when it is not needed.
Think of it this way. On a graph, you cannot display few millions of points at once because you run out of pixels (you should occlusion-cull these points). Similarly, in a table, there isn't enough vertical space on screen (you need data set truncation). Consider streaming data from your source file as you need it. If your source data structure is not appropriate for dynamic retrieval, consider an intermediate, temporary file format. This is one of the ways .Net's JITer works so quickly!
I want to simulate a flash memory architecture in C#. More specifically the architecture looks like the following:
Flash memory is a collection of blocks
1 block = 128 sectors
a sector is composed of a data area and spare area
data area = 8 kB
spare area = 16 B
I wanted to represent this in a struct or in a class but the problem is I don't know how to represent a certain amount of memory space in the code. I can't use int or char arrays since I don't know what is to be stored in that memory space.... I am not very sure but I think I can represent it using byte datatype....
Yes, it sounds like you want a byte array. For example:
public sealed class Block
{
private readonly Sector[] sectors = new Sector[128];
public Sector this[int index] { get { return sectors[index]; } }
}
public sealed class Sector
{
private readonly byte[] data = new byte[8 * 1024];
public byte this[int index]
{
get { return data[index]; }
set { data[index] = value; }
}
}
(You can model the "spare" area as well if you want - it's not clear whether you really need to though.)
That's only allowing single-byte-at-a-time access - you may well want to have GetData and SetData methods on Block which read/write chunks of data at a time. Hopefully this will get you started though.
For any arbitrary instance (collections of different objects, compositions, single objects, etc)
How can I determine its size in bytes?
(I've currently got a collection of various objects and i'm trying to determine the aggregated size of it)
EDIT: Has someone written an extension method for Object that could do this? That'd be pretty neat imo.
First of all, a warning: what follows is strictly in the realm of ugly, undocumented hacks. Do not rely on this working - even if it works for you now, it may stop working tomorrow, with any minor or major .NET update.
You can use the information in this article on CLR internals MSDN Magazine Issue 2005 May - Drill Into .NET Framework Internals to See How the CLR Creates Runtime Objects - last I checked, it was still applicable. Here's how this is done (it retrieves the internal "Basic Instance Size" field via TypeHandle of the type).
object obj = new List<int>(); // whatever you want to get the size of
RuntimeTypeHandle th = obj.GetType().TypeHandle;
int size = *(*(int**)&th + 1);
Console.WriteLine(size);
This works on 3.5 SP1 32-bit. I'm not sure if field sizes are the same on 64-bit - you might have to adjust the types and/or offsets if they are not.
This will work for all "normal" types, for which all instances have the same, well-defined types. Those for which this isn't true are arrays and strings for sure, and I believe also StringBuilder. For them you'll have add the size of all contained elements to their base instance size.
You may be able to approximate the size by pretending to serializing it with a binary serializer (but routing the output to oblivion) if you're working with serializable objects.
class Program
{
static void Main(string[] args)
{
A parent;
parent = new A(1, "Mike");
parent.AddChild("Greg");
parent.AddChild("Peter");
parent.AddChild("Bobby");
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bf =
new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
SerializationSizer ss = new SerializationSizer();
bf.Serialize(ss, parent);
Console.WriteLine("Size of serialized object is {0}", ss.Length);
}
}
[Serializable()]
class A
{
int id;
string name;
List<B> children;
public A(int id, string name)
{
this.id = id;
this.name = name;
children = new List<B>();
}
public B AddChild(string name)
{
B newItem = new B(this, name);
children.Add(newItem);
return newItem;
}
}
[Serializable()]
class B
{
A parent;
string name;
public B(A parent, string name)
{
this.parent = parent;
this.name = name;
}
}
class SerializationSizer : System.IO.Stream
{
private int totalSize;
public override void Write(byte[] buffer, int offset, int count)
{
this.totalSize += count;
}
public override bool CanRead
{
get { return false; }
}
public override bool CanSeek
{
get { return false; }
}
public override bool CanWrite
{
get { return true; }
}
public override void Flush()
{
// Nothing to do
}
public override long Length
{
get { return totalSize; }
}
public override long Position
{
get
{
throw new NotImplementedException();
}
set
{
throw new NotImplementedException();
}
}
public override int Read(byte[] buffer, int offset, int count)
{
throw new NotImplementedException();
}
public override long Seek(long offset, System.IO.SeekOrigin origin)
{
throw new NotImplementedException();
}
public override void SetLength(long value)
{
throw new NotImplementedException();
}
}
Not directly answers the question, but for those who are interested to investigate object sizes while debugging:
Start debugging in VS, make sure the Diagnostics Tools window is shown (Debug > Windows > Show Diagnostic Tools)
Set a breakpoint (optional)
Click Take Snapshot in the Memory Usage while paused
Explore the snapshot (optionally sort the object list alphabetically to find the type you're interested in)
For unmanaged types aka value types, structs:
Marshal.SizeOf(object);
For managed objects the closer i got is an approximation.
long start_mem = GC.GetTotalMemory(true);
aclass[] array = new aclass[1000000];
for (int n = 0; n < 1000000; n++)
array[n] = new aclass();
double used_mem_median = (GC.GetTotalMemory(false) - start_mem)/1000000D;
Do not use serialization.A binary formatter adds headers, so you can change your class and load an old serialized file into the modified class.
Also it won't tell you the real size in memory nor will take into account memory alignment.
[Edit]
By using BiteConverter.GetBytes(prop-value) recursivelly on every property of your class you would get the contents in bytes, that doesn't count the weight of the class or references but is much closer to reality.
I would recommend to use a byte array for data and an unmanaged proxy class to access values using pointer casting if size matters, note that would be non-aligned memory so on old computers is gonna be slow but HUGE datasets on MODERN RAM is gonna be considerably faster, as minimizing the size to read from RAM is gonna be a bigger impact than unaligned.
safe solution with some optimizations
CyberSaving/MemoryUsage code.
some case:
/* test nullable type */
TestSize<int?>.SizeOf(null) //-> 4 B
/* test StringBuilder */
StringBuilder sb = new StringBuilder();
for (int i = 0; i < 100; i++) sb.Append("わたしわたしわたしわ");
TestSize<StringBuilder>.SizeOf(sb ) //-> 3132 B
/* test Simple array */
TestSize<int[]>.SizeOf(new int[100]); //-> 400 B
/* test Empty List<int>*/
var list = new List<int>();
TestSize<List<int>>.SizeOf(list); //-> 205 B
/* test List<int> with 100 items*/
for (int i = 0; i < 100; i++) list.Add(i);
TestSize<List<int>>.SizeOf(list); //-> 717 B
It works also with classes:
class twostring
{
public string a { get; set; }
public string b { get; set; }
}
TestSize<twostring>.SizeOf(new twostring() { a="0123456789", b="0123456789" } //-> 28 B
This doesn't apply to the current .NET implementation, but one thing to keep in mind with garbage collected/managed runtimes is the allocated size of an object can change throughout the lifetime of the program. For example, some generational garbage collectors (such as the Generational/Ulterior Reference Counting Hybrid collector) only need to store certain information after an object is moved from the nursery to the mature space.
This makes it impossible to create a reliable, generic API to expose the object size.
This is impossible to do at runtime.
There are various memory profilers that display object size, though.
EDIT: You could write a second program that profiles the first one using the CLR Profiling API and communicates with it through remoting or something.
For anyone looking for a solution that doesn't require [Serializable] classes and where the result is an approximation instead of exact science.
The best method I could find is json serialization into a memorystream using UTF32 encoding.
private static long? GetSizeOfObjectInBytes(object item)
{
if (item == null) return 0;
try
{
// hackish solution to get an approximation of the size
var jsonSerializerSettings = new JsonSerializerSettings
{
DateFormatHandling = DateFormatHandling.IsoDateFormat,
DateTimeZoneHandling = DateTimeZoneHandling.Utc,
MaxDepth = 10,
ReferenceLoopHandling = ReferenceLoopHandling.Ignore
};
var formatter = new JsonMediaTypeFormatter { SerializerSettings = jsonSerializerSettings };
using (var stream = new MemoryStream()) {
formatter.WriteToStream(item.GetType(), item, stream, Encoding.UTF32);
return stream.Length / 4; // 32 bits per character = 4 bytes per character
}
}
catch (Exception)
{
return null;
}
}
No, this won't give you the exact size that would be used in memory. As previously mentioned, that is not possible. But it'll give you a rough estimation.
Note that this is also pretty slow.
Use Son Of Strike which has a command ObjSize.
Note that actual memory consumed is always larger than ObjSize reports due to a synkblk which resides directly before the object data.
Read more about both here MSDN Magazine Issue 2005 May - Drill Into .NET Framework Internals to See How the CLR Creates Runtime Objects.
AFAIK, you cannot, without actually deep-counting the size of each member in bytes. But again, does the size of a member (like elements inside a collection) count towards the size of the object, or a pointer to that member count towards the size of the object? Depends on how you define it.
I have run into this situation before where I wanted to limit the objects in my cache based on the memory they consumed.
Well, if there is some trick to do that, I'd be delighted to know about it!
For value types, you can use Marshal.SizeOf. Of course, it returns the number of bytes required to marshal the structure in unmanaged memory, which is not necessarily what the CLR uses.
I have created benchmark test for different collections in .NET: https://github.com/scholtz/TestDotNetCollectionsMemoryAllocation
Results are as follows for .NET Core 2.2 with 1,000,000 of objects with 3 properties allocated:
Testing with string: 1234567
Hashtable<TestObject>: 184 672 704 B
Hashtable<TestObjectRef>: 136 668 560 B
Dictionary<int, TestObject>: 171 448 160 B
Dictionary<int, TestObjectRef>: 123 445 472 B
ConcurrentDictionary<int, TestObject>: 200 020 440 B
ConcurrentDictionary<int, TestObjectRef>: 152 026 208 B
HashSet<TestObject>: 149 893 216 B
HashSet<TestObjectRef>: 101 894 384 B
ConcurrentBag<TestObject>: 112 783 256 B
ConcurrentBag<TestObjectRef>: 64 777 632 B
Queue<TestObject>: 112 777 736 B
Queue<TestObjectRef>: 64 780 680 B
ConcurrentQueue<TestObject>: 112 784 136 B
ConcurrentQueue<TestObjectRef>: 64 783 536 B
ConcurrentStack<TestObject>: 128 005 072 B
ConcurrentStack<TestObjectRef>: 80 004 632 B
For memory test I found the best to be used
GC.GetAllocatedBytesForCurrentThread()
For arrays of structs/values, I have different results with:
first = Marshal.UnsafeAddrOfPinnedArrayElement(array, 0).ToInt64();
second = Marshal.UnsafeAddrOfPinnedArrayElement(array, 1).ToInt64();
arrayElementSize = second - first;
(oversimplified example)
Whatever the approach, you really need to understand how .Net works to correctly interpret the results.
For instance, the returned element size is the "aligned" element size, with some padding.
The overhead and thus the size is different depending on the usage of a type: "boxed" on the GC heap, on the stack, as a field, as an array element.
(I wanted to know what would be the memory impact of using "dummy" empty structs (without any field) to mimic "optional" arguments of generics; making tests with different layouts involving empty structs, I can see that an empty struct uses (at least) 1 byte per element; I vaguely remember it is because .Net needs a different address for each field, which wouldn't work if a field really was empty/0-sized).
You can use reflection to gather all the public member or property information (given the object's type). There is no way to determine the size without walking through each individual piece of data on the object, though.
From Pavel and jnm2:
private int DumpApproximateObjectSize(object toWeight)
{
return Marshal.ReadInt32(toWeight.GetType().TypeHandle.Value, 4);
}
On a side note be careful because it only work with contiguous memory objects
Simplest way is: int size = *((int*)type.TypeHandle.Value + 1)
I know this is implementation detail but GC relies on it and it needs to be as close to start of the methodtable for efficiency plus taking into consideration how GC code complex is nobody will dare to change it in future. In fact it works for every minor/major versions of .net framework+.net core. (Currently unable to test for 1.0)
If you want more reliable way, emit a struct in a dynamic assembly with [StructLayout(LayoutKind.Auto)] with exact same fields in same order, take its size with sizeof IL instruction. You may want to emit a static method within struct which simply returns this value. Then add 2*IntPtr.Size for object header. This should give you exact value.
But if your class derives from another class, you need to find each size of base class seperatly and add them + 2*Inptr.Size again for header. You can do this by getting fields with BindingFlags.DeclaredOnly flag.
Arrays and strings just adds that size its length * element size.
For cumulative size of aggreagate objects you need to implement more sophisticated solution which involves visiting every field and inspect its contents.
For anyone looking for a rough approximation comparing the sizes of disparate object graphs/collections, just serialize to JSON - e.g.:
Console.WriteLine($"Size1:\t{(JsonConvert.SerializeObject(someBusyObject)).Length}")); Console.WriteLine($"Size2:\t{(JsonConvert.SerializeObject(someOtherObject)).Length}"));
In my case I have a bunch of IEnumerable's being pulled during a login I'm benchmarking, and I just wanted to roughly size them to see their relative weight.
They're expensive operations and won't give you direct heap allocation size or anything like that, but it was good enough for my use case and was readily available.