Given a Queue<MyMessage>, where MyMessage is the base class for some types of messages: all message types have different fields, so they will use a different amount of bytes. Therefore it would make sense to measure the fill level of this queue in terms of bytes rather than of elements present in the queue.
In fact, since this queue is associated with a connection, I could better control the message flow, reducing the traffic if the queue is nearly full.
In order to get this target, I thought to wrap a simple Queue with a custom class MyQueue.
public class MyQueue
{
private Queue<MyMessage> _outputQueue;
private Int32 _byteCapacity;
private Int32 _currentSize; // number of used bytes
public MyQueue(int byteCapacity)
{
this._outputQueue = new Queue<MyMessage>();
this._byteCapacity = byteCapacity;
this._currentSize = 0;
}
public void Enqueue(MyMessage msg)
{
this._outputQueue.Enqueue(msg);
this._currentSize += Marshal.SizeOf(msg.GetType());
}
public MyMessage Dequeue()
{
MyMessage result = this._outputQueue.Dequeue();
this._currentSize -= Marshal.SizeOf(result.GetType());
return result;
}
}
The problem is that this is not good for classes, because Marshal.SizeOf throws an ArgumentException exception.
Is it possible to calculate in some way the size of an object (instance of a class)?
Are there some alternatives to monitor the fill level of a queue in terms of bytes?
Are there any queues that can be managed in this way?
UPDATE: As an alternative solution I could add a method int SizeBytes() on each message type, but this solution seems a little ugly, although it would perhaps be the most efficient since You cannot easily measure a reference type.
public interface MyMessage
{
Guid Identifier
{
get;
set;
}
int SizeBytes();
}
The classes that implement this interface must, in addition to implementing the SizeBytes() method, also implement an Identifier property.
public class ExampleMessage
{
public Guid Identifier { get; set; } // so I have a field and its Identifier property
public String Request { get; set; }
public int SizeBytes()
{
return (Marshal.SizeOf(Identifier)); // return 16
}
}
The sizeof operator can not be used with Guid because it does not have a predefined size, so I use Marshal.SizeOf(). But at this point perhaps I should use the experimentally determined values: for example, since Marshal.SizeOf() returns 16 for a Guid and since a string consists of N char, then the SizeBytes() method could be as following:
public int SizeBytes()
{
return (16 + Request.Length * sizeof(char));
}
If you could edit the MyMessage base class with a virtual method SizeOf(), then you could have the message classes use the c# sizeof operator on its primitive types. If you can do that, the rest of your code is gold.
You can get an indication of the size of your objects by measuring the length of their binary serialization. Note that this figure will typically be higher than you expect, since .NET may also include metadata in the serialized representation. This approach would also require all your classes to be marked with the [Serializable] attribute.
public static long GetSerializedSize(object root)
{
using (var memoryStream = new MemoryStream())
{
var binaryFormatter = new BinaryFormatter();
binaryFormatter.Serialize(memoryStream, root);
return memoryStream.Length;
}
}
Related
I'm want a unique ID (preferably static, without computation) for each class implementation, but not instance. The most obvious way to do this is just hardcode a value in the class, but keeping the values unique becomes a task for an human and isn't ideal.
class Base
{
abstract int GetID();
}
class Foo: Base
{
int GetID() => 10;
}
class Bar: Base
{
int GetID() => 20;
}
Foo foo1 = new Foo();
Foo foo2 = new Foo();
Bar bar = new Bar();
foo1.GetID() == foo2.GetID();
foo1.GetID() != bar.GetID()
The class name would be an obvious unique identifier, but I need an int (or fixed length bytes). I pack the entire object into bytes, and use the id to know what class it is when I unpack it at the other end.
Hashing the class name every time I call GetID() seems needlessly process heavy just to get an ID number.
I could also make an enum as a lookup, but again I need to populate the enum manually.
EDIT: People have been asking important questions, so I'll put the info here.
Needs to be unique per class, not per instance (this is why the identified duplicate question doesn't answer this one).
ID value needs to be persistent between runs.
Value needs to be fixed length bytes or int. Variable length strings such as class name are not acceptable.
Needs to reduce CPU load wherever possible (caching results or using assembly based metadata instead of doing a hash each time).
Ideally, the ID can be retrieved from a static function. This means I can make a static lookup function that matches ID to class.
Number of different classes that need ID isn't that big (<100) so collisions isn't a major concern.
EDIT2:
Some more colour since people are skeptical that this is really needed. I'm open to a different approach.
I'm writing some networking code for a game, and its broken down into message objects. Each different message type is a class that inherits from MessageBase, and adds it's own fields which will be sent.
The MessageBase class has a method for packing itself into bytes, and it sticks a message identifier (the class ID) on the front. When it comes to unpacking it at the other end, I use the identifier to know how to unpack the bytes. This results in some easy to pack/unpack messages and very little overhead (few bytes for ID, then just class property values).
Currently I hard code an ID number in the classes, but it doesn't seem like the best way of doing things.
EDIT3: Here is my code after implementing the accepted answer.
public class MessageBase
{
public MessageID id { get { return GetID(); } }
private MessageID cacheId;
private MessageID GetID()
{
// Check if cacheID hasn't been intialised
if (cacheId == null)
{
// Hash the class name
MD5 md5 = MD5.Create();
byte[] md5Bytes = md5.ComputeHash(Encoding.UTF8.GetBytes(GetType().AssemblyQualifiedName));
// Convert the first few bytes into a uint32, and create the messageID from it and store in cache
cacheId = new MessageID(BitConverter.ToUInt32(md5Bytes, 0));
}
// Return the cacheId
return cacheId;
}
}
public class Protocol
{
private Dictionary<Type, MessageID> messageTypeToId = new Dictionary<Type, int>();
private Dictionary<MessageID, Type> idToMessageType = new Dictionary<int, Type>();
private Dictionary<MessageID, Action<MessageBase>> handlers = new Dictionary<int, Action<MessageBase>>();
public Protocol()
{
// Create a list of all classes that are a subclass of MessageBase this namespace
IEnumerable<Type> messageClasses = from t in Assembly.GetExecutingAssembly().GetTypes()
where t.Namespace == GetType().Namespace && t.IsSubclassOf(typeof(MessageBase))
select t;
// Iterate through the list of message classes, and store their type and id in the dicts
foreach(Type messageClass in messageClasses)
{
MessageID = (MessageID)messageClass.GetField("id").GetValue(null);
messageTypeToId[messageClass] = id;
idToMessageType[id] = messageClass;
}
}
}
Given that you can get a Type by calling GetType on the instance, you can easily cache the results. That reduces the problem to working out how to generate an ID for each type. You'd then call something like:
int id = typeIdentifierCache.GetIdentifier(foo1.GetType());
... or make GetIdentifier accept object and it can call GetType(), leaving you with
int id = typeIdentifierCache.GetIdentifier(foo1);
At that point, the detail is all in the type identifier cache.
A simple option would be to take a hash (e.g. SHA-256, for stability and making it very unlikely that you'll encounter collisions) of the fully-qualified type name. To prove that you have no collisions, you could easily write a unit test that runs over all the type names in the assembly and hashes them, then checks there are no duplicates. (Even that might be overkill, given the nature of SHA-256.)
This is all assuming that the types are in a single assembly. If you need to cope with multiple assemblies, you may want to hash the assembly-qualified name instead.
Here is one suggestion. I have used a sha256 byte array which is guaranteed to be a fixed size and astronomically unlikely to have a collision. That may well be overkill, you can easily substitute it out for something smaller. You could also use the AssemblyQualifiedName rather than FullName if you need to worry about version differences or the same class name in multiple assemblies
Firstly, here are all my usings
using System;
using System.Collections.Concurrent;
using System.Text;
using System.Security.Cryptography;
Next, a static cached type hasher object to remember the mapping between your types and the resulting byte arrays. You don't need the Console.WriteLines below, they are just there to demonstrate that you are not computing it over and over again.
public static class TypeHasher
{
private static ConcurrentDictionary<Type, byte[]> cache = new ConcurrentDictionary<Type, byte[]>();
public static byte[] GetHash(Type type)
{
byte[] result;
if (!cache.TryGetValue(type, out result))
{
Console.WriteLine("Computing Hash for {0}", type.FullName);
SHA256Managed sha = new SHA256Managed();
result = sha.ComputeHash(Encoding.UTF8.GetBytes(type.FullName));
cache.TryAdd(type, result);
}
else
{
// Not actually required, but shows that hashing only done once per type
Console.WriteLine("Using cached Hash for {0}", type.FullName);
}
return result;
}
}
Next, an extension method on object so that you can ask for anything's id. Of course if you have a more suitable base class, it doesn't need to go on object per se.
public static class IdExtension
{
public static byte[] GetId(this object obj)
{
return TypeHasher.GetHash(obj.GetType());
}
}
Next, here are some random classes
public class A
{
}
public class ChildOfA : A
{
}
public class B
{
}
And finally, here is everything put together.
public class Program
{
public static void Main()
{
A a1 = new A();
A a2 = new A();
B b1 = new B();
ChildOfA coa = new ChildOfA();
Console.WriteLine("a1 hash={0}", Convert.ToBase64String(a1.GetId()));
Console.WriteLine("b1 hash={0}", Convert.ToBase64String(b1.GetId()));
Console.WriteLine("a2 hash={0}", Convert.ToBase64String(a2.GetId()));
Console.WriteLine("coa hash={0}", Convert.ToBase64String(coa.GetId()));
}
}
Here is the console output
Computing Hash for A
a1 hash=VZrq0IJk1XldOQlxjN0Fq9SVcuhP5VWQ7vMaiKCP3/0=
Computing Hash for B
b1 hash=335w5QIVRPSDS77mSp43if68S+gUcN9inK1t2wMyClw=
Using cached Hash for A
a2 hash=VZrq0IJk1XldOQlxjN0Fq9SVcuhP5VWQ7vMaiKCP3/0=
Computing Hash for ChildOfA
coa hash=wSEbCG22Dyp/o/j1/9mIbUZTbZ82dcRkav4olILyZs4=
On the other side, you would use reflection to iterate all of the types in your library and store a reverse dictionary of hash to type.
Have not seen you answer the question if the same value needs to persist between different runs, but if all you need is a unique ID for a class, then use the built-in and simple GetHashCode method:
class BaseClass
{
public int ClassId() => typeof(this).GetHashCode();
}
If you are worried about performance of multiple calls to GetHashCode(), then first, don't, that is ridiculous micro-optimization, but if you insist, then store it.
GetHashCode() is fast, that is its entire purpose, as a fast way to compare values in a hash.
EDIT:
After doing some tests, the same hash code is returned between different runs using this method. I did not test after altering the classes, though, I am not aware of the exact method on how a Type is hashed.
Say we have a POD type:
private class Messages {
public byte[] last;
public byte[] next;
}
and its instance messages.
When a user (caller) requests instance we want to give him deep copy of messages object (that can be not the latest). When a user sets his own version we want to provide it to others as sun as possible yet not interrupting read requests (older version should be removed, not interrupting reads as soon as possible).
How to do such object versioning using System.Collections.Concurrent?
What have I tried:
internal class CuncurrentMessagesHelper {
private readonly ConcurrentStack<Messages> _stack = new ConcurrentStack<Messages>();
public CuncurrentMessagesHelper() {
}
public void SetLatest(Messages m) {
var length = _stack.Count;
_stack.Push(m);
var range = new Messages[length];
_stack.TryPopRange(range, 0, length);
}
public bool ReadLatest(out Messages result) {
return _stack.TryPeek(out result);
}
}
Yet such helper approach seems as ugly hack.
because even we know result is garanteed we use try and return bool instead of object;
it TryPopRange makes us create addintional array with size of all previous versions.
This isn't POD. It's a POCO. I suggest you read up on the difference between .NET's value types and reference types, since their semantics are crucial while writing safe concurrent code.
Since C# references are guaranteed to be atomic, the solution is simple (and doesn't require any special concurrent containers).
Assuming your Messages object is immutable once passed in:
internal class ConcurrentMessagesHelper {
private volatile Messages _current;
public void SetLatest(Messages m) {
_current = m;
}
public Messages ReadLatest() {
return _current;
}
}
Note that it's the reference to the object that's being copied here (atomically), and not the object's byte[] fields. volatile is required since the reference is accessed by multiple threads (it ensures correct behaviour, in particular with regards to memory ordering and limiting the optimizations the JIT can perform to only thread-safe ones).
If the Messages object passed to SetLatest can change while it's the latest, then all you have to do is make a copy first. SetLatest becomes:
public void SetLatest(Messages m) {
_current = DeepClone(m);
}
If readers are allowed to change the Messages object returned, then you have to copy it before letting them have it too. ReadLatest becomes:
public Messages ReadLatest() {
return DeepClone(_current);
}
Note that if the values contained in the byte[] fields of Messages are immutable during each message's lifetime, all you need is a shallow copy, not a deep one.
You can make the interface even nicer by wrapping it in a simple property:
internal class ConcurrentMessagesHelper {
private volatile Messages _current;
public Messages Current {
get { return DeepClone(_current); }
set { _current = DeepClone(value); }
}
private static Messages DeepClone(Messages m)
{
if (m == null)
return null;
return new Messages {
last = m.last == null ? null : (byte[])m.last.Clone(),
next = m.next == null ? null : (byte[])m.next.Clone()
};
}
}
If you actually did have a POD type (e.g. struct Messages), then I suggest the simplest solution would be to wrap it in a class so you can have an atomic reference to a copy of it, which would allow you to use the solution above. StrongBox<T> comes to mind.
The code in such a case becomes even simpler, because no explicit copying is required:
private struct Messages {
public byte[] last;
public byte[] next;
}
internal class ConcurrentMessagesHelper {
private volatile StrongBox<Messages> _current;
public Messages Current {
get { return _current.Value; }
set { _current = new StrongBox<Messages>(value); }
}
}
If the byte arrays in Messages can change during the object's lifetime, then we still need the deep cloning, though:
internal class ConcurrentMessagesHelper {
private volatile StrongBox<Messages> _current;
public Messages Current {
get { return DeepClone(_current.Value); }
set { _current = new StrongBox<Messages>(DeepClone(value)); }
}
private static Messages DeepClone(Messages m)
{
return new Messages {
last = m.last == null ? null : (byte[])m.last.Clone(),
next = m.next == null ? null : (byte[])m.next.Clone()
};
}
}
Is there any known issue or common mistake which would result in a list being deserialized with the correct number of items but for all of those items to be uninitialized? Note: The item type is marked with ProtoContract and has several ProtoMember(s).
My types....
[ProtoContract(SkipConstructor = true)]
[ProtoInclude(2, typeof(LoadBoardAction))]
public abstract class GameBoardAction : IGameBoardAction
{
}
[ProtoContract(SkipConstructor = true)]
public class LoadBoardAction : GameBoardAction
{
[ProtoMember(10)]
private List<GameTileState> _tiles;
}
[ProtoContract]
public struct GameTileState
{
[ProtoMember(1)]
public Point Coordinate;
[ProtoMember(2)]
public TileType Type;
}
Serialization...
// Note: action is a LoadBoardAction with 15 elements in _tiles
var stream = new MemoryStream();
Serializer.Serialize(stream, action);
var buffer = stream.ToArray();
stream.Close();
// buffer gets sent across network here...
Deserialization...
// stream is a MemoryStream initialized to the byte[] which has
// be read from a network packet.
var action = Serializer.Deserialize<GameBoardAction>(stream);
Action will have 15 elements in _tiles but GameTileState.Coordinate is always (0,0) and GameTypeState.Type is always TileType.Null.
Edit:
The enum TileType is actually serialized/deserialized correctly, the issue is Point, which is an XNA struct type and hence its public fields X/Y are not marked for serialization in any way. This is definitely the issue. I just assumed that it knew to read/write the public fields of a struct when that struct was marked as a ProtoMember of a known type.
While I wish this worked automatically it looks like the work around is to add something like this (for all XNA math types)...
RuntimeTypeModel.Default.Add(typeof(Point), false).Add("X", "Y")
Edit:
Verified that adding the above mentioned line fixes the issue.
I'm implementing a system to send Messages between different parts of a program I'm writing. There are some generic message types as well as some specific to each part of the program. I would like to avoid the hierarchy rot inherent in deriving from a base message class for each type, So i'm encapsulating this type in an int or ushort. Then, I centralize the different types with a "Messages" namespace, and a static class with a bunch of constants. However, I ran into the issue of having to maintain a list of unique numbers for each different section:
namespace Messages
{
public static class Generic
{
public const Int32 Unknown = 0;
public const Int32 Initialize = 1;
...
public const Int32 Destroy = 10;
}
}
Then elsewhere
namespace Messages
{
public static class Graphics
{
public const Int32 Unknown = 0;
public const Int32 AddGraphic = 11; // <-- ?
}
}
Having that arbitrary 11 seems difficult, especially if I have several of these, maintaining and updating to make sure there are no collisions seems to be a pain. Is there an easy solution in order to make sure each reference to this is unique? I tried using static readonly, initializing them off of a Unique.ID() function in a static constructor, but if I do that I am unable to switch() over the passed Message type, as it says "A constant type is expected" for each case.
Is there some reason you aren't using enums?
public enum MessageTypes
{
Unknown,
Initialize,
...
}
-- Edit:
Elaborating on my comment, Consider
enum MessageType
{
Update,
Delete,
Destroy
}
MessageType t = ...;
switch(t){
case MessageType.Update:
DoUpdate();
}
}
Versus:
interface IActionable
{
void Do ();
}
public abstract class ActionableBase : IActionable
{
// some other things
public abstract void Do ();
}
public class UpdateAction : ActionableBase
{
public override void Do ()
{
// Update Code
}
}
...
IActionable a = ...;
a.Do();
You can use a number range for each class. Define a base number for the class and add 0, 1, 2, etc to that base number.
If you want to keep them numeric one way is to divide them into different magnitudes:
namespace Messages
{
public static class Generic
{
// these messages are 3-figure numbers
public const Int32 Unknown = 0;
public const Int32 Initialize = 101;
...
public const Int32 Destroy = 110;
}
public static class Graphics
{
// these messages are 4-figure numbers
public const Int32 Unknown = 0;
public const Int32 AddGraphic = 1001; // <-- ?
// and so on...
}
}
Then you just need to make sure that you keep within the boundaries for each type of message.
This isn't automatic, but it may be a bit easier to maintain then copying the values everywhere:
public enum Generic
{
Unknown = 0,
Initialize = 1,
Destroy = 10
}
public enum Graphics
{
AddGraphic = Generic.Destroy + 1
}
So, you can have all of your specific enums start with the value from a previous enum set and build them up like that.
In your actual objects, you could store them as int's and just convert whatever enum value to the appropriate int.
Although, it seems that inheritance may be inevitable in this case since there is a natural hierarchy in your data model.
I suggest you lookup the difference between 'command' & 'message', this might help you come to the conclusion that the use of magic numbers\enums inside messages is a bad idea.
Ideally you want to create 'commands' that are observed and actioned by listeners...
HTH
Ollie
If you really, really want to do this, you can create one generic private Enum that holds all of the possible values.
You can then expose those values through your separate classes as read-only properties that expose the enums as Int32s - e.g.
namespace Messages
{
private enum AllMessageTypes
{
Update,
Delete,
Destroy,
AddGraphic
}
public static class Generic
{
public Int32 Update
{
get { return (Int32)AllMessageTypes.Update; }
}
...
}
public static class Graphics
{
public Int32 AddGraphic
{
get { return (Int32)AllMessageTypes.AddGraphic ; }
}
}
}
However - I'd recommend you redesign your solution. This seems to be asking for trouble (as I'm sure people will comment on)
I have a class library containing several structures each consisting of several value and reference types. Most of the value types are mandatory, a few value types and all reference types are optional. All structures are XmlSerializable (which is mandatory).
As far as the class library is targeted to mobile devices I want to reduce the memory footprint. My first idea was to use Nullable<T> for the value types, but this increases the memory size by 4 bytes per Nullable<T>. My second idea is to pack all optional value types into a separate structure that is only instantiated when any of its members is needed. But this would force me to implement IXmlSerializable on the "main" structure.
Are there any other approaches to "shrink" the structures?
[EDIT]
Beg your pardon for this bad question. I think I have to clarify some things and get more specific:
The class library is designed to serialize data info GPX (GPS Exchange Format). The structures are e.g. Waypoint or Track. They have mandatory fields as latitude, longitude etc. Optional fields are Vertical/Horizontal/Position Dilution of Precision, a description, a link.
The library is mainly targeted to Mobile devices such as PDAs. RAM is short, but plenty of non-volatile Memory is available.
Code examples cannot be shown as far as there are none. I want to think about several pitfalls before starting the implementation.
Here is a technique to aggressively reduce in memory overhead whilst allowing Xml Serialization.
Update: the orignal inline linked list idea is more efficient for 1 and 2 entries than a standard list with count construct but the use of fixed size optionals for zero, one and two cases is even more efficient.
Proviso:
This is predicated on you knowing that you really do need to shave the memory, as such
(since you haven't done any coding yet) this may well be a massively premature
optimization.
Also this design is predicated on the optional fields being very rare.
I use double as a 'placeholder' whatever format best allows you to represent the precision/units involved should be used.
public class WayPoint
{
// consumes IntPtr.Size fixed cost
private IOptional optional = OptionalNone.Default;
public double Latitude { get; set; }
public double Longitude { get; set; }
public double Vertical
{
get { return optional.Get<double>("Vertical") ?? 0.0; }
set { optional = optional.Set<double>("Vertical", value); }
}
[XmlIgnore] // need this pair for every value type
public bool VerticalSpecified
{
get { return optional.Get<double>("Vertical").HasValue; }
}
public void ClearVertical()
{
optional = optional.Clear<double>("Vertical");
}
public string Description // setting to null clears it
{
get { return optional.GetRef<string>("Description"); }
set { optional = optional.SetRef<string>("Description", value); }
}
// Horizontal, Position, DilutionOfPrecision etc.
}
The real heavy lifting is done here:
internal interface IOptional
{
T? Get<T>(string id) where T : struct;
T GetRef<T>(string id) where T : class;
IOptional Set<T>(string id, T value);
IOptional Clear(string id);
}
internal sealed class OptionalNone : IOptional
{
public static readonly OptionalNone Default = new OptionalNone();
public T? Get<T>(string id) where T : struct
{
return null;
}
public T GetRef<T>(string id) where T : class
{
return null;
}
public IOptional Set<T>(string id, T value)
{
if (value == null)
return Clear(id);
return new OptionalWithOne<T>(id, value);
}
public IOptional Clear(string id)
{
return this; // no effect
}
}
The fixed size ones become more interesting to write, there is no point writing these as structs as they would be boxed to be placed within the IOptional field within the WayPoint class.
internal sealed class OptionalWithOne<X> : IOptional
{
private string id1;
private X value1;
public OptionalWithOne(string id, X value)
{
this.id1 = id;
this.value1 = value;
}
public T? Get<T>(string id) where T : struct
{
if (string.Equals(id, this.id1))
return (T)(object)this.value1;
return null;
}
public T GetRef<T>(string id) where T : class
{
if (string.Equals(id, this.id1))
return (T)(object)this.value1;
return null;
}
public IOptional Set<T>(string id, T value)
{
if (string.Equals(id, this.id1))
{
if (value == null)
return OptionalNone.Default;
this.value1 = (X)(object)value;
return this;
}
else
{
if (value == null)
return this;
return new OptionalWithTwo<X,T>(this.id1, this.value1, id, value);
}
}
public IOptional Clear(string id)
{
if (string.Equals(id, this.id1))
return OptionalNone.Default;
return this; // no effect
}
}
Then for two (you can extend this idea as far as you want but as you can see the code gets unpleasant quickly.
internal sealed class OptionalWithTwo<X,Y> : IOptional
{
private string id1;
private X value1;
private string id2;
private Y value2;
public OptionalWithTwo(
string id1, X value1,
string id2, Y value2)
{
this.id1 = id1;
this.value1 = value1;
this.id2 = id2;
this.value2 = value2;
}
public T? Get<T>(string id) where T : struct
{
if (string.Equals(id, this.id1))
return (T)(object)this.value1;
if (string.Equals(id, this.id2))
return (T)(object)this.value2;
return null;
}
public T GetRef<T>(string id) where T : class
{
if (string.Equals(id, this.id1))
return (T)(object)this.value1;
if (string.Equals(id, this.id2))
return (T)(object)this.value2;
return null;
}
public IOptional Set<T>(string id, T value)
{
if (string.Equals(id, this.id1))
{
if (value == null)
return Clear(id);
this.value1 = (X)(object)value;
return this;
}
else if (string.Equals(id, this.id2))
{
if (value == null)
return Clear(id);
this.value2 = (Y)(object)value;
return this;
}
else
{
if (value == null)
return this;
return new OptionalWithMany(
this.id1, this.value1,
this.id2, this.value2,
id, value);
}
}
public IOptional Clear(string id)
{
if (string.Equals(id, this.id1))
return new OptionalWithOne<Y>(this.id2, this.value2);
if (string.Equals(id, this.id2))
return new OptionalWithOne<X>(this.id1, this.value1);
return this; // no effect
}
}
Before finally ending with the relatively inefficient
internal sealed class OptionalWithMany : IOptional
{
private List<string> ids = new List<string>();
// this boxes, if you had a restricted set of data types
// you could do a per type list and map between them
// it is assumed that this is sufficiently uncommon that you don't care
private List<object> values = new List<object>();
public OptionalWithMany(
string id1, object value1,
string id2, object value2,
string id3, object value3)
{
this.ids.Add(id1);
this.values.Add(value1);
this.ids.Add(id2);
this.values.Add(value2);
this.ids.Add(id3);
this.values.Add(value3);
}
public T? Get<T>(string id) where T : struct
{
for (int i= 0; i < this.values.Count;i++)
{
if (string.Equals(id, this.ids[i]))
return (T)this.values[i];
}
return null;
}
public T GetRef<T>(string id) where T : class
{
for (int i= 0; i < this.values.Count;i++)
{
if (string.Equals(id, this.ids[i]))
return (T)this.values[i];
}
return null;
}
public IOptional Set<T>(string id, T value)
{
for (int i= 0; i < this.values.Count;i++)
{
if (string.Equals(id, this.ids[i]))
{
if (value == null)
return Clear(id);
this.values[i] = value;
return this;
}
}
if (value != null)
{
this.ids.Add(id);
this.values.Add(value);
}
return this;
}
public IOptional Clear(string id)
{
for (int i= 0; i < this.values.Count;i++)
{
if (string.Equals(id, this.ids[i]))
{
this.ids.RemoveAt(i);
this.values.RemoveAt(i);
return ShrinkIfNeeded();
}
}
return this; // no effect
}
private IOptional ShrinkIfNeeded()
{
if (this.ids.Count == 2)
{
//return new OptionalWithTwo<X,Y>(
// this.ids[0], this.values[0],
// this.ids[1], this.values[1]);
return (IOptional)
typeof(OptionalWithTwo<,>).MakeGenericType(
// this is a bit risky.
// your value types may not use inhertence
this.values[0].GetType(),
this.values[1].GetType())
.GetConstructors().First().Invoke(
new object[]
{
this.ids[0], this.values[0],
this.ids[1], this.values[1]
});
}
return this;
}
}
OptionalWithMany could be written rather better than this but it gives you the idea.
With restricted type support you could do a global Key -> value map per type 'heap' like so:
internal struct Key
{
public readonly OptionalWithMany;
public readonly string Id;
// define equality and hashcode as per usual
}
Then simply store the list of Id's currently in use within OptionalToMany. Shrinking would be slightly more complex (but better from a type point of view since you would scan each global 'heap' till you found the matching entry and use the type of the heap to construct the OptionalWithTwo. This would allow polymorphism in the property values.
Regardless of the internals the primary benefit of this is that the public surface of the WayPoint class hides all this entirely.
You can then set up the class however you want for serialization though attributes, IXmlSerializable (which would remove the need for the annoying xxxSpecified properties).
I used strings as Id's for simplicity in my example.
If you really care about size and speed you should change the Id's to be enumerations. Given packing behaviour this won't save you much even if you can fit all needed values into a byte but it would give you compile time sanity checking. The strings are all compile time constants so occupy next to no space (but are slower for checking equality).
I urge you to only do something like this after you check that it is needed. The plus side is that this does not limit your xml serialization so you can mould it to whatever format you desire. Also the public face of the 'data packet' can be kept clean (except for the xxxSpecified junk).
If you want to avoid the xxxSpecified hassles and you know you have some 'out of band' values you can use the following trick:
[DefaultValue(double.MaxValue)]
public double Vertical
{
get { return optional.Get<double>("Vertical") ?? double.MaxValue; }
set { optional = optional.Set<double>("Vertical", value); }
}
public void ClearVertical()
{
optional = optional.ClearValue<double>("Vertical");
}
However the rest of you API must be capable of detecting these special values. In general I would say that the specified route is better.
If a particular set of properties become 'always available' on certain devices, or in certain modes you should switch to alternate classes where the properties for those are simple ones. Since the xml form will be identical this means they can interoperate simply and easily but memory usage in those cases will be much less.
If the number of these groups becomes large you may even consider a code-gen scenario (at runtime even, though this increases your support burden considerably)
For some serious fun:
apply Flyweight and store all instances in a bitmap? With a small memory device you don't need 4 byte pointers.
[Edit] With Flyweight, you can have a separate storage strategy for each field. I do not suggest to directly store the string value in the bitmap, but you could store an index.
The type is not stored in the bitmap, but in the unique object factory.
It is probably good to know that the XmlSerializer doesn't care about your internal object layout, it only cares about your public fields and properties. You can hide the internal memory optimizations behind your property accessors, and the XmlSerializer wouldn't even know.
For instance, if you know that you usually have only 2 references set, but on occasion more, you can store the two frequent ones as part of your main object, and hide the infrequent ones inside an object[] or ListDictionary or a specialized private class of your own making. However, be careful that each indirect container object also contains overhead, as it needs to be a reference type. Or when you have 8 nullable integers as part of your public contract, internally you could use 8 regular integers and a single byte containing the is-this-int-null status as its bits.
If you want to specialize even further, perhaps create specialized subclasses depending on the available data, you would have to go the route of IXmlSerializable, but usually that's not really needed.
You can do a couple of things:
Make sure to use the smallest type possible for a particular value. For example, if you look at the schema, dgpsStationType has a min value of 0, and a max of 1023. This can be stored as a ushort. Reduce the size of these items when possible.
Make sure that your fields are 4-byte aligned. The end resulting size of your structure will be some multiple of 4-bytes in size (assuming 32-bit). The default layout for a class has the items stored sequentially. If the fields are not packed correctly, the compiler will have wasted space making sure that your fields are 4-byte aligned. You can specify the layout explicitly using StructLayoutAttribute.
Bad Example: these fields in a class take up 12 bytes. The int must take up 4 contiguous bytes, and the other members must be 4-byte aligned.
public class Bad {
byte a;
byte b;
int c;
ushort u;
}
Better Example: these fields in a class take up 8 bytes. These fields are packed efficiently.
public class Better {
byte a;
byte b;
ushort u;
int c;
}
Reduce the size of your object graph. Each reference type takes up 8 bytes of overhead. If you've got a deep graph, that's a lot of overhead. Pull everything you can into functions that operate on data in you main class. Think more 'C' like, and less OOD.
Its still a good idea to lazy-load some optional parameters, but you should draw your line clearly. Create 1 or maybe 2 sets of 'optional' values that can be loaded or null. Each set will mandate a reference type, and its overhead.
Use structs where you can. Be careful of value-type semantics though, they can be tricky.
Consider not implementing ISerializable. Interface methods are by definition virtual. Any class with virtual methods contains a reference to a vtable (another 4 bytes). Instead implement xml serialization manually in an external class.
Build your own serializing in order to minimize your structure. And serialize to binary and not xml.
Something along the lines of:
internal void Save(BinaryWriter w)
{
w.Write(this.id);
w.Write(this.name);
byte[] bytes = Encoding.UTF8.GetBytes(this.MyString);
w.Write(bytes.Length);
w.Write(bytes);
w.Write(this.tags.Count); // nested struct/class
foreach (Tag tag in this.tags)
{
tag.Save(w);
}
}
and have a constructor which builds it back up
public MyClass(BinaryReader reader)
{
this.id = reader.ReadUInt32();
etc.
}
Some sort of binary serialization will often do much better than XML serialization. You'll have to try it out for your specific data structures to see if you gain much.
Check out MSDN an example using BinaryFormatter.