We Keep Coding

IEnumerable<T> vs. Array

I'm trying to get the idea, what would be the best way to publish a Readonly List of objects as a public method?
From Eric Lippert's Blog, Arrays are kinda bad, because someone could easily add a new Entry. So one would have to pass a new Array every time the method is called.
He suggests, to pass IEnumerable<T>, since this is per definition read only (no add, remove methods), which I practiced for quite sometime.
But in our new project, people even started to create Arrays of these IEnumerables, because they don't know the DataSource behind, so they get a : Handling warning for possible multiple enumeration of IEnumerable
I'm interested in a technical approach, how one would solve this puzzle. The only solution I came up so far would be to use a IReadOnlyCollection, but this would be way more explicit than an IEnumerable.
What is best practice to publish such lists, which shouldn't be changed, but should be declared as In-Memory Lists?

Usually - and since a while - this solved using immutable collections.
Your public properties should be, for example, of type IImmutableList<T>, IImmutableHashSet<T> and so on.
Any IEnumerable<T> can be converted to an immutable collection:
someEnumerable.ToImmutableList();
someEnumerable.ToImmutableHashSet();
... and so on.
This way you can work with private properties using mutable collections and provide a public surface of immutable collections only.
For example:
public class A
{
private List<string> StringListInternal { get; set; } = new List<string>();
public IImmutableList<string> StringList => StringListInternal.ToImmutableList();
}
There's also an alternate approach using interfaces:
public interface IReadOnlyA
{
IImmutableList<string> StringList { get; }
}
public class A : IReadOnlyA
{
public List<string> StringList { get; set; } = new List<string>();
IImmutableList<string> IReadOnlyA.StringList => StringList.ToImmutableList();
}
Check that IReadOnlyA has been explicitly-implemented, thus both mutable and immutable StringList properties can co-exist as part of the same class.
When you want to expose an immutable A, then you return your A objects upcasted to IReadOnlyA and upper layers won't be able to mutate the whole StringList in the sample above:
public IReadOnlyA DoStuff()
{
return new A();
}
IReadOnlyA a = DoStuff();
// OK! IReadOnly.StringList is IImmutableList<string>
IImmutableList<string> stringList = a.StringList;
Avoiding converting the mutable list to immutable list every time
It should be a possible solution to avoid converting the source list into immutable list each time immutable one is accessed.
Equatable members
If type of items overrides Object.Equals and GetHashCode, and optionally implements IEquatable<T>, then both public immutable list property access may look as follows:
public class A : IReadOnlyA
{
private IImmutableList<string> _immutableStringList;
public List<string> StringList { get; set; } = new List<string>();
IImmutableList<string> IReadOnlyA.StringList
{
get
{
// An intersection will verify that the entire immutable list
// contains the exact same elements and count of mutable list
if(_immutableStringList.Intersect(StringList).Count == StringList.Count)
return _immutableStringList;
else
{
// the intersection demonstrated that mutable and
// immutable list have different counts, thus, a new
// immutable list must be created again
_immutableStringList = StringList.ToImmutableList();
return _immutableStringList;
}
}
}
}

I do not think immutable is the way to go
int[] source = new int[10000000];//uses 40MB of memory
var imm1 = source.ToImmutableArray();//uses another 40MB
var imm2 = source.ToImmutableArray();//uses another 40MB
List behaves the same way. If I want to make full copy every time, I do not have to care about what user does with that array. Making it immutable does not protect content of objects in the collection either, they can be changed freely. #HansPassant suggestion seems to be best
public class A
{
protected List<int> list = new List<int>(Enumerable.Range(1, 10000000));
public IReadOnlyList<int> GetList
{
get { return list; }
}
}

For a collection that you don't intend to modify, IEnumerable<T> is still probably the safest option, plus it allows any collection type to be pased in, not just arrays. The reason for that warning is because of the possibility that the IEnumerable represents a query that uses deferred execution, meaning that a potentially expensive operation could be executed multiple times.
Note that there's not an interface that distinguish in-memory collections versus potentially deferred-execution wrappers. That question has been asked before.
The fix for that is do not enumerate the source multiple times. If the code needs perform multiple iteartions (which may be legitimate) then hydrate the collection to a List<T> before iterating.

IEnumerable is a read-only interface which hides implementation from user. Some can argue, that user may cast IEnumerable to list and add new items, but that means two things:
User violates provided API
You can't stop user from reflection usage
IEnumerable describes behavior, while List is an implementation of that behavior. When you use IEnumerable, you give the compiler a chance to defer work until later, possibly optimizing along the way. If you use ToList() you force the compiler to prepare the results right away.
I use IEnumerable Whenever working with LINQ expressions, because by only specifying the behavior, I give LINQ a chance to defer evaluation and possibly optimize the program.
To prevent any modifications to the List<T> object, expose it only through the ReadOnlyCollection<T> wrapper which does not expose methods that modify the collection. However, if changes are made to the underlying List<T> object, the read-only collection reflects those changes as described in MSDN.
Take a look at The New Read-Only Collections in .NET 4.5 please.

I've been programming for longer than I care to remember, and never had such a requirement to protect a list from getting modified. I'm not saying it's not a possible requirement, I'm just saying it is very rare to need such a requirement. If you have a list circulating around in your app, then most likely you have to fix your design. If you need help with that, let us how you're using the list.
The examples you're giving in your question and comments are not good examples for when to require an immutable or read-only list. Let's discuss them one by one.
You mentioned publishing it as an API. By definition, anything you return from an API is no yours anymore and you shouldn't care how it is used. In fact, once it leaves your API, it is now in the API client's domain, and they can do whatever they want with it. Aside from the fact that you cannot protect it once it is in their domain, you should not dictate how they will use it. More importantly, you should never accept anything as input in your API, even if it is the same list that you returned earlier and you think that you protected. All input MUST be validated appropriately.
Perhaps, you did not really mean API, but more like a DLL library that you share in your projects. Whether it is a DLL or just a class in your project, the same principle applies. When you return something, it is up to the user how to use it. And you should never accept the same thing (list or whatever) back without validation. Similarly, you should never expose an internal member of your class, whether it's a list or a single value. After all, that's the whole idea behind using properties instead of marking the members as public. When you create a property for a single-value member, a copy of the value is returned. Similarly, you should create a property for your list and return a copy, not the list itself.
If you really need a list that is globally accessible, and you want to load it only once, expose it to other classes, protect it against modification, and it is too big to make a copy of it. You can look into some designs like wrapping it in a Singleton and/or make it read-only as per Antonín Lejsek's answer. In fact, his answer can be easily converted to a Singleton by marking the constructor as private, thanks to the simplified Singleton implementation in C#.

Efficient usage of TextWriter

Is there any alternative way to perform the operation:
textWriter.Write(myBigObject.ToString())
such that:
myBigObject is 'streamed' into the text representation without creating the whole string object in memory
there are no additional classes or objects used, beside myBigObject and textWriter
Example: Imagine that myBigObject has 50 string fields. There is no point in joining all these fields in a big string and then writing the object to a file, if it is somehow possible to write the strings one by one to the file.

If you have access to the code, you can add a method to MyBigObject that takes a TextWriter and writes out each property. For example:
public class MyBigObject
{
public void Write(TextWriter writer)
{
writer.Write(bigStringField1);
writer.Write(bigStringField2);
// etc.
}
}
If sub-classes of MyBigObject need to write their own representation, then make the method virtual, and the sub-classes call the implementation in the base class.
If you don't own the code, and the fields are exposed through properties, you could build an adapter class that takes a MyBigObject and writes out each property. You could also build some extension methods that do the same thing.
If you cannot access the source code, you could use reflection to do examine the fields on the object, grab the value of each field, and Write() out each value's ToString() representation. However, reflection is slower than direct field access, and it involves a lot more intermediate objects. I don't know if using reflection would be worth it in your case.

Given the limitations you have outlined this isn't possible. You would have to come up with a way to read the data from your object and write it out on char/byte/line at a time.
If you want to be able to loop over your properties and write them out one at a time then this would be possible using reflection. However I suspect going this route would result in using more memory than your original solution as well as being much more complicated than a simple call to .ToString().

Deserialization of changed class

I am working on a program, where I save it's project files by serializing Project class.
Because I am still working on it, some classes, that are part of Project class, do change from time to time (e.g. class got new property). It makes "simple" deserialization impossible.
Is there any way to solve it ? I mean, without writng custom serializer ? (which probably is something high above my level for now)
Just in case, I am using BinaryFormatter.

I hope I understood your problem correctly. You have a class serialized to a file which you have since changed in the program (e.g you have added another property). Now you want to deserialize this class from the file. This is not a problem as long as you have only added new properties. They will be ignored by the deserializer. It creates a new instance of your class (that is the reason why serializable classes have to have a default constructor) and tries to fill the properties it finds in the stream to derserialize. If you change a property's type or remove a property, you won't be able to deserialize the original file.
One workaround for removing properties is to keep them in the class, but just stop using them in the rest of the program. A workaround for properties that have been changed to a different type could look something like this:
[Serializable]
public class MyClass
{
int? newProperty;
[XmlElement("Property")]
public string OldProperty
{
get { return string.Empty; }
set
{
if (!newProperty.HasValue)
{
int temp;
if (int.TryParse(value, out temp))
{
newProperty.Value = temp;
}
}
}
}
public int NewProperty
{
get { return newPropery.HasValue ? newProperty.Value : 0; }
set { newProperty.Value = value; }
}
}

From my experience, I've found using BinaryFormatter for serialization/de-serialization of data types that are going to change a really bad idea. If something changes in your data type, from what I know the BinaryFormatter will fail in the process.
To overcome this issue in the data types I was using, I had to write my own serializer, which wasn't actually that much of a major task. You can use the BinaryReader and BinaryWriter classes to read and write the data in and out of your type. That way you can control the data you are expecting and handle any missing data either by adding default values, skipping the property altogether, or throwing some form of Exception to signify corrupt data. Refer to the MSDN article links above for more information.

With help from Merlyn Morgan-Graham's comments I've found solution, that will work for me.
Versioning described in Version Tolerant Serialization is really good idea, but when I use only [Serializable] attribute.
I forgot to write (my mistake), that I am using ISerializable interface.
I've found, that in deserialization constructor SerializationInfo object has MemberCount property, which solves my problem if I only add new properties/members from time to time. With this information, new members/properties, that can't be deserialized from older file, can be set to default or maybe I can use some prompt form.
Other way here would be using something like assembly version in deserialization, as a first deserialized member. This can solve deserialization problems with more complex class changes.
Either way, I agree with Merylin - "if you can't script something, you shouldn't be building it". ;)

How to use XmlSerializer to deserialize into an existing instance?

Is it somehow possible to use the XmlSerializer to deserialize its data into an existing instance of a class rather than into a new one?
This would be helpful in two cases:
Easily merge two XML files into one object instance.
Let object constructer itself be the one who is loading its data from the XML file.
If the is not possible by default it should work by using reflection (copying each property after the deserialisation) but this would be an ugly solution.

Basically, you can't. XmlSerializer is strictly constructive. The only interesting thing you can do to customize XmlSerializer is to implement IXmlSerializable and do everything yourself - not an attractive option (and it will still create new instances with the default constructor, etc).
Is xml a strict requirement? If you can use a different format, protobuf-net supports merging fragments into existing instances, as simply as:
Serializer.Merge(source, obj);

I think you're on the right track with the Reflection idea.
Since you probably have a wrapper around the XML operations anyway, you could take in the destination object, do the deserialization normally into a new object, then do something similar to cloning by copying over one by one only the properties holding non-default values.
It shouldn't be that complex to implement this, and it would look to consumers from the rest of your application just like in-place deserialization.

I hit the same problem a few weeks ago.
I put a method Deserialize(string serialized form) in the ISelfSerializable interface that an entity class of mine implemented. I also made sure the interface forced the class to have a default constructor.
In my factory I created an object of that type and then deserialized the string into it.

This is not thread safe thing to do... But you can do:
[Serializable]
public class c_Settings
{
static c_Settings Default;
public static SetExistingObject(c_Settings def)
{
Default = def;
}
public string Prop1;
public bool Prop2;
public c_Settings()
{
if (Default == null)
return;
MemberInfo[] members = FormatterServices.GetSerializableMembers(typeof(c_Settings));
FormatterServices.PopulateObjectMembers(this, members, FormatterServices.GetObjectData(Default, members));
}
}
This way you feed your object to deserialiser and deserialiser only overwrites whatever is written in .xml.

Design time serialization in C#

I have created a non-visual component in C# which is designed as a placeholder for meta-data on a form.
The component has a property which is a collection of custom objects, this object is marked as Serializable and implements the GetObjectData for serilizing and public constuctor for deserilizing.
In the resx file for the form it will generate binary data for storing the collection, however any time I make a change to the serialized class I get designer errors and need to delete the data manually out of the resx file and then recreate this data.
I have tried changing the constuctor to have a try / catch block around each property in the class
try
{
_Name = info.GetString("Name");
}
catch (SerializationException)
{
this._Name = string.Empty;
}
but it still crashes. The last error I got was that I had to implement IConvertible.
I would prefer to use xml serialization because I can at least see it, is this possible
for use by the designer?
Is there a way to make the serialization more stable and less resistant to changes?
Edit:
More information...better description maybe
I have a class which inherits from Component, it has one property which is a collection of Rules. The RulesCollection seems to have to be marked as Serializable, otherwise it does not retain its members.
The Rules class is also a Component with the attribute DesignTimeVisible(false) to stop it showing in the component tray, this clas is not marked Serializable.
Having the collection marked as Serializable generates binary data in the resx file (not ideal) and the IDE reports that the Rules class is not Serializable.
I think this issue is getting beyond a simple question. So I will probably close it shortly.
If anyone has any links to something similar that would help a lot.

You might want to try the alternate approach of getting everything to serialize as generated code. To do that is very easy. Just implement your non-visual class from Component. Then expose your collection as you already are but ensure each object placed into the collection is itself derived from Component. By doing that everything is code generated.

I have since discovered where I was going wrong.
The component I was implementing a custom collection (inherited from CollectionBase), I changed this to a List and added the DesignerSerializationVisibility(DesignerSerializationVisibility.Content) attribute to the List property, this list is also read-only. This would then produce code to generate all the components properties and all the entries in the List.
The class stored in the list did not need any particuar attributes or need to be serializble.
private List<Rule> _Rules;
[DesignerSerializationVisibility(DesignerSerializationVisibility.Content)]
public List<Rule> Rules
{
get { return _Rules; }
}

Could you put more code up of the class that is having the serialization issue, maybe the constructor and the property to give reference to the variables you're using.
Just a note:
I've had a lot of issues with the visual designer and code generation, if I've got a property on a control then generally I put
[DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)]
on the property and handle the initialization myself.