We Keep Coding

simulate generics with protocol buffer

I have a class that stores and manipulates some entities. Depending on the number of inputs, I may not be able to store entities in memory, so I'm trying to serialize my objects to be written on hard disk using protocol buffers. I'm using C# and protobuf-csharp-port. I'm aware of protobuf-net as an alternative port; so far I have been working with first option but I'm open for changes if it's required based on my needs.
The class to be serialized in it's simplified form is as follows:
class Entity<T> where T: IComparable<T>
{
int id;
T metaData;
}
So at compile time I have no clues about metaData. Googling I noticed that extensions are the right path to follow (as suggested on google's page and this question); hence I'm defining the Entity.proto file for class Entity as following:
message Entity
{
required int32 id = 1 [default = 0];
extensions 2 to max;
}
and I would like the user to provide his own .proto file for T without the need to access or re-compile Entity.proto. In this regard, my questions are:
Do I need to change Entity.proto ?
What should be the T.proto ?
How can I access T in my C# code ?

With that scheme, any extensions are going to be child values (not subclasses) of the non-generic Entity. That doesn't sound like generics, but ultimately storage (serialization) is often quite different to implementation (Entity<T> etc). If you can manuallyap between them: fine. But it isn't something the library will provide, AFAIK.
For completeness, in protobuf-net terms: it is perfectly fine with Entity<T> - it essentially considers each (Entity<Foo>, Entity<Bar>, etc) to be completely separate messages. Protobuf-net isn't hugely motivated by .proto schemas (although a code-gen tool is provided, for completeness) - it mainly uses runtime metadata.

Dynamic form with no real OOP or objects?

I am tacking a large refactor of a project, and I had asked this question to confirm/understand the direction I should go in and I think I got the answer that I wanted, which is not to throw away years worth of code. So, now begins the challenge of refactoring the code. I've been reading Martine Fowler and Martin Feathers' books, and they have a lot of insight, but I am looking for advice on the ultimate goal of where I want the application to be.
So to reiterate the application a little bit, its a dynamic forms system, with lots of validation logic and data logic between the fields. The main record that gets inserted is the set of form fields that is on the page. Another part of it is 'Actions' that you can do for a person. These 'Actions' can differ client by client, and there are hundreds of 'Actions'. There is also talk that we can somehow make an engine that can eventually take on other similar areas, where a 'person' can be something else (such as student, or employee). So I want to build something very de-coupled. We have one codebase, but different DBs for different clients. The set of form fields on the page are dynamic, but the DB is not - it is translated into the specific DB table via stored procs. So, the generic set of fields are sent to the stored proc and the stored proc then decides what to do with the fields (figure out which table it needs to go to). These tables in fact are pretty static, meaning that they are not really dynamic, and there is a certain structure to it.
What I'm struggling specifically is how to setup a good way to do the dynamic form control page. It seems majority of the logic will be in code on the UI/aspx.cs page, because its loading controls onto the webpage. Is there some way I can do this, so it is done in a streamlined fashion, so the aspx.cs page isn't 5000 lines long? I have a 'FORM' object, and one of the properties is its' 'FIELDS'. So this object is loaded up in the business layer and the Data layer, but now on the fron end, it has to loop through the FIELDS and output the controls onto the page. Also, someway to be able to control the placement would be useful, too - not sure how do get that into this model....
Also, from another point of view - how can I 'really' get this into an object-oriented-structure? Because technically, they can create forms of anything. And those form fields can represent any object. So, for example, today they can create a set of form fields, that represent a 'person' - tomorrow they can create a set of form fields that represent a 'furniture'. How can I possibly translate this to to a person or a furniture object (or should I even be trying to?). And I don't really have controls over the form fields, because they can create whatever....
Any thought process would be really helpful - thanks!

How can I possibly translate this to to a person or a furniture object
(or should I even be trying to?)
If I understand you correctly, you probably shouldn't try to convert these fields to specific objects since the nature of your application is so dynamic. If the stored procedures are capable of figuring out which combination of fields belongs to which tables, then great.
If you can change the DB schema, I would suggest coming up with something much more dynamic. Rather than have a single table for each type of dynamic object, I would create the following schema:
Object {
ID
Name
... (clientID, etc.) ...
}
Property {
ID
ObjectID
Name
DBType (int, string, object-id, etc.)
FormType ( textbox, checkbox, etc.)
[FormValidationRegex] <== optional, could be used by field controls
Value
}
If you can't change the database schema, you can still apply the following to the old system using the stored procedures and fixed tables:
Then when you read in a specific object from the database, you can loop through each of the properties and get the form type and simple add the appropriate generic form type to the page:
foreach(Property p in Object.Properties)
{
switch(p.FormType)
{
case FormType.CheckBox:
PageForm.AddField(new CheckboxFormField(p.Name, p.Value));
break;
case FormType.Email:
PageForm.AddField(new EmailFormField(p.Name, p.Value));
break;
case FormType.etc:
...
break;
}
}
Of course, I threw in a PageForm object, as well as CheckboxFormField and EmailFormField objects. The PageForm object could simply be a placeholder, and the CheckboxFormField and EmailFormField could be UserControls or ServerControls.
I would not recommend trying to control placement. Just list off each field one by one vertically. This is becoming more and more popular anyway, even with static forms who's layout can be controlled completely. Most signup forms, for example, follow this convention.
I hope that helps. If I understood your question wrong, or if you'd like further explanations, let me know.

Not sure I understand the question. But there's two toolboxes suitable for writing generic code. It's generics, and it's reflection - typically in combination.
I don't think I really understand what you're trying to do, but a method using relfection to identify all the properties of an object might look like this:
using System.Reflection;
(...)
public void VisitProperties(object subject)
{
Type subjectType = subject.GetType();
foreach (PropertyInfo info in subjectType.GetProperties()
{
object value = info.GetValue(subject, null);
Console.WriteLine("The name of the property is " + info.Name);
Console.WriteLine("The value is " + value.ToString());
}
}
You can also check out an entry on my blog where I discuss using attributes on objects in conjunction with reflection. It's actually discussing how this can be utilized to write generic UI. Not exactly what you want, but at least the same principles could be used.
http://codepatrol.wordpress.com/2011/08/19/129/
This means that you could create your own custom attributes, or use those that already exists within the .NET framework already, to describe your types. Attributes to specify rules for validation, field label, even field placement could be used.
public class Person
{
[FieldLabel("First name")]
[ValidationRules(Rules.NotEmpty | Rules.OnlyCharacters)]
[FormColumn(1)]
[FormRow(1)]
public string FirstName{get;set;}
[FieldLabel("Last name")]
[ValidationRules(Rules.NotEmpty | Rules.OnlyCharacters)]
[FormColumn(2)]
[FormRow(1)]
public string LastName{get;set;}
}
Then you'd use the method described in my blog to identify these attributes and take the apropriate action - e.g. placing them in the proper row, giving the correct label, and so forth. I won't propose how to solve these things, but at least reflection is a great and simple tool to get descriptive information about an unknown type.

I found xml invaluable for this same situation. You can build an object graph in your code to represent the form easily enough. This object graph can again be loaded/saved from a db easily.
You can turn your object graph into xml & use xslt to generate the html for display. You now also have the benefit of customising this transform for differnetn clients/versions/etc. I also store the xml in the database for performance & to give me a publish function.
You need some specific code to deal with the incoming data, as you're going to be accessing the raw request post. You need to validate the incoming data against what you think you was shown. That stops people spoofing/meddling with your forms.
I hope that all makes sense.

Building a string representation of an object using a "mask" or user supplied Format String

I'm not really sure what tags should be on this sort of question so feel free to give me some suggestions if you think some others are more suited.
I have a dynamic object with an unknown number or properties on it, it's from a sort of dynamic self describing data model that lets the user build the data model at runtime. However because all of the fields holding relevant information to the user are in dynamic properties, it's difficult to determine what should be the human readable identifier, so it's left up to the administrator. (Don't think it matters but this is an ASP.NET MVC3 Application). To help during debugging I had started decorating some classes with DebuggerDisplayAttribute to make it easier to debug. This allow me to do things like
[DebuggerDisplay(#"\{Description = {Description}}")]
public class Group
to get a better picture of what a specific instance of an object is. And this sort of setup would be perfect but I can't seem to find the implementation of this flexibility. This is especially useful on my dynamic objects because the string value of the DebuggerDisplayAttribute is resolved by the .NET framework and I have implementations of TryGetMember on my base object class to handle the dynamic aspect. But this only makes it easier for development. So I've added a field on what part of my object is still strongly typed and called it Title, and I'd like to let the administer set the implementation using their own format, so to speak. So for example they might build out a very simplistic rental tracking system to show rentals and they might specify a format string along the lines of
"{MovieTitle} (Due: {DueDate})"
I would like that when they save the record to add some logic to first update the Title property by resolving the format string to substitute each place holder with the value of the appropriate property on the dynamic object. So this might resolve to a title of
"Inception (Due: May 21, 2011)", or a more realistic scenario of a format string of
"{LastName}, {FirstName}"
I don't want the user to have to update the title of a record when they change the first name field or the last name field. I fully realize this will likely use reflection but I'm hoping some one out there can give me some pointers or even a working example to handle complex format strings that could be a mix if literal text and placeholders.
I've not had much luck looking for an implementation on the net that will do what I want since I'm not really sure what keywords would give me the most relevant search results?

You need two things:
1) A syntax for formatting strings
You have already described a syntax where variables are surrounded by bracers, and if you want to use that you need to build a parser that can parse that. Perhaps you also want to add ways to specify say a date or a number format.
2) Rules for resolving variables
If there is a single context object you can use reflection and match variable names to properties but if your object model is more complex you can add conventions for searching say a hierarchy of objects.
If you are planning to base your model objects on dynamic chances are that you will find the Clay library on CodePlex interesting.

C# Attributes and their uses

I really don't know much about attributes in general in C#, I've seen them in use in a lot of different ways/places but I don't think I see the importance of some of them. Some definitely have importance because they provide a noticeable function, such as [Serializable]. Yet, others don't seem so important, such as one my coworker uses to mark properties with [DataMember].
I suppose my question is, what are attributes and how are they useful? Is there a way to create my own attributes and how can I tell if fields/methods/classes/whatever have particular attributes or what values are set in those attributes?

what are attributes?
Attributes enable you to embed information about a type or method in the metadata which describes that type or method.
You typically want to use attributes to describe facts about the mechanism of the type or method rather than the meaning of the type or method. For example, suppose you have a type Employee. A fact about the meaning of Employee is that it is a kind of Person, that an Employee has a Manager, and so on. A fact about the mechanism of Employee is that it can be the target of data binding, or it can be serialized to disk, or whatever. An employee cannot be serialized to disk, but the class Employee can be. Attributes let you separate information about the technical details from the semantic model.
Is there a way to create my own attributes?
Yes. Create a class which extends Attribute. By convention you want to name it "FooAttribute". If you do so you can use either the [Foo] syntax or the [FooAttribute] syntax at your discretion.
How can I tell if fields/methods/classes/whatever have particular attributes or what values are set in those attributes?
Use the GetCustomAttributes method on the reflection objects.
Where should I read for more information?
Start with the attributes tutorial:
http://msdn.microsoft.com/en-us/library/aa288454(VS.71).aspx
And then read all of chapter 17 of the C# specification.

Attributes are a means by which you can associate metadata with types in .NET. This allows you to check for a type and get information about it that's separate from the "runtime" information of the type.
This can be very useful. You mentioned [Serializable], but other simple examples include many of the System.ComponentModel types, such as Description, which is used by the property grid to "describe" properties when you work with them in the designer. Since the "description" of a property isn't really related to the behavior of the type in a program (at runtime), it doesn't belong in the class. However, it's very handy when you go to edit a control in a visual designer, for example, to see a description (or category, etc) of a property. Attributes are the means by which this is handled.

I think the answer to the following question will provide you some insight to your questions.
How do attribute classes work?
Here is a repost of the answer I provided.
Attributes are essentially meta data that can be attached to various pieces of your code. This meta data can then be interogate and affect the behaviour of certain opperations.
Attributes can be applied to almost every aspect of your code. For example, attributes can be associated at the Assembly level, like the AssemblyVersion and AssemblyFileVersion attributes, which govern the version numbers associated with the assembly.
[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyFileVersion("1.0.0.0")]
Then the Serializable attribute for example can be applied to a type declaration to flag the type as supporting serialization. In fact this attribute has special meaning within the CLR and is actually stored as a special directive directly on the type in the IL, this is optimized to be stored as a bit flag which can be processed much more efficiently, there are a few attributes on this nature, which are known as pseudo custom attributes.
Still other attributes can be applied to methods, properties, fields, enums, return values etc. You can get an idea of the possible targets an attribute can be applied to by looking at this link
http://msdn.microsoft.com/en-us/library/system.attributetargets(VS.90).aspx
Further to this, you can define your own custom attributes which can then be applied to the applicable targets that your attributes are intended for. Then at runtime your code could reflect on the values contained in the custom attributes and take appropriate actions.
For a rather naive example, and this is just for the sake of example :)
You might want to write a persistence engine that will automatically map Classes to tables in your database and map the properties of the Class to table columns. You could start with defining two custom attributes
TableMappingAttribute
ColumnMappingAttribute
Which you can then apply to your classes, as an example we have a Person class
[TableMapping("People")]
public class Person
{
[ColumnMapping("fname")]
public string FirstName {get; set;}
[ColumnMapping("lname")]
public string LastName {get; set;}
}
When this compiles, other than the fact that the compiler emits the additional meta data defined by the custom attributes, little else is impacted. However you can now write a PersistanceManager that can dynamically inspect the attributes of an instance of the Person class and insert the data into the People table, mapping the data in the FirstName property to the fname column and the LastName property to the lname column.
As to your question regarding the instances of the attributes, the instance of the attribute is not created for each instance of your Class. All instances of People will share the same instance of the TableMappingAttribute and ColumnMappingAttributes. In fact, the attribute instances are only created when you actually query for the attributes the first time.

C# provides a mechanism for defining declarative tags, called attributes, which you can place on certain entities in your source code to specify additional information. The information that attributes contain can be retrieved at run time through reflection. You can use predefined attributes or you can define your own custom attributes.
http://msdn.microsoft.com/en-us/library/aa288059%28v=VS.71%29.aspx

How to design to prompt users for new values for properties of deserialized objects?

Right now, I'm currently serializing a class like this:
class Session
{
String setting1;
String setting2;
...etc... (other member variables)
List<SessionAction> actionsPerformed;
}
Where SessionAction is an interface that just has one method. All implementations of the SessionAction interface have various properties describing what that specific SessionAction does.
Currently, I serialize this to a file which can be loaded again using the default .Net binary serializer. Now, I want to serialize this to a template. This template will just be the List of SessionActions serialized to a file, but upon loading it back into memory at another time, I want some properties of these SessionActions to require input from the user (which I plan to dynamically generate GUI controls on the fly depending on the property type). Right now, I'm stuck on determining the best way to do this.
Is there some way I could flag some properties so that upon using reflection, I could determine which properties need input from user? Or what are my other options? Feel free to leave comments if anything isn't clear.

For info, I don't recommend using BinaryFormatter for anything that you are storing long-term; it is very brittle between versions. It is fine for short-lived messages where you know the same version will be used for serialization and deserialization.
I would recommend any of: XmlSerializer, DataContractSerializer (3.0), or for fast binary, protobuf-net; all of these are contract-based, so much more version tolerant.
Re the question; you could use things like Nullable<T> for value-types, and null for strings etc - and ask for input for those that are null? There are other routes involving things like the ShouldSerialize* pattern, but this might upset the serialization APIs.

If you know from start what properties will have that SessionAction, you must implement IDeserializationCallback and put to those props the attribute [NonSerialized]. When you implement the OnDeserialization method you get the new values from the user.