We Keep Coding

How can I extract values as strings from an xml file based on the element/property name in a generated .Net class or the original XSD?

I have a large complex XSD set.
I have C# classes generated from those XSDs using xsd.exe. Naturally, though the majority of properties in the generated classes are strings, many are decimals, DateTimes, enums or bools, just as they should be.
Now, I have some UNVALIDATED data that is structured in the correct XML format, but may well NOT be able to pass XSD validation, let alone be put into an instance of the relevant .Net object. For example, at this stage, for all we know the value for the element that should be a DateTime could be "ABC" - not even parseable as a DateTime - let alone other string elements respecting maxLength or regex pattern restrictions. This data is ready to be passed in to a rules engine that we already have to make everything valid, including defaulting things appropriately depending on other data items, etc.
I know how to use the various types in System.Xml to read the string value of a given element by name. Clearly I could just hand craft code to get out all the elements that exist today by name - but if the XSD changes, the code would need to be reworked. I'd like to be able to either directly read the XSD or use reflection on the generated classes (including attributes like [System.Xml.Serialization.XmlTypeAttribute(TypeName=...] where necessary) to find exactly how to recursively query the XML down to the the raw, unverified string version of any given element to pass through to the ruleset, and then after the rules have made something valid of it, either put it back into the strongly typed object or back into a copy of the XML for serialization into the object.
(It has occurred to me that an alternative approach would be to somehow automatically generate a 'stringly typed' version of the object - where there are not DateTimes etc; nothing but strings - and serialize the xml into that. I have even madly thought of taking the xsd.exe generated .cs file and search/replacing all the enums and base types that aren't strings to strings, but there has to be a better way.)
In other words, is there an existing generic way to pull the XElement or attribute value from some XML that would correspond to a given item in a .Net class if it were serialized, without actually serializing it?

Sorry to self-answer, and sorry for the lack of actual code in my answer, but I don't yet have the permission of my employer to share the actual code on this. Working on it, I'll update here when there is movement.
I was able to implement something I called a Tolerant XML Reader. Unlike most XML deserializing, it starts by using reflection to look at the structure of the required .Net type, and then attempts to find the relevant XElements and interpret them. Any extra elements are ignored (because they are never looked for), any elements not found are defaulted, and any elements found are further interpreted.
The main method signature, in C#, is as follows:
public static T TolerantDeserializeIntoType<T>(
XDocument doc,
out List<string> messagesList,
out bool isFromSuppliedData,
XmlSchemaSet schemas = null,
bool tolerant = true)
A typical call to it might look like this:
List<string> messagesList;
bool defaultOnly;
SomeType result = TolerantDeserializeIntoType<SomeType>(someXDocument, out messagesList, out defaultOnly);
(you may use var; I just explicitly put the type there for clarity in this example).
This will take any XDocument (so the only criteria of the original was that it was well-formed), and make an instance of the specified type (SomeType, in this example) from it.
Note that even if nothing at all in the XML is recognized, it will still not fail. The new instance will simply have all properties / public fields nulled or defaulted, the MessageList would list all the defaulting done, and the boolean out paramater would be FALSE.
The recursive method that does all the work has a similar signature, except it takes an XElement instead of an XDocument, and it does not take a schemaSet. (The present implementation also has an explicit bool to indicate a recursive call defaulting to false. This is a slightly dirty way to allow it to gather all failure messages up to the end before throwing an error if tolerant is false; in a future version I will refactor that to only expose publicly a version without that, if I even want to make the XElement version public at all):
public static T TolerantDeserializeXElementIntoType<T>(
ref XElement element,
ref List<string> messagesList,
out bool isFromSuppliedValue,
bool tolerant = true,
bool recursiveCall = false)
How it works, detail
Starting with the main call, the one with with an XDocument and optional SchemaSet:
If a schema Set that will compile is supplied (actually, it also looks for xsi:noNamespaceSchemaLocation as well) the initial XDocument and schemaSet call runs a standard XDocument.Validate() across the supplied XDocument, but this only collects any issued validation error callbacks. It won't throw an exception, and is done for only two reasons:
it will give some useful messages for the MessageList, and
it will populate the SchemaInfo of all XElements to
possibly use later in the XElement version.
(note, however, that the
schema is entirely optional. It is actually only used to resolve
some ambiguous situations where it can be unclear from the C#
object if a given XElement is mandatory or not.)
From there, the recursive XElement version is called on the root node and the supplied C# type.
I've made the code look for the style of C# objects generated by xsd.exe, though most basic structured objects using Properties and Fields would probably work even without the CustomAttributes that xsd.exe supplies, if the Xml elements are named the same as the properties and fields of the object.
The method looks for:
Arrays
Simple value types, explicitly:
String
Enum
Bool
then anything
else by using the relevant TryParse() method, found by reflection.
(note that nulls/xsi:nill='true' values also have to be specially
handled)
objects, recursively.
It also looks for a boolean 'xxxSpecified' in the object for each field or property 'xxx' that it finds, and sets it appropriately. This is how xsd.exe indicates an element being omitted from the XML in cases where null won't suffice.
That's the outline. As I said, I may be able to put actual code somewhere like GitHub in due course. Hope this is helpful to someone.

What are these namespace properties doing?

I know this is pretty silly but just wondered if anyone had a link or knows exactly what this code is doing on my page?
namespace com.gvinet.EblAdapter.ebl
{
[Serializable]
[DesignerCategory("code")]
[GeneratedCode("System.Xml", "4.0.30319.225")]
[DebuggerStepThrough]
[XmlType(Namespace = "http://addresshere")]
public class TSAPassenger
{
then here is all of the strings for the form like name, address and such
I am thinking it is trying to grab the XML file that was created from the Database but just want to make sure.

It is not. These are all just metadata attributes.
Serializeable - Use the standard XmlSerializer to take public properties and fields and convert to XML for transport using no customization to the format (like ISerializable would). It is usually only used when going out of process (remoting, Web Services, WCF, etc)
DesignerCategory - This can be used a number of ways. This one tends to be used by the property grid in visual studio as a way to organize sections.
GeneratedCode - The application generated it for you, utilizing the System.Xml namespace in version 4.0.
DebuggerStepThrough - If you are stepping through code (F11), by default, skip over anything here (don't step into a property getting for example).
XmlType - Part of the serializer that allows you to provide a specific namespace that is generated in the output.
The items here do not actually get anything, just describe certain aspects of how something may be loaded/handled.
Hope that makes sense.

The Serializable and XmlType attributes are instructing the XML serializer that the class can be serialized and the schema to use when doing so.
XmlType Attribute
Serializable Attribute
DesignerCategory("code") Attribute
DebuggerStepThrough Attribute

These are attributes - used for declarative programming - you can find more about declarative programming online. But here is the link to .net attribute hierarchy page to get you started: http://msdn.microsoft.com/en-us/library/aa311259(VS.71).aspx
Also, these pages may be helpful:
What are attributes: What are attributes in .NET?
Attributes in C#: http://www.codeproject.com/Articles/2933/Attributes-in-C

Building a string representation of an object using a "mask" or user supplied Format String

I'm not really sure what tags should be on this sort of question so feel free to give me some suggestions if you think some others are more suited.
I have a dynamic object with an unknown number or properties on it, it's from a sort of dynamic self describing data model that lets the user build the data model at runtime. However because all of the fields holding relevant information to the user are in dynamic properties, it's difficult to determine what should be the human readable identifier, so it's left up to the administrator. (Don't think it matters but this is an ASP.NET MVC3 Application). To help during debugging I had started decorating some classes with DebuggerDisplayAttribute to make it easier to debug. This allow me to do things like
[DebuggerDisplay(#"\{Description = {Description}}")]
public class Group
to get a better picture of what a specific instance of an object is. And this sort of setup would be perfect but I can't seem to find the implementation of this flexibility. This is especially useful on my dynamic objects because the string value of the DebuggerDisplayAttribute is resolved by the .NET framework and I have implementations of TryGetMember on my base object class to handle the dynamic aspect. But this only makes it easier for development. So I've added a field on what part of my object is still strongly typed and called it Title, and I'd like to let the administer set the implementation using their own format, so to speak. So for example they might build out a very simplistic rental tracking system to show rentals and they might specify a format string along the lines of
"{MovieTitle} (Due: {DueDate})"
I would like that when they save the record to add some logic to first update the Title property by resolving the format string to substitute each place holder with the value of the appropriate property on the dynamic object. So this might resolve to a title of
"Inception (Due: May 21, 2011)", or a more realistic scenario of a format string of
"{LastName}, {FirstName}"
I don't want the user to have to update the title of a record when they change the first name field or the last name field. I fully realize this will likely use reflection but I'm hoping some one out there can give me some pointers or even a working example to handle complex format strings that could be a mix if literal text and placeholders.
I've not had much luck looking for an implementation on the net that will do what I want since I'm not really sure what keywords would give me the most relevant search results?

You need two things:
1) A syntax for formatting strings
You have already described a syntax where variables are surrounded by bracers, and if you want to use that you need to build a parser that can parse that. Perhaps you also want to add ways to specify say a date or a number format.
2) Rules for resolving variables
If there is a single context object you can use reflection and match variable names to properties but if your object model is more complex you can add conventions for searching say a hierarchy of objects.
If you are planning to base your model objects on dynamic chances are that you will find the Clay library on CodePlex interesting.

C# Attributes and their uses

I really don't know much about attributes in general in C#, I've seen them in use in a lot of different ways/places but I don't think I see the importance of some of them. Some definitely have importance because they provide a noticeable function, such as [Serializable]. Yet, others don't seem so important, such as one my coworker uses to mark properties with [DataMember].
I suppose my question is, what are attributes and how are they useful? Is there a way to create my own attributes and how can I tell if fields/methods/classes/whatever have particular attributes or what values are set in those attributes?

what are attributes?
Attributes enable you to embed information about a type or method in the metadata which describes that type or method.
You typically want to use attributes to describe facts about the mechanism of the type or method rather than the meaning of the type or method. For example, suppose you have a type Employee. A fact about the meaning of Employee is that it is a kind of Person, that an Employee has a Manager, and so on. A fact about the mechanism of Employee is that it can be the target of data binding, or it can be serialized to disk, or whatever. An employee cannot be serialized to disk, but the class Employee can be. Attributes let you separate information about the technical details from the semantic model.
Is there a way to create my own attributes?
Yes. Create a class which extends Attribute. By convention you want to name it "FooAttribute". If you do so you can use either the [Foo] syntax or the [FooAttribute] syntax at your discretion.
How can I tell if fields/methods/classes/whatever have particular attributes or what values are set in those attributes?
Use the GetCustomAttributes method on the reflection objects.
Where should I read for more information?
Start with the attributes tutorial:
http://msdn.microsoft.com/en-us/library/aa288454(VS.71).aspx
And then read all of chapter 17 of the C# specification.

Attributes are a means by which you can associate metadata with types in .NET. This allows you to check for a type and get information about it that's separate from the "runtime" information of the type.
This can be very useful. You mentioned [Serializable], but other simple examples include many of the System.ComponentModel types, such as Description, which is used by the property grid to "describe" properties when you work with them in the designer. Since the "description" of a property isn't really related to the behavior of the type in a program (at runtime), it doesn't belong in the class. However, it's very handy when you go to edit a control in a visual designer, for example, to see a description (or category, etc) of a property. Attributes are the means by which this is handled.

I think the answer to the following question will provide you some insight to your questions.
How do attribute classes work?
Here is a repost of the answer I provided.
Attributes are essentially meta data that can be attached to various pieces of your code. This meta data can then be interogate and affect the behaviour of certain opperations.
Attributes can be applied to almost every aspect of your code. For example, attributes can be associated at the Assembly level, like the AssemblyVersion and AssemblyFileVersion attributes, which govern the version numbers associated with the assembly.
[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyFileVersion("1.0.0.0")]
Then the Serializable attribute for example can be applied to a type declaration to flag the type as supporting serialization. In fact this attribute has special meaning within the CLR and is actually stored as a special directive directly on the type in the IL, this is optimized to be stored as a bit flag which can be processed much more efficiently, there are a few attributes on this nature, which are known as pseudo custom attributes.
Still other attributes can be applied to methods, properties, fields, enums, return values etc. You can get an idea of the possible targets an attribute can be applied to by looking at this link
http://msdn.microsoft.com/en-us/library/system.attributetargets(VS.90).aspx
Further to this, you can define your own custom attributes which can then be applied to the applicable targets that your attributes are intended for. Then at runtime your code could reflect on the values contained in the custom attributes and take appropriate actions.
For a rather naive example, and this is just for the sake of example :)
You might want to write a persistence engine that will automatically map Classes to tables in your database and map the properties of the Class to table columns. You could start with defining two custom attributes
TableMappingAttribute
ColumnMappingAttribute
Which you can then apply to your classes, as an example we have a Person class
[TableMapping("People")]
public class Person
{
[ColumnMapping("fname")]
public string FirstName {get; set;}
[ColumnMapping("lname")]
public string LastName {get; set;}
}
When this compiles, other than the fact that the compiler emits the additional meta data defined by the custom attributes, little else is impacted. However you can now write a PersistanceManager that can dynamically inspect the attributes of an instance of the Person class and insert the data into the People table, mapping the data in the FirstName property to the fname column and the LastName property to the lname column.
As to your question regarding the instances of the attributes, the instance of the attribute is not created for each instance of your Class. All instances of People will share the same instance of the TableMappingAttribute and ColumnMappingAttributes. In fact, the attribute instances are only created when you actually query for the attributes the first time.

C# provides a mechanism for defining declarative tags, called attributes, which you can place on certain entities in your source code to specify additional information. The information that attributes contain can be retrieved at run time through reflection. You can use predefined attributes or you can define your own custom attributes.
http://msdn.microsoft.com/en-us/library/aa288059%28v=VS.71%29.aspx

c#: Using Assemblies (via Reflection) as a (meta)data store

SOME CONTEXT
one of my projects requires carrying around some of "metadata" (yes I hate using that word).
What the metadata specifically consists of is not important, only that it's more complex than a simple "table" or "list" - you could think of it as a mini-database of information
Currently I have this metadata stored in an XML file and have an XSD that defines the schema.
I want to package this metadata with my project, currently that means keeping the XML file as a resource
However, I have been looking for a more strongly-typed alternative. I am considering moving it from an XML file to C# code - so instead of using XML apis to traverse my metadata, relying on .NET code via reflection on types
Besides the strong(er) typing, some useful characteristics I see from using an assembly are for this: (1) I can refactor the "schema" to some extent with tools like Resharper, (2) The metadata can come with code, (3) don't have to rely on any external DB component.
THE QUESTIONS
If you have tried something like this, I am curious about what you learned.
Was your experience positive?
What did you learn?
What problems in this approach did you uncover?
What are some considerations I should take into account?
Would you do this again?
NOTES
Am not asking for how to use Reflection - no help is needed there
Am fundamentally asking about your experiences and design considerations
UPDATE: INFORMATION ABOUT THE METADATA
Because people are asking I'll try describing the metadata a bit more. I'm trying to abstract a bit - so this will seem a bit artificial.
There are three entities in the model:
A set of "groups" - each group has a unique name and several properites (usually int values that represent ID numbers of some kind)
Each "group" contains 1 or more "widgets" (never more than 50) - each item has properties like name (therea are multiple names), IDs, and various boolean properties.
Each widget contains a one or more "scenarios". Each "scenario" is documentation- a URL to a description of how to use the widget.
Typically I need to run these kinds of "queries"
Get the names of all the widgets
Get the names of all groups that contain at least one widget where BoolProp1=true
Get given the ID of a widget, which group contains that widget
How I was thinking about modelling the entities in the assembly
There are 3 classes: Group, Widget, Documentation
There are 25 Groups so I will have 25 Group classes - so "FooGroup" will derive from Group, same pattern follows for widgets and documentation
Each class will have attributes to account for names, ids, etc.

I have used and extended Metadata for a large part of my projects, many of them related to describing components, relationships among them, mappings, etc.
(Major categories of using attributes extensively include O/R Mappers, Dependency Injection framework, and Serialization description - specially XML Serialization)
Well, I'm going to ask you to describe a little bit more about the nature of the data you want to embed as resource. Using attributes are naturally good for the type of data that describes your types and type elements, but each usage of attributes is a simple and short one. Attributes (I think) should be very cohesive and somehow independent from each other.
One of the solutions that I want to point you at, is the "XML Serialization" approach. You can keep your current XMLs, and put them into your assemblies as Embedded Resource (which is what you've probably done already) and read the whole XML at once into a strongly-typed hierarchy of objects.
XML Serialization is very very simple to use, much simpler than the typical XML API or even LINQ2XML, in my opinion. It uses Attributes to map class properties to XML elements and XML attributes. Once you've loaded the XML into the objects, you have everything you want in the memory as "typed" data.
Based on what I understand from your description, I think you have a lot of data to be placed on a single class. This means a large and (in my opinion) ugly attribute code above the class. (Unless you can distribute your data among members making each of them small and independent, which is nice.)
I have many positive experiences using XML Serialization for large amount of data. You can arrange data as you want, you get type safety, you get IntelliSence (if you give your XSD to visual studio), and you also get half of the Refactoring. ReSharper (or any other refactoring tool that I know of) don't recognize XML Serialization, so when you refactor your typed classes, it doesn't change the XML itself, but changes all the usage of the data.
If you give me more details on what your data is, I might be able to add something to my answer.
For XML Serialization samples, just Google "XML Serialization" or look it up in MSDN.
UPDATE
I strongly recommend NOT using classes for representing instances of your data. Or even using a class to encapsulate data is against its logical definition.
I guess your best bet would be XML Serialization, provided that you already have your data in XML. You get all the benefits you want, with less code. And you can perform any query on the XML Serializable objects using LINQ2Objects.
A part of your code can look like the following:
[XmlRoot]
public class MyMetadata
{
[XmlElement]
public Group[] Groups { get; set; }
}
public class Group
{
[XmlAttribute]
public string Name { get; set; }
[XmlAttribute]
public int SomeNumber { get; set; }
[XmlElement]
public Widget[] Widgets { get; set; }
}
public class Widget
{
...
}
You should call new XmlSerializer(typeof(MyMetadata)) to create a serializer, and call its Deserialize method giving it the stream of your XML, and you get a filled instance of MyMetadata class.

It's not clear from your description but it sounds like you have assembly-level metadata that you want to be able to access (as opposed to type-level). You could have a single class in each assembly that implements a common interface, then use reflection to hunt down that class and instantiate it. Then you can hard-code the metadata within.
The problems of course are the benefits that you lose from the XML -- namely that you can't modify the metadata without a new build. But if you're going this direction you probably have already taken that into account.