We Keep Coding

Query string properties stored as XML

I am using Entity Framework to query a db which is defined by a model: inside this model I have several classes having a #region dynamic values:
[DataContract]
public class Job : AbstractEntity, IJob
{
[DataMember]
public virtual Guid Id { get; set; }
...
#region dynamic values
[DataMember]
public virtual string MetadataValue { get; set; }
[DataMember]
public virtual string ParametersValue { get; set; }
[DataMember]
public virtual string AttributesValue { get; set; }
#endregion
#region links
...
#endregion
}
AttributesValue, MetadataValue and ParametersValue are declared as string but are stored inside the db as XML documents. I am aware that this is not consistent with the model and should be changed, but for some reasons it has been managed this way and I am not allowed to modify it.
I have created a Unit Test in order to better handle the problem, and here is the code:
public class UnitTest1
{
private ModelContext mc;
[TestInitialize]
public void TestInit()
{
IModelContextFactory mfactory = ModelContextFactory.GetFactory();
mc = mfactory.CreateContextWithoutClientId();
}
[TestMethod]
public void TestMethod1()
{
DbSet<Job> jobs = mc.Job;
IQueryable<string> query = jobs
.Where(elem => elem.AttributesValue == "<coll><item><key>ids:ui:description</key><value>Session Test</value></item><item><key>ids:all:type</key><value>signature</value></item></coll>")
.Select(elem => elem.AttributesValue);
List<string> attrs = new List<string>(query);
foreach (string av in attrs)
{
Console.WriteLine(av ?? "null");
}
Assert.AreEqual(1, 1);
}
}
A quick explanation about the TestInit and ModelContext:
ModelContext inherit from DbContext and is an abstract class implemented by SqlModelContext and OracleModelContext (both override OnModelCreating). Depending on the connection string, CreateContextWithoutClientId return a SqlModelContext or an OracleModelContext. Summary: a Factory pattern.
Let's get down to brass tacks: the TestMethod1.
The problem here is in the Where method and the error returned is, as expected:
SqlException: The data types nvarchar and xml are incompatible in the equal to operator.
(From now on I will only consider the AttributesValue property)
I thought of some possible solutions, which are:
Creating a new property inside the model (but not mapped to the db) and use it as a "proxy" instead of accessing directly AttributesValue. However only mapped properties can be used in Linq, so I discarded it.
Operating directly on the inner SQL query generated by the IQueryable and using a customized CAST for Oracle and Sql Server db. I'd rather avoid go for this for obvious reasons.
Is there a way to specify a custom Property Getter so that I can cast AttributesValue to string before it is accessed? Or maybe some configuration on the DbModelBuilder?
I'm using standard Entity Framework 6, Code-First approach.

There is no standard xml data type or standard canonical function for converting string to xml or vice versa.
Fortunately EF6 supports the so called Entity SQL Language which supports an useful construct called CAST:
CAST (expression AS data_type)
The cast expression has similar semantics to the Transact-SQL CONVERT expression. The cast expression is used to convert a value of one type into a value of another type.
It can be utilized with the help of the EntityFramework.Functions package and Model defined functions.
Model defined functions allow you to associate Entity SQL expression with user defined function. The requirement is that the function argument must be an entity.
The good thing about Entity SQL operators is that they are database independent (similar to canonical functions), so the final SQL is still generated by the database provider, hence you don't need to write separate implementations for SqlServer and Oracle.
Install the EntityFramework.Functions package through Nuget and add the following class (note: all the code requires using EntityFramework.Functions;):
public static class JobFunctions
{
const string Namespace = "EFTest";
[ModelDefinedFunction(nameof(MetadataValueXml), Namespace, "'' + CAST(Job.MetadataValue AS String)")]
public static string MetadataValueXml(this Job job) => job.MetadataValue;
[ModelDefinedFunction(nameof(ParametersValueXml), Namespace, "'' + CAST(Job.ParametersValue AS String)")]
public static string ParametersValueXml(this Job job) => job.ParametersValue;
[ModelDefinedFunction(nameof(AttributesValueXml), Namespace, "'' + CAST(Job.AttributesValue AS String)")]
public static string AttributesValueXml(this Job job) => job.AttributesValue;
}
Basically we add simple extension method for each xml property. The body of the methods doesn't do something useful - the whole purpose of these methods is not to be called directly, but to be translated to SQL when used inside LINQ to Entities query. The required mapping is provided through ModelDefinedFunctionAttribute and applied via package implemented custom FunctionConvention. The Namespace constant must be equal to typeof(Job).Namespace. Unfortunately due to the requirement that attributes can use only constants, we can't avoid that hardcoded string as well as the entity class / property names inside the Entity SQL string.
One thing that needs more explanation is the usage of '' + CAST. I wish we could use simply CAST, but my tests show that SqlServer is "too smart" (or buggy?) and removes the CAST from expression when used inside WHERE. The trick with appending the empty string prevents that behavior.
Then you need to add these functions to entity model by adding the following line to your db context OnModelCreating override:
modelBuilder.AddFunctions(typeof(JobFunctions));
Now you can use them inside your LINQ to Entities query:
IQueryable<string> query = jobs
.Where(elem => elem.AttributesValueXml() == "<coll><item><key>ids:ui:description</key><value>Session Test</value></item><item><key>ids:all:type</key><value>signature</value></item></coll>")
.Select(elem => elem.AttributesValue);
which translates to something like this in SqlServer:
SELECT
[Extent1].[AttributesValue] AS [AttributesValue]
FROM [dbo].[Jobs] AS [Extent1]
WHERE N'<coll><item><key>ids:ui:description</key><value>Session Test</value></item><item><key>ids:all:type</key><value>signature</value></item></coll>'
= ('' + CAST( [Extent1].[AttributesValue] AS nvarchar(max)))
and in Oracle:
SELECT
"Extent1"."AttributesValue" AS "AttributesValue"
FROM "ORATST"."Jobs" "Extent1"
WHERE ('<coll><item><key>ids:ui:description</key><value>Session Test</value></item><item><key>ids:all:type</key><value>signature</value></item></coll>'
= ((('')||(TO_NCLOB("Extent1"."AttributesValue")))))

C# + MongoDB - ObjectId without using MongoDB DataTypes/Attributes

Using MongoDB as my data store makes me to have ObjectID type as primary key by Default. It also can be changed by using Guid with [BsonId] attribute. Which is also defined in MongoDB C# Driver library. I would like to have my Entities independent from Data layer.
Can I just use name Id for the property to identify primary key? What else I can try?

You can use BsonClassMap instead of using attributes to keep your classes "clean".
// 'clean' entity with no mongo attributes
public class MyClass
{
public Guid Id { get; set; }
}
// mappings in data layer
BsonClassMap.RegisterClassMap<MyClass>(cm =>
{
cm.AutoMap();
cm.MapIdMember(c => c.Id).SetIdGenerator(CombGuidGenerator.Instance);
});

OPTION 1: Stick with BsonId and use the Facade Pattern
The [BsonId] property is what you'd use to indicate that the _id property should be linked to a specific property. There isn't a way around that (short of ignoring _id entirely in your crud operations which seems like a bad idea).
So, if you want to separate your "entity" object from your "data layer" then just use a poco class.
-- Use a poco class as a substitute for a record. That class is only for data storage: a quick way to get data in/out of mongo, and a great alternative to working with bson documents.
-- Use a facade on top of that poco class for your entity layer. I don't find it useful to re-invent the wheel, so I typically ask our devs have the entity interface inherit the data-layer (poco) interface, but you can do it however you'd like
Breaking up a sample MyObject class
IMyObjectRecord (declared at the dal and contains only properties and mongo-specific attributes)
IMyObject:IMyObjectRecord (declared at the entity level and may include added properties and methods)
MyObjectRecord:IMyObjectRecord (declared inside the dal, contains mongo-specific attributes. Could be declared internal if you wanted to be really strict about separation).
MyObject:IMyObject (could be, for example, a facade on top of the IMyObjectRecord class you pull from the dal).
Now - you get all the benefits of the facade, and you have a hard-coded link between the properties BUT, you get to keep Bson attributes contained in your dal.
OK, fine. But I really really really HATE that answer.
Yeah. I can accept that. OK, so how about a Convention Pack? If you ABSOLUTELY PROMISE that you'll call your Id's "Id" and you SWEAR that you'll type them as strings (or -- use some other convention that is easy to identify), then we could just use a convention pack like the one I stole from here
namespace ConsoleApp {
class Program {
private class Foo {
// Look Ma! No attributes!
public string Id { get; set; }
public string OtherProperty { get; set; }
}
static void Main(string[] args) {
//you would typically do this in the singleton routine you use
//to create your dbClient, so you only do it the one time.
var pack = new ConventionPack();
pack.Add(new StringObjectIdConvention());
ConventionRegistry.Register("MyConventions", pack, _ => true);
// Note that we registered that before creating our client...
var client = new MongoClient();
//now, use that client to create collections
var testDb = client.GetDatabase("test");
var fooCol = testDb.GetCollection<Foo>("foo");
fooCol.InsertOne(new Foo() { OtherProperty = "Testing", Id="TEST" });
var foundFoo = fooCol.Find(x => x.OtherProperty == "Testing").ToList()[0];
Console.WriteLine("foundFooId: " + foundFoo.Id);
}
//obviously, this belongs in that singleton namespace where
//you're getting your db client.
private class StringObjectIdConvention : ConventionBase, IPostProcessingConvention {
public void PostProcess(BsonClassMap classMap) {
var idMap = classMap.IdMemberMap;
if (idMap != null && idMap.MemberName == "Id" && idMap.MemberType == typeof(string)) {
idMap.SetIdGenerator(new StringObjectIdGenerator());
}
}
}
}
}
What's a Convention Pack
It's a little set of mongo "rules" that get applied during serialize/deserialize. You register it once (when you setup your engine). In this case, the sample pack is telling mongo "if you see a field called 'Id', then save it as a string to _id, please."
These can get really complex and fun. I'd dig into convention packs if you really really really hate the other approach. It's a good way to force all your mongo "attribute driven" logic into one self-contained location.

I have stumbled on the same problem myself, and I didn't want to have mongo attributes inside my classes.
I have created a small wrapper example to show how I save and find elements without having an Id property on the data classes of my business logic.
The wrapper class:
public static class Extensions
{
public static T Unwrap<T>(this MongoObject<T> t)
{
return t.Element;
}
}
public class MongoObject<T>
{
[BsonId]
private ObjectId _objectId;
public T Element { get; }
public MongoObject(T element)
{
Element = element;
_objectId = new ObjectId();
}
}
I have also added an extension method to easily unwrap.
Saving an element is simple
public void Save<T>(T t)
{
_collection.InsertOne(new MongoObject<T>(t));
}
To find an element we can do a linq-like query:
Say we have a data class:
public class Person
{
public string Name { get; set; }
}
then we can find such an element by
public Person FindPersonByName(string name)
{
return _collection.AsQueryable().FirstOrDefault(
personObject => personObject.Element.Name == name).Unwrap();
}
We can also generalize by making MongoObject implement IQueryable<T> and this would make the use of the wrapper even more convenient

If i understand correctly. You want to put your entity to other layer without attribute.
I think you can try this
public object Id { get; set; }
after that you can put your Id which is coming from mongodb without attribute

Factory pattern with a class that can has different class sub types

I have data from multiple organisations (police, fire, office) that need output in different formats.
To achieve this, I defined the following (this is a little simplified):
Transaction class -
"Success" indicator - Boolean.
"Type of department"- String or Enum.
A class which can be of any type - Police, Fire or Office (My question is on this as you will see).
A GenerateOutput() method - to handle generation of file formats.
Police class
Age - String
VehicleNumber - Integer
Supervisor - String
Fire class
Name - String
FireEngineNumber - Integer
County - Enum
WorkTimings - Enum
Office Class
Age - String
DeskNumber - Integer
Department - String
PayScale - Enum
IsManagement - Bool
As you can see, the Police, Fire and Office classes dont share anything in common and are primarily intended as data carrying entities. I intend to use a Factory to return an appropriate generic (not a C# generic) Transaction object with the data (Transaction object with Police, Fire or Office data within it) and then pass the returned object to a Strategy pattern which determines the file format (CSV, Excel, or XML; specified in a configuration file) each one needs.
My problem is in the definition of the Transaction object.
What type does the class in "3." of the Transaction class need to be? The data for each org differs, there are no common members, I am unable to define a common class for all.
Is the overall design appropriate? What other designs should I consider?
Based on Peter's comments below:
I think using generics might work, I ran into a problem though. I would like to use a factory to return the object requested, using GetTransactionObject, as below. What should be the return type of GetTransactionObject to accomodate this.
class TransactionFactory
{
Dictionary<string, Type> typeClassLookup;
public TransactionFactory()
{
typeClassLookup = new Dictionary<string, Type>();
typeClassLookup.Add("Police", typeof(PoliceData));
typeClassLookup.Add("Fire", typeof(FireData));
}
Transaction<????> GetTransactionObject(string org)
{
if( typeClassLookup.TryGetValue(org, out typeValue))
{
switch (typeValue.ToString())
{
case "policeData":
transactionObject = new Transaction<PoliceData>() { Data = new PoliceData(), params = null};
case "FireData":
transactionObject = new Transaction<FireData>() {Data = new FireData(), params = null};
}
}
return transactionObject;

If the types really have nothing in common, then you need no explicit base class. System.Object suffices, just as with many other generic types (i.e. any generic type lacking a constraint).
In other words, you could declare as:
class Transaction<T>
{
public bool Success { get; private set; }
public T Entity { get; private set; }
public Transaction(bool success, T entity)
{
Success = success;
Entity = entity;
}
public void GenerateOutput() { /* something goes here */ }
}
Personally, I would avoid adding a "department type" member. After all, that's implicit from the type parameter T. But you could add that easily to the above if you want.
If and when you find that the types do have something in common, such that your Transaction<T> type needs to do more than simply hold onto an instance of one of those types (which is about all it can do without a constraint), then you will be able to put that commonality into an interface or base class (depending on the specific need), and specify that in a constraint for the Transaction<T> class.
Note that it's not clear what you mean for the GenerateOutput() to do, or how it should work. But assuming that you want output that is specific for each Entity value, it seems to me that that is your "something in common". I.e., it's not the Transaction<T> class at all that needs to implement that method, but rather each entity type. In that case, you have something like this:
interface IDepartmentEntity
{
void GenerateOutput();
}
class Office : IDepartmentEntity
{
public void GenerateOutput() { /* department-specific logic here */ }
}
// etc.
Then you can declare:
class Transaction<T> where T : IDepartmentEntity
{
public bool Success { get; private set; }
public T Entity { get; private set; }
public Transaction(bool success, T entity)
{
Success = success;
Entity = entity;
}
public void GenerateOutput() { Entity.GenerateOutput(); }
}
EDIT:
Per Prasant's follow-up edit, with a request for advice on the GetTransactionObject()…
The right way to do this depends on the caller and the context, a detail not provided in the question. IMHO, the best scenario is where the caller is aware of the type. This allows the full power of generics to be used.
For example:
class TransactionFactory
{
public Transaction<T> GetTransactionObject<T>()
where T : IDepartmentEntity, new()
{
return new Transaction<T>()
{
Data = new T(),
params = null
}
}
}
Then you call like this:
Transaction<FireData> transaction = factory.GetTransactionObject<FireData>();
The caller, of course already knowing the type it is creating, then can fill in the appropriate properties of the transaction.Data object.
If that approach is not possible, then you will need for Transaction<T> itself to have a base class, or implement an interface. Note that in my original example, the IDepartmentEntity interface has only one method, and it's the same as the GenerateOutput() method in the Transaction class.
So maybe, that interface is really about generating output instead of being a data entity. Call it, instead of IDepartmentEntity, something like IOutputGenerator.
In that case, you might have something like this:
class Transaction<T> : IOutputGenerator
{
// all as before
}
class TransactionFactory
{
public IOutputGenerator GetTransactionObject(string org)
{
if( typeClassLookup.TryGetValue(org, out typeValue))
{
switch (typeValue.ToString())
{
case "policeData":
transactionObject = new Transaction<PoliceData>() { Data = new PoliceData(), params = null};
case "FireData":
transactionObject = new Transaction<FireData>() {Data = new FireData(), params = null};
}
}
return transactionObject;
}
}
This is an inferior solution, as it means the caller can only directly access the IOutputGenerator functionality. Anything else requires doing some type-checking and special-case code, something that really ought to be avoided whenever possible.
Note: if the Transaction type has other members which, like the GenerateOutput() method, are independent of the contained type T here, and which would be useful to callers who don't know T, then a possible variation of the above is to not reuse the interface used for the department-specific data types, but instead declare a base class for Transaction<T>, named of course Transaction, containing all those members not related to T. Then the return value can be Transaction.

What type does the class in "3." of the Transaction class need to be?
To decouple your department classes from the various export types, I recommend you make the department classes implement a common interface. Something like this:
public interface Exportable {
// return a list of attribute names, values, and types to export
IList<Tuple<String, String, Type>> GetAttributes();
}
For example:
public class Police : Exportable {
public IList<Tuple<String, String, Type>> GetAttributes() {
// return list size 3 - attribute info for Age, VehicleNumber, Supervisor
}
}
Is the overall design appropriate? What other designs should I consider?
The Transaction class design doesn't seem well suited for this problem.
Consider an Export class with a method for each export type, each method which receives the attributes returned from the Exportable interface method. Basic outline:
public static class Export {
public static boolean CSV(IList<Tuple<String, String, Type>> attributes) {
// export attributes to CSV, return whether succeeded
}
public static boolean Excel(IList<Tuple<String, String, Type>> attributes) {
// export attributes to Excel, return whether succeeded
}
// same thing for XML
}

cast Table<T> to something

I have a datacontext, and it has Authors table.
public partial Author:IProductTag{}
I want to cast Table<Authors> object to Table<IProductTag>, but that appears to be impossible. I am trying to do that because I want my method to be able to work with different tables which come as input parameters. To be more specific, I need to execute OrderBy and Select methods of the table. I have few other tables, entities of which implement IProductTag . Also, I tried to write a function like:
public static void MyF<t>(){
Table<t> t0 = (Table<t>)DataContext.GetMyTableUsingReflection();
}
But it fails at compile-time. And if I cast the table to something like ITable or IQueriable, then the OrderBy and Select functions simply don't work. So how do you deal with it?

I suspect you want to make your method generic - so instead of
void DoSomethingWithTable(Table<IProductTag> table)
you should have
void DoSomethingWithTable<T>(Table<T> table) where T : class, IProductTag
That should work fine, assuming you only need to read entities (and apply query operators etc). If that doesn't work for you, please give more details about what your method needs to do.
(You say that your attempt to use reflection failed, but you haven't said in what way it failed. Could you give more details?)

I have no idea what a ProductTag is so I've used different types to show my solution to this problem. Yes there doesn't seem to be a way to get a Table<T>, but you can get IQueryable<T> which works just as well (at least for my situation).
I have a simple analytics database, where each website has its own table containing both generic and specific items. I wanted to use an interface for the shared data.
public interface ISession
{
public DateTime CreateDt {get; set; }
public string HostAddress {get; set; }
public int SessionDuration {get; set; }
}
public static IQueryable<ISession> GetQueryableTable(MyDataContext db, string site)
{
Type itemType;
switch (item)
{
case "stackoverflow.com":
itemType = typeof(Analytics_StackOverflow);
break;
case "serverfault.com":
itemType = typeof(Analytics_ServerFault);
break;
default: throw Exception();
}
return db.GetTable(itemType).Cast<ISession>();
}
You can then do a query like this :
var table = GetQueryableTable(db, "stackoverflow.com");
var mySessions = table.Where(s => s.HostAddress == MY_IP);
To create a new row you can use reflection :
var rowType = typeof(Analytics_ServerFault);
var newRow = (ISession) rowType.GetConstructor(new Type[0]).Invoke(new object[0]);
(I have a function to get GetRowType - which is not shown here).
Then to insert into the table I have a separate helper function:
public static void Insert(MyDataContext db, ISession item)
{
// GetTable is defined by Linq2Sql
db.GetTable(GetRowType(domain)).InsertOnSubmit(item);
}

Possible ways to return a subset of data from Repository<T>?

Let's say I need to display a list of customers, but only want to display the Name and somehow associate the key to the name within a list control.
It would probably be costly to retrieve the entire list of customers and all it's properties. In this scenario, would it be better to create another class with the properties that are required (in this case Id and Name)?
A basic implementation could look like this:
public class Customer {
public int Id { get; set; }
public string Firstname { get; set; }
public string Lastname { get; set; }
public int Age { get; set; }
.....
}
public class CustomerListView {
public int Id { get; set; }
public string Name { get; set; }
}
public interface IRepository<T> {
public T Find(int id);
public IEnumerable<T> FindAll();
....
}
public class Repository<T>: IRepository<T> {
....
}
public class CustomerRepository: Repository<Customer> {
public IEnumerable<CustomerListView> FindAllListView();
}
Would this approach be appropriate? What other options would there be?

In order to achieve such goals, I create a simple 'View' class, for example CustomerView, which just contains the properties that are needed to display an overview.
My Repository then has a method which returns a collection of these CustomerView objects.
I mostly use NHibernate in my projects. Nhibernate allows you to use 'projections'.
So, what I do in my repository is this:
(note that the code below is just some pseudo-code; it won't compile).
public IList<CustomerView> GetAllCustomers()
{
ICriteria crit = _session.CreateCriteria (typeof(Customer));
crit.AddProjection ( ... );
crit.SetResultTransformer (new EntityToBeanTransformer(typeof(CustomerView));
return crit.ToList();
}
In fact, it comes down to this: I tell my O/R mapper that it should query Customers, but that it should return entities of type 'CustomerView'.
In the defintion of the projection, I also define which properties of the Customer class map to which properties of the CustomerView class.
Then, the O/R mapper is smart enough to generate a very simple query, which only retrieves those fields that are required to populate the CustomerView class.
For instance, the query that is executed can be as simple as:
SELECT customerid, customername FROM tblCustomer

If you use IQueryable as your return instead of IEnumerable than there is no cost of doing:
CustomerRepository().GetAll().Find(1) because AsQueryable doesn't actually execute until you request data. That means LINQ can optimize it out to a:
SELECT .... FROM .... WHERE ID = 1 instead of
GET EVERYTHING. FIND WHERE THE ID = 1
See this post for an explanation:
Why use AsQueryable() instead of List()?
Using this approach you could create an anonymous class and futher narrow down the data going over the wire to just what you want. That way the query generated by LINQ is optimized to the fullest.

If you have to retrieve the list form a Database then your proposal makes some sense but I would look into a Linq and anonymous type solution.
If the list of Customers already exists in memory then there there are no savings.

You could combine the techniques used by Nissan and Frederik (anonymous types and NHibernate) by using Linq-to-NHibernate.
Item #31 in Bill Wagner's More Effective C# says "limit type scope by using anonymous types", and I agree. BTW, I recommend the whole book.