I am running into the "Lazy IO Problem" in Linq and I haven't found a solution that I'm happy with
Setting up the problem
Let say we have SQL tables that looks like
create table Person (
id int primary key not null,
name text not null,
)
create table Dog (
name text primary key not null,
ownerid text primary key not null references Person(name)
)
And in C# we want to use LINQ and Entity Framework to deal with this. Entity Framework classes are defined as partial so we can extend them to add a .Get(string) method and this makes for very clean code.
public partial class Dog
{
public static Dog Get(string dogname)
{
using (var db = new MyDataContext())
{
// LINQ is lazy and doesn't load the referenced Person
return db.Dogs.Single(d => d.name == dogname);
}
}
}
Where the problem happens
Now we attempt to use the Dog object for something
public string DogJson(string dogname)
{
var dog = Dog.Get(dogname);
return JsonConvert.SerializeObject(dog);
}
Since our instance dog contains dog.Owner because of the foreign key, JsonConvert will of course attempt to include it in the json string. But since the DataContext is disposed and LINQ is lazy, an ObjectDisposedException is of course raised because dog.Person had not been evaluated when we disposed the DataContext.
In this case, we don't care about the Owner object at all, we just want to serialize the Dog to json. What is the best way to do that without?
My solution
I have a solution, but I don't particularly like it. Using projection into an anonamous object and casting back to Dog, because we are not allowed to explicitly construct a Dog in the query.
public static Dog Get(string dogname)
{
using (var db = new MyDataContext())
{
var tmpdog = db.Dogs.Where(d => d.name == dogname)
.Select(d => new { name = d.name, ownerid = d.ownerid}).Single();
return new Dog() { name = tmpdog.name, ownerid = tmpdog.ownerid};
}
}
I don't like this solution because it doesn't scale well. This example only has two properties and this gets quickly out of hand. LINQ usually makes for very elegant code and this is not elegant at all. It's also prone to programmer
It sort of feels like I am taking the wrong approach here.
I have had this problem before too, but luckily entity framework provides an easy way around it. You can disable the lazy loading and creation of a dynamic proxy before you query. This will allow the json serializer to run without exception.
public static Dog Get(string dogname)
{
using (var db = new MyDataContext())
{
db.Configuration.ProxyCreationEnabled = false;
db.Configuration.LazyLoadingEnabled = false;
return db.Dogs.Single(d => d.name == dogname);
}
}
The real issue is that your caller should dictate the lifetime of the DbContext, not the callee, since the DogJson method is defining the unit of work. Ideally you should be passing a DbContext instance into the static Get method.
So, rather, your Get code should look more like this:
public static Dog Get(string dogname, MyDataContext db)
{
var result = db.Dogs.SingleOrDefault(d => d.name == dogname);
return result;
}
Then, you can do all of the DTO modifications in your caller, since that's really your unit of work:
public string DogJson(string dogname)
{
using (var db = new MyDataContext())
{
var dog = Dog.Get(dogname, db);
var dogDTO = new Dog { name = dog.name, ownerid = dog.ownerid };
return JsonConvert.SerializeObject(dogDTO);
}
}
See this question about ignoring attributes when serializing objects in Newtonsoft, which I believe is the library that contains the JsonConvert.SerializeObject function.
To summarize the most popular answer is to add the [JsonIgnore] attribute to those fields you do not want to be serialized. In your case, that is Owner so the code would be:
[JsonIgnore]
public Person Owner{ get; set; }
The original poster ended up using virtual properties as noted in their own answer.
Related
Suppose I have a DbSet like so:
DbSet<Teacher>
And a Teacher inherits from a Person (not part of the EF generated classes).
public partial class Teacher: Person
{
}
And I have a list of DbSets like so
var tableList = new List<Type>() { typeof(DbSet<Teacher>), typeof(DbSet<Student>), ... };
Then I want to create a generic function that does the same thing for each of the tables in the list.
How do I do something like the following if I know that all the tables in the list can inherit from the same Person type:
foreach(var tableType in tableList)
{
var table = (DbSet<Person>)Activator.CreateInstance(tableType);
var existing = table.Where(x => x.Name == "whatever");
if (!existing.Any())
{
table.Add(new Person{ Name = "Whatever" });
}
...
The problem is that EF doesn't know which table in the database I want to talk to. How do I tell it that 'in this loop I want you to save the new person to the Student table, in this loop I want you to save to the Teacher table, etc?
I've tried initialising the table like so but neither are useful:
var table = (DbSet<dynamic>)Activator.CreateInstance(tableType);
var table = (DbSet<tableType>)Activator.CreateInstance(tableType);
Note: I know I could've made loads of separate very similar functions for each table type but I've made the above example simple to make the problem easier to understand, I'm not actually saving different types of people in a loop based on whether their name exists. Any ideas? Thanks!
Since there are so many generic methods involved, you would be better off writing your own generic method, then use reflection to call it.
public void ProcessGeneric<T> where T:Person, new()
{
// Perhaps something like;
var dbSet = db.Set<T>();
if (!dbSet.Any(p => p.Name == ...))
dbSet.Add(new T(){ ... });
}
public void Process(Type t)
=> this.GetType()
.GetMethod(nameof(ProcessGeneric))
.MakeGenericMethod(t)
.Invoke(this, new object[] {});
Assuming a model of
[BsonDiscriminator(RootClass = true)]
[BsonKnownTypes(typeof (Dog), typeof (Cat))]
class Animal
{
public ObjectId Id {get;set;}
public string Name {get;set;}
}
class Dog : Animal { public int BarkVolume {get;set;} }
class Cat : Animal { public int PurrVolume {get;set;} }
I can do the following:
var collection = new MongoClient().GetServer().GetDatabase("test").GetCollection<Animal("Animals");
collection.Save( new Dog { Name="spot", BarkVolume=7 } );
collection.Save( new Cat { Name="kitty", PurrVolume=2 } );
However if I then try and query just for cats with
var cats = collection.AsQueryable<Cat>();
foreach(var c in cats)
{
Console.WriteLine("{0} purrs at {1}", c.Name, c.PurrVolume);
}
I'll get an exception "Element BarkVolume does not match any field or property of class Cat".
Of course, if I change my query to:
var cats = collection.AsQueryable<Cat>().Where(x=>x is Cat);
Then no problem, however there then is a warning stating that x is Cat is always true.
Is there a particular reason why the driver doesn't inject a test on the discriminator _t
It's a design decision (which after working with for some time, I agree with). To specify the _t filter you can use the OfType extension method which is cleaner than x => x is Cat.
In the MongoDB C# driver there are typed and untyped options for almost everything. The typed options are only for comfort, they don't change the resulting queries. This is a good thing when you really care about the query performance and index utilization.
For example if you query using a property only the specific type has you don't need to add OfType (and the resulting _t filter) and if you do the query engine might use the _t index which you might not want it to do.
I have been working with entity framework for quite some time and have gotten quite used to using this to dump data into Lists of Classes.
I am now working on a very old project that doesn't have EF and I don't have the time to convert it over to EF (nor would I as it needs a full rewrite).
I am in there though making some minor adjustments and need to pull back data from a stored procedure. I want to store the data in a List of Class, class being one that I created on my own.
I know I can use a DataReader and just read each record passed back and create an item of Class and add that to the list.
List<MyClass> myClassGuy = new List<MyClass>();
dr = cmd.ExecuteReader(CommandBehavior.Default);
while (dr.Read())
{
MyClass myClassItemToAdd = new MyClass();
myClassItemToAdd.VarA = (dr, "VARA");
myClassItemToAdd.VarB = (dr, "VARB");
myClassItemToAdd.VarC = (dr, "VARC");
//etc
myClassGuy.Add(myClassItemToAdd);
}
I know I can do the above but after using EF for such a long time. I have to believe there is a better way of doing this? Is there?
I wrote a whole bunch of code to handle this stuff, then found AutoMapper which (generally) does a pretty good job when working with results from SQL queries:
// setup mapping (this is a once-per-execution thing)
if (AutoMapper.Mapper.FindTypeMapFor<IDataReader, MyData>() == null)
AutoMapper.Mapper.CreateMap<IDataReader, MyData>();
// Read from the DataReader into a list
IList<MyClass> data = AutoMapper.Mapper.Map<IDataReader, IList<MyData>>(dr);
One of the great things about EF is that it automates the process of materializing objects stored in the database.
Using ADO.Net, that work falls on you. You need to read out column values, transform them if appropriate, and put them in the right part of your object.
There's no significantly easier way, without introducing additional frameworks (e.g. AutoMapper), to do that than what you have now.
You should write your own reusable methods to make things easier when you work with your next similar task. As about your exact question, no, there is no simpler solution, but nothing stops you from simplifying your future tasks as much as possible.
You could refactor what you have, one DAL and SomeClass.
The nice thing about not using EF is you have a lot more control on your app's performance. Doesn't matter how much horse power you have still your jockey must be light.
That is of course if your app is critical or some trivial web page.
If you are absolutely unable to use an ORM framework, then you can make then code more generic by modifying your stored procedures to return an XML stream (which should be fairly easy to do) and on the application side, deserialize the stream into objects.
For example, I can define the following data access method which takes a generic type, a name of stored procedure and any number of SqlParameters (optional) and deserializes and returns a list of T. The helper method Deserialize<T> takes in a an XML stream and deserializes it to a List<T>:
public static IEnumerable<T> GetEntity<T>(string storedProcedureName, params SqlParameter[] parameters)
{
try
{
using (SqlConnection connection =
new SqlConnection("connectionString"))
{
connection.Open();
SqlCommand command = new SqlCommand(storedProcedureName, connection);
command.CommandType = System.Data.CommandType.StoredProcedure;
if (parameters != null && parameters.Any())
{
command.Parameters.AddRange(parameters);
}
string result = (string)command.ExecuteScalar();
return Deserialize<T>(result);
}
}
catch (Exception ex)
{
// Handle the exception
return (IEnumerable<T>)default(T);
}
}
private static IEnumerable<T> Deserialize<T>(string xmlStream, params Type[] additionalTypes)
{
XmlSerializer serializer = additionalTypes == null ? new XmlSerializer(typeof(List<T>))
: new XmlSerializer(typeof(List<T>), additionalTypes);
using (var reader = new XmlTextReader(new StringReader(xmlStream)))
{
if (!serializer.CanDeserialize(reader))
{
return (IEnumerable<T>)default(T);
}
return (IEnumerable<T>)serializer.Deserialize(reader);
}
}
And to use it, I can have something as simple as
IEnumerable<MyObject> myObjects = DataAccess.GetEntity<MyObject>("MyStoredProc");
The stored procedure looks like this:
SELECT t.Id as 'Id', t.Name as 'Name'
FROM MyTable t
FOR XML PATH('MyObject'), ROOT('ArrayOfMyObject')
The only tricky part is to make sure column aliases match the properties on the object. You'll also need to add the [XmlRoot("ArrayOfMyObject")] attribute to the MyObject class.
This seems like a lot of code but if you are truly unable to use a framework or library, this would be a decent path to go IMO.
Well it kind of depends. How many properties are you mapping - if it's only a handful then your current approach is probably best, although it's initially onerous it still makes maintenance easy if the SP result set changes.
Then again if we're talking about many fields coming back from the stored procedure, you could use the very lightweight Linq to SQL - it's officially "dead" but still perfect for this kind of thing and it can map to stored procedure results. Or something even lighter like Rob Conery's Massive which at something like 700 loc is a good option.
And of course Automapper is always an option.
However. If for some reason you can't introduce L2S or any 3rd party packages into the legacy codebase, you could go with a simple reflection based mapper class like the example below.
Given this SQL:
create table MyRecords
(
ID int identity,
Name nvarchar(255),
DateCreated datetime,
IsSilly bit
)
Insert into MyRecords select 'John',getdate(),0
Insert into MyRecords select 'Andrew',getdate(),1
Insert into MyRecords select 'Steve',getdate(),0
Insert into MyRecords select 'Max',getdate(),1
Then this console app shows how simple it really is to generalise this kind of thing; error checking, etc, omitted for brevity, and it doesn't support multiple result sets, but you get the idea of how little code is required.
using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Linq;
using System.Reflection;
namespace MiniORM
{
//Attribute to map SQL result set field to class public instance property
public class FieldInfo : Attribute
{
public string FieldName { get; set; }
public Type DataType { get; set; }
}
public class MyRecord
{
[FieldInfo(DataType=typeof(int),FieldName="ID")]
public int ID { get; set; }
[FieldInfo(DataType = typeof(string), FieldName = "Name")]
public string Name { get; set; }
[FieldInfo(DataType = typeof(DateTime), FieldName = "DateCreated")]
public DateTime DateCreated { get; set; }
[FieldInfo(DataType = typeof(bool), FieldName = "IsSilly")]
public bool IsSilly { get; set; }
public override string ToString()
{
return string.Format("{0}-{1}-{2}-{3}", ID, Name, DateCreated, IsSilly);
}
}
public class FieldInfoDBMapper<T> where T: class,new()
{
private readonly Dictionary<string,KeyValuePair<PropertyInfo,Type>> _mapping;
public FieldInfoDBMapper()
{
var t = typeof (T);
_mapping = new Dictionary<string,KeyValuePair<PropertyInfo,Type>>();
foreach (var pi in t.GetProperties(BindingFlags.Instance | BindingFlags.Public))
{
var infos = pi.GetCustomAttributes(typeof(FieldInfo));
if (infos.Any())
{
var fieldInfo = (FieldInfo) infos.First();
_mapping.Add(fieldInfo.FieldName,new KeyValuePair<PropertyInfo, Type>(pi,fieldInfo.DataType));
}
}
}
public List<T> MapFromReader(IDataReader reader)
{
List<T> data = new List<T>();
while (reader.Read())
{
T item = new T();
foreach (var entry in _mapping)
{
var value = Convert.ChangeType(reader[entry.Key],entry.Value.Value);
entry.Value.Key.SetValue(item,value);
}
data.Add(item);
}
return data;
}
}
public class Program
{
static void Main(string[] args)
{
List<MyRecord> records =new List<MyRecord>();
using (
SqlConnection conn =
new SqlConnection("Your connection string here"))
{
conn.Open();
using (SqlCommand comm = new SqlCommand("Select * from MyRecords", conn))
{
var reader = comm.ExecuteReader();
var mapper = new FieldInfoDBMapper<MyRecord>();
records.AddRange(mapper.MapFromReader(reader));
}
}
foreach (var record in records)
{
Console.WriteLine(record.ToString());
}
Console.ReadLine();
}
}
}
Output:
1-John-09/05/2014 00:39:37-False
2-Andrew-09/05/2014 00:39:37-True
3-Steve-09/05/2014 00:39:37-False
4-Max-09/05/2014 00:39:37-True
Hope that helps.
I have a mapping library for stuff like this. https://www.nuget.org/packages/SprocMapper/
Documentation: https://github.com/gtaylor44/SprocMapper
public class MyClass
{
string VarA;
string VarB;
string VarC;
}
public List<MyClass> GetResults()
{
using (SqlConnection conn = new SqlConnection("YourConnection"))
{
return conn.Select().ExecuteReader<MyClass>(conn, "YourStoredProcedure");
}
}
I am not deleting entities. I just sign them by IsDeleted property. The problem is when I get a parent element, its all child elements are loaded even if IsDeleted propery is true or false. Then I did something like below, but I want to know is there a better way to do it ?
var result = from p in context.Set<Product>().Include(x => x.Reviews)
select new
{
Product = x,
ProductReviews = x.ProductReviews.Where(y => !y.IsDeleted)
};
var products = new List<Product>();
foreach (var product in result.OrderBy(x => x.Product.Id).Skip(skipRecords).Take(pageSize))
{
var p = new Product();
p = product.Product;
p.ProductReviews = product.ProductReviews.ToList();
products.Add(p);
}
return products;
How to improve this code block ?
Thanks
What I did to address this type of situation before was to create a specific interface signifying the classes that are "flag deleted" like this and then create an Extension Method that filters them out.
If you only have one class with the IsDeleted property, then you don't need a separate interface and you can just use the class instead. But I'll assume for the moment that you do have multiple classes and that the interface is needed.
So the interface would be defined like so:
public interface IHaveIsDeleted
{
public bool IsDeleted { get; set; }
}
Then my Extension Method would be defined in a static class like so:
public static class MyExtensionMethods
{
public IQueryable<T> FilterDeleted(this IQueryable<T> src) where T : IHaveIsDeleted
{
return src.Where(x => !x.IsDeleted);
}
}
Because this is done on an IQueryable<T>, the where clause gets built into the query that is sent to the database, so there won't be any records returned where IsDeleted is true. So all you would have to do in your example is call x.ProductReviews.FilterDeleted().
Now, the project that I was using this method in was actually using LINQ2SQL. And I'm rather new to EF myself, so there might be a more 'EF specific' way of doing this, (like perhaps some kind of Type Per Hierarchy construct, maybe?), but I just thought this would at least help make your queries simpler.
Hope that helps! ;)
Suppose I have a List of Person (which is a class). It contains about 20 field (Name, Surname, Age, DateOfBirthdate, and so on). So I got this list:
var listOfPersons= MyContext.Persons.Cast<Person>();
Now, I need to iterate through this List, and for each Person adding a new field (which it is not present in the class), called, let's say, CurrentDateTime.
I could create a new object, with the new field, and "copy & paste" values from Person to the new Class. Somethings like:
PersonNew newPerson = new PersonNew("Name", "Surname", "Age", "DateOfBirthdate", ... "CurrentDateTime");
But this is very bad if in the future I change the Person class. So, is there a strategy to "extending Person" with a new field? That takes the Person instance (whatever it is) and adds the new field?
You can create some static method that create PersonNew from Person using Automapper.
public class PersonNew : Person
{
public static PersonNew CreateFromPerson(Person person, DateTime currentDateTime)
{
var newPerson = Mapper.Map<PersonNew>(person);
newPerson.CurrentDateTime = currentDateTime;
}
}
I think that the solution you described works fine. If you want to keep track of each person's birthday without extending the Person class, you might use a Dictionary object
var listOfPersons = MyContext.Perons.Cast<Person>();
Dictionary<Person, DateTime> birthdays = new Dictionary<Person, DateTime>
foreach(Person person in listOfPersons)
{
birthdays.Add(person, getBirthday(person);
}
One solution is to make your class partial, and add your field in another partial definition of your class:
public partial class Person
{
public string Name { get; set; }
public string FirstName { get; set; }
...
}
...
public partial class Person
{
public DateTime CurrentDateTime { get; set; }
}
...
var listOfPersons = MyContext.Persons.Cast<Person>();
foreach (var person in listOfPersons)
{
person.CurrentDateTime = ....
}
Do note that you will use the same instance of your class.
First I would suggest using extension methods for projecting collections instead of iterating. Like that:
var newCollection = oldCollection.Select(entity => MakeNewType(entity))
Second, it's not completely clear what you mean by "extending Person" with a new field. Here are the couple of ways you can accomplish that.
1) Make another class with the new field and map it to the old one. This is a common scenario for asp.net mvc application where you map models to the appropriate viewmodels. Automapper is useful for these types of scenario (see Sławomir Rosiek anwser)
2) Take advantage of dlr in c# 4+. Yuo will lose the intellisense for dynamic objects, but they canned be passed around functions
var newPeople = people.Select(p =>
{
dynamic expando = new ExpandoObject();
expando.Id = p.Id;
expando.FirtName = p.FirtName;
/* ... */
expando.CurrentDateTime = DateTime.Now;
return expando;
});
3) Use Anonymous types. Anonymous types cannot be passed to another functions, so this approach is useful when you need to quickly project data inside a single method and calculate some result
var newPeople = people.Select(p => new
{
Id = p.Id,
FirtName = p.FirtName,
/* ... */
CurrentDateTime = DateTime.Now
});
in both cases you can now access newly "created" property:
foreach(var p in newPeople)
{
Console.WriteLine("CurrentDateTime: {0}", p.CurrentDateTime);
}
4) If you really need to create a fully featured .net class at runtime you can use Reflection.Emit. This scenario is typically used to create dynamic proxies - subclasses which implement some functionality only known at runtime. Entity framework does this.