I've written this code to project one to many relation but it's not working:
using (var connection = new SqlConnection(connectionString))
{
connection.Open();
IEnumerable<Store> stores = connection.Query<Store, IEnumerable<Employee>, Store>
(#"Select Stores.Id as StoreId, Stores.Name,
Employees.Id as EmployeeId, Employees.FirstName,
Employees.LastName, Employees.StoreId
from Store Stores
INNER JOIN Employee Employees ON Stores.Id = Employees.StoreId",
(a, s) => { a.Employees = s; return a; },
splitOn: "EmployeeId");
foreach (var store in stores)
{
Console.WriteLine(store.Name);
}
}
Can anybody spot the mistake?
EDIT:
These are my entities:
public class Product
{
public int Id { get; set; }
public string Name { get; set; }
public double Price { get; set; }
public IList<Store> Stores { get; set; }
public Product()
{
Stores = new List<Store>();
}
}
public class Store
{
public int Id { get; set; }
public string Name { get; set; }
public IEnumerable<Product> Products { get; set; }
public IEnumerable<Employee> Employees { get; set; }
public Store()
{
Products = new List<Product>();
Employees = new List<Employee>();
}
}
EDIT:
I change the query to:
IEnumerable<Store> stores = connection.Query<Store, List<Employee>, Store>
(#"Select Stores.Id as StoreId ,Stores.Name,Employees.Id as EmployeeId,
Employees.FirstName,Employees.LastName,Employees.StoreId
from Store Stores INNER JOIN Employee Employees
ON Stores.Id = Employees.StoreId",
(a, s) => { a.Employees = s; return a; }, splitOn: "EmployeeId");
and I get rid of exceptions! However, Employees are not mapped at all. I am still not sure what problem it had with IEnumerable<Employee> in first query.
This post shows how to query a highly normalised SQL database, and map the result into a set of highly nested C# POCO objects.
Ingredients:
8 lines of C#.
Some reasonably simple SQL that uses some joins.
Two awesome libraries.
The insight that allowed me to solve this problem is to separate the MicroORM from mapping the result back to the POCO Entities. Thus, we use two separate libraries:
Dapper as the MicroORM.
Slapper.Automapper for mapping.
Essentially, we use Dapper to query the database, then use Slapper.Automapper to map the result straight into our POCOs.
Advantages
Simplicity. Its less than 8 lines of code. I find this a lot easier to understand, debug, and change.
Less code. A few lines of code is all Slapper.Automapper needs to handle anything you throw at it, even if we have a complex nested POCO (i.e. POCO contains List<MyClass1> which in turn contains List<MySubClass2>, etc).
Speed. Both of these libraries have an extraordinary amount of optimization and caching to make them run almost as fast as hand tuned ADO.NET queries.
Separation of concerns. We can change the MicroORM for a different one, and the mapping still works, and vice-versa.
Flexibility. Slapper.Automapper handles arbitrarily nested hierarchies, it isn't limited to a couple of levels of nesting. We can easily make rapid changes, and everything will still work.
Debugging. We can first see that the SQL query is working properly, then we can check that the SQL query result is properly mapped back to the target POCO Entities.
Ease of development in SQL. I find that creating flattened queries with inner joins to return flat results is much easier than creating multiple select statements, with stitching on the client side.
Optimized queries in SQL. In a highly normalized database, creating a flat query allows the SQL engine to apply advanced optimizations to the whole which would not normally be possible if many small individual queries were constructed and run.
Trust. Dapper is the back end for StackOverflow, and, well, Randy Burden is a bit of a superstar. Need I say any more?
Speed of development. I was able to do some extraordinarily complex queries, with many levels of nesting, and the dev time was quite low.
Fewer bugs. I wrote it once, it just worked, and this technique is now helping to power a FTSE company. There was so little code that there was no unexpected behavior.
Disadvantages
Scaling beyond 1,000,000 rows returned. Works well when returning < 100,000 rows. However, if we are bringing back >1,000,000 rows, in order to reduce the traffic between us and SQL server, we should not flatten it out using inner join (which brings back duplicates), we should instead use multiple select statements and stitch everything back together on the client side (see the other answers on this page).
This technique is query oriented. I haven't used this technique to write to the database, but I'm sure that Dapper is more than capable of doing this with some more extra work, as StackOverflow itself uses Dapper as its Data Access Layer (DAL).
Performance Testing
In my tests, Slapper.Automapper added a small overhead to the results returned by Dapper, which meant that it was still 10x faster than Entity Framework, and the combination is still pretty darn close to the theoretical maximum speed SQL + C# is capable of.
In most practical cases, most of the overhead would be in a less-than-optimum SQL query, and not with some mapping of the results on the C# side.
Performance Testing Results
Total number of iterations: 1000
Dapper by itself: 1.889 milliseconds per query, using 3 lines of code to return the dynamic.
Dapper + Slapper.Automapper: 2.463 milliseconds per query, using an additional 3 lines of code for the query + mapping from dynamic to POCO Entities.
Worked Example
In this example, we have list of Contacts, and each Contact can have one or more phone numbers.
POCO Entities
public class TestContact
{
public int ContactID { get; set; }
public string ContactName { get; set; }
public List<TestPhone> TestPhones { get; set; }
}
public class TestPhone
{
public int PhoneId { get; set; }
public int ContactID { get; set; } // foreign key
public string Number { get; set; }
}
SQL Table TestContact
SQL Table TestPhone
Note that this table has a foreign key ContactID which refers to the TestContact table (this corresponds to the List<TestPhone> in the POCO above).
SQL Which Produces Flat Result
In our SQL query, we use as many JOIN statements as we need to get all of the data we need, in a flat, denormalized form. Yes, this might produce duplicates in the output, but these duplicates will be eliminated automatically when we use Slapper.Automapper to automatically map the result of this query straight into our POCO object map.
USE [MyDatabase];
SELECT tc.[ContactID] as ContactID
,tc.[ContactName] as ContactName
,tp.[PhoneId] AS TestPhones_PhoneId
,tp.[ContactId] AS TestPhones_ContactId
,tp.[Number] AS TestPhones_Number
FROM TestContact tc
INNER JOIN TestPhone tp ON tc.ContactId = tp.ContactId
C# code
const string sql = #"SELECT tc.[ContactID] as ContactID
,tc.[ContactName] as ContactName
,tp.[PhoneId] AS TestPhones_PhoneId
,tp.[ContactId] AS TestPhones_ContactId
,tp.[Number] AS TestPhones_Number
FROM TestContact tc
INNER JOIN TestPhone tp ON tc.ContactId = tp.ContactId";
string connectionString = // -- Insert SQL connection string here.
using (var conn = new SqlConnection(connectionString))
{
conn.Open();
// Can set default database here with conn.ChangeDatabase(...)
{
// Step 1: Use Dapper to return the flat result as a Dynamic.
dynamic test = conn.Query<dynamic>(sql);
// Step 2: Use Slapper.Automapper for mapping to the POCO Entities.
// - IMPORTANT: Let Slapper.Automapper know how to do the mapping;
// let it know the primary key for each POCO.
// - Must also use underscore notation ("_") to name parameters in the SQL query;
// see Slapper.Automapper docs.
Slapper.AutoMapper.Configuration.AddIdentifiers(typeof(TestContact), new List<string> { "ContactID" });
Slapper.AutoMapper.Configuration.AddIdentifiers(typeof(TestPhone), new List<string> { "PhoneID" });
var testContact = (Slapper.AutoMapper.MapDynamic<TestContact>(test) as IEnumerable<TestContact>).ToList();
foreach (var c in testContact)
{
foreach (var p in c.TestPhones)
{
Console.Write("ContactName: {0}: Phone: {1}\n", c.ContactName, p.Number);
}
}
}
}
Output
POCO Entity Hierarchy
Looking in Visual Studio, We can see that Slapper.Automapper has properly populated our POCO Entities, i.e. we have a List<TestContact>, and each TestContact has a List<TestPhone>.
Notes
Both Dapper and Slapper.Automapper cache everything internally for speed. If you run into memory issues (very unlikely), ensure that you occasionally clear the cache for both of them.
Ensure that you name the columns coming back, using the underscore (_) notation to give Slapper.Automapper clues on how to map the result into the POCO Entities.
Ensure that you give Slapper.Automapper clues on the primary key for each POCO Entity (see the lines Slapper.AutoMapper.Configuration.AddIdentifiers). You can also use Attributes on the POCO for this. If you skip this step, then it could go wrong (in theory), as Slapper.Automapper would not know how to do the mapping properly.
Update 2015-06-14
Successfully applied this technique to a huge production database with over 40 normalized tables. It worked perfectly to map an advanced SQL query with over 16 inner join and left join into the proper POCO hierarchy (with 4 levels of nesting). The queries are blindingly fast, almost as fast as hand coding it in ADO.NET (it was typically 52 milliseconds for the query, and 50 milliseconds for the mapping from the flat result into the POCO hierarchy). This is really nothing revolutionary, but it sure beats Entity Framework for speed and ease of use, especially if all we are doing is running queries.
Update 2016-02-19
Code has been running flawlessly in production for 9 months. The latest version of Slapper.Automapper has all of the changes that I applied to fix the issue related to nulls being returned in the SQL query.
Update 2017-02-20
Code has been running flawlessly in production for 21 months, and has handled continuous queries from hundreds of users in a FTSE 250 company.
Slapper.Automapper is also great for mapping a .csv file straight into a list of POCOs. Read the .csv file into a list of IDictionary, then map it straight into the target list of POCOs. The only trick is that you have to add a propery int Id {get; set}, and make sure it's unique for every row (or else the automapper won't be able to distinguish between the rows).
Update 2019-01-29
Minor update to add more code comments.
See: https://github.com/SlapperAutoMapper/Slapper.AutoMapper
I wanted to keep it as simple as possible, my solution:
public List<ForumMessage> GetForumMessagesByParentId(int parentId)
{
var sql = #"
select d.id_data as Id, d.cd_group As GroupId, d.cd_user as UserId, d.tx_login As Login,
d.tx_title As Title, d.tx_message As [Message], d.tx_signature As [Signature], d.nm_views As Views, d.nm_replies As Replies,
d.dt_created As CreatedDate, d.dt_lastreply As LastReplyDate, d.dt_edited As EditedDate, d.tx_key As [Key]
from
t_data d
where d.cd_data = #DataId order by id_data asc;
select d.id_data As DataId, di.id_data_image As DataImageId, di.cd_image As ImageId, i.fl_local As IsLocal
from
t_data d
inner join T_data_image di on d.id_data = di.cd_data
inner join T_image i on di.cd_image = i.id_image
where d.id_data = #DataId and di.fl_deleted = 0 order by d.id_data asc;";
var mapper = _conn.QueryMultiple(sql, new { DataId = parentId });
var messages = mapper.Read<ForumMessage>().ToDictionary(k => k.Id, v => v);
var images = mapper.Read<ForumMessageImage>().ToList();
foreach(var imageGroup in images.GroupBy(g => g.DataId))
{
messages[imageGroup.Key].Images = imageGroup.ToList();
}
return messages.Values.ToList();
}
I still do one call to the database, and while i now execute 2 queries instead of one, the second query is using a INNER join instead of a less optimal LEFT join.
A slight modification of Andrew's answer that utilizes a Func to select the parent key instead of GetHashCode.
public static IEnumerable<TParent> QueryParentChild<TParent, TChild, TParentKey>(
this IDbConnection connection,
string sql,
Func<TParent, TParentKey> parentKeySelector,
Func<TParent, IList<TChild>> childSelector,
dynamic param = null, IDbTransaction transaction = null, bool buffered = true, string splitOn = "Id", int? commandTimeout = null, CommandType? commandType = null)
{
Dictionary<TParentKey, TParent> cache = new Dictionary<TParentKey, TParent>();
connection.Query<TParent, TChild, TParent>(
sql,
(parent, child) =>
{
if (!cache.ContainsKey(parentKeySelector(parent)))
{
cache.Add(parentKeySelector(parent), parent);
}
TParent cachedParent = cache[parentKeySelector(parent)];
IList<TChild> children = childSelector(cachedParent);
children.Add(child);
return cachedParent;
},
param as object, transaction, buffered, splitOn, commandTimeout, commandType);
return cache.Values;
}
Example usage
conn.QueryParentChild<Product, Store, int>("sql here", prod => prod.Id, prod => prod.Stores)
According to this answer there is no one to many mapping support built into Dapper.Net. Queries will always return one object per database row. There is an alternative solution included, though.
Here is another method:
Order (one) - OrderDetail (many)
using (var connection = new SqlCeConnection(connectionString))
{
var orderDictionary = new Dictionary<int, Order>();
var list = connection.Query<Order, OrderDetail, Order>(
sql,
(order, orderDetail) =>
{
Order orderEntry;
if (!orderDictionary.TryGetValue(order.OrderID, out orderEntry))
{
orderEntry = order;
orderEntry.OrderDetails = new List<OrderDetail>();
orderDictionary.Add(orderEntry.OrderID, orderEntry);
}
orderEntry.OrderDetails.Add(orderDetail);
return orderEntry;
},
splitOn: "OrderDetailID")
.Distinct()
.ToList();
}
Source: http://dapper-tutorial.net/result-multi-mapping#example---query-multi-mapping-one-to-many
Here is a crude workaround
public static IEnumerable<TOne> Query<TOne, TMany>(this IDbConnection cnn, string sql, Func<TOne, IList<TMany>> property, dynamic param = null, IDbTransaction transaction = null, bool buffered = true, string splitOn = "Id", int? commandTimeout = null, CommandType? commandType = null)
{
var cache = new Dictionary<int, TOne>();
cnn.Query<TOne, TMany, TOne>(sql, (one, many) =>
{
if (!cache.ContainsKey(one.GetHashCode()))
cache.Add(one.GetHashCode(), one);
var localOne = cache[one.GetHashCode()];
var list = property(localOne);
list.Add(many);
return localOne;
}, param as object, transaction, buffered, splitOn, commandTimeout, commandType);
return cache.Values;
}
its by no means the most efficient way, but it will get you up and running. I'll try and optimise this when i get a chance.
use it like this:
conn.Query<Product, Store>("sql here", prod => prod.Stores);
bear in mind your objects need to implement GetHashCode, perhaps like this:
public override int GetHashCode()
{
return this.Id.GetHashCode();
}
Related
I have some C# classes which represent database objects, some of which contain one or more other custom objects or enumerables of custom objects. I'm using dapper for queries, and slapper to map to the custom objects. It works great for single object. I can easily grab a parent object with a specific ID from the database, do some inner joins, and map it and all the things it "owns" to my custom objects in C#. Problem comes when I want to do a select over multiple parent-IDs.
Some context, let's say I have a person, that person has a list of hobbies which have an ID and a Description, a list of days they're available which also have an ID and Description, and maybe another custom field such as whether they have or are even willing to be around children which can also boil down to a simple ID and Description. We'll call that last field child status. I'd write a select statement like this:
SELECT
,person.id as Id
,person.first_name as FirstName
,person.last_name as LastName
,hobby.Id as Hobbies_Id
,hobby.Description as Hobbies_Description
,avail.Id as Availabilities_Id
,avail.Description as Availabities_Description
,child.Id as ChildStatus_Id
,child.Description as ChildStatus_Description
FROM
users.users person
JOIN
users.userhobbies uhobby
ON
person.id = uhobby.UserId -- one-to-many with relational table
JOIN
users.avail hobby
ON
uhobby.HobbyId = hobby.Id
JOIN
users.useravailabilities uavail
ON
person.id = uavail.UserId -- one-to-many with relational table
JOIN
users.availabilities avail
ON
uavail.AvailId = avail.Id
JOIN
users.childstatuses child
ON
person.ChildStatusId = child.Id
Then I want this mapped to a user like this:
class User
{
public Guid Id {get; set;}
public string FirstName {get; set;}
public string LastName {get; set;}
public IEnumerable<Hobby> Hobbies {get; set;}
public IEnumerable<Availability> Availabilities {get; set;}
public ChildStatus ChildStatus {get; set;}
}
Since I'm using exact naming conventions and everything here, the query from Dapper and the Automapping work great just like this:
// Using the above sql in a variable
var data = Connection.Query<dynamic>(sql);
var dataReal = Slapper.AutoMapper.MapDynamic<User>(data);
return dataReal;
This works great, but it only ever returns one user. I have a similar method which takes an ID and all of my test users can be retrieved perfectly by passing the ID. I've tried scouring the internet, looking through documentation, and all I found was this: https://github.com/SlapperAutoMapper/Slapper.AutoMapper/issues/57 who seemed to just slip through the cracks. I also tried mapping the dynamic data to various other structures with no luck. Thanks in advance!
Update:
I've come up with a somewhat brutal, "sledgehammer" type solution. I'm not sure if, at this point, I'm forcing myself to use Slapper when there might be a more convenient solution. However, I wanted to ensure anyone in a similar situation might have a chance at making it work. Here's the new C# section:
var data = Connection.Query<dynamic>(sql);
IEnumerable<Guid> Ids = data.Select(row => (Guid)row.id).Distinct();
List<User> results = new List<User>();
foreach (Guid Id in Ids)
{
IEnumerable<dynamic> rows = data.Where(x => { return ((Guid) x.id).Equals(Id); });
User model = (Slapper.AutoMapper.MapDynamic<User>(rows, false) as IEnumerable<User>).FirstOrDefault();
if (model != null)
{
results.Add(model);
}
}
return results;
As you can see, I'm generating a list of unique "primary object" ID's and selecting those rows into their own lists, which I then pass to Slapper. I've passed the "cache = false" parameter to avoid squeezing unrelated data into every object after the first. I could probably get around this by actually keeping the UserHobby/UserAvailability/UserPhoto Ids in place, but I don't like the way that makes my object look. Hopefully this helps someone.
I'm not familiar with Slapper, but I'll show you what I've done with Dapper to construct a complex graph of objects with bi-directional references.
In short, construct a Dictionary or KeyedCollection prior to calling connection.Query<>, then reference it inside the Dapper lambda expression.
This method returns a list of service calls. Each service call is assigned to one technician and one customer. However, a technician may be assigned multiple service calls to multiple customers. And a customer may have multiple technicians on-site.
public ServiceCallResponse GetServiceCallsDapper(ServiceCallRequest Request)
{
var queryParameters = new {statuses = Request.Statuses, createDate = Request.CreateDate};
const string splitOn = "Number,Id"; // Id indicates beginning of second class (Technician). Number indicates begining of third class (Customer).
// Note multiple columns are named "Number". See note below about how Dapper maps columns to class properties.
// Note Dapper supports parameterized queries to protect against SQL injection attacks, including parameterized "where in" clauses.
const string query = #"sql query here..."
ServiceCallResponse response = new ServiceCallResponse(); // Keyed collection properties created in constructor.
using (IDbConnection connection = new SqlConnection("DB connection string here..."))
{
connection.Open();
// Dapper adds a generic method, Query<>, to the IDbConnection interface.
// Query<(1)ServiceCall, (2)Technician, (3)Customer, (4)ServiceCall> means
// construct a (1)ServiceCall, (2)Technician, and (3)Customer class per row, add to an IEnumerable<(4)ServiceCall> collection, and return the collection.
// Query<TFirst, TSecond, TThird, TReturn> expects SQL columns to appear in the same order as the generic types.
// It maps columns to the first class, once it finds a column named "Id" it maps to the second class, etc.
// To split on a column other than "Id", specify a splitOn parameter.
// To split for more than two classes, specify a comma-delimited splitOn parameter.
response.ServiceCalls.AddRange(connection.Query<ServiceCall, Technician, Customer, ServiceCall>(query, (ServiceCall, Technician, Customer) =>
{
// Notice Dapper creates many objects that will be discarded immediately (Technician & Customer parameters to lambda expression).
// The lambda expression sets references to existing objects, so the Dapper-constructed objects will be garbage-collected.
// So this is the cost of using Dapper. We trade unnecessary object construction for simpler code (compared to constructing objects from IDataReader).
// Each row in query results represents a single service call.
// However, rows repeat technician and customer data through joined tables.
// Avoid constructing duplicate technician and customer classes.
// Refer to existing objects in global collections, or add Dapper-mapped objects to global collections.
// This avoid creating duplicate objects to represent same data.
// Newtonsoft JSON serializer preserves object instances from service to client.
Technician technician;
Customer customer;
if (response.Technicians.Contains(Technician.Id))
{
technician = response.Technicians[Technician.Id];
}
else
{
response.Technicians.Add(Technician);
technician = Technician;
}
if (response.Customers.Contains(Customer.Number))
{
customer = response.Customers[Customer.Number];
}
else
{
response.Customers.Add(Customer);
customer = Customer;
}
// Set object associations.
ServiceCall.Technician = technician;
ServiceCall.Customer = customer;
technician.ServiceCalls.Add(ServiceCall);
if (!technician.Customers.Contains(customer))
{
technician.Customers.Add(customer);
}
customer.ServiceCalls.Add(ServiceCall);
if (!customer.Technicians.Contains(technician))
{
customer.Technicians.Add(technician);
}
return ServiceCall;
}, queryParameters, splitOn: splitOn));
}
return response;
}
Using this technique requires you to set PreserveReferencesHandling = true on the JsonSerializer class so object references are preserved on the client-side. Otherwise, Json.NET will construct duplicate objects and technician.Customers.Count will always == 1.
For example, if John Doe is assigned a service call at Acme and another at Contoso, his technician.Customers.Count will equal 1 if you leave PreserveReferencesHandling == false (Json.NET will construct two Technician objects each named John Doe).
Basic idea in psuedocode
Select(a, b) => new Tuple<List<Item1>, List<Item2>>(a, b)
I am trying to accomplish this in:
A single query to the db
obviously using linq (query or method syntax)
Here are the two classes involved
public class Bundle
{
public Guid Id { get; set; }
public string Name { get; set; }
public HashSet<Inventory> Inventories { get; set; }
}
public class Inventory
{
public Guid Id { get; set; }
public string Name { get; set; }
public int Stock { get; set; }
}
Right now all I can think of is
using (var context = new MyEntities())
{
return new Tuple<IEnumerable<Inventory>, IEnumerable<Bundle>>(context.Inventories.OrderBy(a => a.Stock).ToList()
, context.Bundles.Include(b => b.Inventories).OrderBy(c => c.Name).ToList());
}
However, this would hit the database twice.
I know UNION is used to combine result sets from 2 different queries but the two queries must have the same # of columns so I'm assuming it's best used when selecting the same data.
How can I select data from two different tables into two separate lists with only ONE hit on the db?
If you want two result sets, you can do it by throwing two queries. This can be done in a single database call without issue, but it won't magically divide into two sets of objects as you are interested in.
In fact, asking more than one question and getting more than one result set is very common when the cost of establishing connection (instantiation cost or latency cost, etc.) is great enough to warrant it. I have done it myself in a stored procedure, asking for everything a page needs in one query.
But, if performance is the key issue, caching is also very common. And, if these are drop down lists, or something else where the list requested is small, and the list does not change often, you can even pull it into memory when the application starts and let it set on the web server so you are not making the database trip.
I am not fond of LINQ to SQL, as it creates a mess for DBAs, but you can do it something like this (just one example, any method where you can chain commands will work):
var connString = "{conn string here}";
var commandString = "select * from tableA; select * from tableB";
var conn = new SqlConnection(connString);
var command = new SqlCommand(commandString, conn);
try {
conn.Open();
var result = command.Execute();
// work with results here
}
finally {
conn.Dispose();
}
I have not filled in all of the details, but you can do this with any number of commands. Once again, if the information does not change, consider a single hit and holding in memory (caching through programming) or using some other type of cache mechanism.
If you are worried about performance, I don't think that reducing the amount of queries is the way to go. In fact, if you want fine-grain optimizations, LINQ isn't the most appropriate tool, either.
That being said, you could make the two different objects match the same interface/columns, filling with dummy properties for those missing in the other type. This should theoretically be translated to a SQL union containing all of the columns.
var union = context.Inventories
.Select(i => new { i.Id, i.Name, i.Stock, Inventories=null })
.Concat(context.Bundles.Select(b => new { b.Id, b.Name, Stock=0, b.Inventories));
Note that in this case Concat is preferred over of Union, as it doesn't alter the order of your rows and allows duplicate rows.
What you are asking for is not possible.
If you are trying to get two distinct results you need to hit the DB twice, once per result set. You can do an outer join, but this would make your result set larger than it should, and judging from the fact that you want to hit the DB once, you care about performance.
Also, if performance was an issue, I would not use Linq in the first place.
I think that you can do this by asking for multiple result sets (as Gregory pointed out already), I am not sure though what would be the "best" way for this. Why don't you take a look at this msdn article for example?
How to: Use Stored Procedures Mapped for Multiple Result Shape
Or this, for a linq-to-sql approach:
LINQ to SQL: returning multiple result sets
So i've run out of ideas as to why this is so slow. Maybe you can help.
So I am trying to do a simple get on record from an oracle db using nHibernate mapping by code. I'm using nHibernate verison 3.3.1.4 from nuget.
Here is the mapping code:
public class Person
{
public virtual PersonKey Key { get; set; }
public virtual string FirstName { get; set; }
public virtual string LastName { get; set; }
}
public class PersonKey
{
public virtual string PersonId { get; set; }
public override bool Equals(object obj)
{
if (obj == null)
return false;
var t = obj as PersonKey;
if (t == null)
return false;
if (PersonId == t.PersonId)
return true;
return false;
}
public override int GetHashCode()
{
return (PersonId).GetHashCode();
}
}
public class PersonMap : ClassMapping<Person>
{
public PersonMap()
{
Schema("MyDB");
Table("Person");
ComponentAsId(id => id.Key, idMapper => idMapper.Property(p => p.PersonId));
Property(i => i.FirstName);
Property(i => i.LastName);
}
}
Here is the code to create the sessionfactory and retrieve the data.
var mapper = new ModelMapper();
var cfg = new Configuration();
mapper.AddMappings(Assembly.GetExecutingAssembly().GetExportedTypes());
cfg.DataBaseIntegration(c =>
{
c.ConnectionString = #"User Id=user;Password=password;Data Source=MyDB;";
c.Driver<OracleClientDriver>();
c.Dialect<Oracle10gDialect>();
c.LogSqlInConsole = true;
c.LogFormattedSql = true;
c.AutoCommentSql = true;
});
cfg.AddMapping(mapper.CompileMappingForAllExplicitlyAddedEntities());
var sessionFactory = cfg.BuildSessionFactory();
stopwatch.Stop();
Console.WriteLine("Building session factory: {0}", stopwatch.ElapsedMilliseconds);
stopwatch.Restart();
Person entity = null;
using (var session = sessionFactory.OpenSession())
using(var tx = session.BeginTransaction())
{
entity = (Person) session.Get("Person", new PersonKey(){PersonId = "1"});
tx.Commit();
}
The query generated is as follows:
SELECT person0_.PersonId as PersonId0_0_,
person0_.FirstName as FirstName0_0_, person0_.LastName as LastName0_0_,
FROM MyDB.Person person0_
WHERE person0_.PersonId=:p0;
:p0 = '1'
It takes roughly 80-100 seconds on average to get access to the retrieved object. This includes creating the session as well.
So some obvious things I looked for:
The personId coloumn is indexed (it is the primary key of the table).
The database is on a server, so to check it is not the network eating the time, I ran the above generated query using AD0.Net (Oracleconnection + Oraclecommand). This roughly takes 180ms this includes creating the connection and mapping the record to entity class.
Ran the query via PL/SQL developer (took about 32ms).
Check the query plan on the generated query (confirmed the use of the index unique scan and not a full table scan).
I've run the above code against a similar sized sql 2012 db on my local and, it's crazy fast incomparison, around 180ms via the NHibernate code.
Ran nhprof (evaluation) and got the following results:
It seems like the query is run and the results are returned very fast from the database (according to nhprof results), but maybe its the hydration of the query's values into the entity that is taking up the time. I am at a loss at what could be chewing up the time!
My next move is to attach the nHibernate source to the solution and step through it, but download access at work is limited (even to github!). Till then any ideas?
Cheers.
Update: So I've got the nHibernate source in my project and stepped through it. Everything is moving along nicely until the program gets to this piece of code:
for (count = 0; count < maxRows && rs.Read(); count++)
It is on rs.Read() that the execution eats up the time, where rs is the oracle datareader. This piece of code is in the DoQuery function in the Loader.cs file.
The odd thing is if the query passed in is a non-parameterised dynamic query (e.g. select ... from person where personid = '1'), the execution is lighting fast (~ 1ms), but if its parameterised (e.g. select ... from person where personid = :p1), then it's ridiculously slow. Since I want to harness the power of nHibernate, I need to use the generated parameterised queries. I am still trying to figure out why the oracle reader
Found the issue and solution blogged here by Deep Shah.
Essentially, parameterized queries in NHibernate with the microsoft driver for oracle was the reason for the bottleneck.
He goes on to share two solutions:
Replace the current Microsoft oracle driver with an oracle developed Oracle Data Client Driver. Running the same query using new driver eliminates the performance bottelneck. For this to work you will need to have Oracle Data Access Components (ODAC) installed on the machine from which you intend to fire the query.
When setting a parameter on the query via NHibernate use "query.SetAnsiString" instead of "query.SetParameter" when querying against VARCHAR columns.
I have verified both solutions and they both work. Solution one is the one I went with, since I am using mapping-by-code and I am leaving the query generation to nHibernate.
I believe you can also set the data type to ansi string in your mapping as referenced in this post:
NHibernate 3 specify sql data type with loquacious syntax
I don't know Linq2Sql so well yet and I was wondering if there is a trick for this probably common MVVM scenario. I have Linq2Sql data context containing Domain models, but I am fetching data for my customized ViewModel object from it.
var query = from ord in ctx.Table_Orders
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
So i want my ViewModel to inculde both CurrencyId from domain object and the CurrencyText from related table to show it nicely in the View.
This code works great. It generates one DB call with join to fetch the CurrencyText. But the model is simplified, real one has many more fields. I want to make the code reusable because I have many different queries, that returns the same ViewModel. Now every minor change to OrderViewModel requires lots of maintainance.
So I moved the code to OrderViewModel itself as a constructor.
public OrderViewModel(Table_Order ord)
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
}
And call it like this.
var query = from ord in ctx.Table_Orders
select new OrderViewModel(ord);
The Problem: The join is gone DB query is no more optimised. Now I get 1+N calls to database to fetch CurrencyText for every line.
Any comments are welcome. Maybe I have missed different great approach.
This is how far i could get on my own, to get the code reusability. I created a function that does the job and has multiple parameters. Then I need to explicitly pass it everything that has crossed the line of entity.
var query = ctx.Table_Orders.Select(m =>
newOrderViewModel(m, m.Currency.CurrencyText));
The DB call is again optimized. But it still does not feel like I am there yet! What tricks do You know for this case?
EDIT : The final solution
Thanks to a hint by #Muhammad Adeel Zahid I arrived at this solution.
I created an extension for IQueryable
public static class Mappers
{
public static IEnumerable<OrderViewModel> OrderViewModels(this IQueryable<Table_Order> q)
{
return from ord in q
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
}
}
Now i can do this to get all list
var orders = ctx.Table_Order.OrderViewModels().ToList();
or this to get a single item, or anything in between with Where(x => ..)
var order = ctx.Table_Order
.Where(x => x.OrderId == id).OrderViewModels().SingleOrDefault();
And that completely solves this question. The SQL generated is perfect and the code to translate objects is reusable. Approach like this should work with both LINQ to SQL and LINQ to Entities. (Not tested with the latter) Thank You again #Muhammad Adeel Zahid
Whenever we query the database, we mostly require either enumeration of objects (more than one records in db) or we want a single entity (one record in db). you can write your mapping code in method that returns enumeration for whole table like
public IEnumerable<OrderViewModel> GetAllOrders()
{
return from ord in ctx.Table_Orders
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
}
Now you may want to filter these records and return another enumeration for example on currencyID
public IEnumerable<OrderViewModel> GetOrdersByCurrency(int CurrencyID)
{
return GetAllOrders().Where(x=>x.CurrencyId == CurrencyID);
}
Now you may also want to find single record out of all these view models
public OrderViewModel GetOrder(int OrderID)
{
return GetAllOrders().SingleOrDefault(x=>x.OrderId == OrderID);
}
The beauty of IEnumerable is that it keeps adding conditions to query and does not execute it until it is needed. so your whole table will not be loaded unless you really want it and you have kept your code in single place. Now if there are any changes in ViewModel Mapping or in query itself, it has to be done in GetAllOrders() method, rest of code will stay unchanged
You can avoid the N+1 queries problem by having Linq2SQL eagerly load the referenced entites you need to construct your viewmodels. This way you can build one list of objects (and some referenced objects) and use it to construct everything. Have a look at this blog post.
One word of warning though: This technique (setting LoadOptions for the Linq2SQL data context) can only be done once per data context. If you need to perform a second query with a different eager loading configuration, you must re-initalize your data context. I automated this with a simple wrapper class around my context.
I'm having a problem trying to make my LINQ to SQL queries and the mapping to my domain objects DRY without incurring the cost of multiple round trips to the db. Given this example:
var query1 = from x in db.DBProducts
select new MyProduct
{
Id = x.ProductId,
Name = x.ProductName,
Details = new MyProductDetail
{
Id = x.DBProductDetail.ProductDetailId,
Description = x.DBProductDetail.ProductDetailDescription
}
}
The query will make ONE round trip to the DB. Great! However, the problem I see with this is that eventually, I'll also have a 'GetProductDetails' method which will also need to do some of the SAME "data object -> domain object" mapping, very similar to that above.
To alleviate some of the mapping, I thought it might be a cool idea to extend the partial data object classes to do the mapping for me, like so:
public partial class DBProduct
{
MyProduct ToDomainObject()
{
return new MyProduct
{
Id = this.ProductId,
Name = this.ProductName,
Details = this.DBProductDetails.ToDomainObject()
};
}
}
public partial class DBProductDetail
{
MyProductDetail ToDomainObject()
{
return new MyProductDetail
{
Id = this.ProductDetailId,
Description = this.ProductDetailDescription
};
}
}
Nice! Now, I could simply rewrite query1 as follows:
var query1 = from x in db.DBProducts
select x.ToDomainObject();
This makes the code more DRY and more readable. Additionally, other queries that need to do the same type of mapping can simply use the ToDomainObject() method for the mapping. It works, but with a cost. While watching via Profiler, the first query would call the db ONCE, joining tables where necessary. The second query doesn't join appropriately, thus making multiple calls to the DB. Is there a way to accomplish what I'm trying to do: refactor LINQ to SQL queries so that the mapping to domain objects is DRY (no code duplication)?
Use AutoMapper. Once you've tried it, it's unlikely you will ever see code like this:
new MyProduct
{
Id = x.ProductId,
Name = x.ProductName,
Details = new MyProductDetail
{
Id = x.DBProductDetail.ProductDetailId,
Description = x.DBProductDetail.ProductDetailDescription
}
}