I am creating a Data Access Layer in C# for an SQL Server database table. The data access layer contains a property for each column in the table, as well as methods to read and write the data from the database. It seems to make sense to have the read methods be instance based. The question I have is regarding handling the database generated primary key property getter/setter and the write method. As far as I know I have three options...
Option 1: Using a static method while only allowing a getter on the primary key would allow me to enforce writing all of the correct values into the database, but is unwieldy as a developer.
Option 2: Using and instance based write method would be more maintainable, but I am not sure how I would handle the get/set on the primary key and it I would probably have to implement some kind of validation of the instance prior to writing to the database.
Option 3: Something else, but I am wary of LINQ and drag/drop stuff, they have burned me before.
Is there a standard practice here? Maybe I just need a link to a solid tutorial?
You might want to read up on active record patterns and some examples of them, and then implement your own class/classes.
Here's a rough sketch of a simple class that contains some basic concepts (below).
Following this approach you can expand on the pattern to meet your needs. You might be OK with retrieving a record from the DB as an object, altering its values, then updating the record (Option2). Or if that is too much overhead, using a static method that directly updates the record in the database (Option1). For an insert, the database (SP/query) should validate the natural/unique key on the table if you need to, and probably return a specific value/code indicating a unique constraint error). For updates, the same check would need to be performed if allowing natural key fields to be updated.
A lot of this depends on what functionality your application will allow for the specific table.
I tend to prefer retrieving an object from the DB then altering values and saving, over static methods. For me, it's easier to use from calling code and can handle arcane business logic inside the class easier.
public class MyEntityClass
{
private int _isNew;
private int _isDirty;
private int _pkValue;
private string _colValue;
public MyEntityClass()
{
_isNew = true;
}
public int PKValue
{
get {return _pkValue;}
}
public string ColValue
{
get {return _colValue;}
set
{
if (value != _colValue)
{
_colValue = value;
_isDirty = true;
}
}
}
public void Load(int pkValue)
{
_pkValue = pkValue;
//TODO: query database and set member vars based on results (_colVal)
// if data found
_isNew = false;
_isDirty = false;
}
public void Save()
{
if (_isNew)
{
//TODO: insert record into DB
//TODO: return DB generated PK ID value from SP/query, and set to _pkValue
}
else if (_isDirty)
{
//TODO: update record in DB
}
}
}
Have you had a look at the Entity Framework. I know you said you are wary of LINQ, but EF4 takes care of a lot of the things you mentioned and is a fairly standard practice for DALs.
I would stick with an ORM Tool (EF, OpenAccess by Telerik, etc) unless you need a customized dal that you need (not want) total control over. For side projects I use an ORM - at work however we have our own custom DAL with provider abstractions and with custom mappings between objects and the database.
Nhibernate is also a very solid tried and true ORM with a large community backing it.
Entity Framework is the way to go for your initial DAL, then optimize where you need it: Our company actually did some benchmarking in comparing EF vs SQL reader, and found that for querying the database for one or two tables worth of information, the speed is about 6's (neither being appreciably faster than the other). After two tables there is a performance hit, but its not terribly significant. The one place that writing your own SQL statements became worthwhile was in batch commit operations. At which point EF allows you to directly write the SQL queries. So save your self some time and use EF for the basic heavy lifting, and then use its direct connection for the more complicated operations. (Its the best of both worlds)
Related
My WebApi is working with a lot of internal references between my objects and i'm wondering what would be less costly for the application. I'm using EF database first so i don't have access to the generated classes (I know i can edit them but it's not that smart).
For example, i have some areas where i will have 5 relations, and those relations are deep but i don't want to return them all the time to the user because i won't use all that data, sometimes i just need the parent object and to work that around i'm using AutoMapper and creating some ViewModels where i make a copy of my object.
On some point on my Api that i only want to return some entities i would start the AutoMapper and tell him what he should ignore for that case.
My problem is as i said, i have a lot of data, this system is going to be used for 15k - 20k users. Is the AutoMapper ignoring the data be a bottleneck up ahead ? If so would be better i use some other alternative ?
If this isn't the best option, what else could i use ?
This is an example of how i'm working:
Controller:
public async Task<EventVM> Get(int id)
{
var event = await eventService.Get(id);
return event;
}
Service:
public async Task<EventoVM> Get(int id)
{
var event = await _context.Event.FindAsync(id);
return event;
}
Also i checked on my configuration, Lazy Loading is enabled.
Some of the things in your initial post are not clear at all.
You say you use code first but don't have access to generated classes. Well, if you use code first there won't be generated classes, but you must have some classes initially from which your sql tables get generated, right?
As a rule of thumb, do not use anything from EF in your WebApi. Have your Api return only the data and properties you need for each endpoint. This means creating another set of classes, tipically DTOs which are much lighter, don't have any methods only public properties with exactly the data you need. Yes, you will need an extra step in between to transform the data, but that is absolutely fine.
This should help you get started, just remember the important rule : return exactly what you need, nothing more, nothing less.
I often come to a point that when developing an application that connects to a MS SQL database for basic CRUD functions, I need to iterate through the returned results and populate a list of a particular data type that matches the table design.
Specific to .NET, I would just create a method (e.g. GetAllObjA()) and use ADO.NET to specify a SqlCommand that hooks up to a Stored Proc to fetch the data from the return SqlDataReader. I would then iterate through the returned rows, creating a new object and adding that to a list for each.
If however, I wanted to fetch data for different data type, I would rewrite this for methods GetAllObjB(), GetAllObjC() and so forth with the list's data type being different, which feels like a complete waste of rewriting code.
I realise Generics have a purpose here where the data type can be specified in the one method such as GetAll< T >(). Unfortunately this would still require me to define which table I would be fetching the data from and still require me to match the column names with the member names in the object, which doesn't really solve the problem as the application code has no way of knowing how the table is designed.
Is this the extent of Generics' usefulness in this instance? As the applications I am building are fairly small scale, I don't feel an ORM is warranted if it is possible to hand-code the solution instead.
I agree with the comments that micro-ORM's can either solve your scenario or give you some good ideas. Massive is just plain fun to read because it fits in one file. It uses the dynamic functionality of the Framework to solve your issue. Dapper is oriented towards the mapping aspect.
Also, take a look at the Data Access Application Block. Although this is now Open Source, it was originally maintained by Microsoft. It was an overall enterprise application framework, so there are several bloated dependencies that you do not need. But the data access block has some great prototypes for exactly what you are asking for: mapping a IDataReader resultset to a POCO using generics. So you write the mapping code only once, and the only thing you define per table is the actual reader-to-poco property mapping.
Sometimes a table or its mapping may have some quirks, so you want to keep the mapping definition by hand. For other simple tables with basic primitives, the use of dynamics as demonstrated by Rob Connery's Massive combined with the generic rowset mapper can make for some very easy-to-read-and-maintain code.
Disclaimer: this is not a judgement on the merits of this approach over EF. It is simply suggesting a simple, non-EF approach.
So, you might have a small library defining an interface:
public interface IResultSetMapper<TResult>
{
Task<List<TResult>> MapSetAsync(IDataReader reader);
}
and a generic ADO.Net helper class that processes any reader:
// a method with a class that manages the Command (_Command) and Connection objects
public async Task<List<TOut>> ExecuteReaderAsync<TOut>(IResultSetMapper<TOut> resultsMapper)
{
List<TOut> results = null;
try
{
using(var connection = await _DbContext.CreateOpenedConnectionAsync())
{
_Command.Connection = connection;
// execute the reader and iterate the results; when done, the connection is closed
using(var reader = await _Command.ExecuteReaderAsync())
{
results = await resultsMapper.MapSetAsync(reader);
}
}
return results;
}
catch(Exception cmdEx)
{
// handle or log exception...
throw;
}
}
So, the above code would be a helper library, written once. Then your mapper in your application might look like;
internal class ProductReaderMap : IResultSetMapper<Product>
{
public async Task<List<Product>> MapSetAsync(IDataReader reader)
{
List<Product> results = new List<Product>();
using(reader)
{
results.Add(new Product
{
ProductId = r.GetInt32(0),
ProductName = r.GetString(1),
SupplierId = r.GetInt32(2),
UnitsInStock = r.GetInt16(3)
});
}
return results;
}
}
You could break this out even further, defining a row mapper rather that a row set mapper, since the iteration over the reader can be abstracted as well.
I’ve done some Googling but I have yet to find a solution, or even a definitive answer to my problem.
The problem is simple. I want to dynamically create a table per instance of a dynamically named/created object. Each table would then contain records that are specific to the object. I am aware that this is essentially an anti-pattern but these tables could theoretically become quite large so having all of the data in one table could lead to performance issues.
A more concrete example:
I have a base class/interface ACCOUNT which contains a collection of transactions. For each company that uses my software I create a new concrete version of the class, BOBS_SUB_SHOP_ACCOUNT or SAMS_GARAGE_ACCOUNT, etc. So the identifying value for the class is the class name, not a field within the class.
I am using C# and Fluent nHibernate.
So my questions are:
Does this make sense or do I need to clarify more? (or am I trying
to do something I REALLY shouldn’t?)
Does this pattern have a name?
Does nHibernate support this?
Do you know of any documentation on
the pattern I could read?
Edit: I thought about this a bit more and I realized that I don't REALLY need dynamic objects. All I need is a way to tie objects with some identifier to a table through NHibernate. For example:
//begin - just a brain dump
public class Account
{
public virtual string AccountName { get; set; }
public virtual IList Stuff { get; set; }
}
... somewhere else in code ...
//gets mapped to a table BobsGarageAccount (or something similar)
var BobsGarage = new Account{AccountName="BobsGarage"};
//gets mapped to a table StevesSubShop(or something similar)
var StevesSubShop = new Account{AccountName="StevesSubShop"};
//end
That should suffice for what i need, assuming NHibernate would allow it. I am trying to avoid a situation where one giant table would have the heck beat out of it if high volume occurred on the account tables. If all accounts were in one table... it could be ugly.
Thank you in advance.
Rather than creating a class on the fly, I would recommend a dynamic object. If you implement the right interfaces (one example is here, and in any case you can get there by inheriting from DynamicObject), you can write
dynamic bobsSubShopAccount = new DynamicAccount("BOBS_SUB_SHOP_ACCOUNT");
Console.WriteLine("Balance = {0}", bobsSubShopAccount.Balance);
in your client code. If you use the DLR to implement DynamicAccount, all these calls get intercepted at runtime and passed to your class at runtime. So, you could have the method
public override bool TryGetMember(GetMemberBinder binder, out object result)
{
if (DatabaseConnection.TryGetField(binder.Name, out result))
return true;
// Log the database failure here
result = null;
return false; // The attempt to get the member fails at runtime
}
to read the data from the database using the name of the member requested by client code.
I haven't used NHibernate, so I can't comment with any authority on how NHibernate will play with dynamic objects.
Those classes seem awfully smelly to me, and attempt to solve what amounts to be an actual storage layer issue, not a domain issue. Sharding is the term that you are looking for, essentially.
If you are truly worried about performance of the db, and your loads will be so large, perhaps you might look at partitioning the table instead? Your domain objects could easily handle creating the partition key, and you don't have to do crazy voodoo with NHibernate. This will also more easily permit you to not do nutty domain level things in case you change your persistence mechanisms later. You can create collection filters in your maps, or map readonly objects to a view. The latter option would be a bit smelly in the domain though.
If you absolutely insist on doing some voodoo you might want to look at NHibernate.Shards, it was intended for easy database sharding. I can't say what the current dev state and compatibility is, but it's an option.
I am working on converting a web application over to a WPF desktop application. I want to use the existing data access layer so I don't have to rewrite all the SQL queries.
But, as it stands, everything is populated almost purely from DataTables returned from SQL queries. To make things a bit easier to manage, in some instances, it would really be nice to convert these things into objects.
For example, I have a query that pulls out report information. I may get 500 results with columns like ReportID, ReportTitle, ReportDate.
I would like to make a report class that has these public attributes and somehow convert the SQL query results into a collection of these report objects.
What is the best way to go about doing this?
Super bonus points if there is an easy way of going backwards (updating the database if the objects are changed).
You should learn about Object-Relational Mapping (ORM). A good ORM can save you tonnes of work in the future, and gets those nasty queries out of your code.
I'd recommend NHibernate or Entity Framework 4.0
While I would also like to suggest ORM (NHibernate is the way to go:)) a possible solution is:
public IEnumerable<Report> ToReportList(DataTable dt)
{
return dt.AsEnumerable().Select(dr => new Report
{
member1 = dr["column1"].ToString(),
...
});
}
Report is your class here by the way.Such as,
internal class Report
{
public string member1{ get; set;}
...
}
You may also want to check this,
http://msdn.microsoft.com/en-us/library/Bb399375(v=VS.100).aspx
I think if you search stackoverflow, you will find nicer examples as I remember learning this from here.
By the way, if you use NHibernate, you won't have to rewrite your queries at all. All you have to do is map your tables to a class and booya you are good to go. It will handle all your DML stuff (well mostly) and you can easily tell the ORM to do LazyLoad, Batch processing etc which is pretty cool.
Super bonus points if there is an easy
way of going backwards (updating the
database if the objects are changed).
For that , go for ORM i.e. NHibernate (I know I am biased :)). For LINQ to SQL examples check the 2 links below:
http://msdn.microsoft.com/en-us/library/bb386931.aspx
http://csainty.blogspot.com/2008/01/linq-to-sql-insertupdatedelete.html
+1 ORM. Entity Framework is good, LINQ to SQL is good too, but you'd need a good database design and is better for pretty basic SQL CRUD actions. For custom entities from multiple datasources, I'd go EF.
As far as backwards updating - LINQ to SQL has an easy-peasy implementation, sorta like this - say you've got a db called MyDatabase with Dog entities in it:
using(MyDatabaseDataContext db = new MyDatabaseDataContext())
{
//LINQ will automatically pluralize your items (entity named Dog becomes Dogs)
Dog d = db.Dogs.Where(x=>x.DogName.Equals(dogName));
d.Owner = "Steve";
db.SubmitChanges();
//adding new items is easy too
Dog newdog = new Dog();
newDog.DogName = "Scruff";
newDog.Owner = "Jim";
db.Dogs.Add(newDog);
db.SubmitChanges();
}
Check out this method. First you can create a class that inherits DataContext. And then you can use methods of DataContext like to ExecuteQuery<> to convert your results in to objects. With this method you can directly use your queries you wrote earlier. Also, I feel that this way is a lot better than maintaining a .dbml file, because of the issues associated to synchronization with the actual database schema.
Consider the following example. First you need to define a class for interacting with your database
public class DBManagerDataContext : DataContext
{
private static string connectionString = ""; // Your connection string
public static DBManagerDataContext CreateInstance()
{
return new DBManagerDataContext(connectionString);
}
protected DBManagerDataContext(string connectionString)
: base(connectionString, new AttributeMappingSource())
{
}
}
Then you can use this context to execute queries and convert them in to objects as shown below:
public class Report
{
public int ReportID;
public string ReportTitle;
public DateTime ReportDate;
private static string query = "select ReportID, ReportTitle, ReportDate from dbo.Reports"; // Your query
public static List<Report> GetReportList()
{
DBManagerDataContext context = DBManagerDataContext.CreateInstance();
return context.ExecuteQuery<Report>(query).ToList();
}
}
You can use the method "GetReportList()" given above like this for example:
List<Report> reports = Report.GetReportList();
In the case of updates, the method suggested by "jdandison" is a nice option, apart from using the data context as above. In the case of updates it would be "ExecuteCommand" though. Please explore the DataContext class for more information.
Edit: Please note that the query column names should match the definition in the object
My situation is that I screwed up essentially. I inherited my code base about 1.5 years ago when I took this position and rather than reinventing the wheel, even though I know now that I should have, I kept the DAL in pretty much the same structure as the previous developer.
Essentially there is one file (now at 15k lines of code) that serves as a go between to a bunch of DAO's that use DataSets and TableAdapters to retrieve data. My xsd files have grown to such size that they cause R# to crash visual studio every time it opens and the intermediary class that is now 15k lines also takes forever for R# to analyze. Not to mention it is ugly, it works but not well, and is an absolute nightmare to debug.
What I have tried thus far is switching to NHibernate. NHibernate is a great library, but unfortunately it was not adaptable enough to work with my application, from what the lead developer says (Fabio Maulo) it is pretty much a combination of my application requirements and the restrictions upon NHibernate when using identity as a database PK strategy.
So now I am back to essentially designing my own DAL. I am looking at a few different patterns for this, but would like to get your DAL design strategies. There are so many ways and reasons to implement a DAL in a particular manner so if you could please explain your strategy and why it was best fit for you I would greatly appreciate it.
Thanks in advance!
Edit: Let me explain why NHibernate did not work since that seems to be the immediate response. My users create a "job" that is actually just a transient representation of my Job class. Within this job they will give it one or a list of weight factors that are also transient at the time of creation. Finally they provide a list of job details that have a particular weight factor associated to them. Because, in the DB, weight factors are unique when I go to persist the job and it cascades down to weight factor it dies when it finds a duplicate weight factor. I tried running a check before assigning the weight factor to the detail (which I didn't want to do because I don't want the extra calls to the db) but calling CreateCriteria in NH also causes a flush in the session, according to Fabio, which destroys my cache and thus kills the entire in memory representation of the job. Folks over at the NH mailing list said I should switch over to GUID, but that is not a feasible option as the conversion process would be a nightmare.
My experience with NHibernate is that, while it is packed with features and very high-performance, you will eventually need to become an NHibernate expert in order to fix some unexpected behavior. Reading through the pro-NHibernate answers and seeing
Hmm , perhaps he uses long running
Sessions (Session per Business
Transaction model), and in such an
approach, using identity is
discouraged, since it breaks your
unitofwork (it needs to flush directly
after inserting a new entity). A
solution could be to drop the
identity, and use the HiLo identity
generator.
illustrates exactly what I mean.
What I've done is create a base class modeled somewhat off of the ActiveRecord pattern, that I inherit from and mark up the inherited class with attributes that attach it to a stored procedure each for Select, Insert, Update and Delete. The base class uses Reflection to read the attributes and assign the class's property values to SP parameters, and in the case of Select(), assign the result SQLDataReader's column values to the properties of a list of generics.
This is what DataObjectBase looks like:
interface IDataObjectBase<T>
{
void Delete();
void Insert();
System.Collections.Generic.List<T> Select();
void Update();
}
This is an example of a data class deriving from it:
[StoredProcedure("usp_refund_CustRefundDetailInsert", OperationType.Insert)]
[StoredProcedure("usp_refund_CustRefundDetailSelect", OperationType.Select)]
[StoredProcedure("usp_refund_CustRefundDetailUpdate", OperationType.Update)]
public class RefundDetail : DataObjectBase<RefundDetail>
{
[StoredProcedureParameter(null, OperationType.Update, ParameterDirection.Input)]
[StoredProcedureParameter(null, OperationType.Insert, ParameterDirection.Output)]
[StoredProcedureParameter(null, OperationType.Select, ParameterDirection.Input)]
[ResultColumn(null)]
public int? RefundDetailId
{ get; set; }
[StoredProcedureParameter(null, OperationType.Update, ParameterDirection.Input)]
[StoredProcedureParameter(null, OperationType.Insert, ParameterDirection.Input)]
[StoredProcedureParameter(null, OperationType.Select, ParameterDirection.Input)]
[ResultColumn(null)]
public int? RefundId
{ get; set; }
[StoredProcedureParameter(null, OperationType.Update, ParameterDirection.Input)]
[StoredProcedureParameter(null, OperationType.Insert, ParameterDirection.Input)]
[ResultColumn(null)]
public int RefundTypeId
{ get; set; }
[StoredProcedureParameter(null, OperationType.Update, ParameterDirection.Input)]
[StoredProcedureParameter(null, OperationType.Insert, ParameterDirection.Input)]
[ResultColumn(null)]
public decimal? RefundAmount
{ get; set; }
[StoredProcedureParameter(null, OperationType.Update, ParameterDirection.Input)]
[StoredProcedureParameter(null, OperationType.Insert, ParameterDirection.Input)]
[ResultColumn(null)]
public string ARTranId
{ get; set; }
}
I know it seems like I'm reinventing the wheel, but all of the libraries I found either had too much dependence on other libraries (ActiveRecord + NHibernate, for instance, which was a close second) or were too complicated to use and administer.
The library I made is very lightweight (maybe a couple of hundred lines of C#) and doesn't do anything more than assign values to parameters and execute the SP. It also lends itself very well to code generation, so eventually I expect to write no data access code. I also like that it uses a class instance instead of a static class, so that I can pass data to queries without some awkward criteria collection or HQL. Select() means "get more like me".
For me the best fit was a pretty simple concept - use DAO class definitions and with reflection create all SQL necessary to populate and save them. This way there is no mapping file, only simple classes. My DAO's require an Entity base class so it is not a POCO but that doesn't bother me. It does support any type of primary key, be it single identity column or multi column.
If your DAL is written to an interface, it would be much easier to switch to NHibernate or something comperable (I would prefer Fluent-NHibernate, but I digress). So why not spend the time instead refactoring the DAL to use an interface, and then write a new implementation using NH or your ORM of choice?
In recent projects we have stopped programming a separate DAL.
Instead we use an Object Relational Mapper (in our case Entity Framework). We then let the business layer program directly against the ORM.
This has saved us over 90% of development effort in some cases.
My first step would be to break the code out of a 15 KLOC monster, then come up with a strategy for creating a new DAL.
Linq to SQL is nice if you are using SQL Server. There is source out there for a LinqToSQL provider to Access and MySQL. I haven't tested it though. LinqToSql follows the UnitOfWork model which is similar to the way ADO.NET functions. You make a series of changes to a local copy of the data then commit all the changes with one update call. It's pretty clean I think.
You can also extend the DataRow class yourself to provide strongly typed access to your fields. I used XSLT to generate the DataRow descendants based on the metadata of each table. I have a generic DataTable decendant. MyDataTable where T is my derived row. I know that MS's strongly-typed datasets do a similar thing but I wanted a light-weight generic version that I complete control of. Once you have this, you can write static access methods that query the db and fill the DataTable.
You would be in charge of writing the changes from the DataTable back to the DataSource. I would write a generic class or method that creates the update,inserts and deletes.
Good Luck!
I use mine wrapper for SPs for the fastest data retrieving and L2S when perfomance is not a goal. My DAL uses repository pattern and encapsulated logic for TDD.