I need to write a specialized, embedded database for some .net Apps. The database part itself is no problem (as said, it's specialized), but the access to it from my .net code is.
I was thinking of using LINQ as the only data access (so no ADO.net, which is fine for the scenario), but I've never done that and I wonder what I actually need to implement?
Do I need IQueryable? Is there a set of Standard methods/Interfaces I need to implement? Is there a standard for writing data or do I need to create my own semantics for it?
Do I need IQueryable?
Yes, this is where LINQ hooks up.
Is there a standard for writing data
or do I need to create my own
semantics for it?
Your own. OTOH that is normally the easier part - you normally dout do a JOIN or something when you update a row.
I personally use BLToolkit quite often these days (need something fast and leightweight). They have a SqlQuery object that can manipualte classes to tables. GOod enough for most direct db manipulation (and I dont use change tracking - most of my objets never change but are versioned in the database).
LINQ providers are quite complicated. As is writing an execution engine. Sure you want to go that way?
Writing your own linq provider is very difficult - I looked into it a while back and got discouraged... may still do it though.
See the answers here - a couple of tutorials are linked to.
Related
I have a few very large queries which I need to convert it linq because we are using Entity framework and I cant use stored procedures(breaks compatibility with other data bases).
using tool like linqer didnt even help and even if I get it to work with some mods to generated linq, there is a huge performance issue.
so, what is the best option in a situation like this where EF fails?
please don't ask me to divide it into small queries cause that's not possible.
Moving this to an "answer" because what I want to say is too long for a comment.
It sounds like you're running into an inherent limitation to ORMs. You won't get perfect performance trying to do everything in code. It sounds like you're trying to use an ORM like a T-SQL interface rather than a mapping between objects and a relational instance of data.
You say you want to maintain compatibility between databases but that's already a nonstarter if you consider schema differences from database to database. If you're already implementing a schema validation step so you ensure your code doesn't break, then there should be no reason why you can't use something like views.
You can say you don't want to support these things all day long but the simple point is that these things exist because they address certain problems. If you wholesale abandon them, then you can't really expect to get rid of the problem. Some things the database simply does better.
So, I think you're expecting something out of the technology that it wasn't meant to solve. You'll need to either reevaluate your strategy or use another tool to accomplish it. I think you may even need a couple different tools.
What you've been doing may have worked when your scale was smaller. I could see such a thing working for quite a while actually. However, it does have a scale limit, and I think you're coming up against it.
I think you need to make a determination on what databases you want to support. Saying "we support all databases" is untenable. Then, compare features and use the ones in common. If it's a MS SQL vs. MySQL thing, then there's no reason why you can't use views or stored procedures.
Check out LinqKit - this is a very useful tool for building up complex large EF queries.
http://www.albahari.com/nutshell/linqkit.aspx
Can i use a dynamic C# object to access database values. For example if i have a table name Settings, with different values could i simply call from code Settings.SomeSetting, rather then programming them all.
You could also use a C# ORM in order to do something like what you are trying to do.
Dapper.Net - A simple C# ORM.
NHibernate - A mature ORM which has been around while.
I don't have experience with either but both sound like something you are trying to achieve.
Here's a blog post showing one technique using a dynamic wrapper for ADO.NET data access. This should work against a query run against an Access database (as well as any other ADO.NET data source).
Yes, you can.
Probably you will get insane later, but you can. Also, if you want to access the data in a reflective way and without compilation check, you could just use DataSets. They suck, but... You are about to reinvent the weel.
Have a look at Simple.Data. It sounds like what you're looking for.
I'm building a program which has two options: an offline and an online storage option. The online stuff isn't a problem, never has been and probably never will be. To make things a little more simple, I am using the Entity Framework with my online database to do all of my CRUD.
The offline option is where I'm having a problem. I tried adding a local SQL install as part of the install package, but it was failing on certain computers depending upon which version of SQL I was using, some people just couldn't install it at all... honestly, I don't want to be going through that just to get people to install the program. I want to do two things:
1) Store the data. I'm currently thinking Linq to XML is my best bet with nothing extra to install, but is there any way to map the entities to classes within the program that would take care of the XML CRUD?
2) Sync the data. I'm thinking I'll use the Sync Framework, but again, with no normal database in the background, it's a bit of a pain.
Any help/advice/comments very welcome, if there's a much easier way of doing all this, please let me know.
I'm glad to hear you've got something figured out for the syncing logic. That's the bit that would keep me up at night. GUIDs are your friend in a situation like this.
And I've never attempted to use 1 ObjectContext targeted at different datasources. Maybe you can pull off something slick using OLEDB as your provider. I'd go a little down that road. But I suspect you'll find it's more trouble than it's worth.
I recommend following the Repository Pattern. Then you can just implement a XMLRepository and a SQLRepository. This may mean some borderline double coding. But queries against XML versus a database have their own concerns. So, depending on your specifics, I think violating the DRY principle may be justified.
Also, and I acknowledge this is completely subjective, it strikes me that the Entity Framework is overkill for a relational model that can be easily persisted in XML. I'm just curious if you considered using LINQ-to-SQL. And if so, what convinced you to go the way of EF?
I've been taking a look at some different products for .NET which propose to speed up development time by providing a way for business objects to map seamlessly to an automatically generated database. I've never had a problem writing a data access layer, but I'm wondering if this type of product will really save the time it claims. I also worry that I will be giving up too much control over the database and make it harder to track down any data level problems. Do these type of products make it better or worse in the already tough case that the database and business object structure must change?
For example:
Object Relation Mapping from Dev Express
In essence, is it worth it? Will I save "THAT" much time, effort, and future bugs?
I have used SubSonic and EntitySpaces. Once you get the hang of them, I beleive they can save you time, but as complexity of your app and volume of data grow, you may outgrow these tools. You start to lose time trying to figure out if something like a performance issue is related to the ORM or to your code. So, to answer your question, I think it depends. I tend to agree with Eric on this, high volume enterprise apps are not a good place for general purpose ORMs, but in standard fare smaller CRUD type apps, you might see some saved time.
I've found iBatis from the Apache group to be an excellent solution to this problem. My team is currently using iBatis to map all of our calls from Java to our MySQL backend. It's been a huge benefit as it's easy to manage all of our SQL queries and procedures because they're all located in XML files, not in our code. Separating SQL from your code, no matter what the language, is a great help.
Additionally, iBatis allows you to write your own data mappers to map data to and from your objects to the DB. We wanted this flexibility, as opposed to a Hibernate type solution that does everything for you, but also (IMO) limits your ability to perform complex queries.
There is a .NET version of iBatis as well.
I've recently set up ActiveRecord from the Castle Project for an app. It was pretty easy to get going. After creating a new app with it, I even used MyGeneration to script out class files for a legacy app that ActiveRecord could use in a pretty short time. It uses NHibernate to interact with the database, but takes away all the xml mapping that comes with NHibernate. The nice thing is though, if necessary, you already have NHibernate in your project, you can use its full power if you have some special cases. I'd suggest taking a look at it.
There are lots of choices of ORMs. Linq to Sql, nHibernate. For pure object databases there is db4o.
It depends on the application, but for a high volume enterprise application, I would not go this route. You need more control of your data.
I was discussing this with a friend over the weekend and it seems like the gains you make on ease of storage are lost if you need to be able to query the database outside of the application. My understanding is that these databases work by storing your object data in a de-normalized fashion. This makes it fast to retrieve entire sets of objects, but if you need to select data from a perspective that doesn't match your object model, the odbms might have a hard time getting at the particular data you want.
Currently I am using NetTiers to generate my data access layer and service layer. I have been using NetTiers for over 2 years and have found it to be very useful. At some point I need to look at LINQ so my questions are...
Has anyone else gone from NetTiers to LINQ To SQL?
Was this switch over a good or bad thing?
Is there anything that I should be aware of?
Would you recommend this switch?
Basically I would welcome any thoughts
.
No
See #1
You should beware of standard abstraction overhead. Also it's very SQL Server based in it's current state.
Are you using SQL Server, then maybe. If you are using LINQ for other things right now like over XML data (great), Object data, Datasets, then yes you should could switch to have a uniform data syntax for all of them. Like lagerdalek mentioned if it ain't broke don't fix it.
From the quick look at .netTiers Application Framework, I'd say if you already have an investment with that solution it seems to give you much more than a simple Data Access Layer and you should stick with it.
From my experience LINQ to SQL is a good solution for small-medium sized projects. It is an ORM which is a great way to enhance productivity. It also should give you another layer of abstraction that will allow you to change out the layer underneath for something else. The designer in Visual Studio (and I belive VS Express also) is very easy and simple to use. It gives you the common drag-drop and property-based editing of the object mappings.
# Jason Jackson - The Designer does let you add properties by hand, however you need to specify the attributes for that property, but you do this once, it might take 3 minutes longer than the initial dragging of the table into the designer, however it is only necessary once per change in the database itself. This is not too different from other ORMs, however you are correct that they could make this much easier, and find only those properties that have changed, or even implement some kind of refactoring tool for such needs.
Resources:
Why use LINQ to SQL?
Scott Guthrie on LINQ to SQL
10 Tips to Improve your LINQ to SQL Application Performance
LINQ To SQL and Visual Studio 2008 Performance Update
Performance Comparisons LINQ to SQL / ADO / C#
LINQ to SQL 5 Minute Overview
Note that Parallel LINQ is being developed to allow for much greater performance on multi-core machines.
I tried to use Linq to SQL on a small project, thinking that I wanted something I could generate quickly. I ran into a lot of problems in the designer. For example, anytime you need to add a column to a table you basically have to remove and re-add the table definition in the designer. If you have set any properties on the table then you have to re-set those properties. For me this really slowed down the development process.
LINQ to SQL itself is nice. I really like the extensibility. If they can improve the designer I might try it again. I think that the framework would benefit from a little more functionality aimed at a disconnected model like web development.
Check out Scott Guthrie's LINQ to SQL series of blog posts for some great examples of how to use it.
NetTiers is very good for generating a heavy and robust DAL, and we use it internally for core libraries and frameworks.
As I see it, LINQ (in all its incarnations, but specifically as I think you're asking to SQL) is fantastic for quick data access, and we generally use it for more agile cases.
Both technologies are quite inflexible to change without regeneration of the code or dbml layer.
That being said, used properly LINQ 2 SQL is quite a robust solution, and you might even start using it for future development due to it's ease of use, but I wouldn't throw away your current DAL for it - if it aint broke ...
My experience tells me that using by using linq you can get things done faster, however the actual actions to the database are slower.
So... if you have a small database, i'll say go for it. If not, i would wait for some improvements before changing
I'm using LINQ to SQL on fairly large project right now (about 150 tables) and it is working out very well for me. The last ORM I used was IBatis and it worked well but took alot of legwork to get your mappings done. LINQ to SQL performs very well for me and so far has proved to be very easy to use out of the box. There are definately some differences you have to overcome in transition, but I would recommend it's use.
Side note, I have never used or read about NetTiers so I won't discount it's effectiveness, but LINQ to SQL in general has proven to be an extremely viable ORM.
Our team used to use NetTiers and found it to be useful. BUT... the more we used it, the more we found headaches and pain points with it. For example, anytime you make a change to the database, you need to re-generate the DAL with CodeSmith which involved:
re-generating thousands of lines of code in 3 separate projects
re-generating hundreds of stored procedures
Maybe there are other ways of doing it, but this is what we had to do. The re-gen of the source code was ok, scary, but ok. The real issue came with the stored procedures. It didn't clean any unused stored procedures so if you removed a table from your schema and re-gened your DAL, the stored procedures for that table did not get removed. Also, this became quite a headache for database change scripts where we had to compare the old database structure to the new one and create a change script to update client installations. This script could run into the tens of thousands of lines of sql code and if there was an issue executing it, which there invariably was, it was quite a pain to resolve it.
Then the light came on, NHibernate as an ORM. It certainly has a ramp-up time to it but it is well worth it. There is a ton of support for it so if there's something you need done, more than likely it's been done before. It is extremely flexible and allows you to control every aspect of it and then some. It is also becoming easier and easier to use. Fluent Nhibernate is up and coming as a great way to get rid of the xml mapping files that are needed and NHibernate Profiler provides an excellent interface to see what's going on behind the scenes to increase efficiency and remove redundancy.
Moving from NetTiers to NHibernate has been painful, but in a good way. It has forced us to move into a better architecture and re-evaluate functional needs. NetTiers provided tons of data access code, get this entity by its id, get this other entity by its foreign key, get a tlist and vlist of this and that, but most of it was unnecessary and unused. NHibernate with a generic repository and custom repositories only where needed reduced tons of unused code and really increased readability and reliability.