I have this relatively big database for my website, currently in a MySQL database. As the site has been based on "one size fits all" solutions so far, the recent explosive growth of the website forced me to start developing a more fitting, more optimized and less fragmented platform.
Currently working on the back-end, for which an ASP.NET API suits me the best. I've decided to go with MySQL and use the Entity Framework to set things up. While it wasn't too much of an issue, I want to migrate all the data into a new, cleaner and better set up database. The issue is the users themselves - while I think ASP's Membership might be the best solution for me, I'm not sure how to migrate all my users' needed data to the new database, or even if Memberships are actually the best way to go about this. And this is what I need help in - could someone help me how to set up the API for this?
Edit: I've designed the new database inside Visual Studio with the EF Designer, but considering all the material I find, I suppose that's an issue? >_>
Related
I have been using SQL since 1985, so am very comfortable with DB servers.
I see (C#) Code First as yet another fad, that comes and goes. It seems to suit people that have no DBA background. Equally if using Code First and you have not idea what DB you are connecting to eg it might be Mongo later that too is a useful abstraction. Code First does not let itself to Database Diagrams so you can see what is going on.
I would like to know how you promote changes into a production SQL server using code first, where you have no desire to Drop and recreate the DB, unlike using an ALTER TABLE command. I have used tools from Red-Gate that make DB code promotions easy.
So why Code First?
How do you move DB Changes into production?
The code first approach is something that is really nice to have when you're prototyping applications and "don't care" about the "persistence layer" too much in the beginning. It helps getting quick results in a time when things are not at all well defined yet, because you can always very easily drop and re-create your database of your greenfield project.
Unfortunately, most tools that offer this approach do not really help transitioning from the "greenfield project mode" to the more longterm "legacy project mode" where database migrations are essential. In fact, not only are database migrations difficult to achieve in "client model first" approaches, but more importantly, the likelihood of the database model not being well designed is quite high.
Developers pay a high price for the quick win - as always. It is really not a good idea and a lot of more experienced developers who are used to working with legacy, do agree with you: Go database first, and derive your client (e.g. ORM) models from it.
I have written about this topic here, including the advantage of using client model source code generation.
I have a product that I'm currently authoring that relies on SQL server for the backend. One issue I'm trying to resolve is to improve the 'upgrade' story. So v1 will have a particular schema and v2 may include some enhancements to this schema (new tables and new columns).
I'm aware of the SDKs from RedGate and ApexSQL - but would like to avoid.
I've had a read through the SMO docs, but I'm new to it and struggling to see if this can be applied in this situtaiton. Ideally, I'd like this to make this programatic (SMO or other) - the base cases seems straight forward enough, but I really don't want to re-invent the wheel if I can help it. Does anyone have any experience of similar requirements or ideas about how I could approach?
You don't say what version of SQL Server you're using but in (I think) 2005 and beyond, there is the concept of database triggers. These work like their table level cousins but can be used to track any kind of DDL change that happens on the database. We didn't use it to actually generate DDL - more to track when the format of a table changed. Although what you're after should be possible I'd have thought.
Triggers are one of those things that divide developers. Some people think they're the best thing since sliced bread whilst others hate them with a passion. Perhaps because when data changes, these are the last thing you think of.
Maybe not exactly what your're looking for (since it's not SMO) but having a look at Entity Framework Code First Migrations might help you:
http://msdn.microsoft.com/en-us/data/jj591621
Changes in the model-classes can be versioned and can either be applied directly to a database or, if you do not have direct access to your database, you can generate SQL-Code for your new version and hand it to your database-administrator.
I us Database Projects in Visual Studio to mange versioning of schemas. Once you create a baseline in a Database Project, you can make your changes in the project and then use the Schema Compare to create SQL scripts to apply the changes in different environments.
I would recommend doing only additive changes, but it will generate change scripts for destructive changes. If you do not have your environments synced up, I strongly recommend generating a new script for each environment.
This blog post goes over how to create one in Visual Studio 2012: http://candordeveloper.com/2013/01/08/creating-a-sql-server-database-project-in-visual-studio-2012/
Red Gate has a schema compare product too, but I have not really used it.
I recently started a new personal project to learn Entity Framework. My end goal is to make a desktop game that uses SQL compact for data management and uses Entity Framework for the game objects. Not actually knowing there were multiple ways to start EF (model first, code first, db first) I went with the most obvious choice of model first.
I've been working with it successfully now, however one thing concerns me, especially post-development. My goal with the game is that users can update to the latest version without losing any of their existing data. The current issue is that all the generation scripts are destructive by nature (dropping everything then recreating it) - that means I can't run those against the user SQLCE DBs out in "production", so I need to come up with an alternative plan of action.
That said, does anyone have recommended solutions on best practices? In previous desktop apps, I've traditionally used XML/binary to store data, which allows me to easily update the "schema" without affecting existing data (versioning in the app tailors the Load() according to the version, while the Save() always saves in the latest version).
What are some recommendations on handling this problem using SQLCE?
What you need, if understood right, is to utilize migrations which come with EF. Since the question is general this link should best guide you to what you need I think...
http://blogs.msdn.com/b/adonet/archive/2012/02/09/ef-4-3-code-based-migrations-walkthrough.aspx
With migrations which you can tailor manually if needed (and come in the shape of code which is applied at each point of change, incrementally) and you can also supply your 'seeding' if required.
i.e. you should be able to do most of what you require, delete, remove old incompatible data - and seed the new one that you have - and all related to a particular migration step you have.
How would that work with your app deployment specifically, that's a bit more complex I guess, but this should get you started, and then with each db version-breaking change your new code update would contain all the migrations since the previous update (or just one usually is enough, i.e. make it be one with each update) and the code to tear-down or create new things.
hope this helps,
I have worked with Microsoft Access as the back-end of my applications in the past and Visual Studio offers me the choice of copying the database to the installation of the executable that I have created. However, I now want to move onto more complex databases and I figured MySQL was a good start because it's free and popular. I know there may be better options and right now I'm currently only in learning stages so I strictly want to stick with MySQL.
My problem is that I have my MySQL running on my localhost. I have connected to it, ran queries, etc. Now if I wanted to deploy this application to other computers while keeping the database (not web-based) how would I go about doing that? The reason I don't want to go web yet is because I just want to get an understanding without dealing with networking yet. I figured this would be the way to go.
Thank you.
MySQL is not primarily anything. It's a full database as is Oracle, SQLServer, Postgre, etc. that can be used for any application that you feel it applies to.
In your case what you really want is SQLite for "embedded" database needs. The database is represented by a single file that can be opened and queried very similarly to MySQL.
http://www.sqlite.org/
To access the database from your C# code there are many libraries available to you. Here is one I used a while back as an example:
http://www.devart.com/dotconnect/sqlite/features.html
To play around with the data, as you would with MySQL Workbench, there are many front-ends. As an example there's a pretty good firefox addon for this:
https://addons.mozilla.org/en-US/firefox/addon/sqlite-manager/
And don't worry, it's extremely easy to use and most of the query syntax will apply to MySQL as well!
(EDIT: I made it a community wiki as it is more suited to a collaborative format.)
There are a plethora of ways to access SQL Server and other databases from .NET. All have their pros and cons and it will never be a simple question of which is "best" - the answer will always be "it depends".
However, I am looking for a comparison at a high level of the different approaches and frameworks in the context of different levels of systems. For example, I would imagine that for a quick-and-dirty Web 2.0 application the answer would be very different from an in-house Enterprise-level CRUD application.
I am aware that there are numerous questions on Stack Overflow dealing with subsets of this question, but I think it would be useful to try to build a summary comparison. I will endeavour to update the question with corrections and clarifications as we go.
So far, this is my understanding at a high level - but I am sure it is wrong...
I am primarily focusing on the Microsoft approaches to keep this focused.
ADO.NET Entity Framework
Database agnostic
Good because it allows swapping backends in and out
Bad because it can hit performance and database vendors are not too happy about it
Seems to be MS's preferred route for the future
Complicated to learn (though, see 267357)
It is accessed through LINQ to Entities so provides ORM, thus allowing abstraction in your code
LINQ to SQL
Uncertain future (see Is LINQ to SQL truly dead?)
Easy to learn (?)
Only works with MS SQL Server
See also Pros and cons of LINQ
"Standard" ADO.NET
No ORM
No abstraction so you are back to "roll your own" and play with dynamically generated SQL
Direct access, allows potentially better performance
This ties in to the age-old debate of whether to focus on objects or relational data, to which the answer of course is "it depends on where the bulk of the work is" and since that is an unanswerable question hopefully we don't have to go in to that too much. IMHO, if your application is primarily manipulating large amounts of data, it does not make sense to abstract it too much into objects in the front-end code, you are better off using stored procedures and dynamic SQL to do as much of the work as possible on the back-end. Whereas, if you primarily have user interaction which causes database interaction at the level of tens or hundreds of rows then ORM makes complete sense. So, I guess my argument for good old-fashioned ADO.NET would be in the case where you manipulate and modify large datasets, in which case you will benefit from the direct access to the backend.
Another case, of course, is where you have to access a legacy database that is already guarded by stored procedures.
ASP.NET Data Source Controls
Are these something altogether different or just a layer over standard ADO.NET?
- Would you really use these if you had a DAL or if you implemented LINQ or Entities?
NHibernate
Seems to be a very powerful and powerful ORM?
Open source
Some other relevant links;
NHibernate or LINQ to SQL
Entity Framework vs LINQ to SQL
I think LINQ to SQL is good for projects targeted for SQL Server.
ADO.NET Entity Framework is better if we are targeting different databases. Currently I think a lot of providers are available for ADO.NET Entity Framework, Provider for PostgreSQL, MySQL, esql, Oracle and many other (check http://blogs.msdn.com/adonet/default.aspx).
I don't want to use standard ADO.NET anymore because it's a waste of time. I always go for ORM.
Having worked on 20+ different C#/ASP.NET projects I always end up using NHibernate. I often start with a completely different stack - ADO.NET, ActiveRecord, hand rolled wierdness. There are numerous reasons why NHibernate can work in a wide range of situations, but the absolutely stand out for me is the saving in time, especially when linked to code generation. You can change the datamodel, and the entities get rebuilt, but most/all the other code doesn't need to be changed.
MS does have a nasty habit of pushing technologies in this area that parallel existing open source, and then dropping them when they don't take off. Does anyone remember ObjectSpaces?
Added for new technologies:
With Microsoft Sql Server out for Linux in Beta right now, I think it's ok to not be database agnostic. The .Net Core Path and MS-SQL route allows you to run on Linux servers like Ubuntu entirely with no windows dependencies.
As such, imo, a very good flow is to not use a full ORM framework or data controls and leverage the power of SSDT Visual Studio Projects (Sql Server Data Tools) and a Micro ORM.
In Visual Studio you can create a Sql Server Project as a legit Visual Studio Project. Doing so allows you to create the entire database via table designers or raw query editing right inside visual studio.
Secondly, you get SSDT's Schema Compare tool which you can use to compare your database project to a live database in Microsoft Sql Server and update it. You can sync your Visual Studio Project with the server causing updates in your project to go out to the server. Or you can sync the server with your project causing your source code to update. Via this route you can easily pick up changes the DBA made in maintenance last night and push out your new development changes for a new feature easily with a simple tool.
Using that same tool you can compute the migration script without actually running it, if you need to pass that off to an operations department and submit a change order, it works for that flow to.
Now for writing code against you MS-SQL Database, I recommend PetaPoco.
Because PetaPoco works Perfectly inline with the above SSDT solution. PetaPoco comes with T4 text templates you can use to generate all your data entity classes, and it generates the bulk data layer classes for you.
The catch is, you have to write queries yourself, which isn't a bad thing.
So you end up with something like this:
var people = dbContext.Fetch<Person>("SELECT * FROM People where Username Like '%#0%'", "bob");
PetaPoco automatically handles parameterizing #0 for you, it also has the handy Sql class for building queries.
Furthermore, PetaPoco is an order of magnitude faster than EF6 and 8+ times faster than EF7.
So in total, this solution involves using SSDT for SCHEMA management, and PetaPoco for code integration at the gain of high maintainability, customization, and very good performance.
The only downfall to this approach, is that you're hard tieing yourself to Microsoft Sql Server. However, imo, Microsoft Sql Server is one of the best RDBM's out there.
It's got DBMail, Jobs, CLR object capabilities, and on and on. Plus the integration between Visual Studio and MS-SQL server is phenomenal and you don't get any of that if you choose a different RDBMS.
I must say that I never used NHibernate for the immense time that needed to start using... time wasted on the XML setup.
I recently did a web application in MVC2, where I did choose ADO Entities Framework and I use Linq all the time.
I must say, I was impressed with the speed! and our site was having around 35 000 unique visitors per day, in around 60Gb bandwidth per day (I reduced radically this 60Gb number by hosting all static files in Amazon S3 - Great .NET wrapper they have, I must say).
I will always go this way. It's easy to start (just add new data item, choose tables and that's it! for every change in the database we just need to refresh the model - made automatically in just 2 clicks) and it's fun to use - Linq rules!