We are building this .NET application using Entity Framework as our DB connector. I know all about the stuff of picking the right approach based on your circumstances like "do you have an existing database?", "do you prefer modelling instead of coding". But after some reading I've found that it's not the only thing to think of as the upgrade process of the database when it's already in production is really important, espacially for us.
So which approach is best for production use with Entity Framework. For the moment we have an existing database. I prefer to use the model and update the database from it but then we have lack of functionality in default values of columns and the model can be hard to work with in teams so what we need is basically some best practice here.
For production use: Database First, Model First or Code First?
Someone else might chime in here and tell you to use model migrations with the code first approach. That may be a solution, it's just not my preference.
We manage an in-motion database using EF code first, however I would not be able to do it without one hugely beneficial Visual Studio feature: SQL Schema Compare. I believe this feature is only available in the Premium and Ultimate versions of the product.
Each time our model changes, I put 2 copies of the database schema on my local machine: the new version, and the current production version. If you run Schema Compare using the new version as the source and the production version as the target, it will generate a SQL script that you can run against your production db to bring its schema and data in line with the changes.
The SQL it generates often needs some editing before it can be run in production, but it will do a lot of the hard work for you -- disabling constraints, add / drop indexes, and moving data from an old table into a new version of it. It will also warn you of potential issues when changing the schema.
Related
According to this blog post most companies using EF Migrations are supposedly not updating the database schema of production databases with EF migrations. Instead the blog post's author recommends to use Schema update scripts as part of the deployment process.
I've used Schema update scripts for a few years now and while they work, I was planning to use EF migrations instead in the future for the following reasons:
Faster deployment, less downtime
A simpler deployment procedure
Much easier migration of existing data than it would be possible with T-SQL
A more comprehensible syntax of the changes waiting to be applied (DbMigration class with clean C# syntax vs. clunky T-SQL Migration script in a traditional environment).
There is an easy and fast downgrade path to the old db schema if the deployment of the new software version should fail
One reason I can think of that would prohibit the use of EF to migrate a production DB would be if the DB schema was only altered by the DBAs as opposed to the Developers. However, I am both DBA and Developer, so this does not matter in my case.
So, what are the risks of updating a production database using EF?
Edit: I would like to add that, as solomon8718 already suggested, I am always pulling a fresh copy of the production database to my staging server and test the EF Migrations to be applied on the staging server before applying them to a production server. IMO this is essential for any schema update to a production system, whether I'm using EF migrations or not.
Well, I'll try and answer anyhow. I would say No, there's no reason not to use Code First Migrations in production. After all, what's the point of this easy to use system if you can't take it all the way?
The biggest problems I see with it are all problems that you can have with any system, which you've noted already. As long as the whole team (DBA included if applicable) is on board with it, I think allowing EF to manage the schema through migrations is less complex, and hence less error-prone than traditional script-based management. I would still take a backup before performing a migration on a production system, but then you'd do that anyhow.
There's nothing that says a DBA can't perform a migration from Visual Studio, either. The access could still be locked down with privileges at the database level, and he/she could review the migration (in a helpful SQL export format using -Script, if desired) before performing the actual operation. Then they're still in control, but you can use code-first migrations. Hell, they might even end up liking it!
Update: since SPROCs and TVFs were brought up, we handle those in migrations as well, although they are actually done with straight-up SQL statements using a DbMigration.Sql() call in the Up(), and the reverse of them in the Down() (You can also use CreateStoredProcedure and DropStoredProcedure for simple SPROCs, but I think you still have to define the body itself in SQL). I guess you could say that's a caveat; there isn't yet a way for an entire, comprehensive database to be written purely in C#. However, you can use migrations which include SQL scripts to manage the entire schema. One benefit we've found from this process is you can use the C# config file for schema object names (different server names for production vs dev for example) with a simple String.Format, combined with XML Transformation for the config files themselves.
Yes there are good reasons not to use an automated system such as Code First Migrations to make production database changes. But as always there are exceptions to the rules.
One reason which has been mentioned would be access permissions, which would be directly related to your organization's change management rules and security policies.
Another reason would be your level of trust in the Migrations tool itself. Are we sure the tool doesn't have a bug in it? What happens if the tool fails midway through? Are you certain you have up-to-date backups and a process to roll-back if need be?
The change scripts may execute unexpected or inefficient scripts. I've experienced cases where the sql generated copied the data into a temp table, dropped the original table, then recreated the original table for things like adding a new column if you accidentally (or purposefully) change the order in which the column appears, or when you rename the table. If millions of records are involved this could cause serious performance issues.
My recomendation:
Assuming you have a Staging database that mirrors your production schema, use the Migrations tool to generate its change scripts against that system. We usually restore our stage database from a fresh production copy before running. We then examine the change scripts manually to check for issues. After that we run the scripts against our stage database to make sure it executes properly and that all the changes expected took place. Now we are sure that the scripts are both safe to run in production and perform the expected changes. This process would address all three issues I listed above.
One other caveat I found: If you have several websites using the same data context, you need to make sure that all of them are updated at the same time. Otherwise there might be a constant database update / downgrade fight between the websites. Other than that, it worked fine for me.
EDIT: My own perspective one year after starting to use EF Migrations in production:
EF Migrations is actually pretty cool, even for production use, provided that you
Test the migrations on a staging system. I test all migrations by migrating all the way down and up again on my CI server before running integration tests.
Do not trigger migrations automatically, but with a batch file that is launched by an admin. This is essentially the same as running the sql for a migration manually in SSMS.
I use it in production for a couple of projects. Once you get the hang of it I think it's fine.
During development you can keep auto migrations on but at the end you can connect to the live db right from package manager console and generate a migration. It will give you one migration for all the changes.
But always always always use the -script option with update-database and fire the SQL yourself.
I would also advice not using the update db option from web deploy. That way there is no way to tell how much of the migration has already been fired on error. I've ran into trouble with that a few times. So best to get the SQL and fire it manually.
I have a solution which uses Entity Framework model first approach.
The problem I am facing is that whenever I change something on a table, add a column or change a relationship,I right click and go for "Generate Database from Model", which re-generates ALL the code for the solution even if I just changed one table..and that generated code is useless for a production database since it drops every table and then re creates them..
I am wondering, isnt there be an option just to generate the T-SQL with the changes I made ? Otherwise model first would be useless after your app goes into prod.
I am using entity framework 5.0
Personally, I would suggest you to use Red-Gate SQL Compare when you need to sync your databases at Production environment.
This tool helps you to compare and synchronize databases using sync scripts without losing data (it will alert about if so) and its UI is just awesome.
I have a product that I'm currently authoring that relies on SQL server for the backend. One issue I'm trying to resolve is to improve the 'upgrade' story. So v1 will have a particular schema and v2 may include some enhancements to this schema (new tables and new columns).
I'm aware of the SDKs from RedGate and ApexSQL - but would like to avoid.
I've had a read through the SMO docs, but I'm new to it and struggling to see if this can be applied in this situtaiton. Ideally, I'd like this to make this programatic (SMO or other) - the base cases seems straight forward enough, but I really don't want to re-invent the wheel if I can help it. Does anyone have any experience of similar requirements or ideas about how I could approach?
You don't say what version of SQL Server you're using but in (I think) 2005 and beyond, there is the concept of database triggers. These work like their table level cousins but can be used to track any kind of DDL change that happens on the database. We didn't use it to actually generate DDL - more to track when the format of a table changed. Although what you're after should be possible I'd have thought.
Triggers are one of those things that divide developers. Some people think they're the best thing since sliced bread whilst others hate them with a passion. Perhaps because when data changes, these are the last thing you think of.
Maybe not exactly what your're looking for (since it's not SMO) but having a look at Entity Framework Code First Migrations might help you:
http://msdn.microsoft.com/en-us/data/jj591621
Changes in the model-classes can be versioned and can either be applied directly to a database or, if you do not have direct access to your database, you can generate SQL-Code for your new version and hand it to your database-administrator.
I us Database Projects in Visual Studio to mange versioning of schemas. Once you create a baseline in a Database Project, you can make your changes in the project and then use the Schema Compare to create SQL scripts to apply the changes in different environments.
I would recommend doing only additive changes, but it will generate change scripts for destructive changes. If you do not have your environments synced up, I strongly recommend generating a new script for each environment.
This blog post goes over how to create one in Visual Studio 2012: http://candordeveloper.com/2013/01/08/creating-a-sql-server-database-project-in-visual-studio-2012/
Red Gate has a schema compare product too, but I have not really used it.
I recently started a new personal project to learn Entity Framework. My end goal is to make a desktop game that uses SQL compact for data management and uses Entity Framework for the game objects. Not actually knowing there were multiple ways to start EF (model first, code first, db first) I went with the most obvious choice of model first.
I've been working with it successfully now, however one thing concerns me, especially post-development. My goal with the game is that users can update to the latest version without losing any of their existing data. The current issue is that all the generation scripts are destructive by nature (dropping everything then recreating it) - that means I can't run those against the user SQLCE DBs out in "production", so I need to come up with an alternative plan of action.
That said, does anyone have recommended solutions on best practices? In previous desktop apps, I've traditionally used XML/binary to store data, which allows me to easily update the "schema" without affecting existing data (versioning in the app tailors the Load() according to the version, while the Save() always saves in the latest version).
What are some recommendations on handling this problem using SQLCE?
What you need, if understood right, is to utilize migrations which come with EF. Since the question is general this link should best guide you to what you need I think...
http://blogs.msdn.com/b/adonet/archive/2012/02/09/ef-4-3-code-based-migrations-walkthrough.aspx
With migrations which you can tailor manually if needed (and come in the shape of code which is applied at each point of change, incrementally) and you can also supply your 'seeding' if required.
i.e. you should be able to do most of what you require, delete, remove old incompatible data - and seed the new one that you have - and all related to a particular migration step you have.
How would that work with your app deployment specifically, that's a bit more complex I guess, but this should get you started, and then with each db version-breaking change your new code update would contain all the migrations since the previous update (or just one usually is enough, i.e. make it be one with each update) and the code to tear-down or create new things.
hope this helps,
I began recently a new job, a very interesting project (C#,.Net 4, Linq, VS 2010 and SQL Server). And immediately I got a very exciting challenge: I must implement either a new tool or integrate the logic when program start, or whatever, but what must happen is the following: the customers have previous application and database (full with their specific data). Now a new version is ready and the customer gets the update. In the mean time we made some modification on DB (new table, columns, maybe an old column deleted, or whatever). I’m pretty new in Linq and also SQL databases and my first solution can be: I check the applications/databases version and implement all the changes step by step comparing all tables, columns, keys, constrains, etc. (all this new information I have in my dbml and the old I asked from the existing DB). And I’ll do this each time the version changed. But somehow I feel, this is NOT a smart solution so I look for a general solution of this problem.
Is there a way to update customers DB from the dbml file? To create a new one is not a problem (CreateDatabase with DataContext), is there any update/alter database methods? I guess I’m not the only one who search for such a solution (I found nothing in internet – or I looked for bad keywords). How did you solve this problem? I look also for an external tool, but first for a solution with C#, Linq or something similar.
For any idea thank you in advance!
Best regards,
Emil
What I always do is use Red Gate's SQL Compare to compare the schema of the new database to the schema of the old database. It will generate a change script for you and then you can run that script in code.
We have a table that has a single row in it for program setup information. One of the columns in this table is the database version number. This will instantly tell us what database version the customer has when we do an update. Then we run every script that will update them to the latest version they need to be running. Whenever we release a new version (with database changes), we run the SQL Compare and make a script to go from the previous version to the next. We don't do any scripts that will skip versions, just in case of strange conflicts that may arise from that.
This also gives us the opportunity to do any data massaging we may have to do in between versions by writing a custom script and inserting that into the update scripts. Every update script changes that database version field as well.
This allows us to do a lot of automated updating. Having that database version allows the client to take a peek at that version before the user has a chance to use the application. If it's different and the application needs an update, it will go out to our ftp site and download the update and run the setup automatically.
Basically what you want to be able to do is to script the changes - to be able to run "something" that allows you to update one version of the database to the next and also to make any necessary changes to the data required by that change in the schema.
Good news is that you can do this with SQL, you can write DDL statements to create and modify a database schema.
My solution is to put my database schema maintenance entirely in code, I think this is the best version of the writeup I've done so far:
How to create "embedded" SQL 2008 database file if it doesn't exist?
Why in code? Because it works. May not be the best solution but its one I have had some success with and the results are consistent and repeatable. Oh and its version controlled too.
The big problem you may have in this specific instance is that you need to establish a baseline - to make sure that the existing databases are consistent in terms of their schema. This is where more complex and clever tools may serve you better - being able to do a schema diff and then update has a lot of appeal as a concept for example but equally you're somewhat dependent on having your reference database perfect and that raises other issues.