I have a product that I'm currently authoring that relies on SQL server for the backend. One issue I'm trying to resolve is to improve the 'upgrade' story. So v1 will have a particular schema and v2 may include some enhancements to this schema (new tables and new columns).
I'm aware of the SDKs from RedGate and ApexSQL - but would like to avoid.
I've had a read through the SMO docs, but I'm new to it and struggling to see if this can be applied in this situtaiton. Ideally, I'd like this to make this programatic (SMO or other) - the base cases seems straight forward enough, but I really don't want to re-invent the wheel if I can help it. Does anyone have any experience of similar requirements or ideas about how I could approach?
You don't say what version of SQL Server you're using but in (I think) 2005 and beyond, there is the concept of database triggers. These work like their table level cousins but can be used to track any kind of DDL change that happens on the database. We didn't use it to actually generate DDL - more to track when the format of a table changed. Although what you're after should be possible I'd have thought.
Triggers are one of those things that divide developers. Some people think they're the best thing since sliced bread whilst others hate them with a passion. Perhaps because when data changes, these are the last thing you think of.
Maybe not exactly what your're looking for (since it's not SMO) but having a look at Entity Framework Code First Migrations might help you:
http://msdn.microsoft.com/en-us/data/jj591621
Changes in the model-classes can be versioned and can either be applied directly to a database or, if you do not have direct access to your database, you can generate SQL-Code for your new version and hand it to your database-administrator.
I us Database Projects in Visual Studio to mange versioning of schemas. Once you create a baseline in a Database Project, you can make your changes in the project and then use the Schema Compare to create SQL scripts to apply the changes in different environments.
I would recommend doing only additive changes, but it will generate change scripts for destructive changes. If you do not have your environments synced up, I strongly recommend generating a new script for each environment.
This blog post goes over how to create one in Visual Studio 2012: http://candordeveloper.com/2013/01/08/creating-a-sql-server-database-project-in-visual-studio-2012/
Red Gate has a schema compare product too, but I have not really used it.
Related
We are building this .NET application using Entity Framework as our DB connector. I know all about the stuff of picking the right approach based on your circumstances like "do you have an existing database?", "do you prefer modelling instead of coding". But after some reading I've found that it's not the only thing to think of as the upgrade process of the database when it's already in production is really important, espacially for us.
So which approach is best for production use with Entity Framework. For the moment we have an existing database. I prefer to use the model and update the database from it but then we have lack of functionality in default values of columns and the model can be hard to work with in teams so what we need is basically some best practice here.
For production use: Database First, Model First or Code First?
Someone else might chime in here and tell you to use model migrations with the code first approach. That may be a solution, it's just not my preference.
We manage an in-motion database using EF code first, however I would not be able to do it without one hugely beneficial Visual Studio feature: SQL Schema Compare. I believe this feature is only available in the Premium and Ultimate versions of the product.
Each time our model changes, I put 2 copies of the database schema on my local machine: the new version, and the current production version. If you run Schema Compare using the new version as the source and the production version as the target, it will generate a SQL script that you can run against your production db to bring its schema and data in line with the changes.
The SQL it generates often needs some editing before it can be run in production, but it will do a lot of the hard work for you -- disabling constraints, add / drop indexes, and moving data from an old table into a new version of it. It will also warn you of potential issues when changing the schema.
I recently started a new personal project to learn Entity Framework. My end goal is to make a desktop game that uses SQL compact for data management and uses Entity Framework for the game objects. Not actually knowing there were multiple ways to start EF (model first, code first, db first) I went with the most obvious choice of model first.
I've been working with it successfully now, however one thing concerns me, especially post-development. My goal with the game is that users can update to the latest version without losing any of their existing data. The current issue is that all the generation scripts are destructive by nature (dropping everything then recreating it) - that means I can't run those against the user SQLCE DBs out in "production", so I need to come up with an alternative plan of action.
That said, does anyone have recommended solutions on best practices? In previous desktop apps, I've traditionally used XML/binary to store data, which allows me to easily update the "schema" without affecting existing data (versioning in the app tailors the Load() according to the version, while the Save() always saves in the latest version).
What are some recommendations on handling this problem using SQLCE?
What you need, if understood right, is to utilize migrations which come with EF. Since the question is general this link should best guide you to what you need I think...
http://blogs.msdn.com/b/adonet/archive/2012/02/09/ef-4-3-code-based-migrations-walkthrough.aspx
With migrations which you can tailor manually if needed (and come in the shape of code which is applied at each point of change, incrementally) and you can also supply your 'seeding' if required.
i.e. you should be able to do most of what you require, delete, remove old incompatible data - and seed the new one that you have - and all related to a particular migration step you have.
How would that work with your app deployment specifically, that's a bit more complex I guess, but this should get you started, and then with each db version-breaking change your new code update would contain all the migrations since the previous update (or just one usually is enough, i.e. make it be one with each update) and the code to tear-down or create new things.
hope this helps,
In writing an application that runs on Fluent Nhibernate/Nhibernate, something has me a bit concerned. I suppose this would be true of any ORM (and even without using an ORM), but what is the ... I guess the word is 'field of study' that relates to the best practices and methods for updating a database after deployment?
In nHibernate, I establish a SessionFactory and have an initial run where it writes the database out based on the mappings. That's fine and good, I can even write the database out manually. But what about when my client comes back and wants something new added? Can I append to the database without losing my data? I am completely new to all of this and it has been troubling me since the start of this project, and I really do not know what direction to go to make sure I can manage the program after it is deployed.
I have looked at other stack overflow questions that I could find regarding this topic - one of which did not even have an accepted answer (though the question itself was kind of vague), but I did discover the tool http://www.red-gate.com/products/sql-development/sql-compare/ from the question
Tool to upgrade SQL Express database after deployment though I am wondering just how good of a 'strategy' that is.
There are a couple of options, use the AutoMapping feature in Fluent NHibernate to minimize the mapping code you write. If your schema changes comply with the AutoMap conventions then you only need to work with the corresponding domain object changes.
Another less optimal option is to take a database first approach and have something like MyGeneration automatically generate the domain classes and NHibernate mapping files from the schema. This works if you have complete control of the database schema and it can be made to implement a good domain model design (both conditions which very rarely ever happen...)
In either approach, these tools can help handle the database scripting needed to "migrate" the schema changes to a new version
from my experience, after deployment you have to manually keep your db structure up-to-date.
that means that whenever you add / change your db structure, you do so using a script with DDL commands.
when you're ready to deploy, you just run those DDL scripts against your production db.
for example, if you add a 'bar' column to your 'foo' table, your script would be something like (pseudo-code):
ALTER TABLE foo ADD COLUMN 'bar' int(32) not null default(0);
Currently, changes to the database are made through the SQL Server Management program. IF a table changes, sqlmetal is run to regenerate the linqtosql classes and development continues. However, this makes deployment a pain, as you have to go through and manually update the deployment database (and any other databases used in the development cycle). It would be nice if we could use C# to generate these changes, as it would help eliminate human error and have the added benefit of being able to keep the database structure in git. Right now, the only representation of the database is in the generated linqtosql classes.
I've been looking around for a nice library that can handle this sort of thing, but the main solutions seem to be: keep a sql generation script, or embed sql in C# classes that can be run to make changes to the database. Both of these seem to be very non-ideal situations, as you lose the nice strong-typing that C# provides. It seems like there should be a way to do this using pure C#.
I've seen hints of being able to do things like generate databases from POCOs using both the entity framework and linqtosql, but I'm having a hardtime finding specific examples of that being used. Additionally, I haven't been able to discover if those have a graceful (i.e. data preserving) way of handling changes to the database after the initial table generation.
Are there any projects out there that solve this problem?
There exist several tools that help you with schema (and data) migrations of your database: RikMigrations, Migrator.Net and Machine.Migrations. Hope that helps.
Wizardby looks also promising: It provides database independent DDL scripts and automated migrations between different versions of a schema.
VS2010 can operate version control on your database schema through a Database Project. There are other tools out there for DB development that offer version control, you'll need to search to find them and compare pricing.
I prefer to version using sql scripts. Works pretty well, is free, supports updates, easy to version, works well with traditional source control methods.
First,
Create your DB
Use the Database Publishing Wizard to publish the database as a .sql script
Add a version number to the script
Add to your solution
Check into source control.
As updates are made,
Script updates to the previous schema as .sql files
Add a version number to the script that is incremented from the previous version
Add to your solution
Check into source control.
It sounds like you need a tool like Migrator.NET to manage your database migrations. We use it with a call from our site start-up to migrate the database as needed for any particular version.
I have toyed with an idea for creating a cleaner interface and someday hope to get around to implementing it, but other priorities have pushed that back. For now we are using raw sql strings in our migrations because there isn't a sybase driver implementation (outside of a very ugly hack I have written to manage the versioning table).
Redgate software offers something that may be really useful for you. It's called SQL Packager and it does it's job pretty well.
Features:
Easy roll-out of database updates across your client base
Script and compress your schema and data accurately and quickly
Package any pre-existing SQL script as a .exe, or launch as a C# project
Simplify deployments and updates for SQL Server 2000, 2005 and 2008
They also offer SQL Source Control which also may be useful to keep things nice and easy.
As an addon to MadBoy, SQL Packager can also launch the package as a C# project.
Red-Gate's SQL Compare is excellent as well, and as some of the banners on SO indicates, there is new SQL Source Control available as well.
Then they have their SQL Comparison SDK.
The trick here is to rely on the database being the single source of truth for your Linq schema, not the generated classes.
We use Linq to SQL extensively in our dev shop, and work as follows:
1. Create your database (working copy) from version control (baseline).
2. Modify your database any which way you like.
3. Generate Linq to SQL classes from the (working) database.
4. Create patches to update your baseline database to your working copy.
5. Check in and share these patches with all developers.
For a very quick and easy way of generating baseline and working copy databases, try DBSourceTools. http://dbsourcetools.codeplex.com
Have fun.
I began recently a new job, a very interesting project (C#,.Net 4, Linq, VS 2010 and SQL Server). And immediately I got a very exciting challenge: I must implement either a new tool or integrate the logic when program start, or whatever, but what must happen is the following: the customers have previous application and database (full with their specific data). Now a new version is ready and the customer gets the update. In the mean time we made some modification on DB (new table, columns, maybe an old column deleted, or whatever). I’m pretty new in Linq and also SQL databases and my first solution can be: I check the applications/databases version and implement all the changes step by step comparing all tables, columns, keys, constrains, etc. (all this new information I have in my dbml and the old I asked from the existing DB). And I’ll do this each time the version changed. But somehow I feel, this is NOT a smart solution so I look for a general solution of this problem.
Is there a way to update customers DB from the dbml file? To create a new one is not a problem (CreateDatabase with DataContext), is there any update/alter database methods? I guess I’m not the only one who search for such a solution (I found nothing in internet – or I looked for bad keywords). How did you solve this problem? I look also for an external tool, but first for a solution with C#, Linq or something similar.
For any idea thank you in advance!
Best regards,
Emil
What I always do is use Red Gate's SQL Compare to compare the schema of the new database to the schema of the old database. It will generate a change script for you and then you can run that script in code.
We have a table that has a single row in it for program setup information. One of the columns in this table is the database version number. This will instantly tell us what database version the customer has when we do an update. Then we run every script that will update them to the latest version they need to be running. Whenever we release a new version (with database changes), we run the SQL Compare and make a script to go from the previous version to the next. We don't do any scripts that will skip versions, just in case of strange conflicts that may arise from that.
This also gives us the opportunity to do any data massaging we may have to do in between versions by writing a custom script and inserting that into the update scripts. Every update script changes that database version field as well.
This allows us to do a lot of automated updating. Having that database version allows the client to take a peek at that version before the user has a chance to use the application. If it's different and the application needs an update, it will go out to our ftp site and download the update and run the setup automatically.
Basically what you want to be able to do is to script the changes - to be able to run "something" that allows you to update one version of the database to the next and also to make any necessary changes to the data required by that change in the schema.
Good news is that you can do this with SQL, you can write DDL statements to create and modify a database schema.
My solution is to put my database schema maintenance entirely in code, I think this is the best version of the writeup I've done so far:
How to create "embedded" SQL 2008 database file if it doesn't exist?
Why in code? Because it works. May not be the best solution but its one I have had some success with and the results are consistent and repeatable. Oh and its version controlled too.
The big problem you may have in this specific instance is that you need to establish a baseline - to make sure that the existing databases are consistent in terms of their schema. This is where more complex and clever tools may serve you better - being able to do a schema diff and then update has a lot of appeal as a concept for example but equally you're somewhat dependent on having your reference database perfect and that raises other issues.