In our software, we have a customer base with existing databases. The databases are currently accessed via EntitySpaces, but we'd like to switch to EntityFramework (v6), as EntitySpaces is no longer supported. We'd also like to make use of the migrations feature. Automatic migrations are disabled, since we only want to allow database migration to an admin user.
We generated the EF model from an existing database. It all works pretty well, but the real problem we have, is, programmatically distinguishing between existing databases that match the model but have not yet been converted to EF (missing MigrationsHistory table), and empty/new databases. Converting existing databases works well with an empty migration, but for new databases we also need a migration containing the full model. Having an initial migration in the migration chain always clashes with existing databases. Of course we could create a workaround with external SQL scripts or ADO commands, creating and populating the MigrationsHistory table. But that is something we'd like to avoid, because some of our clients use MsSql databases, some use Oracle. So we'd really like to keep the abstraction layer provided by EF.
Is there a way to get EF to handle both existing, and new databases through code-based migrations, without falling back to non-EF workarounds?
My original suggestion was to trap the exception raised by CreateTable, but it turns out this is executed in a different place so this cannot be trapped within the exception.
The simplest method of proceeding will be to use the Seed method to create your initial database if it is not present. To do this...
Starting from a blank database, add an Initial Create migration and grab the generated SQL
Add-Migration InitialCreate
Update-Database -Script
Save this script. You could add it to a resource, static file or even leave it inline in your code if you really want, it's up to you.
Delete all of the code from the InitialCreate migration (leaving it with a blank Up() and Down() function). This will allow your empty migration to be run, causing the MigrationHistory table to be generated.
In your Migration configuration class, you can query and execute SQL dynamically using context.Database.SqlQuery and context.Database.ExecuteSqlCommand. Test for the existence of your main tables, and if it's not present, execute the script generated above.
This isn't very neat, but it's simple to implement. Test it well, as the Seed method runs after EVERY migration runs, not just the initial one. This is why you need to test for the existence of a main table before you do anything.
The more complicated approach would be to write a "CreateTableIfNotExists" method for migrations, but this will involve use of Reflection to call internal methods in the DbMigration class.
Related
I am using EF code first to create my database, the application is in production and test now so I need to run migrations to update the database as my model changes.
I have set up migrations based on the standard instructions and it works fine, against dev database one. I also have a second database that I want to bring in line with the current code base but this second database is in an unknown state.
What I want to know is can I generate a migration class against a different database and how do I do that? I can't seem to generate a migration class against anything other than the original database I specified when I set up migrations. I have changed the DB name in the app.config but it still looks at the original database when doing migrations.
How can I explicitly specify which database to generate the migration script against?
Migrations don't look at the database other than to check if a migration has been applied (__MigrationHistory). When you setup migrations it takes a snapshot of the code models and uses that when moving forward. So if you want to bring a database in an unknown state under migration control you will need to get it to match the code model.
You could do something similar to this: Create an idempotent script from your current database (update-database -Script –SourceMigration $InitialDatabase). This will be the equivalent of all migrations needed to create and bring your database current. Since you have an existing database where some of these objects exist, you will need to manually adjust the script by removing the code for items that already exist. Once you get it to apply, you should have a database that matches your model with a populated __MigrationHistory table.
Another option is to use a schema compare utility to make the databases match and then just copy the __MigrationHistory table over.
See Migrations Under the Hood.
I have a solution where I do not have one default database. I have a master Database that are returning a connection string for the customer that is requesting data, and each customer has their own database. I am using migration (and has AutomaticMigrationsEnabled set to false) and code first.
The command “update-database” is “excecuted” from the code (Database.SetInitializer(new MigrateDatabaseToLatestVersion());).
The first time is it all working fine, but when I afterwards will add a migration, I cannot, because there are pending migrations. I can not run update-database from VS because the connections string is set on runtime.
What is the right way to handle a setup like mine, with migrations and Entity Framework 6?
Thanks very much in advance
What you have is so called multitenant application. And yes, EF migrations can support such scenario, too.
You just have to change your migrator from the MigrateDatabaseToLatestVersion to a custom one that migrates the exact database the context is created for.
Then, forget about executing update-database from a powershell console. This always updates the default database! Instead, your custom migrator will automatically update the database you connect to.
Complete source code of the custom migrator can be found in my blog entry:
http://www.wiktorzychla.com/2013/05/entity-framework-5-code-first-multi.html
Be aware that due to a bug in eariler versions of EF, migrations in a multitenant scenario work correctly starting from EF 6.1.3.
Because of this bug, your multitenant application could incorrectly assign connection strings to newly created db contexts.
https://entityframework.codeplex.com/SourceControl/changeset/4187b1c308316a22b38ef533b1409024bc0f7406
Add-Migration and Update-Database from the console still refer to a database.
To create a DbContext, these commands either use its parameterless constructor (if you have one), or it instantiates an IDbContextFactory<T> if one exists and calls its Create method to get one.
The fact you don't use this specific database at run-time isn't really important, but you do need one somewhere at design time to allow you to add migrations.
I have redesigned my class scheme many times since started the project I'm working on.
At first I didn't know about data annotations, so I have migration1, where my Ids have no DatabaseGenerated.Identity option and migration4 where they already have it.
Turns out that EF doesn't work this way (https://entityframework.codeplex.com/workitem/509 as for EF5), so when I added some seed data, it threw an exception
Cannot insert the value NULL into column 'primarykeycolumn', table 'tablename'; column does not allow nulls. INSERT fails.
What's really interesting is when I deleted all existing migrations and scaffolded new one from scratch, seed method ran without any problems.
So I have a question: do I need to do it any time I make changes to scheme such as adding data annotations like Identity or is there a way to save my previous migrations? Because dropping and recreating db in real-life can result in a huge data loss, which I want to avoid.
Adding Migrations. In your Package Manager Console
Add-Migration Test1
In your Configuration.cs from Migration Class You will see Up method and Down Method. It is like Before, and After Changes. You can call them up if you want to reverse the Changes.
doing Add-Migration "NameoftheMigration" does not implement it
you need to call
Update-Database
If it tells you that there is an error, or potential Data lost.
Update-Database -Force
If you choose to use CodeFirst. You can't really Jump from Altering table by Design, then by code. You have to Choose one. As CodeFirst Migration rely on Models. Data First, Rely on Data Design.
Could someone explain the concept of Migrators (specifically fluentmigrator)?
Here are the (possibly confused) facts Ive gleaned on the subject:
Is it a way to initially create then maintain updates for a database
by way of versioning.
The first migration (or initial version of the
database) would contain all the tables, relationships and properties
required (done either fluently or using a chunk of sql in a script).
When you want to push a change to a database, you would create a new
migration method (Up and Down), something like add a new table or modify a field.
To deploy one of these migrations, you would use a
command line specifying the dll containing the migration, the
connection string and the required version.
If you had a rather complex set of data models, wouldn't it be rather difficult and time consuming to create a migration definition for all of that?
I know with nHibernate/fluent you can easily generate tables for a database without having to define anything other than the models and map files. Is there a way to make this configuration compatible with the Migrator/Versioning?
When nhibernate/fluent is in charge of generating a database, I do not necessarily need to define every thing aspect of the tables. Its done either via convention or via the mapping files. With the migrators I would need to define this level of detail?
Lots of questions here. I'll answer the questions with a focus on FluentMigrator.
Is it a way to initially create then maintain updates for a database
by way of versioning.
FluentMigrator is a way to version control your database schema. Everyone does it in some way. Either manually, with sql scripts, with a tool like SqlCompare or a Visual Studio Database project. All these methods are easy to mess up. It is so easy to make a mistake when releasing a new version and cause the system to crash. Migrations is a better way to handle this.
FluentMigrator allows you to define a change to the schema as code and this is usually checked in to your source control with the other code changes. Meaning that you can say version 1.XX of your system should have version 123 of the database. It means if you roll back your code to the previous version you also know what version of the database to rollback to as well.
It can be used both to create the database schema from the beginning or to start with version control of the schema for an existing database.
A Migration is a way to describe a change to your database schema. FluentMigrator creates a VersionInfo table and stores the unique id (version number) of the Migration after is has been applied.
For example, if I have two Migrations one with Id 1 and one with Id 2. If then I execute the first Migration then Id 1 will be stored in the VersionInfo table and I can look there and know that the version of the database is 1 and that version 2 has not been applied yet.
Being able to know which version the database schema is very useful when pushing changes from Test to Production or if you have multiple copies of the database in Production. For example, I have a customer with offices all around the world and each office has their own copy of the database and all of them are on different versions. Without knowing the database version it would be very difficult to update them safely.
Most of the time I do not need to actually look in the VersionInfo table, FluentMigrator handles this automatically. It compares the assembly with Migrations to the VersionInfo table and figures out which changes have not been applied yet and then executes those.
The first migration (or initial version of the database) would contain
all the tables, relationships and properties required (done either
fluently or using a chunk of sql in a script).
The starting point is up to you. You can have a first migration that is an sql script that you have generated from the current database. You could could also use one of the contrib projects like FluentMigrator.T4 to generate a Fluent Migration. Or you could just decide that the existing database is the starting point and save a copy of it to be able to restore it as version 1.
I have introduced FluentMigrator to a lot of legacy databases without any major problems.
When you want to push a change to a database, you would create a new
migration method (Up and Down), something like add a new table or
modify a field.
Yes, Up is used to apply the change specified in the Migration and Down rolls it back. So Up could be to create a table and Down could be to drop the table.
To deploy one of these migrations, you would use a command line
specifying the dll containing the migration, the connection string and
the required version.
There are three runners available to execute migrations. The command line runner, the Nant task and the MSBuild task. There are usually executed as part of a build script.
The MigrationRunner class can also be used in code. You might do this if you wanted to build your own runner or if you have other needs (like building databases dynamically or automatically updating the database if a new migration is added.)
If you had a rather complex set of data models, wouldn't it be rather
difficult and time consuming to create a migration definition for all
of that?
I have mostly answered this already. It is usually quite easy to generate an sql script for a database. For Sql Server it takes less than a minute to generate the script even for large databases. This script can be saved in a .sql file and executed as the first migration using the Execute.EmbeddedSqlScript expression. It works a treat.
I know with nHibernate/fluent you can easily generate tables for a
database without having to define anything other than the models and
map files. Is there a way to make this configuration compatible with
the Migrator/Versioning?
At the moment, there is no such integration and in practise I, at least, don't miss it. There was some discussion about connecting Fluent NHibernate and FluentMigrator but it would be a lot of work. It would enable scaffolding to generate changes to the model like EF Code First migrations do. It's not on the roadmap at the moment however.
When nhibernate/fluent is in charge of generating a database, I do not
necessarily need to define every thing aspect of the tables. Its done
either via convention or via the mapping files. With the migrators I
would need to define this level of detail?
Yes, you would need to define at that level of detail. FluentMigrators' migrations are a DSL (own little language) for defining schema changes that are translated to sql. You can write sql directly as well using the Execute.Sql expression. Entity Frameworks migrations have that sort of integration which has both advantages and disadvantages.
Check out the wiki or one of the tutorials here, here (part 1) or here (part 2) for more help getting started.
I have an existing application with a SQL database that has been coded using a database first model (I create an EDMX file every time I have schema changes).
Some additional development (windows services which support the original application) has been done which uses EF POCO/DbContext as the data layer instead of an EF EDMX file. No initializer settings were ever configured in the DbContexts, but they never modified the database as the DbSet objects always matched the tables.
Now, I've written a seperate application that uses the existing database but only its own, new tables, which it creates itself using EFs initializer. I had thought this would be a great time to use EF Code First to handle managing these new tables. Everything worked fine the first time I ran the application, but now I am getting this error from some of my original EF POCO DbContexts (which never used an initializer).
The model backing the 'ServerContext' context has changed since the
database was created. Consider using Code First Migrations to update
the database
After some investigation, I've discovered that EF compares a hash of its schema with some stored hash in the sql server somewhere. This value doesn't exist until a context has actually used an initializer on the database (in my case, not until the most recent application added its tables).
Now, my other DbContexts throw an error as they read the now existing hash value and it doesn't match its own. The EF connection using the EDMX doesn't have any errors.
It seems that the solution would be to put this line in protected override void OnModelCreating(DbModelBuilder modelBuilder) in all the DbContexts experiencing the issue
Database.SetInitializer<NameOfThisContext>(null);
But what if later on I wanted to write another application and have it create its own tables again using EF Code first, now I will never be able to reconcile the hash between this theoretical even newer context and the one that is causing the issue right now.
Is there a way to clear the hash that EF stores in the database? Is EF smart enough to only alter tables that exist as a DbSet in the current context? Any insights appreciated.
Yes, Bounded DB contexts is actually good practice.
eg a base context class, to use common connection to DB, each sub class, uses the
Database.SetInitializer(null); as you suggest.
Then go ahead and have 1 large context that has the "view of the DB" and this context is responsible for all migrations and ONLY that context shoudl do that. A single source of truth.
Having multiple contexts responsible for the DB migration is a nightmare I dont think you will solve.
Messing with the system entries created by code first migrations can only end in tears.
Exactly the topic you describe I saw in A julie Lerman video.
Her suggested solution was a single "Migration" context and then use many Bounded DB contexts.
In case you have a pluralsight account:
http://pluralsight.com/training/players/PsodPlayer?author=julie-lerman&name=efarchitecture-m2-boundedcontext&mode=live&clip=11&course=efarchitecture
What EF version are you using? EF Code First used to store hash of the SSDL in the EdmMetadata table. Then in .NET Framework 4.3 thingh changed a little bit and the EdmMetadata table was replaced by __MigrationsHistory table (see this blog post for more details). But it appears to me that what you are really looking after is multi-tenant migrations where you can have multiple context using the same database. This feature has been introduced in EF6 - (currently Aplpha2 version is publicly available) Also, note that EdmMetadata/__MigrationHistory tables are specific to CodeFirst. If you are using the designer (Model First/Database First) no additional information is stored in the database and the EF model is not checked whether it matches the database. This can lead to hard to debug bugs and/or data corruption.