We Keep Coding

How can I use EF Core Migrations API directly by code

With Entity Framework Core, we can generate C# migration files, containing database information to create tables, columns, indexes, etc… and then we can deploy the database using command line, or with a EF DbContext.
Is there a way to use directly the Migrations API by code to modify a database, without using "real" migrations (without code model or dbContext) ?
I would like to do something like this :
var builder = new MigrationBuilder(activeProvider);
builder.CreateTable(
name: "Table1",
columns: table => new
{
Id = table.Column<int>(nullable: false),
Name = table.Column<string>(maxLength: 256, nullable: true)
});
Apply(builder, connectionString);
What should I put in the Apply method to create the table in my database ?

I went in the EF Core internals, and found a way to use the APIs.
I'm not sure if it is a good idea, or if it will work fine, but it works for a simple case (create table, add column).
First, we need to register some EF Core services :
public void ConfigureServices(IServiceCollection services)
{
services.AddScoped<ICurrentDbContext, CurrentDbContext>();
services.AddScoped<RelationalConnectionDependencies>();
services.AddEntityFrameworkSqlServer();
DbContextOptionsBuilder dbContextOptionsBuilder = new DbContextOptionsBuilder<DbContext>();
dbContextOptionsBuilder.UseSqlServer("Data Source=.\\SQLEXPRESS;Initial Catalog=TestDeploy;Integrated Security=True;");
services.AddSingleton<IDbContextOptions>(dbContextOptionsBuilder.Options);
services.AddDbContext<DbContext>();
services.AddScoped<MyMigrationRunner>();
}
We don't use the DbContext directly, but it seems that it is used internally. We don't need anything specific, we can just register the base DbContext from EF Core.
Also, I used SqlServer specific implementation, I assume it would work with other providers, but I didn't test.
To build the database, I created a Migration class
public class MyMigration : Migration
{
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "Table1",
columns: table => new
{
Id = table.Column<int>(nullable: false),
Name = table.Column<string>(maxLength: 256, nullable: true)
});
}
}
Now I can use the IMigrationsSqlGenerator service to generate the SQL commands and apply them to the database
public class MyMigrationRunner
{
private readonly IRelationalConnection connection;
private readonly IMigrationsSqlGenerator migrationsSqlGenerator;
public MyMigrationRunner(IMigrationsSqlGenerator migrationsSqlGenerator, IRelationalConnection connection)
{
this.migrationsSqlGenerator = migrationsSqlGenerator;
this.connection = connection;
}
public void Run()
{
var migration = new MyMigration();
var commands = migrationsSqlGenerator.Generate(migration.UpOperations).ToList();
foreach (var cmd in commands)
{
cmd.ExecuteNonQuery(connection);
}
}
}

Entity Framework, Automatic apply Migrations

I am using Entity Framework Code First approach with AutomaticMigrationsEnabled = true:
Database.SetInitializer(new MigrateDatabaseToLatestVersion<DbContext, MigrateDBConfiguration>());
//////////////////////////////////
public class MigrateDBConfiguration : System.Data.Entity.Migrations.DbMigrationsConfiguration<DbContext>
{
public MigrateDBConfiguration()
{
AutomaticMigrationsEnabled = true;
AutomaticMigrationDataLossAllowed = true;
}
}
The first run of the project creates the database and tables as expected. After changing my model by adding or dropping fields, I ran Add-Migration. The Migration class was generated but after running the project this exception occurs:
An exception of type 'System.InvalidOperationException' occurred in EntityFramework.dll but was not handled in user code
Additional information: The model backing the 'DBContext' context has
changed since the database was created.
EDIT: Per the guidance in the answer of arturo menchaca I changed my code like this:
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
Database.SetInitializer(new MigrateDatabaseToLatestVersion<DBContext, MigrateDBConfiguration<DBContext>>());
...
After the change this exception is occurring:
There is already an object named 'MyTable' in the database.
How can I apply my database migration?

Automatic Migrations means that you don't need to run add-migration command for your changes in the models, but you have to run update-database command manually.
If Automatic Migrations is enabled when you call update-database, if there are pending changes in your models, an 'automatic' migration will be added and database will be updated.
If you want that your database is updated without need to call update-database command, you can add Database.SetInitializer(...) in OnModelCreating() method on your context, like so:
public class MyContext : DbContext
{
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
Database.SetInitializer(new MigrateDatabaseToLatestVersion<MyContext, MigrateDBConfiguration>());
}
...
}
public class MigrateDBConfiguration : System.Data.Entity.Migrations.DbMigrationsConfiguration<MyContext>
{
...
Note that you should declare DbMigrationsConfiguration and MigrateDatabaseToLatestVersion with your real context, not the default DbContext.

Finally, I found a solution to my problem. I call this method in each application start :
public void InitializeDatabase(DataAccessManager context)
{
if (!context.Database.Exists() || !context.Database.CompatibleWithModel(false))
{
var configuration = new DbMigrationsConfiguration();
var migrator = new DbMigrator(configuration);
migrator.Configuration.TargetDatabase = new DbConnectionInfo(context.Database.Connection.ConnectionString, "System.Data.SqlClient");
var migrations = migrator.GetPendingMigrations();
if (migrations.Any())
{
var scriptor = new MigratorScriptingDecorator(migrator);
var script = scriptor.ScriptUpdate(null, migrations.Last());
if (!string.IsNullOrEmpty(script))
{
context.Database.ExecuteSqlCommand(script);
}
}
}
}

If you have change in your entities, you need first run add-migration to create the migration script.
After that in your Global.asax
you need to have some code like this
var configuration = new MyProject.Configuration();
var migrator = new System.Data.Entity.Migrations.DbMigrator(configuration);
migrator.Update();
every time that you run your asp.net project it'll check if you have a new migration to run and run update-database automatically for you.

Microsoft addresses migrations at runtime, here.
For example, you can do this in Program.cs: (tested working in .NET 5.0 preview)
public static void Main(string[] args)
{
var host = CreateHostBuilder(args).Build();
MigrateDatabase(host);
host.Run();
}
private static void MigrateDatabase(IHost host)
{
using var scope = host.Services.CreateScope();
var services = scope.ServiceProvider;
try
{
var context = services.GetRequiredService<ApplicationDbContext>();
context.Database.Migrate();
}
catch (Exception ex)
{
var logger = services.GetRequiredService<ILogger<Program>>();
logger.LogError(ex, "An error occurred creating the DB.");
}
}

using Microsoft.EntityFrameworkCore;
await _dbContext.Database.MigrateAsync();
_dbContext.Database.Migrate();
OR
await _dbContext.Database.EnsureCreatedAsync();
_dbContext.Database.EnsureCreated();
both method check if database exist, if not they both create it.
Migrate() uses migrations and is suitable if you use migrations or relational database.
EnsureCreated() does not use migrations which means once db is created using this method no further migrations can be executed over it.

Schema independent Entity Framework Code First Migrations

I have troubles using Entity Framework migrations targeting Oracle databases since schema name is included in migrations code and for Oracle, schema name is also user name. My goal is to have schema-independent Code First Migrations (to be able to have one set of migrations for testing and production enviroments).
I have already tried this approach (using Entity Framework 6.1.3):
1) I have schema name in Web.config:
<add key="SchemaName" value="IPR_TEST" />
2) My DbContext takes schema name as a constructor parameter:
public EdistributionDbContext(string schemaName)
: base("EdistributionConnection")
{
_schemaName = schemaName;
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.HasDefaultSchema(_schemaName);
}
3) I had to implement IDbContextFactory for Entity Framework Migrations to be able to create my DbContext which does not have parameterless constructor:
public class MigrationsContextFactory : IDbContextFactory<EdistributionDbContext>
{
public EdistributionDbContext Create()
{
return new EdistributionDbContext(GetSchemaName());
}
}
4) I also configured Migration History Table to be placed within correct schema:
public class EdistributionDbConfiguration : DbConfiguration
{
public EdistributionDbConfiguration()
{
SetDefaultHistoryContext((connection, defaultSchema)
=> new HistoryContext(connection, GetSchemaName()));
}
}
5) I modified code generated for migrations to replace hardcoded schema name. Eg. I replaced CreateTable("IPR_TEST.Users") with CreateTable($"{_schema}.Users"). (_schema field is set according to the value in Web.config).
6) I use MigrateDatabaseToLatestVersion<EdistributionDbContext, MigrationsConfiguration>() database initializer.
Having all this set up, I still have problems when I switch to different schema (eg. via web.config transformation) - an exception is thrown telling me that database does not match my model and AutomaticMigrations are disabled (which is desired). When I try to execute add-migration a new migration is generated where all object should be moved to different schema (eg: MoveTable(name: "IPR_TEST.DistSetGroups", newSchema: "IPR");, which is definitely not desired.
For me it seems that schema name is hard-wired somewhere in model string-hash in migration class (eg. 201509080802305_InitialCreate.resx), ie:
<data name="Target" xml:space="preserve">
<value>H4sIAAAAAAAEAO09227jO... </value>
</data>
It there a way how to tell Code First Migrations to ignore schema name?

You can create a derived DbContext and "override" modelBuilder.HasDefaultSchema(...) in OnModelCreating:
public class TestDbContext : ProductionDbContext
{
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.HasDefaultSchema("TestSchema");
}
}
Then you can create migrations for both contexts. See this question on how to create two migrations in one project.
The downside of this approach is that you have to maintain two seperate migrations. But it gives you the opportunity to adjust the configuration of your TestDbContext.

I was faced to same problem and thanks to your aproach I finally found a solution that seems to work pretty well:
1) I have the schema name in Web.config app settings:
<add key="Schema" value="TEST" />
2) I have a history context:
public class HistoryDbContext : HistoryContext
{
internal static readonly string SCHEMA;
static HistoryDbContext()
{
SCHEMA = ConfigurationManager.AppSettings["Schema"];
}
public HistoryDbContext(DbConnection dbConnection, string defaultSchema)
: base(dbConnection, defaultSchema)
{ }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.HasDefaultSchema(SCHEMA);
}
}
3) I have a db configuration that reference my history db context:
public class MyDbConfiguration : DbConfiguration
{
public MyDbConfiguration()
{
SetDefaultHistoryContext((connection, defaultSchema) => new HistoryDbContext(connection, defaultSchema));
}
}
4) And this is my db context:
public partial class MyDbContext : DbContext
{
public MyDbContext()
: base("name=MyOracleDbContext")
{ }
public static void Initialize()
{
DbConfiguration.SetConfiguration(new MyDbConfiguration());
Database.SetInitializer(new MigrateDatabaseToLatestVersion<MyDbContext, Migrations.Configuration>());
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.HasDefaultSchema(string.Empty);
}
}
5) Finally I call the Initialize method from the global.asax
protected void Application_Start()
{
MyDbContext.Initialize();
}
The key is to set the default schema of the db context to String.Empty and the schema of the history context to the correct one.
So when you create your migrations they are schema independant: the DefaultSchema variable of the resx of the migration will be blank. But the history db context schema is still correct to allow migrations checks to pass.
I am using the following nugets packages:
<package id="EntityFramework" version="6.2.0" targetFramework="net452" />
<package id="Oracle.ManagedDataAccess" version="12.2.1100" targetFramework="net452" />
<package id="Oracle.ManagedDataAccess.EntityFramework" version="12.2.1100" targetFramework="net452" />
You can then use Oracle migrations with success on different databases.

EF 6 Code First __MigrationHistory in dbo schema by default

I am new to Code first Entity framework, when logging into the database after running my app for the first time I got a little confused when I saw the "__MigrationHistory" table.
I now understand the need for this table, but do not like it being in the standard dbo schema within the user table, I think its obtrusive and a risk.
My first thought was to move it to the system folder. When researching how to achieve this within the EF context all I could find is how to move it from system to dbo.
I now get the feeling __MigrationHistory should by default be created within the system folder... is this the case?
How can I configure my context to manage/reference the migration history table within the system folder by default?
Here is my context, am I doing something wrong or missing some configuration?
public class MyContext : DbContext, IDataContext
{
public IDbSet<Entity> Entities { get; set; }
public MyContext()
: base("ConnectionString")
{
}
public new IDbSet<TEntity> Set<TEntity>() where TEntity : class
{
return base.Set<TEntity>();
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Conventions.Remove<PluralizingTableNameConvention>();
}
}

There is a technique for moving __MigrationHistory. That table has it's own context (System.Data.Entity.Migrations.History.HistoryContext) that you can override:
public class MyHistoryContext : HistoryContext
{
public MyHistoryContext(DbConnection dbConnection, string defaultSchema)
: base(dbConnection, defaultSchema)
{
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<HistoryRow>().ToTable(tableName: "MigrationHistory", schemaName: "admin");
modelBuilder.Entity<HistoryRow>().Property(p => p.MigrationId).HasColumnName("Migration_ID");
}
}
Then you need to register it:
public class ModelConfiguration : DbConfiguration
{
public ModelConfiguration()
{
this.SetHistoryContext("System.Data.SqlClient",
(connection, defaultSchema) => new MyHistoryContext(connection, defaultSchema));
}
}

You could try executing EXEC sys.sp_MS_marksystemobject __MigrationHistory in your seed method using context.Database.ExecuteSqlCommand();

How to create initializer to create and migrate mysql database?

I have been learning how to use EF for a week or so now and am stuck on the issue of creating/updating my database. I am able to create an initializer to create the database if it is not there:
static class Program
{
static void Main()
{
Database.SetInitializer<GumpDatabase>(new GumpDatabaseInitializer());
....
class GumpDatabaseInitializer : CreateDatabaseIfNotExists<GumpDatabase>
{
public GumpDatabaseInitializer()
{
}
protected override void Seed(GumpDatabase context)
{
context.Database.ExecuteSqlCommand("CREATE UNIQUE INDEX Name ON Stations (Name)");
// Other stuff
}
}
Or I can create a Configuration to migrate the db
static class Program
{
static void Main()
{
Database.SetInitializer<GumpDatabase>(new MigrateDatabaseToLatestVersion<GumpDatabase, Configuration>());
....
internal sealed class Configuration : DbMigrationsConfiguration<GumpDatabase>
{
public Configuration()
{
AutomaticMigrationsEnabled = true;
SetSqlGenerator("MySql.Data.MySqlClient", new MySql.Data.Entity.MySqlMigrationSqlGenerator());
}
protected override void Seed(GumpDatabase context)
{
}
Each works correctly but I haven't figured out a way to do both. I can switch between the two initializers by changing the SetInitializer call but if I want to create the database if it is not there and also migrate it if it is what do I do? Do I need to create a custom initializer?
Thanks
Edit based on NSGaga answer
class CreateOrMigrateDatabaseInitializer<TContext, TConfiguration> : CreateDatabaseIfNotExists<TContext>, IDatabaseInitializer<TContext>
where TContext : DbContext
where TConfiguration : DbMigrationsConfiguration<TContext>, new()
{
private readonly DbMigrationsConfiguration _configuration;
public CreateOrMigrateDatabaseInitializer()
{
_configuration = new TConfiguration();
}
public CreateOrMigrateDatabaseInitializer(string connection)
{
Contract.Requires(!string.IsNullOrEmpty(connection), "connection");
_configuration = new TConfiguration
{
TargetDatabase = new DbConnectionInfo(connection)
};
}
void IDatabaseInitializer<TContext>.InitializeDatabase(TContext context)
{
Contract.Requires(context != null, "context");
if (context.Database.Exists())
{
if (!context.Database.CompatibleWithModel(throwIfNoMetadata: false))
{
var migrator = new DbMigrator(_configuration);
migrator.Update();
}
}
else
{
context.Database.Create();
Seed(context);
context.SaveChanges();
}
}
protected virtual void Seed(TContext context)
{
}
}
and
internal sealed class Configuration : DbMigrationsConfiguration<GumpDatabase>
{
public Configuration()
{
AutomaticMigrationsEnabled = true;
AutomaticMigrationDataLossAllowed = false;
SetSqlGenerator("MySql.Data.MySqlClient", new MySql.Data.Entity.MySqlMigrationSqlGenerator());
}
protected override void Seed(GumpDatabase context)
{
}
}
and
class GumpDatabaseInitializer : CreateOrMigrateDatabaseInitializer<GumpDatabase,Gump.Migrations.Configuration>
{
public GumpDatabaseInitializer()
{
}
protected override void Seed(GumpDatabase context)
{
context.Database.ExecuteSqlCommand("CREATE UNIQUE INDEX Name ON Stations (Name)");
context.Database.ExecuteSqlCommand("CREATE UNIQUE INDEX Name ON Sequences (Name)");
context.Database.ExecuteSqlCommand("CREATE UNIQUE INDEX StationPartNumber ON StationPartNumbers (StationId,PartNumberId)");
}
}
and finally
static void Main()
{
Database.SetInitializer<GumpDatabase>(new GumpDatabaseInitializer());

I think you're pretty much there - you can lookup the source code for MigrateDatabaseToLatestVersion (it's open source http://entityframework.codeplex.com/) - it's pretty simplistic, what it does pretty much is call the DbMigrator - as far as I could see.
All you have to do seems is to merge the two - use one or the other as a basis, add other functionality in there - that should work fine I think.
class CreateAndMigrateDatabaseInitializer<TContext, TConfiguration> : CreateDatabaseIfNotExists<TContext>, IDatabaseInitializer<TContext>
where TContext : DbContext
where TConfiguration : DbMigrationsConfiguration<TContext>, new()
{
private readonly DbMigrationsConfiguration _configuration;
public CreateAndMigrateDatabaseInitializer()
{
_configuration = new TConfiguration();
}
public CreateAndMigrateDatabaseInitializer(string connection)
{
Contract.Requires(!string.IsNullOrEmpty(connection), "connection");
_configuration = new TConfiguration
{
TargetDatabase = new DbConnectionInfo(connection)
};
}
void IDatabaseInitializer<TContext>.InitializeDatabase(TContext context)
{
Contract.Requires(context != null, "context");
var migrator = new DbMigrator(_configuration);
migrator.Update();
// move on with the 'CreateDatabaseIfNotExists' for the 'Seed'
base.InitializeDatabase(context);
}
protected override void Seed(TContext context)
{
}
}
call it like this...
Database.SetInitializer(new CreateAndMigrateDatabaseInitializer<GumpDatabase, YourNamespace.Migrations.Configuration>());
...actually, override it (since it's generic implementation) like you were doing for CreateDatabaseIfNotExists (you just have extra 'param' for Configuration) - and just supply the 'Seed'.
class GumpDatabaseInitializer : CreateAndMigrateDatabaseInitializer<GumpDatabase, YourNamespace.Migrations.Configuration>
{
protected override void Seed(GumpDatabase context)
{
context.Database.ExecuteSqlCommand("CREATE UNIQUE INDEX Name ON Stations (Name)");
}
}
...and call it something like
Database.SetInitializer(new GumpDatabaseInitializer());
EDIT:
Based on the comments - DbMigrator should not run twice. It always checks (spends a bit of time) and does a 'blank' update and moves on. However just in case if you'd like to remove that and 'check' before entering - this should work (change the similar piece above)...
var migrator = new DbMigrator(_configuration);
if (!context.Database.CompatibleWithModel(throwIfNoMetadata: false))
if (migrator.GetPendingMigrations().Any())
migrator.Update();
(this is a redundant / double-check - one of the if-s should be enough. Put a break there - and see exactly what's happening, it should not get in - once Db is migrated. As I mentioned, works fine when I test it.
EDIT:
Replace the inside of InitializeDatabase with...
var doseed = !context.Database.Exists();
// && new DatabaseTableChecker().AnyModelTableExists(context);
// check to see if to seed - we 'lack' the 'AnyModelTableExists' - could be copied/done otherwise if needed...
var migrator = new DbMigrator(_configuration);
// if (doseed || !context.Database.CompatibleWithModel(throwIfNoMetadata: false))
if (migrator.GetPendingMigrations().Any())
migrator.Update();
// move on with the 'CreateDatabaseIfNotExists' for the 'Seed'
base.InitializeDatabase(context);
if (doseed)
{
Seed(context);
context.SaveChanges();
}
This works around (half-way) not-seeding - if migration goes first. And migrations have to be first, otherwise you have issues.
You still need to do it properly - this is the gist if not all you might need - but if any issues w/ MySQL etc., probably some more leg work here.
Note: Still seeding doesn't call if you have a db, but it's empty. Problem is mixing of the two different initializers. So you'll have to work that out - either by implementing what Create... does inside (that call we can't call) or something else.

Actually it should be:
var migrator = new DbMigrator(_configuration);
if (!context.Database.CompatibleWithModel(false) || migrator.GetPendingMigrations().Any())
migrator.Update();
because if we have a migration, that is not related to our db model, for example inserting a row in any of our tables, the migration won't be executed.

To do both (seed and migrate) you really only have to use migrations with a MigrateDatabaseToLatestVersion initializer. When you enable migrations for your context a Configuration class derived from DbMigrationsConfiguration is created and you can override the Seed method to seed your database. Note that the database may already contain seed data when this method executes but the AddOrUpdate extension method conveniently helps you make "upserts" in your database.
This is different compared to the Seed method of some of the other database intitializers where the database is only seeded when it is initially created. However, when you are using migrations you may want to change your seed data when the database changes and using MigrateDatabaseToLatestVersion makes that possible.
To combine seeding with migrations you will have to perform the following steps in a new project:
Create a code-first DbContext with associated entities
In the package manager console execute the command Enable-Migrations
In the Migrations folder a Configuration class is generated with a Seed method. You can modify this method to seed your database:
protected override void Seed(MyContext context) {
// Add two entities with name "Foo" and "Bar".
context.MyEntities.AddOrUpdate(
e => e.Name,
new MyEntity { Name = "Foo" },
new MyEntity { Name = "Bar" }
);
}
You need to create a database initializer that derives from MigrateDatabaseToLatestVersion:
class MyContextInitializer
: MigrateDatabaseToLatestVersion<MyContext, Migrations.Configuration> { }
You will also have to configure the initializer either by calling Database.SetInitializer(new MyContextInitializer()) when you application starts or in the App.config file by using the <databaseInitializer/> element.
In the constructor for the generated Configuration class you can enable automatic migrations:
public Configuration() {
AutomaticMigrationsEnabled = true
}
However, in a team you might prefer to not do that. In that case you will have to create an initial migration (unless it was created when you did Enable-Migrations). In the package manager execute the command Add-Migration InitialCreate. This creates the first migration required to create your database.
At this point you have a DbContext with migrations and a Seed method.
So to sum it: Enable migrations, use the MigrateDatabaseToLatestVersion initializer and add seed data in the Configuration class that was generated when migrations were enabled.

While MigrateDatabaseToLatestVersion does actually create the DB if it does not exist and even allows you to seed it, if you already have a working solution based on CreateDatabaseIfNotExists and/or don't want to complicate it with testing for existence of seed data, you can just use the below by inheriting from it rather than from CreateDatabaseIfNotExists:
public class CreateOrMigrateDatabaseInitializer<TContext, TConfiguration> : CreateDatabaseIfNotExists<TContext>, IDatabaseInitializer<TContext>
where TContext : DbContext
where TConfiguration : DbMigrationsConfiguration<TContext>, new()
{
void IDatabaseInitializer<TContext>.InitializeDatabase(TContext context)
{
if (context.Database.Exists())
{
if (!context.Database.CompatibleWithModel(throwIfNoMetadata: false))
{
var migrationInitializer = new MigrateDatabaseToLatestVersion<TContext, TConfiguration>(true);
migrationInitializer.InitializeDatabase(context);
}
}
base.InitializeDatabase(context);
}
}
This is based on previous answers and OP's own solution. This should work with other providers as well, but I only tested with SQL Server.