I'm working on a .NET 4.5.2 project that creates an In-Memory db from an .mdf file every time we run the Integration tests.
The db works fairly well most of the time, but sometimes, when we make changes on the schema of a table, or add/delete a table altogether, we get problems like the following:
Message:
OneTimeSetUp: System.Data.SqlClient.SqlException : There is already an object named 'SomeTable' in the database
We have narrowed it down to the fact that EF migrations get all messed up when we want to update the test db schema after a merge (known issue). It seems like adding an empty migration everytime we have one of these issues fixes the problem, however, that workaround it's very tacky.
My question is: Is there a way to prevent this kind of issues? Maybe a cleaner workaround?
For some extra content, I'll describe how we're doing things (maybe we're messing up somehow):
We have a Db.mdf file that gets loaded by a LocalDbHelper before running any test:
With this code:
public static void PrepareEmptyDb()
{
var migrationConfiguration = new Configuration()
{
AutomaticMigrationsEnabled = true,
AutomaticMigrationDataLossAllowed = true
};
var migrator = new DbMigrator(migrationConfiguration);
migrator.Update();
}
This code is meant to be run in a OneTimeSetup at the IntegrationTestBase class.
And this is the code in the Configuration class (this is at our code first Persistance project, where all migrations reside):
internal sealed class Configuration : DbMigrationsConfiguration<Context.DbContext>
{
public Configuration()
{
AutomaticMigrationsEnabled = false;
}
protected override void Seed(Context.DbContext context)
{
context.Settings.AddOrUpdate(
p => p.Name,
new Setting { Name = SettingNames.RoundingDirection, Value = "Up" },
new Setting { Name = SettingNames.RoundingValue, Value = "10" },
new Setting { Name = SettingNames.RateCacheLifetimeMinutes, Value = "30" });
}
}
Any help would be greatly appreciated, since this issue has really been annoying us for a while now.
EDIT: I've found this link that seem to suggest that, for older versions of .NET, this workaround was the recommended fix suggested by Microsoft themselves, but if anyone has a better way to fix it (or to automatize it) it would be greatly appreciated.
Adding an empty migration seems to be the only way to fix this. No workarounds or fixes other than upgrading to Core, sadly.
Related
This is both a question and an answer. I've fixed my problem, but it seems a bit wrong.
My original problem is running my asp.net core integration tests in a bitbucket pipeline causes System.IO.IOException: The configured user limit (128) on the number of inotify instances has been reached. Some solutions call for changing some setting through sysctl, but that is restricted by bitbucket, so that isn't an option for me.
The second way of fixing this, as noted in these stackoverflow answers, is to turn reloadOnChange off.
My new problem is now, how do we best do this for the test WebApplicationFactory?
One solution that has worked for me, which is the least amount of code, seems like a total hack. I iterate through all the JsonConfigurationSource and set ReloadOnChange to false.
Full solution:
public class TestApplicationFactory : WebApplicationFactory<Startup>
{
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureAppConfiguration(config =>
{
foreach (var source in config.Sources)
{
if (source is JsonConfigurationSource)
{
var jsonConfigSource = (JsonConfigurationSource) source;
jsonConfigSource.ReloadOnChange = false;
}
}
});
}
}
Another solution, that I haven't tried, may be to override CreateWebHostBuilder(). However, it seems like more code and a lot of copy and paste from the default one.
Am I missing something? Is there a better way to do this?
Just experienced this issue myself running integration tests within a Linux container and followed the previous suggestions to switch off the ReloadOnChange within the WebApplicationFactory. Unfortunately that did not resolve the problem and integration tests were still failing with the same error:
System.IO.IOException: The configured user limit (128) on the number of inotify instances has been reached.
I also tried to configure xUnit to run the integration tests sequentially rather than in parallel, but that did not work either.
The solution that did work for me was to set the appropriate environment variable within the container that runs the integration tests:
export ASPNETCORE_hostBuilder__reloadConfigOnChange=false
builder.ConfigureAppConfiguration is there to configure your (main) application.
You can use builder.ConfigureHostConfiguration (see docs) to explicitly configure files to be read for the host.
builder.ConfigureHostConfiguration((hostingContext, config) =>
{
config.AddJsonFile("appsettings.json", optional: false, reloadOnChange: false);
});
The host configuration is loaded. ASP.NET Core from 3.0 is built based on Generic host (rather than the Web Host of the former versions).
You can do this without inheriting from WebApplicationFactory by using the WithWebHostBuilder and ConfigureAppConfiguration extension methods:
var webAppFactory = new WebApplicationFactory<Startup>().WithWebHostBuilder(webHostBuilder =>
{
webHostBuilder.ConfigureAppConfiguration((hostingContext, configBuilder) =>
configBuilder.Sources.Where(s => s is FileConfigurationSource).ToList()
.ForEach(s => ((FileConfigurationSource)s).ReloadOnChange = false));
});
This accomplishes the same thing as your original idea (which helped me a lot!), but more compact and without the need for a separate class definition.
I just encountered the same issue.
Setting the env variable DOTNET_hostBuilder:reloadConfigOnChange to false fixed it.
This solution works for net6 when you use the Generic Host. For other hosts, maybe try replacing DOTNET_ prefix with ASPNETCORE_
To make it simple, I set it in my code before creating the WebApplicationFactory
Environment.SetEnvironmentVariable("DOTNET_hostBuilder:reloadConfigOnChange", "false");
We run migrations from within our application to create databases on the fly. We also use migrations in our unit/integration tests to create databases for the tests to use. A few months ago we noticed the run-time of our tests substantially increased and we've been able to narrow it down to the instantiation of the DbMigrator class. Even with a very simple Configuration class it still takes ~10 seconds to create the DbMigrator.
For example, here's the Configuration class taken from the "Elastic DB Tools - Entity Framework" sample application.
internal sealed class Configuration : DbMigrationsConfiguration<ElasticScaleContext<int>>
{
public Configuration()
{
this.AutomaticMigrationsEnabled = false;
this.ContextKey = "CodeFirstNewDatabaseSample.BloggingContext";
}
}
Here is the code used to create and run the migrations.
var configurator = new Configuration();
configurator.TargetDatabase = new DbConnectionInfo(connStrBldr.ConnectionString, "System.Data.SqlClient");
var migrator = new DbMigrator(configurator);
migrator.Update();
And here's the trace information from VS2015.
Time to instantiate Configuration
Time to set the TargetDatabase property
Time to instantiate DbMigrator
We're using Entity Framework 6.1.3. Has anyone else experienced this?
I have used EF many times, but i cant seem to figure out why this is happening.
I have a solution with 2 projects, one is my common library used in a few other applications that has all my entities and one is my updater console application. In the common library i have my context which looks like so.
public class DalContext : DbContext
{
public DalContext() : base(ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString)
{
Database.SetInitializer(new CreateDatabaseIfNotExists<DalContext>());
}
public DbSet<UserAccount> UserAccounts { get; set; }
public DbSet<GameImport> GameImports { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Conventions.Remove<PluralizingTableNameConvention>();
}
}
then in my console application i go to create an instance of the context, and i get the error
"Object reference not set to an instance of an object".
I am using a basic using statement to create my context so everything gets disposed properly.
using (var db = new DalContext())
{
....
}
i get the error when the using statement is creating the DalContext object.
both applications are using the same version of EF 6.1.3. I was able to do an add-migration just fine. i dunno, i'm stumped.
I have referenced other applications where i have used this and i cannot find any differences as to why this would be happening. I know its going to be something blatantly obvious.
In this kind of situation, it's always a good practice to do a binary search debugging. That is, keep on removing stuff until it works. Narrow down the error. Until the bare minimum.
You can try:
Replace this by the actuall connection string: ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString. This is the most likely one. "DefaultConnection" may not be on your web.config/app.config file, that would explain why the same code works for other host projects.
Remove/replace this: Database.SetInitializer(new CreateDatabaseIfNotExists<DalContext>());
Remove/replace this: modelBuilder.Conventions.Remove<PluralizingTableNameConvention>();.
Given the info you have provided, it's impossible for us to help you further.
If I build an application and let code first figure out where to put the database and the user inserts data via the application, will that data get lost on a click once update ? If so, how do I approach this problem ?
Thanks
No there is No need to "lose" any data when using automatic migrations.
You migration config class should state no data loss allowed
You will need to build custom scripts/ or tweak the generated scripts when dealing with changes that result in data loss.
public override void MigrateDb() {
Database.SetInitializer(new MigrateDatabaseToLatestVersion<MYDbContext, MYSECIALMigrationConfiguration>());
// Context = GetDefaultContext(); // check if a new context is really needed here
Context.Database.Initialize(true);
}
public class MYSPECIALMigrationConfiguration : MYBaseMigrationConfiguration<MYDbContext>{ }
public abstract class MYBaseMigrationConfiguration<TContext> : DbMigrationsConfiguration<TContext>
where TContext : DbContext{
protected MYBaseMigrationConfiguration() {
AutomaticMigrationsEnabled = true; // you can still chnage this later if you do so before triggering Update
AutomaticMigrationDataLossAllowed = true; // you can still chnage this later if you do so before triggering Update
}
how to approach migrations.
.. Thats actual a big question.
EF6 Migrations - new features and options
Great info on Migrations when working in teams.
This covers many scenarios you may recognise and thus help you understand what approach suits you best.
We are intensively used Entity Framework 4.3 with a code first .NET 4.0 based project within several sites, everything works smooth, fast and bugless, but to enable the last speed improvements made by the Entity Framework team we are now planning an upgrade to EF6 and .NET 4.5
After amending a couple of using clauses (due to the namespaces changes in the latest releases) everything seems to working properly if starting with an empty db, but trying to open an existing customer database will get an exception as soon as the Context() construction happens, apparently something is wrong with the code based migration.
My code based migration configuration is the following:
internal sealed class Configuration : DbMigrationsConfiguration<Context>
{
public Configuration()
{
AutomaticMigrationsEnabled = true;
AutomaticMigrationDataLossAllowed = true;
}
}
With this application initialiser (program.cs)
Database.SetInitializer(new MigrateDatabaseToLatestVersion<Context, Configuration>());
Database.DefaultConnectionFactory = new UniversalDbConnection();
Having UniversalDbConnection declared as
class UniversalDbConnection : IDbConnectionFactory
{
public DbConnection CreateConnection(string givenString)
{
switch (Preferences.DatabaseVendor)
{
case ConnectionPreset.PostgreSQL:
return new PgSqlConnection(Preferences.ConnectionString);
default:
SqlConnectionFactory factory = new SqlConnectionFactory(Preferences.ConnectionString);
return factory.CreateConnection(Preferences.ConnectionString);
}
}
}
Unfortunately an exception claiming:
The object 'FK_StockTests_FormulaSteps_Step_Id' is dependent on column 'Step_Id'.
ALTER TABLE DROP COLUMN Step_Id failed because one or more objects access this column arises