I have a .net core solution that contains a database project (class library) and a console application. The database project contains EF Migrations and to do Add-Migration, most methods use a hard-coded connection string in one place or the other.
To avoid hard-coding (and/or duplication) I have created a shared appsettings.json file in the solution root and I use it in my Main method and the class library
In the console application
static void Main(string[] args)
{
var settingPath = Path.GetFullPath(Path.Combine(#"../appsettings.json"));
var builder = new ConfigurationBuilder()
.AddJsonFile(settingPath, false);
var configuration = builder.Build();
var services = new ServiceCollection()
.AddDbContext<MyContext>(options => options.UseSqlServer(configuration["ConnectionStrings:MyDatabase"]))
.BuildServiceProvider();
}
And in the class library to use migrations
public class DesignTimeDbContextFactory : IDesignTimeDbContextFactory<MyContext>
{
public MyContext CreateDbContext(string[] args)
{
var settingPath = Path.GetFullPath(Path.Combine(#"../appsettings.json"));
var builder = new ConfigurationBuilder()
.AddJsonFile(settingPath, false);
var configuration = builder.Build();
var optionsBuilder = new DbContextOptionsBuilder<MyContext>()
.UseSqlServer(configuration["ConnectionStrings:MyDatabase"]);
return new MyContext(optionsBuilder.Options);
}
}
This is working well for development purposes when I use dotnet run but when I publish the console application, it doesn't include the appsettings file. Other than running a powershell script as part of dotnet publish, is there any other cleaner way of including this file when the project is published?
IDesignTimeDbContextFactory is exactly for the purpose its name describes. You shouldn't be running migrations against your production database in the first place, and if you do, you should be generating specific migrations for production into your app (instead of the class library) and using your app for the migrations. See the docs on using a separate project for migrations. That, then, negates the need to share your appsettings.json. Just leave the connection string hard-coded in your factory, since it's only for development anyways.
Now, you might have an issue I suppose in a team environment. However, even if you're using something like SQLite, you can use project-relative paths that won't be developer-specific, and with LocalDB, you can use a normal SQL Server connection string to the MSSQLLocalDB instance, which will be same for every developer using Visual Studio. Regardless, even if you do need to specify the connection specifically by developer, at that point it would make more sense to use user secrets, anyways, since you wouldn't want that info be committed to source control. Otherwise, each developer would end up clobbering the other's copy of appsettings.json, and you'd have a mess on your hands.
Long and short, just hard-code the connection string in your factory, or if you can't or won't, use user secrets for the connection string. In either case, you do not need to share appsettings.json.
The way I've done this before is to specify the startup project when you run dotnet ef (with the -s switch - the options are at https://learn.microsoft.com/en-us/ef/core/miscellaneous/cli/dotnet#common-options)
It gets messy quickly, and it's probably easiest to write some wrapper scripts for the project that deal with this kind of thing.
Related
I'm using a clear architecture for a NET7 project with Blazor. In the Persistence layer, I have a function to register the database and all the repositories
public static class PersistenceServiceRegistration
{
public static IServiceCollection AddPersistenceServices(
this IServiceCollection services,
IConfiguration configuration,
string cnnStringName = "LIUContextConnection")
{
var cnnString = configuration.GetConnectionString(cnnStringName);
services.AddDbContext<LIUContext>(options =>
options.UseSqlServer(cnnString)
);
services.AddScoped<IArticleRepository, ArticleRepository>();
return services;
}
}
So, in the Program.cs I can register the persistence like
builder.Services.AddPersistenceServices(builder.Configuration);
I want to be sure that the database is created before the application starts. I added the following code an the end of the Program.cs
var app = builder.Build();
LIUContext dbcontext = app.Services.GetRequiredService<LIUContext>();
dbcontext.Database.EnsureCreated();
await app.RunAsync();
When I run the application, I get an error because
Microsoft.Data.SqlClient is not supported on this platform
I added in the Client project and in the Persistence project the NuGet package but I get the some error.
If I create the database from the Package Manager Console is working.
"is not supported on this platform" is of course the main point.
The platform here is dotnet in the Browser and a lot of APIs are not (cannot be) supported. The SqlClient needs sockets and they are not allowed in the browser. Also, you wouldn't be able to keep the credentials secret.
There is some limited support for SqLite and you have the built-in IndexedDb.
For a full SQL Db, the common approach is to use a WebService (Blazor Wasm Hosted template) to handle the Db access.
I need to use the dotnet ef tools utility, but I need to set a connection string in appsettings.json.
I don't want to store it in appsettings.json, because project is stored in the git repository.
At the moment, I keep launchSettings.json in the EnvironmentVariables section, but this does not help me because dotnet ef tools simply does not see these values. I tried to implement IDesignTimeDbContextFactory but everything is the same. He just doesn't see these values.
I found a solution that requires you to explicitly enter the value of the connection string, but this option also does not work.
The thing is that you should not store any secret value in git/repository at all.
Ideally this connnection string should be in a vault/Secret place which you will have access using another password, so eventually you need to "store" a password somewhere.
There is differnt approaches for this, as mention the most common one will be something like _secret.GetConnectionString() -> this calls to another service which will return the connectionstring.
Other solution you can do (probably better in your case) is to "setup" a default/plain appsettings.json just store the value like connection: xxxxxx and then use environment variables to replace that value. The configuration builder is already built with this in mind.
var configuration = new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.AddEnvironmentVariables() //This line here
.Build();
Then you can just set up the value in your own machine.
For deploying to production it will depends which software you use to deploy the app, but it will have some way to allow you to add environment variables.
Finally, as an extension of the previous code you can add your "own" json
var configuration = new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.AddJsonFile("appsettings.private.json")
.Build();
So doing this, in appsettings.private.json you will only need to write the connectionstring property. You will not need to write the entire document.
and then you can just add a rule on .gitignore to ignore it.
I'm using ASP.NET Core data protection with default DI behavior. That works fine when my ASP.NET application is hosted on IIS. Now I have an application that needs to run as a service. So I'm using Microsoft.Extensions.Hosting.WindowsServices to do the windows service part with our standard
Host.CreateDefaultBuilder(args)
.UseWindowsService()
The BackgroundService then hosts ASP.NET Core with your standard
var builder = Host.CreateDefaultBuilder()
.ConfigureAppConfiguration((hostingContext, config) =>
{
config.AddJsonFile("secrets.json", optional: true, reloadOnChange: true);
}
.ConfigureWebHostDefaults(....)
inside the background service, I can then resolve an instance of IDataProtectionProvider, create a protector, and use it to unprotect my secrets
var dataProtectionProvider = Container.Resolve<Microsoft.AspNetCore.DataProtection.IDataProtectionProvider>();
var protector = dataProtectionProvider.CreateProtector(appName);
var decryptedSecret = protector.Unprocect(some secret)
Now that all works fine as long as I run my application from the CLI. But running it as a service (same file, same location, and of course under the same account), I get an 'invalid payload' exception when I call Unprotect.
I know same path and same account is important, so that's taken care of. I also know that the application can find secrets.json as I wrote some probing code that checks if the file is present and can be read before I even try to unprotect. I'm even checking if the string I'm trying to unprotect is null/empty (which it isn't).
I finally installed a debug build as a service and attached the debugger, and when I look at IDataProtectionProvider, it has a Purpose.. and when running as a service, that's c:\windows\system32. When my app runs from the CLI, it's the path to the exe. So, is there a way to specify the purpose on my own so things behave the same regardless of CLI/Service?
So how can I control the purpose?
So having noted the difference in purpose of the IDataProtectionProvider, I was well on my way of solving this. The solution was to set a static purpose as explained here
I have 3 projects in a vs2019 solution which should use the same sqlite database using Entity Framework Core.
First project: an .net core class library with all database models, database context, database repositories and migrations.
Second project: an .net core console application which needs to scan files, retrieve metadata from inet and seed the database with retrieved content. This is done by referencing the first project and use the repositories to access the database.
Third project: an Asp.net core Webapi which need to accept client requests for retrieving, updating, etc data from the same database. This is also done by referencing the first project and using the repositories. This project is the main project and needs to start the second project when needed.
All projects need to be able to get installed on client machines targeting Windows, MacOS and Linux. And be self hosted. It has to act like a content server simular to Plex, Emby and Calibre.
My problem is how to create the datasource option string to be able to use the same db file and have that file copied to the location of the Asp.net core Webapi project.
I now have this in my dbcontext file at the first project:
public class ApplicationDbContext : DbContext
{
public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options) : base(options) { }
#region DbSets
public DbSet<Authors> Authors { get; set; }
...
...
public class DesignTimeDbContextFactory : IDesignTimeDbContextFactory<ApplicationDbContext>
{
public ApplicationDbContext CreateDbContext(string[] args)
{
var builder = new DbContextOptionsBuilder<ApplicationDbContext>();
builder.UseSqlite("Data Source=Chapter.db");
return new ApplicationDbContext(builder.Options);
}
}
And this in my repository class in the first project:
public class GenericRepository<T> : IGenericRepository<T> where T : class
{
private DbContextOptionsBuilder<ApplicationDbContext> _optionsBuilder;
public ApplicationDbContext _context = null;
public DbSet<T> table = null;
public GenericRepository()
{
_optionsBuilder = new DbContextOptionsBuilder<ApplicationDbContext>().UseSqlite("Data Source=Chapter.db");
this._context = new ApplicationDbContext(_optionsBuilder.Options);
table = _context.Set<T>();
}
When I use entity framework to create the database from the migrations in the first project, it creates the Chapter.db file in the main folder of the first project.
In my Asp.net Webapi project i have this to use the database:
public void ConfigureServices(IServiceCollection services)
{
services.AddDbContext<ApplicationDbContext>(options =>
options.UseSqlite("Data Source=Chapter.db"));
Do i need this line of code or can i use the repository classes from the first project?
When I build the solution I have all the projects their dll's in the debug folder of the third project but not the chapter.db database file.
How do you copy the db file automaticly in the correct projects output directories when building for debug and release?
And what do I need to use for the Datasource= connection string to be sure that after people install the application it targets the db file (in windows, linux and macos)?
I have read two answers from simular questions to use:
Var SqlitePath = Path.combine(
Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData),
#"<YourAppName>\Chapter.db");
And:
var builder = new SqliteConnectionStringBuilder(connectionString);
builder.DataSource = Path.GetFullPath( Path.Combine(
AppDomain.CurrentDomain.GetData("DataDirectory") as string ?
AppDomain.CurrentDomain.BaseDirectory, builder.DataSource);
connectionString = builder.ToString();
Are these answers having the same outcome? And am I able to use one of these in the asp.net core project aswell as in the .net core console application?
And does that work cross-platform?
I'm a bit late and I'm assuming that you have already figured this out, but I'm writing this answer since it was a top result to one of my google searches.
The issue at hand is pointing to the correct location of the database.
public void ConfigureServices(IServiceCollection services)
{
services.AddDbContext<ApplicationDbContext>(options =>
options.UseSqlite("Data Source=Chapter.db"));
This line of code will be creating a database called "Chapter.db" within the project directory of whichever project you are running it in. The solution to the problem was provided at the bottom of your question.
Var SqlitePath = Path.combine(
Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData),
#"<YourAppName>\Chapter.db");
This code will convert the relative path to the database (based on the project directory) to an absolute path (one based on your file system instead, i.e. C:...\Chapter.db or /home/.../Chapter.db). The other blob you provided should do the same thing, but with more lines of code. After reading SqliteConnectionStringBuilder documentation it seems that the later has more functionality allowing for passwords and other fun stuff to be added through code rather than manually put through a string.
Absolute pathing will help out in this scenario as it will allow for you to specify a direct path to the database file you are using. Otherwise, you would need to modify the connection string to use relative pathing for each of your projects which could become complex if they were to ever move to different parts of your file system. It can still be done. For example if both projects are in the same location, appending "../" in front of your database name would suffice in providing the same location to all of your projects.
Using absolute pathing, you could just specify a location like "C:\Chapter.db" and all of your projects would use the database at that location rather than their individual files.
How do I build a service that needs access to my database while also using migrations, given that the migrations tool "dotnet ef database update" runs my application BuildWebHost before creating the database? The attempt to configure the service needing the database throws an exception because the database doesn't exist and causes the migration command to fail. The database therefore never gets created.
I'm using asp.net core 2 and EF Core 2.
More specifically, running "dotnet ef database update" with a blank database fails with the following error:
An error occurred while calling method 'BuildWebHost' on class
'Program'.Continuing without the application service provider. Error:
Cannot open database "MyDb" requested by the login. The login failed.
Login failed for user 'MYCOMPUTER\MYNAME'.
This happens because I have built a custom configuration provider backed by my "MyDb" database (with the end goal of binding to it with a custom options class) as per this Microsoft Configuration Tutorial My program.cs looks like:
public static IWebHost BuildWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((builderContext, config) =>
{
var tmpconfig = config.Build();
config.AddMyOptionsConfig(options => options.UseSqlServer(tmpconfig.GetConnectionString("My_Database")));
})
.UseStartup<Startup>()
.Build();
and the AddMyOptionsConfig eventually runs:
public class EFConfigProvider : ConfigurationProvider
{
[...]
// Load config data from EF DB.
public override void Load()
{
var builder = new DbContextOptionsBuilder<MyDbContext>();
OptionsAction(builder);
using (var dbContext = new MyDbContext(builder.Options))
{
// dbContext.Database.EnsureCreated(); // will cause first migration to fail
Data = !dbContext.ConfigurationValue.Any() // throws exception
? CreateAndSaveDefaultValues(dbContext)
: dbContext.ConfigurationValue.ToDictionary(c => c.Id, c => c.Value);
which throws an SQLException when it attempts to access the database because the database hasn't been created yet.
However, if I try to use dbContext.Database.EnsureCreated() then the initial migration fails because the tables already exist. I thought of trying dbContext.Database.Migrate() instead but as a beginner I'm concerned there might be unintended consequences for a production environment. As such I'd prefer to have control over migrations via the command line tools.
Fundamentally, the problem seems to be that "dotnet ef database update" runs the application startup - BuildWebHost - before it creates the database but the custom configuration provider added in BuildWebHost needs the database to already exist.
How do I solve this dilemma?
In a sense, you've created a circular reference. When running migrations on a ASP.NET Core project, the application is initialized to instantiate the DbContext needed to run migrations against. This is due to the fact that DbContext in EF Core now requires a DbContextOptions instance to be injected, as opposed to the old way of doing things in EF, where the connection string name (or the actual full connection string) would be defined directly on the constructor.
Normally, this would work just fine, but, as you've noticed, because the application initialization, itself, requires an already existing database, there's no way to run this before the migrations. As a result, you have two options:
You can attempt to abstract the piece that requires an existing database. That could be as simple as wrapping it in a try-catch and swallowing the exception, or more complex. Since the migration piece doesn't need this particular functionality, it could safely be excluded from the application initialization in that scenario.
You can move your context and entities into a class library and implement an IDesignTimeDbContextFactory to satisfy the ability to construct a DbContext to migrate against. You would then run the migrations against this class library instead of your ASP.NET Core project. That then sidesteps the issue of having to initialize the application in order to do a migration.
BuildWebHost (and Startup.Configure) shouldn't be used for application startup logic. The guidance from the ASP.NET team is to use Program.Main instead.
My solution so far has been to move the entity backing the custom configuration provider into a separate dbContext.
Now the table can be created when needed via MyDbOptionsContext.Database.EnsureCreated() without affecting the migrations, as they are based on the main context (MyDbContext).
The options table no longer participates in migrations but as it's unlikely to change there shouldn't be a problem. The other downside is the need (due to having 2 contexts) to explicitly specify the context for the dotnet ef command line tool.