For some integration tests I want to use LINQ to SQL to drop/re-create the test database. I've had this working fine before, however in this project the database is split up into several schemas.
When I try to run the ctx.CreateDatabase() command I'm getting this exception:
The specified schema name "xyz" either
does not exist or you do not have
permission to use it.
The login I'm using to do this has the role dbcreator - Does it need further permissions? Surely a login with persmissions to create a database should be able to create everything contained in that database also?
Update:
Since it looks like there isn't a solution to this problem using LINQtoSQL, does anyone have recommendations of any similiar tools to generate a db that are preferably free? Ideally I don't want to have to muck about hand writing sql build scripts.
From what I've read, the CreateDatabase() method is limited in what it can reproduce of the original database. It won't recreate things like triggers and check constraints, and I'm guessing it doesn't create custom schemas either. You may want to look into creating the database using a SQL Server .mdf file instead to work around this issue. See this blog entry for more details on some of the limitations of CreateDatabase().
I generally do this sort of work in NAnt to create, initialize the database, create users, add logins, etc....and also roll back capabilities. I have written on this topic quite a bit if you are interested:
Build automation with NAnt
Continuous integration with CruiseControl.NET
I will have to see if I can get LINQ to SQL to work in the way you are trying to use it...that sounds like what we used to do with NHibernate.
The dbcreator fixed server role grants you the permission to create a database. If you create a database, you are the dbo of said database and as dbo you have absolute power in the database, includding the power to create, alter and drop any schema and any object contained in any schema.
the problem with LINQ's CreateDatabase() is not permission, is code quality. The generated SQL code simply does not create the needed schema, so the Create table statements fail because the schema does not exist.
Your best choice, if you can afford it, is to add a VSTS Database Edition GDR R2 project to your solution and declare all your database objects in the Database Edition project (part of your solution). You'll be also getting the added benefit of storing all your database objects in a proper source control solution. The output of the Database project would be a .dbschema file containing the definition of your database. At deployment time (test or real) you would run the VSDBCMD Deployment and Schema Import tool to import your .dbschema into the target server. The tool is capable of doing initial deployment of your schema, as well as further upgrades (deploy only differences). The VSDB solution would allow you to controll all your database objects: tables, indexes, views, schemas, field contraints, table constraints, triggers, procedures, users, permissions, logins etc etc. It really covers all the objects that can be defined in SQL Server.
Actually LINQ to SQL does support schemas, but not every Sql Server edition does. To enable CreateDatabase() to generate them the DataContext must be aware that the target database does support them. It can be done by setting the provider on the DataContext:
[Provider(typeof(Sql2008Provider))]
public class CustomDataContext : DataContext {
...
}
Your user also requires db_dlladmin for that database.
I would definately look at Entity Framework, which I am beginning to look into these days. It's an OR/M, and will most definately suit your needs, and alot more once the next version is released.
Entity Framework is also a brain-child of Microsoft and can be found here: http://msdn.microsoft.com/en-us/library/aa697427(VS.80).aspx
One thing to remember between LINQ to SQL and LINQ to Entities is that you are programming against a model, and not the database.
Related
Before I posted this question, I did some Googling first on how a database was created through C# and mostly it points to either SMO or SQL query files and it was the time of SQL Server 2005 and 2008.
So at this day in age, is there an easier way to create a database with empty tables, tables with data in them by default, stored procedures and views?
I need a suggestion.
I think the answer is probably Entity Framework. You can do 'code first' and use database migrations, allowing you to write your C# code and use that to generate a lot of the database for you.
Ultimately though, 'easier' is subjective. I personally find EF great for the 'normal' stuff, but at the end of the day, if you need a stored procedure to do some custom logic; you need to write the custom logic, in some fashion.
Maybe have a look and see if you think it fits your needs.
https://www.asp.net/mvc/overview/getting-started/getting-started-with-ef-using-mvc/creating-an-entity-framework-data-model-for-an-asp-net-mvc-application
Looked at the database projects in studio 2013. You create a database as a series of scripts using a familiar GUI. However, changes are published - this process creates a unique change script targeting the connection you define. For new databases the whole thing gets created, but publish against a partial or out dated version and the script created in a change script to bring it up to date.
You can even writ unit tests against your database using specialist tools, although I do find them lacking a bit.
More on msdn - here
Depends. right out of gates. Sp and views. Best shot is directly from database through a workbench. I can then capture definitions and store in a file to be replayed through c#
As for tables there are many orms that can generate tables via c#. Look at entity frameworks. Code first examples
I have generated tables using EF Works fine. I then went into database and created views and sps.
The trick is to migrate new views and sps into your EF model U can google entity Frameworks code first ... Adding views and SPs.
Worst case is u create database all through database workbench. Create a script that an be played to recreate eveything. By running. Then use EF DATABASE first approach
In either case u end up with a good set of autogenerated code to manage CRUD and object management and an abstracted data model
We have a project for school in which we have to create a web application. We have access to an online database, but the problem is that we have to connect through VPN before we can connect to it.
For this reason, we're looking for a possibility in which we have a local database (which would be in the project, I suppose?) that we can all use (the project is on a subversion server). But when we deploy the project on our deployment server, we want it to use the real database connection.
I think I've seen it before, but after searching for hours I couldn't find anything relevant.
Is this possible?
EDIT:
We use MVC5 with Entity Framework.
People typically do this in one of several ways depending on what you want to achieve.
Pull down the database as an mdf file and store it in your repository. You can then have a manual step during setup where you ask people to import it into their database (I recommend localdb for local development but sure, you can use sql server or something).
Advantages: Very simple to set up for the person arranging this.
Disadvantages: Manual step is difficult for beginners. If the database is large it will swell your repository. If one developer changes the database (for example by adding a column) then you have to let everyone know to blow away their copy and restore from backup. Also, there is no real explicit history of how your database changes and your test database is not integrated with whatever you have to do for deployment.
Pull down the database as an mdf file. Include this file in your project and set it's properties as Content/Copy if Newer. Then use it directly using a connection string to attachDb such as Server=(localdb)\v11.0;AttachDbFilename=.\MyDataFile.mdf;Database=dbname;.
Advantages: No manual step, everything just works
Disadvantages: Obviously you'd want to use relative paths for AttachDbFilename and I'm not 100% that this is supported. Also, same as above but instead of having to let everyone know when their db needs restoration it just restores behind the scenes. This can mean users suddenly see their data disappear with no notice. It can also fail sometimes due to things like a locked database file and everyone just has to get good at keeping an eye out for that.
Maintain a sql script that can recreate your database in localdb. Provide people with a powershell or batch script (also in source control) to run it easily. Optionally use a post-build script that determines if you need to recreate the database and runs it.
Advantages: Everything is very explicit. Reasonably small size in the repository (which should be able to store text efficiently). You can use the same script as part of deployment.
Disadvantages: More work to set up. Still no real way to deploy changes to existing databases.
Use Entity Framework Database first. I can't speak to what the process looks like exactly when doing this but I know that it is possible.
Advantages: I guess.
Disadvantages: Ewwww EF database first
Use Entity Framework Code First with Migrations. Use explicit migrations (not the silly auto-generate-my-entire-db cruft) and write a proper Seed method to populate your data.
Advantages: This is what professional developers do and is based on tested patterns used frequently by Rails, Django, and many other frameworks. It is very flexible and explicit and supports changing existing databases.
Disadvantages: Can be quite difficult to set up if you don't have experience and especially if you're unaware of the migrations pattern. There's some naming difficulties that make it kind of hard to google (database first EF vs code-first EF, explicit migrations vs auto-generate-the-db, several different Seed methods that depend on your initializer).
You can create a local database using Entity Framework, which saves the database file on the local filesystem, which you can push to your version control server to share with your colleagues. If you decide to deploy the database, you can generate an sql script that you can run on the production database. You can do so by connecting to the local database using SQL Management Studio. You will just need to modify the connection string of the published application after deployment.
You used to be able to use Sql Server Express but it has changed to LocalDB and can be installed and run locally.
http://blogs.msdn.com/b/sqlexpress/archive/2011/07/12/introducing-localdb-a-better-sql-express.aspx
https://msdn.microsoft.com/en-us/library/ms233763.aspx
We have a process where our database guys script changes (and version them using Juneau) to our application's database out-of-band with our code base. They're good at accounting for new columns being null, and not wiping existing data, but occasionally a column rename sneaks in that isn't fully communicated. So they will make some changes to the database schema on a testing server, we'll update Entity Framework to work with those changes, and then commit our code. This process works okay, except for when it's time to deploy.
We have TFS set up to deploy the successful build to the appropriate servers, but there's no guarantee that the database for that environment has been updated. We don't care if extra fields/tables/views/etc. exist in the target database, but we want change the build to check that the database contains at least everything EF is aware of.
I looked at this question, but I don't need the schema to match exactly. Plus, we don't want it creating/modifying the database directly. And this question seems like it's trying to achieve a similar ideal, but still not quite what we're looking to achieve. We just want a integration test of sorts to verify our version of EF will work with the target schema.
I wonder why you try to deploy your application without changes to database. Your application is dependent on the database so the deployment should always be done after the database. It looks like you are going to invest a lot of time to develop validation to fix your incorrect deployment process (where fixing the process itself is the correct solution).
Anyway you can create some "validation" of the database but it will take some time. If you are using EDMX file you can open it as XML and read its SSDL part which describes all expected tables, columns, relations, views (in form of SELECT SQL queries), stored procedures and functions. You can parse this XML part and use system database views (sys.tables, sys.columns, ...) to query if these objects exists in the database.
Another approach can be using database diff. tool to compare your current test database with the target one. This will require the tool which can be executed from command line and you will have to parse its output to find breaking changes.
Currently, changes to the database are made through the SQL Server Management program. IF a table changes, sqlmetal is run to regenerate the linqtosql classes and development continues. However, this makes deployment a pain, as you have to go through and manually update the deployment database (and any other databases used in the development cycle). It would be nice if we could use C# to generate these changes, as it would help eliminate human error and have the added benefit of being able to keep the database structure in git. Right now, the only representation of the database is in the generated linqtosql classes.
I've been looking around for a nice library that can handle this sort of thing, but the main solutions seem to be: keep a sql generation script, or embed sql in C# classes that can be run to make changes to the database. Both of these seem to be very non-ideal situations, as you lose the nice strong-typing that C# provides. It seems like there should be a way to do this using pure C#.
I've seen hints of being able to do things like generate databases from POCOs using both the entity framework and linqtosql, but I'm having a hardtime finding specific examples of that being used. Additionally, I haven't been able to discover if those have a graceful (i.e. data preserving) way of handling changes to the database after the initial table generation.
Are there any projects out there that solve this problem?
There exist several tools that help you with schema (and data) migrations of your database: RikMigrations, Migrator.Net and Machine.Migrations. Hope that helps.
Wizardby looks also promising: It provides database independent DDL scripts and automated migrations between different versions of a schema.
VS2010 can operate version control on your database schema through a Database Project. There are other tools out there for DB development that offer version control, you'll need to search to find them and compare pricing.
I prefer to version using sql scripts. Works pretty well, is free, supports updates, easy to version, works well with traditional source control methods.
First,
Create your DB
Use the Database Publishing Wizard to publish the database as a .sql script
Add a version number to the script
Add to your solution
Check into source control.
As updates are made,
Script updates to the previous schema as .sql files
Add a version number to the script that is incremented from the previous version
Add to your solution
Check into source control.
It sounds like you need a tool like Migrator.NET to manage your database migrations. We use it with a call from our site start-up to migrate the database as needed for any particular version.
I have toyed with an idea for creating a cleaner interface and someday hope to get around to implementing it, but other priorities have pushed that back. For now we are using raw sql strings in our migrations because there isn't a sybase driver implementation (outside of a very ugly hack I have written to manage the versioning table).
Redgate software offers something that may be really useful for you. It's called SQL Packager and it does it's job pretty well.
Features:
Easy roll-out of database updates across your client base
Script and compress your schema and data accurately and quickly
Package any pre-existing SQL script as a .exe, or launch as a C# project
Simplify deployments and updates for SQL Server 2000, 2005 and 2008
They also offer SQL Source Control which also may be useful to keep things nice and easy.
As an addon to MadBoy, SQL Packager can also launch the package as a C# project.
Red-Gate's SQL Compare is excellent as well, and as some of the banners on SO indicates, there is new SQL Source Control available as well.
Then they have their SQL Comparison SDK.
The trick here is to rely on the database being the single source of truth for your Linq schema, not the generated classes.
We use Linq to SQL extensively in our dev shop, and work as follows:
1. Create your database (working copy) from version control (baseline).
2. Modify your database any which way you like.
3. Generate Linq to SQL classes from the (working) database.
4. Create patches to update your baseline database to your working copy.
5. Check in and share these patches with all developers.
For a very quick and easy way of generating baseline and working copy databases, try DBSourceTools. http://dbsourcetools.codeplex.com
Have fun.
I am currently working on a project that include the use of SQLServer. I would like to know what strategy I should use when I install the software to build the database? I need to set up the tables, the stored procedures and the users.
Does my software need to make a check on start up to see if the database exist and then if it doesn't, create it up?
Is there any way that I could automate this when I install SQLServer?
Thank you.
EDIT
Ok right now I have plenty of nice solution, but I am really looking for a solution (free or open source would be awesome) that would allow me to deploy a new application that needs SQLServer to be freshly installed and setuped to the needs of the software.
RedGate software offers SQL Packager which gives you option to produce a script/.exe to deploy whole db including everything (stored procedures, tables etc) from one single .exe. If you can afford it and want to have easy way to do it (without having to do it yourself) it's the way to go ;-)
Easy roll-out of database updates across your client base
Script and compress your schema and data accurately and quickly
Package any pre-existing SQL script as a .exe, or launch as a C# project
Simplify deployments and updates for SQL Server 2000, 2005 and 2008
You could use migration framework like Migrator.Net, then you could run the migrations every time your application starts. The good thing about this approach is that you can update your database when you release a new version of your software.
Go take a look at their Getting started page. This might clear up the concept.
I have succesfully used this approach to solve the problem you are confronted with.
You do all of that with the SQL scripts. And then your installation program runs them against the customer's SQL Server.
You can write a T-SQL script that only creates objects when they do not exist. For a database:
if db_id('dbname') is null
create database dbname
For a stored procedure (see MSDN for a list of object types):
if object_id('spname', 'P') is null
exec ('create procedure dbo.spname as select 1')
go
alter procedure dbo.spname
as
<procedure definition>
The good thing about such scripts is that running them multiple times doesn't create a problem- all the objects will already exist, so the script won't do anything a second time.
Setting up the server is pretty straight forward if you're using MS SQL Server. As for creating the database and tables, you generally only do this once. The whole point of a database is that the data is persistent, so if there's a chance that the database won't exist you've either got a) major stability problems, or b) no need for an actual database.
Designing the database, tables, and procedures is an entire component of the software development process. When I do this I usually keep all of my creation scripts in source control. After creation you will write the program in such a way that it assumes the database already exists - checking for connectivity is one thing, but the program should never think that there is no database at all.
you can make a script from all of objects that exist in your db. after that you can run this script from your code.
when you create your db script with script wizard in sql server, in "choose script options" section, set "Include If Not Exist" to yes. with this work done only if db not exists.