Our company inherited some software that runs on C# Visual Studio 2010, Windows 7 and Oracle 11g. After some effort we got the software working and got a stable database (schema) set up.
We are now starting the process of migrating some data from an old system to this "new" system. However, I don't want to mess up our working schema as I expect a bit of trial and error work will be needed with our data import.
I wanted to do the following:
Let's say our existing schema is called PROD. I wanted to create a second schema called TEST that we can use for the imported data. Then, in the C# code I can just switch the name of the datasource when switching between our two database schemas. The catch is that the username and password for this connection appears in a multitude of places scattered in the code. To avoid having to change user credentials in multiple places every time we switch between "db environments", I wanted to create a single user to have access to PROD and to TEST.
However, how to grant user privilege on specific schema? suggests this is not possible. Correct way to give users access to additional schemas in Oracle suggests a method for granting access on an object level, but this is insufficient: I basically want one single user to have access to two identical schemas (PROD and TEST). Once I've achieved this, I want to start modifying TEST to start with our data import.
I have also tried creating TEST as a separate Oracle Database installation on a different port, but when trying to create my user on this new instance I still get a conflict that the user already exists (since it was created for PROD in the original database installation).
My user already exists and has access to PROD. How do I give him access to TEST as well? Or how would one solve the more general problem of having a PROD and TEST database defined in an application that uses Oracle?
In MySQL this would be trivial, but I don't have any idea how to do this in Oracle. I am very new to Oracle.
The question of giving permissions has already been answered.
Now to your question, as a whole: Am I reading correctly that you want to update the database schema, but you want to keep it in the same database as another schema and run both in what appears to be a production database? If so, read that again to let it sink in how extremely dangerous that is.
When migrating from one "schema" to another, as a software update, it is safer to create a new database and migrate the data. This gives you plenty of shots, as you can blow away the new database as you tweak scripts.
If you want as little friction as possible in your software, you need to do a couple of things:
Refactor out the code from the moron who decided to hard code connection information in multiple places. You need to get the strings in one place and make sure you extract out the Data access layer (DAL) code into its own class.
Consider creating domain objects that do not rely on the database schema(s). I consider this mandatory, but you could get away without doing this. I would still create domain objects, even if they match the PROD schema tables, as you should not be using data constructs if you are moving from one schema to another.
Create an interface for your data access layer (DAL)
Map the current data schema, through the current DAL, to the domain objects, using the interface.
Map the new schema, through a new DAL, to the domain objects, using the interface.
Create a factory (or use the provider pattern) to determine which DAL object you are going to use (this makes the application configurable to old or new "schema"
I'm assuming that you have a schema PROD and a DBUSER which have some privileges to objects in this schema.
DBUSER's name and password is hardcoded all over the application.
You've created a new schema TEST which looks the same as PROD (including grants to DBUSER).
You want that wherever the application does something like:
UPDATE some_table set ...
It will update some_table table in TEST and not in PROD.
My suggestion is to use and change SYNONYMS, ie-
When you want to update some_table in PROD, do:
CREATE OR REPLACE PUBLIC SYNONYM some_table for prod.some_table;
and when you want to update some_table in TEST, do:
CREATE OR REPLACE PUBLIC SYNONYM some_table for test.some_table;
The connection to Oracle is not handled correctly in the C# code and this is what is causing difficulty.
If the Data Access Layer were defined separately as Gregory suggests or if a more generic naming convention were used in SQL statements as A.B's answer points to, then it would be much simpler to switch between two databases.
Since our mandate currently doesn't involve making any changes/refactoring of code, I am using a backup and recovery approach:
I create a backup of the working database. Then I do the necessary tests and changes on the database. If I need to revert back to the working database, I create a backup of the "testing" database again and restore the original working database, using the appropriate flag to replace existing tables in the case of a restore. This enables me to switch back and forth between the "working" database and the "test" database.
This is not ideal as it does take some time to execute the backups and restores, but works without affecting the C# code and gives the ability to do work on a "testing" database without affecting the working one. Since this is a temporary scenario until the "testing" database becomes working, this is the approach I'll be following.
As the other answers point out - there is a more generic need to fix/refactor/generalize the connection code - I believe that is the best approach and the only reason I'm not doing that immediately is because we are not yet mandated to change the code.
Related
I am working on a project where I need to create a database to track the status of units throughout the production process. My current blockade involves getting the users to interact with a DataGridView that is supplied from a Microsoft Access Query instead of a Microsoft Access Table.
What I want to do is create a query in Microsoft Access and have it link to the DataGridView so end users can interact with a query instead of the actual tables, while populating all parent tables.
I am not sure if what I am attempting to do is possible or advised. This is the first time I have built a database in the professional world and want to make sure I am doing things properly. I have also never built a C# application for business use and have very limited experience with the language itself.
I have tried creating the Query in Access and linking it to the application in the same way you would add a table from a data source. That would allow me to view the data in the query...but it would display as a read-only and not allow for any data to be altered (the query builder in the TableAdapter Query Configuration Wizard indicated it was a read-only) . I have tried adding all related table adapters to the TableAdapterManager and it still didn't help.
I apologize if this question sounds disjointed as I am trying to overcome one obstacle at a time and do not want to overload one question with multiple issues. I can supply my ERD if it will make things easier and I have it normalized to at least 2NF.
Both of these work in my app:
INSERT INTO PLATYPUS (Bla, Blee, Bloo, Blah) VALUES (:Bla, :Blee, :Bloo, :Blah)
INSERT INTO CRITTERS.PLATYPUS (Bla, Blee, Bloo, Blah) VALUES (:Bla, :Blee, :Bloo, :Blah)
...Is one way preferred over the other?
If you are using multiple schemas, then no. It's all about context. Explicit is better than implicit.
After working on a number of systems that have used both implicit and explicit (I call them "hardcoded") schema references, I've found that in all cases, hardcoding the schema name in application code makes life more difficult.
This is why Oracle has synonyms.
The only time I hardcode schema names is in deployment scripts, e.g. when creating an object I want to explicitly state which schema the object should be created in.
It means that when the developers ask, "can we have a copy of our dev database in the same instance?", I can say, "no problem - give me a few minutes". I create a new schema, copy the tables etc. into it, then update their login user's synonyms to point to the new schema. Voila, two databases on one instance. However, if I let them hardcode a schema name in the application code, this becomes impossible because synonym translation doesn't get done.
They both work because you're probably logged in as a user whose default schema is CRITTERS. If you log in as another user when the default schema is something different, you will have to use the qualified form of your query.
The first statement in any Oracle application is explicitly to state which objects you will access:
alter session set current_schema = MYSCHEMA;
This way I can point my application to different schemas (databases) within ONE database server (instance). So never login as the schema-account. The schema-account should be locked, and only opened during ddl-changes (upgrades).
Let's say I am a broadcaster and I run several channels: RTL1, RTL2, RTL3, RTL4, etc... The same application can logon to different databases (Oracle term schema) within the database server
Now I run my application for RTL1:
alter session set current_schema = RTL1;
select * from top_stories;
Now I run my application for RTL2:
alter session set current_schema = RTL2;
select * from top_stories;
etc...
The #1 design flaw I see is the one-to-one relationship: application - database server.
It makes administration, storage and backup a fulltime job.
Oracle can run thousands of applications and databases/schemas on one database server.
But as always, it depends. Sometimes it makes sense to run one application on one database server.
So I try not to prefix and design my applications to be installed in any schema. If I need to access objects in other schemas, I make use of views and/or synonyms.
This approach has worked very well and I know what objects I am accessing by querying:
select sys_context('USERENV','CURRENT_SCHEMA') from dual;
I made an application that generates reports based on data from a database.
The functionality of my application is correct, but I have a following problem: my client has 2 identical databases - one for testing and one for actual work he does.
My application should work with both databases (it should have a "switching mechanism"), but I don't know how to implement it.
I know that I could just switch between connection strings but the problem is that in my reports I use datasets that are bound to one database.
Is it possible to fill those datasets with the data from both databases (since the databases are identical in schema, it should be possible), and how would that be done, or do I have to use duplicate dataset/report pairs?
I'm using C# in VS 2010 with SQL Server 2005, and .rdlc for my reports.
Thanks.
Ideally you should should be able to change the connection string in one place and it should affect project-wide.
This will work ONLY IF you get the connection string from one place. Keep it in the app.config file.
See this article to see how you can store and read the connection string from the app.config file.
You've hit upon the reason why people implement the Repository pattern or at least a version of it.
You really need to remove your business logic away from the database, so that it is database agnostic. It shouldn't care where the data comes from only what it is.
From what you' said the implication is that your client doesn't wants more than just a change in the app.config connection string used for database access.
If that is so then, I know that it will entail some work, your best bet is to have a singleton pattern type class to control all data access to and from your data layer.
Using a known inteface you can use a factory pattern to create access to your development or live database at runtime (perhaps based on an app.config setting or even a test class that has no database access at all, but just returns hard coded test data.
We have a process where our database guys script changes (and version them using Juneau) to our application's database out-of-band with our code base. They're good at accounting for new columns being null, and not wiping existing data, but occasionally a column rename sneaks in that isn't fully communicated. So they will make some changes to the database schema on a testing server, we'll update Entity Framework to work with those changes, and then commit our code. This process works okay, except for when it's time to deploy.
We have TFS set up to deploy the successful build to the appropriate servers, but there's no guarantee that the database for that environment has been updated. We don't care if extra fields/tables/views/etc. exist in the target database, but we want change the build to check that the database contains at least everything EF is aware of.
I looked at this question, but I don't need the schema to match exactly. Plus, we don't want it creating/modifying the database directly. And this question seems like it's trying to achieve a similar ideal, but still not quite what we're looking to achieve. We just want a integration test of sorts to verify our version of EF will work with the target schema.
I wonder why you try to deploy your application without changes to database. Your application is dependent on the database so the deployment should always be done after the database. It looks like you are going to invest a lot of time to develop validation to fix your incorrect deployment process (where fixing the process itself is the correct solution).
Anyway you can create some "validation" of the database but it will take some time. If you are using EDMX file you can open it as XML and read its SSDL part which describes all expected tables, columns, relations, views (in form of SELECT SQL queries), stored procedures and functions. You can parse this XML part and use system database views (sys.tables, sys.columns, ...) to query if these objects exists in the database.
Another approach can be using database diff. tool to compare your current test database with the target one. This will require the tool which can be executed from command line and you will have to parse its output to find breaking changes.
Situation
I'm creating a C#/WPF 4 application using a SQL Compact Edition database as a backend with the Entity Framework and deploying with ClickOnce.
I'm fairly new to applications using databases, though I don't suspect I'll have much problem designing and building the original database. However, I'm worried that in the future I'll need to add or change some functionality which will require me to change the database design after the database is already deployed and the user has data in the database.
Questions
Is it even possible to push an updated database design out to users via a clickonce update in the same way it is for code changes?
If I did, how would the user's data be affected?
How is this sort of thing done in real situations? What are some best-practices?
I figure that in the worst case, I'd need to build some kind of "version" number into the database or program settings and create some routine to migrate the user's current version of the database to the new one.
I appreciate any insight into my problem. Thanks a lot.
There are some 'tricks' that are employed when designing databases to allow for design changes.
Firstly, many database designers create views to code against, rather than coding directly to the tables. This allows tables to be altered (split or merged, etc) while only requiring that the views are updated. You may want to investigate database refactoring techniques for this.
Secondly, you can indeed add versioning information to the database (commonly done as a 'version' table with a single field). Updating the database can be done through code or through scripts. One system I worked on would automatically check the database version and then progressively update the schema through versions in code until it matched the required version for the runtime. This was quite an undertaking.
I think your "worst" case is actually a pretty good route to go in this situation. Maintain a database version in the DB and have your application check and update the DB as necessary. If you build your updater correctly, it should be able to maintain the user's data. Depending on the update this might involve creating temporary tables to hold the existing data and repopulating new versions of the tables from them. You might be able to include a new SDF file with the new schema in place in the update process and simply transfer the data. It might be slightly easier that way -- you could use file naming to differentiate versions and trigger the update code that way.
Unfortunately version control and change management for databases is desperately, desperately far from what you can do with the rest of your code.
If you have an internal-only environment there are a number of tools which will help you (DBGhost, Red Gate has a newish app, some deployment management apps) but all of them are less than full solutions imho, but they are mostly good enough.
For client-shipped solutions you really don't have anything better than your worst case I'm afraid. Just try and design with flexibility in mind - see Dr.Herbie's answer.
This is not a solved problem basically.
"Smart Client Deployment with ClickOnce" by Brian Noyes has an excellent chapter on this issue. (Chapter 5)
ISBN 978-0-32-119769-6
He suggests something like this:
if(ApplicationDeployment.CurrentDeployment.IsFirstRun) {
MigrateData();
}
private void MigrateData() {
string previousDb = Path.Combine(ApplicationDeployment.CurrentDeployment.DataDirectory, #".\pre\mydb.sdf");
if(!File.Exists(previousDb))
return;
string oldConnString = #"Data Source=|DataDirectory|\.pre\mydb.sdf";
string newConnString = #"Data Source=|DataDirectory|\mydb.sdf";
//If you are using datasets perform any migration here, with the old and new table adapters.
//Otherwise use an .sql data migration script.
//Store the version of the database in the database, and check that in the beginning of your update script and GOTO the correct line in the SQL script.
}
A common solution is to include a version number somewhere in the database. If you have a table with miscellaneous system data, throw it in there, or create a table with one record just to hold the DB version number. Then whenever the program starts up, check if the database version is less than the expected version. If so, execute the required SQL CREATE, ALTER, etc, commands to bring it up to speed. Have a script or function for each version change. So if you see the database is currently at version 6 and the code expects version 8, execute the 6 to 7 update and the 7 to 8 update.
Another method we used on one project I worked was to ship a schema-only, no data database with the code. Every time you installed a new version the installer would also install the latest copy of this new blank database. Then when the program started it up it would compare the user's current database schema with the new database schema, and determine what database changes were needed on the fly. Like, if in the "reference schema" table Foo had a column named Bar, and there was no column Bar in the user's current database, we would generate a "alter table Foo add Bar ..." and execute it. While writing the first draft of the program to do this was a fair amount of work, once we'd done it there was pretty much zero maintenance to keep the DB schema up to date. The conversion was just done on the fly.
Note that this scheme doesn't handle DB changes that require changing data values, like if you add a new column that must be initially populated by doing some computation on data from other tables or some such. But if you can generate new data from old data, that must mean that the new data is redundant and your database is not normalized. I don't think the situation ever came up for us.
I had the same issue with an app in Android with an SQLite database adding a table. I changed the name of the database to include a version extension, like: theDataBaseV1, deleted the previous one and the app works fine.
I just changed the name of the database and the name in this line of code
private static final String DATABASE_NAME = "busesBogotaV2.db";
in the DBManager when its going to open.
Does anybody knows if this trivial solution has any unintended consequences?