I am new to C# and SQL Server.
I am developing an application using Winforms.
I am using dataset.
In master and details transactions, suppose I retrieve one transaction from the database.
At the same time another user also retrieves the same transaction.
My requirement is when I am changing this transaction no one else should be allowed to update or delete the same transaction.
As dataset is in disconnect mode, how can I achieve the locking mechanism?
using (var transaction = connection.BeginTransaction())
{
adapter.Update(dataSet);
transaction.Commit();
}
If the update is to a small number of rows within a table, SQL server grants a row level lock to the caller. If the change is to a large number of rows, SQL server grants a table level lock. Its all automatic. Hence concurrency is taken care of.
The problem however is that with many users simultaneously working on the same set of rows, chance of a dead lock are high. The new CQRS design pattern promoted by Udi Dahan takes care of that. How ever if your application is small, applying CQRS would be an overkill.
If im correct in assuming you are using C# then you should look into some ORM frameworks as they can handle collisions like this for you or at least alert you when they have happened so you can handle them in your code. So you could for instance inform the user someone else has made a change and refresh their display or merge the changes and save the merged data.
Have a look at entity framework. There are literally loads of tutorials and examples available for you. This should get you started.
http://www.asp.net/entity-framework
http://msdn.microsoft.com/en-us/library/bb386876.aspx
This specifically references data concurrency (its MVC but the principles are the same)
http://www.asp.net/mvc/tutorials/getting-started-with-ef-using-mvc/handling-concurrency-with-the-entity-framework-in-an-asp-net-mvc-application
Related
So I'm developing an application that works as sort of a "sidekick" to a large proprietary application which I do not have the source code for nor the rights to modify. The proprietary application does store all of its data in a Microsoft SQL database (version 2008 R2 or higher, I believe), however, and I have a good idea what the data represents. What I need my application to do is to constantly monitor the data as it is being added, updated, and deleted, and then act on the data automatically (such as raising alerts).
The issue is figuring out the best approach to receiving changes made to the database by the other application as they're happening, because I don't wanna miss a beat.
Here is what I have done so far:
LINQ to SQL: As far as I know, each time I run a query, I receive a new set of data, but I do not get the ability to receive the changes only or be notified of changes.
Typed DataSet using DataSet.Load:
using (IDataReader reader = dataSetInstance.CreateDataReader())
{
dataSetInstance.Load(reader, LoadOption.OverwriteChanges, dataSetInstance.Table1, dataSetInstance.Table2, dataSetInstance.Table3);
}
This didn't work out too well when I did it. dataSetInstance only contained a set of unfilled tables after calling the Load method. I was hoping to call dataSetInstance.GetChanges and dataSetInstance.AcceptChanges at regular intervals after the first call to dataSetInstance.Load to get only the changes. Am I doing it wrong?
Typed DataSet with tables filled individually using their associated table adapters:
using (Table1TableAdapter adapter = new Table1TableAdapter())
{
adapter.Fill(dataSetInstance.Table1);
}
using (Table2TableAdapter adapter = new Table2TableAdapter())
{
adapter.Fill(dataSetInstance.Table2);
}
using (Table3TableAdapter adapter = new Table3TableAdapter())
{
adapter.Fill(dataSetInstance.Table3);
}
Of course, the problem is that there are actually way more than 3 tables which can add up to quite a lot of repetitive code (and maintenance work), but the real problem is that I will not receive any change notifications since I'm not using the Load/AcceptChanges methods (according to the documentation).
Row retrieval by date/time field: This was something I started work on, but something I stopped after observing the other application modify fields in the rows after creating them. Consider this:
There is a row with a time stamp of a transaction and a boolean field that specifies if the transaction was canceled later on. If it is canceled, the other application simply goes back to that row and toggles the value. The time stamp remains the same, and my application will never know of the news. There is no statute of limitations; the other application can change this field any time in the future.
By the way, I should mention that this other application does not implement any constraints within the database such as foreign and primary keys. I believe I read somewhere in the documentation that for row update events and such to fire on the typed DataTable classes, some sort of primary key is needed.
There must be some way to do this!!!
Have you considered SQL Server Query Notifications? This uses SQL Server Service Broker under the covers.
SqlDependency is the C# class to look at.
Using SqlDependency in a Windows Application (.NET Framework 2.0 example: should be very similar to later versions.)
SqlDependency in an ASP.NET Application
I’d consider solving this at SQL Server level by implementing auditing triggers or SQL Server traces.
Triggers – idea is to add triggers to all tables you want to monitor. Triggers will catch all changes and store the data in some other “history” table. Once this is setup all your application needs to do is to read from these tables.
Check this for more details Creating audit triggers in SQL Server
Traces – you can setup SQL Server traces that will store all info in trace files and then your app can parse trace files and see what’s going on.
There appears to be no silver bullet to the problem given the conditions, but anything is better than polling the database for changes every minute. What I will probably do now is take Mitch Wheat's suggestion and work from there:
Some tables have rows that are highly likely to change. A recent purchase, for example, is more likely to be cancelled than one from 7 days ago, or 6 months ago, or in the case of 1 year—probably never. The application will only need to monitor queries restricted to a certain time range. Older (in terms of creation time) rows will simply be refreshed at a much slower rate and without prompting from SQL Server query notifications. The application is going to have to tolerate some stale data in order to not needlessly pull entire tables from the database every minute.
For tables without chronological information, the application will have to receive notifications for queries on conditions that are important or have to be acted on right away such as WHERE Quantity < 0.
Some more clever approaches will need to be taken for the rest of the tables. Some tables are never updated nor their rows deleted, but they will gain new rows whenever some other table's rows changes. For example: every time the NumberOfPeople value changes for a row in table Room, another row is added to one of the tables CheckIn or CheckOut.
A lot more code needs to be written, but the application is probably going to be doing a lot less unnecessary work when it's running.
In the beginning of our app's development, we were using SqlDependency quite heavily to cache DB results until the notifications told our app to grab a fresh copy.
During testing, we've noticed that the SQL DB's performance was getting hammered by the SqlDependency notification service. We scaled back the number of tables that we were using SqlDependency and noticed a large gain in performance. So, we thought we were just over using it and we moved on. We are down to only a few tables now.
Later, we discovered that we couldn't scale back the security access level for the username that will establish the dependency. We could have more than one connection string for each DB (one for dependency and one for the rest of the app), but with multiple DBs and DB mirroring, this is a pain (from SQL DB admin point of view and app development).
At this point, we are just thinking about moving away from SqlDependency altogether based on the following logic:
We don't need "instant" notification that the data has changed. If we knew within 1 second, that would be fast enough.
With some slight refactoring, we could get it down to just 1 table and poll that table once a second.
Does anyone see a flaw in this logic?
Would polling one table once a second cause more or less load on the DB than SqlDependency?
Has anyone had similar performance issue with SqlDependency?
I do dare try answer your question. But I am not sure you'll get the answer you was hoping for...
I remember back in the early 90ies when Borland promoted this grand new feature of 'callbacks' in their database Interbase that would give the caller (Delphi) 'notifications' via some very nifty new tech where promises was made that the database could be 'active'.
This was later known as the 'waste of time theory'.
And I guess why this never took of is perhaps that while the concept of DBMS was looking very promising, the database is one of your tiers that you can only scale up and not horizontally.
So programming languages to the rescue. Or rather the idea of Service Oriented Architecture (SOA). Many confuse SOA for 'Webservices' that was indeed an included hype in this new concept.
But if you check out the Fiefdom/Emissary design pattern (or Master/Agent pattern renamed to make it sound more cool and professional), you will find that the major idea is having exclusive control of its resources (read databases) and that all calls are being funneled via one single data adapter.
Obviously such a design does not work at all with triggers nor any callback frameworks.
But I think you should reconsider your entire design. If you funnel all actions and all calls via a single 'DataLayer', perhaps using Entity Framework, and perhaps on top on that a Caching mechanism you would not have to rely on your database to forward messages back up the food chain.
To show how weird things can get when being to 'database-centric', here is an extreme actual live example of how not to send an email, written a long long time ago, by a coder I was not so much impressed with:
Fact 1: Sql Server can send emails.
Fact 2: Asp3 coder does not know if or how this can be done in VbScript.
Asp3: read textbox email-address, send to com+ layer
Com+: take email-address and forward to datalayer
Datalayer: take email-address and forward to a stored procedure
Sproc: take email-address and forward to sql function
function: do weird sub-string things to check that email-adress has # . in it. return true or false.
Sproc: return a recordset with one column and one row containing 1 or 0
Datalayer: return the table as is.
Com+: convert the first column and row with value 1 or 0 to true or false
Asp3: if true, send email-adress with email subject and email text to com+
Com+: sends the exact information to datalayer
Datalayer: calls an stored procedure..
Sproc: calls a sql-function...
function: uses sql server email agent to send the email
If you read this far, my advice is to let sql server manage tables, relations, indexes and transactions. It is very good at that. Anything beyond those tasks, and with that I do include cursors in stored procedures, is better handled via proper code.
I have a form with few tabs, and in each tab an grid control. When user select a row to be deleted i want to remove it from the grid, and if the object exist in the database remove it too, but not permanent - only if and when user clicks save on form.
For now, if object doesn't exist in db i remove it from the list, and if objects exist in db i delete it from db and remove it from the list. But, if user clicks Cancel button he expects row/s not to be deleted from database.
I have two possible solutions on my mind: 1) - remove object from list, and if objects exist in db add it to the list of objects to be deleted 2) - implement another list, getter will return only objects with state != ToBeDeleted (performance?)
Note: i'm not using ORM tool, working with my own ado.net based data access framework.
I think the case you are descibing just asks pretty much for a Transaction.
ADO.Net handles them easily, provided you are using a reasonable database engine (so: no SqlServerCE for example:))
See for example the TransactionScope class. You construct such object before interacting with the database, and the changes will be "commited" if and only if you call Complete(). If you just leave it alone or if you Dispose() it, the transaction will be cancelled and all changes on the DB will be "rolledback", so, reverted.
So, in your case, you may open the transaction in the Form's ctor or onLoaded(), and Complete() at "save", and Dispose() at any other window closing.
While this is the normal way of handling such things for small systems, especially single-user ones, but be careful: if your system has to handle many concurent useres, you may be not able to use it in this way. The Transaction blocks rows and tables until it is completed or cancelled, and the therefore "other users" may see large delays..
So, how many users do you have to support and how often they will try to edit the same things?
-- edit: (10 users)
With that many users, you will want to avoid long-running transactions. Opening transaction at form-load will be unacceptable, and will lock many users away until that one current user closes the window. But, using transactions at Save() that push all the changes in one batch are OK.
Of course, if you can eliminate transactions at all - that's great! But, it is very hard thing to do if you also need to preserve data integrity.. To eliminate the need of transactions, almost always you have to redesign both the data structure on the DB side, and the way you obtain and work with the data. If you want to redesign both, then I'd really recommend to first try redesigning it to use some existing data-access framework, as even the basic .Net ADO has really nice features for online editing of databases held at SqlClient-compliant databases..
So, assuming you don't want to rewrite/rethink most of your code, you just need to buffer the data and also, delay all of the actual operations on the database.
You may want to do it in a "simple" form: when you display your form, instead of binding your Form directly to the database-driven datasources - download all required data to some BindingList<>s, DataTables, etc - whatever container you like. And bind your form to them instead. Probably you have something like that already set up. But, the important thing is that all those datacontainers must be offline or at least readonly+delayloaded.
Next, you've got to intercept all operations that the user performs on the UI. Surely you have it done already, as I'm assuming the application works:) As your Forms are bound to that offline cached items, your application should perform the operation on that cached data, and don't touch the database at all. But there's more: along with performing them on cached data, you should record what happens to which table.
Then, when finally the user stops playing around and presses CANCEL :) - you just trash everything and close the form. database not changed.
On Save - you open a fresh transaction, then iterate over the list of changes and effectively replay your recorder changes on the database, then commit transaction.
Please note two things though: the database could have changed during the time the users cached the data and the time he pressed Save. You have to detect this and abort, or resolve conflicts. You should do that inside that transaction, either during or before executing the recorded changes. You may detect it by simply comparing the online data with offline cached data (the unchanged original values, not those modified by user), or you may use some other mechanisms like OptimisticLocking and just compare the version tags on the rows.
If you don't like record-replay, you may implement a "DIFF"ing utility that takes the modified offline data and compares it in a generic way with the current-online tables. This is somewhat harder, but has a bonus: with such utility, you can initially doubly-cache the data: one copy for offline reference (just stored and never touched by the user) and one copy for offline editing (all those bound to the Forms). Now, upon Save you open transaction and diff the reference data against the online database. If there are any difference - you've just detected a collision. Solve/merge/abort/etc. If no differences, then you diff the modified data against online-data, and apply all differences found to the database and commit transaction.
Either of those methods has its pros and cons: aside from difficulty of implementation, there's memory issues of caching, latency issues if you dare to copy too large tables, etc.
But - once solved, it would work pretty nice.
And as you finish, you can go and boast that you have just implemented a smaller sis' of the DataSet+DataTable. I'm not joking, and I'm not laughing at you. I'm just trying to show you why everyone is telling you to rewise your DAO layer and try understanding and using the hard work that was already done for you by the platform designers/developers :)
Anyways, I've said you can avoid the clashes and transactions at all if you rethink your data structure.. For example: why do you DELETE the rows at all? I know there's a nifty DELETE statement in the SQL, but, well, do you really need to delete that row? Can't you just add some 'bool isDeleted' column and when user deletes the row from the Grid - just make set that rowcell to True and make the application filter-out any isDeleted=true rows and not show them? and not include them in views and aggregations? Bonus: sys/db admins now have a magic tool: undelete..
Let's take it further: do you need to UPDATE the rows? Maybe you can just APPEND some information that from (this-date) that row should have a new price? of course, the structure must be greatly altered: entities doesn't have properties, but have logs of timestamped property changes (or either the rows must have version numbers and be duplicated..), queries must be done against only the newest versiosn data, etc. Pros: database is now append-only. Transactions, if needed at all, are hyper-short. Cons: SELECT queries are complicated and may be slow, especially when joining many tables..
Pro/Con: and your db actually starts looking very meta- instead of data-base...
Con: and this is really hard task to "upgrade" existing application to such db structure. Writing a new app from scratch and importing data from odl system may be few times faster.
Now, to summarise:
I do not recommend any of the ways described.
First, I recommend you to take some ORM framework like NHibernate, EntityFramework, XPO from DevExpress, or whetever else. Any of them will save you lots of time. Those three I list here even have OptimisticLocking collision detection built-in. Why use SQL-self-written framework when such tools exist?
If not, then next I recommed to use existing tools found in the framework. you use SqlClient, whydontya use DataSet and DataTables? They are provided along with SqlClient and they have many useful mechanisms just built-in, which otherwise you will spend weeks on implementing and testing all by yourself. Learn to use DataSets and its collision detection, and its merging algorithms, and use them. You will loose a bit of time on experimenting and learning, but you will save huge amounts of time on not-reinventing the wheel.
If you really want to do it manually, start with data-caching and record-replay. It is easy to comprehend, it is quite easy to introduce anywhere where you currently use plain SQL queries, and will quickly introduce you to all kinds of cache-syncing and version-checking problems, and you will soon learn in details why all those strange mechanisms in the above-mentioned frameworks were implemented, how they work and what pros/cons they have.
and about the doubly-cached diffing approach.. it will be more tempting to write that record-repay, but please: use it only if you know very well how to detect/solve/merge collisions. Have at least one record-replay approach implemented before you try it..
..and of course yo umay use long-lasting transactions. Dumb-easy to introduce, and they "just irritate" the users.. Well, or even make the system unusable when >90% of the users constantly collide and hit the locks, heh.. No, that was a joke. Don't use long-lasting transactions. They are ok for 1-4 users, or for very sparse databases..
I am developing an application in C# WPF which will have Client-Server architecture (Client will do products sales billing). I am novice in this area and I asked this question to start my development process.
Click here to view question.
So, ultimately I have selected MySQl, WCf & WPF. Now I have one silly question. Do i need to handle DB concurrency explicitly in my application (like 3 clients inserting data same time) or MySQl will handle this without any conflict?
To accomplish my project i thought, I will create a service in WCf which will do DB queries from client application. Do you have any suggestion to improve my application performance.
With respect to your question about concurrency, your application should be designed to keep connections to the database open as short as possible. Each action on the database should involve: open connection, act on the database, close the connection rather than: open connection, do a bunch of work that may or may not be related to getting/updating/inserting data and then at "the end" close the connection.
Now, with respect to application concurrency, you end up with two scenarios. In scenario one, which I'll call "last write wins", whatever connection writes to a given row last is the version of the data that gets stored. If Alice then Bob write to the Name column on the same row at the same time, Bob's version will be what is stored. This is by far the simplest but if you might have a lot of people updating the same data, it might be problematic.
An alternative is "first write wins" also called Optimistic Concurrency. In this scenario, the second call checks that the data has not changed since it was last retrieved and if it has, then its transaction is rolled back. What happens next depends on your application. Some systems simply throw an error and require the user to re-enter their information (discarding their original change). This is obviously easier to implement. Some applications tell the user that the data has changed and provide some information about what is different and ask whether they want to overwrite this change. That can be more complicated depending on the architecture of your system.
See Optimistic Concurrency for more.
There are a number of ways to handle concurrency, each with their pro's and con's.
This article gives a good general introduction.
If you wish to share more about your requirements around concurrency:
What do you WANT to happen when
multiple people try to edit the same
data?
How frequently do you expect users to edit the same data?
I would be glad to give more specific advice.
Database servers are pretty good at handling multiple updates.
Use netTcpBinding.
I was wondering how to monitor a database for changes programmatically.
Suppose I want to have a .net application which would run after every 100th or(nth) row insertion or row deletion or updation . how can it be achieved?
I know little about triggers.they can be used to fire executable.
But I heard that it isn't a good practice.
Is there any other way?
2]Do database fire events on table updations? and can they be caught in a program?
3]Can SQL reporting services be used here?
(Also assuming that this application is independent from the actual program which does
database manipulation.)
SQL Server 2005 introduced query
notifications, new functionality that
allows an application to request a
notification from SQL Server when the
results of a query change. Query
notifications allow programmers to
design applications that query the
database only when there is a change
to information that the application
has previously retrieved.
Check out the MSDN link for more clarity
and sample immplementation
A trigger is really going to be your only way unless you aren't concerned about the accuracy of "100th" or "nth".
The answer to 2 and 3 are no.
You can write managed stored procedures (MSDN example) but that doesn't help you here really. In general triggers can be bad practice since they can block the initial caller but sometimes they are the only solution.
I think you need to question your requirement to place this low-level data monitoring in a separate application. Think about where your data changes could originate -
Do you have full understanding of every:
stored proc within your db (now and future) and which ones update this table?
application that may hit your database (now and future)
If not, then watching the changes right down at the data level (ie within the db) is probably the best option, and that probably means triggers...
Read about "Service Broker" at http://msdn.microsoft.com/en-us/library/ms166104(v=SQL.90).aspx