I am using the managed ESENT PersistentDictionary class to get my C# application to store data on a network share. It all works great, with one caveat. When I try to access the database from multiple client computers (all running my app), I sometimes get an exception with the message "System path already used by another database instance".
From the documentation, I gather than ESENT supports concurrency, so this should not be a problem. What am I doing wrong?
Thank you.
There's a slight misunderstanding. ESENT supports multi-threaded concurrency, but not multi-process concurrency. esent.dll opens the file exclusively, so any other process will get ACCESS_DENIED (with the exception of read-only mode -- multiple processes can open a database in read-only mode).
In addition, the file-locking over SMB isn't quite as rigid as with local file system access, and the caching behaviour is also different. It's not recommended that you have a database on a remote SMB share, although you'll probably not have a problem with it in real life. (And some of that guidance was based on older versions of SMB. Newer versions might have changed the implementation details enough so that it works perfectly -- I guess I just don't know enough. :)
In order to have multi-machine access, you'll have to write your own server process to handle requests from other machines. Sorry.
-martin
Related
Basically, I am making a program that blocks the internet access after 11h PM. But my only problem is that there is many ways to bypass it, such as shutting down the computer and the user just have to wait until the process gets closed by the OS itself then cancel the shutdown operation (Windows 7).
Any ways to make sure that the program won't get terminated before the pc shutdowns or anything?
If your goal is to block internet access, I recommend enforcing this rule on your router rather than on your PCs. It would be a much simpler, much more reliable solution. Your router probably already supports the feature, but if it doesn't you can buy a new consumer-grade router (dirt-cheap) and/or install a custom firmware that does (see Tomato Firmware for the Linksys WRT-54GL and company).
If the router approach just won't work for you, and you must block internet access in software, I would first suggest investigating Windows "local policy" or "group policy" to see if they can do what you want.
If that's too complex for your taste, try finding an off-the-shelf solution. Look into ZoneAlarm or NetNanny to see if one of them will do the trick.
But if you are bent on writing a C# program to do it for you, you probably want to look into writing a Windows Service. Services are more complex to write and deploy, but they can be configured to run at boot and are not slaved to a user session like regular desktop apps.
That's actually somewhat complex. It's like a virus - how do you keep it running, always?
You might want to read about drivers. Drivers have the highest "trust" by the operating system. They can physically access anything in the computer. Anything but a driver or a core file may be closed by the user manually, is some way or another.
Another thing you can do is to "burn" the file into Kernal.DLL or such. You can do it with a different operating system on the computer (e.g Linux) or by physically writing to the hard disk (not via Windows's API). To physically access the driver, check this out.
I have a Java application running on Debian OS and communicate with a Windows C# server program. My Java application will connect to C# server program via TCP/IP. A problem I am facing now is that my Debian OS system time is always slower than Windows Server System Time. Both applications are mostly run in an internal network, which has no access to Internet.
May I know is there any way to synchronize the time between these two applications?
I read about NTP, can Java use NTP to synchronize time with C# program?
Must C# program be running as NTP Server? (Any way to do it?)
If writing simply message exchange between these two applications, will there be any problem?
Will be greatly appreciated if anyone can provide links to study the implementations.
The best of all worlds would be to run your own local NTP server, and then sync both boxes to your local NTP server independently. You can even run the server on one of the boxes. Then you will have a common timing baseline from which to operate.
Alternately, if you have no access or support to do this, why not send the time in the data package that is transmitted between systems. Then, you will have the information needed to understand a differential between the machines. (This does not take into account any transmission delays between boxes, but it may be good enough to get the job done.)
Your network should have one centralized NTP service against which all other clocks in that network precisely synchronize themselves. Ideally, that NTP-server would synchronize itself against an Internet time-standard, but whether it does or not, it should be "the one and only source of Truth" for your entire network.
The old adage literally applies here: "a man with one watch always knows what time it is; a man with two watches never does."
It is not appropriate for your application to attempt to manage time-sync, even if somehow it possessed the necessary privileges to do so. (And it shouldn't!!) Instead, it should require that the clocks of all systems must at all times be properly synchronized against one master source. This should be its mandatory prerequisite, but not its personal responsibility.
I have a data file and from time to time I need to write a change to the file. The change consists of changing information in more than one place. For example, changing some data near the end of the file and also changing some information near the start. I want the two separate writes to either both succeed or both fail, otherwise it is left in uncertain state and effectively corrupted. Is there any builtin support for this scenario in .NET or in general?
If not then how to others solve this issue? How does a database on Windows solve this issue?
UPDATE: I do not want to use the Transactional NTFS capability because it is not available on older version of Windows such as XP and it is slow in the file overwrite scenario as described above.
DB basically uses a Journal concept (at least those one I'm aware of). An idea is, that a write operation is written in journal until Writer doesn't commit a transaction. (Sure it's just basic description, it's so easy)
In your case, it could be a copy of your file, where you're going to write a data, and if everything finished with success, substitute original file with it's copy.
Substitution is: rename original file like a old, rename backup file like a original.
If substitution fails: this is a critical error, that application should handle via fault tolerance strategies. Could be that it informed a user about a failed save operation, and tries to recover. By the way in any moment you have both copies of your file. That one when write operation just started, and that one when write operation finished.
This techniques we used on past projects on VS IDE like systems for industrial control with pretty good success.
If you are using Windows 6 or later (Vista/7/2008/2008R2) the NTFS filesystem supports transactions (including within a distributed transaction): but you will need to use P/Invoke to call Win32 APIs (see this question).
If you need to run on older versions of Windows, or non-NTFS partitions you would need to perform the transactions yourself. This is decidedly non-trivial: getting full ACID functionality while handling multiple processes (including remote access via shares) across process and system crashes even with the assumption that only your access methods will be used (some other process using normal Win32 APIs would of course break things).
In this case a database will almost certainly be easier: there are a number of in-process databases (SQL Compact Edition, SQL Lite, ...) so a database doesn't require a server process.
Question: I currently store ASP.net application data in XML files.
Now the problem is I have asynchronous operations, which means I ran into the problem of simultanous write access on a XML file...
Now, I'm considering moving to an embedded database to solve the issue.
I'm currently considering SQlite and embeddable Firebird.
I'm not sure however if SQlite or Firebird can handle multiple concurrent write access.
And I certainly don't want the same problem again.
Anybody knows ?
SQlite certainly is better known, but which one is better - SQlite or Firebird ? I tend to say Firebird, but I don't really know.
No MS-Access or MS-SQL-express recommodations please, I'm a sane person.
I wll choose Firebird for many reasons and for this too
Although it is transactional, SQLite
does not support concurrent
transactions, so if your embedded
application needs two or more
connections, they must be serialized.
An embedded Firebird database is
simple to upgrade to a fully shared
database - just change the shared
library.
May be you can also check this
SQLITE can be configured to gracefully handle simultaneous writes in most situations. What happens is that when one thread or process begins a write to the db, the file is locked. When the second write is attempted, and encounters the lock, it backs off for a short period before attempting the write again, until it succeeds or times out. The timeout is configurable, but otherwise all this happens without the application code having to do anything special except enabling the option, like this:
// set SQLite to wait and retry for up to 100ms if database locked
sqlite3_busy_timeout( db, 100 );
All this works very well and without any difficulty, except in two circumstances:
If an application does a great many writes, say a thousand inserts, all in one transaction, then the database will be locked up for a significant period and can cause problems for any other application attempting to write. The solution is to break up such large writes into seperate transactions, so other applications can get access to the database.
If the database is shared by different processes running on different machines, sharing a network mounted disk. Many operating systems have bugs in network mounted disks that making file locking unreliable. There is no answer to this. If you need to share a db on a network mounted disk, you need another database engine such as MySQL.
I do not have any experience with Firebird. I have used SQLITE in situations like this for many applications over several years.
Have you looked into Berkeley DB with the SQLite API for SQL support?
It sounds like SQLite will be a good fit. We use SQLite in a number of production apps, it supports, actually, it prefers transactions which go a long way to handling concurrency.
transactional sqlite? in C#
I would add #3 to the list from ravenspoint above: if you have a large call-center or order-processing center, say, where dozens of people might be hitting the SAVE button at the same time, even if each is updating or inserting just one record, you can run into problems using the busy timeout approach.
For scenario #3, a true SQL engine that can serialize is ideal; less ideal but serviceable is a dbms that can do byte-range record locking of a shared-file. But be aware that even a byte-range record lock will be inadequate for a large number of concurrent writes when new records are appended to the end of the file like a caboose on the end of a freight train, so that multiple processes are trying at the same time to set a lock on the same byte-range. On the other hand, a byte-range record locking scheme coupled with a hashed-key sparse file approach (e.g. the old Revelation/OpenInsight database for LANs) will be far superior to ISAM for this scenario.
I would like to be able to do an "inplace" update with my program. Basically, I want to be able to login remotely where the software is deployed, install it while other users are still using it (in a thin client way), and it update their program.
Is this possible without too much of a hassle? I've looked into clickonce technology, but I don't think that's really what I'm looking for.
What about the way firefox does it's updates? Just waits for you to restart the program, and notifies you when it's been updated.
UPDATE: I'm not remoting into the users' PC. This program is ran on a server, and I remote in and update it, the users run it directly off the server through remote access.
ClickOnce won't work because it requires a webserver.
I had some example code that I can't find right now but you can do something similar to Firefox with the System.Deployment.Application namespace.
If you use the ApplicationDeployment class, you should be able to do what you want.
From MSDN, this class...
Supports updates of the current deployment programmatically, and handles on-demand downloading of files.
Consider the MS APIs with BITS, just using bitsadmin.exe in a script or the Windows Update Services.
Some questions:
Are the users running the software locally, but the files are located on a networked share on your server?
Are they remoting into the same server you want to remote into, and execute it there?
If 2. are they executing the files where they are placed on the server, or are they copying them down to a "private folder"?
If you cannot change the location of the files, and everyone is remoting in, and everyone is executing the files in-place, then you have a problem. As long as even 1 user is running the program, the files will be locked. You can only update the files once everyone is out.
If, on the other hand, the users are able to run their own private copy of the files, then I would set up a system where you have a central folder with the latest version of the files, and when a user starts his program, it checks if the central folder has newer versions than the user is about to execute. If it does, copy the new version down first.
Or, if that will take too long, and the user will get impatient (what, huh, users getting impatient?), then having the program check the versions after startup, and remind the user to exit would work instead. In this case, the program would set a flag that upon next startup would do the copying, only now the user is aware of it happening.
The copying part would easily be handled by either having a separate executable that does the actual copying, and executing that instead, or the program could copy itself temporarily to another location and run that copy with parameters that says "update the original files".
While you can design your code to modify itself (maybe not in C#?), this is generally a bad idea. This means that you must restart something to get the update. (In Linux you are able to replace files that are in use, however an update does not happen until the new data is loaded into memory i.e. application restart)
The strategy used by Firefox (never actually looked into it) is storing the updated executable in a different file which is checked for when program starts to load. This allows the program to overwrite the program with the update before the resource is locked by the OS. You can also design you program more modular so that portions of it can be "restarted" without requiring a restart of the entire program.
How you actually do this is probably provided by the links given by others.
Edit:: In light of a response given to Lasse V. Karlsen
You can have your main program looking for the latest version of the program to load (This program wouldn't be able to get updates without everyone out). You then can remove older versions once people are no longer using it. Depending on how frequent people restart their program you may end up with a number of older programs versions.
ClickOnce and Silverlight (Out of browser) both support your scenario, if we talk about upgrades. Remote login to your users machine? Nope. And no, Firefox doesn't do that either as far as I can tell..
Please double-check both methods and add them to your question, explaining why they might not do what you need. Otherwise it's hard to move on and suggest better alternatives.
Edit: This "I just updated, please restart" thing you seem to like is one method call for Silverlight applications running outside of the browser. At this point I'm fairly certain that this might be the way to go for you.
ClickOnce doesn't require a webserver, it will let you publish updates while users are running the software. You can code your app to check for new update every few minutes and prompt the user to restart the app if a new version is found which will then take them through the upgrade process.
Another option is a Silverlight OOB application, but this would be more work if your app is already built as WinForms/WPF client app.
Various deployment/update scenarios (for .NET applications) are discussed with there pros and cons in Microsoft's Smart Client Architecture and Design Guide. Though a little bit old I find that most still holds today, as it is describing rather the basic architectural principles than technical details. There is a PDF version, but you find it online as well:
Deploying and Updating Smart Client Applications
Is this possible without too much of a hassle?
Considering the concurrency issues with thin clients and the complexity of Windows installations, yes hot updates will be a hassel without doing it the way the system demands.