In order to give my application an autosave functionality, I'm looking at the best implementation that would optimise the 3 followings requirements:
safety: in order to reduce the risk of data corruption
user friendly: the user is not computer expert so the solution must be intuitive and friendly
quick to develop: I don't want to spend weeks over this implementation never
I have three solutions witch doesn't fit the 3 criteria and I'm looking for an alternative:
creating a simple shadow file so when the application crashes or the PC shutdown unexpectedly the application try to restore it
working the same way than above but storing several version of the file at different time in a temp folder
implement a true roll back system allowing the extend the undo/redo functionnality even the the application is restarted by keeping trace of the modification in a temp folder.
Does someone have anything to suggest?
For autosave, I'd simply have a background running thread that would run your Save() method silently (no popups) to a temp location (AppData system folder). You should probably keep a separate file for each session, so that you can always offer to return to a previous crashed session. On normal exit, you should delete the file to indicate the session has completed successfully.
I'd even keep 2 files for every session an alternate saving to each, so that if a crash happens during an autosave, it won't corrupt the previous autosave.
Related
I'm currently working on a C# project of an application we'd like to develop. We're brainstorming over the question of sharing the data between users. We'd like to be able to specify a folder where all the files of the application are going to be saved and we'd like to be able to save them on a shared folder (server, different PC or Mac, Nas, etc.).
The deployment would be like so :
Installation on the first PC, we choose a network drive, share, whatever and create all the files for the application in this location.
On the second PC we install the application and we choose the same location (on the network), the application doesn't create anything, it sees that it's already existing and it uses these files as the application's data
Same thing on the other clients
The application's files are going to be documents (most likely XML formatted documents) and when opening the application we want to show all the existing documents. The thing is, we don't only want to have the list of documents and be able to edit their content, we also would like to be able to edit the document's property, so in a way we'd like a file (Sqlite, XML, whatever) representing the list of all the documents and their attributes. Same thing for a list of addresses.
I know all that looks exactly like a client / server with database solution, but this solution is out of the question. I was first looking at SQLite for my data files, but I know concurrency can be a real problem and file lock doesn't work well. The thing is, I would have the same problem with simple XML files (refreshing the content when several users are working, accessing locked files).
So I guess my final question is : Is it feasable? Is there an alternative I didn't see which would allow us to do that more easily?
EDIT :
OK I'm not responding to every post or comment, because I'm currently testing concurrency with SQLite. What I did, and please correct me if the way I test this is wrong, is launch X BackgroundWorker which are all going to insert record in a sample database (which is recreated everytime I start the application). I tried launching 100 iterations of INSERT in the database via these backgroundWorkers.
Of course concurrency is working with one application running, it's simply waiting for the last BackgroundWorker to do it's job and then writing the next record. I also tried inserting at (almost) the same time, meaning I put a loop in every BackgroundWorker waiting for a modulo 5 timestamp (every 5 seconds, every BackgroundWorker runs). Again, it's waiting for the previous insert query to end before doing the next and everything's working fine. I even tried it with 500 BackgroundWorkers and it worked fine.
I then tried launching my app several times and running them simultaneously. When doing this I did have some issue. With two instances of my app it was still working fine, but when trying this with 4-5 instances, it got really buggy and I got two types of error : 1. database is locked 2. disk I/O failure. But mostyle locked databases.
What I did was pretty intensive, in the scenario of my application, it will never ever come to 5 processes trying to simultaneously insert 500 hunded rows at the same time (maybe I'll get a concurrency of two or three connections). But what really bugged me and what makes me think my testing method is not really a good one, is that I got these errors trying to work on a database on a shared network, on a NAS AND on my own HDD. Everytime it worked for maybe 30-40 queries then throwing me "database is locked" error.
Am I testing it wrong? Maybe I shouldn't be trying so hard to make this work, but I'm still not convinced that SQLite is not a good alternative to what I'm trying to do, since the concurrency is going to be really small.
With your optimistic/pessimistic locking, you are ultimately trying to build a database. Also, you WILL have issues with consistency while trying to keep multiple files in sync with each other. Think about if you update the "metadata" file, and the write fails half-way through because of a network blip. File corruption will ensue, and you will be left trying to reconstruct things from backups.
I would suggest a couple of likely solutions:
1) Host the content yourselves, and let them be pure clients (cloud based deployments are ideal for this). Most network/firewall issues can be circumvented by using HTTP as your transport (web services).
2) Have one of the workstations be the "server", which keeps it data files on the NFS. This will give you transactional integrity, incremental backups, etc. There are lots of good embedded database managements systems to help you manage this complexity. MS SQL Server even has some great options for this.
You right, Sqlite uses file locks on database file, so storing all data files in database would bring write-starvation problem for editing your documents.
May be it's better choice to implement simple optimistic/pessimistic locking by yourself on particular-file level? For example, in case of using pessimistic lock you just don't allow anyone to edit particular file, if somebody already in process of editing it. In this case you will hold lock just on one file, but not on the entire database. If possibility of conflict(editing particular file at the same time) is pretty low, it is better to go with optimistic locking.
Simple optimistic locking implementation:
When user get file for reading - it's OK, no problem here. If user get file for editing, you could calculate hash for this file(or get timestamp of last updated time of the file), and then, when user tries to save edited file, compare current(at the moment of saving) hash/timestamp to make sure that file has not been changed by somebody else. If file has not been changed then it's ok to save it. IF file has been changed, then current user is out of luck, you need to inform him about it. This optimistic scenario is nice when possibility of this "out of luck" is pretty low. Otherwise it's better to stick with pessimistic locking, when you do not allow user even to start file editing if somebody else is doing it.
hello guys I have been given the task to create a web-app that would allow an end user to load a file to one of our servers. the server would then manipulate the file and then give it back to the end user.The server has IIS7 and .net framework 4
Here is my problem.
I have everything figured out except that I'm having a hard time getting rid of the file from the server once i manipulated. Is there a way that i can put a timer say after 30 mins to have the file removed from the server in my code? or is there another solution to doing this?
I know the timer suggestion perhaps does not make sense however i cant think of another way to do it so i'm looking for an opinion or another method.
A timer is a good method to schedule something in the future. You could even reset the timer if the user requests the file again. Just give the timer a delegate that deletes the file when the timer fires.
Are you able to set up a scheduled task on the server?
This sort of thing is perfect for a simple console app that simply deletes files that have a modified date/time that is older than, say, now.AddMinutes(-10).
The task can run every 10 minutes or so if you like too.
Sometimes best to keep this sort of thing away from your website. Let your site serve your users and create something else to serve your server.. :)
Update
If its a heavy-traffic site you could simply delete all old files the next time someone uploads a file.
So:
User Selects file to upload, clicks Upload -> you get file -> you
delete old files (regardless of who they belong to) -> you manipulate
file -> file will be deleted by next users upload...
Why not delete it after manipulating? Or whatever the last step in the process is? That would seem to be the best and easiest way.
Depending on volume, it's probably not a great idea to do a single task for each file - rather you should batch them into a queue and have a single thread process the queue.
For instance, you can spin up a background thread in global.asax (perhaps using a Timer) to handle cleanup tasks by comparing file times or something.
Or, as step 1 of the process you could clean any old files. Not exactly the same thing, but may be close enough.
Or, you could abuse the Cache remove callback as a timer.
If you can ensure the app stays running all the time you can skip scheduled tasks and use Quartz.NET. In this case, even if it shuts down using quartz would not be that bad -- unless there is something else to this having a few old files hanging around while the app is idle wouldn't hurt.
Insofar as handling this, I would store in an appropriate manner (eg -- your database) an list of the files with a marker for the job being complete as well as deleted. Your quartz task could then grab the files that are marked done but not marked deleted and clean those up. Bonus points for using transactions around the file deletes and updates and logging so you know what happened while the world was sleeping.
I need to uniquely identify a file on Windows so I can always have a reference for that file even if it's moved or renamed. I did some research and found the question Unique file identifier in windows with a way that uses the method GetFileInformationByHandle with C++, but apparently that only works for NTFS partitions, but not for the FAT ones.
I need to program a behavior like the one on DropBox: if you close it on your computer, rename a file and open it again it detects that change and syncs correctly. I wonder whats the technique and maybe how DropBox does if you guys know.
FileSystemWatcher for example would work, but If the program using it is closed, no changes can be detected.
I will be using C#.
Thanks,
The next best method (but one that involves reading every file completely, which I'd avoid when it can be helped) would be to compare file size and a hash (e.g. SHA-256) of the file contents. The probability that both collide is fairly slim, especially under normal circumstances.
I'd use the GetFileInformationByHandle way on NTFS and fall back to hashing on FAT volumes.
In Dropbox' case I think though, that there is a service or process running in background observing file system changes. It's the most reliable way, even if it ceases to work if you stop said service/process.
What the user was looking for was most likely Windows Change Journals. Those track changes like renames of files persistently, no need to have a watcher observing file system events running all the time. Instead, one simply needs to maintain when last looked at the log and continue looking again beginning at that point. At some point a file with an already known ID would have an event of type RENAME and whoever is interested in that event could do the same for its own version of that file. The important thing is to keep track of the used IDs for files of course.
An automatic backup application is one example of a program that must check for changes to the state of a volume to perform its task. The brute force method of checking for changes in directories or files is to scan the entire volume. However, this is often not an acceptable approach because of the decrease in system performance it would cause. Another method is for the application to register a directory notification (by calling the FindFirstChangeNotification or ReadDirectoryChangesW functions) for the directories to be backed up. This is more efficient than the first method, however, it requires that an application be running at all times. Also, if a large number of directories and files must be backed up, the amount of processing and memory overhead for such an application might also cause the operating system's performance to decrease.
To avoid these disadvantages, the NTFS file system maintains an update sequence number (USN) change journal. When any change is made to a file or directory in a volume, the USN change journal for that volume is updated with a description of the change and the name of the file or directory.
https://learn.microsoft.com/en-us/windows/win32/fileio/change-journals
I was wondering if it was possible to disable users copying and pasting an external file while running my C# application?
example user runs application while it is running clipboard cannot be used, when the application is finished it then enables the clipboard again user can copy and paste now.
I found this prevent-cut-paste-copy-delete-re-naming-of-files-folders
Thanks for any help!
Answer to: "The user runs my launcher this runs the game and then connects to server where they download a file, this file is stored in a appdata this is the file i dont want people to copy".
The only option to prevent user from copying file on its own computer is to not send file there in a first place.
If you just want merely discourage people from copying the file (as it would be the case of "disable copy/paste") then opening file as non-sharable, delete-on-close may be enough.
Very difficult, if not impossible, and most likely totally unnecessary - what you have in plan. Clipboard belongs to the OS, and not just to your application. Think about how to solve the root of your problem in another way. If you explain what you're trying to do, maybe somebody will suggest how they would solve that particular problem. Why are you using the clipboard to maintain user/application state? If you accept input that way, then copy that data into your application's memory (or elsewhere), then work with it. Don't expect it to stay in the clipboard until your app is done working with it. However, also note that, it'd be against all usability rules to update/change the content of the clipboard with the result of that calculation - if that's what your mind is going as you're reading this.
That would be plain evil. Whatever the purpose. Remember, whenever you get to a task that need you to do some hacks just to provide a workaround to avoid dealing with some security layer being there with a reason or (as in your case) messing with some low-level operating system functionalities to change their bahavior, ask yourself if it even makes any sense.
You either don't need that feature, or you are searching for a security issue in the system/software which will be fixed within weeks or months.
You may actually implement some ugly non-reliable obstacles preventing the user to do those operations, but the user will always be able to find a different way to do them. Except if you are dealing with some DRM stuff, which I doubt.
And however, preventing the user to copy-paste? That definitely won't be some happy user ...
If you were going to download the file on every execution anyway, then you could download it at the game's "loading screen" and keep it in an in-memory stream. Less evil than having to hook the clipboard, and pulling it out involves debuggers or the ability to extract from the swapfile...
I'm not a fan of this solution (and I suspect you/you client will not be either) due to the bandwidth costs of downloading the core data of the game at every launch...
I am trying to implement file based autoincrement identity value (at int value stored in TXT file) and I am trying to come up with the best way to handle concurrency issues. This identity will be used for unique ID for my content. When saving new content this file gets opened, the value gets read, incremented, new content is saved and the incremented value is written back to the file (whether we store the next available ID or the last issued one doesn't really matter). While this is being done another process might come along and try to save new content. The previous process opens the file with FileShare.None so no other process will be able to read the file until it is released by the first process. While the odds of this happening are minimal it could still happen.
Now when this does happen we have two options:
wait for the file to become available -
Emulate waiting on File.Open in C# when file is locked
we are talking about miliseconds here, so I guess this wouldn't be an issue as long as something strange happens and file never becomes available, then this solution would result in an infinite loop, so not an ideal solution
implement some sort of a queue and run all operations on files within a queue. My user experience requirements are such that at the time of saving/modifying files user should never be informed about exceptions or that something went wrong - he would get informed about them through a very friendly user interface later when operations would fail on the queue too.
At the moment of writing this, the solution should work within ASP.NET MVC application (both synchronously and async thru AJAX) but, if possible, it should use the concepts that could also work in Silverlight or Windows Forms or WPF application.
With regards to those two options which one do you think is better and for the second option what are possible technologies to implement this?
The ReaderWriterLockSlim class seems like a good solution for synchronizing access to the shared resource.