I know this question is somewhat subjective, but I think it might be a valid question to ask.
I want to create a program that watches folders on a file server. The program itself runs on the server, so network folders don't have to be monitored.
I want to get an event in case a folder/file gets deleted, moved, created and such. These information will be written to the disk (where the network users can't access it). I would need the name of the file, and the user who caused it, and maybe more info, but that's the minimum requirement for now.
In C# we can use the FileSystemWatcher-Class, which is very
unreliable. (Examples for that can be found around StackOverflow a
lot.)
We also could use the Auditing Feature of Windows 7 Professional
(which I am running on), but this also gives many, many confusing
entries in the system log. I just can't get reliable information
from those.
Third, one could just poll the files and compare. This is kind of
the brute force attack I would like to omit. Also, the other methods
might be almost realtime, this one is not.
So, I could think of combining 1. and 3., and maybe even 2., too, but what is the clean, the good way to do this?
Related
I'm currently working on a C# project of an application we'd like to develop. We're brainstorming over the question of sharing the data between users. We'd like to be able to specify a folder where all the files of the application are going to be saved and we'd like to be able to save them on a shared folder (server, different PC or Mac, Nas, etc.).
The deployment would be like so :
Installation on the first PC, we choose a network drive, share, whatever and create all the files for the application in this location.
On the second PC we install the application and we choose the same location (on the network), the application doesn't create anything, it sees that it's already existing and it uses these files as the application's data
Same thing on the other clients
The application's files are going to be documents (most likely XML formatted documents) and when opening the application we want to show all the existing documents. The thing is, we don't only want to have the list of documents and be able to edit their content, we also would like to be able to edit the document's property, so in a way we'd like a file (Sqlite, XML, whatever) representing the list of all the documents and their attributes. Same thing for a list of addresses.
I know all that looks exactly like a client / server with database solution, but this solution is out of the question. I was first looking at SQLite for my data files, but I know concurrency can be a real problem and file lock doesn't work well. The thing is, I would have the same problem with simple XML files (refreshing the content when several users are working, accessing locked files).
So I guess my final question is : Is it feasable? Is there an alternative I didn't see which would allow us to do that more easily?
EDIT :
OK I'm not responding to every post or comment, because I'm currently testing concurrency with SQLite. What I did, and please correct me if the way I test this is wrong, is launch X BackgroundWorker which are all going to insert record in a sample database (which is recreated everytime I start the application). I tried launching 100 iterations of INSERT in the database via these backgroundWorkers.
Of course concurrency is working with one application running, it's simply waiting for the last BackgroundWorker to do it's job and then writing the next record. I also tried inserting at (almost) the same time, meaning I put a loop in every BackgroundWorker waiting for a modulo 5 timestamp (every 5 seconds, every BackgroundWorker runs). Again, it's waiting for the previous insert query to end before doing the next and everything's working fine. I even tried it with 500 BackgroundWorkers and it worked fine.
I then tried launching my app several times and running them simultaneously. When doing this I did have some issue. With two instances of my app it was still working fine, but when trying this with 4-5 instances, it got really buggy and I got two types of error : 1. database is locked 2. disk I/O failure. But mostyle locked databases.
What I did was pretty intensive, in the scenario of my application, it will never ever come to 5 processes trying to simultaneously insert 500 hunded rows at the same time (maybe I'll get a concurrency of two or three connections). But what really bugged me and what makes me think my testing method is not really a good one, is that I got these errors trying to work on a database on a shared network, on a NAS AND on my own HDD. Everytime it worked for maybe 30-40 queries then throwing me "database is locked" error.
Am I testing it wrong? Maybe I shouldn't be trying so hard to make this work, but I'm still not convinced that SQLite is not a good alternative to what I'm trying to do, since the concurrency is going to be really small.
With your optimistic/pessimistic locking, you are ultimately trying to build a database. Also, you WILL have issues with consistency while trying to keep multiple files in sync with each other. Think about if you update the "metadata" file, and the write fails half-way through because of a network blip. File corruption will ensue, and you will be left trying to reconstruct things from backups.
I would suggest a couple of likely solutions:
1) Host the content yourselves, and let them be pure clients (cloud based deployments are ideal for this). Most network/firewall issues can be circumvented by using HTTP as your transport (web services).
2) Have one of the workstations be the "server", which keeps it data files on the NFS. This will give you transactional integrity, incremental backups, etc. There are lots of good embedded database managements systems to help you manage this complexity. MS SQL Server even has some great options for this.
You right, Sqlite uses file locks on database file, so storing all data files in database would bring write-starvation problem for editing your documents.
May be it's better choice to implement simple optimistic/pessimistic locking by yourself on particular-file level? For example, in case of using pessimistic lock you just don't allow anyone to edit particular file, if somebody already in process of editing it. In this case you will hold lock just on one file, but not on the entire database. If possibility of conflict(editing particular file at the same time) is pretty low, it is better to go with optimistic locking.
Simple optimistic locking implementation:
When user get file for reading - it's OK, no problem here. If user get file for editing, you could calculate hash for this file(or get timestamp of last updated time of the file), and then, when user tries to save edited file, compare current(at the moment of saving) hash/timestamp to make sure that file has not been changed by somebody else. If file has not been changed then it's ok to save it. IF file has been changed, then current user is out of luck, you need to inform him about it. This optimistic scenario is nice when possibility of this "out of luck" is pretty low. Otherwise it's better to stick with pessimistic locking, when you do not allow user even to start file editing if somebody else is doing it.
I was wondering if it was possible to disable users copying and pasting an external file while running my C# application?
example user runs application while it is running clipboard cannot be used, when the application is finished it then enables the clipboard again user can copy and paste now.
I found this prevent-cut-paste-copy-delete-re-naming-of-files-folders
Thanks for any help!
Answer to: "The user runs my launcher this runs the game and then connects to server where they download a file, this file is stored in a appdata this is the file i dont want people to copy".
The only option to prevent user from copying file on its own computer is to not send file there in a first place.
If you just want merely discourage people from copying the file (as it would be the case of "disable copy/paste") then opening file as non-sharable, delete-on-close may be enough.
Very difficult, if not impossible, and most likely totally unnecessary - what you have in plan. Clipboard belongs to the OS, and not just to your application. Think about how to solve the root of your problem in another way. If you explain what you're trying to do, maybe somebody will suggest how they would solve that particular problem. Why are you using the clipboard to maintain user/application state? If you accept input that way, then copy that data into your application's memory (or elsewhere), then work with it. Don't expect it to stay in the clipboard until your app is done working with it. However, also note that, it'd be against all usability rules to update/change the content of the clipboard with the result of that calculation - if that's what your mind is going as you're reading this.
That would be plain evil. Whatever the purpose. Remember, whenever you get to a task that need you to do some hacks just to provide a workaround to avoid dealing with some security layer being there with a reason or (as in your case) messing with some low-level operating system functionalities to change their bahavior, ask yourself if it even makes any sense.
You either don't need that feature, or you are searching for a security issue in the system/software which will be fixed within weeks or months.
You may actually implement some ugly non-reliable obstacles preventing the user to do those operations, but the user will always be able to find a different way to do them. Except if you are dealing with some DRM stuff, which I doubt.
And however, preventing the user to copy-paste? That definitely won't be some happy user ...
If you were going to download the file on every execution anyway, then you could download it at the game's "loading screen" and keep it in an in-memory stream. Less evil than having to hook the clipboard, and pulling it out involves debuggers or the ability to extract from the swapfile...
I'm not a fan of this solution (and I suspect you/you client will not be either) due to the bandwidth costs of downloading the core data of the game at every launch...
The Problem
Our company make specialized devices running Windows XP (Windows XPe, to be precise). One of the unbending legal requirements we face is that we must quickly detect when a fixed IDE drive is removed. Quickly as in within a few seconds.
The drives in question are IDE drives. They are also software-protected from writes with an EWF (Enhanced Write Filter) layer. The EWF layer sits under the file system, protecting the disk from writes. If you change or write something on an EWF-protected volume, the actual changes happen only in a memory layer (but the file system isn't aware of that).
The problem is that Windows itself doesn't seem to notice fixed drive removal. You can pull the drive out of the machine, and Windows Explorer will be happy to let you browse directories and even open files if they happen to still be cached in memory. And thanks to the EWF layer, I can even seem to write files to the missing drive.
I need a clean software-only solution. Ideally in C#/.Net 1.1, but I have no problem with using pinvoke or C++.
Things I can't do
No, I can't retrofit thousands of devices with new hardware.
No, we can't just super-glue drives in to meet legal requirements.
No, a normal file write/read won't detect the situation, thanks to the EWF layer.
No, we can't turn off the EWF layer.
No, I can't ignore legal requirements, even if they are silly.
No, I can't detect fixed drive removal the way I would for a USB or other removable drive. These are fixed drives.
No, I can't use WMI (Windows Management Instrumentation). It isn't installed on our machines.
No I can't use versions of .Net past 1.1. It won't fit on our small drives. (But if an easy solution exists in a higher version of .Net, I might be able to port it back to 1.1.)
Current awkward solution
I'm not happy with our current solution. I'm looking for something more elegant and efficient.
What I'm currently doing involves two threads.
Thread A polls the drive. It first creates a special file on the drive using Kernel32.dll:
Kernel32.CreateFile(
filename,
File_Access.GenericRead | File_Access.GenericWrite,
File_Share.Read | File_Share.Write,
IntPtr.Zero,
CreationDisposition.CreateAlways,
CreateFileFlagsAndAttributes.File_Attribute_Hidden | CreateFileFlagsAndAttributes.File_Attribute_System,
IntPtr.Zero);
Then it polls the drive by calling
Kernel32.FlushFileBuffers(fileHandle);
If the drive has been removed, then thread A will hang for a long time before returning an error code.
Thread B polls thread A.
If thread B sees that thread A has locked up (hasn't updated a special variable in a while), then thread B raises an event that the drive has been removed.
My current solution works, but I don't like it. If anyone knows a cleaner software-only solution, I would appreciate it.
I'm shocked and amazed that the system doesn't fall over dead if you yank out a fixed IDE drive. Like, really shocked. But, hey...
Are you sure can't just fix this with super glue? :)
First, the reason why Windows doesn't notice is because notification of device removal has to come from the bus driver. In this case, the IDE bus doesn't support what we call "surprise remove" so no one ever gets told that the disk is unplugged. I suspect that communications just start timing out, which is why your flush trick works.
Not sure if you're going to come up with any cleaner solution though. If you really, really need this and can restrict it to just a particular release of XP, someone might be able to analyze the drivers involved here and exploit a path that would give you a quicker result. But there's clearly nothing architected in Windows to deal with this and so that's like real work.
-scott
Have you looked in here:
http://msdn.microsoft.com/en-us/library/aa363217(VS.85).aspx
Looks like what you are looking for.
I would like to be able to do an "inplace" update with my program. Basically, I want to be able to login remotely where the software is deployed, install it while other users are still using it (in a thin client way), and it update their program.
Is this possible without too much of a hassle? I've looked into clickonce technology, but I don't think that's really what I'm looking for.
What about the way firefox does it's updates? Just waits for you to restart the program, and notifies you when it's been updated.
UPDATE: I'm not remoting into the users' PC. This program is ran on a server, and I remote in and update it, the users run it directly off the server through remote access.
ClickOnce won't work because it requires a webserver.
I had some example code that I can't find right now but you can do something similar to Firefox with the System.Deployment.Application namespace.
If you use the ApplicationDeployment class, you should be able to do what you want.
From MSDN, this class...
Supports updates of the current deployment programmatically, and handles on-demand downloading of files.
Consider the MS APIs with BITS, just using bitsadmin.exe in a script or the Windows Update Services.
Some questions:
Are the users running the software locally, but the files are located on a networked share on your server?
Are they remoting into the same server you want to remote into, and execute it there?
If 2. are they executing the files where they are placed on the server, or are they copying them down to a "private folder"?
If you cannot change the location of the files, and everyone is remoting in, and everyone is executing the files in-place, then you have a problem. As long as even 1 user is running the program, the files will be locked. You can only update the files once everyone is out.
If, on the other hand, the users are able to run their own private copy of the files, then I would set up a system where you have a central folder with the latest version of the files, and when a user starts his program, it checks if the central folder has newer versions than the user is about to execute. If it does, copy the new version down first.
Or, if that will take too long, and the user will get impatient (what, huh, users getting impatient?), then having the program check the versions after startup, and remind the user to exit would work instead. In this case, the program would set a flag that upon next startup would do the copying, only now the user is aware of it happening.
The copying part would easily be handled by either having a separate executable that does the actual copying, and executing that instead, or the program could copy itself temporarily to another location and run that copy with parameters that says "update the original files".
While you can design your code to modify itself (maybe not in C#?), this is generally a bad idea. This means that you must restart something to get the update. (In Linux you are able to replace files that are in use, however an update does not happen until the new data is loaded into memory i.e. application restart)
The strategy used by Firefox (never actually looked into it) is storing the updated executable in a different file which is checked for when program starts to load. This allows the program to overwrite the program with the update before the resource is locked by the OS. You can also design you program more modular so that portions of it can be "restarted" without requiring a restart of the entire program.
How you actually do this is probably provided by the links given by others.
Edit:: In light of a response given to Lasse V. Karlsen
You can have your main program looking for the latest version of the program to load (This program wouldn't be able to get updates without everyone out). You then can remove older versions once people are no longer using it. Depending on how frequent people restart their program you may end up with a number of older programs versions.
ClickOnce and Silverlight (Out of browser) both support your scenario, if we talk about upgrades. Remote login to your users machine? Nope. And no, Firefox doesn't do that either as far as I can tell..
Please double-check both methods and add them to your question, explaining why they might not do what you need. Otherwise it's hard to move on and suggest better alternatives.
Edit: This "I just updated, please restart" thing you seem to like is one method call for Silverlight applications running outside of the browser. At this point I'm fairly certain that this might be the way to go for you.
ClickOnce doesn't require a webserver, it will let you publish updates while users are running the software. You can code your app to check for new update every few minutes and prompt the user to restart the app if a new version is found which will then take them through the upgrade process.
Another option is a Silverlight OOB application, but this would be more work if your app is already built as WinForms/WPF client app.
Various deployment/update scenarios (for .NET applications) are discussed with there pros and cons in Microsoft's Smart Client Architecture and Design Guide. Though a little bit old I find that most still holds today, as it is describing rather the basic architectural principles than technical details. There is a PDF version, but you find it online as well:
Deploying and Updating Smart Client Applications
Is this possible without too much of a hassle?
Considering the concurrency issues with thin clients and the complexity of Windows installations, yes hot updates will be a hassel without doing it the way the system demands.
I have written a program that is sort of an unofficial, standalone plugin for an application. It allows customers to get a service that is a lower priced alternative then the vendor-owned one. My program is not illegal, against any kind of TOS, and is certainly not a virus, adware, or anything like that. That being said, the vendor of course is not happy about me taking his competition, and is trying to block my application from running.
He has already tried some tactics to stop people from running my app alongside his. He makes it so if it is detected, his app throws a fake error.
First, he checked to see if my program was running by looking for an open window with the right title. I countered this by randomizing the program title at startup.
Next, he looked for the running process name. I countered this by making the app copy itself when it is started as [random string].exe and then running that.
Anyways, my question is this: what else can he do to detect if my program running? I know that you can read window text (ie status bar, labels). I'm prepared to counter this by replacing the labels with images (ugh, any other way?).
But what else is there? Can you detect what .dlls a program has loaded? If so, could this be solved by randomizing the dll names before loading them?
I know that it's possible to get a program's signature in memory and track it that way (like a virus scanner), but the chances of him doing that probably aren't good because that sounds pretty advanced.
Even though this is kinda crappy of him to be doing, its kind of fun. It's like a nerdy fist fight.
EDIT:
When I said it's a plugin, that is just the (incorrect) term I used. It's a standalone EXE. The "API" between my program and the other is mine is simply entering data into the controls (like textboxes, etc).
I feel a little dirty answering this but it's late and I'm waiting for a drive copy to finish so....
He could use a checksum to identify your executable/dll. This gets around the renaming tricks.
You can get around this by randomly modifying bits in the program on start (e.g., change a resource, play with the embedded version, etc...).
If I were him I'd also start looking for patterns of network traffic; e.g., if you're directing customers to competitors you're looking that information up from somewhere so kill the process and/or unload the library if a plugin accesses a site that's on the blacklist.
If you take the cat and mouse game far enough (e.g., shell hooks to re-create your executable/library if it gets deleted) you'll probably get flagged as a virus by antivirus software.
Not very sporting of your competitor.
Deploy your project as uncompiled encrypted source code. Write a decryption and deployment program that can randomize, renames classes, re-arranges code to avoid any particular signature detection.
Then compile the code on the client machine using CSharpCodeProvider to compile your code. You can generate random assemblies, with totally random function signatures (I suggest using a large dictionary of real, common, words instead of being totally random. You can concatenate them together for more fun. e.g. Live, Virtual, Space, Office, Network, Utility. Space.Live.Network.dll, Utility.Virtual.Live.dll ).
Every version of your program on every client will be different. Make sure to cloak your deployment program. Maybe it should delete itself after it has installed your customized version.
Anyways, my question is this: what else can he do to detect if my program running?
Is your program an EXE or a DLL?
You call it a plugin: what is plugging in to?
How is your program started/launched/run?
What does your program do to "plug in"?
What's the API between your program and the other program?
This is not an answer to your final question but rather to the problem described.
How about fixing the other application.
Find the string it is looking for in the titles and change some letter in it.
Let your customers know where the problem lies by supplying them with a fix to the other application rather than your own.
#ryeguy ... The best defense is a good offense imho. Do what you can to disable his process before it disables yours.