Debug mysteriously disappearing files from file server - c#

We've been having an issue in our production system lately where some files have been disappearing, or their names changed which breaks the reference to them in the database.
I've added logs to places where I suspect the bug might be, but the codebase is large and there are many flows where files are deleted/copied/moved, so it's difficult to find the problem like this.
My question is whether it's possible to somehow extend the File Delete/CopyTo/MoveTo methods from .NET in order to catch every file operation, and log it, along with the StackTrace to know where the request came from?
If not, then I'd appreciate some tips on how to tackle this issue.
Thank you.

This one is going to be pretty hard for you to capture. You have FileSystemWatcher that you could use to at least capture and bring the file back if it is that critical. But it doesn't give you any context information on how/what made the change, just that a change happened.
The same even goes for shell hooks, which you wouldn't want to do either. (https://msdn.microsoft.com/en-us/library/windows/desktop/cc144063(v=vs.85).aspx)
As for hooking into all calls to the File classes, if would be possible if you had a wrapping class, but otherwise it is going to be hard.

Try Ionad.Fody. It allows to replace static method calls on whole assembly.
PM> Install-Package Ionad.Fody
But it replaces methods completely, so you can't use original functionality by default.
To extend the methods you have to create wrappers in separate project / assembly.
For example. Add new project to your solution. Do NOT apply Fody.Ionad to it. Add wrappers for all of the methods you need to extend:
public static class FileHelper
{
public static void Delete(string path)
{
Console.WriteLine(path);
File.Delete(path);
}
}
Apply Fody.Ionad to your main project where you need to replace methods and add substitutes with StaticReplacement attribute:
[StaticReplacement(typeof(File))]
public static class FileSubstitute
{
public static void Delete(string path)
{
FileHelper.Delete(path);
}
}
Now when you call File.Delete("path") it will output the path to console and delete the file.
If you using some framework that hides File calls in own methods, you have to apply Fody.Ionad add substitutes to that framework too.

Related

Validating Static Resources in a Web Application

Like most web applications mine has static resources that must be part of the deployment or the user receives a 404 response from the server. My thought was to use unit testing to validate two things 1. the resource exists, and 2. the content was not modified. I have tried the following code but it expects (I think) that the files exist in the unit test project.
Solution structure:
WebApplicationProject
- ...
- public
- file.*
- otherfile.*
- web.config
WebApplicationProject.Tests
- AssetTests.cs
Am I going about this all wrong, should this not be part of a unit test and some other gait on the CI build process (Azure DevOps), or am I missing something super obvious? I'm likely asking the wrong questions and going about this the wrong way, because I know I'm not the first person to want to do something like this.
I've read other posts around testing with files, but they all are using test files to drive data for input in some method that consumes the file, or some process that generates a file for comparison. I don't want to do either of these things.
I have also played with the settings making the file an embedded resource, and to always deploy with the project, but the unit test project still cannot access the file the way I'm going about this.
[TestClass]
public class AssetTests
{
[TestMethod]
[DeploymentItem(#".\files\file.*")]
public void AwardLetters()
{
string path = Path.Combine(Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location), "file.*");// gets the working path for the testing dll, no files exist here.
Assert.IsTrue(File.Exists("file.*"), "FAIL: file {0} not found", "file.*");// nothing I have tried has access to the projects static resources
}
}
All results end in a file not found exception so far.
I did try to load the reference manually using:
Assembly a = Assembly.LoadFrom("WebApplicationProject");// also used WebApplicationProject.dll
This fails to find the reference. Yes, the reference property copy local is set to true.
I am open to all suggestions, but if you suggest that I have two copies of the files, please fully explain why this is desirable.
Alright here's my MVP. I'll leave this open for a while though in hopes someone has a better solution, it cant be this difficult to access the resources like this, I feel like there should be a way to access the applications directory without having to embed the file in the assembly just to pass it to a test method.
[TestMethod]
public void FileExists()
{
Assembly a = Assembly.LoadFrom(#"..\..\..\WebApplicationProject\bin\WebApplicationProject.dll");
string t = string.Join("", a.GetManifestResourceNames());
Assert.IsTrue(t.Contains("file.*"));
}
Now that I have the file I can also create a test to test the content of the file to validate it's contents.
I still think this is duck tape and it's not elegant at all. So please share your answers and critiques of my solution.

How to replace a static class at runtime

I know you cant override or inherit from a static class and why. That is clear.
I am looking for some advice on how to replace that static class with my own static class. Any hackish or wildest attempts please.
I am basically writing a MOD for a game and the way the game writer wrote one class in particular, he set it as static and put the implementation in there. So when we write our own DLL with this thing, the only way to execute a calculation on the pixel grid is when his code calls this particular calculation in his static class. Both classes are static but I only need to change one.
That is great for him but I want my thing to do another calculation and make it more awesome. I used ILspy and can see all the code in that static class of the base game, so I can copy and paste it and I only need to modify two or three lines.
But now I want to nuke the games core static class and make mine the only implementation.
I want to force replace that static class at runtime, before the static class is ever called and after loading my mod, how? There must be a way to swap static classes?
I read about creating a proxy DLL that redirects all methods to the old DLL and my method to my DLL but that would require gamers to replace a core game DLL and that is even dirtier than just telling people what my mod does. I am changing thas implementation for this mod, if you dont like don use my mod. That is more reasonable.
I will assume you don't have access to the source and thus can't modify it directly.
You COULD (probably shouldn't) use microsoft fakes since it is mainly for testing. You could create a fakes assembly based on the original author's dll, and override just the type you want. It even supports overriding static classes. Again, I am not saying that you necessarily SHOULD do this, but you COULD.
Here is the page for isolating code under test, it includes an example for shimming a static class (DateTime) https://msdn.microsoft.com/en-us/library/hh549175.aspx
A few options ...
Review how the original developer said to modify the game
You could use something like JustDecompile to get their code.
Use Fakes as suggested above
Create your own assembly that calls into their assembly and hack the IL dynamically
This seems pretty close to this question: Can I redirect .NET method calls to a new method at runtime?
One of the answers to this post suggests looking at a library called Moles which seems to be similar to Detours and may help
Moles allows to replace any .NET method with a delegate. Moles supports static or non-virtual methods

How can I get a portion of my C# app to load dynamically without app restart?

My C# application server.exe is critical to my business operations and ideally needs to run without interruption 24/7. The code is rock solid but one thing that's out of my control is the poor quality of inbound data feeds that are generated by third parties. I'll occasionally receive data feeds that contain anomalies in which case I must:
update the feed processing code within server.exe to accommodate the anomaly
recompile
restart server.exe using the new code and allow the syntactically flawed feed to be processed
The whole process usually takes less than a couple of minutes but the restart of server.exe causes the reset of certain non-critical state information and worse, causes a disruption of external processes that depend upon server.exe.
My goal: Isolate the feed processing code into a separate DLL whose contents can be updated without restarting server.exe. How do I do this?
Allow me to explain what I've done so far prior to writing this forum post:
The feed processor interface has been moved to a new assembly called common.dll. The interface looks something like this:
public interface IFeedProcessor{
bool ProcessFeed(String filePath); //returns false on failure, true on success
}
Server.exe now references common.dll.
The feed processors themselves have been moved to a new assembly called feedProcessors.dll. The implementations look something like this:
internal class FeedProcessor1:IFeedProcessor{
public FeedProcessor1(){}
bool ProcessFeed(String filePath){/*implementation*/return true;}
}
internal class FeedProcessor2:IFeedProcessor{
public FeedProcessor2(){}
public bool ProcessFeed(String filePath){/*implementation*/return true;}
}
[... and so on...]
feedProcessors.dll also contains a class named FeedProcessorUtils that's used to create a specific feed processor based on some configuration inputs. It looks something like this:
public class FeedProcessorUtils{
public static void CreateFeedProcessor(int feedType /*and other configuration params*/){
switch(feedType){
case 1:return new FeedProcessor1();
case 2:return new FeedProcessor2();
default: throw new ApplicationException("Unhandled feedType: "+feedType);
}
}
}
Everything works just as before but of course it doesn't solve my dynamic loading problem; If I updated feedProcessors.dll with new code and copy it to the production server, I'm unable to do so because the file is in use. No surprise there. So what's the solution?
Ideally I want to be able to copy an updated feedProcessors.dll to the production server without a file-in-use error and without restarting server.exe. Then, the next time server.exe makes a call to FeedProcessorUtils.CreateFeedProcessor(), it'll be executing from my revised DLL instead of the old one.
Where do I start?
You want to use shadow copy assemblies for the dynamically loaded DLL
http://msdn.microsoft.com/en-us/library/ms404279(v=vs.110).aspx
Sounds like a classic place to use MEF: http://msdn.microsoft.com/en-us/library/dd460648(v=vs.110).aspx
I suggest you look into it and ask questions as you proceed.
In addition to the shadow copy option, you should create an AppDomain and check when your dll is changed to restart the domain and reload the dll, you can check this with a FileSystemWatcher.
Take care of not directly referencing your classes between AppDomains, or your old assembly will never be unloaded.

Configuration vs. static properties, security concerns

I'm developing a class library and I need to provide a way to set configuration parameters. I can create a configuration section or I can expose static properties. My concern about static properties is security. What prevents a malicious component from making changes at runtime? For instance, in ASP.NET MVC you configure routes using a static property. Is this secure? Can a malicious component add/remove routes?
How would the "untrusted component" get in my application in the first place? NuGet for example. We don't know what's out there, who did it, and if it contains small bits of undesired state changes.
How would the "untrusted component" run? In ASP.NET all you need is PreApplicationStartMethodAttribute to run some code when the application is starting.
When you consider something as a security threat, you should also think about from whom you are trying to protect.
In order for "malicious code" to alter the values of your static properties, this code would need to be loaded into your AppDomain and run. Now think that a malicious attacker has managed to get his code to run in your AppDomain - are your static properties really your major concern? Such an attacker can probably do a lot worst.
Unless you have a scenario where you need to load an assembly/code originating from external untrusted sources, I think you don't really need to defend against your user accessing your properties (Not from security perspective anyway - usability is another thing).
EDIT - about external untrusted code
I still think this is not really your concern. If I understand correctly, you are developing and providing a library, to be used by some 3rd party in their application.
If the application owner decided to take some external library which he does not trust, add it to his application, and allow it to run, then this is not your concern, it is the application owner's concern.
In this scenario, everything I said above still applies. The malicious code can do much worse then setting your properties. It can mess with memory, corrupt data, flood the thread pool, or even easily crash the AppDomain.
The point is, if you don't own the application because you are only providing a class library, you don't need to defend from code running inside the AppDomain where you classes are loaded.
Note: Re. NuGet, I wouldn't be too worried about that. NuGet is sort of a static tool. If I understand correctly, it doesn't do things in runtime such as downloading code and running it. It is only used in design time to download binaries, add references, and possibly add code. I think it's perfectly reasonable to assume that an application owner that uses NuGet to download a package will do his due diligence to ensure that the package is safe. And he has to do it only once, during development.
As the previous answers note, there isn't really much of a difference here.
Malicious code could set a static property, and malicious code could change a configuration file. The latter is probably a bit easier to figure out from the outside, and can be done no matter what way the code is run (it wouldn't have to be .NET, wouldn't have to be run in your app domain, and indeed wouldn't have to be code, should someone gain the ability to change the file manually), so there's a bit of a security advantage in the use of a static property, though it's a rather bogus one considering that we may well have just moved the issue around a bit, since the calling code could very well be using configuration itself to decide what to set the properties to!
There's a third possibility, which is that you have an instance with instance members that set the properties, and it's the calling code that makes that instance static. This might be completely irrelevant to what you are doing, but it can be worth considering cases where someone might want to have your code running with two sets of configuration parameters in the same app domain. As a matter of security, it is much the same as the matter of static members, except that it could affect serialisation concerns.
So, so far there's the disadvantage of configuration files in that they can be attacked by code completely separate to yours, but with the noted caveat that the information might end up in a configuration file somewhere else anyway.
Whichever approach you take, the safety of access comes down to the way that you load in partially-trusted code.
The code should be loaded into its own app domain, and the security on that app domain set appropriately to how well it can be trusted. If at all possible, it shouldn't be your library that is doing so, but left to the calling code to decide upon the policies to be set by any partially-trusted code it loads in. Of course, if it's inherent to your libraries purpose that it loads in partially-trusted code, then it must do so, but generally it should remain agnostic as to whether the code is fully or partially trusted except in demanding certain permissions when appropriate. If it is up to your library to load in this code, then you will need to decide upon the appropriate permissions to give the app domain. Really, this should be the lowest amount of permission where it is still possible to do the job it was loaded in for. This would presumably not include FileIOPermission, hence preventing it from writing to a config file.
Now, whether your library or the calling code has loaded the partially trusted code, you need to consider what permissions are necessary on your exposed classes and their members. This covers the static setter properties, but would still be necessary if you took the config-file approach given that your scenario still involves that there is partially-trusted code accessing your library.
In some cases, the methods won't need any more protection, because they inherently have it due to what they do. For example, if you try to access a file but the calling code does not have permission to do so, then your code will fail with a security exception that will be passed up to the calling code. Indeed, you may have to do the opposite and take measures to allow the partially-trusted code to call your method (if you access a file in a way that is safe because the caller cannot affect which file is accessed or how, you may want to Assert file-access permissions at that point).
In other cases, you may need to add protection because calling code won't do anything that immediately attempts a security-restricted operation but which may cause trusted code to behave in an inappropriate manner. For example, if your code stores paths that are used by later operations, then essentially calling that code allows for file access to happen in a particular way. E.g.:
public string TempFilePath{get;set;}
public void WriteTempData(string data)
{
using(sw = new StreamWriter(TempFilePath, true))
sw.Write(data);
}
Here if malicious code set TempDirPath it could cause a later call by trusted code to WriteTempData to damage an important file by over-writing it. An obvious approach here is to call Demand on an appropriate FileIOPermission object, so that the only code that could set it would be code that was already trusted to write to arbitrary locations anyway (this could of course be combined by both restricting the possible values for TempDirPath and demanding the ability to write within the set of locations that allowed).
You can also demand certain unions of permission, and of course create your own permissions, though using unions of those defined by the framework has an advantage of better fitting in with existing code.
What prevents a malicious component from making changes at runtime?
This depends on the definition of "malicious component". Configuration is really intended to allow changes at runtime.
If you handle this via code (whether static or instance properties, etc), you do have the distinct advantage of controlling the allowable settings directly, as your property setter can control this however you wish. You could also add some form of security, if your application requires it, as you'd control the way this was set.
With a configuration section, your only control would be in reading the values - you couldn't control the writing, but instead would have to validate settings on read.
For sure, it can be changed by underlying classes which provide those abstractions, even in case of being defined as private members.
Think of a security interceptor that provision every request against defined privileges of authenticated or anonymous users.
I generally use Config file and Static variables together. I define static variable as private, and i make only "get" method to expose value. so it is can not be changed out of class.
I create a class to handle configuration implementing "IConfigurationSectionHandler" interface. My implementation is for ASP.NET Web applications.
Step 1: Create a section in web.config file to process later.
<configuration>
<configSections>
<section name="XXXConfiguration" type="Company.XXXConfiguration, Company"/>
...
</configSections>
<XXXConfiguration>
<Variable>Value to set static variable</Variable>
</XXXConfiguration>
...
<configuration>
Step 2: Create a class to handle previous configuration section.
using System;
using System.Collections;
using System.Collections.Specialized;
using System.Xml;
using System.Configuration;
namespace Company{
public class XXXConfiguration : IConfigurationSectionHandler
{
/// <summary>
/// Initializes a new instance of LoggingConfiguration class.
/// </summary>
public XXXConfiguration() {}
private static string _variable;
public static string Variable
{
get {return XXXConfiguration._variable; }
}
public object Create(object parent, object configContext, XmlNode section)
{
// process config section node
XXXConfiguration._variable = section.SelectSingleNode("./Variable").InnerText;
return null;
}
}
}
Step 3: Use GetSection method of System.Configuration.ConfigurationManager at startup of application. In Global.asax
void Application_Start(object sender, EventArgs e)
{
// Code that runs on application startup
System.Configuration.ConfigurationManager.GetSection("LoggingConfiguration");
...
}

.NET 3.5: Dynamically adding classes without recompiling

I'd like to build an infrastructure that will monitor a server and check ping, response time, and more.
The catch is that future checks (for example: does a certain file exist) will be added without recompiling the infrastructure.
How do I build it in such a way that will enable me to attach inherited classes with different functionality, and execute them without recompiling?
Thanks!
In addition to creating an interface and defining a single entry point for your new library, you could create an attribute which identifies the classes that you need to load or the methods that you need to call. You then use reflection to look at all the DLLs in a certain path, and instantiate / run whatever contains your attribute.
I've built a similar application that had to perform a number of health checks on a system, and needed to be extensible. The application started, looked through all the DLLs in a specified path and for each class with the 'TestAttribute' decoration it would create an instance and run the 'Execute' method.
The use of an attribute means that you don't need to specify which DLLs to process (doesn't need to be in config / database) because it's safe to process every DLL, and only those decorated with the attribute will do anything.
Implement an interface, and the provider pattern, then you can plug anything in that you like. MSBuild is a great example of this, with a simple interface you can add any type of task you like to your build process - follow the same sort of pattern.
Sounds like you could use some kind of 'plugin' mechanism. Define a basic interface and you can compile every "check/action" into a separate assembly. Load all your assemblies dynamically from file and call execute the check/action via the defined interface.
The interface could be just as simple as this, for starters:
public interface IMonitorAction
{
bool Exectute();
}
This infrastructure allows you to add more checks by just creating another assembly file implementing the interface next to the existing ones.
Of the top of my head.
I presume you can re-start you application.
Have a file that lists all the DLL's to load that implement your required functionality. Each DLL should have the same name entry point. Load each DLL, call the method, unload DLL. loop.
Caveat: I've never done anything like this, so I may be talking hot air.
Adding to #slugsters answer, instead of building your own extensibility infrastructure, take a look at extensibility libraries like MEF.

Categories