We Keep Coding

Garbage collection of global Application object in ASP.NET MVC C#

I have ran into a problem where a long running singleton added to HttpApplicationState which does some data masking (GDPR), stops masking data after running in the background for some time.
It's hard to debug because it only happens in our UAT enviroment and it usually happens overnight.
The problem is that the data masking library is third party, and is still work in progress (or at the end of that work in progress).
But I'd appreciate if anyone with better GC knowledge could look at the init code below, and confirm this is out of GC domain.
The Translator.GetInstance() is a Lazy loader of the GDPR masking/translation singleton. So it's initialized the first time the user masks/unmasks the data.
protected void Application_Start()
{
if (Translator)
{
Application["MaskDataUtility"] = new MaskDataUtility(Translator.GetInstance());
}
else
{
Application["MaskDataUtility"] = new MaskDataUtility(new CustomTranslator());
}
}

How do I properly dispose and free the memory used for V8.Net.V8Engine instances?

I'm running into an issue when using my V8Engine instance, it appears to have a small memory leak, and disposing of it, as well as forcing the garbage collection doesn't seem to help much. It will eventually throw an AccessViolationException on V8Enging local_m_negine = new V8Engine() claiming a Fatal error in heap setup, Allocation failed - process out of memory and Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Monitoring the program's memory usage through Task manager whilst running confirms that it is leaking memory, around 1000 KB every couple of seconds I think. I suspect it is the variables being declared within the executed script not being collected, or something to do with the GlobalObject.SetProperty method. Calling V8Engine.ForceV8GarbageCollection(), V8Engine.Dispose() and even GC.WaitForPendingFinalizers() & GC.Collect() doesn't prevent this memory being leaked (Although it is worth noting that it seems to leak it slower with these commands in place, and I know I shouldn't use GC but it was there as a last resort to see if it would fix the issue.)
A tangential issue that could also provide a solution is the inability to clear the execution context for V8Engine. I am required to dispose and re-instantiate the engine for each script, which I believe is where the memory leak is happening, otherwise I run into issues where variables have already been declared, causing V8Engine.Execute() to throw an exception saying such.
I can definitely confirm that the memory leak is something to do with the V8Engine Implementation, as running the older version of this program that uses Microsoft.JScript has no such memory leak, and the memory used remains consistent.
The affected code is as follows;
//Create the V8Engine and dispose when done
using (V8Engine local_m_engine = new V8Engine())
{
//Set the Lookup instance as a global object so that the JS code in the V8.Net wrapper can access it
local_m_engine.GlobalObject.SetProperty("Lookup", m_lookup, null, true, ScriptMemberSecurity.ReadOnly);
//Execute the script
result = local_m_engine.Execute(script);
//Please just clear everything I can't cope.
local_m_engine.ForceV8GarbageCollection();
local_m_engine.GlobalObject.Dispose();
}
EDIT:
Not sure how useful this will be but I've been running some memory profiling tools on it and have learnt that after running an isolated version of the original code, My software ends up with a large amount of instances of IndexedObjectList's full of null values (see here: http://imgur.com/a/bll5K). It appears to have one instance of each class for each V8Engine instance that is made, but they aren't being disposed or freed. I cant help but feel like I'm missing a command or something here.
The code I'm using to test and recreate the memory leak that the above implementation causes is as follows:
using System;
using V8.Net;
namespace V8DotNetMemoryTest
{
class Program
{
static void Main(string[] args)
{
string script = #" var math1 = 5;
var math2 = 10;
result = 5 + 10;";
Handle result;
int i = 0;
V8Engine local_m_engine;
while (true)
{
//Create the V8Engine and dispose when done
local_m_engine = new V8Engine();
//Set the Lookup instance as a global object so that the JS code in the V8.Net wrapper can access it
//local_m_engine.GlobalObject.SetProperty("Lookup", m_lookup, null, true, ScriptMemberSecurity.ReadOnly);
//Execute the script
result = local_m_engine.Execute(script);
Console.WriteLine(i++);
result.ReleaseManagedObject();
result.Dispose();
local_m_engine.Dispose();
GC.WaitForPendingFinalizers();
GC.Collect();
local_m_engine = null;
}
}
}
}

Sorry, I had no idea this question existed. Make sure to use the v8.net tag.
Your problem is this line:
result = local_m_engine.Execute(script);
The result returned is never disposed. ;) You are responsible for returned handles. Those handles are struct values, not class objects.
You could also do using (result = local_m_engine.Execute(script)) { ... }
There is a new version released. I am finally resurrecting this project again as I will need it for the FlowScript VPL project - and it now supports .Net Standard as well for cross-platform support!

Unmanaged Leak in Calling Web service in C#

I need help with my code.
Ive been trying to use RedGate to monitor the memory usage of my application, after hours of testing It pointed out the unmanaged code of my application, and I could only think of my webservice calling as the only or somewhat unmanaged part of my code. Ive been debugging for hours and cant seem to find out where or what really happened. below is my code.
private void btnstart_Click(object sender, EventArgs e)
{
btnPressed = !btnPressed //boolean
if(btnPressed)
{
myTask = Task.Factory.StartNew(() =>
{
do {
_checkMatches(token.Token);
} while (token.IsCancellationRequested != true);
},token.Token);
}
else
{
token.Cancel();
try {
Task.WaitAny(myTask);
}
catch(Exception Error) {
//Put to logs
}
finally
{
if(myTask.isCancelled || myTask.IsCompleted || myTask.IsFaulted)
{
myTask.Dispose();
}
}
}
}
private void _checkMatches(CancellationToken token)
{
try
{
if(token.IsCancellationRequested)
{
token.ThrowIfCancellationRequested();
}
EndpointAddressBuilder ServiceEndPoint = new EndpointAddressBuilder(//Read Endpoint From Notepad);
ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(delegate { return true; });//just bypassing validation for test purposes
// WebService is an ASMX web service
using (WebService.SoapClient client = new WebService.SoapClient())
{
WebService.checkResponse clientRes = client.checkClient();
if(clientRes.response == 1 || clientRes.response == 2) {
//Put to Logs
}
}
}
catch(Exception error){
//Put to logs
}
}
I cant seem to find any error in this code. Can Someone Help me with what is the problem why my unmanaged code is leaking? or could someone suggest what tools to be used or even suggest to find the leaking part of my code? Any Help would be great.

If you are using RedGate memory profiler it should tell you more than just pointing unmanaged resources. Because you are not directly creating any unamanged resources, most likely your memory leak is caused by a managed object or an event subsciption(or callback subscription).
Standard .net managed objects which have unmanaged resources (ie : WebService.SoapClient) have implemented the finalizer(destructor) in order to get rid of its unmanaged resources in any case if Dispose is not called. If you use a standard .net managed object which has unmanaged resources and don't call the dispose (this is bad), still it will release unamanged resources when it is going through the finalization. Generally this shouldn't cause unmanaged resource memory leak.
How to check if there is any memory leak:
Execute your memory consumption logic a few times. Go to the Instance list and see if there is any instances are growing. Select the 'Objects with Source' option in order to get rid of heaps of system object. You should see more instances that there should be if there is any memory leak.
If there is any growing instances, pick one and see the objects retention graph which will show you exactly which instance holds the reference.
And also, make sure that you have implemented IDisposable properly and dispose all disposable objects and unsubscribe from all event subscriptions.
Have a look at below walkthroughs
http://www.red-gate.com/products/dotnet-development/ants-memory-profiler/walkthrough http://www.red-gate.com/products/dotnet-development/ants-memory-profiler/

How do I implement asynchrounous caching?

We're using the following pattern to handle caching of universal objects for our asp.net application.
private object SystemConfigurationCacheLock = new object();
public SystemConfiguration SystemConfiguration
{
get
{
if (HttpContext.Current.Cache["SystemConfiguration"] == null)
lock (SystemConfigurationCacheLock)
{
if (HttpContext.Current.Cache["SystemConfiguration"] == null)
HttpContext.Current.Cache.Insert("SystemConfiguration", GetSystemConfiguration(), null, DateTime.Now.AddMinutes(1), Cache.NoSlidingExpiration, new CacheItemUpdateCallback(SystemConfigurationCacheItemUpdateCallback));
}
return HttpContext.Current.Cache["SystemConfiguration"] as SystemConfiguration;
}
}
private void SystemConfigurationCacheItemUpdateCallback(string key, CacheItemUpdateReason reason, out object expensiveObject, out CacheDependency dependency, out DateTime absoluteExpiration, out TimeSpan slidingExpiration)
{
dependency = null;
absoluteExpiration = DateTime.Now.AddMinutes(1);
slidingExpiration = Cache.NoSlidingExpiration;
expensiveObject = GetSystemConfiguration();
}
private SystemConfiguration GetSystemConfiguration()
{
//Load system configuration
}
The problem is that when under load (~100,000 users) we see a huge jump in TTFB as the CacheItemUpdateCallback blocks all the other threads from executing until it has finished refreshing the cache from the database.
So what I figured we needed is solution that when the first thread after an expiry of the cache attempts to access it, an asynchronous thread is fired off to update the cache but still allows all other executing threads to read from the old cache until it has sucessfully updated.
Is there anything built into the .NET framework that can natively handle what I'm asking, or will I have to write it from scratch? Your thoughts please...
A couple of things...
The use of the HttpContext.Current.Cache is incidental and not necessarily essential as we've got no problem using private members on a singleton to hold the cached data.
Please don't comment on the cache times, SPROC effeciency, why we're caching in the first place etc as it's not relevent. Thanks!

AppFabric might be a good fit for what you're looking for.
http://msdn.microsoft.com/en-us/windowsserver/ee695849
http://msdn.microsoft.com/en-us/library/ff383731.aspx

So it turns out after several hours of investigation that the problem is not the CacheItemUpdateCallback blocking other threads as I originally thought, in fact it did exactly what I wanted it to asynchronously but it was the garbage collector stopping everything to clean up the LOH.

Window Service and C# design pattern question

I have recently taken over a legacy windows service and it has been writing the following event in the system event log:
Event ID: 7034 Description: The
MyService service terminated
unexpectedly. It has done this X
time(s).
I was looking over source code and found the following code pattern in the service class library:
(It has been simplified to protect the innocent..)
public static void StartService()
{
//do some stuff...
ManageCycle();
}
public static void ManageCycle()
{
//do some stuff
ManageCycle();
}
What is this coding patten called and could it possibly cause the windows service to shutdown (i.e. memory leak)?

This looks like the stack overflow exception pattern. Eran is correct. Use a while loop:
public static void StartService()
{
//do some stuff...
isRunning = true;
ManageCycle();
}
public static void ManageCycle()
{
while(isRunning)
{
//do some stuff and wrap in exception handling
}
}
public static void StopService()
{
isRunning=false;
}

It suppose to throw StackOverflow (HA HA :) ) Exception, because of the endless recursive calling.
Take a look at this example - you should choose the technique that fits your architecture.

That's a recursive call that will ultimately blow the stack.

The best answer for this kind of situation:
Don't use Recursive Algorithms unless your algorithm has a recursive structure. For example, if you're analyzing a file system, and want to scan a specific Directory, you'd want to do something like:
void ScanDirectory(Directory)
{
// Handle Files
if (currfile.directory)
ScanDirectory(currfile)
}
This makes sense because it's much easier than doing it iteratively. But otherwise, when you're just repeating an action over and over again, making it a recursion is completely unnecessary and will cause code inefficiency and eventually stack overflows.

This is a recursive call with apparently no exit criteria. Eventually it will run out of stack since a call to ManageCycle never returns.
In addition the StartService method will never return, it ought to be spining up at least one foreground thread and then return.

Recursion, it ooks like it's recursively calling itself. I'm surprised there isn't a stack overflow exception. Perhaps the service property on the machine running this is configured to restart the service on failure.

It's recursive alright. It will keep calling itself repeatedly (a bad thing) and that will result in a stackoverflow.
What the "//do some stuff" do? Maybe there is a good reason that it calls itself,
bBut without a way to get out of the loop (recursive), the application will exit.