We Keep Coding

ScriptService execute eating memory

When I run Execute on a ScriptService run request, it takes some memory and fails to release it. Testing this by running on monodevelop on a Raspberry Pi shows the memory rising at an alarming rate, and will eventually crash the program. The GC.Collect was an attempt at re-claiming this memory. Is there any insight into what I am doing wrong?
public MainWindow() : base(Gtk.WindowType.Toplevel)
{
Build();
while (true)
{
getDashRow();
Thread.Sleep(1000);
Console.WriteLine("Total available memory before collection: {0:N0}", System.GC.GetTotalMemory(false));
System.GC.Collect();
Console.WriteLine("Total available memory collection: {0:N0}", System.GC.GetTotalMemory(true));
}
}
private int getDashRow()
{
ScriptsResource.RunRequest runreq;
DriveService driveservice;
ExecutionRequest exrequest;
Console.WriteLine("getDashRow");
int retval = 0;
exrequest = new ExecutionRequest();
exrequest.Function = "getMacRow";
IList<object> parameters = new List<object>();
parameters.Add(spreadsheetname);
exrequest.Parameters = parameters;
exrequest.DevMode = false;
try
{
// run a Google Apps Script function on the online sheet to find number of rows (more efficient)
runreq = scriptservice.Scripts.Run(exrequest, dashscriptid);
// following line consumes the memory
Operation op = runreq.Execute();
retval = Convert.ToInt16(op.Response["result"]);
parameters = null;
exrequest = null;
op = null;
}
catch (Exception ex)
{
Console.WriteLine("getDashRow: " + ex.Message);
}
return retval;
}

I solved this, installing Mono Preview v6.0.0.277 has resolved this problem, memory is now correctly freed up without manually calling GC.Collect().
Related issue which led to the solution

Start a process, and if one of its threads is hogging CPU, suspend it

There's a well-known problem that Skype on Windows 8 takes up 100% of one CPU core on some users' PCs. Including mine! There's a workaround courtesy of techfreak in Skype Community:
Download and run the latest version of process explorer. (http://download.sysinternals.com/files/ProcessExplorer.zip)
With Skype running search for Skype.exe in the list of active programs and double click on it.
Go to the threads tab and Suspend or Kill the Skype thread that is consuming the highest resources when IDLE. (like 50%+ CPU)
I'm getting annoyed with manually doing this after every reboot, so I'd like to automate the steps above, to write a simple C++ or C# "Skype launcher" program that does the following:
launch SKYPE.EXE
wake up every 1 second and look to see if one particular Skype thread is taking up over 98% of the CPU cycles in the process
if found, suspend that thread and exit the launcher process
otherwise loop up to 10 times until the bad thread is found.
After a quick Google search I got intimidated by the Win32 thread-enumeration APIs, and this "find and kill/suspend evil thread" problem seems to be fairly generic, so I'm wondering if there's an existing sample out there that I could re-purpose. Any pointers?

After much more googling and some dead ends with powershell (too many security hassles, too confusing for a newbie) and WMI (harder than needed), I finally found a great C# sample on MSDN Forums that will enumerate and suspend threads. This was easy to adapt to first check CPU time of each thread before suspending the culprit.
Here's code. Just compile and drop into your startup menu and Skype will no longer heat your office!
// code adapted from
// http://social.msdn.microsoft.com/Forums/en-US/d51efcf0-7653-403e-95b6-bf5fb97bf16c/suspend-thread-of-a-process
using System;
using System.Linq;
using System.Threading.Tasks;
using System.Runtime.InteropServices;
using System.Diagnostics;
using System.Threading;
using System.ComponentModel;
namespace SkypeLauncher
{
class Program
{
static void Main(string[] args)
{
Process[] procs = Process.GetProcessesByName("skype");
if (procs.Length == 0)
{
Console.WriteLine("Skype not loaded. Launching. ");
Process.Start(Environment.ExpandEnvironmentVariables(#"%PROGRAMFILES(X86)%\Skype\Phone\Skype.exe"));
Thread.Sleep(8000); // wait to allow skype to start up & get into steady state
}
// wait to allow skype to start up & get into steady state, where "steady state" means
// a lot of threads created
Process proc = null;
for (int i = 0; i < 50; i++)
{
procs = Process.GetProcessesByName("skype");
if (procs != null)
{
proc = procs[0];
if (proc.Threads.Count > 10)
break;
}
Thread.Sleep(1000); // wait to allow skype to start up & get into steady state
}
// try multiple times; if not hanging after a while, give up. It must not be hanging!
for (int i = 0; i < 50; i++)
{
// must reload process to get updated thread time info
procs = Process.GetProcessesByName("skype");
if (procs.Length == 0)
{
Console.WriteLine("Skype not loaded. Exiting. ");
return;
}
proc = procs[0];
// avoid case where exception thrown if thread is no longer around when looking at its CPU time, or
// any other reason why we can't read the time
var safeTotalProcessorTime = new Func<ProcessThread, double> (t =>
{
try { return t.TotalProcessorTime.TotalMilliseconds; }
catch (InvalidOperationException) { return 0; }
}
);
var threads = (from t in proc.Threads.OfType<ProcessThread>()
orderby safeTotalProcessorTime(t) descending
select new
{
t.Id,
t.ThreadState,
TotalProcessorTime = safeTotalProcessorTime(t),
}
).ToList();
var totalCpuMsecs = threads.Sum(t => t.TotalProcessorTime);
var topThread = threads[0];
var nextThread = threads[1];
var topThreadCpuMsecs = topThread.TotalProcessorTime;
var topThreadRatio = topThreadCpuMsecs / nextThread.TotalProcessorTime;
// suspend skype thread that's taken a lot of CPU time and
// and it has lots more CPU than any other thread.
// in other words, it's been ill-behaved for a long time!
// it's possible that this may sometimes suspend the wrong thread,
// but I haven't seen it break yet.
if (topThreadCpuMsecs > 10000 && topThreadRatio > 5)
{
Console.WriteLine("{0} bad thread. {0:N0} msecs CPU, {1:N1}x CPU than next top thread.",
topThread.ThreadState == System.Diagnostics.ThreadState.Wait ? "Already suspended" : "Suspending",
topThreadCpuMsecs,
topThreadRatio);
Thread.Sleep(1000);
IntPtr handle = IntPtr.Zero;
try
{
//Get the thread handle & suspend the thread
handle = OpenThread(2, false, topThread.Id);
var success = SuspendThread(handle);
if (success == -1)
{
Win32Exception ex = new Win32Exception(Marshal.GetLastWin32Error());
Console.WriteLine(ex.Message);
}
Console.WriteLine("Exiting");
Thread.Sleep(1000);
return;
}
finally
{
if (handle != IntPtr.Zero)
CloseHandle(handle);
};
}
Console.WriteLine("Top thread: {0:N0} msecs CPU, {1:N1}x CPU than next top thread. Waiting.",
topThreadCpuMsecs,
topThreadRatio);
Thread.Sleep(2000); // wait between tries
}
Console.WriteLine("No skype thread is ill-behaved enough. Giving up.");
}
[DllImport("kernel32.dll", SetLastError = true)]
private static extern int SuspendThread(IntPtr hThread);
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool CloseHandle(IntPtr handle);
[DllImport("kernel32.dll", SetLastError = true)]
private static extern
IntPtr OpenThread(int dwDesiredAccess, [MarshalAs(UnmanagedType.Bool)]bool bInheritHandle, int dwThreadId);
}
}

C# application with unmanaged DLL freezes whole system

I'm currently writing a software in Visual Studio 2012 for communication with RFID-cards.
I got a DLL written in Delphi to handle the communication with the card reader.
The problem is: My software is running fine on machines, that have VS2012 installed. On other systems it freezes itself or the whole system.
I tried it on Win XP / 7 / 8 with x32 and x64 configuration.
I'm using .NET 4.0.
After connecting to the reader, the software starts a backgroundWorker, which polls (at 200ms rate) the reader with a command to inventory cards in the readers RF-field. The crash usally happens ca. 10 to 20 seconds after the reader connect. Here is the code:
[DllImport("tempConnect.dll", CallingConvention = CallingConvention.StdCall)]
private static extern int inventory(int maxlen, [In] ref int count,
IntPtr UIDs, UInt32 HFOffTime);
public String getCardID()
{
if (isConnectet())
{
IntPtr UIDs = IntPtr.Zero;
int len = 2 * 8;
Byte[] zero = new Byte[len];
UIDs = Marshal.AllocHGlobal(len);
Thread.Sleep(50);
Marshal.Copy(zero, 0, UIDs, len);
int count = 0;
int erg;
String ret;
try
{
erg = inventory(len, ref count, UIDs, 50);
}
catch (ExternalException) // this doesn't catch anything (iI have set <legacyCorruptedStateExceptionsPolicy enabled="true"/>)
{
return "\0";
}
finally
{
ret = Marshal.PtrToStringAnsi(UIDs, len);
IntPtr rslt = LocalFree(UIDs);
GC.Collect();
}
if (erg == 0)
return ret;
else
return zero.ToString();
}
else
return "\0";
}
The DLL is written in Delphi, the code DLL command is:
function inventory (maxlen: Integer; var count: Integer;
UIDs: PByteArray; HFOffTime: Cardinal = 50): Integer; STDCALL;
I think there may be a memory leak somewhere, but I have no idea how to find it...
EDIT:
I added some ideas (explicit GC.Collect(), try-catch-finally) to my code above, but it still doesnt work.
Here is the code, that calls getCardID():
The action, that runs every 200ms:
if (!bgw_inventory.IsBusy)
bgw_inventory.RunWorkerAsync();
Async backgroundWorker does:
private void bgw_inventory_DoWork(object sender, DoWorkEventArgs e)
{
if (bgw_inventory.CancellationPending)
{
e.Cancel = true;
return;
}
else
{
String UID = reader.getCardID();
if (bgw_inventory.CancellationPending)
{
e.Cancel = true;
return;
}
if (UID.Length == 16 && UID.IndexOf("\0") == -1)
{
setCardId(UID);
if (!allCards.ContainsKey(UID))
{
allCards.Add(UID, new Card(UID));
}
if (readCardActive || deActivateCardActive || activateCardActive)
{
if (lastActionCard != UID)
actionCard = UID;
else
setWorkingStatus("OK", Color.FromArgb(203, 218, 138));
}
}
else
{
setCardId("none");
if (readCardActive || deActivateCardActive || activateCardActive)
setWorkingStatus("waiting for next card", Color.Yellow);
}
}
}
EDIT
Till now I have made some little reworks (updates above) at the code. Now only the App. crashes with 0xC00000FD (Stack overflow) at "tempConnect.dll". This does not happen on Systems with VS2012 installed or if I use the DLL with native Delphi!
Do anyone have any other ideas ?
EDIT
Now I made the DLL logging it's stacksize and found something weird:
If it's called and polled from my C# Programm, the stacksize is changing continuously up and down.
If i do the same from a natural Deplhi Program the stacksize is constant!
So I'll do further investigations, but I have no really idea, what I have to search for...

I'm a little concerned about how're using that Marshal object. As you fear with the memory leak, it seems to be allocating memory quite often but I don't see it ever explicitly releasing it. The garbage collector should (operative word) be taking care of that, but you say yourself you have some unmanaged code in the mix. It is difficult with the posted information to tell where the unmanaged code begins.
Check out this question for some good techniques to finding memory leaks in .NET itself - this will give you a ton of information on how memory is being used in the managed end of your code (that is, the part you can directly control). Use the Windows Performance Monitor with breakpoints to keep an eye on the overall health of the system. If .NET appears to be behaving, but WPM is showing some sharp spikes, it's probably in the unmanaged code. You can't really control anything but your usage there, so it would probably be time to go back to the documentation at that point.

Unload loaded dlls of external process

Does somebody know how to unload a dll or any other type of module loaded by an external process?
I tried to do GetModuleHandle and then FreeLibrary with no result...
Thank you for all your replies
Thank you for all your replies. I found an interesting msdn article here :
http://blogs.msdn.com/b/jmstall/archive/2006/09/28/managed-create-remote-thread.aspx
The problem is that when i try to do a OpenProcess the external process crashes.
What are the minimum process access rights to unload a module from it ?
Here is what i am trying to do in c# :
[code]
protected const int PROCESS_ALL_ACCESS = (STANDARD_RIGHTS_REQUIRED | SYNCHRONIZE | 0xFFF);
protected const int STANDARD_RIGHTS_REQUIRED = 0xF0000;
protected const int SYNCHRONIZE = 0x100000;
public static bool UnloadRemoteModule(FileEntry le)
{
try
{
Process process = System.Diagnostics.Process.GetProcessById(le.ProcessID);
if (process == null) return false;
StringBuilder sb = new StringBuilder(le.File);
UnloadModuleThreadProc umproc = new UnloadModuleThreadProc(UnloadModule);
IntPtr fpProc = Marshal.GetFunctionPointerForDelegate(umproc);
SafeProcessHandle processHandle = null;
IntPtr currentProcess = NativeMethods.GetCurrentProcess();
int processId = le.ProcessID;
bool remote = (processId != NativeMethods.GetProcessId(currentProcess));
try
{
if (remote)
{
MessageBox.Show("OPENING PROCESS !");
processHandle = NativeMethods.OpenProcess(PROCESS_ALL_ACCESS, true, processId);
System.Threading.Thread.Sleep(200);
uint dwThreadId;
if (processHandle.DangerousGetHandle() == IntPtr.Zero)
{
MessageBox.Show("COULD NOT OPEN HANDLE !");
}
else
{
// Create a thread in the first process.
IntPtr hThread = CreateRemoteThread(
processHandle.DangerousGetHandle(),
IntPtr.Zero,
0,
fpProc, IntPtr.Zero,
0,
out dwThreadId);
System.Threading.Thread.Sleep(200);
WaitForThreadToExit(hThread);
}
}
return true;
}
finally
{
if (remote)
{
if (processHandle != null)
{
processHandle.Close();
}
}
}
return false;
}
catch (Exception ex)
{
//Module.ShowError(ex);
return false;
}
}
public delegate int UnloadModuleThreadProc(IntPtr sb_module_name);
static int UnloadModule(IntPtr sb_module_name2)
{
using (StreamWriter sw = new StreamWriter(#"c:\a\logerr.txt"))
{
sw.AutoFlush = true;
sw.WriteLine("In Unload Module");
StringBuilder sb_module_name =new StringBuilder(#"C:\Windows\System32\MyDll.dll");
IntPtr mh = DetectOpenFiles.GetModuleHandle(sb_module_name.ToString());
sw.WriteLine("LAST ERROR="+Marshal.GetLastWin32Error().ToString());
sw.WriteLine("POINTER="+mh.ToInt32());
if (mh != IntPtr.Zero)
{
return (FreeLibrary(mh) ? 1 : 0);
}
sw.WriteLine("LAST ERROR 2 =" + Marshal.GetLastWin32Error().ToString());
sw.WriteLine("EXIT " + mh.ToInt32());
}
return 0;
}[/code]

You can do it, but honestly I must ask why? You're most likely going to screw things up beyond what you realize. Seriously, there's nothing that can go right if you do this. Don't read the rest of this post, close your browser, do some meditation, and figure out what you're doing wrong that made you ask this question.
HERE BE DRAGONS
That said, it can be done, and rather easily too.
All you have to do is use CreateRemoteThread, pass a handle to the process you want to force unload in, and a function pointer to a function that calls GetModuleHandle and FreeLibrary. Easy as pie.
Sample code (untested, written in vi, and not to be used no matter what):
DWORD WINAPI UnloadNamedModule(void *)
{
//If you value your life, don't use this code
LPCTSTR moduleName = _T("MYMODULE.DLL");
HMODULE module = GetModuleHandle(moduleName);
if (module != NULL)
{
UnloadModule(hModule);
//All hell breaks loose. Not even this comment will be reached.
//On your own head be it. Don't say I didn't warn you.
}
}
//Warning: this function should never be run!
void UnloadRemoteModule(HANDLE hProcess)
{
CreateRemoteThread(hProcess, NULL, 0, UnloadNamedModule, NULL, 0);
}

You cannot force an external process to unload it's modules. You would need to run that code from inside the external process. The best you can hope for is to kill the process that owns the external DLL. It would be extremely dangerous if you could unload a dll from an external process, the code could be running at the time that you pull it out of RAM.
If you are looking to replace the DLL, the best you can do is to rename the DLL and save the new one. That way, the DLL will get use the next time the external process loads it.
Correction to italics above: You can do it but you are asking for big trouble if you do. I still think the best approach is to do what I listed above, rename the DLL and put the new one in it's place for the next time the external process starts. It's a far safer approach if you would like to replace a DLL.

c# memory leak in loop

public void DoPing(object state)
{
string host = state as string;
m_lastPingResult = false;
while (!m_pingThreadShouldStop.WaitOne(250))
{
Ping p = new Ping();
try
{
PingReply reply = p.Send(host, 3000);
if (reply.Status == IPStatus.Success)
{
m_lastPingResult = true;
}
else
{
m_lastPingResult = false;
}
}
catch
{
}
numping = numping + 1;
}
}
Any idea why this code gives me a memory leak? I can see it's this code as changing the wait value to smaller or larger values increases the rate of the memory usage. Does any one have any idea how to resolve it? or how to see what part of the code is causing it?

In some garbage collected languages, there is a limitation that the object isn't collected if the method that created it still hasn't exited.
I believe .net works this way in debug mode. Quoting from this article; note the bolded statement.
http://www.simple-talk.com/dotnet/.net-framework/understanding-garbage-collection-in-.net/
A local variable in a method that is currently running is considered
to be a GC root. The objects referenced by these variables can always
be accessed immediately by the method they are declared in, and so
they must be kept around. The lifetime of these roots can depend on
the way the program was built. In debug builds, a local variable lasts
for as long as the method is on the stack. In release builds, the JIT
is able to look at the program structure to work out the last point
within the execution that a variable can be used by the method and
will discard it when it is no longer required. This strategy isn’t
always used and can be turned off, for example, by running the program
in a debugger.

Garbage collection only happens when there is memory pressure, thus just seeing your memory usage go up doesn't mean there is a memory leak and in this code I don't see how there could be a legitimate leak. You can add
GC.Collect();
GC.WaitForPendingFinalizers();
to double check but shouldn't leave that in production.
Edit: someone in comments pointed out that Ping is Disposable. not calling dispose can cause leaks that will eventually get cleaned up but may take a long time and cause non memory related problems.

Add a finally statement to your try-catch, like this:
catch() {}
finally
{
Ping.Dispose();
}

using(var p = new Ping())
{
try
{
var reply = p.Send(host, 3000);
if (reply.Status == IPStatus.Success)
_lastPingResult = true;
else
_lastPingResult = false;
}
catch(Exception e)
{
//...
}
}

This can be used from a static Class:
public static bool testNet(string pHost, int pTimeout)
{
Ping p = new Ping();
bool isNetOkay = false;
int netTries = 0;
do
{
PingReply reply = p.Send(pHost, pTimeout);
if (reply.Status == IPStatus.Success)
{
isNetOkay = true;
break;
}
netTries++;
} while (netTries < 4);
//Void memory leak
p.Dispose();
return isNetOkay;
}