I'm currently writing a software in Visual Studio 2012 for communication with RFID-cards.
I got a DLL written in Delphi to handle the communication with the card reader.
The problem is: My software is running fine on machines, that have VS2012 installed. On other systems it freezes itself or the whole system.
I tried it on Win XP / 7 / 8 with x32 and x64 configuration.
I'm using .NET 4.0.
After connecting to the reader, the software starts a backgroundWorker, which polls (at 200ms rate) the reader with a command to inventory cards in the readers RF-field. The crash usally happens ca. 10 to 20 seconds after the reader connect. Here is the code:
[DllImport("tempConnect.dll", CallingConvention = CallingConvention.StdCall)]
private static extern int inventory(int maxlen, [In] ref int count,
IntPtr UIDs, UInt32 HFOffTime);
public String getCardID()
{
if (isConnectet())
{
IntPtr UIDs = IntPtr.Zero;
int len = 2 * 8;
Byte[] zero = new Byte[len];
UIDs = Marshal.AllocHGlobal(len);
Thread.Sleep(50);
Marshal.Copy(zero, 0, UIDs, len);
int count = 0;
int erg;
String ret;
try
{
erg = inventory(len, ref count, UIDs, 50);
}
catch (ExternalException) // this doesn't catch anything (iI have set <legacyCorruptedStateExceptionsPolicy enabled="true"/>)
{
return "\0";
}
finally
{
ret = Marshal.PtrToStringAnsi(UIDs, len);
IntPtr rslt = LocalFree(UIDs);
GC.Collect();
}
if (erg == 0)
return ret;
else
return zero.ToString();
}
else
return "\0";
}
The DLL is written in Delphi, the code DLL command is:
function inventory (maxlen: Integer; var count: Integer;
UIDs: PByteArray; HFOffTime: Cardinal = 50): Integer; STDCALL;
I think there may be a memory leak somewhere, but I have no idea how to find it...
EDIT:
I added some ideas (explicit GC.Collect(), try-catch-finally) to my code above, but it still doesnt work.
Here is the code, that calls getCardID():
The action, that runs every 200ms:
if (!bgw_inventory.IsBusy)
bgw_inventory.RunWorkerAsync();
Async backgroundWorker does:
private void bgw_inventory_DoWork(object sender, DoWorkEventArgs e)
{
if (bgw_inventory.CancellationPending)
{
e.Cancel = true;
return;
}
else
{
String UID = reader.getCardID();
if (bgw_inventory.CancellationPending)
{
e.Cancel = true;
return;
}
if (UID.Length == 16 && UID.IndexOf("\0") == -1)
{
setCardId(UID);
if (!allCards.ContainsKey(UID))
{
allCards.Add(UID, new Card(UID));
}
if (readCardActive || deActivateCardActive || activateCardActive)
{
if (lastActionCard != UID)
actionCard = UID;
else
setWorkingStatus("OK", Color.FromArgb(203, 218, 138));
}
}
else
{
setCardId("none");
if (readCardActive || deActivateCardActive || activateCardActive)
setWorkingStatus("waiting for next card", Color.Yellow);
}
}
}
EDIT
Till now I have made some little reworks (updates above) at the code. Now only the App. crashes with 0xC00000FD (Stack overflow) at "tempConnect.dll". This does not happen on Systems with VS2012 installed or if I use the DLL with native Delphi!
Do anyone have any other ideas ?
EDIT
Now I made the DLL logging it's stacksize and found something weird:
If it's called and polled from my C# Programm, the stacksize is changing continuously up and down.
If i do the same from a natural Deplhi Program the stacksize is constant!
So I'll do further investigations, but I have no really idea, what I have to search for...
I'm a little concerned about how're using that Marshal object. As you fear with the memory leak, it seems to be allocating memory quite often but I don't see it ever explicitly releasing it. The garbage collector should (operative word) be taking care of that, but you say yourself you have some unmanaged code in the mix. It is difficult with the posted information to tell where the unmanaged code begins.
Check out this question for some good techniques to finding memory leaks in .NET itself - this will give you a ton of information on how memory is being used in the managed end of your code (that is, the part you can directly control). Use the Windows Performance Monitor with breakpoints to keep an eye on the overall health of the system. If .NET appears to be behaving, but WPM is showing some sharp spikes, it's probably in the unmanaged code. You can't really control anything but your usage there, so it would probably be time to go back to the documentation at that point.
Related
There's a well-known problem that Skype on Windows 8 takes up 100% of one CPU core on some users' PCs. Including mine! There's a workaround courtesy of techfreak in Skype Community:
Download and run the latest version of process explorer. (http://download.sysinternals.com/files/ProcessExplorer.zip)
With Skype running search for Skype.exe in the list of active programs and double click on it.
Go to the threads tab and Suspend or Kill the Skype thread that is consuming the highest resources when IDLE. (like 50%+ CPU)
I'm getting annoyed with manually doing this after every reboot, so I'd like to automate the steps above, to write a simple C++ or C# "Skype launcher" program that does the following:
launch SKYPE.EXE
wake up every 1 second and look to see if one particular Skype thread is taking up over 98% of the CPU cycles in the process
if found, suspend that thread and exit the launcher process
otherwise loop up to 10 times until the bad thread is found.
After a quick Google search I got intimidated by the Win32 thread-enumeration APIs, and this "find and kill/suspend evil thread" problem seems to be fairly generic, so I'm wondering if there's an existing sample out there that I could re-purpose. Any pointers?
After much more googling and some dead ends with powershell (too many security hassles, too confusing for a newbie) and WMI (harder than needed), I finally found a great C# sample on MSDN Forums that will enumerate and suspend threads. This was easy to adapt to first check CPU time of each thread before suspending the culprit.
Here's code. Just compile and drop into your startup menu and Skype will no longer heat your office!
// code adapted from
// http://social.msdn.microsoft.com/Forums/en-US/d51efcf0-7653-403e-95b6-bf5fb97bf16c/suspend-thread-of-a-process
using System;
using System.Linq;
using System.Threading.Tasks;
using System.Runtime.InteropServices;
using System.Diagnostics;
using System.Threading;
using System.ComponentModel;
namespace SkypeLauncher
{
class Program
{
static void Main(string[] args)
{
Process[] procs = Process.GetProcessesByName("skype");
if (procs.Length == 0)
{
Console.WriteLine("Skype not loaded. Launching. ");
Process.Start(Environment.ExpandEnvironmentVariables(#"%PROGRAMFILES(X86)%\Skype\Phone\Skype.exe"));
Thread.Sleep(8000); // wait to allow skype to start up & get into steady state
}
// wait to allow skype to start up & get into steady state, where "steady state" means
// a lot of threads created
Process proc = null;
for (int i = 0; i < 50; i++)
{
procs = Process.GetProcessesByName("skype");
if (procs != null)
{
proc = procs[0];
if (proc.Threads.Count > 10)
break;
}
Thread.Sleep(1000); // wait to allow skype to start up & get into steady state
}
// try multiple times; if not hanging after a while, give up. It must not be hanging!
for (int i = 0; i < 50; i++)
{
// must reload process to get updated thread time info
procs = Process.GetProcessesByName("skype");
if (procs.Length == 0)
{
Console.WriteLine("Skype not loaded. Exiting. ");
return;
}
proc = procs[0];
// avoid case where exception thrown if thread is no longer around when looking at its CPU time, or
// any other reason why we can't read the time
var safeTotalProcessorTime = new Func<ProcessThread, double> (t =>
{
try { return t.TotalProcessorTime.TotalMilliseconds; }
catch (InvalidOperationException) { return 0; }
}
);
var threads = (from t in proc.Threads.OfType<ProcessThread>()
orderby safeTotalProcessorTime(t) descending
select new
{
t.Id,
t.ThreadState,
TotalProcessorTime = safeTotalProcessorTime(t),
}
).ToList();
var totalCpuMsecs = threads.Sum(t => t.TotalProcessorTime);
var topThread = threads[0];
var nextThread = threads[1];
var topThreadCpuMsecs = topThread.TotalProcessorTime;
var topThreadRatio = topThreadCpuMsecs / nextThread.TotalProcessorTime;
// suspend skype thread that's taken a lot of CPU time and
// and it has lots more CPU than any other thread.
// in other words, it's been ill-behaved for a long time!
// it's possible that this may sometimes suspend the wrong thread,
// but I haven't seen it break yet.
if (topThreadCpuMsecs > 10000 && topThreadRatio > 5)
{
Console.WriteLine("{0} bad thread. {0:N0} msecs CPU, {1:N1}x CPU than next top thread.",
topThread.ThreadState == System.Diagnostics.ThreadState.Wait ? "Already suspended" : "Suspending",
topThreadCpuMsecs,
topThreadRatio);
Thread.Sleep(1000);
IntPtr handle = IntPtr.Zero;
try
{
//Get the thread handle & suspend the thread
handle = OpenThread(2, false, topThread.Id);
var success = SuspendThread(handle);
if (success == -1)
{
Win32Exception ex = new Win32Exception(Marshal.GetLastWin32Error());
Console.WriteLine(ex.Message);
}
Console.WriteLine("Exiting");
Thread.Sleep(1000);
return;
}
finally
{
if (handle != IntPtr.Zero)
CloseHandle(handle);
};
}
Console.WriteLine("Top thread: {0:N0} msecs CPU, {1:N1}x CPU than next top thread. Waiting.",
topThreadCpuMsecs,
topThreadRatio);
Thread.Sleep(2000); // wait between tries
}
Console.WriteLine("No skype thread is ill-behaved enough. Giving up.");
}
[DllImport("kernel32.dll", SetLastError = true)]
private static extern int SuspendThread(IntPtr hThread);
[DllImport("kernel32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool CloseHandle(IntPtr handle);
[DllImport("kernel32.dll", SetLastError = true)]
private static extern
IntPtr OpenThread(int dwDesiredAccess, [MarshalAs(UnmanagedType.Bool)]bool bInheritHandle, int dwThreadId);
}
}
Days of research and programming have led me to try all variants of inpout32.dll and inpoutx64.dll: binaries, source code, 32-bit, 64-bit, address wrappers. None work: no change is seen to the output bits of the port.
However, I know it is possible, because using another commercially available program that does parallel port output, I can detect a trigger (state change) on all eight output bits (D0-D7) by passing a value between 0 and 255, exactly what I want to do in my own application.
I have followed all the advice from at least these pages:
Write to parallel port in Windows 7
C# LPT inpout32.dll
C# - Read Parallel Port State (Simple Push Switch)
Write to a parallel port on windows 7
Parallel port with C#
http://www.lvr.com/parport.htm
I am using Windows 7, 64-bit; and my SIIG Cyberpro port is mapped as LPT3 at address 0xCCD8, with four status bits at address 0xCCD4. I have another ECP printer port mapped as LPT1 at 0x0378, but that does not work either.
I know better than to try direct _inp(), _outp() calls on Win7.
Can anyone help?
If I need to download and modify the driver code, I can do that if I have to, but I think it should not be that difficult.
My final version of code uses 32-bit compilation, interfacing to inpout32.dll:
using System;
using System.Runtime.InteropServices;
namespace ParallelPort
{
public class PortAccess
{
//inpout.dll
[DllImport("inpout32.dll")]
private static extern void Out32(ushort PortAddress, short Data);
[DllImport("inpout32.dll")]
private static extern short Inp32(ushort PortAddress);
private ushort _PortAddress = 0;
public ushort PortAddress { get { return _PortAddress; } }
public PortAccess(ushort portAddress)
{
_PortAddress = portAddress;
short result = 0;
try
{
result = Inp32(portAddress);
}
catch (DllNotFoundException e)
{
throw new ArgumentException("Unable to find InpOut32.dll");
}
catch (BadImageFormatException)
{
result = 0;
}
if (0 == result)
throw new ArgumentException("Unable to open parallel port driver");
}
//Public Methods
public void Write(ushort Data)
{
Out32(_PortAddress, (short)Data);
}
public byte Read()
{
return (byte)Inp32(_PortAddress);
}
}
}
FYI:
When I added
[DllImport("inpout32.dll")]
private static extern void DlPortWritePortUshort(ushort PortAddress, ushort Data);
and called that function rather than Out32(ushort addr, ushort value) the code worked.
I don't know why the exact interface matters, but it does; and perhaps that is indeed because of sign extension on the 16-bit port address, as suggested [somewhere TBD].
I have been researching this issue pretty extensively and cannot seem to find an answer.
I know that the Only part of a ReadProcessMemory or WriteProcessMemory request was completed exception is thrown when a 32-bit process tries to access a 64-bit process and the same for a 64-bit modifying a 32-bit process.
The solution to that issue is to change the Platform Target to 'Any CPU'. I have tried this and unfortunately this does not solve my issue.
The next block of code is what keeps throwing the exception. The program that runs this code is used to open up applications on remote computers and keeps a list of all the processes that the program itself opened so that I don't have to loop through all the processes.
Process processToRemove = null;
lock (_runningProcesses)
{
foreach (Process p in _runningProcesses)
{
foreach (ProcessModule module in p.Modules)
{
string[] strs = text.Split('\\');
if (module.ModuleName.Equals(strs[strs.Length - 1]))
{
processToRemove = p;
break;
}
}
if (processToRemove != null)
{
break;
}
}
if (processToRemove != null)
{
processToRemove.Kill();
_runningProcesses.Remove(processToRemove);
}
}
These processes can and most likely will be 32-bit and 64-bit, mixed together.
Is there anything I am doing that I shouldn't be doing, or is there just a better way to do all of this?
As detailed in the comments of the MSDN page for Process.Modules and this thread there is a known issue in Process.Modules when enumerating 32 bit processes from a 64 bit process and visa-versa:
Internally .NET's Process.Modules is using function EnumProcessModules
from PSAPI.dll. This function has a known issue that it cannot work
across 32/64 bit process boundary. Therefore enumerating another
64-bit process from 32-bit process or vice versa doesn't work
correctly.
The solution seems to be to use the EnumProcessModulesEx function, (which must be called via P/Invoke), however this function is only available on later versions of Windows.
We fixed this issue by adding
a new function called EnumProcessModulesEx to PSAPI.dll
(http://msdn2.microsoft.com/en-us/library/ms682633.aspx), but we
currently cannot use it in this case:
it only works on Windows Vista or Windows Server 2008
currently .NET 2.0 Framework don't have a service pack or hotfix to make Process.Modules use this new API
There are only some issues regarding the handling of the processes and the locking that I would change:
object lockObject = new object();
List<Process> processesToRemove = new List<Process>();
foreach (Process p in _runningProcesses)
{
foreach (ProcessModule module in p.Modules)
{
string[] strs = text.Split('\\');
if (module.ModuleName.Equals(strs[strs.Length - 1]))
{
processesToRemove.Add(p);
break;
}
}
}
lock (lockObject)
{
foreach (Process p in processesToRemove)
{
p.Kill();
_runningProcesses.Remove(p);
}
}
I'm not answering for the bounty, just wanted to give some ideas. This code isn't tested because I don't exactly know what you are trying to do there.
Just consider not to lock the process-list and to keep the lock as short as possible.
I agree with #sprinter252 that _runningProcesses should not be used as your sync object here.
//Somewhere that is accessible to both the thread getting the process list and the thread the
//code below will be running, declare your sync, lock while adjusting _runningProcesses
public static readonly object Sync = new object();
IList<Process> runningProcesses;
lock(Sync)
{
runningProcesses = _runningProcesses.ToList();
}
Process processToRemove = null;
foreach (Process p in _runningProcesses)
{
foreach (ProcessModule module in p.Modules)
{
string[] strs = text.Split('\\');
if (module.ModuleName.Equals(strs[strs.Length - 1]))
{
processToRemove = p;
break;
}
}
if (processToRemove != null)
{
break;
}
}
if (processToRemove != null)
{
//If we've got a process that needs killing, re-lock on Sync so that we may
//safely modify the shared collection
lock(Sync)
{
processToRemove.Kill();
_runningProcesses.Remove(processToRemove);
}
}
If this code is wrapped in a loop to continue to check _runningProcesses for the process you wish to kill, consider changing processToRemove to processesToRemove and change it's type to a collection, iterate over that list in the bottom block after a check for a non-zero count and lock outside of that loop to decrease the overhead of obtaining and releasing locks per process to kill.
Does somebody know how to unload a dll or any other type of module loaded by an external process?
I tried to do GetModuleHandle and then FreeLibrary with no result...
Thank you for all your replies
Thank you for all your replies. I found an interesting msdn article here :
http://blogs.msdn.com/b/jmstall/archive/2006/09/28/managed-create-remote-thread.aspx
The problem is that when i try to do a OpenProcess the external process crashes.
What are the minimum process access rights to unload a module from it ?
Here is what i am trying to do in c# :
[code]
protected const int PROCESS_ALL_ACCESS = (STANDARD_RIGHTS_REQUIRED | SYNCHRONIZE | 0xFFF);
protected const int STANDARD_RIGHTS_REQUIRED = 0xF0000;
protected const int SYNCHRONIZE = 0x100000;
public static bool UnloadRemoteModule(FileEntry le)
{
try
{
Process process = System.Diagnostics.Process.GetProcessById(le.ProcessID);
if (process == null) return false;
StringBuilder sb = new StringBuilder(le.File);
UnloadModuleThreadProc umproc = new UnloadModuleThreadProc(UnloadModule);
IntPtr fpProc = Marshal.GetFunctionPointerForDelegate(umproc);
SafeProcessHandle processHandle = null;
IntPtr currentProcess = NativeMethods.GetCurrentProcess();
int processId = le.ProcessID;
bool remote = (processId != NativeMethods.GetProcessId(currentProcess));
try
{
if (remote)
{
MessageBox.Show("OPENING PROCESS !");
processHandle = NativeMethods.OpenProcess(PROCESS_ALL_ACCESS, true, processId);
System.Threading.Thread.Sleep(200);
uint dwThreadId;
if (processHandle.DangerousGetHandle() == IntPtr.Zero)
{
MessageBox.Show("COULD NOT OPEN HANDLE !");
}
else
{
// Create a thread in the first process.
IntPtr hThread = CreateRemoteThread(
processHandle.DangerousGetHandle(),
IntPtr.Zero,
0,
fpProc, IntPtr.Zero,
0,
out dwThreadId);
System.Threading.Thread.Sleep(200);
WaitForThreadToExit(hThread);
}
}
return true;
}
finally
{
if (remote)
{
if (processHandle != null)
{
processHandle.Close();
}
}
}
return false;
}
catch (Exception ex)
{
//Module.ShowError(ex);
return false;
}
}
public delegate int UnloadModuleThreadProc(IntPtr sb_module_name);
static int UnloadModule(IntPtr sb_module_name2)
{
using (StreamWriter sw = new StreamWriter(#"c:\a\logerr.txt"))
{
sw.AutoFlush = true;
sw.WriteLine("In Unload Module");
StringBuilder sb_module_name =new StringBuilder(#"C:\Windows\System32\MyDll.dll");
IntPtr mh = DetectOpenFiles.GetModuleHandle(sb_module_name.ToString());
sw.WriteLine("LAST ERROR="+Marshal.GetLastWin32Error().ToString());
sw.WriteLine("POINTER="+mh.ToInt32());
if (mh != IntPtr.Zero)
{
return (FreeLibrary(mh) ? 1 : 0);
}
sw.WriteLine("LAST ERROR 2 =" + Marshal.GetLastWin32Error().ToString());
sw.WriteLine("EXIT " + mh.ToInt32());
}
return 0;
}[/code]
You can do it, but honestly I must ask why? You're most likely going to screw things up beyond what you realize. Seriously, there's nothing that can go right if you do this. Don't read the rest of this post, close your browser, do some meditation, and figure out what you're doing wrong that made you ask this question.
HERE BE DRAGONS
That said, it can be done, and rather easily too.
All you have to do is use CreateRemoteThread, pass a handle to the process you want to force unload in, and a function pointer to a function that calls GetModuleHandle and FreeLibrary. Easy as pie.
Sample code (untested, written in vi, and not to be used no matter what):
DWORD WINAPI UnloadNamedModule(void *)
{
//If you value your life, don't use this code
LPCTSTR moduleName = _T("MYMODULE.DLL");
HMODULE module = GetModuleHandle(moduleName);
if (module != NULL)
{
UnloadModule(hModule);
//All hell breaks loose. Not even this comment will be reached.
//On your own head be it. Don't say I didn't warn you.
}
}
//Warning: this function should never be run!
void UnloadRemoteModule(HANDLE hProcess)
{
CreateRemoteThread(hProcess, NULL, 0, UnloadNamedModule, NULL, 0);
}
You cannot force an external process to unload it's modules. You would need to run that code from inside the external process. The best you can hope for is to kill the process that owns the external DLL. It would be extremely dangerous if you could unload a dll from an external process, the code could be running at the time that you pull it out of RAM.
If you are looking to replace the DLL, the best you can do is to rename the DLL and save the new one. That way, the DLL will get use the next time the external process loads it.
Correction to italics above: You can do it but you are asking for big trouble if you do. I still think the best approach is to do what I listed above, rename the DLL and put the new one in it's place for the next time the external process starts. It's a far safer approach if you would like to replace a DLL.
I run through millions of records and sometimes I have to debug using Console.WriteLine to see what is going on.
However, Console.WriteLine is very slow, considerably slower than writing to a file.
BUT it is very convenient - does anyone know of a way to speed it up?
If it is just for debugging purposes you should use Debug.WriteLine instead. This will most likely be a bit faster than using Console.WriteLine.
Example
Debug.WriteLine("There was an error processing the data.");
You can use the OutputDebugString API function to send a string to the debugger. It doesn't wait for anything to redraw and this is probably the fastest thing you can get without digging into the low-level stuff too much.
The text you give to this function will go into Visual Studio Output window.
[DllImport("kernel32.dll")]
static extern void OutputDebugString(string lpOutputString);
Then you just call OutputDebugString("Hello world!");
Do something like this:
public static class QueuedConsole
{
private static StringBuilder _sb = new StringBuilder();
private static int _lineCount;
public void WriteLine(string message)
{
_sb.AppendLine(message);
++_lineCount;
if (_lineCount >= 10)
WriteAll();
}
public void WriteAll()
{
Console.WriteLine(_sb.ToString());
_lineCount = 0;
_sb.Clear();
}
}
QueuedConsole.WriteLine("This message will not be written directly, but with nine other entries to increase performance.");
//after your operations, end with write all to get the last lines.
QueuedConsole.WriteAll();
Here is another example: Does Console.WriteLine block?
I recently did a benchmark battery for this on .NET 4.8. The tests included many of the proposals mentioned on this page, including Async and blocking variants of both BCL and custom code, and then most of those both with and without dedicated threading, and finally scaled across power-of-2 buffer sizes.
The fastest method, now used in my own projects, buffers 64K of wide (Unicode) characters at a time from .NET directly to the Win32 function WriteConsoleW without copying or even hard-pinning. Remainders larger than 64K, after filling and flushing one buffer, are also sent directly, and in-situ as well. The approach deliberately bypasses the Stream/TextWriter paradigm so it can (obviously enough) provide .NET text that is already Unicode to a (native) Unicode API without all the superfluous memory copying/shuffling and byte[] array allocations required for first "decoding" to a byte stream.
If there is interest (perhaps because the buffering logic is slightly intricate), I can provide the source for the above; it's only about 80 lines. However, my tests determined that there's a simpler way to get nearly the same performance, and since it doesn't require any Win32 calls, I'll show this latter technique instead.
The following is way faster than Console.Write:
public static class FastConsole
{
static readonly BufferedStream str;
static FastConsole()
{
Console.OutputEncoding = Encoding.Unicode; // crucial
// avoid special "ShadowBuffer" for hard-coded size 0x14000 in 'BufferedStream'
str = new BufferedStream(Console.OpenStandardOutput(), 0x15000);
}
public static void WriteLine(String s) => Write(s + "\r\n");
public static void Write(String s)
{
// avoid endless 'GetByteCount' dithering in 'Encoding.Unicode.GetBytes(s)'
var rgb = new byte[s.Length << 1];
Encoding.Unicode.GetBytes(s, 0, s.Length, rgb, 0);
lock (str) // (optional, can omit if appropriate)
str.Write(rgb, 0, rgb.Length);
}
public static void Flush() { lock (str) str.Flush(); }
};
Note that this is a buffered writer, so you must call Flush() when you have no more text to write.
I should also mention that, as shown, technically this code assumes 16-bit Unicode (UCS-2, as opposed to UTF-16) and thus won't properly handle 4-byte escape surrogates for characters beyond the Basic Multilingual Plane. The point hardly seems important given the more extreme limitations on console text display in general, but could perhaps still matter for piping/redirection.
Usage:
FastConsole.WriteLine("hello world.");
// etc...
FastConsole.Flush();
On my machine, this gets about 77,000 lines/second (mixed-length) versus only 5,200 lines/sec under identical conditions for normal Console.WriteLine. That's a factor of almost 15x speedup.
These are controlled comparison results only; note that absolute measurements of console output performance are highly variable, depending on the console window settings and runtime conditions, including size, layout, fonts, DWM clipping, etc.
Why Console is slow:
Console output is actually an IO stream that's managed by your operating system. Most IO classes (like FileStream) have async methods but the Console class was never updated so it always blocks the thread when writing.
Console.WriteLine is backed by SyncTextWriter which uses a global lock to prevent multiple threads from writing partial lines. This is a major bottleneck that forces all threads to wait for each other to finish the write.
If the console window is visible on screen then there can be significant slowdown because the window needs to be redrawn before the console output is considered flushed.
Solutions:
Wrap the Console stream with a StreamWriter and then use async methods:
var sw = new StreamWriter(Console.OpenStandardOutput());
await sw.WriteLineAsync("...");
You can also set a larger buffer if you need to use sync methods. The call will occasionally block when the buffer gets full and is flushed to the stream.
// set a buffer size
var sw = new StreamWriter(Console.OpenStandardOutput(), Encoding.UTF8, 8192);
// this write call will block when buffer is full
sw.Write("...")
If you want the fastest writes though, you'll need to make your own buffer class that writes to memory and flushes to the console asynchronously in the background using a single thread without locking. The new Channel<T> class in .NET Core 2.1 makes this simple and fast. Plenty of other questions showing that code but comment if you need tips.
A little old thread and maybe not exactly what the OP is looking for, but I ran into the same question recently, when processing audio data in real time.
I compared Console.WriteLine to Debug.WriteLine with this code and used DebugView as a dos box alternative. It's only an executable (nothing to install) and can be customized in very neat ways (filters & colors!). It has no problems with tens of thousands of lines and manages the memory quite well (I could not find any kind of leak, even after days of logging).
After doing some testing in different environments (e.g.: virtual machine, IDE, background processes running, etc) I made the following observations:
Debug is almost always faster
For small bursts of lines (<1000), it's about 10 times faster
For larger chunks it seems to converge to about 3x
If the Debug output goes to the IDE, Console is faster :-)
If DebugView is not running, Debug gets even faster
For really large amounts of consecutive outputs (>10000), Debug gets slower and Console stays constant. I presume this is due to the memory, Debug has to allocate and Console does not.
Obviously, it makes a difference if DebugView is actually "in-view" or not, as the many gui updates have a significant impact on the overall performance of the system, while Console simply hangs, if visible or not. But it's hard to put numbers on that one...
I did not try multiple threads writing to the Console, as I think this should generally avoided. I never had (performance) problems when writing to Debug from multiple threads.
If you compile with Release settings, usually all Debug statements are omitted and Trace should produce the same behaviour as Debug.
I used VS2017 & .Net 4.6.1
Sorry for so much code, but I had to tweak it quite a lot to actually measure what I wanted to. If you can spot any problems with the code (biases, etc.), please comment. I would love to get more precise data for real life systems.
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading;
namespace Console_vs_Debug {
class Program {
class Trial {
public string name;
public Action console;
public Action debug;
public List < float > consoleMeasuredTimes = new List < float > ();
public List < float > debugMeasuredTimes = new List < float > ();
}
static Stopwatch sw = new Stopwatch();
private static int repeatLoop = 1000;
private static int iterations = 2;
private static int dummy = 0;
static void Main(string[] args) {
if (args.Length == 2) {
repeatLoop = int.Parse(args[0]);
iterations = int.Parse(args[1]);
}
// do some dummy work
for (int i = 0; i < 100; i++) {
Console.WriteLine("-");
Debug.WriteLine("-");
}
for (int i = 0; i < iterations; i++) {
foreach(Trial trial in trials) {
Thread.Sleep(50);
sw.Restart();
for (int r = 0; r < repeatLoop; r++)
trial.console();
sw.Stop();
trial.consoleMeasuredTimes.Add(sw.ElapsedMilliseconds);
Thread.Sleep(1);
sw.Restart();
for (int r = 0; r < repeatLoop; r++)
trial.debug();
sw.Stop();
trial.debugMeasuredTimes.Add(sw.ElapsedMilliseconds);
}
}
Console.WriteLine("---\r\n");
foreach(Trial trial in trials) {
var consoleAverage = trial.consoleMeasuredTimes.Average();
var debugAverage = trial.debugMeasuredTimes.Average();
Console.WriteLine(trial.name);
Console.WriteLine($ " console: {consoleAverage,11:F4}");
Console.WriteLine($ " debug: {debugAverage,11:F4}");
Console.WriteLine($ "{consoleAverage / debugAverage,32:F2} (console/debug)");
Console.WriteLine();
}
Console.WriteLine("all measurements are in milliseconds");
Console.WriteLine("anykey");
Console.ReadKey();
}
private static List < Trial > trials = new List < Trial > {
new Trial {
name = "constant",
console = delegate {
Console.WriteLine("A static and constant string");
},
debug = delegate {
Debug.WriteLine("A static and constant string");
}
},
new Trial {
name = "dynamic",
console = delegate {
Console.WriteLine("A dynamically built string (number " + dummy++ + ")");
},
debug = delegate {
Debug.WriteLine("A dynamically built string (number " + dummy++ + ")");
}
},
new Trial {
name = "interpolated",
console = delegate {
Console.WriteLine($ "An interpolated string (number {dummy++,6})");
},
debug = delegate {
Debug.WriteLine($ "An interpolated string (number {dummy++,6})");
}
}
};
}
}
Just a little trick I use sometimes: If you remove focus from the Console window by opening another window over it, and leave it until it completes, it won't redraw the window until you refocus, speeding it up significantly. Just make sure you have the buffer set up high enough that you can scroll back through all of the output.
Try using the System.Diagnostics Debug class? You can accomplish the same things as using Console.WriteLine.
You can view the available class methods here.