We Keep Coding

Creating a pool of locks in C#

I would like to use exactly one of a set of resources from a multi-threaded C# application. These resources are not thread-safe, so some lock or mutex must be for them. How should I do it?
I would like to get something like the following pseudo code:
lockAny([obj1, obj2, obj3]) {
achievedLock = getAchievedLock(); // returns e. g. obj2
myResource = getResourceForLock(achievedLock); // some function written by me, looks up the resource belonging to the particular lock
myResource.DoSomething();
}

The best way to do this is with some sort of pooling. A pool object can attempt to ensure that only 1 thing has a reference at a time (but .NET can't guarantee that like Rust).
.NET has recently added Microsoft.Extensions.ObjectPool<T>. You can configure how it creates the items. Some examples here.

How to find and close any open OleDbConnections in winforms?

I have a C# Winforms app that is large and complex. It makes OleDB connections to an Access database at various times for various reasons. In a certain function we need to MOVE (copy + delete) the mdb file, but it can't be done because it's locked. I've tried lots of different things to unlock/release the mdb file, and sometimes it works.
But in a certain 100% reproducible scenario, it cannot be unlocked. We have 2 global oledb connection variables we reuse everywhere, for efficiency, and to avoid having 1-off connections everywhere. And these 2 connection vars are useful for when we want to CLOSE the connections, so we can delete the mdb.
Here is my function (which normally works - just not in this 1 case) to forcibly close/release the 2 oledb connections from our winforms app:
public static void CloseOleDBConnections(bool forceReleaseAll = false) {
if ( DCGlobals.Connection1 != null )
DCGlobals.Connection1.Close();
if ( DCGlobals.Connection2 != null )
DCGlobals.Connection2.Close();
if ( forceReleaseAll ) {
DCGlobals.Connection1.Dispose();
DCGlobals.Connection2.Dispose();
OleDbConnection.ReleaseObjectPool();
GC.Collect(GC.MaxGeneration);
GC.WaitForPendingFinalizers();
}
}
I am passing true into the above function.
One other thought: Certainly my Winforms app knows about all open oledbconnections. Is there no way to tell c# to find and iterate all open connections? When I close/exit my application - poof - the open connection to the mdb is released and I can delete the file. So something in .net knows about the connection and knows how to release it -- so how can I tap into that same logic without exiting the application?
Post Script
(I am aware that Access is bad, non-scalable, etc. - it's a legacy requirement and we're stuck with it for now).
I have seen numerous stack discussions (and on other forums) on this topic. I have tried numerous recommendations to no avail.

Disposed IDataReaders?
Do you disable all IDataReader objects properly? They may prevent the connection closing properly.
Tracking Solution
In any case, you need to at least better track all your connections. It sounds like a very large project. You need to be absolutely sure that all connections are being disposed.
1. New TrackedOleDbConnection object
Create a TrackedOleDbConnection object which inherits from OleDbConnection, but adds a static ConcurrentList named StillOpen. When the TrackedOleDbConnection is constructed, add to the list, when it's disposed (override that function), remove it.
public class TrackedOleDbConnection: OleDbConnection
{
public TrackedOleDbConnection() : base()
{
}
public TrackedOleDbConnection(string ConnectionString) : base(ConnectionString)
{
}
//You don't need to create a constructor for every overload of the baseclass, only for overloads your project uses
ConcurrentList<TrackedOleDbConnection> ActiveConnections = new ConcurrentList<TrackedOleDbConnection>();
void AddActiveConnection()
{
ActiveConnections.Add(this);
}
override void Dispose()
{
ActiveConnections.RemoveIfExists(this); //Pseudo-function
GC.SuppressFinalise(this);
}
//Destructor, to ensure the ActiveConnection is always removed, if Dispose wasn't called
~TrackedOleDbConnection()
{
//TODO: You should log when this function runs, so you know you still have missing Dispose calls in your code, and then find and add them.
Dispose();
}
}
2. Don't directly reference OleDbConnection anymore
Then do a simple Find and Replace across your solution to use TrackedOleDbConnection.
Then finally, during your CloseOleDBConnections function, you can access TrackedOleDbConnection.StillOpen to see if you've got a problem of an untracked connection around somewhere.
Wherever you find such untracked problems, don't use the single central references, but instead using to ensure your connection is disposed properly.

Probably if the only thing you need is to copy the file probably there is no need to mess with connections. Please take a look at this:
https://www.raymond.cc/blog/copy-locked-file-in-use-with-hobocopy/

It's highly likely that ADOX is not releasing the connection to the database. Make sure that you:
explicitly call 'Close' the ADOX Connection objects
call 'Dispose' them
call System.Runtime.InteropServices.Marshal.FinalReleaseComObject(db.ActiveConnection);
call System.Runtime.InteropServices.Marshal.Marshal.FinalReleaseComObject(db);
set them to Nothing/null
Also when something calls close on a file handle the close request is put in a queue to be processed by the kernel. In other word even closing a simple file doesn't happen instantly. For this, you may have to put in a time-boxed loop to check that the .LDB file is removed...though that will ultimately require the user to wait. Seek any other alternative to this approach, though it has been necessary with other formats/connections IME in the past.

How to find out memory consumed by classes, objects, variables, etc [duplicate]

This question already has answers here:
How to get object size in memory? [duplicate]
(6 answers)
Closed 7 years ago.
I am trying to play with memory profiling (for the first time, so please forgive my ignorance), just to see how much memory is consumed by classes, objects, variables, methods etc. I wrote this sample c# console program called MemPlay.exe:
using System;
using System.Text;
namespace MemPlay
{
class Program
{
static void Main(string[] args)
{
SomeClass myObject = new SomeClass();
StringNineMethod();
}
private static void StringNineMethod()
{
string someString0 = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
string someString1 = string.Empty;
string someString2 = string.Empty;
for (int i = 0; i < 9999; i++) {
someString1 += "9";
someString2 += someString1;
}
}
}
class SomeClass
{
}
}
Once the program ran, I want to find out:
How much memory was consumed by
MemPlay.exe
Program class
SomeClass
Main method
myObject
StringNineMethod
someString0
someString1
someString2
and how much processor was used by:
MemPlay.exe
Main method
StringNineMethod
I tried using VisualStudio's 'Performance Diagnostics' tool, but all I can see is how much memory was used by the whole function (i.e. Main method, StringNineMethod, and the String.Concat method).
Is there any way/tool which can help me see all the details of how much memory each variable, object, class, method consumed? Thanks in advance.
EDIT: No my question is not duplicate of the question suggested, as that question is trying to get object sizes at runtime, I am asking how can I get this information after the program has ended. Just what Visual Studio's Performance Diagnostics tool does, it gives this information after the program has ended execution.

You can use System.Diagnostics namespace classes to get different kind of measurements and statistics. To get total memory allocated for the process use WorkingSet property (more details on MSDN):
Process currentProcess = System.Diagnostics.Process.GetCurrentProcess();
long processAllocatedMemory = currentProcess.WorkingSet64;
So that's process.
To get specific object allocation you probably can use GC to check initial memory, then allocate an object and finally to check memory again:
// Check initial memory
var memoryStart = System.GC.GetTotalMemory(true);
// Allocate an object.
var myClass = new SomeClass;
// Check memory after allocation
var memoryEnd = System.GC.GetTotalMemory(true);
To check memory consumption on specific process after specific operation you probably can use the same trick as with the GC only on the current process (like in the first example).
To check executables and programs use Visual Studio profiler. In VS2013 Community Edition go to ANALYZE -> Performance and Diagnostics menu (or hit Alt+F2). It allows you to analyze standard project, an exe, an ASP.NET website, and Windows Phone App:
There, you select Performance Wizard, click Start, and in the next step you have a choice of metrics you would like to run. One of which is memory consumption:

I use this one :
RedGate ANTS
I've also used this one in the past :
SciTech Memory Profiler
They both have free trials and are worth a look at. I've been impressed enough at various times to buy versions of both.
(I don't work for either company - just recommending what has worked for me - there are other tools out there too such as the JetBrains Memory Profiler, but I've not tried that personally and can't offer an opinion).

How to minimize concurrent database connections?

My hosting company blocked my website for using more than 15 concurrent database connections. But in my code I closed each and every connection that I opened. But still they are saying that there are too many concurrent connections. And suggested me the solution that I should change the source code of my website. So please tell me the solution about this? And my website is dynamic, so would making it static simple HTML old days type will make a difference or not?
Also note that I tried this when no solution I can think of, before every con.open(), I added con.Close(), So that any other connection opened will be closed.

The first thing to do is to check when you open connections - see if you can minimise that. For example, and you doing "n+1" on different connections?
If you have a single server, the technical solution here is a semaphore - for example, something like:
someSemaphore.TakeOne();
try {
using(var conn = GetConnection()) {
...
}
} finally {
someSemaphore.Release();
}
which will (assuming someSemaphore is shared, for example static) ensure that you can only get into that block "n" times at once. In your case, you would create the semaphore with 15 spaces:
static readonly Semaphore someSemaphore = new Semaphore(15,15);
However! Caution is recommended: in some cases you could get a deadlock: imagine 2 threads poorly written each need 9 connections - thread A takes 7 and thread B takes 8. They both need more - and neither will ever get them. Thus, using WaitOne with a timeout is important:
static void TakeConnection() {
if(!someSemaphore.TakeOne(3000)) {
throw new TimeoutException("Unable to reserve connection");
}
}
static void ReleaseConnection() {
someSemaphore.Release();
}
...
TakeConnection();
try {
using(var conn = GetConnection()) {
...
}
} finally {
ReleaseConnection();
}
It would also be possible to wrap that up in IDisposable to make usage more convenient.

Change Hosting Company.
Seriously.
Unless you run a pathetic Little home blog.
You can easily have more than 15 pages / requests being handled at the same time. I am always wary of "run away Connections" but I would not consider 15 Connections to even be something worth mentioning. This is like a car rental Company complaining you drive more than 15km - this simply is a REALLY low Limit.
On a busy Website you can have 50, 100, even 200 open Connections just because you ahve that many requests at the same time.

This is something not so obvious, but even if you care about opening and closing your connections properly, you have to look at something particular.
If you make the smallest change on the text you use to build a connection string, .net will create a whole new connection instead of using one already opened (even if the connection uses MARS), so just in case, look for your code if you are creating connection strings on the fly instead of using a single one from your web config.

I believe SQL Connections are pooled. When you close one, you actually just return it to connection pool.
You can use SqlConnection.ClearPool(connection) or SqlConnection.ClearAllPools to actually close the connection, but it will affect the performance of your site.
Also, you can disable pooling by using connection string parameter Pooling=false.
There are also Max Pool Size (default 100), you may want to set it to a lower number.
This all might work, but i would also suggest you to switch providers ....

If you only fetch data from database then it is not very difficult to create some sort of cache. But if there full CRUD then the better solution is to change hosting provider.

MySQL Threading Issues in my static DAL

I'm using the MySQL .Net libraries in an older C# application I'm rewriting. The Data Access Layer is rather obsolete but I'm trying to make the best of it. But now I ran into some really nasty threading issues.
I have a series of about 20 Select statements which are used to process a report. They take about 5 seconds to complete and I'm displaying a progress bar while the Select statements run. I'm launching the operations via a simple ThreadPool call:
[LATER EDIT: What happens is that I called the method below twice due to a bug in my UI - this doesn't devalue the question though, merely explains why my threads were racing against each other.]
ThreadPool.QueueUserWorkItem(new WaitCallback(UpdateChart));
Sometimes it works.
Sometimes it crashes with a "possible IO stream race condition".
Sometimes it crashes with "connection should be open and valid".
Sometimes it crashes with "object reference not set...".
All classes in my DAL are Static because I thought this is a good way of improving performance (not having to create new class instances for every little operation).
And all my DAL classes use the same "root" DAL class which builds Connections:
public static class MySQLConnectionBuilder
{
private static MySqlConnectionStringBuilder ConnectionStringBuilder = new MySqlConnectionStringBuilder();
//I'm initializing the ConnectionStringBuilder with my server password & address.
public static MySqlConnection GetConnection ()
{
return new MySqlConnection(ConnectionStringBuilder.ConnectionString);
}
}
All my DAL classes have functions which are similar to the crashing function. The crashing function looks like this:
public static STDS.UserPresence.user_presenceDataTable GetPresence (int aUserID, DateTime aStart, DateTime aEnd)
{
ta.Connection = MySQLConnectionBuilder.GetConnection();
ds = ta.GetPresenceForUserBetweenDates(aUserID, aStart, aEnd);
ta.Connection.Close();
return ds;
}
Ideas? Tips on improvement? Will the threading issue go away if I switch to a more object-oriented (instance-driven) DAL?

The line
ta.Connection.Close()
Closes the connection last assigned to ta.Connection - not always the connection created in the same thread. This may close a connection on which a query is currently running in another thread.
If you want to quickly determine if this is what's happening, mark the connection variable with a [ThreadStatic] attribute in the class ta points to:
[ThreadStatic]
private static MySqlConnection connection;
I wouldn't use that approach for your final solution though, as it may cause the GC not to collect them.
A simple solution (for that problem, I can't determine if your classes have other multithreading issues) is to add the connection as a parameter to each of your DAL methods, allowing you to remove the class global Connection:
public static STDS.UserPresence.user_presenceDataTable GetPresence (int aUserID, DateTime aStart, DateTime aEnd)
{
using (MySqlConnection connection = MySQLConnectionBuilder.GetConnection())
{
ds = ta.GetPresenceForUserBetweenDates(connection, aUserID, aStart, aEnd);
return ds;
}
}
Threading issues never simply go away - they require attention. If you are unsure about what's happening, forget about the slight performance boost (if a query takes 5 seconds, any possible performance gain of using static classes would be below 1% anyway).