I've written a C# program that talks to a COM server to conduct simulations. It works without any trouble, but the simulations being carried out by the COM server are fairly processor intensive, and only run single core.
As such, I've used Parallel.For to distribute the workload amongst multiple threads. It appears, however, that all the simulation results generated by the COM server are shared amongst all instances of its classes, so when I'm running the parallel task with only 1 thread, everything works as expected, but when I'm running the task with multiple threads, the results are completely garbled (as multiple threads are effectively causing the simulation engine to replace its results with new ones as they are being read).
I was wondering if there was a way to connect to the COM server multiple times in order to stop the results-sharing of class instances?
Edit
My process for connecting to the COM server was to:
Add a reference using Project->Add References->COM (VS2010)
Use the following code to instantiate the simulator object:
dss = new OpenDSSengine.DSS();
dss.Start(0);
The above code is called in the local thread data initialiser (localInit) parameter of Parallel.For, and thus a new dss object is created for each thread, but the results obtained seem to be common across all threads.
The COM server is a dll.
As you specify that your COM server is actually an in-proc server (a .dll instead of .exe), it means that every time you execute new DSS() you actually create a new instance (unless it is created with singleton class factory, which is rare but possible).
The problem, according to your description, seems to be with the fact that the DSS implementation uses some static/global state which results in garbled data when you parallelize the execution.
In that case you can run each instance of the server in a separate process by using DllSurrogate. If the default surrogate (dllhost.exe) doesn't suffice, it is possible to write the custom one. Be aware that moving the server into another process will introduce marshaling overhead for each method call done against the server.
Please also not that if you are using an STA COM server, your parallelization will have no effect, as all the calls to the server are serialized by COM infrastructure.
All that being said, before going there make sure that the problem is not on the caller side, i.e. with your parallelization and not the server itself.
First try creating multiple instances of the COM object (just call new OpenDSSengine.DSS() multiple times, storing the results in separate variables, or in an array). If the COM server was implemented well, those multiple instances will co-exist in your process without interfering with each other, and your multi-threaded client code can use them simultaneously.
If you still find that those instances are interfering with each other, that means the COM server is using some state that is global to the process. The only way to get around that would be to invoke the multiple COM objects via multiple surrogate processes, as others have suggested.
Related
Let's say, I am executing an exe written in c#(just my choice of language). It has the following piece of code:
var comObj=new ComClass();
comObj.DoSomething();
Now, I would like to know in which process is the DoSomething method executed. Is it the same process where the current exe is running or a different process responds to the DoSomething call?
This is entirely transparent in COM, you cannot find out from your program either. It is determined by configuration information that is stored in the registry. The core reason why COM servers need to be registered. The different scenarios are:
On the same thread that creates the object. Used when the server is registered as an in-process server and the thread's apartment is compatible with the threading model of the COM object. The most common case, particularly so when you create objects on the UI thread of a program.
On another thread, if necessary created by COM, to give the object a thread-safe home. This commonly happens when your new statement runs on a thread that's in the MTA, the multi-threaded apartment. Commonly from a worker thread. The object you create is a proxy, its primary job is to serialize the arguments you pass to a method and deserialize them in the stub which runs on the other thread. It ensures that all calls on the object are thread-safe. Otherwise the same kind of mechanism as used in .NET Remoting. The underlying layer that takes care of the marshaling is LRPC, an obscure Windows component that was optimized to make inter-thread and inter-process calls as fast as possible.
Inside a surrogate process for an in-process component. Not very common but surrogates can be very handy to bridge a process bitness problem for example. Allowing you to use a 32-bit server in a 64-bit process. Requires both 32-bit and 64-bit proxy/stubs.
Inside another process that was registered as an out-of-process server. The canonical example are Microsoft Office programs like Word and Excel, very common in .NET programming. This is where COM starts to get brittle, unexpected program aborts tend to cause a mess when the server keeps running. A very common question at SO.
Inside another process on another machine. Called DCOM or Distributed COM. An extra configuration step is necessary to ensure the target machine and proper account privileges can be selected. Pretty notorious for giving humans a splitting headache, it doesn't get used much anymore these days. DCOM's biggest claim to fame was enabling Java to eat Microsoft's lunch in the middle-ware wars of the late 90s.
If you have no idea which of these scenarios applies in your case then a utility like SysInternals' Process Monitor tends to provide insight. You'll see your program reading the registry, telling you where to look, and load a DLL or start an EXE.
From COM Clients and Servers
There are two main types of servers, in-process and out-of-process. In-process servers are implemented in a dynamic linked library (DLL), and out-of-process servers are implemented in an executable file (EXE). Out-of-process servers can reside either on the local computer or on a remote computer.
I do think that the names are quite explicit :-)
Note that even for out-of-process COM servers, there will be some code in-process that will do the marshaling between the COM client and the COM server
We have very old legacy vb6 application, that has one global object that serves as Application Core, that stores different application settings, invokes database operations and so on. Multiple modules with different progid use this global object and have no problem with it due to single-thread apartment.
Not long ago new WPF application was created, that provide us transition from vb6, however it is still limited by vb6 legacy due to some architectural mistakes. It is able to connect to only one database per app instance. It hold static instance of vb6 global object in wrapper class, that serves as bridge to reach legacy functionality.
Now, we are developing new application that should not be limited by old legacy code, in particular it new application should be able to connect to several databases at once, but there is a catch: vb6 code limited to single database, so there should be several instances of vb6 global objects, one per each database.
So the question is: is it possible, and if it is, how is it possible to use several separated intstances of global vb6 objects in same C# application?
I presume that each instance of such object should live in it's own STA-Thread, but i don't know how to create such threads, that are kept alive for entire application runtime and that have assotiated wrappers, containing instances of global vb6 objects and supporting invocation of some functions from GUI thread (and how to organize such cross-thread communication, there is no thread.invoke(...)). I thinked about using wpf dispatcherization model ( wrapper class is DispatcherObject, each instance has it's own Dispatcher with it's own STA-Thread), but i cannot see how to implement such thing. Also I think it could be imlemented by loading each instance of wrapper class (static) in different AppDomains, but i don't know if it resolves STA problem for COM.
It is possible, using what is referred to as unmanaged code. Check out this:
http://www.codeproject.com/Articles/154144/Using-Unmanaged-VB6-Code-in-NET
You might get away with running one instance of the global object per AppDomain.
To be safe, you should run them each in its own process, since that's the assumption they were created with. I guarantee you that horrible things can happen when you violate the assumptions of ancient code - especially when you introduce them to things which simply did not exist when they were written.
If I deploy a C# console app, which does the following:
reads message (ActiveMQ)
processes message contents
writes result to database (SQL Server)
Would there be any issues with running this multiple times e.g. what if I created a batch file and ran 100 instances? Would there be any conflict given that each instance would be using the same shared DLLs e.g. Apache.NMS.ActiveMQ.
The other option would be to deploy the app multiple times, but I'd rather not have to manage duplicated folders. I'm also avoiding threading at the moment but that will be an option for further development in future.
Just want to clarify what happens with those DLLs, and check that there wouldn't be a threading type conflict, e.g. one instance writing the results of another instance's processing to the database...
No, there will be no problem with loading the same DLL files into multiple processes as you describe. You would only run into problems running multiple instances of the same application if the process needed exclusive access to a shared resource, like a file. With regard to writing to a database, as long as you design your application so that multiple clients can write data without overwriting data or causing some sort of inconsistency with the domain integrity of the data then again, no problem.
However, I would strongly suggest you look at making you application multi-threaded if it is concurrency you need, or Application Domains if it is isolation you need. Running multiple processes is much more expensive in terms of resources than either of these two options.
UPDATE: The following error was actually due to a simple bug which I missed. The only real message here that tired and stupid is a bad combination.
For reasons to do with some specific features of an ODBC driver we're forced to use, I've been trying to write a small application which directly uses ODBC calls. Since C# 2.0 is what I know most, I've been doing this using P/Invoke calls into ODBC32.dll.
The code I've written initially has been multithreaded. But I've noticed that as soon as I jump threads I'm getting AccessViolationExceptions. For instance, when I generate IntPtr references to an Environment and Connection in one thread and then try to use these in another thread in the generation of a Statement (SQLAllocStmt), it all goes pop.
I'm sure I can work around this, but is there some obvious reason for this? Is the unmanaged memory allocated by the calls into ODBC32.dll somehow bound to a particular thread?
This depends on:
The odbc driver: What is it?
Your code: Are you freeing the memory without realizing it?
Consider:
Whether you really need to do this. Can't you control the behaviour of the driver using the connection string, or using driver-specific commands through a command object?
If you do have to do this, can you isolate it into a single STA thread and use marshalling, or a task queue, to simplify your job?
If you do have to use it from multiple threads, can't you make sure each thread has it's own Connection and Environment?
Our connectivity to EMS code was initially ill-designed and created one TopicConnection object per topic that we listened to. So, in effect, whenever we subscribed to a topic, we create a new connection, a new session and, lastly, a new listener.
We would like to switch to a single connection model. Although I am able to do this easily in our code by sharing one connection object and creating a new session object per topic, we are unsure whether this is going to cause any issues without code.
My understanding is that the Tibco EMS client library is thread safe with regards to sharing a connection. In effect, a connection is just a pipe and the sessions can resuse the this pipe in a thread safe manner.
Is this assumption correct or is there more to this?
The .NET EMS API is based on JMS. In JMS, the Connection and Session objects are specified to be thread-safe and can be reused within the program. You are quite correct in that the Connection object simply represent a network pipe to the EMS server. The EMS User's Guide states:
A connection is a fairly heavyweight object, so most clients will create a connection once and keep it open until the client exits. Your application can create multiple connections, if necessary.
And regarding sessions:
A Session is a single-threaded context for producing or consuming messages. You create Message Producers or Message Consumers using Session objects.
Essentially, unless you need very large volumes and are bumping into performance limitations, it's perfectly safe to use just one connection in your application. The session controls the transaction/acknowledgement semantics of any producers or consumers created within, but is again safe to reuse. I would probably use separate sessions for modules exist within the application with distinct life cycles (think separate deployment units within an application server).
Your EMS server installation will contain a samples directory with various code (something like C:\tibco\ems\5.0\samples\cs). The code in csTopicSubscriber.cs shows how to write a single-threaded topic consumer. There is no multi-threaded topic consumer example but csMsgConsumerPerf.cs demonstrates how to do it with queues.
Be sure to clean up any objects you create after you're done with them - e.g. close the topic consumer object, the session, and the connection when you're finished. Leaking handles without closing them can result in unpredictable behaviour when combined with prefetch and fault-tolerant reconnect settings.
I think yes as long as sharing is within the same application (exe, binary).
We have shared same connection object, and used it as a singleton in our code.
Agree with an earlier answer: the JMS Session must not be shared between threads, but the Connection can/should be. So one connection per application is ok (make sure you start/close it only once - best before/after the individual threads creation).
And then create and use one Session per thread. Remember that when you close() a Session, it will block until all receive callbacks have really returned. So do NOT call close() from within a callback's onMessage().