I have a batch application that processes a number of inputs to generate some reports. It reads an input text file, generates a list of report definitions, and then calls external code to run each report.
However, the external report generation code contains some sort of memory leak. When running a single report it has no issue, but with multiple the machine will quickly run out of memory. I've tried clearing the report instances and disposing what objects I can, but the only way I've found to clear the memory is to close the process. If I run each report as a separate process, this solves the issue.
So I require my application to create a new process, pass the report definition object and server information, and get that process to generate the report, and then close. Then repeat for the next definition. I have been looking at .Net Remoting using IPCChannel, however I can't figure out how to simply create the process, pass the information (the report definition and server information) and let it run.
Another option could be using memory mapped files. Or potentially using a console application, but that woulds require serializing my object and passing a large amount of text and many arguments, which isn't elegant.
So what option would be best for creating a process, passing a 'report definition' object and server information, and then allowing the process to close before running again?
I suggest you using Named Pipes or Anonymous Pipes. These are specifically for inter process communication and they are very fast. See one example using Anonymous Pipes here: https://msdn.microsoft.com/en-us/library/bb546102(v=vs.110).aspx
Related
I have a WPF client application which can have maximum of one instance at any point of time, if the user runs the second instance it should pass the input arguments to first instance of application and quit.
i.e i run the first instance with argument "hello", and again starting new instance of same appln with new argument "welcome", noe the second instance should pass the data "welcome" to first instance and exit.
Other that WCF and .Net remoting, is there any easiest and clean way to do it?
Currently I'm able to make the first instance as active, using WindowsEventHandle, but could not able to pass the data. This is in same machine within same login
Have a look at Memory-Mapped Files Since .NET 4 they have been available and can even share memory with unmanaged applications.
Non-persisted memory-mapped files Non-persisted files are
memory-mapped files that are not associated with a file on a disk.
When the last process has finished working with the file, the data is
lost and the file is reclaimed by garbage collection. These files are
suitable for creating shared memory for inter-process communications
(IPC).
Very basic, but you could simply write to a file and have the other application instance monitor the file for new commands with unique IDs for messages based on the current timestamp down to millisecond.
Yes of course. We tried multiple approaches, but the one using mutex`es seems to work the best.
here a some samples :
How can I check for a running process per user session?
a good article about mutex`es http://odetocode.com/blogs/scott/archive/2004/08/20/the-misunderstood-mutex.aspx
a simple implementation
https://stackoverflow.com/a/19326/444149
If You search deeper, I'm sure there's a good sample in SO which shows how mutex events can be used to trigger something in another app instance
I have two separate programs, one is a console application, and the other one is a windows application.
My windows application:
Has a graphic interface, buttons, and others functions.
One of the buttons, named "research": when I click on it, I launch the console application with this line of code:
string strResult = ProcessHelper.LaunchProcessWaitForPipedResult("MyExecFile.exe", strArguments, 10 * 60 * 1000, true); // 10 mins max
My console Application:
do a query on all existing files in a directory.
My problem:
I want to create a progress-bar on the windows application to show the progress of the console application. The problem is I don't know how to pass this information between the two processes. The only restriction is to not use a database or file.
Given two processes in the same user session, and wanting to avoid any communication outside that session I would look at three options:
1. Using named pipes.
The parent process creates a named pipe using a random name (and confirms that name is not in use by opening it). It passes that name to the child process. A simple protocol is used that allows the child to send updates.
There are a number of challenges to overcome:
Getting the logic to ensure the name is unique right (named pipe names are global).
Ensuring no other process can connect (the default named pipe ACL limits connections to the session: this might be enough).
Handling the case where a different parent process does not support progress updates.
Handling the child or parent crashing.
Avoiding getting too clever with the communication protocol, but allowing room for growth (what happens when more than a simple progress bar is wanted?)
2. Using Shared Memory
In this case names of objects are, by default, local to the session. By default this is more secure.
The parent process creates a sufficiently large amount of shared memory (for a simple progress update: not much), a mutex and an event.
The parent process then, concurrently with the GUI waits for the event to be signalled, when it is it enters the mutex and reads the content of shared memory. It then unsets the event and leaves the mutex.
Meanwhile to send an update the child enters the mutex, updates and memory and sets the event before leaving the mutex.
The challenges here include:
Defining the layout of the shared memory. Without a shared assembly this is likely to be error prone.
Avoiding others using the shared memory and synchronisation objects. .NET makes things harder here: in Win32 I would make the handles inheritable thus not needing to name the objects (except for debugging) and pass to the child directly.
Getting the sequencing of shared memory, mutex and event correct is critical. Memory corruption and more subtle bugs await any errors.
It is harder to do variable length data with shared memory, not an issue for a simple progress count but customers always want more.
Summary
I would probably look at named pipes in the first place (or perhaps custom WMI types if I wanted greater flexibility). BUT I would do that only after trying everything to avoid needing multiple processes in the first place. A shared library plus console wrapper for others, while I use the library directly would be a far easier option.
I have many instances of a process I've written on a server. I'd like to associate some information with each process. In this specific case I'd like to store the "CurrentState" of the process - "RUNNING|DRAINING|STOPPING", but it would be useful for me to store a "Friendly Name" and so on.
I want to query this information from another "mother" process - this mother process will query the processes running and collate the data.
I've thought of a couple of different ways I could achieve this. For example I might open up a NetPipe to each process of interest and ask for the data, or have each process broadcast it's state regularly.
I was wondering: is there a way to store key value pair information against a process built into Windows itself? Is there an accepted pattern for doing this?
I control the source for the child processes and the mother process. They are written in C#, P/Invoking is fine. The operating system is Windows 2012 R2.
You can host WCF services that use named pipes:
http://msdn.microsoft.com/en-us/library/ms733769(v=vs.110).aspx
Based on some of your comments, it looks like you could also consider the System.AddIn (aka Managed AddIn Framework (MAF)) functionality to create, host, and communicate with Add-ins. MAF supports loading addins in your app domain, a separate app domain, or in a completely separate process. The downside with MAF is that it requires 5 DLLs to get started, but in doing that gives you a lot of flexibility with API compatibility as you version and change your pipeline.
If you're controlling the data from a Mother process, you can also use AppDomains to load your other processes and communicate via Marhsaled data such as a Status class, or use the AppDomains to Set and Get data.
Be aware that any Status data you transfer needs to either be a class which derives from the Marshaling class or be marked as Serializable. The reason for this is because AppDomains are treated in the OS the same as different processes, so they can't access each others memory an actually have to serialize data as if it were being passed through IPC.
Take a look at the .Net Process Class:
http://msdn.microsoft.com/en-us/library/system.diagnostics.process(v=vs.110).aspx
You can use it to get all running processes, start a process, get the processes unique Id, and be alerted when the process exits. This should give you everything you need to track processes.
Children can call Process.GetCurrentProcess to get their own process id, then make a call to the "mother" process to associate arbitrary data about itself.
If I deploy a C# console app, which does the following:
reads message (ActiveMQ)
processes message contents
writes result to database (SQL Server)
Would there be any issues with running this multiple times e.g. what if I created a batch file and ran 100 instances? Would there be any conflict given that each instance would be using the same shared DLLs e.g. Apache.NMS.ActiveMQ.
The other option would be to deploy the app multiple times, but I'd rather not have to manage duplicated folders. I'm also avoiding threading at the moment but that will be an option for further development in future.
Just want to clarify what happens with those DLLs, and check that there wouldn't be a threading type conflict, e.g. one instance writing the results of another instance's processing to the database...
No, there will be no problem with loading the same DLL files into multiple processes as you describe. You would only run into problems running multiple instances of the same application if the process needed exclusive access to a shared resource, like a file. With regard to writing to a database, as long as you design your application so that multiple clients can write data without overwriting data or causing some sort of inconsistency with the domain integrity of the data then again, no problem.
However, I would strongly suggest you look at making you application multi-threaded if it is concurrency you need, or Application Domains if it is isolation you need. Running multiple processes is much more expensive in terms of resources than either of these two options.
I thought this could've been a common question, but it has been very difficult to find an answer. I've tried searching here and other forums with no luck.
I'm writing a C# (.net version 4) program to monitor a process. It already raises an event when the process starts and when it stops, but I also need to check where is this process reading from and writing to; specially writing to since I know this process writes a large amount of data every time it runs. We process batches of data, and the path where the process writes to contains the Batch ID, which is an important piece of information to log the results of the process.
I've looked into the System.Diagnostics.Process.BeginOutputReadLine method, but since the documentation says that StandardOutput must be redirected, I'm not sure if this can be done on a process that is currently running, or if it affects the write operation originally intended by the process.
It is a console application in C#. If anyone have any idea on how to do this, it would be much appreciated.
Thanks in advance!
Output redirection would only help you solve the problem of intercepting the process' standard output stream. This would have no effect on read/write operations to other files or streams that the program would use.
The easiest way to do this would be to avoid reverse engineering this information and exert some control over where the process writes its data (e.g. pass a command line parameter to it to specify the output path and you can monitor that output path yourself).
If that is impossible for some reason, you can look into these approaches, all of which are quite advanced and have various drawbacks:
Use Detours to launch the process and redirect calls to CreateFile to a function that you define (e.g. you could call into some other function to track the file name that it used and then call the real CreateFile). Note that a license to use Detours costs money and it requires you to build an an unmanaged DLL to define your replacement function.
Read the data from the Microsoft-Windows-Kernel-File event tracing provider. This provider tracks all file operations for everything on the system. Using this data requires advanced knowledge of ETW and a lot of P/Invoke calls if you are trying to consume it from C#.
Enumerate the open handles of the process once it is started. A previous stackoverflow.com question has several possible solutions. Note that this is not foolproof as it only gives you a snapshot of the activity at a point in time (e.g. the process may open and close handles too quickly for you to observe it between calls to enumerate them) and most of those answers require calling into undocumented functions.
I came across this implementation recently: DetectOpenFiles but i have not used and/or test it. Feel free to try it. It seems to deliver open file handle information for a given process id. Looking forward to read your experience with it! ;-)