I created a C# program which does some stuff. Let's call it Program B. I want to open Program B's executable in another program, Program A, I am creating also in C#. What I want to do is to get some information(actually a custom object) from Program B on a button click which also will close the program and add the information in a List in Program A. I want to use Memory Mapped Files for this, but somehow I didn't manage to do this until now. I have created a class:
public class MemoryMappedFileCommunicator{
public MemoryMappedFileCommunicator()
{
MemoryMappedFile = MemoryMappedFile.CreateOrOpen(MMF_FILE_NAME, MMF_MAX_SIZE);
// Create a stream which allows it to write data from offset 0 to 1024 (whole memory)
MemoryMappedViewStream = MemoryMappedFile.CreateViewStream(0, MMF_VIEW_SIZE);
_commandToSend = new Command();
//callback = new SendOrPostCallback(OnDataReceivedInternal);
//operation = AsyncOperationManager.CreateOperation(null);
}
}
I tried using SendOrPostCallback seen in a tutorial, but it didn't make sense to me. I also tried using an event handler, but my event did not fire.
What I actually want to do is to fire an event every time I finished writing in the file, so Program A could read the information and put it into a List.
Write function looks like this:
public void WriteData(MyCustomObject obj)
{
FileStream writer = new FileStream(MMF_FILE_NAME, FileMode.Append);
DataContractSerializer ser = new(typeof(MyCustomObject ));
ser.WriteObject(writer, obj);
writer.Close();
DataWrittenEvent();
}
My delegate is void and has no parameters
Read function looks like this:
public MyCustomObject ReadData()
{
FileStream reader = new FileStream(MMF_FILE_NAME, FileMode.Open);
XmlDictionaryReader xmlDictionaryReader = XmlDictionaryReader.CreateTextReader(reader, new XmlDictionaryReaderQuotas());
DataContractSerializer ser = new DataContractSerializer(typeof(MyCustomObject ));
MyCustomObject deserializedObj = (MyCustomObject )ser.ReadObject(xmlDictionaryReader, true);
xmlDictionaryReader.Close();
reader.Close();
return deserializedObj;
}
In every program I create an instance of MemoryMappedFileCommunicator, but only in Program A's constructor I attach the DataWrittenEvent event, but it is null all the time.
What am I doing wrong? How could I manage to do this?
I expected to make a callback from Program B to Program A like this:
Program B: get information -> serialize information -> write serialized information to MMF (this should rise an event to Program A)
Program A: read from MMF -> deserialize information -> add to list
So I figured it out finally. The way I work with the 2 processes is the following:
I have process A from which I will open process B. Process B will
calculate some stuff and on the button click "Add" it will add the
information into a MemoryMappedFile (created in Process A before
opening Process B). It is important to give it a name and set the
HandleInheritability to HandleInheritability.Inheritable. Also an EventWaitHandle is created in Process A in order to wait for Process B to do its job. I am sending objects between the two processes, so I
serialize the object and after that I write the information in the
MMF. The same at reading, I deserialize the object after reading it
from the MMF.
The solution looks like this:
Process A:
EventWaitHandle ewh = new EventWaitHandle(false, EventResetMode.AutoReset, "ObjectAddedEvent");
MyObject obj= new();
using (MemoryMappedFile mmf = MemoryMappedFile.CreateOrOpen("ObjectMMF", 1024, MemoryMappedFileAccess.ReadWrite, MemoryMappedFileOptions.None, HandleInheritability.Inheritable))
{
Process myProcess = Process.Start(psi);
ewh.WaitOne(); // Here we wait for Process B to do its job
// This will be executed AFTER Process B finishes its task (after ewh.Set())
using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
BinaryReader reader = new BinaryReader(stream);
string receivedObjectString = reader.ReadString();
if (!string.IsNullOrEmpty(receivedObjectString ))
obj= JsonSerializer.Deserialize<MyObject>(receivedObjectString);
}
// AddReceivedObjectToList(obj); -- this is a function I need in my program. It adds the object received from Process B into a list in Process A, after it is read by Process A from the MMF
ewh.Dispose();
myProcess.Kill(); // I will kill Process B manually after Process A read everything from the MMF
Process B:
MyObject obj= GetObject();
string serializedObject = JsonSerializer.Serialize(obj);
using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("ObjectMMF", MemoryMappedFileRights.ReadWrite, HandleInheritability.Inheritable))
{
bool ewhExists = EventWaitHandle.TryOpenExisting("ObjectAddedEvent", out EventWaitHandle ewh);
using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
BinaryWriter writer = new BinaryWriter(stream);
writer.Write(serializedObject);
}
ewh.Set(); // This gives Process A a signal that it finished its job and Process A will continue to read the data from the MMF
}
Related
I'm trying to use NAudio to record some sound in C# using WasapiLoopbackCapture and WaveFileWriter.
The problem is that after the recording is finished, the "size" field in the WAV/RIFF header is set to 0, rendering the file unplayable.
I'm using the following code:
WasapiLoopbackCapture CaptureInstance = null;
WaveFileWriter RecordedAudioWriter = null;
void StartSoundRecord()
{
string outputFilePath = #"C:\RecordedSound.wav";
// Redefine the capturer instance with a new instance of the LoopbackCapture class
CaptureInstance = new WasapiLoopbackCapture();
// Redefine the audio writer instance with the given configuration
RecordedAudioWriter = new WaveFileWriter(outputFilePath, CaptureInstance.WaveFormat);
// When the capturer receives audio, start writing the buffer into the mentioned file
CaptureInstance.DataAvailable += (s, a) =>
{
// Write buffer into the file of the writer instance
RecordedAudioWriter.Write(a.Buffer, 0, a.BytesRecorded);
};
// When the Capturer Stops, dispose instances of the capturer and writer
CaptureInstance.RecordingStopped += (s, a) =>
{
RecordedAudioWriter.Dispose();
RecordedAudioWriter = null;
CaptureInstance.Dispose();
};
// Start audio recording !
CaptureInstance.StartRecording();
}
void StopSoundRecord()
{
if(CaptureInstance != null)
{
CaptureInstance.StopRecording();
}
}
(Borrowed from: https://ourcodeworld.com/articles/read/702/how-to-record-the-audio-from-the-sound-card-system-audio-with-c-using-naudio-in-winforms )
Which I'm testing with simply:
StartSoundRecord();
Thread.Sleep(10000);
StopSoundRecord();
What am I missing, why isn't WaveFileWriter writing the size field? I have tried also calling the Flush() and Close() methods before disposing. But it makes no difference.
Sure, I could write a method to find out the size of the file and manually writing it to the final file, but that seems unnecessary.
Found the solution.
Calling RecordedAudioWriter.Flush() after every Write made it work just fine.
Don't know if doing so might be inefficient (as I assume flush is blocking until data is written to disk), but for my application that is not an issue.
I am using MemoryMappedFile for communication between 2 programs. Program "A" creates the mmf and reads it's contents on a timer. Program "B" writes xml data to the mmf on a timer. I have the memory map working but I run into an issue where the previous iteration of the XML data is longer than the current and old data gets carried over to the next round.
so for simplicity lets say program B writes
aaaa
Program A will read correctly,
Then the next write from program B is:
b
Program A reads
baaa
It seems like there should be some simple way to flush the contents of the memory mapped file but I can't seem to figure it out. It's very possible that I'm totally wrong in the way I'm going about this.
Here's what I'm currently doing.
Program A:
using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("testmap",MemoryMappedFileRights.ReadWrite))
{
Mutex mutex = Mutex.OpenExisting("testmapmutex");
mutex.WaitOne();
string outputtext;
using (MemoryMappedViewStream stream = mmf.CreateViewStream(0,0))
{
XmlSerializer deserializer = new XmlSerializer(typeof(MyObject));
TextReader textReader = new StreamReader(stream);
outputtext = textReader.ReadToEnd();
textReader.Close();
}
mutex.ReleaseMutex();
return outputtext; //ends up in a textbox for debugging
}
Program B
using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("testmap", MemoryMappedFileRights.ReadWrite))
{
Mutex mutex = Mutex.OpenExisting("testmapmutex");
mutex.WaitOne();
using (MemoryMappedViewStream stream = mmf.CreateViewStream(0, 0))
{
XmlSerializer serializer = new XmlSerializer(typeof(MyObject));
TextWriter textWriter = new StreamWriter(stream);
serializer.Serialize(textWriter, myObjectToExport);
textWriter.Flush();
}
mutex.ReleaseMutex();
}
Assuming length is reasonably small, you could really clear it out
textWriter.BaseStream.Seek(0, System.IO.SeekOrigin.Begin);
textWriter.BaseStream.Write(new byte[length], 0, length);
textWriter.BaseStream.Seek(0, System.IO.SeekOrigin.Begin);
EDIT: I think I misunderstood the OP's question. The problem he was having was not with clearing the contents of the MMF, but with stream manipulation. This should fix the problem:
textWriter.BaseStream.Seek(0, System.IO.SeekOrigin.Begin);
textWriter.Write("");
textWriter.Flush();
That being said, you might want to do both.
I haven't really worked with MemoryMappedStreams much but this question seemed interesting so I took a crack at it. I wrote a really basic windows example with two buttons (read/write) and a single text box. I didn't pass in "0, 0" to the CreateViewStream calls and I created the file with a fixed length using a call to "CreateOrOpen" and everything worked well! The following are the key pieces of code that I wrote:
WRITE The File
// create the file if it doesn't exist
if (sharedFile == null) sharedFile = MemoryMappedFile.CreateOrOpen("testmap", 1000, MemoryMappedFileAccess.ReadWrite);
// process safe handling
Mutex mutex = new Mutex(false, "testmapmutex");
if (mutex.WaitOne()) {
try {
using (MemoryMappedViewStream stream = sharedFile.CreateViewStream()) {
var writer = new StreamWriter(stream);
writer.WriteLine(txtResult.Text);
writer.Flush();
}
}
finally { mutex.ReleaseMutex(); }
}
READ The File
// create the file if it doesn't exist
if (sharedFile == null) sharedFile = MemoryMappedFile.CreateOrOpen("testmap", 1000, MemoryMappedFileAccess.ReadWrite);
// process safe handling
Mutex mutex = new Mutex(false, "testmapmutex");
if (mutex.WaitOne()) {
try {
using (MemoryMappedViewStream stream = sharedFile.CreateViewStream()) {
var textReader = new StreamReader(stream);
txtResult.Text = textReader.ReadToEnd();
textReader.Close();
}
}
finally { mutex.ReleaseMutex(); }
}
Dispose the file (after finished)
if (sharedFile != null) sharedFile.Dispose();
For the full example, see here: https://github.com/goopyjava/memory-map-test. Hope that helps!
EDIT/NOTE - If you look at the example provided you can write to the file as many times as you want and any time you read you will read exactly/only what was written last. I believe this was the original goal of the question.
I have simple page, that loads XML from filesystem, fills textboxes, these can be updated and saved. For serializing and deserializing I am using these methods:
private static readonly object FormDataLock = new object();
public static FormData getFormData(string filename)
{
FormData fd;
lock (FormDataLock)
{
XmlSerializer x = new XmlSerializer(typeof(FormData));
using (Stream s = new FileStream(filename, FileMode.Open, FileAccess.Read))
{
return (FormData)x.Deserialize(s);
}
}
}
public void saveFormData(string filename)
{
lock (FormDataLock)
{
XmlSerializer x = new XmlSerializer(typeof(FormData));
using (Stream s = new FileStream(filename, FileMode.Create, FileAccess.Write))
{
x.Serialize(s, this);
}
}
}
But the problem is, that I am gettig sometimes (as I have notticed when I click the "save" button too fast after PageLoad) the IOException:
IOException: The process cannot access the file ".." because it is being used by another process.
I was trying to lock the block with mutex, but it is still not working properly. The page form is quite simple, but I am using UpdatePanel on it (is it important?).
When the page is loaded and first save request was done OK, I can click the button as fast as I can and everything is OK (no exception).
XmlSerialization creates new dll's on the fly which are specific to the class you're trying to serialise in the temp directory. These are created to increase performance.
See http://msdn.microsoft.com/en-us/library/swxzdhc0.aspx
Instead of calling the GC.Collect etc... try creating the serializer as a static field on your class. This should improve performance and might solve your problem as it's only ever going to be created once.
This code will create a single xmlserializer in a thread safe way. Do NOT add a [ThreadStatic] attribute to this as this will ensure the code gets executed once per thread and make it thread unsafe again!
private static readonly XmlSerializer xmlSerializer =
new XmlSerializer(typeof(FormData));
I had similar problem and I hope this will help you too.
The problem was that garbage collector didn't clean up before your second click, so you should try to call it manually. Try to call GC before living using
GC.Collect();
GC.WaitForPendingFinalizers();
I've made a simple test with a MemoryMappedFile as msdn says :
2 processes, 1 memory mapped file :
the first process adds the string "1"
the first process waits
the second process adds the string "2" and terminates
the first process now reads the whole memory mapped file
process A:
using (MemoryMappedFile mmf = MemoryMappedFile.CreateNew("testmap", 10000))
{
bool mutexCreated;
Mutex mutex = new Mutex(true, "testmapmutex", out mutexCreated);
using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
BinaryWriter writer = new BinaryWriter(stream, Encoding.UTF8);
writer.Write("1");
}
mutex.ReleaseMutex();
Console.WriteLine("Start Process B and press ENTER to continue.");
Console.ReadLine();
mutex.WaitOne();
using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
BinaryReader reader = new BinaryReader(stream, Encoding.UTF8);
Console.WriteLine("Process A says: {0}", reader.ReadString());
Console.WriteLine("Process B says: {0}", reader.ReadString());
}
mutex.ReleaseMutex();
}
process B:
using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("testmap"))
{
Mutex mutex = Mutex.OpenExisting("testmapmutex");
mutex.WaitOne();
using (MemoryMappedViewStream stream = mmf.CreateViewStream(1, 0))
{
BinaryWriter writer = new BinaryWriter(stream, Encoding.UTF8);
writer.Write("2");
}
mutex.ReleaseMutex();
}
The result is :
Hu ?
Where is "1", "2" ?
However, if I run ONLY the first process ( without activating process B) I get :
What am I missing ?
I expect to see :
Process A says: 1
Process B says: 2
You are battling an implementation detail of BinaryWriter.Write(string). It writes the length of the string first, required so that BinaryReader knows how many characters it needs to read when reading the string back. For short strings, like "1", it writes a single byte to store the length.
So the offset you pass to CreateViewStream() is wrong, passing 1 will make it overwrite part of the string written by process A. The smiley character you see is the glyph for (char)1. The length byte of the string written by process B.
Memory mapped files are troublesome in managed code. You normally read and write to them by declaring a struct to set the layout and using pointers to access the view but that requires unsafe code. Streams are a pretty poor abstraction for a chunk of memory but a necessary evil. Also the reason it took so long for MMFs to become available in .NET.
EDIT
I noticed one apparently strange thing in the code of ProcessB. This code
using (MemoryMappedViewStream stream = mmf.CreateViewStream(1, 0))
creates a view from the first byte, but the strings in .NET are 2 bytes. I think it should be enough to you to make that 1->2 become 2. So the offset of the ProcessB view from the start of the mapped file will be after already inserted "1" string from ProcessA.
In your case seems that you overlap them.
Hope this helps.
In C# (.NET 4.0 running under Mono 2.8 on SuSE) I would like to run an external batch command and capture its ouput in binary form. The external tool I use is called 'samtools' (samtools.sourceforge.net) and among other things it can return records from an indexed binary file format called BAM.
I use Process.Start to run the external command, and I know that I can capture its output by redirecting Process.StandardOutput. The problem is, that's a text stream with an encoding, so it doesn't give me access to the raw bytes of the output. The almost-working solution I found is to access the underlying stream.
Here's my code:
Process cmdProcess = new Process();
ProcessStartInfo cmdStartInfo = new ProcessStartInfo();
cmdStartInfo.FileName = "samtools";
cmdStartInfo.RedirectStandardError = true;
cmdStartInfo.RedirectStandardOutput = true;
cmdStartInfo.RedirectStandardInput = false;
cmdStartInfo.UseShellExecute = false;
cmdStartInfo.CreateNoWindow = true;
cmdStartInfo.Arguments = "view -u " + BamFileName + " " + chromosome + ":" + start + "-" + end;
cmdProcess.EnableRaisingEvents = true;
cmdProcess.StartInfo = cmdStartInfo;
cmdProcess.Start();
// Prepare to read each alignment (binary)
var br = new BinaryReader(cmdProcess.StandardOutput.BaseStream);
while (!cmdProcess.StandardOutput.EndOfStream)
{
// Consume the initial, undocumented BAM data
br.ReadBytes(23);
// ... more parsing follows
But when I run this, the first 23bytes that I read are not the first 23 bytes in the ouput, but rather somewhere several hundred or thousand bytes downstream. I assume that StreamReader does some buffering and so the underlying stream is already advanced say 4K into the output. The underlying stream does not support seeking back to the start.
And I'm stuck here. Does anyone have a working solution for running an external command and capturing its stdout in binary form? The ouput may be very large so I would like to stream it.
Any help appreciated.
By the way, my current workaround is to have samtools return the records in text format, then parse those, but this is pretty slow and I'm hoping to speed things up by using the binary format directly.
Using StandardOutput.BaseStream is the correct approach, but you must not use any other property or method of cmdProcess.StandardOutput. For example, accessing cmdProcess.StandardOutput.EndOfStream will cause the StreamReader for StandardOutput to read part of the stream, removing the data you want to access.
Instead, simply read and parse the data from br (assuming you know how to parse the data, and won't read past the end of stream, or are willing to catch an EndOfStreamException). Alternatively, if you don't know how big the data is, use Stream.CopyTo to copy the entire standard output stream to a new file or memory stream.
Since you explicitly specified running on Suse linux and mono, you can work around the problem by using native unix calls to create the redirection and read from the stream. Such as:
using System;
using System.Diagnostics;
using System.IO;
using Mono.Unix;
class Test
{
public static void Main()
{
int reading, writing;
Mono.Unix.Native.Syscall.pipe(out reading, out writing);
int stdout = Mono.Unix.Native.Syscall.dup(1);
Mono.Unix.Native.Syscall.dup2(writing, 1);
Mono.Unix.Native.Syscall.close(writing);
Process cmdProcess = new Process();
ProcessStartInfo cmdStartInfo = new ProcessStartInfo();
cmdStartInfo.FileName = "cat";
cmdStartInfo.CreateNoWindow = true;
cmdStartInfo.Arguments = "test.exe";
cmdProcess.StartInfo = cmdStartInfo;
cmdProcess.Start();
Mono.Unix.Native.Syscall.dup2(stdout, 1);
Mono.Unix.Native.Syscall.close(stdout);
Stream s = new UnixStream(reading);
byte[] buf = new byte[1024];
int bytes = 0;
int current;
while((current = s.Read(buf, 0, buf.Length)) > 0)
{
bytes += current;
}
Mono.Unix.Native.Syscall.close(reading);
Console.WriteLine("{0} bytes read", bytes);
}
}
Under unix, file descriptors are inherited by child processes unless marked otherwise (close on exec). So, to redirect stdout of a child, all you need to do is change the file descriptor #1 in the parent process before calling exec. Unix also provides a handy thing called a pipe which is a unidirectional communication channel, with two file descriptors representing the two endpoints. For duplicating file descriptors, you can use dup or dup2 both of which create an equivalent copy of a descriptor, but dup returns a new descriptor allocated by the system and dup2 places the copy in a specific target (closing it if necessary). What the above code does, then:
Creates a pipe with endpoints reading and writing
Saves a copy of the current stdout descriptor
Assigns the pipe's write endpoint to stdout and closes the original
Starts the child process so it inherits stdout connected to the write endpoint of the pipe
Restores the saved stdout
Reads from the reading endpoint of the pipe by wrapping it in a UnixStream
Note, in native code, a process is usually started by a fork+exec pair, so the file descriptors can be modified in the child process itself, but before the new program is loaded. This managed version is not thread-safe as it has to temporarily modify the stdout of the parent process.
Since the code starts the child process without managed redirection, the .NET runtime does not change any descriptors or create any streams. So, the only reader of the child's output will be the user code, which uses a UnixStream to work around the StreamReader's encoding issue,
I checked out what's happening with reflector. It seems to me that StreamReader doesn't read until you call read on it. But it's created with a buffer size of 0x1000, so maybe it does. But luckily, until you actually read from it, you can safely get the buffered data out of it: it has a private field byte[] byteBuffer, and two integer fields, byteLen and bytePos, the first means how many bytes are in the buffer, the second means how many have you consumed, should be zero. So first read this buffer with reflection, then create the BinaryReader.
Maybe you can try like this:
public class ThirdExe
{
private static TongueSvr _instance = null;
private Diagnostics.Process _process = null;
private Stream _messageStream;
private byte[] _recvBuff = new byte[65536];
private int _recvBuffLen;
private Queue<TonguePb.Msg> _msgQueue = new Queue<TonguePb.Msg>();
void StartProcess()
{
try
{
_process = new Diagnostics.Process();
_process.EnableRaisingEvents = false;
_process.StartInfo.FileName = "d:/code/boot/tongueerl_d.exe"; // Your exe
_process.StartInfo.UseShellExecute = false;
_process.StartInfo.CreateNoWindow = true;
_process.StartInfo.RedirectStandardOutput = true;
_process.StartInfo.RedirectStandardInput = true;
_process.StartInfo.RedirectStandardError = true;
_process.ErrorDataReceived += new Diagnostics.DataReceivedEventHandler(ErrorReceived);
_process.Exited += new EventHandler(OnProcessExit);
_process.Start();
_messageStream = _process.StandardInput.BaseStream;
_process.BeginErrorReadLine();
AsyncRead();
}
catch (Exception e)
{
Debug.LogError("Unable to launch app: " + e.Message);
}
private void AsyncRead()
{
_process.StandardOutput.BaseStream.BeginRead(_recvBuff, 0, _recvBuff.Length
, new AsyncCallback(DataReceived), null);
}
void DataReceived(IAsyncResult asyncResult)
{
int nread = _process.StandardOutput.BaseStream.EndRead(asyncResult);
if (nread == 0)
{
Debug.Log("process read finished"); // process exit
return;
}
_recvBuffLen += nread;
Debug.LogFormat("recv data size.{0} remain.{1}", nread, _recvBuffLen);
ParseMsg();
AsyncRead();
}
void ParseMsg()
{
if (_recvBuffLen < 4)
{
return;
}
int len = IPAddress.NetworkToHostOrder(BitConverter.ToInt32(_recvBuff, 0));
if (len > _recvBuffLen - 4)
{
Debug.LogFormat("current call can't parse the NetMsg for data incomplete");
return;
}
TonguePb.Msg msg = TonguePb.Msg.Parser.ParseFrom(_recvBuff, 4, len);
Debug.LogFormat("recv msg count.{1}:\n {0} ", msg.ToString(), _msgQueue.Count + 1);
_recvBuffLen -= len + 4;
_msgQueue.Enqueue(msg);
}
The key is _process.StandardOutput.BaseStream.BeginRead(_recvBuff, 0, _recvBuff.Length, new AsyncCallback(DataReceived), null); and the very very important is that convert to asynchronous reads event like Process.OutputDataReceived.