We Keep Coding

Can a text file be updated by two programs simultaneously (live communication)?

I have a program that has c/c# abilities, and I have python. I want that program to update a text file, almost in milliseconds, and have the python to read that text file in milliseconds as well. How can I go achieve this?
Is it possible for a text file to be updated live by another program and be read live by python? Is there any alternative way to do this instead of relying on text file.
Basically what I want to do is a bunch of computations on live data from that program using python and send back those computations to the program in form of commands.Can a file not be closed and reopened and yet updated in the memory?

If you start the C/C# process from python with subprocess.Popen then your two programs can communicate via the stdin and stdout pipes:
c_program = subprocess.Popen(["ARGS","HERE"],
stdin = subprocess.PIPE, # PIPE is actually just -1
stdout= subprocess.PIPE, # it indicates to create a new pipe
stderr= subprocess.PIPE #not necessary but useful
)
Then you can read the output of the process with:
data = c_program.stdout.read(n) #read n bytes
#or read until newine
line = c_program.stdout.readline()
Note that both of these are blocking methods, although non blocking alternatives exist.
Also note that in python 3 these will return bytes objects, you can convert into a str with the .decode() method.
Then to send input to the process you can simply write to the stdin:
c_program.stdin.write(DATA)
Like the read above, in python 3 this method expects a bytes object. You can use the str.encode method to encode it before writing it to the pipe.
I have extremely limited knowledge of C# but from limited research it seems that you can read data from System.Console.In and write data to System.Console.Out, although if you have written programs in C# that run in a terminal, the same methods used to write data to the screen and read input from the user will work here too. (you can imagine the .stdout as the terminal screen and data python writes to .stdin the user input)

Reading from Standard Input in C# [duplicate]

This question already has answers here:
C# Console receive input with pipe
(8 answers)
Closed 7 years ago.
I'm trying to read in a text file from the command prompt in C#, via
program.exe < textfile.txt
However, I cannot find the correct way to do this.
So far, I've only been able to pass the path and the filename to string [] args and then opening the file with the StreamReader class. While this is an acceptable alternative, I've been told that the method with "<", which I suppose is a redirection of standard input, offers advantages like not requiring file handling.
Can anyone give some insight into this?
edit: Program.exe is my C# application.

You've got the right idea - the '<' symbol means the Console class reads from the file you specify, instead of reading user input from the console. When you do this, you read from it using the Console class.
The advantages of reading from STDIN is that the user can either run the program as program.exe, and manually type the input the program is expecting, or they can run it as program.exe < input.txt. The only time this is a disadvantage is if you know you will always supply a file, and consider the effort of typing the '<' symbol too much...

In a command prompt the < sign is one of several redirection operators. Specifically, the < sign is the input redirection operator. When you type this: program.exe < textfile.txt, you are telling the command prompt to execute program.exe and read the command input from a file, instead of reading input from the keyboard. In this way, the command prompt basically opens textfile.txt and takes its content and "stuffs it" into the keyboard buffer, so, as far as program.exe is concerned, the input is being read from the keyboard and has no idea you are actually "stuffing" the keyboard buffer with contents from a file.
If your program currently is reading from a file, you will need to modify your program. You no longer want to read from a file and instead read from the keyboard, using commands such as Console.ReadLine or Console.Read or Console.ReadKey.
As far as advantages, the advantages are minimal.

How to monitor windows service text output

For my windows service, I have a procedure that outputs log entries to a text file. That all works well, but I can not see it live. As soon as I open the file to see the log, it stops writing to it, because the service can't write to an open file, I'm assuming.
My question: Is there a way to monitor this text output with something like the console? This solution will only be used for debugging purposes. I don't want a GUI or anything like that--just something simple to watch what my logger is doing.
Right now to write to the text file I'm doing:
RecordDataToLog("Log this");
private void RecordDataToLog(string txtData)
{
using (System.IO.StreamWriter file = new System.IO.StreamWriter(logFileDir, true))
file.WriteLine(txtData);
}
I tried to use Console like this, but it didn't work--no console window showed.
Console.WriteLine("Log this");
Or is there another text editor or text file type that supports writing to an open file?

I opened the file with notepad and Notepad++ and was still able to write to the file with StreamWriter, so it must be the editor you are opening the file with. Notepad doesn't refresh the contents at all. You need to reopen the file. Notepadd++ should prompt you and ask if you want to reload.
PowerShell v3 includes an option on the Get-Content cmdlet to open a file and print new content as it is added, similar to tail on *nix OSes:
Get-Content "C:\path\to\log\file" -Wait

You will reduce your pain in this area significantly by using a logging framework, such as log4net or NLog.
These will allow you to configure multiple output formats in a separate configuration file. I've converted a home-grown logging system to log4net. Took about half a day to do this - and this was a relatively large commercial application.

You can use Notepad++.
If you start it in the command-line by writing
notepad++.exe -ro "MyLogFile.txt"
then it will open the file in read-only mode.

You can use the Get-Content cmdlet from PowerShell to tail your log file

You can use also the "monitor" tool of NotePad++ (it's an icon with an eye).. doing so NotePad will automatically show you the updated file every time it will be modified by your app

How to open / use a file descriptor

I have a particular problem, I have some program that I cannot modify but that provides some functionality I'd like to use inside office. So I am writing a plugin for Office that takes my document, executes the program on the background, puts the document on the stdin. The program writes to the stdout, and I take that back to my program to post process that.
This all works fine except that the program asks for a password which I don't want to put on stdin. The tool has a way to read the password from any other input stream but it needs the number of the file-descriptor it should read from.
So here is my question: how do I (within the .net environment) open a stream on a file descriptor with a number that I can give as parameter to this program? Ideally I want to write something like:
process.start("start-program --password-fd " + x);
stream = new StreamWriter(x);
stream.write("secritpwd");
ect.. (but then magically corrected so it will work ;) )
I hope someone can help me.
Thanks

I'm not sure exactly what this app means by "file descriptor", but you may be able to pass the handle of an inheritable anonymous pipe. See AnonymousPipeServerStream. (This assumes you're on at least .NET 3.5.)
The basic outline would be something like this:
Instantiate an AnonymousPipeServerStream.
Pass the pipe handle (pipeServer.GetClientHandleAsString()) as a command-line parameter to your C executable.
Write to the AnonymousPipeServerStream.

File descriptors aren't part of Windows - they're part of the C runtime library. You would have to write a DLL in C or C++ to do your file I/O, then call it from your C# program. Get the file descriptor number from the C DLL to pass to your other code.

Maby this links will help you to start and to get the logic from:
How to use OpenFileById to open a file
Opening pipe connection to a file descriptor in C#
OpenFileById Function

How to get around the command line length limit?

I've been working on a small and simple program that I drop files onto and, based on certian rules, they are moved to diffrent places.
The program works fine unless I drop more than a few files, then it kicks back an error (that appears to be more Windows than anything) that the start up command "c:\myapp.exe \file \file \file" is too long.
I realize I could set up a background proccess, but I really would prefer that this program not run in the backround (where it would be idle most of the time).
Is there a way around this limitation?

From this blog:
The maximum command line length for the CreateProcess function is 32767 characters. This limitation comes from the UNICODE_STRING structure.
If you are using the CMD.EXE command processor, then you are also subject to the 8192 character command line length limit imposed by CMD.EXE.
If you are using the ShellExecute/Ex function, then you become subject to the INTERNET_MAX_URL_LENGTH (around 2048) command line length limit imposed by the ShellExecute/Ex functions.
The maximum size of your environment is 32767 characters. The size of the environment includes all the variable names plus all the values.
So you're gonna have to settle with some of the mentioned workarounds (also, there's another workaround on the msdn blog I linked).

If you want to drop files with respect of Windows Explorer, then you can implement your own Drop Handlers as a Shell Extension Handlers see:
How to Create Drop Handlers (Windows)
Creating Shell Extension Handlers
On The Complete Idiot's Guide to Writing Shell Extensions you will find a good introduction how to write such extensions.
The part VI gives an example of Drop Handler (for a little other use case, but it dose not matter).
With respect of Drop Shell Extension Handler your program will receive full information about all dropped files and you need not start a child program with all the files as command like parameters.

I think the drag and drop handler is possibly one way to go, but it seems quite heavy.
An alternative solution is to use a Explorer Context Menu handler. With this in place, you would select all the files, but rather than dragging them, right click, and choose your new menu item "Send to ".
When the menu item is selected, it passes the list of commands to your program. There are a couple of ways of doing this:
launch your program, and feed the list of files to standard input
write the list of files to a temporary file, and launch your program with just one command argument - the temporary file listing the files to process. List files are usually prefixed with '#' on the command line to distinguish them from ordinary file names.

When the files are drag&dropped onto your application write the list of files names out to a text file, then give the location of this file to your program. The target program can then read in this file and process it line-by-line. This way you only ever pass a single file name.

I'd modify the application to pick up the files from a specific location (e.g. specific folder on a filesystem) rather than specifying each file on the command line.
UPDATE:
If the requirement is to be able to drag an item onto an .exe via Windows Explorer to start the application as Mark mentioned then you could put all of your files into one folder and drop the entire folder on the .exe.

I think the easiest solution is to modify your application to accept a directory as the parameter as opposed to a list of files. This would allow you to copy the multiple files into a single directory. You can then drag and drop the folder onto your executable. It would then run a command like "c:\myapp.exe \folder-with-files-in-it" which should not run into the command line parameter limitation that you are experiencing now.

I experienced a similar issue when I was trying to get security details on a long path.
My solution was to map drives whenever the path length got too long. Check out my solution on
How do I get the security details for a long path?

Unix commands frequently have one or two parameters that can be of unbounded length. Several of these commands have added parameters which can source those arguments from a file, or from standard input. So you have one command which gins up the list of arguments, and either pipes them to a temporary file, or pipes them to stdout.
See also, xargs, which can take a list of arguments and either invoke your command with all of the parameters, or in batches.

How to get around the command line length limit? Write your whole command to a batch file, e.g. to "C:\Users\Johnny\Documents\mybatch.bat". Write it as you would in the cmd (no need to escape anything). Then, in your code simply call that file:
strCmdText = "C:\\Users\\Johnny\\Documents\\mybatch.bat";
ProcessStartInfo ProcessInfo = new ProcessStartInfo("cmd.exe", "/K " + strCmdText);
ProcessInfo.CreateNoWindow = true;
ProcessInfo.UseShellExecute = true;
Process.Start(ProcessInfo);