We Keep Coding

what are the best practices to use python in c# application?

I have a requirement for using Python analytics into c# application.
To be precise: c# should call a python script and get the output back into c# application for further processing.
I have tried using IronPython as recommend by many.
ScriptEngine engine = Python.CreateEngine();
ScriptScope scope = engine.CreateScope();
engine.ExecuteFile(#"DemoPythonApplication\SentimentAnalysis.py", scope);
dynamic testFunction = scope.GetVariable("create_sentiment_analysis"); //calling a function from python script file
var result = testFunction(); //This function is returning a dynamic dictionary, which I can use in my c# code further
But the limitation with IronPython is, it is not providing support for many python libraries like pandas, numpy, nltk etc, These libraries are getting used in python scripts. (since we have a different team working on python I don't have control over them for using specific libraries.)
Another option i tried is to run the python process and calling the script
private static readonly string PythonLocation = #"Programs\Python\Python37\python.exe"; //Location of Python.exe
private static readonly string PythonScript = #"DemoPythonApplication\SentimentAnalysis.py"; //Location of Python Script
private static void ProcessInPython(int a, int b)
{
ProcessStartInfo start = new ProcessStartInfo();
start.FileName = PythonLocation;
start.Arguments = string.Format("{0} {1} {2}", PythonScript, a, b);
start.UseShellExecute = false;
start.RedirectStandardOutput = true;
using (Process process = Process.Start(start))
{
using (StreamReader reader = process.StandardOutput)
{
var result = reader.ReadToEnd();
Console.Write(result);
}
}
}
There are limitations of using this approach though, I am only able to get whatever is getting printed on the console as a string, and could not get the output that python functions are returning.
Also if I use the 2nd approach, I don't know how to call a specific function from the python script file.
Can someone help with the best practices of using python in c# for this kind of situation?

How to avoid avoid broken pipes using c# and Python?

When trying to use stdin and stdout in C# (Unity) to pipe to a Python process, I get about a dozen or so transactions and the process breaks and the error "ObjectDisposedException: The object was used after being disposed."
After trying several of the more obvious things, I'm bringing the problem here perhaps someone know just the right technique. Thanks in advance.
Here's the C# Startup code:
Process pyProcess; // <=== fixed
ProcessStartInfo pyStartInfo;
public StreamReader pyStreamReader;
public StreamWriter pyStreamWriter;
public void startPython()
{
// Create new process start info
pyStartInfo = new ProcessStartInfo(pyPath)
{
UseShellExecute = false,
RedirectStandardInput = true,
RedirectStandardOutput = true,
Arguments = pyApp + " " + pyArgs
};
pyProcess = new Process { StartInfo = pyStartInfo };
pyProcess.Start();
pyStreamReader = pyProcess.StandardOutput;
pyStreamWriter = pyProcess.StandardInput;
pyStreamWriter.WriteLine("Hello!");
string str = pyStreamReader.ReadLine();
Debug.LogFormat(str + "\n");
}
void Start()
{
if(testPython == true)
startPython();
Here is the fragment that generates data sent to python at each update...
if (controller.testPython)
{
string str, python;
str = String.Format("data to send");
pyStreamWriter.DiscardBufferedData(); #<==== fixed
pyStreamWriter.WriteLine(str);
python = pyStreamReader.ReadLine();
Debug.LogFormat("python says: " + python + "\n");
}
And here is the simplified python process that's echoing the data
while True:
cmd = input() # read a command from c#
print(cmd) # process the cmd, here we just echo it back to c#

After a little experimentation, I discovered that adding
pyStreamReader.DiscardBufferedData();
before
pyStreamWriter.WriteLine(str);
solves the main problem and this simple form of piping seems to work, at least for hundreds of transactions that I observed.
I also had to declare pyProcess outside the scope so the code so that its handle is not released. That resolved the ObjectDisposed exception.

execute a python script in C#

I am trying to execute a python code in C#. Normally it should be done using IronPython and after installing PTVS (I'm using VS 2010).
var pyEngine = Python.CreateEngine();
var pyScope = pyEngine.CreateScope();
try
{
pyEngine.ExecuteFile("plot.py", pyScope);
}
catch (Exception ex)
{
Console.WriteLine("There is a problem in your Python code: " + ex.Message);
}
The problem is that it seems that IronPython doesn't recognize some libraries like numpy, pylab or matplotlib. I took a look a little bit and found some people talking about Enthought Canopy or Anaconda, which i have both installed without fixing the problem.
What should I do to get the problem solved?

In order to execute a Python script which imports some libraries such as numpy and pylab, it is possible to make this:
string arg = string.Format(#"C:\Users\ayed\Desktop\IronPythonExamples\RunExternalScript\plot.py"); // Path to the Python code
Process p = new Process();
p.StartInfo = new ProcessStartInfo(#"D:\WinPython\WinPython-64bit-2.7.5.3\python-2.7.5.amd64\python.exe", arg);
p.StartInfo.UseShellExecute = false;
p.StartInfo.CreateNoWindow = true; // Hide the command line window
p.StartInfo.RedirectStandardOutput = false;
p.StartInfo.RedirectStandardError = false;
Process processChild = Process.Start(p.StartInfo);

If you execute your code, IronPython will only look for the script in the current working directory. You need to add some more search paths. This is a part of some old integration code in my application using ironpython:
var runtimeSetup = Python.CreateRuntimeSetup(null);
runtimeSetup.DebugMode = false;
runtimeSetup.Options["Frames"] = true;
runtimeSetup.Options["FullFrames"] = true;
var runtime = new ScriptRuntime(runtimeSetup);
var scriptEngine = runtime.GetEngineByTypeName(typeof(PythonContext).AssemblyQualifiedName);
// Set default search paths
ICollection<string> searchPaths = scriptEngine.GetSearchPaths();
searchPaths.Add("\\Scripts\\Python");
scriptEngine.SetSearchPaths(searchPaths);
The trick is to add all paths in this code line: scriptEngine.SetSearchPaths(searchPaths);. If you add the directory which contains plot.py here, all should work.
Hope this helps.

How do I execute and return the results of a python script in c#?

How do I execute and return the results of a python script in c#?
I am trying to run a python script from my controller.
I have python.exe setup in a virtual environment folder created with the virtualenv command.
So just for testing purposes at the moment I would like to just return resulting string from my phython script:
# myscript.py
print "test"
And display that in a view in my asp.net mvc app.
I got the run_cmd function from a related stackoverflow question.
I've tried adding the -i option to force interactive mode and calling process.WaitForExit() with no luck.
namespace NpApp.Controllers
{
public class HomeController : Controller
{
public ActionResult Index(string returnUrl)
{
ViewBag.ReturnUrl = returnUrl;
ViewBag.textResult = run_cmd("-i C:/path/to/virtualenv/myscript.py", "Some Input");
return View();
}
private string run_cmd(string cmd, string args)
{
ProcessStartInfo start = new ProcessStartInfo();
start.FileName = #"C:/path/to/virtualenv/Scripts/python.exe";
start.CreateNoWindow = true;
start.Arguments = string.Format("{0} {1}", cmd, args);
start.UseShellExecute = false;
start.RedirectStandardOutput = true;
using (Process process = Process.Start(start))
{
using (StreamReader reader = process.StandardOutput)
{
string result = reader.ReadToEnd();
//Console.Write(result);
process.WaitForExit();
return result;
}
}
}
}
}
It seems like myscript.py never even runs. But I get no errors, just a blank variable in my view.
Edit:
I had tried to simplify the above stuff because I thought it would be easier to explain and get an answer. Eventually I do need to use a package called "nameparser" and store the result of passed name argument into a database. But if I can just get the run_cmd to return a string I think I can take care of the rest of it. This is why I think the rest api and IronPython mentioned in the comments may not work for me here.

Ok, I figured out what the issue was thanks to some leads from the comments. Mainly it was the spaces in the path to the python.exe and the myscript.py. Turns out I didn't need -i or process.WaitForExit(). I just moved the python virtual environment into a path without spaces and everything started working. Also made sure that the myscript.py file was executable.
This was really helpful:
string stderr = process.StandardError.ReadToEnd();
string stdout = process.StandardOutput.ReadToEnd();
Debug.WriteLine("STDERR: " + stderr);
Debug.WriteLine("STDOUT: " + stdout);
That shows the python errors and output in the Output pane in Visual Studio.

Using Ghostscript in a Webapplication (PDF Thumbnails)

i am using the ghostscriptsharp wrapper for c# and ghostscript. I want to generate thumbnails out of pdf-files.
There are different Methods imported form the ghostscript-c-dll "gsdll32.dll".
[DllImport("gsdll32.dll", EntryPoint = "gsapi_new_instance")]
private static extern int CreateAPIInstance(out IntPtr pinstance,
IntPtr caller_handle);
[DllImport("gsdll32.dll", EntryPoint = "gsapi_init_with_args")]
private static extern int InitAPI(IntPtr instance, int argc, IntPtr argv);
//...and so on
I am using the GhostscriptWrapper for generating the thumbnails in a webapplication (.net 2.0). This class uses the methods imported above.
protected void Page_Load(object sender, EventArgs e){
GhostscriptWrapper.GeneratePageThumb("c:\\sample.pdf", "c:\\sample.jpg", 1, 100, 100);
}
When i debug the Web-Application in Visual Studio 2008 by hitting key "F5" it works fine (a new instance of webserver is generated). When i create a WindowsForm Application it works too. The thumbnails get generated.
When i access the application with the webbrowser directly (http://localhoast/mywebappliation/..) it doesn't work. No thumbnails are generated. But there is also no exception thrown.
I placed the gsdll32.dll in the system32-folder of windows xp. The Ghostscript Runtime is installed too. I have given full access in the IIS-Webproject (.Net 2.0).
Does anybody know why i can't access Ghostscript from my webapplication? Are there any security-issues for accessing dll-files on the IIS-Server?
Greetings
Klaus

Try changing the current directory
string workingDirectory = #"C:\tmp";
Directory.SetCurrentDirectory(workingDirectory);
GhostscriptWrapper.GeneratePageThumb("c:\\sample.pdf", "c:\\sample.jpg", 1, 100, 100);

I now have a workaround. I am not accessing Ghostscript with the GhostscriptWrapper-Class. Instead i access the cmd.exe on the server directly. The following method takes a command (ghostscript syntax) and runs it in cmd.exe. I used the following method for doing this:
public static string runCommand(string workingDirectory, string command)
{
// Create the ProcessInfo object
System.Diagnostics.ProcessStartInfo psi = new System.Diagnostics.ProcessStartInfo("cmd.exe");
psi.UseShellExecute = false;
psi.RedirectStandardOutput = true;
psi.RedirectStandardInput = true;
psi.RedirectStandardError = true;
psi.WorkingDirectory = workingDirectory;
// Start the process
System.Diagnostics.Process proc = System.Diagnostics.Process.Start(psi);
// Attach the output for reading
System.IO.StreamReader sOut = proc.StandardOutput;
// Attach the in for writing
System.IO.StreamWriter sIn = proc.StandardInput;
sIn.WriteLine(command);
// strm.Close();
// Exit CMD.EXE
string stEchoFmt = "# {0} run successfully. Exiting";
// sIn.WriteLine(String.Format(stEchoFmt, targetBat));
sIn.WriteLine("EXIT");
// Close the process
proc.Close();
// Read the sOut to a string.
string results = sOut.ReadToEnd().Trim();
// Close the io Streams;
sIn.Close();
sOut.Close();
// Write out the results.
string fmtStdOut = "<font face=courier size=0>{0}</font>";
return String.Format(fmtStdOut, results.Replace(System.Environment.NewLine, "<br>"));
}

Its possible that the identity you are running the web site under does not have write permissions for c:\