We Keep Coding

Unable to run python script in c# which runs scrapy spider

I followed this_link and I was able to run a dummy python file from my c# code like this...
public JsonResult FetchscrapyDataUrl(String website)
{
ProcessStartInfo start = new ProcessStartInfo();
start.FileName = #"C:\ProgramData\Anaconda3\python.exe";
start.Arguments = #"C:\Users\PycharmProjects\scraping_web\scrape_info\main.py";
//this is path to .py file from scrapy project
start.CreateNoWindow = false; // We don't need new window
start.UseShellExecute = false; // Do not use OS shell
//start.RedirectStandardOutput = true;// Any output, generated by application will be redirected back
start.RedirectStandardError = true; // Any error in standard output will be redirected back (for example exceptions)
Console.WriteLine("Python Starting");
start.RedirectStandardOutput = true;
using (Process process = Process.Start(start))
{
using (StreamReader reader = process.StandardOutput)
{
string stderr = process.StandardError.ReadToEnd(); // Here are the exceptions from our Python script
string result = reader.ReadToEnd(); // Here is the result of StdOut(for example: print "test")
Console.Write(result);
}
}
}
Now I know that I can run scrapy spider from a single file main.py like this...
from scrapy import cmdline
cmdline.execute("scrapy crawl text".split())
When I run main.py file from cmd in windows it works fine but it does not work when I run it from C# code .Net framework. The error is ...
"Scrapy 1.4.0 - no active project\r\n\r\nUnknown command: crawl\r\n\r\nUse \"scrapy\" to see available commands\r\n"
Any Idea how to run this...Or am i missing some path setting in windows ??
Or should I run my spider from C# in some other way??

You need to set the WorkingDirectory property
start.WorkingDirectory = #"C:\Users\PycharmProjects\scraping_web\scrape_info\"
Or you need to cd to that directory to make it work

Inno command line compilation not working

I'm trying to compile .iss file with command line
string INNOCLI = Application.StartupPath + #"\Inno\ISCC.exe";
string Argument = string.Format("iscc /q \"{0}\"", INNOSCRIPTFILE);
using (Process cli = new Process())
{
//cli.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
cli.StartInfo.FileName = INNOCLI;
cli.StartInfo.Arguments = Argument;
cli.StartInfo.UseShellExecute = false;
cli.StartInfo.RedirectStandardError = true;
cli.StartInfo.RedirectStandardOutput = true;
//cli.StartInfo.CreateNoWindow = true;
cli.OutputDataReceived += cli_OutputDataReceived;
cli.ErrorDataReceived += cli_ErrorDataReceived;
cli.Start();
cli.BeginErrorReadLine();
cli.BeginOutputReadLine();
cli.WaitForExit();
}
But i'm getting nothing out of it, i'm using c#
Edit:
I disabled output redirect, now i see its saying "Script file name specified more than once" on console window.

You've said that the output you get from the ISCC tool that you execute is:
Script file name specified more than once
which comes from this exception which is raised if you pass more than one parameter longer than 1 char with no starting /, or - char. And that's what happens because you have mistakenly passed iscc and a file name to your arguments. Remove that mistyped iscc from there. Change this line:
string Argument = string.Format("iscc /q \"{0}\"", INNOSCRIPTFILE);
to this:
string Argument = string.Format("/q \"{0}\"", INNOSCRIPTFILE);

I was experiencing this issue and the fix for me was to specific the script file as the very first argument.

How to catch %ERRORLEVEL% when running Psexec.exe in C#, save to log file

I am using psexec.exe to install some software on a large amount of computers. I would like to be able to catch the %ERRORLEVEL% (or its c# equivalent) and write them to a TXT file. ( cmd.exe exited on 6440-nbpb00f51w with error code 0.)
After extensive research, I am finding several possibilities. My end-goal will be to have a .CSV file. Each row would have the computer name and the output of the psexec command. Using this data I will be able to determine which computers the update was successful on.
string[] pcnamearray = new string[] {"COMPUTERNAME1", "COMPUTERNAME2"};
foreach (string i in pcnamearray)
{
Process p = new Process();
string psexeclaunch = "psexec.exe";
p.StartInfo.FileName = psexeclaunch;
p.StartInfo.RedirectStandardOutput = false;
p.StartInfo.UseShellExecute = true;
p.StartInfo.CreateNoWindow = false;
string arg = "/silent";
p.StartInfo.Arguments = #" -i \\" + i + #" -u mydomain\admin -p mypassword -h -e cmd.exe /c start \\networkserver\sw\myfile.exe" + arg;
p.Start();
}
I am not here looking for someone to write the code for me. I am here to ask people for their advice, or past experiences with this. I know there are several different methods available. Ideally, I would prefer to have a short sweet line for each result, but if i have to take the entire output and use macros to shave them down to a more readable form, that is fine too.
So far, I have tried:
Console.WriteLine(p.StandardOutput.ReadToEnd());
And...
System.IO.File.WriteAllLines(#"C:\Users\areu2447\Desktop\UpdateDAT\Final\out.txt", lines);
And...
FileStream filestream = new FileStream(#"C:\Users\areu2447\Desktop\UpdateDAT\Final\out.txt", FileMode.Append);
var streamwriter = new StreamWriter(filestream);
streamwriter.AutoFlush = true;
Console.SetOut(streamwriter);
Console.SetError(streamwriter);
The last one seems to me like the best one but cannot seem to get it to work. I have tried including it in my foreach loop, but file is in use by another process, so after looking into it i found that it needed to be on its own. Even with it alone, i still cannot get it to work.
Please help/advise!
I was able to get it to actaully modify my TXT file, even though there was nothing added to it, by using the following:
System.IO.StreamReader reader = p.StandardOutput;
String sRes = reader.ReadToEnd();
StreamWriter SW;
SW = File.CreateText(#"C:\Users\areu2447\Desktop\UpdateDAT\Final\out.txt");
SW.WriteLine(sRes);
SW.Close();
Console.WriteLine("File Created Successfully");
reader.Close();

I would recommend using System.Management.Automation to create a PowerShell pipeline and run the script in process.
Executing PowerShell scripts from C#

C# and psexec process output redirection issues

I'm trying to create a small program to run on a centralized device. This program will run
"psexec \server(s) netstat -na | findstr "LISTENING""
to collect netstat data from remote nodes (should redirect output to string), then parse the data and compare against a known list. I can run the psexec cmd above without any issues from the cmd line, but when I try to run the same command as a process within my C# program, no data is returned to be parsed. I can see that the netstat is being run (cmd window flashes with netstat results), but the process.standardoutput is not catching the stream. If I use ping or pretty much anything other than psexec as an argument, the stream is caught and the results are shown in my text box. I've also tried setting the filename to psexec.exe and specifying the arguments but I get the same results. Last but not least, if I run psexec without any arguments, I get the help kickback info returned in my textbox. This is true if I'm running psexec.exe as the filename OR if I run cmd.exe as filename with "/c psexec" specified as args.
I'm just trying to get psexec output to be caught when executing locally at this point. I'll worry about psexec to remote machines later. Any help would be MUCH appreciated.
Here's the code:
System.Diagnostics.Process pProcess = new System.Diagnostics.Process();
pProcess.StartInfo.FileName = "cmd.exe";
pProcess.StartInfo.Arguments = "/c psexec netstat";
pProcess.StartInfo.UseShellExecute = false;
pProcess.StartInfo.RedirectStandardOutput = true;
pProcess.Start();
string strOutput = pProcess.StandardOutput.ReadToEnd();
pProcess.WaitForExit();
if (pProcess.HasExited)
{
textBox1.Text = strOutput;
}
else
{
textBox1.Text = "TIMEOUT FAIL";
}

I would recommend also capturing the standard error output in case anything is being reported there.
Also, you may have a disconnect between "bitness" of psexec and your application if you are running on a 64-bit OS. If this is the case, change the platform for the project to match that of psexec rather than building as Any CPU.

Came across a few things to be changed but your recommendation of capturing standard error output was dead on and a good start. Turns out some info was being sent to the error output (even though really wasn't error, just run status 0 from psexec) so I knew at that point psexec wasn't just eating ALL the output. Once I started trying to pass remote hosts as args, I started getting user/pass error data back. Also needed to catch standard input if I wanted to supply credentials for proc run. Threw in some str literals and credentials for the remote exec, works perfectly. Thanks for the help. Here is the updated code--
System.Diagnostics.Process pProcess = new System.Diagnostics.Process();
pProcess.StartInfo.Domain = "domain";
pProcess.StartInfo.UserName = "user with priv";
pProcess.StartInfo.Password = new System.Security.SecureString();
char [] pass = textBox3.Text.ToArray();
for (int x = 0; x < pass.Length; ++x)
{
pProcess.StartInfo.Password.AppendChar(pass[x]);
}
pProcess.StartInfo.FileName = #"psexec.exe";
pProcess.StartInfo.Arguments = #"\\remoteHost netstat -ano";
pProcess.StartInfo.UseShellExecute = false;
pProcess.StartInfo.RedirectStandardInput = true;
pProcess.StartInfo.RedirectStandardOutput = true;
pProcess.StartInfo.RedirectStandardError = true;
pProcess.Start();
pProcess.WaitForExit(30000);
if (!pProcess.HasExited)
{
pProcess.Kill();
}
string strOutput = pProcess.StandardOutput.ReadToEnd();
string errOutput = pProcess.StandardError.ReadToEnd();
textBox1.Text = strOutput;
textBox2.Text = errOutput;

C#: How would I execute this command line directly to java.exe?

I am writing a program that needs to run a java.jar server. I need to run the process directly so I can rewrite the output to a textbox and all-in-all have complete control of it. I tried just doing it through CMD.exe, but that wouldnt work because CMD.exe would just call a new process java.exe and I wouldn't have control of it. I need to call java.exe directly so I can have the control and get the output. Can any of you tell me how to convert this command so I could create a process in C# and call it?
I need this CMD command converted:
"java -Xmx1024m -cp ./../libs/*;l2jserver.jar net.sf.l2j.gameserver.GameServer"
into
a command line I can put into the Process.Arguments so I can call Java.exe directly.
I've tried to do it... and it just won't work.
I've been looking at this for hours and hours... please someone help!

Part of the problem might be that despite what the Framework documentation says using Process doesn't always resolve things against the PATH environment variable properly. If you know the name of the folder Java is in then use the full path to Java.exe, otherwise use a function like the following:
private void LocateJava()
{
String path = Environment.GetEnvironmentVariable("path");
String[] folders = path.Split(';');
foreach (String folder in folders)
{
if (File.Exists(folder + "java.exe"))
{
this._javadir = folder;
return;
}
else if (File.Exists(folder + "\\java.exe"))
{
this._javadir = folder + "\\";
return;
}
}
}
It's somewhat hacky but it will find java.exe provided the Java Runtime is installed and it's folder is in the windows PATH variable. Make a call to this function the first time your program needs to find Java and then subsequently start Java using the following:
//Prepare the Process
ProcessStartInfo start = new ProcessStartInfo();
if (!_javadir.Equals(String.Empty)) {
start.FileName = this._javadir + "java.exe";
} else {
start.FileName = "java.exe";
}
start.Arguments = "-Xmx1024m -cp ./../libs/*;l2jserver.jar net.sf.l2j.gameserver.GameServer";
start.UseShellExecute = false;
start.RedirectStandardInput = true;
start.RedirectStandardOutput = true;
//Start the Process
Process java = new Process();
java.StartInfo = start;
java.Start();
//Read/Write to/from Standard Input and Output as required using:
java.StandardInput;
java.StandardOutput;