We Keep Coding

Capturing output from powershell script

I have been working on converting a GUI script from another language to C# in VS2017 for a customer. With help from the folks here I am 95% of the way there, but have run into a couple of snags; just not sure I am doing things in the best way. I'm including just the relevant portions of code below, please let me know if I am not providing enough:
The majority of the code is centered on the wpf form, which collects data for low level technicians to batch deploy a number of Virtual Machines into the VMware environment. This number could easily range into the dozens or even a hundred VMs at once. The information for each VM is specified in the form, then collected in a listview. Once the listview is fully populated it is exported to a csv. Up to this point everything works just fine.
I've next been working on actually launching the powershell/powerCLI script (also working) and capturing output. The log file is opened with a specific reader application the customer uses, which updates in real time, and the captured output is fed to the log. It is important for the technicians to see the output from the code line by line so they can react if there is an issue.
I started with something like this as a test:
string sPSScript = "C:\\Users\\Admin\\Desktop\\TestC#.ps1";
string logFile = "C:\\Users\\Admin\\Desktop\\My.log";
string logReader = "C:\\Users\\Admin\\Documents\\CMTrace.exe";
string standard_output;
System.Diagnostics.Process PSScript = new System.Diagnostics.Process();
PSScript.StartInfo.FileName =
Environment.GetFolderPath(Environment.SpecialFolder.SystemX86) +
"\\WindowsPowerShell\\v1.0\\powershell.exe";
PSScript.StartInfo.Arguments = "-command . '" + sPSScript + "' " +
vCenter.Text;
PSScript.StartInfo.RedirectStandardOutput = true;
PSScript.StartInfo.UseShellExecute = false;
PSScript.Start();
System.Diagnostics.Process LogFile = new System.Diagnostics.Process();
LogFile.StartInfo.FileName = logReader;
LogFile.StartInfo.Arguments = logFile;
LogFile.Start(); while ((standard_output =
PSScript.StandardOutput.ReadLine()) != null)
{
if (standard_output != "")
{
using (StreamWriter file = new StreamWriter(logFile, append: true))
{
file.WriteLine(standard_output);
}
}
}
While this writes to the log file in real time as expected, it creates 100 instances of the logReader application. I understand why, since I am declaring a new StreamWriter object through every pass, but am unsure how better to go about this.
I tried creating the file outside the loop, like this:
StreamWriter file = new StreamWriter(logFile, append: true) { };
System.Diagnostics.Process LogFile = new System.Diagnostics.Process();
LogFile.StartInfo.FileName = logReader;
LogFile.StartInfo.Arguments = logFile;
System.Diagnostics.Process PSScript = new System.Diagnostics.Process();
PSScript.StartInfo.FileName = Environment.GetFolderPath(Environment.SpecialFolder.SystemX86) + "\\WindowsPowerShell\\v1.0\\powershell.exe";
PSScript.StartInfo.Arguments = "-command . '" + sPSScript + "' " + vCenter.Text;
PSScript.StartInfo.RedirectStandardOutput = true;
PSScript.StartInfo.UseShellExecute = false;
LogFile.Start();
PSScript.Start();
System.Threading.Thread.Sleep(1500);
while ((standard_output = PSScript.StandardOutput.ReadLine()) != null)
{
if (standard_output != "")
{
file.WriteLine(standard_output);
}
}
It doesn't create multiple instances, but it also does not update the log file in real time as the previous code does. It only updates once the script runs, and then only partially. The script produces ~1000 lines of output, and I consistently see only about 840 written to the log file.
I thought about doing something like this:
FileStream logFS;
logFS = new FileStream(logFile, FileMode.Append);
but it appears the only options available to me to write to the file are expecting a byte array.
I am sure that I am missing something stupid simple in this, but would appreciate any suggestions on the easiest way to create the log file, open it in the reader, and then update it with the standard output from the powershell script.

why did the previous code writes in real time?
because you are wrapping it with using. And at the end of using block its gonna call dispose which calls .Flush to write to disk
Your second code block calls WriteLine but never called Flush so it writes to the disk whenever the buffer is full. Just add a .Flush call after WriteLine and you will have real time logging

Getting full command output from ShellStream of C# SSH.NET

Using Renci.SshNet library. I am trying to execute some commands.
After executing "command 1", I am executing "command 2" which takes more time.
I am only getting the first line of the output. (reader.ReadToEnd() is not working in a proper way).
I also tried while (!reader.EndOfStream){ } with no luck.
I think it is because the delay of the response from server. When there is no response the stream read nothing and finishes.
I found a solution
String tmp;
TimeSpan timeout = new TimeSpan(0, 0, 3);
while ((tmp = s.ReadLine()) != null)
{
Console.WriteLine(tmp);
}
But this is not professional. I need some way in which the stream ends when it ends.
using (var vclient = new SshClient("host", "username", "password"))
{
vclient.Connect();
using (ShellStream shell = vclient.CreateShellStream("dumb", 80, 24, 800, 600, 1024))
{
Console.WriteLine(SendCommand("comand 1", shell));
Console.WriteLine(SendCommand("comand 2", shell));
shell.Close();
}
vclient.Disconnect();
}
public static string SendCommand(string cmd, ShellStream sh)
{
StreamReader reader = null;
try
{
reader = new StreamReader(sh);
StreamWriter writer = new StreamWriter(sh);
writer.AutoFlush = true;
writer.WriteLine(cmd);
while (sh.Length == 0)
{
Thread.Sleep(500);
}
}
catch (Exception ex)
{
Console.WriteLine("exception: " + ex.ToString());
}
return reader.ReadToEnd();
}

The shell is an endless stream. There's no send command – receive output sequence. The ReadToEnd cannot know where an output of one command ends. All you can do is to read until you yourself can tell that the output ended.
If you cannot tell that, you can help yourself by appending some kind of end-of-output mark like:
command 1 ; echo this-is-the-end-of-the-output
and read until you get "this-is-the-end-of-the-output" line.
Generally a "shell" channel is not an ideal solution for automation. It's intended for an interactive sessions.
You better use "exec" channel using SshClient.CreateCommand. With CreateCommand the channel closes once the command finishes. So there's clear "end of the stream", what makes the ReadToEnd() work as you expect. And SSH.NET even makes whole command output available in SshCommand.Result (which internally uses ReadToEnd()).

-ERR Exceeded the login limit for a 15 minute period. Reduce the frequency of requests to the POP3 server

using following code i have reading msg from my hotmail account . But sometimes the following error coming . -ERR Exceeded the login limit for a 15 minute period. Reduce the frequency of requests to the POP3 server . can anyone tell me whats the reason for this ? Is that server problem or anything else ? other than pop3 anyother protocol can we use for hotmail?
public string hotmail(string username, string password)
{
string result = "";
string str = string.Empty;
string strTemp = string.Empty;
try
{
TcpClient tcpclient = new TcpClient();
tcpclient.Connect("pop3.live.com", 995);
System.Net.Security.SslStream sslstream = new SslStream(tcpclient.GetStream());
sslstream.AuthenticateAsClient("pop3.live.com");
System.IO.StreamWriter sw = new StreamWriter(sslstream);
System.IO.StreamReader reader = new StreamReader(sslstream);
strTemp = reader.ReadLine();
sw.WriteLine("USER" + " " + username);
sw.Flush();
strTemp = reader.ReadLine();
sw.WriteLine("PASS" + " " + password);
sw.Flush();
strTemp = reader.ReadLine();
string[] numbers = Regex.Split(strTemp, #"\D+");
int a = 0;
foreach (string value in numbers)
{
if (!string.IsNullOrEmpty(value))
{
int i = int.Parse(value);
numbers[a] = i.ToString();
a++;
}
}
sw.WriteLine("RETR" + " " + numbers[0]);
sw.Flush();
strTemp = reader.ReadLine();
while ((strTemp = reader.ReadLine()) != null)
{
if (strTemp == ".")
{
break;
}
if (strTemp.IndexOf("-ERR") != -1)
{
break;
}
str += strTemp;
}
sw.WriteLine("Quit ");
sw.Flush();
result = str;
return result;
}
Catch ( Exception ex)
{}
return result;
}
thanks in advance ..

Any other protocol you can use? Yes, hotmail/outlook.com now supports IMAP.
But the issue with the code here seems to be that you're creating a new TcpClient every time you run this. If you're running it many times in in a row, Outlook.com/Hotmail will eventually complain. It's as if you've got tons of clients from a single source connecting to their server, which is, when it's not testing code, often a sign of email abuse.
TcpClient tcpclient = new TcpClient(); // Hello, new.
tcpclient.Connect("pop3.live.com", 995);
If you've got a lot to do on the server, keep a single connection active longer, and close it up when you're done.
Every time you run the code in your question, you're creating (and not tcpclient.Close()-ing) a connection to pop3.live.com. I usually only get this error when I've had a lot of connections that don't close properly due to errors when I'm messing with my code.
MSDN actually has a decent example for TcpClient, but you might be more interested in another example from SO here. Check out how it uses using, and nests a loop inside.
using (TcpClient client = new TcpClient())
{
client.Connect("pop3.live.com", 995);
while(variableThatRepresentsRunning)
{
// talk to POP server
}
}
By the way, the best advice I can give here is to tell you not to reinvent the wheel (unless you're just having fun playing with the POP server. Throwing commands via TCP can be lots of fun, especially with IMAP).
OpenPop.NET is a great library to handle POP requests in C#, includes a good MIME parser, and, if you're still working on this, should speed you along quite a bit. Its examples page is excellent.

Go to the mail inbox , you may get mail regarding this and accept it. Otherwise Try to give the request after some time. Because google having some restriction to read mail using pop settings.

Transfer file from Windows Mobile device to...anywhere

I can't seem to find a solution to this issue. I'm trying to get my Compact Framework application on Windows Mobile 6 to have the ability to move a file on its local filesystem to another system.
Here's the solutions I'm aware of:
FTP - Problem with that is most of
the APIs are way to expensive to use.
HTTP PUT - As far as I have been able to find, I can't use anonymous PUT with IIS7, and that's the web server the system is running. (An extreme workaround for this would be to use a different web server to PUT the file, and have that other system transfer it to the IIS system).
Windows share - I would need authentication on the shares, and I haven't seen that a way to pass this authentication through windows mobile.
The last resort would be to require that the devices be cradled to transfer these files, but I'd really like to be able to have these files be transferred wirelessly.

FTP: define "too expensive". Do you mean performance or byte overhead or dollar cost? Here's a free one with source.
HTTP: IIS7 certainly supports hosting web services or custom IHttpHandlers. You could use either for a data upload pretty easily.
A Windows Share simply requires that you to P/Invoke the WNet APIs to map the share, but it's not terribly complex.

I ended up just passing information to a web server via a PHP script.
The options provided above just didn't work out for my situation.
Here's the gist of it. I've got some code in there with progress bars and various checks and handlers unrelated to simply sending a file, but I'm sure you can pick through it. I've removed my authentication code from both the C# and the PHP, but it shouldn't be too hard to roll your own, if necessary.
in C#:
/*
* Here's the short+sweet about how I'm doing this
* 1) Copy the file from mobile device to web server by querying PHP script with paramaters for each line
* 2) PHP script checks 1) If we got the whole data file 2) If this is a duplicate data file
* 3) If it is a duplicate, or we didn't get the whole thing, it goes away. The mobile
* device will hang on to it's data file in the first case (if it's duplicate it deletes it)
* to be tried again later
* 4) The server will then process the data files using a scheduled task/cron job at an appropriate time
*/
private void process_attempts()
{
Uri CheckUrl = new Uri("http://path/to/php/script?action=check");
WebRequest checkReq = WebRequest.Create(CheckUrl);
try
{
WebResponse CheckResp = checkReq.GetResponse();
CheckResp.Close();
}
catch
{
MessageBox.Show("Error! Connection not available. Please make sure you are online.");
this.Invoke(new Close(closeme));
}
StreamReader dataReader = File.OpenText(datafile);
String line = null;
line = dataReader.ReadLine();
while (line != null)
{
Uri Url = new Uri("http://path/to/php/script?action=process&line=" + line);
WebRequest WebReq = WebRequest.Create(Url);
try
{
WebResponse Resp = WebReq.GetResponse();
Resp.Close();
}
catch
{
MessageBox.Show("Error! Connection not available. Please make sure you are online.");
this.Invoke(new Close(closeme));
return;
}
try
{
process_bar.Invoke(new SetInt(SetBarValue), new object[] { processed });
}
catch { }
process_num.Invoke(new SetString(SetNumValue), new object[] { processed + "/" + attempts });
processed++;
line = dataReader.ReadLine();
}
dataReader.Close();
Uri Url2 = new Uri("http://path/to/php/script?action=finalize&lines=" + attempts);
Boolean finalized = false;
WebRequest WebReq2 = WebRequest.Create(Url2);
try
{
WebResponse Resp = WebReq2.GetResponse();
Resp.Close();
finalized = true;
}
catch
{
MessageBox.Show("Error! Connection not available. Please make sure you are online.");
this.Invoke(new Close(closeme));
finalized = false;
}
MessageBox.Show("Done!");
this.Invoke(new Close(closeme));
}
In PHP (thoroughly commented for your benefit!):
<?php
//Get the GET'd values from the C#
//The current line being processed
$line = $_GET['line'];
//Which action we are doing
$action = $_GET['action'];
//# of lines in the source file
$totalLines = $_GET['lines'];
//If we are processing the line, open the data file, and append this new line and a newline.
if($action == "process"){
$dataFile = "tempdata/SOME_KIND_OF_UNIQUE_FILENAME.dat";
//open the file
$fh = fopen($dataFile, 'a');
//Write the line, and a newline to the file
fwrite($fh, $line."\r\n");
//Close the file
fclose($fh);
//Exit the script
exit();
}
//If we are done processing the original file from the C# application, make sure the number of lines in the new file matches that in the
//file we are transferring. An expansion of this could be to compare some kind of hash function value of both files...
if($action == "finalize"){
$dataFile = "tempdata/SOME_KIND_OF_UNIQUE_FILENAME.dat";
//Count the number of lines in the new file
$lines = count(file($dataFile));
//If the new file and the old file have the same number of lines...
if($lines == $totalLines){
//File has the matching number of lines, good enough for me over TCP.
//We should move or rename this file.
}else{
//File does NOT have the same number of lines as the source file.
}
exit();
}
if($action == "check"){
//If a file with this unique file name already exists, delete it.
$dataFile = "tempdata/SOME_KIND_OF_UNIQUE_FILENAME.dat";
if(file_exists($dataFile)){
unlink($dataFile);
}
}
?>

how to create a fresh database before tests run?

how to create a fresh database (everytime) before tests run from a schema file ?

You can use the SchemaExport class in NHibernate to do this in code:
var schema = new SchemaExport(config);
schema.Drop(true, true);
schema.Execute(true, true, false);

drop the entire database - don't drop table by table - that adds too much maintenance overhead

I have used the following utility methods for running SQL scripts for setting up databases and test data in a project that I am working with every now and then. It has worked rather well:
internal static void RunScriptFile(SqlConnection conn, string fileName)
{
long fileSize = 0;
using (FileStream stream = File.OpenRead(fileName))
{
fileSize = stream.Length;
using (StreamReader reader = new StreamReader(stream))
{
StringBuilder sb = new StringBuilder();
string line = string.Empty;
while (!reader.EndOfStream)
{
line = reader.ReadLine();
if (string.Compare(line.Trim(), "GO", StringComparison.InvariantCultureIgnoreCase) == 0)
{
RunCommand(conn, sb.ToString());
sb.Length = 0;
}
else
{
sb.AppendLine(line);
}
}
}
}
}
private static void RunCommand(SqlConnection connection, string commandString)
{
using (SqlCommand command = new SqlCommand(commandString, connection))
{
try
{
command.ExecuteNonQuery();
}
catch (Exception ex)
{
Console.WriteLine(string.Format("Exception while executing statement: {0}", commandString));
Console.WriteLine(ex.ToString());
}
}
}
I have used the Database Publishing Wizard to generate SQL scripts (and in some cases edited them to include only the data I want to use in the test), and just pass the script file paths into the RunScriptFile method before the tests. The method parses the script file and executes each part that is separated by a GO line separately (I found that this greatly helped in troubleshooting errors that happened while running the SQL scripts).
I has been a while since I wrote the code, but I think it requires the the script file ends with a GO line in order for the last part of it to be executed.

Have a look at these posts.
Ayende Rahien - nhibernate-unit-testing
Scott Muc - unit-testing-domain-persistence-with-ndbunit-nhibernate-and-sqlite
I have found them to be very usefull and basically they are extending the example by Mike Glenn

I use Proteus (Unit Test Utility), available on Google code here :
http://code.google.com/p/proteusproject/
You create a set of data. Each time, you run a unit test, the current data are saved, the set of data is loaded, then you use all the time the same set of data to make your tests. At the end the original data are restored.
Very powerfull
HTH