Use Log Parser to bulk insert IIS Logs into SQL Server - c#

I have a lot of IIS logs that need to be inserted into a SQL Server database. After some research it looks like Log Parser is the way to go but I am having a hard time figuring out how to automate the process. We have hundreds of these files, some of which are up to 3Gb in size.
The console command to insert a file is
"C:\Program Files (x86)\Log Parser 2.2\LogParser.exe" "SELECT * INTO LogTable FROM C:\Logs\ex1234.log" -i:iisw3c -o:SQL -server:ServerName -database:DatabaseName -driver:"SQL Server" -createTable:ON
In order to iterate through the directories and files I created a small console app in which the main method enumerates through the directories and then sends each file to a method that runs the Log Parser:
static void RunLogParser(string fileName)
{
var logParserPath = #"""C:\\Program Files (x86)\\Log Parser 2.2\\LogParser.exe""";
var query = $"\"SELECT * INTO LogTable FROM {fileName}\"";
var startInfo = new ProcessStartInfo
{
FileName = "cmd.exe",
Arguments = logParserPath + query + "-i:iisw3c -o:SQL -server:ServerName -database:DatabaseName -driver:\"SQL Server\" -createTable:ON",
CreateNoWindow = true,
UseShellExecute=false
};
try
{
using (var process = Process.Start(startInfo))
{
process.WaitForExit();
}
}
catch(Exception ex)
{
throw new Exception(ex.Message);
}
Console.ReadKey();
}
When I run this nothing seems to happen. Executing the command from the console takes about 20 seconds to insert a test file, but when I run this same command from the code nothing.
I have also tried using Log Parser as the FileName arg in the ProcessStartInfo but when the process executes all that happens is that the command window seems to display the arguments for the Log Parser command with an error message but I can't the window closes too fast to read the message.
Edit: Solved
It seems that I had a typo, much to my chagrin. I changed the ProcessStartInfo filename to the LogParser.exe from cmd.exe, added a space after the select statement and changed the args to a literal string and double quoted the driver argument.
static void RunLogParser(string fileName)
{
var logParserPath = #"""C:\\Program Files (x86)\\Log Parser 2.2\\LogParser.exe""";
var query = $"\"SELECT * INTO LogTable FROM {fileName}\"";
var startInfo = new ProcessStartInfo
{
FileName = logParserPath ,
Arguments = query + #" -i:iisw3c -o:SQL -server:ServerName -database:DatabaseName -driver:""SQL Server"" -createTable:ON",
CreateNoWindow = true,
UseShellExecute=false
};
try
{
using (var process = Process.Start(startInfo))
{
process.WaitForExit();
}
}
catch(Exception ex)
{
throw new Exception(ex.Message);
}
}

Related

Using Command line application with my asp.net application

I doing my final year project by implementing learning web application using ASP.NET Core
however, in some page i need the user to interact with some command line application where these commands need to be sent based on the given output from the command line application. I have no issue with sending on command, but when it comes to the second command i got several issues. here is the code
public static Process process = new Process()
{
StartInfo = new ProcessStartInfo
FileName = "cmd",
RedirectStandardOutput = true,
RedirectStandardInput = true,
UseShellExecute = false,
CreateNoWindow = true,
}
};
This is my definition for my process i have created outside the function because i do need to use the same process several time until the user leave the page.
For the process.Start(); I have putted with the function of viewing the page, so the process is started already and no issue with it.
public IActionResult acceptingInputFromUser(string input)
{
string result;
try
{
StreamWriter sw = process.StandardInput;
StreamReader rd = process.StandardOutput;
using (process.StandardInput)
{
sw.WriteLine(input);
sw.Flush();
sw.Close();
result = process.StandardOutput.ReadToEnd();
}
}catch (Exception ex)
{
result = ex.ToString();
}
return Json(result);
}
Here where is the issue, once the user submit any input the the first time will accept and give the result, however when he send another request it says
Cannot write to a closed TextWriter
I have tried several ways but so far i couldn't find solution. Can anyone help me with this error like how I can open the Textwriter again or if there is any other efficient way to where I can use CMD freely.

text file not updating in the run problem c#

I have the following code:
strCmdText = "/C cppcheck.exe --enable=all " + StrFile1 + " 2> " + StrfolderOut + "\\fileOut.txt";
//Console.WriteLine(strCmdText);
System.Diagnostics.Process.Start("CMD.exe", strCmdText);
but when I'm trying to read it afterwards is like the program executes it after finishing the execution
I ran 2 test:
First one I use as an input "file1.txt" (fileOut does not exist) the file is created by cmd redirecting the output using ">". When trying to read it it says "the file does not exist"
Secondly, I use "file2.txt" as an input (fileOut does existe and has the info of file2.txt) and it reads but the content of "file1.txt".
So I think that the cmd command runs after finishing the execution. I triple checked the paths so is not a problem since im reading the path using file folder gui.
I'm using this code to read
public List<Error> LeerArchivo()
{
List<Error> listaErrores = new List<Error>();
string #folderLocation = StrfolderOut + "\\fileOut.txt";
Console.WriteLine(#folderLocation);
//OpenFileDialog openFileDialog = new OpenFileDialog();
//openFileDialog.ShowDialog();
if (File.Exists(#folderLocation))
{
try
{
// Create an instance of StreamReader to read from a file.
// The using statement also closes the StreamReader.
using (StreamReader sr = new StreamReader(#folderLocation))
{
string line = "";
// Read and display lines from the file until the end of
// the file is reached.
Console.WriteLine("reading");
while ((line = sr.ReadLine()) != null)
{
Console.WriteLine(line);
}
Console.WriteLine(line);
}
}
catch (Exception e)
{
// Let the user know what went wrong.
Console.WriteLine("The file could not be read:");
Console.WriteLine(e.Message);
}
}
else
{
Console.WriteLine("no existe");
}
return listaErrores;
}
I manage it to work, I used:
Process process = new Process();
process.StartInfo.FileName = "CMD.exe";
{do cppcheck... etc}
process.Start();
process.WaitForExit();
It works now, I think it is since the main program waits for that process to finish. Maybe I can have a deeper explanation of what happening.

Using C# fetch Registered time of file (dll,exe,ocx)

I am installing an application, post installation i am performing few verification. One of which is to check the registered time and other is to check if the expected files list is registered.
I have list of files which should get registered and i have written code to verify that. But somehow i am not able to find a way to get the registered time[Date and Time] of the registered file.
Below is the code i wrote for fetching complete list of registry file and then run a loop of expected files on the obtained files to check if they are present or not.
static void Main(string[] args)
{
string keyPath = #"SOFTWARE\Classes";
//string keyPath = "InprocServer32";
RegistryKey topRegKey = RegistryKey.OpenBaseKey(RegistryHive.LocalMachine, RegistryView.Default).OpenSubKey(keyPath);
GetAllKeys(topRegKey);
int b = result.FindIndex(p => p.ToLower().Contains("MSCOMCT2.OCX".ToLower()));
}
public List<string> RegKeysList = new List<string>();
public static void GetAllKeys(RegistryKey regKey)
{
if (regKey != null)
{
foreach (string key in regKey.GetSubKeyNames())
{
GetAllKeys(regKey.OpenSubKey(key));
}
if (regKey.GetSubKeyNames().Length == 0)
{
try
{
result.Add(regKey.GetValue(string.Empty).ToString());
}
catch (Exception)
{
}
}
}
}
I was able to find a C++ function RegQueryInfoKey function - ftLastWriteTime
to perform it(Didn't explore more on this). But i want to keep that as my last option.
Is there a way using C# to perform this. Am i missing something?
I ran RegDllView through command prompt and generated a file with all registered files using C#. Which had all registered files list with version,date,etc.
Through C# i copied RegDllView.exe to my server machine.
Followed by fetching required files list from the file. PFB working code:
System.Diagnostics.Process process = new System.Diagnostics.Process();
System.Diagnostics.ProcessStartInfo startInfo = new System.Diagnostics.ProcessStartInfo();
startInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;
startInfo.FileName = "cmd.exe";
startInfo.Arguments = "/C " + RegDllViewExePath + " /scomma " + RegDllOutputFilePath;
process.StartInfo = startInfo;
process.Start();

Calling WSL bash.exe from C#

Mostly just as a curiosity, I wrote a little app to start up Terminator shell on Windows, using Ubuntu/WSL and Xming window server.
Doing things manually from the shell, I can run Firefox, gedit, Terminator, etc on Windows, it's pretty cool.
So I checked the location of bash.exe using where bash and it returned...
C:\Windows\System32\bash.exe
However when I tried to run this code...
using (var xminProc = new Process())
{
xminProc.StartInfo.FileName = #"C:\Program Files (x86)\Xming\Xming.exe";
xminProc.StartInfo.Arguments = ":0 -clipboard -multiwindow";
xminProc.StartInfo.CreateNoWindow = true;
xminProc.Start();
}
using (var bashProc = new Process())
{
bashProc.StartInfo.FileName = #"C:\Windows\System32\bash.exe";
bashProc.StartInfo.Arguments = "-c \"export DISPLAY=:0; terminator; \"";
bashProc.StartInfo.CreateNoWindow = true;
bashProc.Start();
}
I get the error...
System.ComponentModel.Win32Exception: 'The system cannot find the file specified'
And checking my entire system for bash.exe reveals it really be in another place altogether...
I'm not sure if this location is one that I can rely on, I'm worried it's ephemeral and can change during a Windows Store update, although I may be wrong about that.
Why does the command prompt show bash.exe to be in System32 but it's really in another location altogether?
Can I get C# to also use the System32 location?
As #Biswapriyo stated first set the platafrom to x64 on your solution:
Then you may run on your ubuntu machine from c# as:
Console.WriteLine("Enter command to execute on your Ubuntu GNU/Linux");
var commandToExecute = Console.ReadLine();
// if command is null use 'ifconfig' for demo purposes
if (string.IsNullOrWhiteSpace(commandToExecute))
{
commandToExecute = "ifconfig";
}
// Execute wsl command:
using (var proc = new Process
{
StartInfo = new ProcessStartInfo
{
FileName = #"cmd.exe",
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardInput = true,
CreateNoWindow = true,
}
})
{
proc.Start();
proc.StandardInput.WriteLine("wsl " + commandToExecute);
System.Threading.Thread.Sleep(500); // give some time for command to execute
proc.StandardInput.Flush();
proc.StandardInput.Close();
proc.WaitForExit(5000); // wait up to 5 seconds for command to execute
Console.WriteLine(proc.StandardOutput.ReadToEnd());
Console.ReadLine();
}

Retrieve cmd prompt output from PsExec command?

Use case: I am checking certain credentials on a remote system by running commands via PsExec (i.e. for this example, I am trying to retrieve the KB articles currently installed on the remote system).
I have the following to retrieve command output:
public string GetCmDOutput(string cmd)
ProcessStartInfo startInfo = new ProcessStartInfo("control", cmd)
{
WindowStyle = ProcessWindowStyle.Hidden,
UseShellExecute = false,
RedirectStandardOutput = true,
CreateNoWindow = true
};
string output = string.Empty;
Process process = Process.Start(startInfo);
process.OutputDataReceived += (sender, e) => output = string.Concat(output, e.Data);
process.BeginOutputReadLine();
process.Start();
process.WaitForExit();
Delay.Milliseconds(1500) //API-specific delay
return output;
}
Whenever I use GetCmdOutput() to run a command locally it works like a charm, but if I try to run a command with PsExec, my output is empty.
For instance, I ran the following:
string cmd = #"\psexec.exe \\remoteComputerName -u username -p password -c cmd /c wmic qfe";
GetCmdOutput(cmd);
Report.Info(cmd); //API-specific reporting
And an empty string was returned.
After playing around with this for a couple of hours, I feel I may need a second set of eyes. What might be causing this issue?
I have run into this same problem. My solution was to run cmd and have it call psexec. I have psexec's output saved to a temp file for further manipulation. My code is returning a List.
public List<string> ExecutePSExec(string hostname)
{
List<string> recordNames = new List<string>();
string command = #"\\path\to\psexec.exe /accepteula \\" + hostname + ". exe-to-run-remotely";
try
{
string location = AppDomain.CurrentDirectory.BaseDirectory;
string cmdWithFileOutput = string.Format("{0} >{1}temp.log", command, location);
procStartInfo.UseShellExecute = true;
procStartInfo.CreateNoWindow = true;
procStartInfo.WindowStyle = ProcessWindowStyle.Hidden;
Process proc = new Process();
proc.StartInfo = procStartInfo;
proc.Start();
proc.WaitForExit();
// Read file contents, manipulate data and then delete temp file here
}
catch (Exception e)
{
Console.WriteLine("Failure to run psexec: {0}", e.Message);
}
return recordNames;
}
NOTE: I ran into another problem and found out that running psexec this way requires the remote hostname (not IP Address) in the command to end in a period \\" + hostname + ".
This code assumes you can run psexec on the remote machine as your current user.

Categories