I don't agree that this question was answered effectively: decode mysqlbinlog in C#.
I have, what I think is the same question: I want to read the MySql binlogs from within a c# application, but do not know the format of the file. How can I properly parse these files' data?
First, what I learned:
Most of the source code files for MySql are installed along with the assemblies and generally located in [basedir]\include. For example, a typical install would place the files in Program Files\MySql\MySql 5.6\include.
mysqlbin.cc was NOT in that folder. However, I was easily able to get the file by doing a quick Google search. The file can be found here: https://code.google.com/p/mg-common-utils/source/browse/trunk/myreplicator/src/mysqlbinlog.cc?r=4 . It is well documented and easy to read.
Second, my solution:
As akuzminsky pointed out, the format of MySql's binlog is subject to change. However, the format returned from the mysqlbinlog.exe utility is consistent. This application is typically included in a MySql install and should be located in [basedir]\bin. I now run this application from within a c# Console Application and parse the results. I used the following steps to accomplish this:
Enabled binlogging on the MySql server from within the options file. In MySql Workbench, check 'log-bin' under the logging tab. Or, type 'log-bin=' in the settings file (often located in [basedir]. Might be called 'my.ini' or 'my.cnf' or something else. Generally, with .cnf or .ini extension). A filename is not required. When one is not specified, MySql will automatically create filenames for the logs. However, review MySql documentation on possible issues with this.
From within my client application, I query the server to get the path of each binary log (there could be many). To do this:
query show global variables like 'datadir' //returns the data directory.
query show binary logs //returns the filename of each binary log, along with its file size (helpful for reading).
Parsing these together gets the path for each binary log.
Since mysqlbinlog.exe is located in [basedir]\bin, I query the server to get the path of the base directory:
query show global variables like 'basedir'
Then, I parse the result with '\bin\mysqlbinlog.exe'
I use the Process class to create a new process, execute each binary log using mysqlbinlog.exe, and read each files results into a string variable:
private static string GetLogTexts(Liststring> logfilenames)
{
List<string> _logtexts = new List<string>();
string _basedir = GetBaseDir();
foreach(string logfilename in logfilenames)
{
Process proc = new Process();
proc.StartInfo.FileName = _basedir + "\\bin\\mysqlbinlog";
proc.StartInfo.Arguments = string.Format("\"{0}\"", logfile);
proc.StartInfo.UseShellExecute = false;
proc.StartInfo.RedirectStandardInput = proc.StartInfo.RedirectStandardOutput = true;
proc.Start();
_logtexts.Add(proc.StandardOutput.ReadToEnd());
}
return _logtexts;
}
private static string GetBaseDir()
{
string path = "";
using (MySqlConnection conn = new MySqlConnection(RemoteServerConnectionString))
{
conn.Open();
using (MySqlCommand cmd1 = new MySqlCommand("show global variables like 'basedir'", conn))
{
using (MySqlDataReader reader = cmd1.ExecuteReader())
{
while (reader.Read())
{
path = reader.GetString(1);
}
}
}
}
return path;
}
Finally, I parse the results using my own logic (specific for what I am looking for). The results are very easy to read: mysqlbinlog uses regular line breaks and statements are terminated by a delimiter, which is defined prior to the statement (as usual, there can be multiple delimiters).
I hope this helps someone!
Related
Let's say I have a string. " db.getCollection("somecollection").find({})". Can I execute this string as a query in C#? i.e. I get a string. And I just execute it as a query but in c#
I just want like this
string query = "db.getCollection("somename")";
Mongo.execute(query);
no, the best you can do in this context is to use db.RunCommand<BsonDocument>("{ ping : 1 }") (c#) which is close to the shell db.runCommand({ ping : 1 })
UPDATE:
you may look at this as well How to execute mongo commands through shell scripts?, I'm not familiar with this and it doesn't work for me on windows and 5.0 server in most of the cases mentioned there other than simple one: mongo --eval "printjson(db.serverStatus())", but if you will be able to make this suggested script mongo < script.js (or similar) work in, for example, the shell, you will be able to put your random query in this file(script.js) and then, add this file as argument into Process creating similar to:
using (var process = new Process())
{
// arguments below may require the whole path to the files
process.StartInfo.Arguments = "script.js";
process.StartInfo.FileName = "mongo";
process.Start();
}
to read results, you will need analyzing process.StandardOutput/process.StandardError stream.
I have a scenario where i would like to compress my folder due to presence of large number of files present in them using SSIS 2008. Consider it like i have one Source Folder and one Target Folder and while moving files from "SRC" to "TGT" the folder must be compressed in destination.Now feasible option for doing this i think is SSIS Script task ,since I cannot use Execute Process task due to restriction of using any third party software like 7z/Winrar etc.But i am not able to implement this even after using SSIS Script Component.Tried many online solutions but it did not work.How can i implement such thing using SSIS 2008?
You can use the ZipPackage class if you are targeting .Net 3 and above. Complete example here:
https://msdn.microsoft.com/en-us/library/system.io.packaging.zippackage.aspx
There is also a ZipArchive class, example here:
https://msdn.microsoft.com/en-us/library/system.io.compression.ziparchive(v=vs.110).aspx
I did this exercise , I created a Script Task to perform compression of a folder by using the compression provided by Windows;
the folder name can dynamically change.
In this way it is not necessary to use third party software like 7z/Winrar etc..
You need to provide to the Script Task the folder to be zipped and the name of the compressed folder as ReadOnlyVariables (to be added in the tab ReadOnlyVariables)
These two variables must be defined in the Variables tab (String type) of the package and can be changed dynamically through a cycle (eg. for each)
I use these two variables:
sFolderCompressed - the folder '.zip' that you want to obtain eg. \\XX.XX.XX.XX\C$\.....\folderCompressed
sFolderSource - the source folder containing the files affected eg. \\XX.XX.XX.XX\C$\.....\folderSource
(*)
The script is made using c#, choose Script Language: Microsoft Visual C# 2008
This is the code to be added in the Main method:
public void Main()
{
// TODO: Add your code here
try
{
// variables used in process
string l_sFolderCompressed = (string)Dts.Variables["User::sFolderCompressed"].Value;
string l_sFolderSource = (string)Dts.Variables["User::sFolderSource"].Value;
string l_sCommand = "zip -j " + l_sFolderCompressed + " " + l_sFolderSource + "/*";
// create the ProcessStartInfo using "cmd" as the program to be run,
// and "/C " as the parameters.
// Incidentally, /C tells cmd that we want it to execute the command that follows,
// and then exit.
System.Diagnostics.ProcessStartInfo procStartInfo = new System.Diagnostics.ProcessStartInfo("cmd", "/C " + l_sCommand);
// The following commands are needed to redirect the standard output.
// This means that it will be redirected to the Process.StandardOutput StreamReader.
procStartInfo.RedirectStandardOutput = true;
procStartInfo.UseShellExecute = false;
// Do not create the black window.
procStartInfo.CreateNoWindow = true;
// Now we create a process, assign its ProcessStartInfo and start it
System.Diagnostics.Process proc = new System.Diagnostics.Process();
proc.StartInfo = procStartInfo;
proc.Start();
// Get the output into a string
string result = proc.StandardOutput.ReadToEnd();
// Possibly display the command output.
}
catch (Exception objException)
{
Dts.TaskResult = (int)ScriptResults.Failure;
// Log the exception
}
Dts.TaskResult = (int)ScriptResults.Success;
}
You can also manage a single file
"cmd", "/C zip -j c:\\...\file.zip c:\\..\file.txt");
I hope can help
My sql file contains
`SPOOL &1;
//sql code to execute
SPOOL OFF;`
The sql file is executed using SQL Plus and SQL Plus is being called from C# code using Process.Start... Code snipped
`var m_StartInfo = new ProcessStartInfo();
m_StartInfo.FileName = "SQLPLUS.EXE";
m_StartInfo.CreateNoWindow = true;
m_StartInfo.UseShellExecute = false;
m_StartInfo.Arguments = String.Format("{0}\"{1}\" \"{2}\"", connectionString, sqlfile, sqlLogFileName);
m_Process = Process.Start(m_StartInfo);
Other code.....`
It works fine and the sqlplus log is created fine at sqlLogFileName location. However if the sqlLogFileName has spaces in between (say like "C:\My Application\log.txt"), then the log file is not created, instead gives the error SP2-0768 Illegal SPOOL command on SQL Plus window
Any suggestion how to resolve this? I am using Oracle 11GR2
You just have to surround your file name with double quotes.
Something like:
spool "Test with spaces.txt"
Or in your case with a parameter:
SPOOL "&1"
I think is best not to use spaces though, as #tvCa explained.
Side Note:
What StarPilot is refering to, I believe, is about the redirect a command output to a file, and that is why it didn't work when you tried to use it.
For example in command prompt you would write:
dir > dir.txt
And that saves the output of dir inside dir.txt.
Oracle software is designed to be used with directories not having whitespaces (which is an accepted standard in Linux/Unix, even though technically you can do otherwise). On Windows, things are different, but the Oracle software has the same idea : it does not like whitespaces. So, the fix is clear : spool to a directory without whitespaces. This is advice, anybody is free to take or not.
As of now I've been given a piece of software that reads XML files and presents these graphically, to give the user some overview to the events these files describe.
The crux as of now is that one of the engineers using the software has requested that all the data you load into the program should be possible to save to a database.
My pickle is as follows:
public void addToDBO(List<List<EventElement>> insertList)
{
SqlConnection s1 = new SqlConnection();
s1.ConnectionString = filePath;
s1.Open();
SqlCommand command = new SqlCommand();
command.Connection = s1;
foreach (List<EventElement> item in insertList)
---------------------SNIP----------------------------
//Adding-data-to-database logic
I mean to write the loaded data into a database deployed localy, but the the problem is that it does not accept my hamhanded methods of simply adding the database's filepath as the filepath string.
So, to summarize:
How do I correctly connect to a database situated on my C:/?
Thank you in advance.
You do not simply pass a file path. The connection string needs to follow a certain convention. For example
Provider=SQLNCLI10;Server=.\SQLExpress;AttachDbFilename=c:\asd\qwe\mydbfile.mdf;
Database=dbname;Trusted_Connection=Yes;
See http://www.connectionstrings.com/sql-server/ for more
I using C# .NET , vs 2008 , .net 3.5
For me, is difficult, but I need sample code in C# for this:
Check if a file or a folder is in use
If file or a folder is in use, the name of Process that use it
For example, in my issue.
I try delete file, and I get "The process cannot access the file 'XYZ' because it is being used by another process." Exception.
File.Delete(infoFichero.Ruta);
I want check if a file is in use, and the name of Process that use it.
I need sample code, source code, please. I dont want use c++, I dont know c, c++, unmanaged code, or WinApi. I want use only C# code (managed code .net).
I have read several references but not get sample code source,
How to check if a file is in use?
Emulate waiting on File.Open in C# when file is locked
http://social.msdn.microsoft.com/Forums/en-US/csharpgeneral/thread/9dabc172-237a-42db-850e-ada08885a5d5
How to check if a file is in use?
Easiest way to read text file which is locked by another application
Using C# is it possible to test if a lock is held on a file
EDIT:
From Yan Jun - MSFT
string path = "D:\\temp2.xlsx";
foreach (Process c in Process.GetProcesses()) {
if (c.MainWindowTitle.Contains(Path.GetFileName(path))){
MessageBox.Show(c.ProcessName);
return;
}
}
try{
FileInfo f = new FileInfo(path);
f.Delete();
}
catch (Exception ex){
MessageBox.Show(ex.Message);
}
...
But it is difficult get solution for all 100% issues.
Problem if c.MainWindowTitle == null or not contains filename.
Problem for shared folder in another machine, PC, server,... like:
File.Delete(#\desiis\TEmporal\Project\script.targets);
any sample code, I ask for help gurus, MVPs, anyone.
UPDATE: the same issue for a folder
There's not going to be a way to find the process that has the file opened without stepping into the WinApi, I don't think. And as far as checking whether its in use, the only thing you can really do, as the SO questions you linked to state, is to wrap the file access attempts in a try/catch block.
The code to find which file has it opened is likely to be ugly, but there may be an API out there that wraps this up nicely. There are 3rd party utilities that will tell you this (Unlocker being the best known example). You can also use ProcessExplorer to search for open file handles by the filename. Those don't really help you though.
The short answer of what I'm trying to get across here is you have the answer for the first part of your question in the SO questions you already linked, and the second part would probably require WIN32 calls, which you want to avoid, but you're probably going to have to get your hands dirty in Win32... Still want help?
EDIT: You could shell out to sysinternals Handle utility. You would need to get the output of that command and parse it yourself. You can read the executed process's output like this
string result = proc.StandardOutput.ReadToEnd();
The issue with this is you're going to get a license agreement popup the first time you run the Handle utility. Not to mention the whole licensing issues if this is something you hope to deploy...
If you're still interested, I can show you how you'd go about this.
EDIT: Here's a runnable program that will find the exe name and pid of any program that has an open handle to a file. I added comments, but can elaborate further if necessary. I use Regular Expressions here to parse the output as that makes the most sense given the task at hand.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.Text.RegularExpressions;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
ProcessStartInfo si = new ProcessStartInfo();
si.FileName = "handle.exe"; //name of the handle program from sysinternals
//assumes that its in the exe directory or in your path
//environment variable
//the following three lines are required to be able to read the output (StandardOutput)
//and hide the exe window.
si.RedirectStandardOutput = true;
si.WindowStyle = ProcessWindowStyle.Hidden;
si.UseShellExecute = false;
si.Arguments = "test.xlsx"; //this is the file you're trying to access that is locked
//these 4 lines create a process object, start it, then read the output to
//a new string variable "s"
Process p = new Process();
p.StartInfo = si;
p.Start();
string s = p.StandardOutput.ReadToEnd();
//this will use regular expressions to search the output for process name
//and print it out to the console window
string regex = #"^\w*\.EXE";
MatchCollection matches = Regex.Matches(s, regex, RegexOptions.Multiline);
foreach (var match in matches)
{
Console.WriteLine(match);
}
//this will use regex to search the output for the process id (pid)
//and print it to the console window.
regex = #"pid: (?<pid>[0-9]*)";
matches = Regex.Matches(s, regex, RegexOptions.Multiline);
foreach (var obj in matches)
{
Match match = (Match)obj; //i have to cast to a Match object
//to be able to get the named group out
Console.WriteLine(match.Groups["pid"].Value.ToString());
}
Console.Read();
}
}
}
There is no purely managed way to do this. You have to use some low-level APIs through P/invoke or similar.
There's good information here on a way to do it, but it's C++ code. You'd have to do the porting yourself.
http://www.codeproject.com/KB/shell/OpenedFileFinder.aspx
Note there are some complex issues with this, namely the issues around kernel vs. userspace memory. This is not a simple problem you're trying to solve.
Try the windows Process Explorer:
http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx
Won't let you do it from code, but at least you can figure out what the source of your locks are.