We Keep Coding

How to Parse BlobColumn into Words, while removing spaces and carriage returns, in SSIS? [duplicate]

Struggling with a C# Component. What I am trying to do is take a column that is ntext in my input source which is delimited with pipes, and then write the array to a text file. When I run my component my output looks like this:
DealerID,StockNumber,Option
161552,P1427,Microsoft.SqlServer.Dts.Pipeline.BlobColumn
Ive been working with the GetBlobData method and im struggling with it. Any help with be greatly appreciated! Here is the full script:
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
string vehicleoptionsdelimited = Row.Options.ToString();
//string OptionBlob = Row.Options.GetBlobData(int ;
//string vehicleoptionsdelimited = System.Text.Encoding.GetEncoding(Row.Options.ColumnInfo.CodePage).GetChars(OptionBlob);
string[] option = vehicleoptionsdelimited.Split('|');
string path = #"C:\Users\User\Desktop\Local_DS_CSVs\";
string[] headerline =
{
"DealerID" + "," + "StockNumber" + "," + "Option"
};
System.IO.File.WriteAllLines(path + "OptionInput.txt", headerline);
using (System.IO.StreamWriter file = new System.IO.StreamWriter(path + "OptionInput.txt", true))
{
foreach (string s in option)
{
file.WriteLine(Row.DealerID.ToString() + "," + Row.StockNumber.ToString() + "," + s);
}
}

Try using
BlobToString(Row.Options)
using this function:
private string BlobToString(BlobColumn blob)
{
string result = "";
try
{
if (blob != null)
{
result = System.Text.Encoding.Unicode.GetString(blob.GetBlobData(0, Convert.ToInt32(blob.Length)));
}
}
catch (Exception ex)
{
result = ex.Message;
}
return result;
}
Adapted from:
http://mscrmtech.com/201001257/converting-microsoftsqlserverdtspipelineblobcolumn-to-string-in-ssis-using-c

Another very easy solution to this problem, because it is a total PITA, is to route the error output to a derived column component and cast your blob data to a to a STR or WSTR as a new column.
Route the output of that to your script component and the data will come in as an additional column on the pipeline ready for you to parse.
This will probably only work if your data is less than 8000 characters long.

Finding ® in a string of text

Let me rephrase my question:
I am reading in text where one of the characters is the registered symbol, ®, from a text file that has no problem displaying the symbol. When I try to print the string after reading it from the file, the symbol is an unprintable character. When I read in the string and split the string to characters and convert the character to an Int16 and print out the hex, I get 0xFFFD. I specify Encoding.UTF8 when I open the StreamReader.
Here is what I have
using (System.IO.StreamReader sr = new System.IO.StreamReader(HttpContext.Current.Server.MapPath("~/App_Code/Hormel") + "/nutrition_data.txt", System.Text.Encoding.UTF8))
{
string line;
while((line = sr.ReadLine()) != null)
{
//after spliting the file on '~'
items[i] = scrubData(utf8.GetString(utf8.GetBytes(items[i].ToCharArray())));
//items[i] = scrubData(items[i]); //original
}
}
Here is the scrubData function
private String scrubData(string data)
{
string newStr = String.Empty;
try
{
if (data.Contains("HORMEL"))
{
string[] s = data.Split(' ');
foreach(string str in s)
{
if (str.Contains("HORMEL"))
{
char[] ch = str.ToCharArray();
for(int i=0; i<ch.Length; i++)
{
EventLogProvider.LogInformation("LoadNutritionInfoTask", "Test", ch[i] + " = " + String.Format("{0:X}", Convert.ToInt16(ch[i])));
}
}
}
}
return String.Empty;
}
catch (Exception ex)
{
EventLogProvider.LogInformation("LoadNutritionInfoTask", "ScrubData", ex.Message);
return data;
}
}
I'm not concerned with what is being returned right now, I am printing out the characters and the hex codes that correspond to them.

First, you need to make sure you're reading the text with the correct encoding. It appears to me that you are using UTF-8, since you say ® (Unicode code point U+00AE) is 0xC2AE, which is the same as UTF-8. You can use that like:
Encoding.UTF8.GetString(new byte[] { 0xc2, 0xae }) // "®", the registered symbol
// or
using (var streamReader = new StreamReader(file, Encoding.UTF8))
Once you've got it as a string in C#, you should use HttpUtility.HtmlEncode to encode it as HTML. E.g.
HttpUtility.HtmlEncode("SomeStuff®") // result is "SomeStuff®"

Check encoding you are decoding bytes with.

Try this:
string txt = "textwithsymbol";
string html = "<html></html>";
txt = txt.Replace("\u00ae", html);
Obviously you would replace the txt variable with the text you have read in and "\u00ae" is the symbol you are looking for.

c# Searching for a string that contains quotes

I am fairly new to c# and am working on a little project but got stuck on this. I have a file that contains some assembly code. I want my program to search this file for a string, actually a value right after my string. One of the strings i am searching for is:
setproperty QName(PackageNamespace(""), "font")
getlocal 4
pushint
My search code is this:
private void searchFile(String searchText)
{
System.IO.StreamReader reader = new System.IO.StreamReader(file);
String text = reader.ReadToEnd();
if (Regex.IsMatch(text, searchText))
{
MessageBox.Show(searchText + " was found in the given file", "Finally!!");
}
else
{
MessageBox.Show("Sorry, but " + searchText + " could not be found in the given file", "No Results");
}
}
//when i click a button//
searchFile(#"setproperty QName(PackageNamespace(""""), ""font"")
getlocal 4
pushint ");
I know that the string is in the file but the result comes up with not found. I don't know if it is the quotes or tabs or both that is causing this.
Here is part of the file:
getlocal 4
pushstring "Verdana"
setproperty QName(PackageNamespace(""), "font")
getlocal 4
pushint 16764170
setproperty QName(PackageNamespace(""), "color")
getlocal 4
pushbyte 12
setproperty QName(PackageNamespace(""), "size")
My second question is how can i get the value of the first int after my search result?
Thanks in advance.
-Leen

You should change your method like this:
private static string searchFile(String searchText)
{
System.IO.StreamReader reader = new System.IO.StreamReader("test.txt");
String text = reader.ReadToEnd();
int poz = text.IndexOf(searchText);
if (poz >= 0)
{
int start = poz + searchText.Length;
int end = text.IndexOf("\n", start);
Console.WriteLine(searchText + " was found in the given file", "Finally!!");
return text.Substring(start, end - start);
}
else
{
Console.WriteLine("Sorry, but " + searchText + " could not be found in the given file", "No Results");
return string.Empty;
}
}
The call:
string val = searchFile("setproperty QName(PackageNamespace(\"\"), \"font\")\r\n\r\n getlocal 4\r\n pushint ");

So I think you may be use to VB.net. C-based languages (like c#) used the backslash character "\" as an escape character.
So in a searching for a double-quote in a string you would need to escape it using \".
I believe what you're looking for is:
searchFile(#"setproperty QName(PackageNamespace(\"\"), \"font\")
getlocal 4
pushint ");
But this isn't really a regular expression, which is what the Regex class is meant for. So I would (well not really, I would clean it up a bit, like not mix my UI and bizlogic) do this:
// Added String as the function type so you can return the matched "Integer" as a string, you could always do a Int32.TryParse(...)
private String searchFile(String file, String searchText)
{
System.IO.StreamReader reader = new System.IO.StreamReader(file);
String text = reader.ReadToEnd();
int32 index = text.IndexOf(searchText);
if (index >= 0) //We could find it at the very beginning
{
MessageBox.Show(searchText + " was found in the given file", "Finally!!");
int32 start = index + searchText.Length;
int32 end = Regex.Match(text, "[\n\r\t]", index).Index; // This will search for whitespace
String value = text.Substring(start, end - start);
// Now you can do something with your value, like...
return value;
}
else
{
MessageBox.Show("Sorry, but " + searchText + " could not be found in the given file", "No Results");
return "";
}
}

How to find dd/mm/yyyy date on string using c#?

I have a file on which I want to replace the dates but I don't know what dates are on it and I want to do generic code for me to use the same executable for other files.
I put all content of the file in a string and I want to replace all dates with format dd/mm/yyyy (ex: 19/12/2011) with the actual date (20/12/2011).
How can I do that?
Code so far:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
namespace ReplaceDates
{
class Program
{
static int Main(string[] args)
{
string FileIn = retreiveArgument(args, "i");
int AddDays = Int32.Parse(retreiveArgument(args, "d"));
string date = (System.DateTime.Now).AddDays(AddDays).ToString("dd/MM/yyyy");
string content = "", date2replace = "";
if (File.Exists(FileIn))
{
File.Copy(FileIn, FileIn + ".bkp", true);
try
{
content = File.ReadAllText(FileIn);
// here is what I need to do
}
catch (Exception)
{
Console.WriteLine("Error replacing dates in " + FileIn + ".");
return 0;
}
try
{
File.WriteAllText(FileIn, content);
Console.WriteLine("Dates replaced in " + FileIn + ".");
return 0;
}
catch (Exception)
{
Console.WriteLine("Couldn't write the file " + FileIn);
return 2;
}
}
else
{
Console.WriteLine("File " + FileIn + " does not exist.");
return 1;
}
}
private static string retreiveArgument(string[] argument, string argumentName)
{
for (int i = 0; i < argument.Length; i++)
{
if (argument[i].ToLower().Equals("-h") || argument[i].ToLower().Equals("help") || argument[i].ToLower().Equals("-help") || argument[i].Equals("?") || argument[i].Equals("-?"))
{
Console.WriteLine("Usage : ");
Console.WriteLine("ReplaceDates.exe -i [Input File] -d [Addition]");
Console.WriteLine("[Input File] -> Complete path to the file.");
Console.WriteLine("[Addition] -> Adds the specified value in days to the actual date.");
Console.WriteLine();
Console.WriteLine("This executable replaces the data from the input file.");
}
else
{
if (argument[i].Equals("-" + argumentName))
{
return argument[i + 1].Trim();
}
}
}
return "";
}
}
}

Here's a solution using a MatchEvaluator. Tweak the regular expression if desired;
string UpdateDates(string input)
{
return Regex.Replace(input, #"\d{2}/\d{2}/\d{4}", m => DateTime.Now.ToString("dd/MM/yyyy"));
}

you can use a regex for that http://www.regular-expressions.info/dates.html

Use another file that contains the layout of the original file but in the dates that you need to replace put another string, something like "DD/MM/YYYY".
Now, you simply need to open the layout file, read all content, replace the "DD/MM/YYYY" for the date you want and voilá :)

Extracting text from a file where date -time is the index

I have got around 800 files of maximum 55KB-100KB each where the data is in this format
Date,Time,Float1,Float2,Float3,Float4,Integer
Date is in DD/MM/YYYY format and Time is in the format of HH:MM
Here the date ranges from say 1st May to 1June and each day, the Time varies from 09:00 to 15:30.
I want to run a program so that, for each file, it extracts the data pertaining to a particular given date and writes to a file.
I am trying to get around, to form a to do a search and extract operation. I dont know, how to do it, would like to have some idea.
I have written the code below:
static void Main(string[] args)
{
string destpath = Directory.GetCurrentDirectory();
destpath += "\\DIR";
DirectoryInfo Dest = Directory.CreateDirectory(destpath);
DirectoryInfo Source = new DirectoryInfo(Directory.GetCurrentDirectory() + "\\IEOD");
FileInfo[] fiArr = Source.GetFiles("*.csv");
Console.WriteLine("Search Date:");
string srchdate = Console.ReadLine();
String FileNewLine;
String FileNewdt;
FileInfo r;
foreach (FileInfo f in fiArr)
{
r = new FileInfo(destpath + "\\" + f.Name);
r.Create();
StreamWriter Sw = r.AppendText();
StreamReader Sr = new StreamReader(f.FullName);
while (Sr.Peek() >= 0)
{
FileNewLine = Sr.ReadLine();
FileNewdt = FileNewLine.Substring(0,10);
if (String.Compare(FileNewdt, srchdate, true) == 0)
{
//write it to a file;
Console.WriteLine(FileNewLine);
}
}
}
Console.ReadKey();
}
As of now, it should write into the Console. The writing with the help of StreamWriter will be done later, but I am facing a runtime error. It says, " 'C:\Documents and Settings\Soham Das\Desktop\Test\DIR\ABAN.csv' because it is being used by another process."
Here ABAN is a newly created file, by the code. The problem is faced at StreamWriter Sw = r.AppendText()
Help appreciated.
Thanks
Soham

Now that you have edited the question to show that the delimiter is actually a comma instead of a slash (which would have conflicted with the date format) this becomes a lot easier. I've re-posted the answer from last night below.
// This would come from Stream.ReadLine() or something
string line = "02/06/2010,10:05,1.0,2.0,3.0,4.0,5";
string[] parts = line.Split(',');
DateTime date = DateTime.ParseExact(parts[0], "dd/MM/yyyy", null);
TimeSpan time = TimeSpan.Parse(parts[1]);
date = date.Add(time); // adds the time to the date
float float1 = Single.Parse(parts[2]);
float float2 = Single.Parse(parts[3]);
float float3 = Single.Parse(parts[4]);
float float4 = Single.Parse(parts[5]);
int integer = Int32.Parse(parts[6]);
Console.WriteLine("Date: {0:d}", date);
Console.WriteLine("Time: {0:t}", date);
Console.WriteLine("Float1: {0}", float1);
Console.WriteLine("Float2: {0}", float2);
Console.WriteLine("Float3: {0}", float3);
Console.WriteLine("Float4: {0}", float4);
Console.WriteLine("Integer: {0}", integer);
Obviously you can make it more resilient by adding error handling, using TryParse, etc. But this should give you a basic idea of how to manipulate strings in .NET.

So 800 files with around 100KB sums up to 80 KBytes. So why don't built up a little class like
public class Entry
{
public DateTime Date {get; set;}
public float Float1 {get; set;}
public int Integer1 {get; set;}
public Entry(string values)
{
//ToDo: Parse single line into properties
// e.g. use String.Split, RegEx, etc.
}
}
Also you should take care about implementing GetHashCode() and Equals() (there is a good explanation in the book Essential C#). And you should add the interface IComparable to that class which just makes somethine like
public int CompareTo(Entry rhs)
{
return this.Date.CompareTo(rhs.Date);
}
If you got this you can easily do the following:
var allEntries = new SortedList<Entry>();
string currentLine = null;
using (var streamReader = new StreamReader("C:\\MyFile.txt"))
while ((currentLine = streamReader.ReadLine()) != null)
{
try
{
var entry = new Entry(currentLine);
allEntries.Add(entry);
}
catch (Exception ex)
{
//Do whatever you like
//maybe just
continue;
//or
throw;
}
}
So what's missing is to read in all the files (instead of a single one). But this can be done by another loop on Directory.GetFiles() which maybe itself is looped through a Directory.GetDirectories().
After reading all the files into your List you can do whatever LINQ query comes to your mind.