SSIS remove carriage returns - c#

I've a flat file which has 48 columns. The columns are tab delimited and the rows CR-LF (return).
Now I've the problem that there is a column which sometimes contains carriage returns and there is no possibility to change the file before the import process.
At the moment I wrote a C# script task which looks in every row, counts the tabs and when there is a return and the counted tabs are not divisible by 48 it deletes the return. This way works but it's to slow because my files are very big and with that way I've to read every character in the file.
Does someone knows a better way to get rid of these carriage returns?
Cheers!

https://community.spiceworks.com/topic/2130093-ssis-package-flat-file-destination-blank-row-at-end-of-file
public void Main()
{
string filename = #"C:\Temp\Gerard.txt";
string fileinfo = "";
string curline = "";
TextReader tr = new StreamReader(filename);
while ((curline = tr.ReadLine()) != null)
{
if (fileinfo != "")
{
fileinfo = fileinfo + Environment.NewLine;
}
fileinfo = fileinfo + curline;
}
tr.Close();
TextWriter tw = new StreamWriter(filename, false);
tw.Write(fileinfo);
tw.Close();
Dts.TaskResult = (int)ScriptResults.Success;
}

Related

C# unique file naming with String from Streamreader

ok i have solved my problem of finding a unique word within the file that is then used as the newly created .txt file name.
for example: current.txt files have 200 lines of words/data per file but one of the words is unique("92222225") with every current.txt file.
so the newly created output files from streamwriter becomes 92222225.txt, 933333334.txt and so on.
the whole time i though what i needed was within streamreader or streamwriter.
but what i need to add to the two was "Regex.Match".
here is the code i figured out to use for pulling strings out of a .txt file to use as a name for the output files. also to add other words to the new output file.
string mydocpath = Environment.GetFolderPath(Environment.SpecialFolder.Desktop) + #"\reporting";
foreach (string txtName in Directory.EnumerateFiles(mydocpath, "*.txt"))
{
StringBuilder sb = new StringBuilder();
StreamReader sr = new StreamReader(txtName);
string content = sr.ReadToEnd();
sb.AppendLine(txtName.ToString());
sb.AppendLine("= = = = = =");
sb.Append(content);
if (content.Contains("helloworld"))
{
sb.AppendLine();
sb.AppendLine("byeworld");
}
sb.AppendLine();
sb.AppendLine();
//string fileName = content.Contains("helloworld").ToString();
string FindMatch = content;
Match match = Regex.Match(FindMatch, #"9(([A-Za-z0-9\-])\d+)");
if (match.Success)
{
//this is what adds unique word as 922225.txt file name.
string capture = match.Groups[1].Captures[0].Value;
using (StreamWriter outfile = new StreamWriter(mydocpath + #"\" + capture + ".txt"))
{
outfile.Write(sb.ToString());
}
}
}
i updated this whole post so if anyone else may need this.
never even used Regex.Match before, nor knew about it or maybe i forgot about it.
If you want to name your output file use:
using (StreamWriter outfile = new StreamWriter(mydocpath + #"\901232lOi.txt"))

Read last empty line of a text file

I have funny problem - I tried several scripts that will read text files, and that's ok.
Problem occur when text file have empty line at the end - that line is "ignored".
Code I use is "usual" code for file read, like next one:
string fullFileName;
fullFileName = "myFile.txt";
var lines = File.ReadAllLines(fullFileName);
string fileContent = null;
bool firstLine = true;
foreach (var line in lines) {
if (firstLine != true)
{
//textBox1.Text += System.Environment.NewLine;
fileContent += System.Environment.NewLine;
}
else
{
firstLine = false;
}
//textBox1.Text += line;
fileContent += line;
}
textBox1.Text = fileContent;
So, if last line of file myFile.txt is empty, it is not showed in a TextBox.
Can you help me where is a problem?
I think you could avoid the loop altogether and just do:
textBox1.Text = File.ReadAllText(fullFileName);
This will preserve all the newlines.
It is a problem with the file representation, not with ReadAllLines.
See this thread: http://www.pcreview.co.uk/forums/file-readalllines-doesnt-read-last-blank-line-weird-t3765200.html
Other solution:
using (FileStream fileStream = File.OpenRead("C:\myFile.txt"))
using (StreamReader streamReader = new StreamReader(fileStream))
{
string fileContent = streamReader.ReadToEnd();
textBox1.Text = fileContent;
}
File.ReadAllLines(fullFileName);
does not reads carriage return ('\r'). i think your last line contains only carriage return thats why its not being read. put space in last line to check.
http://msdn.microsoft.com/en-us/library/s2tte0y1.aspx

File cannot be accessed because it is being used by another program

I am trying to remove the space at the end of line and then that line will be written in another file.
But when the program reaches to FileWriter then it gives me the following error
Process can't be accessed because it is being used by another process.
The Code is as below.
private void FrmCounter_Load(object sender, EventArgs e)
{
string[] filePaths = Directory.GetFiles(#"D:\abc", "*.txt", SearchOption.AllDirectories);
string activeDir = #"D:\dest";
System.IO.StreamWriter fw;
string result;
foreach (string file in filePaths)
{
result = Path.GetFileName(file);
System.IO.StreamReader f = new StreamReader(file);
string newFileName = result;
// Combine the new file name with the path
string newPath = System.IO.Path.Combine(activeDir, newFileName);
File.Create(newPath);
fw = new StreamWriter(newPath);
int counter = 0;
int spaceAtEnd = 0;
string line;
// Read the file and display it line by line.
while ((line = f.ReadLine()) != null)
{
if (line.EndsWith(" "))
{
spaceAtEnd++;
line = line.Substring(0, line.Length - 1);
}
fw.WriteLine(line);
fw.Flush();
counter++;
}
MessageBox.Show("File Name : " + result);
MessageBox.Show("Total Space at end : " + spaceAtEnd.ToString());
f.Close();
fw.Close();
}
}
File.Create itself returns a stream.
Use that stream to write file. Reason you are receiving this error is because Stream returned by File.Create is open and you are trying to open that file again for write.
Either close the stream returned by File.Create or better use that stream for file write or use
Stream newFile = File.Create(newPath);
fw = new StreamWriter(newFile);
Even though you solved your initial problem, if you want to write everything into a new file in the original location, you can try to read all of the data into an array and close the original StreamReader. Performance note: If your file is sufficiently large though, this option is not going to be the best for performance.
And you don't need File.Create as the StreamWriter will create a file if it doesn't exist, or overwrite it by default or if you specify the append parameter as false.
result = Path.GetFileName(file);
String[] f = File.ReadAllLines(file); // major change here...
// now f is an array containing all lines
// instead of a stream reader
using(var fw = new StreamWriter(result, false))
{
int counter = f.Length; // you aren't using counter anywhere, so I don't know if
// it is needed, but now you can just access the `Length`
// property of the array and get the length without a
// counter
int spaceAtEnd = 0;
// Read the file and display it line by line.
foreach (var item in f)
{
var line = item;
if (line.EndsWith(" "))
{
spaceAtEnd++;
line = line.Substring(0, line.Length - 1);
}
fw.WriteLine(line);
fw.Flush();
}
}
MessageBox.Show("File Name : " + result);
MessageBox.Show("Total Space at end : " + spaceAtEnd.ToString());
Also, you will not remove multiple spaces from the end of the line using this method. If you need to do that, consider replacing line = line.Substring(0, line.Length - 1); with line = line.TrimEnd(' ');
You have to close any files you are reading before you attempt to write to them in your case.
Write stream in using statement like:
using (System.IO.StreamReader f = new StreamReader(file))
{
//your code goes here
}
EDIT:
Zafar is correct, however, maybe this will clear things up.
Because File.Create returns a stream.. that stream has opened your destination file. This will make things clearer:
File.Create(newPath).Close();
Using the above line, makes it work, however, I would suggest re-writing that properly. This is just for illustrative purposes.

Fastest way to find strings in a file

I have a log file that is not more than 10KB (File size can go up to 2 MB max) and I want to find if atleast one group of these strings occurs in the files. These strings will be on different lines like,
ACTION:.......
INPUT:...........
RESULT:..........
I need to know atleast if one group of above exists in the file. And I have do this about 100 times for a test (each time log is different, so I have reload and read the log), so I am looking for fastest and bets way to do this.
I looked up in the forums for finding the fastest way, but I dont think my file is too big for those silutions.
Thansk for looking.
I would read it line by line and check the conditions. Once you have seen a group you can quit. This way you don't need to read the whole file into memory. Like this:
public bool ContainsGroup(string file)
{
using (var reader = new StreamReader(file))
{
var hasAction = false;
var hasInput = false;
var hasResult = false;
while (!reader.EndOfStream)
{
var line = reader.ReadLine();
if (!hasAction)
{
if (line.StartsWith("ACTION:"))
hasAction = true;
}
else if (!hasInput)
{
if (line.StartsWith("INPUT:"))
hasInput = true;
}
else if (!hasResult)
{
if (line.StartsWith("RESULT:"))
hasResult = true;
}
if (hasAction && hasInput && hasResult)
return true;
}
return false;
}
}
This code checks if there is a line starting with ACTION then one with INPUT and then one with RESULT. If the order of those is not important then you can omit the if () else if () checks. In case the line does not start with the strings replace StartsWith with Contains.
Here's one possible way to do it:
StreamReader sr;
string fileContents;
string[] logFiles = Directory.GetFiles(#"C:\Logs");
foreach (string file in logFiles)
{
using (StreamReader sr = new StreamReader(file))
{
fileContents = sr.ReadAllText();
if (fileContents.Contains("ACTION:") || fileContents.Contains("INPUT:") || fileContents.Contains("RESULT:"))
{
// Do what you need to here
}
}
}
You may need to do some variation based on your exact implementation needs - for example, what if the word spans two lines, does the line need to start with the word, etc.
Added
Alternate line-by-line check:
StreamReader sr;
string[] lines;
string[] logFiles = Directory.GetFiles(#"C:\Logs");
foreach (string file in logFiles)
{
using (StreamReader sr = new StreamReader(file)
{
lines = sr.ReadAllLines();
foreach (string line in lines)
{
if (line.Contains("ACTION:") || line.Contains("INPUT:") || line.Contains("RESULT:"))
{
// Do what you need to here
}
}
}
}
Take a look at How to Read Text From a File. You might also want to take a look at the String.Contains() method.
Basically you will loop through all the files. For each file read line-by-line and see if any of the lines contains 1 of your special "Sections".
You don't have much of a choice with text files when it comes to efficiency. The easiest way would definitely be to loop through each line of data. When you grab a line in a string, split it on the spaces. Then match those words to your words until you find a match. Then do whatever you need.
I don't know how to do it in c# but in vb it would be something like...
Dim yourString as string
Dim words as string()
Do While objReader.Peek() <> -1
yourString = objReader.ReadLine()
words = yourString.split(" ")
For Each word in words()
If Myword = word Then
do stuff
End If
Next
Loop
Hope that helps
This code sample searches for strings in a large text file. The words are contained in a HashSet. It writes the found lines in a temp file.
if (File.Exists(#"temp.txt")) File.Delete(#"temp.txt");
String line;
String oldLine = "";
using (var fs = File.OpenRead(largeFileName))
using (var sr = new StreamReader(fs, Encoding.UTF8, true))
{
HashSet<String> hash = new HashSet<String>();
hash.Add("house");
using (var sw = new StreamWriter(#"temp.txt"))
{
while ((line = sr.ReadLine()) != null)
{
foreach (String str in hash)
{
if (oldLine.Contains(str))
{
sw.WriteLine(oldLine);
// write the next line as well (optional)
sw.WriteLine(line + "\r\n");
}
}
oldLine = line;
}
}
}

Delete specific line from a text file?

I need to delete an exact line from a text file but I cannot for the life of me workout how to go about doing this.
Any suggestions or examples would be greatly appreciated?
Related Questions
Efficient way to delete a line from a text file (C#)
If the line you want to delete is based on the content of the line:
string line = null;
string line_to_delete = "the line i want to delete";
using (StreamReader reader = new StreamReader("C:\\input")) {
using (StreamWriter writer = new StreamWriter("C:\\output")) {
while ((line = reader.ReadLine()) != null) {
if (String.Compare(line, line_to_delete) == 0)
continue;
writer.WriteLine(line);
}
}
}
Or if it is based on line number:
string line = null;
int line_number = 0;
int line_to_delete = 12;
using (StreamReader reader = new StreamReader("C:\\input")) {
using (StreamWriter writer = new StreamWriter("C:\\output")) {
while ((line = reader.ReadLine()) != null) {
line_number++;
if (line_number == line_to_delete)
continue;
writer.WriteLine(line);
}
}
}
The best way to do this is to open the file in text mode, read each line with ReadLine(), and then write it to a new file with WriteLine(), skipping the one line you want to delete.
There is no generic delete-a-line-from-file function, as far as I know.
One way to do it if the file is not very big is to load all the lines into an array:
string[] lines = File.ReadAllLines("filename.txt");
string[] newLines = RemoveUnnecessaryLine(lines);
File.WriteAllLines("filename.txt", newLines);
Hope this simple and short code will help.
List linesList = File.ReadAllLines("myFile.txt").ToList();
linesList.RemoveAt(0);
File.WriteAllLines("myFile.txt"), linesList.ToArray());
OR use this
public void DeleteLinesFromFile(string strLineToDelete)
{
string strFilePath = "Provide the path of the text file";
string strSearchText = strLineToDelete;
string strOldText;
string n = "";
StreamReader sr = File.OpenText(strFilePath);
while ((strOldText = sr.ReadLine()) != null)
{
if (!strOldText.Contains(strSearchText))
{
n += strOldText + Environment.NewLine;
}
}
sr.Close();
File.WriteAllText(strFilePath, n);
}
You can actually use C# generics for this to make it real easy:
var file = new List<string>(System.IO.File.ReadAllLines("C:\\path"));
file.RemoveAt(12);
File.WriteAllLines("C:\\path", file.ToArray());
This can be done in three steps:
// 1. Read the content of the file
string[] readText = File.ReadAllLines(path);
// 2. Empty the file
File.WriteAllText(path, String.Empty);
// 3. Fill up again, but without the deleted line
using (StreamWriter writer = new StreamWriter(path))
{
foreach (string s in readText)
{
if (!s.Equals(lineToBeRemoved))
{
writer.WriteLine(s);
}
}
}
Read and remember each line
Identify the one you want to get rid
of
Forget that one
Write the rest back over the top of
the file
I cared about the file's original end line characters ("\n" or "\r\n") and wanted to maintain them in the output file (not overwrite them with what ever the current environment's char(s) are like the other answers appear to do). So I wrote my own method to read a line without removing the end line chars then used it in my DeleteLines method (I wanted the option to delete multiple lines, hence the use of a collection of line numbers to delete).
DeleteLines was implemented as a FileInfo extension and ReadLineKeepNewLineChars a StreamReader extension (but obviously you don't have to keep it that way).
public static class FileInfoExtensions
{
public static FileInfo DeleteLines(this FileInfo source, ICollection<int> lineNumbers, string targetFilePath)
{
var lineCount = 1;
using (var streamReader = new StreamReader(source.FullName))
{
using (var streamWriter = new StreamWriter(targetFilePath))
{
string line;
while ((line = streamReader.ReadLineKeepNewLineChars()) != null)
{
if (!lineNumbers.Contains(lineCount))
{
streamWriter.Write(line);
}
lineCount++;
}
}
}
return new FileInfo(targetFilePath);
}
}
public static class StreamReaderExtensions
{
private const char EndOfFile = '\uffff';
/// <summary>
/// Reads a line, similar to ReadLine method, but keeps any
/// new line characters (e.g. "\r\n" or "\n").
/// </summary>
public static string ReadLineKeepNewLineChars(this StreamReader source)
{
if (source == null)
throw new ArgumentNullException(nameof(source));
char ch = (char)source.Read();
if (ch == EndOfFile)
return null;
var sb = new StringBuilder();
while (ch != EndOfFile)
{
sb.Append(ch);
if (ch == '\n')
break;
ch = (char)source.Read();
}
return sb.ToString();
}
}
Are you on a Unix operating system?
You can do this with the "sed" stream editor. Read the man page for "sed"
What?
Use file open, seek position then stream erase line using null.
Gotch it? Simple,stream,no array that eat memory,fast.
This work on vb.. Example search line culture=id where culture are namevalue and id are value and we want to change it to culture=en
Fileopen(1, "text.ini")
dim line as string
dim currentpos as long
while true
line = lineinput(1)
dim namevalue() as string = split(line, "=")
if namevalue(0) = "line name value that i want to edit" then
currentpos = seek(1)
fileclose()
dim fs as filestream("test.ini", filemode.open)
dim sw as streamwriter(fs)
fs.seek(currentpos, seekorigin.begin)
sw.write(null)
sw.write(namevalue + "=" + newvalue)
sw.close()
fs.close()
exit while
end if
msgbox("org ternate jua bisa, no line found")
end while
that's all..use #d

Categories