We Keep Coding

How can I read big txt file?

i have a 1000 text file and i want read single to single and Each file has a 4700000 record,for example one of line in file is:
43266200 6819 43295200 1393/05/23 14:28:45 113 1
and i want save into sql server for example:
field1:43266200
field2:6819
how can i do this?

var seperators = " ".ToCharArray();
foreach(var line in File.ReadLines(path))
{
var fields = line.Split(seperators, StringSplitOptions.RemoveEmptyEntries);
//now you have fields[0] and fields[1], save them in your database
}

This may help you
var message ="43266200 6819 43295200 1393/05/23 14:28:45 113 1";
//Split your data into pieces
var messages=message.Split(' ').Where( o => !string.IsNullOrEmpty(o));
var i=0;
foreach(var item in messages)
{
// do whatever you wanna to do with pieces
Console.Write( "field {0}:{1}",++i,item);
}

If you're reading the text from a file, and you can reasonably assume that the space character will be your only delimiter, you should use the String.Split() method to tokenize each line:
// instantiate FileInfo of your file as yourFile
foreach (string line in yourFile.ReadLines())
{
string[] lineTokens = line.Split(new char[] { ' ' }, StringSplitOptions.RemoveEmptyEntries);
}
String.Split() allows you to separate any string into a string[] of substrings based on the char delimiters you provide in the first argument. The second argument in the code above is one of the values in the StringSplitOptions enumeration, which has values of either None (provide all strings) or RemoveEmptyEntries (do not return any substrings that consist solely of delimiter characters).
Then, from there, you can iterate through lineTokens and assemble an object from each token, or you can assemble an SQL query where any given index corresponds to a column in the row you intend to add.

Comparing string array with starting values in another

I have a string array with fixed values and a richtextbox whose text is dynamically changed. Some of the lines in the richtextbox start with values in the string array. I want to select only the lines of the richtextbox which do not start with values in the string array.
The following code returns all lines in the richtextbox.
string[] parts = new string[] { "Definition:", "derivation:", "derivations:"};
IEnumerable<string> lines = richTextBox1.Lines.Where(
c =>parts.Any(b=>!c.StartsWith(b)));
My question is: How can I select only the lines of the richtextbox which do not start with values in the string array?

Change Any to All. As it's written, it returns all lines because a line can't start with more than one word.
Your current code says, "return true if there is any word in parts that isn't the first word of the line." Obviously, the line can't start with "foo" and with "derivation:". So you always get true.
You want to say, "return true if all of the words in parts are not the first word of the line."
Another way to do it is:
lines = richTextBox1.Lines.Where(c => !parts.Any(b => c.StartsWith(b)));
Which is probably how I would have written it.

You put the (!) operator in the wrong place. If you want to use Any then
string[] parts = new string[] { "Definition:", "derivation:", "derivations:"};
IEnumerable<string> lines = richTextBox1.Lines.Where(
c => !parts.Any(b => c.StartsWith(b)));

Find and Delete Characters in String

My program reads registry key values and combines those values with the installation path. I also read the installation path from the registry.
i.e. String dllInstpath = installPath + rKey which equals to:
C:\Program Files (x86)\NSi\AutoStore Workflow 6\HpOXPdCaptureRes.dll
I then use FileVersionInfo on the string above to get the file information of HpOXPdCaptureRes.dll from it's install path and write all the values to a notepad.
My problem is the TRUE dll name does not have 'Res' in the file name. The registry only has the file name with 'Res' in the file name. What I need to do is read from a text file and find all 'Res' and remove them from the line of text within the notepad file.
So the output should look like this:
Current:
HpOXPdCaptureRes.dll
New:
HpOXPdCapture.dll
I have read online and I see the best way to do this is to use ReadAllLines and WriteAllLines. However I am not sure how to implement the find and replace. I have seen a lot of examples on how to remove spaces, invalid characters, etc., but I haven't been able to find an example for what I need.
Summary:
Read text file
Fine Res in all lines of text and remove
Retain current text file, i.e. remove Res and close file
Any help is greatly appreciated.
Thank you!

You can use File.ReadAllLines and File.WriteAllLines.
Example:
Read all the lines and replace the value you want on each line, then write the lines again
File.WriteAllLines("textFilePath",File.ReadAllLines("textFilePath").Select(line => line.Replace("Res.dll", ".dll")));

Just open the file and read all lines using 'File.ReadAllLines'. Then use Replace to remove the Res:
string[] lines = File.ReadAllLines("yourFileName");
var output = lines.Select(x => x.Replace("Res.dll", ".dll")).ToArray();
To later save them back you can use File.WriteAllLines:
File.WriteAllLines("yourFileName", output);

Read everything from file, replace all occurrences of 'res' and write to file:
String filename = "fileName";
StreamReader sr = new StreamReader(filename);
String content = sr.ReadToEnd();
sr.Close();
StreamWriter sw = new StreamWriter(filename);
sw.Write(content.Replace("res", ""));
sw.Close();

If the string you are replacing is guaranteed to be unique in the string - "res.dll" at the end of the string for instance - then you can use Replace method of the String class to do the replacement:
List<string> lines = File.ReadAllLines(sourceFile);
lines = lines.select(l => l.Replace("res.dll", ".dll").ToList();
Or if case sensitivity is an issue:
lines = lines.Select(l => l.Substring(l.Length - 7).ToLower() == "res.dll" ? l.Substring(0, l.Length - 7) + ".dll" : l).ToList();
For more complex cases you might need to use a regular expression to identify the section of the string to replace. Or you might want to split the string int path and filename, modify the filename and join it back together.

how to find indexof substring in a text file

I have converted an asp.net c# project to framework 3.5 using VS 2008. Purpose of app is to parse a text file containing many rows of like information then inserting the data into a database.
I didn't write original app but developer used substring() to fetch individual fields because they always begin at the same position.
My question is:
What is best way to find the index of substring in text file without having to manually count the position? Does someone have preferred method they use to find position of characters in a text file?

I would say IndexOf() / IndexOfAny() together with Substring(). Alternatively, regular expressions. It the file has an XML-like structure, this.

If the files are delimited eg with commas you can use string.Split
If data is: string[] text = { "1, apple", "2, orange", "3, lemon" };
private void button1_Click(object sender, EventArgs e)
{
string[] lines = this.textBoxIn.Lines;
List<Fruit> fields = new List<Fruit>();
foreach(string s in lines)
{
char[] delim = {','};
string[] fruitData = s.Split(delim);
Fruit f = new Fruit();
int tmpid = 0;
Int32.TryParse(fruitData[0], out tmpid);
f.id = tmpid;
f.name = fruitData[1];
fields.Add(f);
}
this.textBoxOut.Clear();
string text=string.Empty;
foreach(Fruit item in fields)
{
text += item.ToString() + " \n";
}
this.textBoxOut.Text = text;
}
}

The text file I'm reading does not contain delimiters - sometimes there spaces between fields and sometimes they run together. In either case, every line is formatted the same. When I asked the question I was looking at the file in notepad.
Question was: how do you find the position in a file so that position (a number) could be specified as the startIndex of my substring function?
Answer: I've found that opening the text file in notepad++ will display the column # and line count of any position where the curser is in the file and makes this job easier.

You can use indexOf() and then use Length() as the second substring parameter
substr = str.substring(str.IndexOf("."), str.Length - str.IndexOf("."));

How to split lines from Windows text file (/r/n separation)

My question is quite simple. I need to get all text lines from Windows text file.
All lines are separated by \r\n symbols. I use String.Split, but its not cool, because
it only splits 'by one symbol' leaving empty string that I need to remove with options flag. Is there a better way?
My implementation
string wholefile = GetFromSomeWhere();
// now parsing
string[] lines = operationtext.Split("\r\n".ToCharArray(), StringSplitOptions.RemoveEmptyEntries);
// ok now I have lines array
UPDATE
File.ReadAllXXX is of no use here coz GetFromSomeWhere is actually RegEx, so I've no file after this point.

You can use this overload of String.Split, which takes an array of strings that can serve as delimiters:
string[] lines = operationtext.Split(new[] { Environment.NewLine },
StringSplitOptions.RemoveEmptyEntries);
Of course, if you already have the file-path, it's much simpler to use File.ReadAllLines:
string[] lines = File.ReadAllLines(filePath);

String.Split does accept a string (like "\r\n"). Not just a chararray.
string[] lines = wholetext.Split(new string[] { "\r\n" }, StringSplitOptions.RemoveEmptyEntries);

You may find it much easier to simply use File.ReadAllLines() or File.ReadLines()

you could use an extension method like the one below and your code would then look like this:
var lines = operationText.ReadAsLines();
Extension method implementation:
public static IEnumerable<string> ReadAsLines(this string text)
{
TextReader reader = new StringReader(text);
while(reader.Peek() >= 0)
{
yield return reader.ReadLine();
}
}
I'm guessing it's not as performant as the split option which is usually very performant but if that's not an issue...