We Keep Coding

Develop a file reading routine in c#

I have developed a file reading routine using c#.net that will read the entire file contents into memory using a suitable Data Class or structure.
I have a text file of 600MB that has RoadId and many other entries. I have to read that file using query method so I used Stream Reader in c#.net that reads line by line. But I want to know is there any other method in c#.net that will be memory efficient and less time taking or by converting the text to binary and then reading.
Not sure please guide me through this.
I am putting my code for reading whole file line by line...
public static void read_time()
{
DateTime end;
StreamReader file =
new StreamReader(#"C:\Users\Reva-Asus1\Desktop\DTF Test\F_Network_out.txt");
DateTime start = DateTime.Now;
while ((file.ReadLine()) != null) ;
end = DateTime.Now;
Console.WriteLine();
Console.WriteLine("Full File Read Time: " + (end - start));
Console.WriteLine();
file.Close();
Console.WriteLine("Data is read");
Console.ReadLine();
return;
}
// This querying method is to take roadId from user from console and display the record....
public static void querying_method()
{
Console.WriteLine("Give a RoadId to search record\n");
DateTime start, end;
string id =Console.ReadLine().Trim();
try
{
System.IO.StreamReader file =
new System.IO.StreamReader(#"C:\Users\Reva-Asus1\Desktop\DTF Test\F_Network_out.txt");
string line1;
int count = 1;
start = DateTime.Now;
while ((line1 = file.ReadLine()) != null)
{
if(line1 == id)
{
string line2 = " ";
while (count != 14)
{
Console.WriteLine(line2 = file.ReadLine());
count++;
}
int n = Convert.ToInt16(line2);
while (n != 0)
{
Console.WriteLine(line2 = file.ReadLine());
n--;
}
break;
}
}
end = DateTime.Now;
Console.WriteLine("Read Time for the data record: " + (end - start));
Console.ReadLine();
return;
}
catch (Exception)
{
Console.WriteLine("No ID match found in the file entered by user");
Console.ReadLine();
return;
}
}

You could maybe use this:
foreach (var line in File.ReadLines(path))
{
// TODO: Parse the line and convert to your object...
}

File.ReadLines(YourPath) is using using StreamReader in the background, so you can continue using it. Here reference code. So if you already using StreamReader and reading only by one line, you don't need to change anything.
using (StreamReader sr = new StreamReader(path, encoding))
{
while ((line = sr.ReadLine()) != null)
{
//You are reading the file line by line and you load only the current line in the memory, not the whole file.
//do stuff which you want with the current line.
}
}

how to skip a line inside stream reader?

using streamreader, if line starts with '0' skip the line???
string FileToProcess = System.IO.File.ReadAllText(#"up.FolderPath");
using (StreamReader sr = new StreamReader(FileToProcess))
{
while (!sr.EndOfStream)
{
string line = sr.ReadLine();
if (line.StartsWith("0"))
{
line.Skip();
// ????have to put a number line in here
// But i just want to skip the line it is on
}
}
}

You can skip a line by just not doing anything with it. Quickest way, use continue;...
while (!sr.EndOfStream)
{
string line = sr.ReadLine();
if (line.StartsWith("0"))
{
continue;
}
//process line logic
}
Using continue will effectively jump to the start of the while loop again, which will then proceed to reading the next line,
This is based on my understanding that you want to skip lines that start with "0"

string FileToProcess = System.IO.File.ReadAllText(#"up.FolderPath");
using (StreamReader sr = new StreamReader(FileToProcess))
{
while (!sr.EndOfStream)
{
string line = sr.ReadLine();
if (line.StartsWith("0"))
{
continue;
}
else{
//Do stuff here
}
}
}

Do something with the line only when it do not start with '0'

Filtering a line out of a string c#

I want to read a .txt file in c# and filter a line out of the string and only show that line. If the match is on the first line, i get a good output using streamreader.ReadLine. But if it's on the second line, i need to get it filtered. (i tought by creating a ReadLine loop?)
Thanks in advance
private void comboBox3_SelectedIndexChanged(object sender, EventArgs e)
{
StreamReader sr = new StreamReader(textBox1.Text);
string BoxLM1 = sr.ReadLine();
if (comboBox3.Text == "Anderlecht")
{
if (BoxLM1.Contains("Anderlecht"))
{
label5.Text = BoxLM1;
}
else
{
string BoxLM2 = sr.ReadToEnd();
MessageBox.Show(BoxLM2);
}

You can check all lines at once using File.ReadLines() method and LINQ:
var firstAnderlecht = File.ReadLines(textBox1.Text).FirstOrDefault(s => s.Contains("Anderlecht"));
if (firstAnderlecht != null) {
label5.Text = firstAnderlecht;
}
The ReadLines produces an enumerable of lines from the file; the FirstOrDefault method lets you apply a condition to all lines of the file without a loop, and pick the first line where the condition applies.

If you are manipulating big files i recommend to use this iterator:
private static IEnumerable FileIterator(String filePathe)
{
using (StreamReader streamReader = new StreamReader(filePathe))
{
String line;
while ((line = streamReader.ReadLine()) != null)
{
yield return line;
}
yield break;
}
}
it will prevent the loading of full file to RAM

Delete Lines in a textfile

Hi I have a text file with table schema and data when user checks not schema required then i need to delete schema and leave the data . I am using StreamReader to read the file and checking one condition and it should delete all the lines in the file till it satisfies my condition .
Let say if i am checking
using (StreamReader tsr = new StreamReader(targetFilePath))
{
do
{
string textLine = tsr.ReadLine() + "\r\n";
{
if (textLine.StartsWith("INSERT INTO"))
{
// It should leave these lines
// and no need to delete lines
}
else
{
// it should delete the lines
}
}
}
while (tsr.Peek() != -1);
tsr.Close();
Please suggest me how to delete lines and note if textline finds "InsertInto" it should not delete any content from there .

Use a second file where to put only required lines, and, at the end of the process, remove original file and rename new one to target file.
using (StreamReader tsr = new StreamReader(targetFilePath))
{
using (StreamWriter tsw = File.CreateText(targetFilePath+"_temp"))
{
string currentLine;
while((currentLine = tsr.ReadLine()) != null)
{
if(currentLine.StartsWith("A long time ago, in a far far away galaxy ..."))
{
tsw.WriteLine(currentLine);
}
}
}
}
File.Delete(targetFilePath);
File.Move(targetFilePath+"_temp",targetFilePath);

You could use Linq:
File.WriteAllLines(targetFilePath, File.ReadAllLines(targetFilePath).Where(x => x.StartsWith("INSERT INTO")));

You read in the file just the same way you were doing. However, if the line doesn't contain what you are looking for, you simply skip it. In the end, whatever data you are left over with you then write to a new text file.
private void button1_Click(object sender, EventArgs e)
{
StringBuilder newText = new StringBuilder();
using (StreamReader tsr = new StreamReader(targetFilePath))
{
do
{
string textLine = tsr.ReadLine() + "\r\n";
{
if (textLine.StartsWith("INSERT INTO"))
{
newText.Append(textLine + Environment.NewLine);
}
}
}
while (tsr.Peek() != -1);
tsr.Close();
}
System.IO.TextWriter w = new System.IO.StreamWriter(#"C:\newFile.txt");
w.Write(newText.ToString());
w.Flush();
w.Close();
}

Delete specific line from a text file?

I need to delete an exact line from a text file but I cannot for the life of me workout how to go about doing this.
Any suggestions or examples would be greatly appreciated?
Related Questions
Efficient way to delete a line from a text file (C#)

If the line you want to delete is based on the content of the line:
string line = null;
string line_to_delete = "the line i want to delete";
using (StreamReader reader = new StreamReader("C:\\input")) {
using (StreamWriter writer = new StreamWriter("C:\\output")) {
while ((line = reader.ReadLine()) != null) {
if (String.Compare(line, line_to_delete) == 0)
continue;
writer.WriteLine(line);
}
}
}
Or if it is based on line number:
string line = null;
int line_number = 0;
int line_to_delete = 12;
using (StreamReader reader = new StreamReader("C:\\input")) {
using (StreamWriter writer = new StreamWriter("C:\\output")) {
while ((line = reader.ReadLine()) != null) {
line_number++;
if (line_number == line_to_delete)
continue;
writer.WriteLine(line);
}
}
}

The best way to do this is to open the file in text mode, read each line with ReadLine(), and then write it to a new file with WriteLine(), skipping the one line you want to delete.
There is no generic delete-a-line-from-file function, as far as I know.

One way to do it if the file is not very big is to load all the lines into an array:
string[] lines = File.ReadAllLines("filename.txt");
string[] newLines = RemoveUnnecessaryLine(lines);
File.WriteAllLines("filename.txt", newLines);

Hope this simple and short code will help.
List linesList = File.ReadAllLines("myFile.txt").ToList();
linesList.RemoveAt(0);
File.WriteAllLines("myFile.txt"), linesList.ToArray());
OR use this
public void DeleteLinesFromFile(string strLineToDelete)
{
string strFilePath = "Provide the path of the text file";
string strSearchText = strLineToDelete;
string strOldText;
string n = "";
StreamReader sr = File.OpenText(strFilePath);
while ((strOldText = sr.ReadLine()) != null)
{
if (!strOldText.Contains(strSearchText))
{
n += strOldText + Environment.NewLine;
}
}
sr.Close();
File.WriteAllText(strFilePath, n);
}

You can actually use C# generics for this to make it real easy:
var file = new List<string>(System.IO.File.ReadAllLines("C:\\path"));
file.RemoveAt(12);
File.WriteAllLines("C:\\path", file.ToArray());

This can be done in three steps:
// 1. Read the content of the file
string[] readText = File.ReadAllLines(path);
// 2. Empty the file
File.WriteAllText(path, String.Empty);
// 3. Fill up again, but without the deleted line
using (StreamWriter writer = new StreamWriter(path))
{
foreach (string s in readText)
{
if (!s.Equals(lineToBeRemoved))
{
writer.WriteLine(s);
}
}
}

Read and remember each line
Identify the one you want to get rid
of
Forget that one
Write the rest back over the top of
the file

I cared about the file's original end line characters ("\n" or "\r\n") and wanted to maintain them in the output file (not overwrite them with what ever the current environment's char(s) are like the other answers appear to do). So I wrote my own method to read a line without removing the end line chars then used it in my DeleteLines method (I wanted the option to delete multiple lines, hence the use of a collection of line numbers to delete).
DeleteLines was implemented as a FileInfo extension and ReadLineKeepNewLineChars a StreamReader extension (but obviously you don't have to keep it that way).
public static class FileInfoExtensions
{
public static FileInfo DeleteLines(this FileInfo source, ICollection<int> lineNumbers, string targetFilePath)
{
var lineCount = 1;
using (var streamReader = new StreamReader(source.FullName))
{
using (var streamWriter = new StreamWriter(targetFilePath))
{
string line;
while ((line = streamReader.ReadLineKeepNewLineChars()) != null)
{
if (!lineNumbers.Contains(lineCount))
{
streamWriter.Write(line);
}
lineCount++;
}
}
}
return new FileInfo(targetFilePath);
}
}
public static class StreamReaderExtensions
{
private const char EndOfFile = '\uffff';
/// <summary>
/// Reads a line, similar to ReadLine method, but keeps any
/// new line characters (e.g. "\r\n" or "\n").
/// </summary>
public static string ReadLineKeepNewLineChars(this StreamReader source)
{
if (source == null)
throw new ArgumentNullException(nameof(source));
char ch = (char)source.Read();
if (ch == EndOfFile)
return null;
var sb = new StringBuilder();
while (ch != EndOfFile)
{
sb.Append(ch);
if (ch == '\n')
break;
ch = (char)source.Read();
}
return sb.ToString();
}
}

Are you on a Unix operating system?
You can do this with the "sed" stream editor. Read the man page for "sed"

What?
Use file open, seek position then stream erase line using null.
Gotch it? Simple,stream,no array that eat memory,fast.
This work on vb.. Example search line culture=id where culture are namevalue and id are value and we want to change it to culture=en
Fileopen(1, "text.ini")
dim line as string
dim currentpos as long
while true
line = lineinput(1)
dim namevalue() as string = split(line, "=")
if namevalue(0) = "line name value that i want to edit" then
currentpos = seek(1)
fileclose()
dim fs as filestream("test.ini", filemode.open)
dim sw as streamwriter(fs)
fs.seek(currentpos, seekorigin.begin)
sw.write(null)
sw.write(namevalue + "=" + newvalue)
sw.close()
fs.close()
exit while
end if
msgbox("org ternate jua bisa, no line found")
end while
that's all..use #d