C# remember current file stream location and jump to the location - c#

I want to remember where, that my program had processed.
I call StreamReader ReadLine()
Then next time I can jump right back to the location keep processing.
I don't want to do the line count and skip, I want to store a physicial location or something like that. So next time I can jump right back.
Thanks

you can count your text length, and then seek it before reading like below code:
hope it can help you.
internal class Program
{
static void Main(string[] args)
{
var fileName = "your file path";
long count = 0;
var firstLine = GetContent(fileName, count);
count += firstLine.position;
var secondLine = GetContent(fileName, count);
count += secondLine.position;
var thirdLine = GetContent(fileName, count);
}
public static (string content, int position) GetContent(string fileName, long position)
{
using var x = File.OpenRead(fileName);
using var ww = new StreamReader(x);
x.Seek(position, SeekOrigin.Begin);
var k = ww.ReadLine();
return (k, ww.CurrentEncoding.GetByteCount(k) + 2);
}
}

Related

Counting total characters of a file

Hi I'm pretty new to C# and trying to do some exercises to get up to speed with it. I'm trying to count the total number of characters in a file but it's stopping after the first word, would someone be able to tell me where I am going wrong? Thanks in advance
public void TotalCharacterCount()
{
string str;
int count, i, l;
count = i = 0;
StreamReader reader = File.OpenText("C:\\Users\\Lewis\\file.txt");
str = reader.ReadLine();
l = str.Length;
while (str != null && i < l)
{
count++;
i++;
str = reader.ReadLine();
}
reader.Close();
Console.Write("Number of characters in the file is : {0}\n", count);
}
If you want to know the size of a file:
long length = new System.IO.FileInfo("C:\\Users\\Lewis\\file.txt").Length;
Console.Write($"Number of characters in the file is : {length}");
If you want to count characters to play around with C#, then here is some sample code that might help you
int totalCharacters = 0;
// Using will do the reader.Close for you.
using (StreamReader reader = File.OpenText("C:\\Users\\Lewis\\file.txt"))
{
string str = reader.ReadLine();
while (str != null)
{
totalCharacters += str.Length;
str = reader.ReadLine();
}
}
// If you add the $ in front of the string, then you can interpolate expressions
Console.Write($"Number of characters in the file is : {totalCharacters}");
it's stopping after the first word
It is because you have check && i < l in the loop and then increment it so the check doesn't pass you don't change the value of l variable(by the way, the name is not very good, I was sure it was 1, not l).
Then if you need to get total count of characters in the file you could read the whole file to a string variable and just get it from Count() Length
var count = File.ReadAllText(path).Count();
Getting Length property of the FileInfo will give the size, in bytes, of the current file, which is not necessary will be equal to characters count(depending on Encoding a character may take more than a byte)
And regarding the way you read - it also depends whether you want to count new line symbols and others or not.
Consider the following sample
static void Main(string[] args)
{
var sampleWithEndLine = "a\r\n";
var length1 = "a".Length;
var length2 = sampleWithEndLine.Length;
var length3 = #"a
".Length;
Console.WriteLine($"First sample: {length1}");
Console.WriteLine($"Second sample: {length2}");
Console.WriteLine($"Third sample: {length3}");
var totalCharacters = 0;
File.WriteAllText("sample.txt", sampleWithEndLine);
using(var reader = File.OpenText("sample.txt"))
{
string str = reader.ReadLine();
while (str != null)
{
totalCharacters += str.Length;
str = reader.ReadLine();
}
}
Console.WriteLine($"Second sample read with stream reader: {totalCharacters}");
Console.ReadKey();
}
For the second sample, first, the Length will return 3, because it actually contains three symbols, while with stream reader you will get 1, because The string that is returned does not contain the terminating carriage return or line feed. The returned value is null if the end of the input stream is reached

c# position counter in a stream reader

Developing a small quiz application, thus far I have a text file containing two example questions to see if i can get it working properly before adding more. There are 5 question levels of increasing difficulty, each of these levels contains 10 questions
I'm using a streamreader to read my level 1 questions in from the text file, it reads the first question in fine and the program reads user input and compares it to the answer. If correct the user will move to the next level, but if incorrect I want the program to ask the second question in the file - but it continues to read the first.
I have
static int pos = 0;
serving as my position counter for the reader, but whenever I try to include the position when addressing my struct in the reader like
_question1[pos].q_No = Convert.ToInt32(sreader.ReadLine());
I get an error message:
Cannot apply indexing with [] to an expression of type
'Quiz_Application.Program.question1'
Variables n stuff:
static question1[] _questions1 = new question1[10];
static question2[] _questions2 = new question2[10];
static question3[] _questions3 = new question3[10];
static question4[] _questions4 = new question4[10];
static question5[] _questions5 = new question5[10];
static int score = 0;
static int asked = 0;
static int pos = 0;
static int user_input = 0;
static int user_level = 1;
struct question1
{
public int q_No;
public string Question;
public string Choices;
public int Answer;
}
My reader:
static void QuestionReader_Level1()
{
Console.Clear();
question1 _question1 = new question1();
string filename = #"C:\Users\Craigo\Desktop\Quiz_Application\Files\question1.txt";
while (user_level == 1)
{
using (StreamReader sreader = new StreamReader(filename, true))
{
pos += 1;
asked += 1;
_question1.q_No = Convert.ToInt32(sreader.ReadLine());
Console.WriteLine(_question1.q_No);
_question1.Question = sreader.ReadLine();
Console.WriteLine(_question1.Question);
_question1.Choices = sreader.ReadLine();
Console.WriteLine(_question1.Choices);
_question1.Answer = Convert.ToInt32(sreader.ReadLine());
user_input = Convert.ToInt32(Console.ReadLine());
if (user_input == _question1.Answer)
{
score += 1;
user_level += 1;
Console.WriteLine("\nCongratulations, you have scored 1 point and advanced to level 2");
Console.WriteLine("Score = {0}, Questions Asked = {1}", score, asked);
}
}
}
}
What do I do?
the variable "_question1" is not an array:
question1 _question1 = new question1();
So, when you have "_question1[pos]", it won't work.
Your array is "_questions1" (you are missing the 's'):
question1[] _questions1 = new question1[10];
_questions1[pos] should work
The while loop only loops when the variable user_level is 1. Also , you have added that if the user's input is equal to the answer , user_level will increment by 1 which will equal to 2 and hence , the loop will not run again. However , the fact that the program is still looping means that the condition was not met , which tells us that THE USER INPUT IS NOT EQUAL TO THE ANSWER. Hence , the error might be caused by the file. Would you mind showing the contents of the file?

ASP.net C# : How to read 20 to 200 GB file line by line using File.ReadLines(fileName).GetEnumerator()?

We are trying with below code.
public static int SplitFile(string fileName, string tmpFolder, List<string> queue, int splitSize = 100000)
{
int chunk = 0;
if (!Directory.Exists(tmpFolder))
Directory.CreateDirectory(tmpFolder);
using (var lineIterator = File.ReadLines(fileName).GetEnumerator())
{
bool stillGoing = true;
for (chunk = 0; stillGoing; chunk++)
{
stillGoing = WriteChunk(lineIterator, splitSize, chunk, tmpFolder, queue);
}
}
return chunk;
}
private static bool WriteChunk(IEnumerator<string> lineIterator,
int splitSize, int chunk, string tmpFolder, List<string> queue)
{
try
{
//int tmpChunkSize = 1000;
//int tmpChunkInc = 0;
string splitFile = Path.Combine(tmpFolder, "file" + chunk + ".txt");
using (var writer = File.CreateText(splitFile))
{
queue.Add(splitFile);
for (int i = 0; i < splitSize; i++)
{
if (!lineIterator.MoveNext())
{
return false;
}
writer.WriteLine(lineIterator.Current);
}
}
return true;
}
catch (Exception)
{
throw;
}
}
It creates around 36 text files (around 800 MB), but starting throwing "Out of memory exception" in creation of 37th File at lineIterator.MoveNext().
While lineIterator.Current shows the value in debugger.
As It s a huge file you should read it Seek and ReadBytes methods of BinaryReader.
You can see a simple example here. After you use the ReadBytes check for the last new lines and write the process file in certain amount of lines you read. Don t write every line you read and also don t keep all the data in the memory.
The rest is in your hands.
Maybe it is realted to that one When does File.ReadLines free resources
IEnumerable doesn't inherit from IDisposable because typically, the class that implements it only gives you the promise of being enumerable, it hasn't actually done anything yet that warrants disposal.

Requesting direction for loading my file in chunks (Only needing advice)

I've been working for the last few days on a method to compress 144 million tile representation for my xna game down to a very small size when saved. Having managed to pull that off I now find myself stumped on how to go about getting them back from the file in chunks.
In the file I have.
An integer (it gets compressed to bytes using the 7BitEncodedInt method)
A byte
The compressed integer represents the number of tiles and the byte that follows determines what type the tiles are. This is all well and good and works really well. Most importantly it shrinks the file size down to just 50mb on average.
The problem is that I am currently reading back the entire file.
From the file I'm getting this.
The index value of each tile (just a basic iteration as I grab the tiles)
The type for each tile as a byte value
A byte value representing a texture for that tile (this is hard to explain but its necessary on a per tile basis)
The end result of all this is that I'm managing to save the file and only use about 50mb. But by loading the whole thing back in it expands out to nearly 1.5gigs on the ram. I can't really afford to sacrifice anymore tile info. so I need a way to only load portions of the map based on the player location. The goal is to be around the 100-200mb range
I have been looking at memory mapping the file, using quadtrees, pretty much anything I could find for loading files in chunks. While these options all seem pretty good I'm not sure which is best or if given the situation there may be another even better one. The other problem with all this is that these solutions all seem very involved (especially since this is my first time using them) and while I'm not against devoting myself to some lengthy coding I'd like to know that its gonna do what I need it to before hand.
My question is, given how I have to process the file as I pull it in and the fact that it needs to be done based on the players location what would be the best way to do this ? I'm just looking for some direction here. Code is always welcome but not required.
You want to have fixed length variables in your Tile class and implement something like such:
This is an example of a collection class (People) that can get a value based on index from a collection that was serialised into a file.
Person is the class that is the base of the People collection.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace FileStreamDatabaseTest
{
class Program
{
static void Main(string[] args)
{
People.OpenCollection();
People.Test_WillOverwriteData();
People.CloseCollection();
Console.ReadLine();
}
}
public class Person
{
// define maxium variable sizes for serialisation
protected static int pMaxLength_FirstName = 64;
protected static int pMaxLength_Age = 10;
public static int MaxObjectSize
{
get
{
// return the sum of all the maxlegnth variables to define the entire object size for serialisation
return pMaxLength_FirstName + pMaxLength_Age;
}
}
// define each object that will be serialised as follows
protected string pFirstName;
public string Firstname
{
set
{
// ensure the new value is not over max variable size
if (value.Length > pMaxLength_FirstName)
throw new Exception("the length of the value is to long.");
pFirstName = value;
}
get
{
return pFirstName;
}
}
protected int pAge;
public int Age
{
get
{
return pAge;
}
set
{
pAge = value;
}
}
public byte[] Serialise()
{
// Output string builder
StringBuilder Output = new StringBuilder();
// Append firstname value
Output.Append(Firstname);
// Add extra spaces to end of string until max length is reached
if (Firstname.Length < pMaxLength_FirstName)
for (int i = Firstname.Length; i < pMaxLength_FirstName; i++)
Output.Append(" ");
// Append age value as string
Output.Append(Age.ToString());
// Add extra spaces to end of string until max length is reached
int AgeLength = Age.ToString().Length;
if (AgeLength < pMaxLength_Age)
for (int i = AgeLength; i < pMaxLength_Age; i++)
Output.Append(" ");
// Return the output string as bytes using ascii encoding
return System.Text.Encoding.ASCII.GetBytes(Output.ToString());
}
public void Deserialise(byte[] SerialisedData)
{
string Values = System.Text.Encoding.ASCII.GetString(SerialisedData);
pFirstName = Values.Substring(0, pMaxLength_FirstName).Trim();
pAge = int.Parse(Values.Substring(pMaxLength_FirstName, pMaxLength_Age).Trim());
}
}
public static class People
{
private static string tileDatasource = #"c:\test.dat";
private static System.IO.FileStream FileStream;
public static void OpenCollection()
{
FileStream = new System.IO.FileStream(tileDatasource, System.IO.FileMode.OpenOrCreate, System.IO.FileAccess.ReadWrite, System.IO.FileShare.None);
}
public static void CloseCollection()
{
FileStream.Close();
FileStream.Dispose();
FileStream = null;
}
public static void SaveCollection(Person[] People)
{
FileStream.SetLength(People.Length * Person.MaxObjectSize);
FileStream.Position = 0;
foreach (Person PersonToWrite in People)
{
// call serialise to get bytes
byte[] OutputBytes = PersonToWrite.Serialise();
// write the output buffer
// note: this will always be the same size as each variable should
// append spaces until its max size is reached
FileStream.Write(OutputBytes, 0, OutputBytes.Length);
}
}
public static Person GetValue(int Index)
{
// set the stream position to read the object by multiplying the requested index with the max object size
FileStream.Position = Index * Person.MaxObjectSize;
// read the data
byte[] InputBytes = new byte[Person.MaxObjectSize];
FileStream.Read(InputBytes, 0, Person.MaxObjectSize);
// deserialise
Person PersonToReturn = new Person();
PersonToReturn.Deserialise(InputBytes);
// retun the person
return PersonToReturn;
}
public static void Test_WillOverwriteData()
{
long StartTime;
long EndTime;
TimeSpan TimeTaken;
Console.WriteLine("-------------------------------------------------------------------");
Console.WriteLine("*** Creating 2,000,000 test people... ");
StartTime = DateTime.Now.Ticks;
Person[] People = new Person[2000000];
for (int i = 0; i < 2000000; i++)
{
People[i] = new Person();
People[i].Firstname = "TestName." + i;
People[i].Age = i;
}
EndTime = DateTime.Now.Ticks;
TimeTaken = new TimeSpan(EndTime - StartTime);
Console.WriteLine("-> Completed in " + TimeTaken.TotalSeconds + " seconds");
Console.WriteLine("-------------------------------------------------------------------");
Console.WriteLine("*** Serialising Collection to disk... ");
StartTime = DateTime.Now.Ticks;
SaveCollection(People);
EndTime = DateTime.Now.Ticks;
TimeTaken = new TimeSpan(EndTime - StartTime);
Console.WriteLine("-> Completed in " + TimeTaken.TotalSeconds + " seconds");
Console.WriteLine("-------------------------------------------------------------------");
Console.WriteLine("*** Redundancy Test... ");
StartTime = DateTime.Now.Ticks;
bool Parsed = true;
int FailedCount = 0;
for (int i = 0; i < 2000000; i++)
{
if (GetValue(i).Age != i)
{
Parsed = false;
FailedCount++;
}
}
EndTime = DateTime.Now.Ticks;
TimeTaken = new TimeSpan(EndTime - StartTime);
Console.WriteLine("-> " + (Parsed ? "PARSED" : "FAILED (" + FailedCount + " failed index's"));
Console.WriteLine("-> Completed in " + TimeTaken.TotalSeconds + " seconds");
Console.WriteLine("-------------------------------------------------------------------");
Console.WriteLine("*** Reading 10,000 index's at once... ");
StartTime = DateTime.Now.Ticks;
Person[] ChunkOfPeople = new Person[10000];
for (int i = 0; i < 10000; i++)
ChunkOfPeople[i] = GetValue(i);
EndTime = DateTime.Now.Ticks;
TimeTaken = new TimeSpan(EndTime - StartTime);
Console.WriteLine("-> Completed in " + TimeTaken.TotalSeconds + " seconds");
Console.WriteLine("-------------------------------------------------------------------");
Console.WriteLine("*** Reading 100,000 index's at once... ");
StartTime = DateTime.Now.Ticks;
ChunkOfPeople = new Person[100000];
for (int i = 0; i < 100000; i++)
ChunkOfPeople[i] = GetValue(i);
EndTime = DateTime.Now.Ticks;
TimeTaken = new TimeSpan(EndTime - StartTime);
Console.WriteLine("-> Completed in " + TimeTaken.TotalSeconds + " seconds");
Console.WriteLine("-------------------------------------------------------------------");
Console.WriteLine("*** Reading 1,000,000 index's at once... ");
StartTime = DateTime.Now.Ticks;
ChunkOfPeople = new Person[1000000];
for (int i = 0; i < 1000000; i++)
ChunkOfPeople[i] = GetValue(i);
EndTime = DateTime.Now.Ticks;
TimeTaken = new TimeSpan(EndTime - StartTime);
Console.WriteLine("-> Completed in " + TimeTaken.TotalSeconds + " seconds");
}
}
}
There are a number of options, not all of them may be appropriate for your particular project:
Don't use a single file for all the data. Divide the map in smaller "rooms" and store each one in its own file. Load only the "room" the player starts in and preemptively load neighboring "rooms" and unload old ones.
Reduce the number of tiles you need to store. Use procedural generation to create an area's layout.
If you have a 10x10 room with the floor made of a single tile type then don't store 100 separate tiles but instead use a specific marker that says "this area has a 10x10 floor with this tile". If it's a wall then save the start and end positions and the texture type. If you have a multi-tile doodad in the middle of an open field and it's position is not relevant to the story then position it randomly in the field (and save the seed for the random number generator in the map file so next time it will appear in the same place).

How to read last "n" lines of log file [duplicate]

This question already has answers here:
Get last 10 lines of very large text file > 10GB
(21 answers)
Closed 1 year ago.
need a snippet of code which would read out last "n lines" of a log file. I came up with the following code from the net.I am kinda new to C sharp. Since the log file might be
quite large, I want to avoid overhead of reading the entire file.Can someone suggest any performance enhancement. I do not really want to read each character and change position.
var reader = new StreamReader(filePath, Encoding.ASCII);
reader.BaseStream.Seek(0, SeekOrigin.End);
var count = 0;
while (count <= tailCount)
{
if (reader.BaseStream.Position <= 0) break;
reader.BaseStream.Position--;
int c = reader.Read();
if (reader.BaseStream.Position <= 0) break;
reader.BaseStream.Position--;
if (c == '\n')
{
++count;
}
}
var str = reader.ReadToEnd();
Your code will perform very poorly, since you aren't allowing any caching to happen.
In addition, it will not work at all for Unicode.
I wrote the following implementation:
///<summary>Returns the end of a text reader.</summary>
///<param name="reader">The reader to read from.</param>
///<param name="lineCount">The number of lines to return.</param>
///<returns>The last lneCount lines from the reader.</returns>
public static string[] Tail(this TextReader reader, int lineCount) {
var buffer = new List<string>(lineCount);
string line;
for (int i = 0; i < lineCount; i++) {
line = reader.ReadLine();
if (line == null) return buffer.ToArray();
buffer.Add(line);
}
int lastLine = lineCount - 1; //The index of the last line read from the buffer. Everything > this index was read earlier than everything <= this indes
while (null != (line = reader.ReadLine())) {
lastLine++;
if (lastLine == lineCount) lastLine = 0;
buffer[lastLine] = line;
}
if (lastLine == lineCount - 1) return buffer.ToArray();
var retVal = new string[lineCount];
buffer.CopyTo(lastLine + 1, retVal, 0, lineCount - lastLine - 1);
buffer.CopyTo(0, retVal, lineCount - lastLine - 1, lastLine + 1);
return retVal;
}
Had trouble with your code. This is my version. Since its' a log file, something might be writing to it, so it's best making sure you're not locking it.
You go to the end. Start reading backwards until you reach n lines. Then read everything from there on.
int n = 5; //or any arbitrary number
int count = 0;
string content;
byte[] buffer = new byte[1];
using (FileStream fs = new FileStream("text.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
// read to the end.
fs.Seek(0, SeekOrigin.End);
// read backwards 'n' lines
while (count < n)
{
fs.Seek(-1, SeekOrigin.Current);
fs.Read(buffer, 0, 1);
if (buffer[0] == '\n')
{
count++;
}
fs.Seek(-1, SeekOrigin.Current); // fs.Read(...) advances the position, so we need to go back again
}
fs.Seek(1, SeekOrigin.Current); // go past the last '\n'
// read the last n lines
using (StreamReader sr = new StreamReader(fs))
{
content = sr.ReadToEnd();
}
}
A friend of mine uses this method (BackwardReader can be found here):
public static IList<string> GetLogTail(string logname, string numrows)
{
int lineCnt = 1;
List<string> lines = new List<string>();
int maxLines;
if (!int.TryParse(numrows, out maxLines))
{
maxLines = 100;
}
string logFile = HttpContext.Current.Server.MapPath("~/" + logname);
BackwardReader br = new BackwardReader(logFile);
while (!br.SOF)
{
string line = br.Readline();
lines.Add(line + System.Environment.NewLine);
if (lineCnt == maxLines) break;
lineCnt++;
}
lines.Reverse();
return lines;
}
Does your log have lines of similar length? If yes, then you can calculate average length of the line, then do the following:
seek to end_of_file - lines_needed*avg_line_length (previous_point)
read everything up to the end
if you grabbed enough lines, that's fine. If no, seek to previous_point - lines_needed*avg_line_length
read everything up to previous_point
goto 3
memory-mapped file is also a good method -- map the tail of file, calculate lines, map the previous block, calculate lines etc. until you get the number of lines needed
Here is my answer:-
private string StatisticsFile = #"c:\yourfilename.txt";
// Read last lines of a file....
public IList<string> ReadLastLines(int nFromLine, int nNoLines, out bool bMore)
{
// Initialise more
bMore = false;
try
{
char[] buffer = null;
//lock (strMessages) Lock something if you need to....
{
if (File.Exists(StatisticsFile))
{
// Open file
using (StreamReader sr = new StreamReader(StatisticsFile))
{
long FileLength = sr.BaseStream.Length;
int c, linescount = 0;
long pos = FileLength - 1;
long PreviousReturn = FileLength;
// Process file
while (pos >= 0 && linescount < nFromLine + nNoLines) // Until found correct place
{
// Read a character from the end
c = BufferedGetCharBackwards(sr, pos);
if (c == Convert.ToInt32('\n'))
{
// Found return character
if (++linescount == nFromLine)
// Found last place
PreviousReturn = pos + 1; // Read to here
}
// Previous char
pos--;
}
pos++;
// Create buffer
buffer = new char[PreviousReturn - pos];
sr.DiscardBufferedData();
// Read all our chars
sr.BaseStream.Seek(pos, SeekOrigin.Begin);
sr.Read(buffer, (int)0, (int)(PreviousReturn - pos));
sr.Close();
// Store if more lines available
if (pos > 0)
// Is there more?
bMore = true;
}
if (buffer != null)
{
// Get data
string strResult = new string(buffer);
strResult = strResult.Replace("\r", "");
// Store in List
List<string> strSort = new List<string>(strResult.Split('\n'));
// Reverse order
strSort.Reverse();
return strSort;
}
}
}
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine("ReadLastLines Exception:" + ex.ToString());
}
// Lets return a list with no entries
return new List<string>();
}
const int CACHE_BUFFER_SIZE = 1024;
private long ncachestartbuffer = -1;
private char[] cachebuffer = null;
// Cache the file....
private int BufferedGetCharBackwards(StreamReader sr, long iPosFromBegin)
{
// Check for error
if (iPosFromBegin < 0 || iPosFromBegin >= sr.BaseStream.Length)
return -1;
// See if we have the character already
if (ncachestartbuffer >= 0 && ncachestartbuffer <= iPosFromBegin && ncachestartbuffer + cachebuffer.Length > iPosFromBegin)
{
return cachebuffer[iPosFromBegin - ncachestartbuffer];
}
// Load into cache
ncachestartbuffer = (int)Math.Max(0, iPosFromBegin - CACHE_BUFFER_SIZE + 1);
int nLength = (int)Math.Min(CACHE_BUFFER_SIZE, sr.BaseStream.Length - ncachestartbuffer);
cachebuffer = new char[nLength];
sr.DiscardBufferedData();
sr.BaseStream.Seek(ncachestartbuffer, SeekOrigin.Begin);
sr.Read(cachebuffer, (int)0, (int)nLength);
return BufferedGetCharBackwards(sr, iPosFromBegin);
}
Note:-
Call ReadLastLines with nLineFrom starting at 0 for the last line and nNoLines as the number of lines to read back from.
It reverses the list so the 1st one is the last line in the file.
bMore returns true if there are more lines to read.
It caches the data in 1024 char chunks - so it is fast, you may want to increase this size for very large files.
Enjoy!
This is in no way optimal but for quick and dirty checks with small log files I've been using something like this:
List<string> mostRecentLines = File.ReadLines(filePath)
// .Where(....)
// .Distinct()
.Reverse()
.Take(10)
.ToList()
Something that you can now do very easily in C# 4.0 (and with just a tiny bit of effort in earlier versions) is use memory mapped files for this type of operation. Its ideal for large files because you can map just a portion of the file, then access it as virtual memory.
There is a good example here.
As #EugeneMayevski stated above, if you just need an approximate number of lines returned, each line has roughly the same line length and you're more concerned with performance especially for large files, this is a better implementation:
internal static StringBuilder ReadApproxLastNLines(string filePath, int approxLinesToRead, int approxLengthPerLine)
{
//If each line is more or less of the same length and you don't really care if you get back exactly the last n
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
var totalCharsToRead = approxLengthPerLine * approxLinesToRead;
var buffer = new byte[1];
//read approx chars to read backwards from end
fs.Seek(totalCharsToRead > fs.Length ? -fs.Length : -totalCharsToRead, SeekOrigin.End);
while (buffer[0] != '\n' && fs.Position > 0) //find new line char
{
fs.Read(buffer, 0, 1);
}
var returnStringBuilder = new StringBuilder();
using (StreamReader sr = new StreamReader(fs))
{
returnStringBuilder.Append(sr.ReadToEnd());
}
return returnStringBuilder;
}
}
Most log files have a DateTime stamp. Although can be improved, the code below works well if you want the log messages from the last N days.
/// <summary>
/// Returns list of entries from the last N days.
/// </summary>
/// <param name="N"></param>
/// <param name="cSEP">field separator, default is TAB</param>
/// <param name="indexOfDateColumn">default is 0; change if it is not the first item in each line</param>
/// <param name="bFileHasHeaderRow"> if true, it will not include the header row</param>
/// <returns></returns>
public List<string> ReadMessagesFromLastNDays(int N, char cSEP ='\t', int indexOfDateColumn = 0, bool bFileHasHeaderRow = true)
{
List<string> listRet = new List<string>();
//--- replace msFileName with the name (incl. path if appropriate)
string[] lines = File.ReadAllLines(msFileName);
if (lines.Length > 0)
{
DateTime dtm = DateTime.Now.AddDays(-N);
string sCheckDate = GetTimeStamp(dtm);
//--- process lines in reverse
int iMin = bFileHasHeaderRow ? 1 : 0;
for (int i = lines.Length - 1; i >= iMin; i--) //skip the header in line 0, if any
{
if (lines[i].Length > 0) //skip empty lines
{
string[] s = lines[i].Split(cSEP);
//--- s[indexOfDateColumn] contains the DateTime stamp in the log file
if (string.Compare(s[indexOfDateColumn], sCheckDate) >= 0)
{
//--- insert at top of list or they'd be in reverse chronological order
listRet.Insert(0, s[1]);
}
else
{
break; //out of loop
}
}
}
}
return listRet;
}
/// <summary>
/// Returns DateTime Stamp as formatted in the log file
/// </summary>
/// <param name="dtm">DateTime value</param>
/// <returns></returns>
private string GetTimeStamp(DateTime dtm)
{
// adjust format string to match what you use
return dtm.ToString("u");
}

Categories