I use the following code to write log file:
Encoding enc = Encoding.GetEncoding(932);
MemoryStream msLog = new MemoryStream();
StreamWriter swLog = new StreamWriter(msLog, enc);
swLog.WriteLine("Line Number,Error,Additional Information"); //log header
After some complex processing I'd like to know whether there any log line was added except the header. Obviously, one way is to set some boolean variable to true whenever I use swLog.WriteLine(), but because of long and complex code I'd like to avoid this approach. How can I easily check line count of memory stream?
As you noted, there are other better ways to do this. However, here is a direct answer to your question:
First, make sure that the StreamWriter has flushed the data into the stream like this:
swLog.Flush();
Then, you can use the following method to detect if the MemoryStream has more than one line:
private bool HasMoreThanNumberOfLines(Stream stream, Encoding enc, int number_of_lines)
{
long current_position = stream.Position;
stream.Position = 0;
try
{
using(StreamReader sr = new StreamReader(stream, enc, true, 1024, true))
{
for (int i = 0; i < number_of_lines + 1 ; i++)
{
string line = sr.ReadLine();
if (line == null)
return false;
}
}
return true;
}
finally
{
stream.Position = current_position;
}
}
Please note that I am using a special constructor of StreamReader to make sure that it does not close the underlying stream (stream) when it is disposed of.
Notice also how this method saves the current position of the stream, and then restores it after executing its logic so that the StreamWriter would continue to work normally.
You can use this method like this:
var has_another_line = HasMoreThanNumberOfLines(msLog, enc, 1);
Please note that this is not thread-safe. I am assuming that the stream will be accessed by a single thread at any point in time. You would need to put some locks to make it thread-safe.
Related
Do I need a Seek call in this code?
// Assume bytes = byte[] of some bytes
using (var memoryStream = new MemoryStream(bytes))
{
memoryStream.Seek(0, SeekOrigin.Begin);
return new BinaryFormatter().Deserialize(memoryStream);
}
No, there is no need to Seek on stream that was just created.
You need to Seek or set Position is you wrote something to the stream before.
I.e. common question is "how to return MemoryStream with some serialized data" - you need to write data to the stream and than Seek to the beginning of the stream so Read will start from the beginning and not the last position of Write (hence always saying that there nothing left to read). Sample question - Can't create MemoryStream.
No, you don't have to. For proving that, you can check the constructor code:
public MemoryStream(byte[] buffer, bool writable)
{
if (buffer == null) throw new ArgumentNullException("buffer", Environment.GetResourceString("ArgumentNull_Buffer"));
Contract.EndContractBlock();
_buffer = buffer;
_length = _capacity = buffer.Length;
_writable = writable;
_exposable = false;
_origin = 0;
_isOpen = true;
}
Seek changes _position (in your example to 0), which is not assigned in the constructor, so upon construction of the object Position will have the default long value of 0.
It's a different story though if you perform further operations on the stream that could change its Position before reading from it.
I'm trying to read file into a string and rewrite that string into a new file, but there is a small check, if the current character is one of special characters that I want to rewrite.
I've debugged it, and the code seems to work fine, but the output file is empty.. I think I'm missing something... but what?
StreamWriter file = new StreamWriter(newname, true);
char current;
int j;
string CyrAlph = "йцукен";
string LatAlph = "ysuken";
string text = File.ReadAllText(filename);
for (int i = 0; i < text.Length; i++)
{
if (CyrAlph.IndexOf(text[i]) != -1)
{
j = CyrAlph.IndexOf(text[i]);
current = LatAlph[j];
}
else current = text[i];
file.Write(current);
}
What happens if you set file.AutoFlush = true after your StreamWriter instantiation or call file.Close at the end of writing everything or you can instantiate your StreamWriter in a using statement. My guess is that it is empty because the buffer needs flushed
You're missing a stream flush. The standard pattern is to add a using statement around the allocation of the StreamWriter. That also takes care of closing the file and releasing the operating system's file handle:
using (StreamWriter file = new StreamWriter(path, true))
{
// Work with your file here
} // After this block, you have "disposed" of the file object.
// That takes care of flushing the stream and releasing the file handle
The using statement has the added benefit, over explicitly closing the stream, of disposing the stream correctly even in the case of an exception within the block.
StreamWriter implements IDisposable. You "have" to Dispose it after using it. To do so, use a using statement. This will automatically flushes and closes the stream at the end of the using body.
using(StreamWriter file = new StreamWriter(newname,true))
{
char current;
int j;
string CyrAlph="йцукен";
string LatAlph = "ysuken";
string text = File.ReadAllText(filename);
for (int i = 0; i < text.Length; i++)
{
if (CyrAlph.IndexOf(text[i]) != -1)
{
j=CyrAlph.IndexOf(text[i]);
current = LatAlph[j];
}
else current=text[i];
file.Write(current);
}
}
Here is my code. :
FileStream fileStreamRead = new FileStream(pathAndFileName, FileMode.OpenOrCreate, FileAccess.Read, FileShare.None);
FileStream fileStreamWrite = new FileStream(reProcessedFile, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None);
StreamWriter sw = new StreamWriter(fileStreamWrite);
int readIndex = 0;
using (StreamReader sr = new StreamReader(fileStreamRead))
{
while (!sr.EndOfStream) {
Console.WriteLine("eof" + sr.EndOfStream);
readIndex++;
Console.WriteLine(readIndex);
string currentRecord = "";
currentRecord = sr.ReadLine();
if (currentRecord.Trim() != "")
{
Console.WriteLine("Writing " + readIndex);
sw.WriteLine(currentRecord);
}
else {
Console.WriteLine("*******************************************spaces ***********************");
}
}
It is cutting off 2 lines with one test file and half a line, and then 1 line and half a line with the other test file I am running it against.
I am not a streamreader/writer expert you can probably see.
Any ideas or suggestions would be greatly appreciated as this is driving me batty. I am sure it is me using these incorrectly.
You are missing Flush/Close or simply using for your writer.
using(FileStream fileStreamWrite =
new FileStream(reProcessedFile, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None);
{
using(StreamWriter sw = new StreamWriter(fileStreamWrite))
{
// .... write everything here
}
}
Right after the closing brace of the using statement, do this:
sw.Flush();
sw.Close();
There, that should do it.
You need to Flush your StreamWriter. A StreamWriter has a buffer, and it writes to disk only when the buffer is full. By flushing at the end you make sure all the text in the buffer is written to the disk.
In addition to other answers (use using, and/or flush/close), would say that they do not actually respond to the question: "why it may cut several lines."
I have an idea on subject that it is related to a fact that you use StreamReader and call EndOfStream twice: in a while loop header, and another inside it.
The only possible way of understanding if the stream ends is try to read some data from it. So I suspect EnfOfStream does it, and reading it twice, may create a problem in stream processing.
To resolve an issue:
Or use simple TextReader, considering that you are reading text file (seems to me)
Or change your logic to call only once, so no more call to Console.WriteLine("eof" + sr.EndOfStream);
Or change your logic, so do not use EndOFStream at all, but read line by line till the line is null.
You're not using StreamWriter properly. Also, since you're always reading lines, I would use a method that already does all that for you (and manages it properly).
using (var writer = new StreamWriter("path"))
{
foreach(var line in File.ReadLines("path"))
{
if (string.IsNullOrWhiteSpace(line))
{ /**/ }
else
{ /**/ }
}
}
... or ...
/* do not call .ToArray or something that will evaluate this _here_, let WriteAllLines do that */
var lines = File.ReadLines("path")
.Select(line => string.IsNullOrWhiteSpace(line) ? Stars : line);
var encoding = Encoding.ASCII; // whatever is appropriate for you.
File.WriteAllLines("path", lines, encoding);
I need to read the first line from a stream to determine file's encoding, and then recreate the stream with that Encoding
The following code does not work correctly:
var r = response.GetResponseStream();
var sr = new StreamReader(r);
string firstLine = sr.ReadLine();
string encoding = GetEncodingFromFirstLine(firstLine);
string text = new StreamReader(r, Encoding.GetEncoding(encoding)).ReadToEnd();
The text variable doesn't contain the whole text. For some reason the first line and several lines after it are skipped.
I tried everything: closing the StreamReader, resetting it, calling a separate GetResponseStream... but nothing worked.
I can't get the response stream again as I'm getting this file from the internet, and redownloading it again would be bad performance wise.
Update
Here's what GetEncodingFromFirstLine() looks like:
public static string GetEncodingFromFirstLine(string line)
{
int encodingIndex = line.IndexOf("encoding=");
if (encodingIndex == -1)
{
return "utf-8";
}
return line.Substring(encodingIndex + "encoding=".Length).Replace("\"", "").Replace("'", "").Replace("?", "").Replace(">", "");
}
...
// true
Assert.AreEqual("windows-1251", GetEncodingFromFirstLine(#"<?xml version=""1.0"" encoding=""windows-1251""?>"));
** Update 2 **
I'm working with XML files, and the text variable is parsed as XML:
var feedItems = XElement.Parse(text);
Well you're asking it to detect the encoding... and that requires it to read data. That's reading it from the underlying stream, and you're then creating another StreamReader around the same stream.
I suggest you:
Get the response stream
Retrieve all the data into a byte array (or MemoryStream)
Detect the encoding (which should be performed on bytes, not text - currently you're already assuming UTF-8 by creating a StreamReader)
Create a MemoryStream around the byte array, and a StreamReader around that
It's not clear what your GetEncodingFromFirstLine method does... or what this file really is. More information may make it easier to help you.
EDIT: If this is to load some XML, don't reinvent the wheel. Just give the stream to one of the existing XML-parsing classes, which will perform the appropriate detection for you.
You need to change the current position in the stream to the beginning.
r.Position = 0;
string text = new StreamReader(r, Encoding.GetEncoding(encoding)).ReadToEnd();
I found the answer to my question here:
How can I read an Http response stream twice in C#?
Stream responseStream = CopyAndClose(resp.GetResponseStream());
// Do something with the stream
responseStream.Position = 0;
// Do something with the stream again
private static Stream CopyAndClose(Stream inputStream)
{
const int readSize = 256;
byte[] buffer = new byte[readSize];
MemoryStream ms = new MemoryStream();
int count = inputStream.Read(buffer, 0, readSize);
while (count > 0)
{
ms.Write(buffer, 0, count);
count = inputStream.Read(buffer, 0, readSize);
}
ms.Position = 0;
inputStream.Close();
return ms;
}
I have a text file that I want to read line by line and record the position in the text file as I go. After reading any line of the file the program can exit, and I need to resume reading the file at the next line when it resumes.
Here is some sample code:
using (FileStream fileStream = new FileStream("Sample.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
fileStream.Seek(GetLastPositionInFile(), SeekOrigin.Begin);
using (StreamReader streamReader = new StreamReader(fileStream))
{
while (!streamReader.EndOfStream)
{
string line = streamReader.ReadLine();
DoSomethingInteresting(line);
SaveLastPositionInFile(fileStream.Position);
if (CheckSomeCondition())
{
break;
}
}
}
}
When I run this code, the value of fileStream.Position does not change after reading each line, it only advances after reading a couple of lines. When it does change, it increases in multiples of 1024. Now I assume that there is some buffering going on under the covers, but how can I record the exact position in the file?
It's not FileStream that's responsible - it's StreamReader. It's reading 1K at a time for efficiency.
Keeping track of the effective position of the stream as far as the StreamReader is concerned is tricky... particularly as ReadLine will discard the line ending, so you can't accurately reconstruct the original data (it could have ended with "\n" or "\r\n"). It would be nice if StreamReader exposed something to make this easier (I'm pretty sure it could do so without too much difficulty) but I don't think there's anything in the current API to help you :(
By the way, I would suggest that instead of using EndOfStream, you keep reading until ReadLine returns null. It just feels simpler to me:
string line;
while ((line = reader.ReadLine()) != null)
{
// Process the line
}
I would agree with Stefan M., it is probably the buffering which is causing the Position to be incorrect. If it is just the number of characters that you have read that you want to track than I suggest you do it yourself, as in:
using(FileStream fileStream = new FileStream("Sample.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
fileStream.Seek(GetLastPositionInFile(), SeekOrigin.Begin);
/**Int32 position = 0;**/
using(StreamReader streamReader = new StreamReader(fileStream))
{
while(!streamReader.EndOfStream)
{
string line = streamReader.ReadLine();
/**position += line.Length;**/
DoSomethingInteresting(line);
/**SaveLastPositionInFile(position);**/
if(CheckSomeCondition())
{
break;
}
}
}
}
Provide that your file is not too big, why not read the whole thing in big chuncks and then manipulate the string - probably faster than the stop and go i/o.
For example,
//load entire file
StreamReader srFile = new StreamReader(strFileName);
StringBuilder sbFileContents = new StringBuilder();
char[] acBuffer = new char[32768];
while (srFile.ReadBlock(acBuffer, 0, acBuffer.Length)
> 0)
{
sbFileContents.Append(acBuffer);
acBuffer = new char[32768];
}
srFile.Close();