We Keep Coding

Placing a string in memory and saving it later

I have a program that creates a huge string, too large for StringBuilder to handle. I want to store it in memory, then, at a later date, save it to a user defined directory.
I have tried a StreamWriter but I need to declare the directory as I create it and that is problem because I get the directory later on, if the user selects the "Save to text file".
Basically I need it to:
-Create instance of whatever it is (public and in a class scope, can not be in a method scope)
-Add a string to instance
-Then, later on, whenever the user decides, save that to a file. I am aware it will take up a large amount of RAM and I believe it will only be around 50-100Mb, which isn't too bad.
public class ImageProcessor
{
MemoryStream ms = new MemoryStream();
StreamWriter sw;
public Bitmap Rendering(string bmpPath)
{
sw = new StreamWriter(ms, System.Text.Encoding.UTF8);
using (ms)
sw.Write(Convert.ToString(ArGBformat)); //Error is here
}
}
`

Here's a code snippet that may help you. I just tested it in VS 2012:
using (MemoryStream ms = new MemoryStream)
{
StreamWriter sw = new StreamWriter(ms, System.Text.Encoding.UTF8)
sw.WriteLine("This is a test.");
sw.WriteLine("This is a second line.");
sw.Flush();
using (FileStream fs = new FileStream("Test.txt", FileMode.Create))
{
ms.CopyTo(fs);
}
sw.Close();
}
The file contents are:
This is a test.
This is a second line.
You'll want to modify this to fit your program's design, but the basic idea is using a StreamWriter to write the text to the MemoryStream, and then writing the MemoryStream to the file with the MemoryStream.CopyTo method (which takes a Stream).
Be careful with how you construct things - if you close the StreamWriter it will close the MemoryStream as well (the first time I tested this I had the StreamWriter in a using block inside the MemoryStream using block, and then got an error trying to access a closed stream).
Hopefully this will at least get you going in the right direction.
EDIT
You'll need to initialize the StreamWriter in a method, not as a field variable.
Something like this:
public class ImageProcessor
{
MemoryStream ms = new MemoryStream();
StreamWriter sw;
public Bitmap Rendering(string bmpPath)
{
sw = new StreamWriter(ms, System.Text.Encoding.UTF8);
}
}

Using StreamReader / StreamWriter to grab logs causes program to cease responding

I'm attempting to use StreamReader and StreamWriter to grab a temporary output log (.txt format) from another application.
The output log is always open and constantly written to.
Unhelpfully if the application closes or crashes, the log file ends up deleted - hence the need for a tool that can grab the information from this log and save it.
What my program currently does is:
Create a new .txt file, and stores the path of that file as the
string "destinationFile".
Finds the .txt log file to read, and stores the path of that file as
the string "sourceFile"
It then passes those two strings to the method below.
Essentially I'm trying to read the sourceFile one line at a time.
Each time one line is read, it is appended to destinationFile.
This keeps looping until the sourceFile no longer exists (i.e. the application has closed or crashed and deleted its log).
In addition, the sourceFile can get quite big (sometimes 100Mb+), and this program may be handling more than one log at a time.
Reading the whole log rather than line by line will most likely start consuming a fair bit of memory.
private void logCopier(string sourceFile, string destinationFile)
{
while (File.Exists(sourceFile))
{
string textLine;
using (var readerStream = File.Open(sourceFile,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
using (var reader = new StreamReader(readerStream))
{
while ((textLine = reader.ReadLine()) != null)
{
using (FileStream writerStream = new FileStream(destinationFile,
FileMode.Append,
FileAccess.Write))
using (StreamWriter writer = new StreamWriter(writerStream))
{
writer.WriteLine(textLine);
}
}
}
}
}
The problem is that my WPF application locks up and ceases to respond when it reaches this code.
To track down where, I put a MessageBox just before the writerStream line of the code to output what the reader was picking up.
It was certainly reading the log file just fine, but there appears to be a problem with writing it to the file.
As soon as it reaches the using (FileStream writerStream = new FileStream part of the code, it stops responding.
Is using the StreamWriter in this manner not valid, or have I just gone and dome something silly in the code?
Am also open to a better solution than what I'm trying to do here.

Simply what I understand is you need to copy a file from source to destination which may be deleted at any time.
I'll suggest you to use FileSystemWatcher to watch for source file changed event, then just simply copy the whole file from source to destination using File.Copy.

I've just solved the problem, and the issue was indeed something silly!
When creating the text file for the StreamWriter, I had forgotten to use .Dispose();. I had File.Create(filename); instead of File.Create(filename).Dispose(); This meant the text file was already open, and the StreamWriter was attempting to write to a file that was locked / in use.
The UI still locks up (as expected), as I've yet to implement this on a new thread as SteenT mentioned. However the program no longer crashes and the code correctly reads the log and outputs to a text file.
Also after a bit of refinement, my log reader/writer code now looks like this:
private void logCopier(string sourceFile, string destinationFile)
{
int num = 1;
string textLine = String.Empty;
long offset = 0L;
while (num == 1)
{
if (File.Exists(sourceFile))
{
FileStream stream = new FileStream(sourceFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
using (new StreamReader(stream))
{
stream.Seek(offset, SeekOrigin.Begin);
TextReader reader2 = new StreamReader(stream);
while ((textLine = reader2.ReadLine()) != null)
{
Thread.Sleep(1);
StreamWriter writer = new StreamWriter(destinationFile, true);
writer.WriteLine(textLine);
writer.Flush();
writer.Close();
offset = stream.Position;
}
continue;
}
}
else
{
num = 0;
}
}
}
Just putting this code up here in case anyone else is looking for something like this. :)

Why does my C# program only write 251 rows to a file?

I've got a list of 369 different names and I want to print these names into a csv file. All's going well until I take a look at the outputted csv file and it only has 251 rows. I've tried outputting to a .txt instead, and still it only outputs 251 rows. Ive stepped through with the debugger and it is still calling writer.WriteLine() 369 times.
Is there some sort of writing restriction in place? If so, why 251? How do I write all 369 names?
Here's my code just in case:
List<String> names = new List<String>();
//Retrieve names from a separate source.
var writer = new StreamWriter(File.OpenWrite(#"C:names.txt"));
for (int i = 0; i < names.Count; i++ )
{
System.Console.WriteLine(names[i].ToString());
writer.WriteLine(names[i].ToString());
}
System.Console.Write(names.Count);
The output on the console shows all 369 names and the names.Count prints 369.

You need to close your StreamWriter, the best way is to use a using block like so:
using(StreamWriter writer = new StreamWriter(File.OpenWrite("C:\\names.txt")) {
// code here
}
The using block will always call the .Dispose method of StreamWriter which has the effect of flushing the stream. Presently you have buffered-but-unwritten data in your StreamWriter instance.

You do not show anywhere that you properly close writer. If your program terminates abnormally, the writer would never be flushed to disk.
Try making use of a using block.
// NOTE: The is should be C:\names.txt. The posted code is missing a \
using (var writer = new StreamWriter(File.OpenWrite(#"C:names.txt")))
{
// Your code here
}

You have to flush buffer after last write. Put writer inside using statement.
Dispose method of writer flushes buffer. You can also call writer.Flush(). But since you still have to make sure that writer is disposed just put it in a using statement as other suggested.

List<String> names = new List<String>();
var sb = new StringBuilder()
//Retrieve names from a separate source.
for (int i = 0; i < names.Count; i++ )
{
System.Console.WriteLine(names[i].ToString());
sb.WriteLine(names[i].ToString());
}
using (var writer = new StreamWriter(File.OpenWrite(#"C:\names.txt")))
{
writer.WriteLine(sb.ToString());
}

Streamwriter is cutting off my last couple of lines sometimes in the middle of a line?

Here is my code. :
FileStream fileStreamRead = new FileStream(pathAndFileName, FileMode.OpenOrCreate, FileAccess.Read, FileShare.None);
FileStream fileStreamWrite = new FileStream(reProcessedFile, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None);
StreamWriter sw = new StreamWriter(fileStreamWrite);
int readIndex = 0;
using (StreamReader sr = new StreamReader(fileStreamRead))
{
while (!sr.EndOfStream) {
Console.WriteLine("eof" + sr.EndOfStream);
readIndex++;
Console.WriteLine(readIndex);
string currentRecord = "";
currentRecord = sr.ReadLine();
if (currentRecord.Trim() != "")
{
Console.WriteLine("Writing " + readIndex);
sw.WriteLine(currentRecord);
}
else {
Console.WriteLine("*******************************************spaces ***********************");
}
}
It is cutting off 2 lines with one test file and half a line, and then 1 line and half a line with the other test file I am running it against.
I am not a streamreader/writer expert you can probably see.
Any ideas or suggestions would be greatly appreciated as this is driving me batty. I am sure it is me using these incorrectly.

You are missing Flush/Close or simply using for your writer.
using(FileStream fileStreamWrite =
new FileStream(reProcessedFile, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None);
{
using(StreamWriter sw = new StreamWriter(fileStreamWrite))
{
// .... write everything here
}
}

Right after the closing brace of the using statement, do this:
sw.Flush();
sw.Close();
There, that should do it.

You need to Flush your StreamWriter. A StreamWriter has a buffer, and it writes to disk only when the buffer is full. By flushing at the end you make sure all the text in the buffer is written to the disk.

In addition to other answers (use using, and/or flush/close), would say that they do not actually respond to the question: "why it may cut several lines."
I have an idea on subject that it is related to a fact that you use StreamReader and call EndOfStream twice: in a while loop header, and another inside it.
The only possible way of understanding if the stream ends is try to read some data from it. So I suspect EnfOfStream does it, and reading it twice, may create a problem in stream processing.
To resolve an issue:
Or use simple TextReader, considering that you are reading text file (seems to me)
Or change your logic to call only once, so no more call to Console.WriteLine("eof" + sr.EndOfStream);
Or change your logic, so do not use EndOFStream at all, but read line by line till the line is null.

You're not using StreamWriter properly. Also, since you're always reading lines, I would use a method that already does all that for you (and manages it properly).
using (var writer = new StreamWriter("path"))
{
foreach(var line in File.ReadLines("path"))
{
if (string.IsNullOrWhiteSpace(line))
{ /**/ }
else
{ /**/ }
}
}
... or ...
/* do not call .ToArray or something that will evaluate this _here_, let WriteAllLines do that */
var lines = File.ReadLines("path")
.Select(line => string.IsNullOrWhiteSpace(line) ? Stars : line);
var encoding = Encoding.ASCII; // whatever is appropriate for you.
File.WriteAllLines("path", lines, encoding);

How to append data to a binary file?

I have a binary file to which I want to append a chunk of data at the end of the file, how can I achieve this using C# and .net? Also is there anything to consider when writing to the end of a binary file? Thanks a lot for your help.

private static void AppendData(string filename, int intData, string stringData, byte[] lotsOfData)
{
using (var fileStream = new FileStream(filename, FileMode.Append, FileAccess.Write, FileShare.None))
using (var bw = new BinaryWriter(fileStream))
{
bw.Write(intData);
bw.Write(stringData);
bw.Write(lotsOfData);
}
}

You should be able to do this via the Stream:
using (FileStream data = new FileStream(path, FileMode.Append))
{
data.Write(...);
}
As for considerations - the main one would be: does the underlying data format support append? Many don't, unless it is your own raw data, or text etc. A well-formed xml document doesn't support append (without considering the final end-element), for example. Nor will something like a Word document. Some do, however. So; is your data OK with this...

Using StreamWriter and referencing DotNetPerls, make sure to add the True boolean to the StreamWriter constructor, if otherwise left blank, it'll overwrite as usual:
using System.IO;
class Program
{
static void Main()
{
// 1: Write single line to new file
using (StreamWriter writer = new StreamWriter("C:\\log.txt", true))
{
writer.WriteLine("Important data line 1");
}
// 2: Append line to the file
using (StreamWriter writer = new StreamWriter("C:\\log.txt", true))
{
writer.WriteLine("Line 2");
}
}
}
Output
(File "log.txt" contains these lines.)
Important data line 1
Line 2
This is the solution that I was actually looking for when I got here from Google, although it wasn't a binary file though, hope it helps someone else.