The following code works when I work with CSV files under 1MB but fails when I try to read 600MB file. Any reason why? Or any fixes?
What I am trying to do is read a large raw CSV file in Visual C# 2010 and manipulate the contents, could be line by line or to memory at one go and export 5 files with certain selections using LINQ. These 5 files are to be used in various processes so need them to be split into 5 different files with very different content.
When the file is small the codes work perfect but when it's too big it gives me the messagebox from Exception handling "Cannot write to source destination". I have tried both ReadAllLines() and ReadLines() Please could you advise me. Thanks.
public void button2_Click(object sender, EventArgs e)
{
string file_name = textBox1.Text.ToString();
// Get the directories to split the file in.
string directoryPath = Path.GetDirectoryName(textBox1.Text.ToString());
if (File.Exists(file_name) == true)
{
try
{
StreamReader readerfile = new StreamReader(file_name);
var BillSummaryQuery1 =
(from line in File.ReadAllLines(file_name)
let linerecord = line.Split(',').ToList()
select line).ToList();
#region Start Writing BillSummary.CSV
//lines removed
#endregion End writing BillSummary.CSV
#region Start writing Notes.CSV
//lines removed
#endregion Notes.CSV
string message =
"Bill Translated Successfully! \r\nFiles located in: " + directoryPath;
MessageBox.Show(message, "Success", MessageBoxButtons.OK, MessageBoxIcon.Information);
}
catch (Exception)
{
string message2 = "Cannot write to source destination";
MessageBox.Show(message2, "Error");
}
}
else
{
MessageBox.Show("No such file exists","Error",MessageBoxButtons.OK,MessageBoxIcon.Error);
}
If you are using a StreamReader, why don't use it ?
public void button2_Click(object sender, EventArgs e)
{
string file_name = textBox1.Text.ToString();
// Get the directories to split the file in.
string directoryPath = Path.GetDirectoryName(textBox1.Text.ToString());
if (File.Exists(file_name) == true)
{
try
{
using (StreamReader reader= new StreamReader(file_name))
{
string line = null;
while ((line = reader.ReadLine()) != null)
{
// Do your stuff
}
}
}
catch (Exception ex)
{
Trace.TraceError(ex.Message);
string message2 = "Cannot write to source destination";
MessageBox.Show(message2, "Error");
}
}
else
{
MessageBox.Show("No such file exists", "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
}
Rolling your own CSV reader is a waste of time unless the files that you're reading are guaranteed to be very simple. Use a pre-existing, tried-and-tested implementation instead.
Related
I'm trying to actively scan recent file inputs from a directory and StreamReading the file to console. I'm having problems finding a solution to using the most recent file input and printing the whole file text lines to console. I'm using FileSystemWatcher for recent inputs.
Here's my code (I'm a beginner):
private static void filesys_created(object sender, FileSystemEventArgs e)
{
Console.WriteLine(DateTime.Now + " : New file input: {0}", e.Name + Enviroment.Newline);
try
{
using (StreamReader sr = new StreamReader(directorypath))
{
string path = directorypath
String line = sr.ReadLine(e.Name);
string filename = e.Name;
Console.WriteLine(line);
}
}
catch (exception)
{
Console.WriteLine("File could not be read to console");
}
}
Only error showing is "No overload for method "ReadLine" takes 1 argument.
& File is not streamreading text to console
You must open the file from FullPath in FileSystemEventArgs
Then, you can write to console the content of the file by ReadToEnd
using (StreamReader sr = new StreamReader(e.FullPath))
{
Console.WriteLine(sr.ReadToEnd());
}
I'm working on a web installer and one of the things I have currently is
void MoveFiles()
{
lbldlstatus.Text = "Moving Files";
string InstallDirectory = Directory.GetCurrentDirectory() + "/DoxramosRepack-master";
DirectoryInfo d = new DirectoryInfo(InstallDirectory);
foreach(var file in d.GetFiles("*"))
{
try
{
if (File.Exists(file.Name)) {
File.Delete(file.Name);
}
Directory.Move(file.FullName, file.Name);
Cleanup();
}
catch(Exception e)
{
MessageBox.Show(e.ToString());
lbldlstatus.Text = "Repack Installation Failed";
}
}
}
void Cleanup()
{
lbldlstatus.Text = "Cleaning Up Files";
try
{
if (File.Exists("Repack.zip"))
{
File.Delete("Repack.zip");
}
if(Directory.Exists("DoxramosRepack-master"))
{
Directory.Delete("DoxramosRepack-master");
}
lbldlstatus.Text = "Repack Installed Successfully";
}
When I get to Cleanup() I have a System.IO.IOException.
Process cannot access the file Repack.zip because it being used by
another process.
The full code runs
Download->Extract->Move->Cleanup.
I'm not sure what process is being used, but I'm looking to find a way for each process to wait for the previous to finish before starting.
According to extract code below
void Extract()
{
string zipPath = #"Repack.zip";
string extractPath = #".";
try
{
using (ZipFile unzip = ZipFile.Read(zipPath))
{
unzip.ExtractAll(extractPath);
lbldlstatus.Text = "Extracting Files";
MoveFiles();
}
}
catch (ZipException e)
{
MessageBox.Show(e.ToString());
lbldlstatus.Text = "Repack Installation Failed";
}
}
You are calling the move files before you are finish with the zip file. Seeing as the move file method is responsible for calling the clean up function then you should make sure that the zip file is already disposed of before trying to delete it.
void Extract()
{
string zipPath = #"Repack.zip";
string extractPath = #".";
try
{
using (ZipFile unzip = ZipFile.Read(zipPath))
{
unzip.ExtractAll(extractPath);
lbldlstatus.Text = "Extracting Files";
}
MoveFiles();
}
catch (ZipException e)
{
MessageBox.Show(e.ToString());
lbldlstatus.Text = "Repack Installation Failed";
}
}
Clean up should also be called after everything has been moved. Currently the example code is calling it repeatedly in the for loop.
The code which you pasted on pastebin is different from what you have posted here. The code in pastebin never calls cleanup.
Anyways the problem is because you are calling MoveFiles() from within the using block here:
using (ZipFile unzip = ZipFile.Read(zipPath))
{
unzip.ExtractAll(extractPath);
lbldlstatus.Text = "Extracting Files";
MoveFiles();
}
Move it outside the using block.
List<string> errorLog = new List<string>();
foreach (DirectoryInfo dir in directories)
{
try
{
dir.Delete(true);
}
catch (System.IO.IOException msg)
{
code = 5;
errorLog.Add(String.Concat(dir.FullName, " ", msg.Message));
Console.WriteLine("Error Removing the directory: {0}", dir.FullName);
}
}
I have a for each loop that will go through a list of directories and remove them, but keep the parent directory. Should an error occur, I would like to log it. I created a list and in the catch add the errors. At the end, I can check the length of errorLog list and if it's more than zero, I can print them. I've seen posts where they call using and streamwriter within the catch, but what happens if something were to occur while writing the error log?
Is what I'm doing considered bad practice? If so, what should I do ?
I think you've got the right idea. There are many solutions, but recently I tried rerouting the console output to file and it worked pretty well. Regarding your solution, it would look like:
try
{
FileStream oStream;
StreamWriter sWriter;
var oldOut = Console.Out;
var desktopPath = Environment.GetFolderPath(Environment.SpecialFolder.Desktop);
const string outputFileName = "\\errorlog.txt";
var fullPath = string.Concat(desktopPath, outputFileName);
Console.SetOut(sWriter);
foreach (DirectoryInfo dir in directories)
{
try
{
dir.Delete(true);
}
catch(System.IO.IOException msg)
{
code = 5;
errorLog.Add(String.Concat(dir.FullName," ",msg.Message));
Console.WriteLine("Error Removing the directory: {0}", dir.FullName);
}
}
}
catch(Exception e)
{
//handle error with streams or file
}
finally
{
//ensures that we close the connections and such
Console.SetOut(oldOut);
sWriter.Close();
oStream.Close();
}
The finally block ensures that if any unhandled thing happens, the stream and file will still be closed.
I have the following code which...
Checks if a folder exists
If it exists, check if a file exists
if file exists, read all the lines from the file
once all the line has been read, show the length of the line in a messagebox
Code:
private void button2_Click(object sender, EventArgs e)
{
strPath = #"C:\QRXS";
string strFile = #"C:\QRXS\download.lst";
if (Directory.Exists(strPath))
{
try
{
if (File.Exists(strFile))
{
try
{
ln = File.ReadAllLines(strPath);
}
catch (Exception ex)
{
// inform user or log depending on your usage scenario
MessageBox.Show(ex.Message, "FILE ACCESS");
}
if (ln != null)
{
MessageBox.Show(ln.Length + "");
// do something with lines
}
}
}
catch (Exception ce)
{
MessageBox.Show(ce.Message, "FOLDER ACCESS");
}
}
}
Everytime I run the application (used Run as Administrator as well), the following line keeps being invoked:
MessageBox.Show(ex.Message, "FILE ACCESS");
How can I fix it?
Replace:
File.ReadAllLines(strPath);
with:
File.ReadAllLines(strFile);
Reason: strPath denotes a directory. You're trying to read its contents as if it were a file, and that obviously won't work.
You need to use :
File.ReadAllLines(strFile);
When I try to open a .txt file it only shows its location in my textbox.
I am out of ideas:( hope you can help me...
code:
private void OpenItem_Click(object sender, EventArgs e)
{
openFileDialog1.ShowDialog();
System.IO.StringReader OpenFile = new System.IO.StringReader(openFileDialog1.FileName);
richTextBox1.Text = OpenFile.ReadToEnd();
OpenFile.Close();
}
A StringReader reads the characters from the string you pass to it -- in this case, the file's name. If you want to read the contents of the file, use a StreamReader:
var OpenFile = new System.IO.StreamReader(openFileDialog1.FileName);
richTextBox1.Text = OpenFile.ReadToEnd();
Use File.ReadAllText
richTextBox1.Text = File.ReadAllText(openFileDialog1.FileName);
I'd use the File.OpenText() method for reading text-files. You should also use using statements to properly dispose the object.
if(openFileDialog1.ShowDialog() == DialogResult.OK)
{
try
{
// Make sure a file was selected
if ((myStream = openFileDialog1.OpenFile()) != null) {
// Open stream
using (StreamReader sr = File.OpenText(openFileDialog1.FileName))
{
// Read the text
richTextBox1.Text = sr.ReadToEnd();
}
}
}
catch (Exception ex)
{
MessageBox.Show("An error occured: " + ex.Message);
}
}
That's easy. This is what you need to do:
1) Put using System.IO; above namespace.
2) Create a new method:
public static void read()
{
StreamReader readme = null;
try
{
readme = File.OpenText(#"C:\path\to\your\.txt\file.txt");
Console.WriteLine(readme.ReadToEnd());
}
// will return an invalid file name error
catch (FileNotFoundException errorMsg)
{
Console.WriteLine("Error, " + errorMsg.Message);
}
// will return an invalid path error
catch (Exception errorMsg)
{
Console.WriteLine("Error, " + errorMsg.Message);
}
finally
{
if (readme != null)
{
readme.Close();
}
}
}
3) Call it in your main method: read();
4) You're done!