Reading multiple .csv files C# - c#

I've a program where i need to read in multiple .csv files from a directory, take some info from each one and then create one large .csv file. However i'm having problems reading them in but not sure why. I have this piece of code in my main method:
string sourceDirectory = #"sourceDirectory/test";
var csvFiles = Directory.EnumerateFiles(sourceDirectory, "*.csv", SearchOption.AllDirectories);
foreach (string currentFile in csvFiles)
{
readFile(currentFile);
}
And then the following in my readFile method:
public static void readFile(string currentFile)
{
StreamWriter writer = new StreamWriter(#"destinationFile.csv");
StreamReader reader = new StreamReader(currentFile);
while(**){
object[] array;
array = new object[11];
array[0] = info1;
array[1] = info2;
array[2] = info3;
//........
writer.WriteLine(string.Join(", ", array));
}
reader.DiscardBufferedData();
writer.Close();
reader.Close();
Without the while loop it only reads in one line of the file, understanably. I can't seem to understand what or even if the while loop should contain. If it was a .txt file i would simply put while ((line = reader.ReadLine()) != null). My code never seems to read in more than one .csv file from the direcotry but there are 6 .csv files in there.
The only data i really need from them is to count certain lines between dates(one of the .csv columns).

Related

File is already use in some other process

Here I want to delete line in a textfiles containg specific string like "21309#003" where item1 is a filename but It shows runtime exception that item1 (file) is already use in some process.How I Solve this problem.I am new in .net C#.
private void button1_Click(object sender, EventArgs e)
{
var selectedItems = listBox1.SelectedItems.Cast<String>().ToList();
foreach (var item in selectedItems)
{
listBox1.Items.Remove(item);
}
foreach (var item1 in selectedItems)
{
listBox1.Items.Remove(item1);
string line = null;
//string line_to_delete = "the line i want to delete";
using (StreamReader reader = new StreamReader(item1))
//item1= "C:\\IMP2711\\textpresent.txt"
{
using (StreamWriter writer = new StreamWriter(item1))
{
while ((line = reader.ReadLine()) != null)
{
//if (String.Compare(line, #"*21349#003*") == 0)
//if (!line.Contains("21349#003") )
if (!line.StartsWith("21349#003"))
{**strong text**
writer.WriteLine(line);
}
}
}
You are reading and writing to the same file at the same time.
var item2 = item1;
If the file is not to big you can read the lines into memory and then write the lines you want to keep back to the file. We can even simplify your code a little bit.
File.WriteAllLines(item1,
File.ReadLines(item1).Where(l => !l.StartsWith("21349#003")).ToList());
Another option if the file is very large is to write to a temporary file. Delete the original and then rename the temporary.
var tmp = Path.GetTempFileName();
File.WriteAllLines(tmp, File.ReadLines(item1).Where(l => !l.StartsWith("21349#003")));
File.Delete(item1);
File.Move(tmp, item1);
If your file is small first read it to memory and then try to write on it, you have two stream on the same file, a file can share between multiple streams but you can not modify a file when it is open by another stream, if your file is huge and you can not moved to memory you can create a temp file and write to temp file when your reading finished replacing original file with temp file and removing temp file.
There's some process that's locking the file c:\imp2711\textpresent.txt. You have to find and kill it.
To find it out, please refer to this question: https://superuser.com/questions/117902/find-out-which-process-is-locking-a-file-or-folder-in-windows

Adding to listbox from multiple files and deleting doubled items

As you can read in the title, I am trying to add listbox items to a listbox from multiple files. But I don't know how to read from all these files and how to delete the doubled lines (as some txt-files contain the same information).
A new file gets added every day, so I can't just read them all manually.
My code so far:
string directory = System.AppDomain.CurrentDomain.BaseDirectory;
DirectoryInfo dinfo = new DirectoryInfo(directory);
FileInfo[] Files = dinfo.GetFiles("*.txt");
First of all you need to identify every file you need to read.
Once you have all the files you need to read the data from each file into some form of storage for example a DataTable.
Once you have filled the DataTable you will need to populate the ListBox with the data.
From what you've got so far it looks like your next step will be to collect the data from each of the files (we can deal with removing duplicates afterwards).
So perhaps:
HashSet<something> myCollection = new HashSet<something>();
// perhaps <something> is just a string?
foreach (var file in Files)
{
// Collect what you need and pop it in the collection
}
// Remove duplicates
To get the info out of the files you'll probably need a StreamReader.
Fore removal of duplicates try HashSets.
you can try this code:
in this code all unique data will be stored in lstData and you can bind your control using this
string directory = System.AppDomain.CurrentDomain.BaseDirectory;
DirectoryInfo dinfo = new DirectoryInfo(directory);
FileInfo[] Files = dinfo.GetFiles("*.txt");
List<string> lstData = new List<string>();
foreach (var file in Files)
{
using (StreamReader sr = File.OpenText(file.FullName))
{
string s = String.Empty;
while ((s = sr.ReadLine()) != null)
{
if (!lstData.Contains(s))
{
lstData.Add(s);
}
}
}
}

Reading csv file in chunks for processing

I have a .csv file with 100 000 records with five columns in it. I am reading it line by line and storing it in a remote database .
Previously, I was following a performance oriented approach. I was reading the .csv file line by line and in the same transaction I was opening the connection to database and closing it. This was taking a serious performance overhead.
For just writing 10 000 lines, it took one hour.
using (FileStream reader = File.OpenRead(#"C:\Data.csv"))
using (TextFieldParser parser = new TextFieldParser(reader))
{
parser.TrimWhiteSpace = true; // if you want
parser.Delimiters = new[] { " " };
parser.HasFieldsEnclosedInQuotes = true;
while (!parser.EndOfData)
{
//Open a connection to a database
//Write the data from the .csv file line by line
//Close the connection
}
}
Now I have changed the approach. For testing purpose I have taken a .csv file with 10 000 lines and after reading all the 10 000 lines, I am making one connection to database and writing it there.
Now, the only issue is:
I want to read first 10 000 lines and write it, similarly read the next 10 000 lines and write it,
using (FileStream reader = File.OpenRead(#"C:\Data.csv"))
using (TextFieldParser parser = new TextFieldParser(reader))
but the above two lines will read the entire file . I don’t want to read it completely.
Is there any way to read the .csv file chunk by chunk of 10 000 lines each?
Try below code it reads data from csv chunk by chunk
IEnumerable<DataTable> GetFileData(string sourceFileFullName)
{
int chunkRowCount = 0;
using (var sr = new StreamReader(sourceFileFullName))
{
string line = null;
//Read and display lines from the file until the end of the file is reached.
while ((line = sr.ReadLine()) != null)
{
chunkRowCount++;
var chunkDataTable = ; ////Code for filling datatable or whatever
if (chunkRowCount == 10000)
{
chunkRowCount = 0;
yield return chunkDataTable;
chunkDataTable = null;
}
}
}
//return last set of data which less then chunk size
if (null != chunkDataTable)
yield return chunkDataTable;
}

How can I write and read a text doucument in a folder?

string curetn = Environment.CurrentDirectory;
string path = curetn.ToString() + #"\DATA\SaveGame.txt";
Console.WriteLine(path);
TextReader tr = new StreamReader(path);
Hello, I am making a text-adventure, and I do not like having all my save files, and mp3 file in the same place as my application. I would like for the files to be in a folder. I want to be able to use StreamWriter and StreamReader, to be able to write and read files that are in a folder. This file is also in a distributable folder, not just in the Visual Studios Projects folders. I have tried everything I can, and this is what I have. I also have one of these for StreamWriter. Please help!
Edit:
The thing that does not work, is that it does not read the lines, and assigns them to a variable. I have it in a try-catch, and it catches, and displays the error message that I wrote.
If you are looking for simply read and write lines from file you can try this
using (StreamReader sr = new StreamReader(path))
{
while (!sr.EndOfStream)
{
sr.ReadLine();
}
}
string s;
using (StreamWriter sw = new StreamWriter(path))
{
sw.WriteLine(s);
}
So basically what you want to do is read the text file:
string data[] = File.ReadAllLines(path); // Read the text file.
var x = data[1]; // Replace the '1' with the line number you want.
Console.WriteLine(x);
This is a good way to read the text file, I think it's better than opening a stream.
You can also write to it, so every time you want to save, just do this:
// When you want to write:
File.WriteAllText(path, "");
File.AppendAllText(path, "Add a data line" + Environment.NewLine); // Environment.NewLine adds a line.
Keep appending text to the file for the data you need.

Merging namespaces in C#

How can I merge the same namespaces that are scattered over different files in to one file?
I am using Resharper but it doesn't seem to have this option.
Order is not important.
Something like: copy *.cs output.cs but this doesn't merge namespaces.
Here is the code to join the namespaces. I did not put in a duplicate line checker because you may have different classes that may cointain identical lines. Though, LIke all the comments, i dont think its a good idea to have so many classes in one namespace. Its like haveing 500 woman in one house. None the less, Just change the file extension to .cs and your IDE will tell you the duplicate names and remove them. It should take about 2 minutes if you do find and edit all that meet search criteria.
here is the codeL
class Program
{
static void Main(string[] args)
{
StreamReader reader;
StreamWriter writer = new StreamWriter(/*where you want the final txt to be saved. rename the extension to .cs */);
string line;
int count = 0;
string[] fileEntries = Directory.GetFiles(/*where your cs files are*/");
foreach (string fileName in fileEntries)
{
// do something with fileName
File.Copy( fileName, /* location to save the now txt .cs files */ + count.ToString() + ".txt");
count += 1;
}
string[] Completetxt = Directory.GetFiles(/*location where you saved the txt files */);
foreach (string TxtFile in Completetxt)
{
reader = new StreamReader(TxtFile);
while ((line = reader.ReadLine()) != null)
{
Console.WriteLine(line);
writer.WriteLine(line);
}
reader.Close();
}
}
}
Just remember this copies all the cs files into one file with all the imports below each other as well as namespace names. Just use your IDE to find the copies and delete them.
Best of luck

Categories