We Keep Coding

I want to combine three txt-files into one single file in c# [duplicate]

This question already has answers here:
Easiest way to read from and write to files
(14 answers)
Closed 5 years ago.
Im new to C# but learning as I go forward, so forgive me if my question might be a little too easy for most of you guys. :-)
My question has two parts.
First: I have three textfiles (with fixed names) with totaly
different contents. I now want these combine into one single
textfile.
Second: During this combination process I also want every comma(,)
in these files to be changed to a dot(.) in the output combined file.
I have managed to do something similar with StreamReader and StreamWriter in combination with ReadLine in vb but cant get it to work in C#.
Im thankful for any help I can get.
/Tomas

If files are not big you can use this snippet:
File.WriteAllText("newfile", String.Concat(File.ReadAllText("file1"),File.ReadAllText("file2"),File.ReadAllText("file3")).Replace(",","."));

In general case when files are long with arbitrary number of files you can try Linq SelectMany:
using System.IO;
using System.Linq;
...
string[] fileNames = new string[] {
#"C:\MyFile1.txt",
#"C:\MyFile2.txt",
#"C:\MyFile3.txt",
};
...
File.WriteAllLines(#"C:\MyCombinedFile.txt", fileNames
.SelectMany(file => File.ReadLines(file))
.Select(line => line.Replace(',', '.')));

string[] files = new string[]{ #"E:\myfile1.txt", #"E:\myfile2.txt", #"E:\myfile3.txt" };
string fileContent = string.Empty;
foreach (var fileName in files)
{
using (System.IO.StreamReader Reader = new System.IO.StreamReader(fileName))
{
fileContent += Reader.ReadToEnd();
}
}
fileContent = fileContent.Replace(',', '.');
System.IO.File.WriteAllText(#"E:\myfile.txt", fileContent);

Slow loading of .CSV files using EPPLUS

I have loads of .csv files I need to convert to .xslx after applying some formatting.
A file containing approx 20 000 rows and 7 columns takes 12 minutes to convert.
If the file contains more than 100 000 it runs for > 1 hour.
This is unfortunately not acceptable for me.
Code snippet:
var format = new ExcelTextFormat();
format.Delimiter = ';';
format.Encoding = new UTF7Encoding();
format.Culture = new CultureInfo(System.Threading.Thread.CurrentThread.CurrentCulture.ToString());
format.Culture.DateTimeFormat.ShortDatePattern = "dd.mm.yyyy";
using (ExcelPackage package = new ExcelPackage(new FileInfo(file.Name))){
ExcelWorksheet worksheet = package.Workbook.Worksheets.Add(Path.GetFileNameWithoutExtension(file.Name));
worksheet.Cells["A1"].LoadFromText(new FileInfo(file.FullName), format);
}
I have verified that it is the LoadFromText command that spends the time used.
Is there a way to speed things up?
I have tried without the "format" parameter, but the loadtime was the same.
What loadtimes are you experiencing?

My suggestion here is to read the file by yourself and then use the library to create the file.
The code to read the CSV could be as simple as:
List<String> lines = new List<String>();
using (StreamReader reader = new StreamReader("file.csv"))
{
String line;
while((line = reader.ReadLine()) != null)
{
lines.add(line);
}
}
//Now you got all lines of your CSV
//Create your file with EPPLUS
foreach(String line in lines)
{
var values = line.Split(';');
foreach(String value in values)
{
//use EPPLUS library to fill your file
}
}

I ran into a very similar problem with LoadFromCollection. EPPlus has to account for all situations in their methods to load data generically like that so there is a good deal of overhead. I ended up narrowing done the bottleneck to that method and ended up just manually coverting the data from the collection to Excel Cell object in EPPlus. Probably saved several minutes in my exports.
Plenty of examples on how to read csv data:
C# Read a particular value from CSV file

Joining two files in C#, split options

I am currently trying to work on files, joining multiple of them and having problem because the last work from file 1 is linked with first word from file 2. For example:
File 1:John has got new haircut
File 2: Mike has got new haircut
and it prints me "haircutMike".
The code I am using to split words:
input.Split(' ').ToList().ForEach(n =>{});
I am also making one big file from multiple ones like so:
string[] files = { "f1.txt", "f2.txt" };
FileStream outputFile = new FileStream("new.txt", FileMode.Create);
using (StreamWriter ws = new StreamWriter(outputFile))
{
foreach (string file in files)
{
ws.Write(System.IO.File.ReadAllText(file) + " ");
}
}
#EDIT
Changed some code, of course I meant to use stream not binary,also I am using split because I want to count the number of each word in files so I have to split spaces, dots etc.
You mentioned to use + " " option, although it works, but it added me 1 letter to the total count.

EDIT: for multiple input files:
string[] files = { "f1.txt", "f2.txt" };
var allLines = files.SelectMany(i => System.IO.File.ReadAllLines(i));
System.IO.File.WriteAllLines("new.txt", allLines.ToArray());

parsing large textfile output to another textfile [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I want to parse a large textfile and if the line contains a certain substring then append that line to my new text file. I need the solution with the lowest memory usage, This is what I have so far, the comments is what I need help adding:
.
.
.
if (File.ReadLines(filepath).Any(line => line.Contains(myXML.searchSTRING)))
{
// code to grab that line and append it to the a new text file
// if new text file doesn't exist then create it.
// All text files im parsing have the same header, I want to grab
// the third line and use it as my new text file header.
// Only write the header once, I do not want it written every time a new
// text file is opened for parsing
}

Try :
var count = 1;
File.WriteAllLines(newFilePath,
File.ReadLines(filepath)
.Where(count++ == 3 || l => l.Contains(myXML.searchSTRING))
);
Both WriteAllLines() and ReadLines() use enumerators, so should have relatively low memory usage.
I'm not sure how you would know to write the header only once, it depends on how you have your list of files to open available. Are they in an array? If so wrap the File.WriteAllLines call in a foreach loop around that array.

Something like this should do it (edited to reflect #JimMischel's comments):
private static void WriteFile(string mySearchString, string fileToWrite, params string[] filesToRead)
{
using (var sw = new StreamWriter(fileToWrite, true))
{
var count = 1;
foreach (var file in filesToRead)
{
using (var sr = new StreamReader(file))
{
string line;
while ((line = sr.ReadLine()) != null)
{
if (count == 3)
{
sw.WriteLine(line);
}
if (count > 3 && line.Contains(mySearchString))
{
sw.WriteLine(line);
}
count++;
}
}
}
}
}
You would call it like this:
WriteFile("Foobar", "fileToWrite.txt", "input1.txt", "input2.txt", "input3.txt");

You can use a StreamWriter for that :
using (var fs = new FileStream(outpuFilePath, FileMode.Append, FileAccess.Write))
{
using (var sw = new StreamWriter(fs))
{
foreach (var line in File.ReadLines(filepath).Where(line => line.Contains(myXML.searchSTRING)))
{
sw.WriteLine(line);
}
}
}

I think the most important thing is to use "Where" instead of "Any" Any returns a true/false, if a collection matches, whereas you want to filter the collection. Below should get you started in combination with the answers above (I would use Linq for clarity though).
StreamWriter outFile = new StreamWriter("output.txt");
string filepath = "infile.txt";
var header=File.ReadLines(filepath).Skip(2).First();
outFile.WriteLine(header);
var searchString = "temp";
File.ReadLines(filepath).Where(x => x.Contains(searchString))
.Select(x =>outFile.WriteLine(x));

Please read article for MemoryMappedFile
http://www.dotnetperls.com/memorymappedfile-benchmark

read only given last x lines in txt file [duplicate]

This question already has answers here:
Get last 10 lines of very large text file > 10GB
(21 answers)
Closed 9 years ago.
Currently I'm reading file content using File.ReadAllText(), but now I need to read last x lines in my txt file. How can I do that?
content of myfile.txt
line1content
line2content
line3content
line4content
string contentOfLastTwoLines = ...

What about this
List <string> text = File.ReadLines("file.txt").Reverse().Take(2).ToList()

Use Queue<string> to store last X lines and replace the first one with currently read:
int x = 4; // number of lines you want to get
var buffor = new Queue<string>(x);
var file = new StreamReader("Input.txt");
while (!file.EndOfStream)
{
string line = file.ReadLine();
if (buffor.Count >= x)
buffor.Dequeue();
buffor.Enqueue(line);
}
string[] lastLines = buffor.ToArray();
string contentOfLastLines = String.Join(Environment.NewLine, lastLines);

You can use ReadLines to avoid reading the entire file into memory, like this:
const int neededLines = 5;
var lines = new List<String>();
foreach (var s in File.ReadLines("c:\\myfile.txt")) {
lines.Add(s);
if (lines.Count > neededLines) {
lines.RemoveAt(0);
}
}
Once the for loop is finished, the lines list contains up to the last neededLines of text from the file. Of course if the file does not contain as many lines as required, fewer lines will be placed in the lines list.

Read the lines into an array, then extract the last two:
string[] lines = File.ReadAllLines();
string last2 = lines[lines.Count-2] + Environment.NewLine + lines[lines.Count-1];
Assuming your file is reasonably small, it's easier to just read the whole thing and throw away what you don't need.

Since reading a file is done linearly, usually line-by-line. Simply read line-by-line and remember last two lines (you can use queue or something if you want... or just two string variables). When you get to EOF, you'll have your last two lines.

You want to read the file backwards using ReverseLineReader:
How to read a text file reversely with iterator in C#
Then run .Take(2) on it.
var lines = new ReverseLineReader(filename);
var last = lines.Take(2);
OR
Use a System.IO.StreamReader.
string line1, line2;
using(StreamReader reader = new StreamReader("myFile.txt")) {
line1 = reader.ReadLine();
line2 = reader.ReadLine();
}