We Keep Coding

c# Find file encoding

i m trying to write a method that does an estimate of the encoding of a file, i searched the msdn site and found this :
using (StreamReader sr = new StreamReader(openFileDialog1.FileName, true))
{
using (var reader = new StreamReader(openFileDialog1.FileName, defaultEncodingIfNoBom, true))
{
reader.Peek(); // you need this!
var encoding = reader.CurrentEncoding;
}
while (sr.Peek() >= 0)
{
Console.Write((char)sr.Read());
}
Console.WriteLine("The encoding used was {0}.", sr.CurrentEncoding);
Console.ReadLine();
Console.WriteLine();
textBox4.Text = sr.CurrentEncoding.ToString();
}
My problem with the code above is that for large files, it reads the entire file before there's any sort of output, is there a way to limit this to reading just say, the first ten lines of a file?

You could use the System.Linq derivative and use the following;
string[] first10Lines = File.ReadLines(path).Take(10).ToList();
That will read the first 10 lines of the file and store it in an array. you can then do a for loop to write the data in the array like this which should add a new line after each data is written.
foreach (string line in first10Lines)
{
File.AppendAllText(newPathName, line);
File.AppendAllText(newPathName, string.Format("{0}{1}", " ", Environment.NewLine));
}

Merging 2 Text Files in C#

Firstly, i'd just like to mention that I've only started learning C# a few days ago so my knowledge of it is limited.
I'm trying to create a program that will parse text files for certain phrases input by the user and then output them into a new text document.
At the moment, i have it the program searching the original input file and gathering the selected text input by the user, coping those lines out, creating new text files and then merging them together and also deleting them afterwards.
I'm guessing that this is not the most efficient way of creating this but i just created it and had it work in a logical manor for me to understand as a novice.
The code is as follows;
private void TextInput1()
{
using (StreamReader fileOpen = new StreamReader(txtInput.Text))
{
using (StreamWriter fileWrite = new StreamWriter(#"*DIRECTORY*\FIRSTFILE.txt"))
{
string file;
while ((file = fileOpen.ReadLine()) != null)
{
if (file.Contains(txtFind.Text))
{
fileWrite.Write(file + "\r\n");
}
}
}
}
}
private void TextInput2()
{
using (StreamReader fileOpen = new StreamReader(txtInput.Text))
{
using (StreamWriter fileWrite = new StreamWriter(#"*DIRECTORY*\SECONDFILE.txt"))
{
string file;
while ((file = fileOpen.ReadLine()) != null)
{
if (file.Contains(txtFind2.Text))
{
fileWrite.Write("\r\n" + file);
}
}
}
}
}
private static void Combination()
{
ArrayList fileArray = new ArrayList();
using (StreamWriter writer = File.CreateText(#"*DIRECTORY*\FINALOUTPUT.txt"))
{
using (StreamReader reader = File.OpenText(#"*DIRECTORY*\FIRSTFILE.txt"))
{
writer.Write(reader.ReadToEnd());
}
using (StreamReader reader = File.OpenText(#"*DIRECTORY*\SECONDFILE.txt"))
{
writer.Write(reader.ReadToEnd());
}
}
}
private static void Delete()
{
if (File.Exists(#"*DIRECTORY*\FIRSTFILE.txt"))
{
File.Delete(#"*DIRECTORY*\FIRSTFILE.txt");
}
if (File.Exists(#"*DIRECTORY*\SECONDFILE.txt"))
{
File.Delete(#"*DIRECTORY*\SECONDFILE.txt");
}
}
The output file that is being created is simply outputting the first text input followed by the second. I am wondering if it is possible to be able to merge them into 1 file, 1 line at a time as it is a consecutive file meaning have the information from Input 1 followed 2 is needed rather than all of 1 then all of 2.
Thanks, Neil.

To combine the two files content in an one merged file line by line you could substitute your Combination() code with this
string[] file1 = File.ReadAllLines("*DIRECTORY*\FIRSTFILE.txt");
string[] file2 = File.ReadAllLines("*DIRECTORY*\SECONDFILE.txt");
using (StreamWriter writer = File.CreateText(#"*DIRECTORY*\FINALOUTPUT.txt"))
{
int lineNum = 0;
while(lineNum < file1.Length || lineNum < file2.Length)
{
if(lineNum < file1.Length)
writer.WriteLine(file1[lineNum]);
if(lineNum < file2.Length)
writer.WriteLine(file2[lineNum]);
lineNum++;
}
}
This assumes that the two files don't contains the same number of lines.

try this method. You can receive three paths. File 1, File 2 and File output.
public void MergeFiles(string pathFile1, string pathFile2, string pathResult)
{
File.WriteAllText(pathResult, File.ReadAllText(pathFile1) + File.ReadAllText(pathFile2));
}
If the pathResult file exists, the WriteAllText method will overwrite it. Remember to include System.IO namespace.
Important: It is not recommended for large files! Use another options available on this thread.

If your input files are quite large and you run out of memory, you could also try wrapping the two readers like this:
using (StreamWriter writer = File.CreateText(#"*DIRECTORY*\FINALOUTPUT.txt"))
{
using (StreamReader reader1 = File.OpenText(#"*DIRECTORY*\FIRSTFILE.txt"))
{
using (StreamReader reader2 = File.OpenText(#"*DIRECTORY*\SECONDFILE.txt"))
{
string line1 = null;
string line2 = null;
while ((line1 = reader1.ReadLine()) != null)
{
writer.WriteLine(line1);
line2 = reader2.ReadLine();
if(line2 != null)
{
writer.WriteLine(line2);
}
}
}
}
}
Still, you have to have an idea how many lines you have in your input files, but I think it gives you the general idea to proceed.

Using a FileInfo extension you could merge one or more files by doing the following:
public static class FileInfoExtensions
{
public static void MergeFiles(this FileInfo fi, string strOutputPath , params string[] filesToMerge)
{
var fiLines = File.ReadAllLines(fi.FullName).ToList();
fiLines.AddRange(filesToMerge.SelectMany(file => File.ReadAllLines(file)));
File.WriteAllLines(strOutputPath, fiLines.ToArray());
}
}
Usage
FileInfo fi = new FileInfo("input");
fi.MergeFiles("output", "File2", "File3");

I appreciate this question is almost old enough to (up)vote (itself), but for an extensible approach:
const string FileMergeDivider = "\n\n";
public void MergeFiles(string outputPath, params string[] inputPaths)
{
if (!inputPaths.Any())
throw new ArgumentException(nameof(inputPaths) + " required");
if (inputPaths.Any(string.IsNullOrWhiteSpace) || !inputPaths.All(File.Exists))
throw new ArgumentNullException(nameof(inputPaths), "contains invalid path(s)");
File.WriteAllText(outputPath, string.Join(FileMergeDivider, inputPaths.Select(File.ReadAllText)));
}

How to split the large text file(32 GB) using C#

I tried to split the file about 32GB using the below code but I got the memory exception.
Please suggest me to split the file using C#.
string[] splitFile = File.ReadAllLines(#"E:\\JKS\\ImportGenius\\0.txt");
int cycle = 1;
int splitSize = Convert.ToInt32(txtNoOfLines.Text);
var chunk = splitFile.Take(splitSize);
var rem = splitFile.Skip(splitSize);
while (chunk.Take(1).Count() > 0)
{
string filename = "file" + cycle.ToString() + ".txt";
using (StreamWriter sw = new StreamWriter(filename))
{
foreach (string line in chunk)
{
sw.WriteLine(line);
}
}
chunk = rem.Take(splitSize);
rem = rem.Skip(splitSize);
cycle++;
}

Well, to start with you need to use File.ReadLines (assuming you're using .NET 4) so that it doesn't try to read the whole thing into memory. Then I'd just keep calling a method to spit the "next" however many lines to a new file:
int splitSize = Convert.ToInt32(txtNoOfLines.Text);
using (var lineIterator = File.ReadLines(...).GetEnumerator())
{
bool stillGoing = true;
for (int chunk = 0; stillGoing; chunk++)
{
stillGoing = WriteChunk(lineIterator, splitSize, chunk);
}
}
...
private static bool WriteChunk(IEnumerator<string> lineIterator,
int splitSize, int chunk)
{
using (var writer = File.CreateText("file " + chunk + ".txt"))
{
for (int i = 0; i < splitSize; i++)
{
if (!lineIterator.MoveNext())
{
return false;
}
writer.WriteLine(lineIterator.Current);
}
}
return true;
}

Do not read immediately all lines into an array, but use StremReader.ReadLine method, like:
using (StreamReader sr = new StreamReader(#"E:\\JKS\\ImportGenius\\0.txt"))
{
while (sr.Peek() >= 0)
{
var fileLine = sr.ReadLine();
//do something with line
}
}

File.ReadAllLines
That will read the whole file into memory.
To work with large files you need to only read what you need now into memory, and then throw that away as soon as you have finished with it.
A better option would be File.ReadLines which returns a lazy enumerator, data is only read into memory as you get the next line from the enumerator. Providing you avoid multiple enumerations (eg. don't use Count()) only parts of the file will be read.

Instead of reading all the file at once using File.ReadAllLines, use File.ReadLines in a foreach loop to read the lines as needed.
foreach (var line in File.ReadLines(#"E:\\JKS\\ImportGenius\\0.txt"))
{
// Do something
}
Edit: On an unrelated note, you don't have to escape your backslashes when prefixing the string with a '#'. So either write "E:\\JKS\\ImportGenius\\0.txt" or #"E:\JKS\ImportGenius\0.txt", but #"E:\\JKS\\ImportGenius\\0.txt" is redundant.

The problem here is that you are reading the entire file's content into memory at once with File.ReadAllLines(). What you need to do is open a FileStream with File.OpenRead() and read/write smaller chunks.
Edit: Actually for your case ReadLine is obviously better. See other answers. :)

Use a StreamReader to read the file, write with a StreamWriter.

File cannot be accessed because it is being used by another program

I am trying to remove the space at the end of line and then that line will be written in another file.
But when the program reaches to FileWriter then it gives me the following error
Process can't be accessed because it is being used by another process.
The Code is as below.
private void FrmCounter_Load(object sender, EventArgs e)
{
string[] filePaths = Directory.GetFiles(#"D:\abc", "*.txt", SearchOption.AllDirectories);
string activeDir = #"D:\dest";
System.IO.StreamWriter fw;
string result;
foreach (string file in filePaths)
{
result = Path.GetFileName(file);
System.IO.StreamReader f = new StreamReader(file);
string newFileName = result;
// Combine the new file name with the path
string newPath = System.IO.Path.Combine(activeDir, newFileName);
File.Create(newPath);
fw = new StreamWriter(newPath);
int counter = 0;
int spaceAtEnd = 0;
string line;
// Read the file and display it line by line.
while ((line = f.ReadLine()) != null)
{
if (line.EndsWith(" "))
{
spaceAtEnd++;
line = line.Substring(0, line.Length - 1);
}
fw.WriteLine(line);
fw.Flush();
counter++;
}
MessageBox.Show("File Name : " + result);
MessageBox.Show("Total Space at end : " + spaceAtEnd.ToString());
f.Close();
fw.Close();
}
}

File.Create itself returns a stream.
Use that stream to write file. Reason you are receiving this error is because Stream returned by File.Create is open and you are trying to open that file again for write.
Either close the stream returned by File.Create or better use that stream for file write or use
Stream newFile = File.Create(newPath);
fw = new StreamWriter(newFile);

Even though you solved your initial problem, if you want to write everything into a new file in the original location, you can try to read all of the data into an array and close the original StreamReader. Performance note: If your file is sufficiently large though, this option is not going to be the best for performance.
And you don't need File.Create as the StreamWriter will create a file if it doesn't exist, or overwrite it by default or if you specify the append parameter as false.
result = Path.GetFileName(file);
String[] f = File.ReadAllLines(file); // major change here...
// now f is an array containing all lines
// instead of a stream reader
using(var fw = new StreamWriter(result, false))
{
int counter = f.Length; // you aren't using counter anywhere, so I don't know if
// it is needed, but now you can just access the `Length`
// property of the array and get the length without a
// counter
int spaceAtEnd = 0;
// Read the file and display it line by line.
foreach (var item in f)
{
var line = item;
if (line.EndsWith(" "))
{
spaceAtEnd++;
line = line.Substring(0, line.Length - 1);
}
fw.WriteLine(line);
fw.Flush();
}
}
MessageBox.Show("File Name : " + result);
MessageBox.Show("Total Space at end : " + spaceAtEnd.ToString());
Also, you will not remove multiple spaces from the end of the line using this method. If you need to do that, consider replacing line = line.Substring(0, line.Length - 1); with line = line.TrimEnd(' ');

You have to close any files you are reading before you attempt to write to them in your case.

Write stream in using statement like:
using (System.IO.StreamReader f = new StreamReader(file))
{
//your code goes here
}

EDIT:
Zafar is correct, however, maybe this will clear things up.
Because File.Create returns a stream.. that stream has opened your destination file. This will make things clearer:
File.Create(newPath).Close();
Using the above line, makes it work, however, I would suggest re-writing that properly. This is just for illustrative purposes.

fast text reader c#

I am using the following code to read and combine number of texts in one string:
foreach (string path in filePaths)
{
StreamReader singfile = new StreamReader(path);
string file_text = singfile.ReadToEnd();
combinetexts += file_text + "\n";
fs.Close();
}
and as I know, the string combinetexts will copy n times as much as the number of filepaths. is it possible the to do that orcedure using string builder ? I tried but it doesn't.
thanks in advance.

Here's a short LINQ way of doing it:
string result = string.Join("\n", filePaths.Select(x => File.ReadAllText(x)));
Or with C# 4 (which has better handling of type inference wrt method group conversions):
string result = string.Join("\n", filePaths.Select(File.ReadAllText));
If you're using .NET 3.5 you'll need to create an array of the strings, as string.Join didn't have as many overloads then:
string result = string.Join("\n", filePaths.Select(x => File.ReadAllText(x))
.ToArray());
This has the disadvantage of reading all of all the files before performing the concatenation, admittedly - but it's still better than the repeated concatenation in the original code. It might also be more efficient than using StringBuilder - it depends on the string.Join implementation.
See my article on StringBuilder for why the original code is really inefficient.
EDIT: Note that this does not include a trailing \n at the end. If you really want to add that, you can :)

Here's your example using a StringBuilder instead of a string:
var sb = new StringBuilder();
foreach (string path in filePaths)
sb.AppendLine(File.ReadAllText(path));
string result = sb.ToString();
(I've also taken the liberty to shorten/optimize your code a bit. File.ReadAllText reads the complete contents of a file without having to open a StreamReader manually. In addition, AppendLine automatically adds a \n at the end.)

Of course it's possible, use
StringBuilder combinetexts = new StringBuilder();
...
combinetexts.Append(file_text);
combinetexts.Append("\n");;

It's more efficient to use an StringBuilder to manipulates strings.
http://www.codeproject.com/Articles/14936/StringBuilder-vs-String-Fast-String-Operations-wit
Best regards

Try the below code:
StringBuilder strBuilder= new StringBuilder();
foreach (string path in filePaths)
{
StreamReader singfile = new StreamReader(path);
string file_text = singfile.ReadToEnd();
strBuilder.AppendLine(file_text);
fs.Close();
}
Console.WriteLine(strBuilder.ToString());

Yes, it is possible to use StringBuilder, there are various ways to "optimize" this code.
TL;DR: Skip to the last part of this post for the best way to do this.
Here's stage 1 changes to your code:
StringBuilder combinetexts = new StringBuilder();
foreach (string path in filePaths)
{
StreamReader fs = new StreamReader(path);
string file_text = fs.ReadToEnd();
combinetexts.Append(file_text).Append("\n");
fs.Close();
}
Secondly, before building the StringBuilder you can calculate how much space you actually need, this will reduce the chance of copying the string even further:
long totalSize = 0;
foreach (string path in filePaths)
totalSize += new FileInfo(path).Length + 1; // +1 = \n
StringBuilder sb = new StringBuilder(Convert.ToInt32(totalSize));
foreach (string path in filePaths)
{
StreamReader fs = new StreamReader(path);
string file_text = fs.ReadToEnd();
combinetexts.Append(file_text).Append("\n");
fs.Close();
}
Lastly I would use using (...) instead of the fs.Close(); call:
long totalSize = 0;
foreach (string path in filePaths)
totalSize += new FileInfo(path).Length + 1; // +1 = \n
StringBuilder sb = new StringBuilder(Convert.ToInt32(totalSize));
foreach (string path in filePaths)
{
using (StreamReader fs = new StreamReader(path))
{
string file_text = fs.ReadToEnd();
combinetexts.Append(file_text).Append("\n");
}
}
Then I would use LINQ a bit more and switch to using File.ReadAllText instead of an explicit StreamReader, and then combine the lines of code a bit:
long totalSize = filePaths.Sum(path => new FileInfo(path).Length + 1);
StringBuilder sb = new StringBuilder(Convert.ToInt32(totalSize));
foreach (string path in filePaths)
{
combinetexts.Append(File.ReadAllText(path)).Append("\n");
}
However, as it turns out, there's an even better way to do this:
string combinetexts = String.Join("\n", filePaths.Select(path => File.ReadAllText(path)));
or in C# 4.0 which can better infer the right way to handle method group conversions:
string combinetexts = String.Join("\n", filePaths.Select(File.ReadAllText));
This will do all of the above, it will:
Read in all the files
String.Join will calculate the total size needed to hold the entire string
Then it will combine all the texts, with a \n between each