We Keep Coding

Effective way of reading text file in c# for line to line operation

I'm having problem in reading text file for line to line addition operation. I have used following syntax
StreamReader reader = new StreamReader("input.txt");
string line;
while ((line = reader.ReadLine()) != null)
string[] splitted = line.Split('#');
string first = splitted[0].Trim();
string second = splitted[1].Trim();
I have used this syntax to separate the input from text file if file has following values.
12#15
15#7
13#14
23#31
x= Convert.ToInt32(first);
y= Convert.ToInt32(second);
sum = x+y;
txtBox.text = Convert.ToString(sum);
the problem is it only executes the last line. It only calculate the sum of 23 and 31 and show only but I want to add 12 and 15 first and show it in textbox similarly I want to add others. please help me in forming appropriate syntax.

The question is vague one, however, I suggest using Linq:
var source = File
.ReadLines("input.txt") // read line by line
.Select(line => line.Split('#')) // split each line
//.Where(items => items.Length == 2) // you may want to filter out the lines
.Select(items => new { // convert each line into anonymous class
first = items[0].Trim(),
second = items[1].Trim()
});
You can add as many Select (line to line opetations) as you want. Then you can proceed the items in a foreach loop:
foreach (var item in source) {
...
// Let's read some fields from the anonymous object
var first = item.first;
var second = item.second;
...
}
Edit: according to the edited question you want just to sum up which can be done via Linq as well:
var result = File
.ReadLines("input.txt")
.Select(line => line.Split('#'))
//.Where(items => items.Length == 2) // you may want to filter out the lines
.Sum(items => int.Parse(items[0]) + int.Parse(items[1]));
txtBox.text = result.ToString();

It doesn't only read the last line, you just never do anything with the other iterations. Currently you just keep reassigning line with the value of the latest line that has been read, I presume you hope to save these to a list or similar
StreamReader reader = new StreamReader("input.txt");
string line;
List<string> allLines = new List<string>();
while ((line = reader.ReadLine()) != null)
allLines.Add(line);

here you can test your File and Load The data into a DataTable this should be pretty straight forward.
DataTable dtTextFileData = new DataTable();
dtTextFileData.Columns.AddRange(new []
{
new DataColumn("First", typeof(string)),
new DataColumn("Second", typeof(string))
});
StreamReader file = new StreamReader(#"c:\YourFilePath\input.txt");
string line = file.ReadLine();
while (line != null)
{
string[] fields = line.Split('#');
DataRow dr = dtTextFileData.NewRow();
dr["First"] = fields[0].ToString();
dr["Second"] = fields[1].ToString();
dtTextFileData.Rows.Add(dr);
line = file.ReadLine();
}

Copying CSV file while reordering/adding empty columns

Copying CSV file while reordering/adding empty columns.
For example if ever line of incoming file has values for 3 out of 10 columns in order different from output like (except first which is header with column names):
col2,col6,col4 // first line - column names
2, 5, 8 // subsequent lines - values for 3 columns
and output expected to have
col0,col1,col2,col3,col4,col5,col6,col7,col8,col9
then output should be "" for col0,col1,col3,col5,col7,col8,col9,and values from col2,col4,col4 in the input file. So for the shown second line (2,5,8) expected output is ",,2,,5,,8,,,,,"
Below code I've tried and it is slower than I want.
I have two lists.
The first list filecolumnnames is created by splitting a delimited string (line) and this list gets recreated for every line in the file.
The second list list has the order in which the first list needs to be rearranged and re concatenated.
This works
string fileName = "F:\\temp.csv";
//file data has first row col3,col2,col1,col0;
//second row: 4,3,2,1
//so on
string fileName_recreated = "F:\\temp_1.csv";
int count = 0;
const Int32 BufferSize = 1028;
using (var fileStream = File.OpenRead(fileName))
using (var streamReader = new StreamReader(fileStream, Encoding.UTF8, true, BufferSize))
{
String line;
List<int> list = new List<int>();
string orderedcolumns = "\"\"";
string tableheader = "col0,col1,col2,col3,col4,col5,col6,col7,col8,col9,col10";
List<string> tablecolumnnames = new List<string>();
List<string> filecolumnnames = new List<string>();
while ((line = streamReader.ReadLine()) != null)
{
count = count + 1;
StringBuilder sb = new StringBuilder("");
tablecolumnnames = tableheader.Split(',').ToList();
if (count == 1)
{
string fileheader = line;
//fileheader=""col2,col1,col0"
filecolumnnames = fileheader.Split(',').ToList();
foreach (string col in tablecolumnnames)
{
int index = filecolumnnames.IndexOf(col);
if (index == -1)
{
sb.Append(",");
// orderedcolumns=orderedcolumns+"+\",\"";
list.Add(-1);
}
else
{
sb.Append(filecolumnnames[index] + ",");
//orderedcolumns = orderedcolumns+ "+filecolumnnames["+index+"]" + "+\",\"";
list.Add(index);
}
// MessageBox.Show(orderedcolumns);
}
}
else
{
filecolumnnames = line.Split(',').ToList();
foreach (int items in list)
{
//MessageBox.Show(items.ToString());
if (items == -1)
{
sb.Append(",");
}
else
{
sb.Append(filecolumnnames[items] + ",");
}
}
//expected format sb.Append(filecolumnnames[3] + "," + filecolumnnames[2] + "," + filecolumnnames[2] + ",");
//sb.Append(orderedcolumns);
var result = String.Join (", ", list.Select(index => filecolumnnames[index]));
}
using (FileStream fs = new FileStream(fileName_recreated, FileMode.Append, FileAccess.Write))
using (StreamWriter sw = new StreamWriter(fs))
{
sw.WriteLine(sb.ToString());
}
}
I am trying to make it faster by constructing a string orderedcolumns and remove the second for each loop which happens for every row and replace it with constructed string.
so if you uncomment the orderedcolumns string construction orderedcolumns = orderedcolumns+ "+filecolumnnames["+index+"]" + "+\",\""; and uncomment the append sb.Append(orderedcolumns); I am expecting the value inside the constructed string but when I append the orderedcolumns it is appending the text i.e.
""+","+filecolumnnames[3]+","+filecolumnnames[2]+","+filecolumnnames[1]+","+filecolumnnames[0]+","+","+","+","+","+","+","
i.e. I instead want it to take the value inside the filecolumnnames[3] list and not the filecolumnnames[3] name itself.
Expected value: if that line has 1,2,3,4
I want the output to be 4,3,2,1 as filecolumnnames[3] will have 4, filecolumnnames[2] will have 3..

String.Join is the way to construct comma/space delimited strings from sequence.
var result = String.Join (", ", list.Select(index => filecolumnnames[index]);
Since you are reading only subset of columns and orders in input and output don't match I'd use dictionary to hold each row of input.
var row = tablecolumnnames
.Zip(line.Split(','), (Name,Value)=> new {Name,Value})
.ToDictionary(x => x.Name, x.Value);
For output I'd fill sequence from defaults or input row:
var outputLine = String.Join(",",
filecolumnnames
.Select(name => row.ContainsKey(name) ? row[name] : ""));
Note code is typed in and not compiled.

orderedcolumns = orderedcolumns+ "+filecolumnnames["+index+"]" + "+\",\""; "
should be
orderedcolumns = orderedcolumns+ filecolumnnames[index] + ",";
you should however use join as others have pointed out. Or
orderedcolumns.AppendFormat("{0},", filecolumnnames[index]);
you will have to deal with the extra ',' on the end

Merge 2 lines in .CSV file using StreamReader

I am currently trying to merge some lines in a .csv file. The file follows a specific format which is split by "," and the last element uses \n ascii code. This means the last element gets put onto a new line and i return an array with only one Element. I am looking to merge this element with the line above it.
So my line would be:
192.168.60.24, ACD_test1,86.33352, 07/12/2014 13:33:13, False, Annotated, True,"Attribute1
Attribute 2
Attribute 3"
192.168.60.24, ACD_test1,87.33352, 07/12/2014 13:33:13, False, Annotated, True
Is it possible to merge/join the new line attributes with the line above?
My code is shown below:
var reader = new StreamReader(File.OpenRead(#path));
string line1 = reader.ReadLine();
if (line1.Contains("Server, Tagname, Value, Timestamp, Questionable, Annotated, Substituted"))
{
while (!reader.EndOfStream)
{
List<string> listPointValue = new List<string>();
var line = reader.ReadLine();
var values = line.Split(',');
if (values.Count() < 2)
{
//*****Trying to Add Attribute to listPointValue.ElememtAt(0) here******
}
else
{
foreach (string value in values)
{
listPointValue.Add(value);
}
allValues.Add(listPointValue);
}
}
// allValues.RemoveAt(0);
return allValues;
}

I think you want to read the next line before you do the allValues.Add. That way you can decide whether to add the previous line to allValues (starting a new line). This gives you an idea of what I mean:
var reader = new StreamReader(File.OpenRead(#path));
string line1 = reader.ReadLine();
if (line1.Contains("Server, Tagname, Value, Timestamp, Questionable, Annotated, Substituted"))
{
List<string> listPointValue = new List<string>();
// Add first line to listPointValue
var line = reader.ReadLine();
var values = line.Split(',');
foreach (string value in values)
{
listPointValue.Add(value);
}
while (!reader.EndOfStream)
{
// Read next line
line = reader.ReadLine();
values = line.Split(',');
// If next line is a full line, add the previous line and create a new line
if (values.Count() > 1)
{
allValues.Add(listPointValue);
listPointValue = new List<string>();
}
// Add values to line
foreach (string value in values)
{
listPointValue.Add(value);
}
}
allValues.Add(listPointValue);
}

Why is this code not replacing data in a text file?

I'm working on a small app which should read a file (ANSI 835) and replace data at certain positions with generic data. Basically I'm trying to scrub a person's first and last name from the file.
The line I'm searching for that contains the name looks like this:
NM1*QC*1*Doe*John*R***MI*010088307 01~
My code looks like this:
string[] input_file = (string[])(e.Data.GetData(DataFormats.FileDrop));
string output_file = #"c:\scrubbed.txt";
foreach (string file in input_file)
{
string[] lines = File.ReadAllLines(file);
foreach (string line in lines)
{
if (line.StartsWith("NM1*QC"))
{
line.Split('*')[1] = "Lastname";
line.Split('*')[2] = "Firstname";
}
}
File.WriteAllLines(output_file, lines);
}
The File.WriteAllLines works, but the data isn't being changed. I'm trying to get any line that starts with NM1*QC to look like this:
NM1*QC*1*Lastname*Firstname*R***MI*010088307 01~
There are many lines in the file that start with NM1*QC. What's the proper way to 'find and replace' and then create a new file in this situation?
As always, thanks for your time!

The calls to String.Split return variables that you neither capture, nor use, they do not change the underlying string. So your code equates to this:
if (line.StartsWith("NM1*QC"))
{
string[] split1 = line.Split('*')[1] = "Lastname";
string[] split2 = line.Split('*')[2] = "Firstname";
}
You would need to take the results of split1 and split2 and use those to recreate your string. Here is how I would re-write your code:
string[] input_file = (string[])(e.Data.GetData(DataFormats.FileDrop));
string output_file = #"c:\scrubbed.txt";
foreach (string file in input_file)
{
string[] lines = File.ReadAllLines(file);
for (int i=0; i < lines.length; i++)
{
string line = lines[i];
if (line.StartsWith("NM1*QC"))
{
string[] values = line.Split('*');
values[1] = "Lastname";
values[2] = "Firstname";
lines[i] = String.Join("*", values);
}
}
File.WriteAllLines(output_file, lines);
}
Notice I am recombining the individual values using the String.Join method, and inserting the new string back into the array of lines. That will then get written out as you expect.

Here you are creating a temporary array:
line.Split('*')
And you are changing its contents:
line.Split('*')[1] = "Lastname";
After the line has been executed the reference to this temporary array is lost and along with it go your changes.
In order to persist the changes you need to write directly to lines:
for (var i = 0; i < lines.Length; ++i)
{
var line = lines[i];
if (!line.StartsWith("NM1*QC"))
{
continue;
}
var parts = line.Split('*');
parts[3] = "Lastname";
parts[4] = "Firstname";
lines[i] = string.Join("*", parts);
}

is there any way to insert data from text file to dataset?

i have text file that looks like this:
1 \t a
2 \t b
3 \t c
4 \t d
i have dataset: DataSet ZX = new DataSet();
is there any way for inserting the text file values to this dataset ?
thanks in advance

You will have to parse the file manually. Maybe like this:
string data = System.IO.File.ReadAllText("myfile.txt");
DataRow row = null;
DataSet ds = new DataSet();
DataTable tab = new DataTable();
tab.Columns.Add("First");
tab.Columns.Add("Second");
string[] rows = data.Split(new char[] { '\n' }, StringSplitOptions.RemoveEmptyEntries);
foreach (string r in rows)
{
string[] columns = r.Split(new char[] { '\t' }, StringSplitOptions.RemoveEmptyEntries);
if (columns.Length <= tab.Columns.Count)
{
row = tab.NewRow();
for (int i = 0; i < columns.Length; i++)
row[i] = columns[i];
tab.Rows.Add(row);
}
}
ds.Tables.Add(tab);
UPDATE
If you don't know how many columns in the text file you can modify my original example as the following (assuming that the number of columns is constant for all rows):
// ...
string[] columns = r.Split(new char[] { '\t' }, StringSplitOptions.RemoveEmptyEntries);
if (tab.Columns.Count == 0)
{
for(int i = 0; i < columns.Length; i++)
tab.Columns.Add("Column" + (i + 1));
}
if (columns.Length <= tab.Columns.Count)
{
// ...
Also remove the initial creation of table columns:
// tab.Columns.Add("First");
// tab.Columns.Add("Second")
-- Pavel

Sure there is,
Define a DataTable, Add DataColumn with data types that you want,
ReadLine the file, split the values by tab, and add each value as a DataRow to DataTable by calling NewRow.
There is a nice sample code at MSDN, take a look and follow the steps

Yes, create data tabel on the fly, refer this article for how-to
Read your file line by line and add those value to your data table , refer this article for how-to read text file

Try this
private DataTable GetTextToTable(string path)
{
try
{
DataTable dataTable = new DataTable
{
Columns = {
{"MyID", typeof(int)},
"MyData"
},
TableName="MyTable"
};
// Create an instance of StreamReader to read from a file.
// The using statement also closes the StreamReader.
using (StreamReader sr = new StreamReader(path))
{
String line;
// Read and display lines from the file until the end of
// the file is reached.
while ((line = sr.ReadLine()) != null)
{
string[] words = line.Split(new string[] { "\\t" }, StringSplitOptions.RemoveEmptyEntries);
dataTable.Rows.Add(words[0], words[1]);
}
}
return dataTable;
}
catch (Exception e)
{
// Let the user know what went wrong.
throw new Exception(e.Message);
}
}
Call it like
GetTextToTable(Path.Combine(Server.MapPath("."), "TextFile.txt"));
You could also check out CSV File Imports in .NET

I'd like also to add to the "volpan" code the following :
String _source = System.IO.File.ReadAllText(FilePath, Encoding.GetEncoding(1253));
It's good to add the encoding of your text file, so you can be able to read the data and in my case export those after modification to another file.