We Keep Coding

Why a method is modifying a List<string> variable in my Load Page?

I want to create two List with methods, the first method create a new list, and the second modify the first one but return a new one. For example, the first list has 200 items, and after add and delete some items in the second method, the returned one has 120 items.
But the second method is actually modifying the first list (now both list has 120 items).
What am I doing wrong?.
PD. I'm learning
protected void Page_Load(object sender, EventArgs e)
{
List<string> firstList = OEEClass.GetCompleteListDocument(DateTime.Today, "BZX"); // Let say it has 200 items
List<string> modifiedList = OEEClass.ModifyList(firstList); // The returned list has less items
}
public class OEEClass
{
public static List<string> GetCompleteListDocument(DateTime Fecha, string noMaquina)
{
var rawDoc = new HtmlDocument();
var tempList = new List<string>();
string url = #"C:/www/WPCS-Feedback/" + Fecha.Year + "/" + Fecha.Month + "/" + Fecha.Day.ToString("d2") + "/" + "Production state data.htm";
if (File.Exists(url) == true)
{
rawDoc.Load(url);
string cleanString = rawDoc.DocumentNode.InnerText.Trim().Replace(" ", "");
cleanString = Regex.Replace(cleanString, #"^\s+$[\r\n]*", string.Empty, RegexOptions.Multiline);
tempList = Regex.Split(cleanString, "\r\n|\r|\n").ToList();
tempList.RemoveRange(0, 5);
for (int j = 0; j < tempList.Count; j++)
{
if (tempList[j].StartsWith("ProductionTerminated") || tempList[j].StartsWith("ProductionInterrumpted"))
{
tempList.Insert(j + 4, "PressSingleCycleActivated=0");
}
}
}
return tempList;
}
public static List<string> ModifyList(List<string> completeListDocument)
{
for (int i = 0; i < completeListDocument.Count; i++)
{
if (completeListDocument[i].StartsWith("MachineSetup"))
{
completeListDocument.RemoveRange(i, 6);
i--;
}
}
return completeListDocument;
}
}

The simplest thing you can do is make a copy of your list before modifying it:
public static List<string> ModifyList(List<string> completeListDocument)
{
var results = new List<string>(completeListDocument);
for (int i = 0; i < results.Count; i++)
{
if (results[i].StartsWith("MachineSetup"))
{
results.RemoveRange(i, 6);
i--;
}
}
return results;
}

binary search in a sorted list in c#

I am retrieving client id\ drum id from a file and storing them in a list.
then taking the client id and storing it in another list.
I need to display the client id that the user specifies (input_id) on a Datagrid.
I need to get all the occurrences of this specific id using binary search.
the file is already sorted.
I need first to find the occurrences of input_id in id_list.
The question is: how to find all the occurrences of input_id in the sorted list id_list using binary search?
using(StreamReader sr= new StreamReader(path))
{
List<string> id_list = new List<string>();
List<string> all_list= new List<string>();
List<int> indexes = new List<int>();
string line = sr.ReadLine();
line = sr.ReadLine();
while (line != null)
{
all_list.Add(line);
string[] break1 = line.Split('/');
id_list.Add(break1[0]);
line = sr.ReadLine();
}
}
string input_id = textBox1.Text;
Data in the file:
client id/drum id
-----------------
123/321
231/3213
321/213123 ...

If the requirement was to use binary search I would create a custom class with a comparer, and then find an element and loop forward/backward to get any other elements. Like:
static void Main(string[] args
{
var path = #"file path...";
// read all the Ids from the file.
var id_list = File.ReadLines(path).Select(x => new Drum
{
ClientId = x.Split('/').First(),
DrumId = x.Split('/').Last()
}).OrderBy(o => o.ClientId).ToList();
var find = new Drum { ClientId = "231" };
var index = id_list.BinarySearch(find, new DrumComparer());
if (index != -1)
{
List<Drum> matches = new List<Drum>();
matches.Add(id_list[index]);
//get previous matches
for (int i = index - 1; i > 0; i--)
{
if (id_list[i].ClientId == find.ClientId)
matches.Add(id_list[i]);
else
break;
}
//get forward matches
for (int i = index + 1; i < id_list.Count; i++)
{
if (id_list[i].ClientId == find.ClientId)
matches.Add(id_list[i]);
else
break;
}
}
}
public class Drum
{
public string DrumId { get; set; }
public string ClientId { get; set; }
}
public class DrumComparer : Comparer<Drum>
{
public override int Compare(Drum x, Drum y) =>
x.ClientId.CompareTo(y.ClientId);
}

If i understand you question right then this should be a simple where stats.
// read all the Ids from the file.
var Id_list = File.ReadLines(path).Select(x => new {
ClientId = x.Split('/').First(),
DrumId = x.Split('/').Last()
}).ToList();
var foundIds = Id_list.Where(x => x.ClientId == input_id);

C# Constructing a Dynamic Query From DataTable

Trying to Generate a Dynamic Linq Query, based on DataTable returned to me... The column names in the DataTable will change, but I will know which ones I want to total, and which ones I will want to be grouped.
I can get this to work with loops and writing the output to a variable, then recasting the parts back into a data table, but I'm hoping there is a more elegant way of doing this.
//C#
DataTable dt = new DataTable;
Dt.columns(DynamicData1)
Dt.columns(DynamicData1)
Dt.columns(DynamicCount)
In this case the columns are LastName, FirstName, Age. I want to total ages by LastName,FirstName columns (yes both in the group by). So one of my parameters would specify group by = LastName, FirstName and another TotalBy = Age. The next query may return different column names.
Datarow dr =..
dr[0] = {"Smith","John",10}
dr[1] = {"Smith","John",11}
dr[2] = {"Smith","Sarah",8}
Given these different potential columns names...I'm looking to generate a linq query that creates a generic group by and Total output.
Result:
LastName, FirstName, AgeTotal
Smith, John = 21
Smith, Sarah = 8

If you use a simple converter for Linq you can achieve that easily.
Here a quick data generation i did for the sample :
// create dummy table
var dt = new DataTable();
dt.Columns.Add("LastName", typeof(string));
dt.Columns.Add("FirstName", typeof(string));
dt.Columns.Add("Age", typeof(int));
// action to create easily the records
var addData = new Action<string, string, int>((ln, fn, age) =>
{
var dr = dt.NewRow();
dr["LastName"] = ln;
dr["FirstName"] = fn;
dr["Age"] = age;
dt.Rows.Add(dr);
});
// add 3 datarows records
addData("Smith", "John", 10);
addData("Smith", "John", 11);
addData("Smith", "Sarah", 8);
This is how to use my simple transformation class :
// create a linq version of the table
var lqTable = new LinqTable(dt);
// make the group by query
var groupByNames = lqTable.Rows.GroupBy(row => row["LastName"].ToString() + "-" + row["FirstName"].ToString()).ToList();
// for each group create a brand new linqRow
var linqRows = groupByNames.Select(grp =>
{
//get all items. so we can use first item for last and first name and sum the age easily at the same time
var items = grp.ToList();
// return a new linq row
return new LinqRow()
{
Fields = new List<LinqField>()
{
new LinqField("LastName",items[0]["LastName"].ToString()),
new LinqField("FirstName",items[0]["FirstName"].ToString()),
new LinqField("Age",items.Sum(item => Convert.ToInt32(item["Age"]))),
}
};
}).ToList();
// create new linq Table since it handle the datatable format ad transform it directly
var finalTable = new LinqTable() { Rows = linqRows }.AsDataTable();
And finally here are the custom class that are used
public class LinqTable
{
public LinqTable()
{
}
public LinqTable(DataTable sourceTable)
{
LoadFromTable(sourceTable);
}
public List<LinqRow> Rows = new List<LinqRow>();
public List<string> Columns
{
get
{
var columns = new List<string>();
if (Rows != null && Rows.Count > 0)
{
Rows[0].Fields.ForEach(field => columns.Add(field.Name));
}
return columns;
}
}
public void LoadFromTable(DataTable sourceTable)
{
sourceTable.Rows.Cast<DataRow>().ToList().ForEach(row => Rows.Add(new LinqRow(row)));
}
public DataTable AsDataTable()
{
var dt = new DataTable("data");
if (Rows != null && Rows.Count > 0)
{
Rows[0].Fields.ForEach(field =>
{
dt.Columns.Add(field.Name, field.DataType);
});
Rows.ForEach(row =>
{
var dr = dt.NewRow();
row.Fields.ForEach(field => dr[field.Name] = field.Value);
dt.Rows.Add(dr);
});
}
return dt;
}
}
public class LinqRow
{
public List<LinqField> Fields = new List<LinqField>();
public LinqRow()
{
}
public LinqRow(DataRow sourceRow)
{
sourceRow.Table.Columns.Cast<DataColumn>().ToList().ForEach(col => Fields.Add(new LinqField(col.ColumnName, sourceRow[col], col.DataType)));
}
public object this[int index]
{
get
{
return Fields[index].Value;
}
set
{
Fields[index].Value = value;
}
}
public object this[string name]
{
get
{
return Fields.Find(f => f.Name == name).Value;
}
set
{
var fieldIndex = Fields.FindIndex(f => f.Name == name);
if (fieldIndex >= 0)
{
Fields[fieldIndex].Value = value;
}
}
}
public DataTable AsSingleRowDataTable()
{
var dt = new DataTable("data");
if (Fields != null && Fields.Count > 0)
{
Fields.ForEach(field =>
{
dt.Columns.Add(field.Name, field.DataType);
});
var dr = dt.NewRow();
Fields.ForEach(field => dr[field.Name] = field.Value);
dt.Rows.Add(dr);
}
return dt;
}
}
public class LinqField
{
public Type DataType;
public object Value;
public string Name;
public LinqField(string name, object value, Type dataType)
{
DataType = dataType;
Value = value;
Name = name;
}
public LinqField(string name, object value)
{
DataType = value.GetType();
Value = value;
Name = name;
}
public override string ToString()
{
return Value.ToString();
}
}

I think I'd just use a dictionary:
public Dictionary<string, int> GroupTot(DataTable dt, string[] groupBy, string tot){
var d = new Dictionary<string, int>();
foreach(DataRow ro in dt.Rows){
string key = "";
foreach(string col in groupBy)
key += (string)ro[col] + '\n';
if(!d.ContainsKey(key))
d[key] = 0;
d[key]+= (int)ro[tot];
}
return d;
}
If you want the total on each row, we could get cute and create a column that is an array of one int instead of an int:
public void GroupTot(DataTable dt, string[] groupBy, string tot){
var d = new Dictionary<string, int>();
var dc = dt.Columns.Add("Total_" + tot, typeof(int[]));
foreach(DataRow ro in dt.Rows){
string key = "";
foreach(string col in groupBy)
key += (string)ro[col] + '\n'; //build a grouping key from first and last name
if(!d.ContainsKey(key)) //have we seen this name pair before?
d[key] = new int[1]; //no we haven't, ensure we have a tracker for our total, for this first+last name
d[key][0] += (int)ro[tot]; //add the total
ro[dc] = d[key]; //link the row to the total tracker
}
}
At the end of the operation every row will have an array of int in the "Total_age" column that represents the total for that First+Last name. The reason I used int[] rather than int, is because int is a value type, whereas int[] is a reference. Because as the table is being iterated each row gets assigned a reference to an int[] some of them with the same First+Last name will end up with their int[] references pointing to the same object in memory, so incrementing a later one increments all the earlier ones too (all "John Smith" rows total column holds a refernece to the same int[]. If we'd made the column an int type, then every row would point to a different counter, because every time we say ro[dc] = d[key] it would copy the current value of d[key] int into ro[dc]'s int. Any reference type would do for this trick to work, but value types wouldn't. If you wanted your column to be value type you'd have to iterate the table again, or have two dictionaries, one that maps DataRow -> total and iterate the keys, assigning the totals back into the row

CSV-file values to List<List>

I have a CSV file that I want some values from. One problem is that I don't know how many columns the file has. The number can be different every time I get a new CSV file. It will always have columns and rows with values. I will get it from a normal excel-file.
I want the method to return a List<List>.
ListA(FirstName, LastName, PhoneNumber... and so on) here I don't know how many items ListA will have. It can be different every time.
Inside ListA I want lists of persons like this:
ListA[FirstName] = List1(Zlatan, Lionel, Anders.....)
ListA[LastName] = List2(Ibrahimovic, Messi, Svensson.....) .. and so on.

You could create a class Person
class person {
private string FirstName;
private string LastName;
// others
}
Open the File and split each row in the file with the String.Split()-Method then convert each value and create Objects, which you can add to a List.
List<Person> persons = new List<Person>();
persons.Add(personFromFile);
Thats a pretty short solution but it works
Edit: Variable Fields per Row
If thats the case you could use a List<string[]> stringArraylist; and then add the results of the String.Split()-Method to it.
List<string[]> stringArraylist;
stringArraylist = new List<string[]>();
stringArraylist.Add("Andrew;Pearson;...;lololo;".Split(';'));
Is that more of what you wanted?

There are a lot of questions on SO that deal with parsing CSV files. See here for one: Reading CSV files in C#. I am fairly certain there are some solutions built in to .NET, though I can't recall what they are at the moment. (#ZoharPeled suggested TextFieldParser)
Most of the parsing solutions with give you a collection of rows where each item is a collection of columns. So assuming you have something like a IEnumerable<IList<string>>, you could create a class and use LINQ queries to get what you need:
public class CSVColumns
{
public IEnumerable<IList<string>> CSVContents { get; private set; }
public CSVColumns(IEnumerable<IList<string>> csvcontents)
{
this.CSVContents = csvcontents;
}
public List<string> FirstNames
{
get { return GetColumn("FirstName"); }
}
public List<string> LastNames
{
get { return GetColumn("LastName"); }
}
/// <summary>
/// Gets a collection of the column data based on the name of the column
/// from the header row.
/// </summary>
public List<string> GetColumn(string columnname)
{
//Get the index of the column with the name
var firstrow = CSVContents.ElementAtOrDefault(0);
if (firstrow != null)
{
int index = -1;
foreach (string s in firstrow)
{
index++;
if (s == columnname)
{
return GetColumn(index, true);
}
}
}
return new List<string>();
}
/// <summary>
/// Gets all items from a specific column number but skips the
/// header row if needed.
/// </summary>
public List<string> GetColumn(int index, bool hasHeaderRow = true)
{
IEnumerable<IList<string>> columns = CSVContents;
if (hasHeaderRow)
columns = CSVContents.Skip(1);
return columns.Select(list =>
{
try
{
return list[index];
}
catch (IndexOutOfRangeException ex)
{
return "";
}
}
).ToList();
}
}

I finally got a solution and it's working for me. My friend made it so all creed to him. No user here on stackoverflow so I post it instead.
private List<Attributes> LoadCsv()
{
string filename = #"C:\Desktop\demo.csv";
// Get the file's text.
string whole_file = System.IO.File.ReadAllText(filename);
// Split into lines.
whole_file = whole_file.Replace('\n', '\r');
string[] lines = whole_file.Split(new char[] { '\r' },
StringSplitOptions.RemoveEmptyEntries);
// See how many rows and columns there are.
int num_rows = lines.Length;
int num_cols = lines[0].Split(';').Length;
// Allocate the data array.
string[,] values = new string[num_rows, num_cols];
// Load the array.
for (int r = 0; r < num_rows; r++)
{
string[] line_r = lines[r].Split(';');
for (int c = 0; c < num_cols; c++)
{
values[r, c] = line_r[c];
}
}
var attr = new List<Attributes>();
for (var r = 0; r < num_rows; r++)
{
if (r == 0)
{
for (var c = 0; c < num_cols; c++)
{
attr.Add(new Attributes());
attr[c].Name = values[r, c];
attr[c].Value = new List<String>();
}
}
else
{
for (var b = 0; b < num_cols; b++)
{
var input = values[r, b];
attr[b].Value.Add(input);
}
}
}
// Return the values.
return attr;
}

Performance problems with Linq To CSV

I have found a library which connects csv-files with linq. I understood the principles and my code works well. But i have some problems with big csv files.
http://www.codeproject.com/Articles/25133/LINQ-to-CSV-library
Now i want to access specific single items from my _dataTable object. I get them like this:
public class CsvFile
{
private IEnumerable<DataRow> _dataTable = cc.Read<DataRow>(_filePath, _inputFileDescription);
public string GetItem(int row, int column)
{
return _dataTable.ElementAt<DataRow>(row).ElementAt<DataRowItem>(column).Value;
}
}
When i now call the method like this in a loop:
CsvFile file1 = new CsvFile("C:\\dev_csvcompare\\Master.csv", ';', true);
for(int i = 0; i < 10000; i++)
{
string dummy = file1.GetItem(1, i); //Does not make sense, my loop is a bit more complicated
}
it gets very slow, because the IEnumerable opens the stream every call.
In the documentation(link) under "Deferred Reading" they say i can access the ienumerable "_dataTable" with a foreach loop (this does work fine), but this is in my case no option because i want access to specific items in the csv.
Are there possibilities to keep the filestream open so that the performace increases?
EDIT (My code, maybe a lot of nosense, im not so experienced with .net, c# and oop):
public void Compare(int key1, int key2, int col1, int col2)
{
string lastKeyCol1 = null;
string lastKeyCol2 = null;
List<string> sortedKeyColFile1 = new List<string>();
List<string> sortedKeyColFile2 = new List<string>();
int file1counter = 0;
int file2counter = 0;
int cnt = 0;
sortedKeyColFile1 = _file1.GetCol(key1);
sortedKeyColFile1.Sort();
sortedKeyColFile2 = _file2.GetCol(key2);
sortedKeyColFile2.Sort();
while ((file1counter < sortedKeyColFile1.Count) || (file2counter < sortedKeyColFile2.Count))
{
_outputList.Add(new OutputValues(key1, key2, col1, col2));
//Keys are in both files
if (sortedKeyColFile1[file1counter] == sortedKeyColFile2[file2counter])
{
if (lastKeyCol1 == sortedKeyColFile1[file1counter])
{
//Keys are redundant
_outputList[cnt].RedundantKeyF1 = true;
}
if (lastKeyCol2 == sortedKeyColFile2[file2counter])
{
//Keys are redundant
_outputList[cnt].RedundantKeyF2 = true;
}
lastKeyCol1 = sortedKeyColFile1[file1counter];
lastKeyCol2 = sortedKeyColFile2[file2counter];
_outputList[cnt].ValF1 = _file1.GetItem(file1counter, col1);
_outputList[cnt].ValF2 = _file2.GetItem(file2counter, col2);
_outputList[cnt].LineNumF1 = file1counter;
_outputList[cnt].LineNumF2 = file2counter;
//compare the values (because keys do match at this place)
_outputList[cnt].CompareResult = CompareString(_file1.GetItem(file1counter, col1), _file2.GetItem(file2counter, col2));
if (file1counter < sortedKeyColFile1.Count)
{
file1counter++;
}
if (file2counter < sortedKeyColFile2.Count)
{
file2counter++;
}
}
//Key sortedKeyColFile2[file2counter] is not in file 1
else if (file2counter < sortedKeyColFile2.Count && 0 < (string.Compare(sortedKeyColFile1[file1counter], sortedKeyColFile2[file2counter])))
{
_outputList[cnt].LineNumF2 = file2counter;
if (lastKeyCol2 == sortedKeyColFile2[file2counter])
{
//Keys are redundant
_outputList[cnt].RedundantKeyF2 = true;
}
lastKeyCol2 = sortedKeyColFile2[file2counter];
file2counter++;
}
//Key sortedKeyColFile1[file1counter] is not in file 2
else if (file1counter < sortedKeyColFile1.Count)
{
_outputList[cnt].LineNumF1 = file1counter;
if (lastKeyCol1 == sortedKeyColFile1[file1counter])
{
//Keys are redundant
_outputList[cnt].RedundantKeyF1 = true;
}
lastKeyCol1 = sortedKeyColFile1[file1counter];
file1counter++;
}
cnt++;
}
}
//And here the important part of the csv-file class, maybe not so interesting
public class CsvFile
{
private string _filePath = null;
private char _separator = ',';
private bool _hasHeader = true;
private CsvContext _cc = null;
private CsvFileDescription _inputFileDescription = null;
private List<string> _headers = null;
private IEnumerable<DataRow> _dataTable = null;
/// <summary>
/// Constructor for a new CsvFile object.
/// The Constructor initiates the Object and read the values out of the File
/// </summary>
/// <param name="filePath">Full path of the csv-file</param>
/// <param name="separator">Seperator of the csv-file, eg: ';' or ',' or '\t'</param>
/// <param name="hasHeader">Is true if the first col of the csv-file contains a headers</param>
public CsvFile(string filePath, char separator, bool hasHeader = true)
{
//Throws an exception if something is wrong with the file
File.OpenRead(filePath);
_filePath = filePath;
_separator = separator;
_hasHeader = hasHeader;
_cc = new CsvContext();
_inputFileDescription = new CsvFileDescription
{
SeparatorChar = separator,
FirstLineHasColumnNames = hasHeader
};
_dataTable = _cc.Read<DataRow>(_filePath, _inputFileDescription);
if (hasHeader)
{
ParseHeaders();
}
}
public List<string> GetCol(int col)
{
List<string> column = new List<string>();
int cnt = 0;
foreach(DataRow x in _dataTable)
{
column.Add(x[col].Value);
cnt++;
}
return column;
}
private void ParseHeaders()
{
System.IO.StreamReader file = new System.IO.StreamReader(_filePath);
if (!file.EndOfStream)
{
//_headers = file.ReadLine().Split(_separator);
_headers = new List<string> (file.ReadLine().Split(_separator));
}
file.Close();
}
}

Try this:
public class CsvFile
{
private IEnumerable<DataRow> rows = cc.Read<DataRow>(_filePath, _inputFileDescription);
//...
public IEnumerable<DataRow> Rows { get { return rows; } }
}
And then:
CsvFile file1 = new CsvFile("C:\\dev_csvcompare\\Master.csv", ';', true);
foreach(DataRow row in file1.Rows)
{
string dummy = row[1];
}