We Keep Coding

C# Node Collection using HTMLAgilityPack to Data table

I am trying to scrape data from this site. I am able to obtain the data but now I need add the selected data to a data table. The following is very near complete but it is only returning the last record. I know its something simple but just cant figure it out. Essentially, the commented out portion for the Console.Writeline portion in the for loop will be the desired results in the datagridview after all said and done.
private void button1_Click(object sender, EventArgs e)
{
var doc = new HtmlWeb().Load("https://www.sportingcharts.com/nba/defense-vs-position/");
HtmlAgilityPack.HtmlNodeCollection teams = doc.DocumentNode.SelectNodes("//div[#class='col col-md-3']//tr/td[2]");
HtmlAgilityPack.HtmlNodeCollection points = doc.DocumentNode.SelectNodes(".//div[#class='col col-md-3']//tr/td[3]");
HtmlAgilityPack.HtmlNodeCollection positions = doc.DocumentNode.SelectNodes(".//div[#class='col col-md-3']//span[1]");
DataTable dvp_dt = new DataTable();
dvp_dt.Columns.Add("Team", typeof(string));
dvp_dt.Columns.Add("Points", typeof(string));
dvp_dt.Columns.Add("Position", typeof(string));
string[] positions_aux = positions.Where(x => x.InnerText.Length >= 6).Select(y => y.InnerText).ToArray();
DataRow row = dvp_dt.NewRow();
for (int i = 0; i < teams.Count - 1; i++)
{
var aux = i / 30;
row["Team"] = (teams[i].InnerText);
row["Points"] = (points[i].InnerText);
row["Position"] = (positions_aux[aux]);
// Console.WriteLine(teams[i].InnerText + ' ' + points[i].InnerText + ' ' + positions_aux[aux]);
}
dvp_dt.Rows.Add(row);
dataGridView2.DataSource = dvp_dt;
}

Try following :
for (int i = 0; i < teams.Count - 1; i++)
{
DataRow row = dvp_dt.NewRow();
var aux = i / 30;
row["Team"] = (teams[i].InnerText);
row["Points"] = (points[i].InnerText);
row["Position"] = (positions_aux[aux]);
}
dataGridView2.DataSource = null;
dataGridView2.DataSource = dvp_dt;

Linq Extension Methods and Error Handling

I have the following C# code...
// We're essentially pivoting the data, using LINQ's GroupBy.
var pivotedOperands = Operands.GroupBy(o => new { outputid = (Guid)o[DETAILS_OUTPUTID], unitid = o[DETAILS_UNITID] })
.Select(g => new
{
PivotKey = g.Key,
c1 = g.Where(x => (int)x[DETAILS_OV_SEQUENCEID] == 1).Sum(x => double.Parse(x[DETAILS_VALUE].ToString())),
r1 = g.Where(x => (int)x[DETAILS_OV_SEQUENCEID] == 2).Sum(x => double.Parse(x[DETAILS_VALUE].ToString())),
a1 = g.Where(x => (int)x[DETAILS_OV_SEQUENCEID] == 3).Sum(x => double.Parse(x[DETAILS_VALUE].ToString()))
});
It takes the data in Operands (which is a List object) and uses the GroupBy() extension method to perform a pivot on the data. Essentially c1, r1 and a1 are all values in different DataRow objects with sequence IDs of 1, 2 and 3 respectively. (I can go into more detain on that if it becomes necessary, but I think it won't.)
So sometimes the value for c1 might be empty. (It's not supposed to, but bugs have happened further upstream in the process from time to time.) If c1 is not a numeric value, the double.Parse() call will raise an exception. That's fine. Here's my problem. If the Operands object contains, for example, 9 rows that will be pivoted into 3 rows and one of those nine values is not numeric, is it possible to determine which DataRow object raised the exception?
example:
If Operands contains the following values for SequenceID and Value...
OutputID UnitID SequenceID Value
A 1 1 '0'
A 1 2 '0'
A 1 3 '0'
A 2 1 ''
A 2 2 '0'
A 2 3 '0'
B 1 1 '0'
B 1 2 '0'
B 1 3 '0'
...then we will get an "Input string was not in a correct format" exception when it tries to process the empty string through the double.Parse() method for the 4th row of my data set. I want to raise a friendly exception to the users telling them which row is the problem; not just that there was a problem somewhere in this set of data. Is it possible to identify exactly what caused the exception?
If you create a new C# console application in Visual studio and dump the following code into the Main method, you will be able to reproduce my problem.
// Create a DataTable so that we can easily create new DataRows to add to our List.
DataTable dt = new DataTable();
DataColumn col = new DataColumn();
col.DataType = System.Type.GetType("System.String");
col.ColumnName = "OutputID";
dt.Columns.Add(col);
col = new DataColumn();
col.DataType = System.Type.GetType("System.Int32");
col.ColumnName = "UnitID";
dt.Columns.Add(col);
col = new DataColumn();
col.DataType = System.Type.GetType("System.Int32");
col.ColumnName = "SequenceID";
dt.Columns.Add(col);
col = new DataColumn();
col.DataType = System.Type.GetType("System.String");
col.ColumnName = "Value";
dt.Columns.Add(col);
// Create the List and add our sample data
List<DataRow> Operands = new List<DataRow>();
DataRow dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "1";
dr["SequenceID"] = 1;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "1";
dr["SequenceID"] = 2;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "1";
dr["SequenceID"] = 3;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "2";
dr["SequenceID"] = 1;
dr["Value"] = ""; // This should cause an error.
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "2";
dr["SequenceID"] = 2;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "2";
dr["SequenceID"] = 3;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "B";
dr["UnitID"] = "1";
dr["SequenceID"] = 1;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "B";
dr["UnitID"] = "1";
dr["SequenceID"] = 2;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "B";
dr["UnitID"] = "1";
dr["SequenceID"] = 3;
dr["Value"] = "0";
Operands.Add(dr);
// Now pivot the data
try
{
var pivotedOperands = Operands.GroupBy(o => new { outputid = o[0], unitid = o[1] })
.Select(g => new
{
PivotKey = g.Key,
c1 = g.Where(x => (int)x[2] == 1).Sum(x => double.Parse(x[3].ToString())),
r1 = g.Where(x => (int)x[2] == 2).Sum(x => double.Parse(x[3].ToString())),
a1 = g.Where(x => (int)x[2] == 3).Sum(x => double.Parse(x[3].ToString()))
});
foreach (var o in pivotedOperands)
{
Console.WriteLine(string.Format("c1 = {0}; r1 = {1}; a1 = {2}", o.c1, o.r1, o.a1));
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
Console.WriteLine("Done.");
Console.ReadLine();

Depending on how you want the information surfaced, you can either change the type of your results to account for the possibility of failure, or you can capture contextual information about the exception and throw a new exception with more information in it.
In either case, don't be afraid to use helper methods. For example, suppose you got rid of the repetitive code in your selector by creating a method like this:
string GetSumOrErrorMessage(int idToMatch, IEnumerable<DataRow> dataRow)
{
try
{
var sum = dataRow.Where(x => (int)x[2] == idToMatch).Sum(x => double.Parse(x[3].ToString()));
return sum.ToString();
}
catch (Exception)
{
return "Error happened here"; // or something more specific
}
}
Now you can change your query like this:
var pivotedOperands = Operands.GroupBy(o => new { outputid = o[0], unitid = o[1] })
.Select(g => new
{
PivotKey = g.Key,
c1 = GetSumOrErrorMessage(1, g),
r1 = GetSumOrErrorMessage(2, g),
a1 = GetSumOrErrorMessage(3, g)
});
And your output turns into:
c1 = 0; r1 = 0; a1 = 0
c1 = Error happened here; r1 = 0; a1 = 0
c1 = 0; r1 = 0; a1 = 0
If you like this pattern, rather than just returning a string you may want to look into specialized Monadic types that can help with this. For example, you could create a class that has a generic value when an action is successful, or an error message when it's not. You can create a variety of extension methods and helpers to make this easier to deal with, similar to how my CallMeMaybe library would allow you to attempt to parse a value, but just return an empty Maybe<> if parsing fails. (e.g. Maybe.From(x[3].ToString()).ParseInt64().Select(i => i.ToString()).Else("Error happened here")).
Alternatively, if you actually want to halt when you get bad input, but still want to know where the bad input was, you can catch and throw:
double GetSum(int idToMatch, IGrouping<object, DataRow> dataRows)
{
try
{
return dataRows.Where(x => (int)x[2] == idToMatch).Sum(x => double.Parse(x[3].ToString()));
}
catch (Exception e)
{
throw new Exception($"Failure when matching {idToMatch} with group {dataRows.Key}", e);
}
}
...
var pivotedOperands = Operands.GroupBy(o => new { outputid = o[0], unitid = o[1] })
.Select(g => new
{
PivotKey = g.Key,
c1 = GetSum(1, g),
r1 = GetSum(2, g),
a1 = GetSum(3, g)
});
Output:
c1 = 0; r1 = 0; a1 = 0
Failure when matching 1 with group { outputid = A, unitid = 2 }

You try using TryParse to get around the exception. If TryParse is false then default to zero (0)
.Sum(x => {
double value = 0;
return double.TryParse(x[DETAILS_VALUE].ToString(), out value) ? value : 0;
})

how to copy a row of a dataset into another dataset? [duplicate]

This question already has answers here:
Copy rows from one Datatable to another DataTable?
(15 answers)
Closed 7 years ago.
i am looping through a dataset with 2 loops trying to find the rows which have have their ID matched with the assigned value to an array and if they matched i would like to copy that row into another table. for example:
DataSet dsWinners = new DataSet();
for(int i =0;i<=TotalWinners;i++)
{
for (int j = 1; j <= ds.Tables[0].Rows.Count; j++)
{
//this is my ds table 0
left = Convert.ToInt16(ds.Tables[0].Rows[i]["ID"]);
//this is my array
right = Convert.ToInt16(Winners[i, 0]);
if ( left == right )//if array value matechs with ds.table[0] ID
{
dsWinners.Tables[0].Rows[i] = ds.Tables[0].Rows[j];
}
}
}
how can i get record/row And copy it into a new table and if i cant copy that row like this, is there an alternative way to do it?

DataTable tempDt = new DataTable();
tempDt = ds.Tables[0].Clone();
ds.Tables[0].Rows.OfType<DataRow>().ToList().ForEach(x =>
{
int rowIndex = ds.Tables[0].Rows.IndexOf(x);
if (rowIndex < TotalWinners &&
Convert.ToInt16(x["ID"]) == Convert.ToInt16(Winners[rowIndex, 0]))
{
tempDt.ImportRow(x);
}
});
DataSet dsWinners = new DataSet();
dsWinners.Tables.Add(tempDt);
EDIT:
Dictionary<string, string> winnersList = new Dictionary<string, string>();
for (int i = 0; i < TotalWinners; i++)
{
winnersList.Add(Winners[i, 0], Winners[i, 1]);
}
string idList = string.Join("','", winnersList.Select(x => x.Key));
DataSet dsWinners = new DataSet();
dsWinners.Tables.Add(ds.Tables[0].Select("ID IN ('" + idList + "')").CopyToDataTable());
dsWinners.Tables[0].Columns.Add("prize_amt", typeof(string));
dsWinners.Tables[0].Rows.OfType<DataRow>().ToList().ForEach(x =>
{
x["prize_amt"] = winnersList[x["ID"].ToString()];
});
dsWinners.Tables[0].AcceptChanges();
Hope this helps...

DataSet dsWinners = new DataSet();
DataTable dataTable= ds.Tables[0].Clone();
for(int i =0;i<=TotalWinners;i++)
{
for (int j = 1; j <= ds.Tables[0].Rows.Count; j++)
{
//this is my ds table 0
left = Convert.ToInt16(ds.Tables[0].Rows[i]["ID"]);
//this is my array
right = Convert.ToInt16(Winners[i, 0]);
if ( left == right )//if array value matechs with ds.table[0] ID
{
// dsWinners.Tables[0].Rows[i] = ds.Tables[0].Rows[j];
dataTable.Rows.Add( ds.Tables[0].Rows[j]);
}
}
}
dsWinners.Add(dataTable);

DataSet ds = new DataSet();
DataTable dt = new DataTable("ExampleTable");
dt.Columns.Add(new DataColumn("Age",typeof(int)));
dt.Columns.Add(new DataColumn("Name", typeof(string)));
DataRow dr = dt.NewRow();
dr["Age"] = 30;
dr["Name"] = "Mike";
dt.Rows.Add(dr);
ds.Tables.Add(dt);

If i read your question right then you need to copy rows from one dataset when certain condition is met and transfer it to other one.Assuming that both the dataset have same structure this example should suffice.
DataSet DSResult = new DataSet();
DSResult.Tables.Add();
DSResult.Tables[0].Columns.Add("ID", typeof(int));
DSResult.Tables[0].Columns.Add("Name", typeof(string));
DSResult.Tables[0].Rows.Add(1,"Jon");
DSResult.Tables[0].Rows.Add(2, "Kyle");
DSResult.Tables[0].Rows.Add(3, "Sam");
DSResult.Tables[0].Rows.Add(4, "Peter");
DSResult.Tables[0].Rows.Add(5, "Lily");
DataSet DSWinners = new DataSet();
DSWinners.Tables.Add();
DSWinners = DSResult.Clone();
int[] Id = new int[] { 1, 4, 5 }; //condition to match
foreach (int val in Id)
{
DataRow[] Samplerow =
DSResult.Tables[0].AsEnumerable()
.Select((row, index) => new { row, index })
.Where(item => item.row.Field<int>("ID") == val)
.Select(item => item.row)
.ToArray();
DSWinners.Tables[0].ImportRow(Samplerow[0]);
// If both tables are not same then
string YourVal=Samplerow[0]["ID"].ToString();
DSWinners.Tables[0].Rows.Add();
DSWinners.Tables[0].Rows[0]["YourColumnname"]=Yourval //Should have same Datataype
}

A data item was not found in the container. The container must either implement IDataItemContainer, or have a property named DataItem

I am trying to bind string array to grid view. While using the given below code showing the error "A data item was not found in the container. The container must either implement IDataItemContainer, or have a property named DataItem." Please help me to find a proper solution. Thank you.
Code:
protected void ddlCircle_SelectedIndexChanged(object sender, EventArgs e)
{
ShadingAnalysisDataSetTableAdapters.tbl_CadEngineersTeamTableAdapter cd;
cd = new ShadingAnalysisDataSetTableAdapters.tbl_CadEngineersTeamTableAdapter();
DataTable dt = new DataTable();
dt = cd.GetAvailableData(ddlCircle.SelectedValue); // Getting details of unassigned site
int x, y; //z;
DataTable dt3 = new DataTable();
dt3 = cd.GetTeam();
y = dt3.Rows.Count;
x = dt.Rows.Count; // counting the unassinged sites
DataTable dt2 = new DataTable();
dt2 = cd.GetAssignTeam(x); //Getting team based on count
string[] arr = new string[dt2.Rows.Count];
int i = 0;
foreach (DataRow r in dt2.Rows)
{
arr[i] = r["Team"].ToString(); // assigning available team to array
i++;
}
string[] strArr = new string[100]; // another array to copy arr values.
i = 0; int j = 0;
while (j <= x)
{
strArr[j]= arr[i] ; // copying the arr[] values into strArr[] based on count.
i++;
j++;
if (i == 3)
{
i = 0;
}
}
GridView2.DataSource = strArr;
GridView2.DataBind(); // error popup here
}

Define a GridView's column such that it binds to the Team column of your DataTable and assign the DataTable directly to the GridView as DataSource. Then DataBind to the DataTable.

Binding Array to DataGrid is just like putting bananas in egg tray. Please you have to bind a source having structure according to datagrid. As suggested by #Konstantin D - Infragistics

Now the gridview showing strArr[j] array values
protected void ddlCircle_SelectedIndexChanged(object sender, EventArgs e)
{
ShadingAnalysisDataSetTableAdapters.tbl_CadEngineersTeamTableAdapter cd;
cd = new ShadingAnalysisDataSetTableAdapters.tbl_CadEngineersTeamTableAdapter();
DataTable dt = new DataTable();
dt = cd.GetAvailableData(ddlCircle.SelectedValue); // Getting details of unassigned site
int x, y; //z;
DataTable dt3 = new DataTable();
dt3 = cd.GetTeam();
y = dt3.Rows.Count;
x = dt.Rows.Count; // counting the unassinged sites
DataTable dt2 = new DataTable();
dt2 = cd.GetAssignTeam(x); //Getting team based on count
string[] arr = new string[dt2.Rows.Count];
int i = 0;
foreach (DataRow r in dt2.Rows)
{
arr[i] = r["Team"].ToString(); // assigning available team to array
i++;
}
string[] strArr = new string[x+1]; // another array to copy arr values.
i = 0; int j = 0;
while (j <= x)
{
strArr[j]= arr[i] ; // copying the arr[] values into strArr[] based on count.
i++;
j++;
if (i == 3)
{
i = 0;
}
}
GridView2.DataSource = strArr;
GridView2.DataBind();
}

Update two columns in a DataTable using LINQ

I want to update two columns of DataTable in a single line using LINQ query. Currently I am using following two lines to do the same:
oldSP.Select(string.Format("[itemGuid] = '{0}'", itemGuid)).ToList<DataRow>().ForEach(r => r["startdate"] = stDate);
oldSP.Select(string.Format("[itemGuid] = '{0}'", itemGuid)).ToList<DataRow>().ForEach(r => r["enddate"] = enDate);
How can I do this in one line, using one Select?

You can do it in one 'line', just pass appropriate action delegate to ForEach method:
oldSP.Select(string.Format("[itemGuid] = '{0}'", itemGuid))
.ToList<DataRow>()
.ForEach(r => {
r["startdate"] = stDate;
r["enddate"] = enDate;
});
Also you can use LINQ to DataSet (looks more readable to me, than one-liner):
var rowsToUpdate =
oldSP.AsEnumerable().Where(r => r.Field<string>("itemGuid") == itemGuid);
foreach(var row in rowsToUpdate)
{
row.SetField("startdate", stDate);
row.SetField("enddate", enDate);
}

Use curly bracers to do two on more operations:
oldSP.Select(string.Format("[itemGuid] = '{0}'", itemGuid))
.ToList<DataRow>()
.ForEach(r => { r["enddate"] = enDate); r["startdate"] = stDate; });
But for code readability I would use old-fashioned foreach loop.

Try this :
oldSP.Select(string.Format("[itemGuid] = '{0}'", itemGuid)).ToList<DataRow>()
.ForEach(r => { r["startdate"] = stDate; r["enddate"] = enDate; });

I didn't like any of the examples I saw on the web, so here's my example
DataTable dt = new DataTable();
dt.Columns.Add("Year");
dt.Columns.Add("Month");
dt.Columns.Add("Views");
for (int year = 2011; year < 2015; year++)
{
for (int month = 1; month < 13; month++)
{
DataRow newRow = dt.NewRow();
newRow[0] = year;
newRow[1] = month;
newRow[2] = 0;
dt.Rows.Add(newRow);
}
}
dataGridView1.DataSource = dt;
//if using Lambda
//var test = dt.AsEnumerable().Where(x => x.Field<string>("Year") == "2013" && x.Field<string>("Month") == "2").ToList();
var test = (from x in dt.AsEnumerable()
where x.Field<string>("Year") == "2013"
where x.Field<string>("Month") == "2"
select x).ToList();
test[0][0] = "2015";
dt.AcceptChanges();
//if writing to sql use dt.SubmitChanges() instead