We Keep Coding

Scrape data from local HTML file

I am trying to extract the values for Design Capacity mWh and Full Charged Capacity mWh from windows battery-report.html the HTML document stores these values in a table but with no attribute name I can easily access
I do have AngleSharp added but don't have much idea how to use it in this case to get the data I need it may not be right for the job though.
</td>
</tr></thead>
<tr>
<td><span class="label">NAME</span></td>
<td>Blade</td>
</tr>
<tr>
<td><span class="label">MANUFACTURER</span></td>
<td>Razer</td>
</tr>
<tr>
<td><span class="label">SERIAL NUMBER</span></td>
<td>CNB1RC30-027097A00283-A05</td>
</tr>
<tr>
<td><span class="label">CHEMISTRY</span></td>
<td>Li-I</td>
</tr>
<tr>
<td><span class="label">DESIGN CAPACITY</span></td>
<td>65,003 mWh
</td>
</tr>
<tr style="height:0.4em;"></tr>
<tr>
<td><span class="label">FULL CHARGE CAPACITY</span></td>
<td>72,395 mWh
</td>
</tr>
<tr>
<td><span class="label">CYCLE COUNT</span></td>
<td>
I generate the battery report and pass that togetBattery
private void BatteryHealthBtn_Click(object sender, EventArgs e)
{
string designCap = null;
string fullCap = null;
ManagementObjectSearcher mybatteryObject = new ManagementObjectSearcher("select * from Win32_Battery");
foreach (ManagementObject obj in mybatteryObject.Get())
{
if (obj["DesignCapacity"] != null || obj["FullChargeCapacity"] != null)
{
designCapTxt.Text = obj["DesignCapacity"].ToString();
fullCapTxt.Text = obj["FullChargeCapacity"].ToString();
}
else
{
MessageBox.Show("No WMI Data Found Running Manually", "Error No WMI",
MessageBoxButtons.OK, MessageBoxIcon.Error);
var saveLocation = System.AppDomain.CurrentDomain.BaseDirectory + "battery-report.html";
if (saveLocation != null)
{
System.Diagnostics.Process process = new System.Diagnostics.Process();
System.Diagnostics.ProcessStartInfo startInfo = new System.Diagnostics.ProcessStartInfo();
startInfo.FileName = "cmd.exe";
startInfo.Arguments = "/C powercfg /batteryreport /output " + '"' + saveLocation + '"';
process.StartInfo = startInfo;
process.Start();
System.Diagnostics.Process.Start(saveLocation);
GetBattery(saveLocation);
}
}
}
}
Image of the Hmtl Document
public async void GetBattery(string html)
{
var config = Configuration.Default.WithDefaultLoader();
string address = html;
IDocument document = await
BrowsingContext.New(config).OpenAsync(address);
var designCap = document.GetElementsByClassName("label");
MessageBox.Show(designCap.ToString(), "a",
MessageBoxButtons.OK, MessageBoxIcon.Error);
}
think i am getting closer with this but getting null reference still on line 4
var config = Configuration.Default.WithDefaultLoader();
var address = html;
var document = await BrowsingContext.New(config).OpenAsync(address);
var cellSelector = "tr td:nth-child(2)";
var cells = document.QuerySelectorAll(cellSelector);
var designCap = cells.Select(m => m.TextContent);

had to swap over to html agility pack but i got it
HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument();
// There are various options, set as needed
htmlDoc.OptionFixNestedTags = true;
// filePath is a path to a file containing the html
htmlDoc.Load(saveLocation);
foreach (HtmlNode table in htmlDoc.DocumentNode.SelectNodes("//table"))
{
foreach (HtmlNode row in table.SelectNodes("tr"))
{
if(row.InnerText.Contains("DESIGN CAPACITY"))
{
designCapTxt.Text = row.InnerText;
}
if (row.InnerText.Contains("FULL CHARGE CAPACITY"))
{
fullCapTxt.Text = row.InnerText;
}
}
}

Trying to use id generated dynamically in asp.net

I am trying to use the id generated dynamically in asp.net.
<tr id="stdColTr" runat="server">
<td style="font-size:10pt">
<b>Nominal Tol(<asp:Label id="lblStdSize_Unit" runat="server" Text=""></asp:Label>)</b>
</td>
<td>
<asp:Label id="lblStdSize_Toler" runat="server" Text=""></asp:Label>
</td>
</tr>
I have a label in uw.aspx and I am trying to bind the table in it and trying to check whether if value read from database is zero then that particular td should not be visible in that table.
uw.aspx.cs
lblStdSize_Toler.Text = "<table ><thead><tr><th colspan='2'>Diameter</th><th colspan='2'>Tolerance</th></tr></thead><tr><td id='STd1'>From</td><td id='STd2'>To</td><td id='STd3'>+</td><td id='STd4'>-</td></tr>";
while (ReadData.Read())
{
prop_name = ReadData["prop_name"].ToString();
t_property = ReadData["tprop"].ToString();
}
lblStdSize_Unit.Text = ReadData["WD_PROP"].ToString();
if (prop_name == "1sizMin")
{
if (t_property=="0")
{
first td with id SD1 should be hidden.
}
lblStdSize_Toler.Text = lblStdSize_Toler.Text + "<td>" + tdc_property + "</td>";
}
Similarly remaining td checked with t_property if value is "0" then it should be visible false.

If I understand you correctly this should do what you need. It creates a row per recordset returned an applies a style (visibility) to the first column (STd1). I don't know anything about the code environment and I also just added the code as I think it could work. Please adjust variable names or typos to your liking.
lblStdSize_Toler.Text = "<table ><thead><tr><th colspan='2'>Diameter</th><th colspan='2'>Tolerance</th></tr></thead>";
while (ReadData.Read())
{
var row = "<tr><td id='STd1' style='visibility: {0};'>From</td><td id='STd2'>To</td><td id='STd3'>+</td><td id='STd4'>-</td></tr>";
var visibility = "visible";
prop_name = ReadData["prop_name"].ToString();
t_property = ReadData["tprop"].ToString();
lblStdSize_Unit.Text = ReadData["WD_PROP"].ToString();
if (prop_name == "1sizMin")
{
if (t_property=="0")
{
visibility = "hidden";
}
}
lblStdSize_Toler.Text += string.Format(row, visibility) + "<td>" + tdc_property + "</td>";
}

reference datatable column in html string C#

Building an string named html in the aspx.CS page using C# language.
Assuming DataTable dt.
This obviously doesn't work:
string html = "dt["FirstName"]";
Neither does this:
string html = "dt[\"FirstName\"]";
Neither does this:
string html = ""+dt+"[\"FirstName\"]";
Full Code (was really just trying to keep this question simple, but this might help for understanding):
public static string ConvertDataTableToHtmlTable(DataTable dt)
{
string html = "<h3>Locations:</h3><table class=\"table table-condensed\" style=\"border-collapse:collapse;\">";
//add header row
html += "<thead><tr><th> </th><th>City</th><th>Contact</th><th>Stations</th></tr></thead><tbody>";
int i = 1;
//add rows
foreach (DataRow row in dt.Rows)
{
//visible row
html += "<tr data-toggle=\"collapse\" data-target=\"#demo"+i+"\" class=\"accordion-toggle\">";
html += "<td><button class=\"btn btn-default btn-xs\" onclick=\"return false;\"><span class=\"glyphicon glyphicon-option-horizontal\"></span></button></td>";
html += "<td>"+dt[\"CompanyCity\"]+</td>";
html += "<td>"+dt[\"FirstName\"]+</td>";
html += "<td> 11 </td></tr>";
//collapsable row
html += "<tr><td colspan = \"12\" class=\"hiddenRow\"><div class=\"accordian-body collapse\" id=\"demo"+i+"\">";
html += "<table class=\"table table-striped\"><thead>";
html += "<tr><th>License Key</th><th>Start</th><th>Expire</th><th>Product</th><th>Version</th><th>Level</th><th>Model</th><th>User</th><th>Email</th></tr></thead><tbody>";
html += "<tr><td>Instalcode </td><td> start </td><td> expire </td><td> product </td><td> version </td><td> level </td><td> model </td><td> user </td><td> email </td></tr>";
html += "</tbody></table></div></td></tr>";
i++;
}
html += "</tbody></table>";
return html;
}
Page load (dtActive is the datatable with data)
protected void Page_Load(object sender, EventArgs e)
{
locationsTable.InnerHtml = ConvertDataTableToHtmlTable(dtActive);
}
ASPX Page:
<div id="locationsTable" runat="server"></div>
Any help is appreciated and of course if you need me to be more specific, or if this is a duplicate please comment below with your request/link. Thank you

You need to iterate through the records of the datatable
foreach (DataRow dr in dt)
{
String firstname = dr["FirstName"] != null ? dr["FirstName"].ToString() : string.Empty;
}
edit
Based on your editing:
You're iterating but using the wrong reference, use row["FirstName"] instead of dt["FirstName"]

Getting data from HTML table into a datatable

Ok so I need to query a live website to get data from a table, put this HTML table into a DataTable and then use this data. I have so far managed to use Html Agility Pack and XPath to get to each row in the table I need but I know there must be a way to parse it into a DataTable. (C#) The code I am currently using is:
string htmlCode = "";
using (WebClient client = new WebClient())
{
htmlCode = client.DownloadString("http://www.website.com");
}
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(htmlCode);
//My attempt at LINQ to solve the issue (not sure where to go from here)
var myTable = doc.DocumentNode
.Descendants("table")
.Where(t =>t.Attributes["summary"].Value == "Table One")
.FirstOrDefault();
//Finds all the odd rows (which are the ones I actually need but would prefer a
//DataTable containing all the rows!
foreach (HtmlNode cell in doc.DocumentNode.SelectNodes("//tr[#class='odd']/td"))
{
string test = cell.InnerText;
//Have not gone further than this yet!
}
The HTML table on the website I am querying looks like this:
<table summary="Table One">
<tbody>
<tr class="odd">
<td>Some Text</td>
<td>Some Value</td>
</tr>
<tr class="even">
<td>Some Text1</td>
<td>Some Value1</td>
</tr>
<tr class="odd">
<td>Some Text2</td>
<td>Some Value2</td>
</tr>
<tr class="even">
<td>Some Text3</td>
<td>Some Value3</td>
</tr>
<tr class="odd">
<td>Some Text4</td>
<td>Some Value4</td>
</tr>
</tbody>
</table>
I'm not sure whether it is better/easier to use LINQ + HAP or XPath + HAP to get the desired result, I tried both with limited success as you can probably see. This is the first time I have ever made a program to query a website or even interact with a website in any way so I am very unsure at the moment! Thanks for any help in advance :)

Using some of Jack Eker's code above and some code from Mark Gravell (see post here) , I managed to come with a solution.
This code snippet is used to obtain the public holidays for the year of 2012 in South Africa as of writing this article
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Web;
using System.Net;
using HtmlAgilityPack;
namespace WindowsFormsApplication
{
public partial class Form1 : Form
{
private DataTable dt;
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
string htmlCode = "";
using (WebClient client = new WebClient())
{
client.Headers.Add(HttpRequestHeader.UserAgent, "AvoidError");
htmlCode = client.DownloadString("http://www.info.gov.za/aboutsa/holidays.htm");
}
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(htmlCode);
dt = new DataTable();
dt.Columns.Add("Name", typeof(string));
dt.Columns.Add("Value", typeof(string));
int count = 0;
foreach (HtmlNode table in doc.DocumentNode.SelectNodes("//table"))
{
foreach (HtmlNode row in table.SelectNodes("tr"))
{
if (table.Id == "table2")
{
DataRow dr = dt.NewRow();
foreach (var cell in row.SelectNodes("td"))
{
if ((count % 2 == 0))
{
dr["Name"] = cell.InnerText.Replace(" ", " ");
}
else
{
dr["Value"] = cell.InnerText.Replace(" ", " ");
dt.Rows.Add(dr);
}
count++;
}
}
}
dataGridView1.DataSource = dt;
}
}
}
}

There's no such method out of the box from the HTML Agility Pack, but it shouldn't be too hard to create one. There's samples out there that do XML to Datatable from Linq-to-XML. These can be re-worked into what you need.
If needed I can help out creating the whole method, but not today :).
See also:
HTML Agility pack - parsing tables
parsing html with HTMLAGILITYPACK and loading into datatable C#

This is my solution. May be a bit messy but it is working perfectly at the moment :D
string htmlCode = "";
using (WebClient client = new WebClient())
{
client.Headers.Add(HttpRequestHeader.UserAgent, "AvoidError");
htmlCode = client.DownloadString("http://www.website.com");
}
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(htmlCode);
DataTable dt = new DataTable();
dt.Columns.Add("Name", typeof(string));
dt.Columns.Add("Value", typeof(decimal));
int count = 0;
decimal rowValue = 0;
bool isDecimal = false;
foreach (var row in doc.DocumentNode.SelectNodes("//table[#summary='Table Name']/tbody/tr"))
{
DataRow dr = dt.NewRow();
foreach (var cell in row.SelectNodes("td"))
{
if ((count % 2 == 0))
{
dr["Name"] = cell.InnerText.Replace(" ", " ");
}
else
{
isDecimal = decimal.TryParse((cell.InnerText.Replace(".", "")).Replace(",", "."), out rowValue);
if (isDecimal)
{
dr["Value"] = rowValue;
}
dt.Rows.Add(dr);
}
count++;
}
}

Simple logic to convert a htmltable to datatable :
//Define your webtable
public static HtmlTable table
{
get
{
HtmlTable var = new HtmlTable(parent);
var.SearchProperties.Add("id", "searchId");
return var;
}
}
//Convert a webtable to datatable
public static DataTable getTable
{
get
{
DataTable dtTable= new DataTable("TableName");
UITestControlCollection rows = table.Rows;
UITestControlCollection headers = rows[0].GetChildren();
foreach (HtmlHeaderCell header in headers)
{
if (header.InnerText != null)
dtTable.Columns.Add(header.InnerText);
}
for (int i = 1; i < rows.Count; i++)
{
UITestControlCollection cells = rows[i].GetChildren();
string[] data = new string[cells.Count];
int counter = 0;
foreach (HtmlCell cell in cells)
{
if (cell.InnerText != null)
data[counter] = cell.InnerText;
counter++;
}
dtTable.Rows.Add(data);
}
return dtTable;
}
}

You can try
DataTable.Rows[i].Cells[j].InnerText;
Where DataTable is the id of your table, i is the row and j is the cells.

Create HTML table from a list

I am trying to get some values from a List and then create a html table with this data but I can't get it to work properly.
I have:
HtmlTable table = new HtmlTable();
HtmlTableRow row;
HtmlTableCell cell;
foreach(var item in Name)
{
row = new HtmlTableRow();
foreach(var familyName in item.familyName)
{
cell = new HtmlTableCell();
cell.InnerText = item.familyName.ToString();
row.Cells.Add(cell);
}
foreach (var givenName in item.givenName)
{
cell = new HtmlTableCell();
cell.InnerText = item.givenName.ToString();
row.Cells.Add(cell);
}
table.Rows.Add(row);
}
this.Controls.Add(table);
When I step through the debugger I can see that row.Cells.Add(cell) contains the family name in the first loop and given name in the second loop but then something seems to be wrong and I can't get the table to show up on the page with this data.
When I check the table.rows.add(row) it says that
base {System.SystemException} = {"'HtmlTableRow' does not support the InnerText property."}
What am I doing wrong here?

I've stepped through your code and I can't replicate the error you mention.
It's difficult to say for sure without seeing your data structure Name but a couple of observations:
I. If familyName is a string, your inner foreach will execute once for each character in the string. This may not be what you want as it'll output a surname x number of times where x = surname.length.
This will result in unequal numbers of table cells per row unless all your surnames are the same length.
So I would say get rid of the
foreach(var familyName in item.familyName){...}
loop and just leave the code inside so it'll output surname just once.
II. I'm guessing that item.givenName is an array or collection e.g. List<> of strings? If so you could just use
cell.InnerText = givenName;
Note that this is will still give you uneven numbers of table cells per row because people have different numbers of forenames ;-)
Having said that you really ought to use the built in controls for doing this kind of thing - the Repeater is probably the way to go.
E.g.
Markup
<asp:Repeater runat="server" id="rptNames" onItemDataBound="rptName_ItemDataBound" >
<HeaderTemplate>
<table>
<tr>
<td>Given Name(s)</td>
<td>Family Name</td>
</tr>
</HeaderTemplate>
<ItemTemplate>
<tr>
<td><%# Eval("FamilyName") %></td>
<td>
<asp:Label runat="server" id="lGivenNames" />
</td>
</tr>
<ItemTemplate>
<FooterTemplate>
</table>
</FooterTemplate>
</asp:Repeater>
CodeBehind
Probably triggered by Page_Load - just bind your repeater to your Name collection:
rptNames.DataSource = Name;
rptNames.DataBind();
To output the GivenNames you use the ItemDataBound event which gets called for each row of the repeater:
protected void rptNames_ItemDataBound(object sender, RepeaterItemEventArgs e){
//Not interested the Header and Footer rows
if (e.Item.ItemType == ListItemType.Item || e.Item.ItemType == ListItemType.AlternatingItem){
Label l = ((Label)e.Item.FindControl("lGivenNames"));
string[] arrGivenNames = ((FullName)e.Item.DataItem).GivenNames;
foreach (string n in arrGivenNames){//could use a StringBuilder for a performance boost.
l.Text += n + " "; //Use a regular space if using it for Winforms
}
//For even slicker code, replace the Label in your repeater with another repeater and bind to that. Google `nested repeater` for a how to.
}
}
HTH.
Full Code
<h2>Doing it by hand - manually building up an HTML Table</h2>
<asp:Panel runat="server" ID="pnl1">
</asp:Panel>
<h2>With a Repeater</h2>
<asp:Repeater runat="server" id="rptNames" onItemDataBound="rptName_ItemDataBound" >
<HeaderTemplate>
<table border="1" style="border-color:Red;">
<tr>
<td>Given Name(s)</td>
<td>Family Name</td>
</tr>
</HeaderTemplate>
<ItemTemplate>
<tr>
<td><%# Eval("FamilyName") %></td>
<td>
<asp:Label runat="server" id="lGivenNames" />
</td>
</tr>
</ItemTemplate>
<FooterTemplate>
</table>
</FooterTemplate>
</asp:Repeater>
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.HtmlControls;
namespace Testbed.WebControls
{
internal class FullName{
public string FamilyName{get;set;}
public string[] GivenNames{get;set;}
public FullName(){
}
public FullName(string[] _givenNames, string _familyName)
{
FamilyName = _familyName;
GivenNames = _givenNames;
}
}
public partial class HTMLTables : System.Web.UI.Page
{
List<FullName> Name;
protected void Page_Load(object sender, EventArgs e)
{
this.Name = new List<FullName>();
Name.Add(new FullName(new string[]{"Kylie"},"Minogue"));
Name.Add(new FullName(new string[]{"Angelina", "Kate", "Very-Lovely"}, "Jolie"));
Name.Add(new FullName(new string[]{"Audrey", "Veronica"},"Hepburn"));
HtmlTable table = new HtmlTable();
table.Border = 1;
HtmlTableRow row;
HtmlTableCell cell;
row = new HtmlTableRow();
cell = new HtmlTableCell();
cell.InnerText = "Given Name";
row.Cells.Add(cell);
cell = new HtmlTableCell();
cell.InnerText = "Family Name";
row.Cells.Add(cell);
foreach (var item in Name)
{
row = new HtmlTableRow();
//foreach (var familyName in item.FamilyName){
cell = new HtmlTableCell();
cell.InnerText = item.FamilyName.ToString();
row.Cells.Add(cell);
//}
foreach (string givenName in item.GivenNames)
{
cell = new HtmlTableCell();
cell.InnerText = givenName.ToString();
row.Cells.Add(cell);
}
table.Rows.Add(row);
}
this.pnl1.Controls.Add(table);
//Or do it with a repeater
rptNames.DataSource = Name;
rptNames.DataBind();
}
//This gets called everytime a data object gets bound to a repeater row
protected void rptName_ItemDataBound(object sender, RepeaterItemEventArgs e){
switch(e.Item.ItemType){
case ListItemType.Item:
case ListItemType.AlternatingItem:
string[] arrGivenNames = ((FullName)e.Item.DataItem).GivenNames;
foreach(string n in arrGivenNames){
((Label)e.Item.FindControl("lGivenNames")).Text += n + #" ";
}
break;
default:
break;
}
}
}
}