We Keep Coding

Getting data from table using Selenium - c#

I have a table of fees I am trying to parse through to return data, but it is returning a few blanks before it actually returning the string of data.
<table id="Fees">
<thead>
<tr>
<th>Rate Code</th>
<th>Description</th>
<th>Amount</th>
</tr>
</thead>
<tbody>
<tr>
<td class="code">A1</td>
<td>Charge Type 1</td>
<td class="amount">$11.20</td>
</tr>
<tr>
<td class="code">C2</td>
<td>Charge Type 2</td>
<td class="amount">$36.00</td>
</tr>
<tr>
<td class="code">CMI</td>
<td>Cuba Medical Insurance</td>
<td class="amount">$25.00</td>
</tr>
</tbody>
<tfoot>
<tr>
<td colspan="2">Total:</td>
<td class="amount">$145.16</td>
</tr>
</tfoot>
</table>
I return by xpath
private By lst_Fee
{
get { return By.XPath("//*[#id=\"Fees\"]/tbody/tr"); }
}
Selenium code:
IList<IWebElement> fees = GetNativeElements(lst_Fee, 5);
List<string> actual = new List<string>();
foreach (IWebElement elem in fees)
{
actual.Add(GetText(elem, ControlType.Label));
}
Questions
Is ControlType.Label correct for a table? I am getting a few blank elems before actually getting to the data.
If I wanted to separate each Rate, Description and Fee out in each item to make sure the cost adds up to Total correctly, how can I do that?

I would do something like the below. I created a class Fee that holds the parts of a fee: the code, description, and amount. For each table row , you would extract these three values and store them in an instance of the Fee class. The function returns a collection of Fee instances. To get the sum of the fees themselves, you would call the GetFees() method and then iterate through the Fee instances summing the amount into the final Total.
public class Fee
{
private String code;
private String desc;
private BigDecimal amount;
private Fee(String _code, String _desc, BigDecimal _amount)
{
this.code = _code;
this.desc = _desc;
this.amount = _amount;
}
}
public List<Fee> GetFees()
{
List<Fee> fees = new ArrayList<Fee>();
List<WebElement> rows = driver.findElements(By.cssSelector("#Fees > tbody > tr"));
for (WebElement row : rows)
{
List<WebElement> cells = row.findElements(By.cssSelector("td"));
fees.add(new Fee(cells.get(0).getText(), cells.get(1).getText(), parse(cells.get(2).getText(), Locale.US)));
}
return fees;
}
// borrowed from http://stackoverflow.com/a/23991368/2386774
public BigDecimal parse(final String amount, final Locale locale) throws ParseException
{
final NumberFormat format = NumberFormat.getNumberInstance(locale);
if (format instanceof DecimalFormat)
{
((DecimalFormat) format).setParseBigDecimal(true);
}
return (BigDecimal) format.parse(amount.replaceAll("[^\\d.,]", ""));
}

You can grab all the column headers and as well the row data by the below code:
Happy coding =
//// Grab the table
IWebElement grid;
grid = _browserInstance.Driver.FindElement(By.Id("tblVoucherLookUp"));
IWebElement headercolumns = grid.FindElement(By.Id("tblVoucherLookUp"));
_browserInstance.Driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(75));
_browserInstance.ScreenCapture("Voucher LookUp Grid");
//// get the column headers
char[] character = "\r\n".ToCharArray();
string[] Split = headercolumns.Text.Split(character);
for (int i = 0; i < Split.Length; i++)
{
if (Split[i] != "")
{
_log.LogEntry("INFO", "Voucher data", true,
Split + " Text matches the expected:" + Split[i]);
}
}

Related

I am trying to verify if the table contains filetype (1st cell) is figure then 3ed cell should contain jpg or png, etc and 5th cell should contain (something)
the function should return true or false.
here is my code I don't know what I am doing wrong, because it always returns false even is the conditions are true. Thanks for the help !!
public bool Results()
{
var aqcTable = Browsers.getDriver.FindElement(By.TagName("table"));
var lstTrElem = aqcTable.FindElements(By.TagName("tr"));
var i = 0;
foreach (var cell in lstTrElem)
{
if (lstTrElem[i].Text== "figure")
if (lstTrElem[ i+ 2].Text.Contains("jpg"))
if (lstTrElem[i + 4].Text.Contains("pass"))
return true;
}
return false;
}

You are trying to get the Text of a <tr> element.
You need to get the text of <td>. So if what you are trying to achieve is:
I am trying to verify if the table contains filetype (1st cell) is
figure then 3ed cell should contain jpg or png, etc and 5th cell
should contain (something)
And you HTML content looks like this:
<table>
<tbody>
<tr>
<td>fig_ure</td>
<td>Some value</td>
<td>jpg</td>
<td>Some other value</td>
<td>Some other value</td>
</tr>
<tr>
<td>Figure</td>
<td>Some value</td>
<td>jpg</td>
<td>Some other value</td>
<td>Some other value which contains the pass word</td>
</tr>
</tbody>
</table>
Note: The text comparison is case sensitive. "figure" will not match "Figure"
Note: Name your variables as such, that the name should immediately convey what it holds
Note: First check existence and access later (tdElements.Count > 0 ?)
var firstTableElement = driver.FindElement(By.TagName("table"));
var trElementList = firstTableElement.FindElements(By.TagName("tr"));
foreach (var trElement in trElementList)
{
var tdElementList = trElement.FindElements(By.TagName("td"));
if (tdElementList.Count > 0)
{
bool trMatchesRequirements =
tdElementList[0].Text == "figure" &&
tdElementList[2].Text.Contains("jpg") &&
tdElementList[4].Text.Contains("pass");
if (trMatchesRequirements)
{
return true;
}
}
}
return false;

I have this list so far:
List<object> lista = new List<object>();
foreach (var item in group)
{
lista.Add(new
{
ver = item.FirstOrDefault().vereda.DESCRIPCION,
prod = item.Count()
});
}
ViewBag.veredasEncu = lista;
i have to send that data to a view in order to build a table:
<tbody>
#{
if (ViewBag.veredasEncu != null)
{
List<object> lis = ViewBag.veredasEncu;
for (int i = 0; i < lis.Count(); i++)
{
<tr>
<td></td>
<td></td>
</tr>
}
}
}
</tbody>
I cant get to place the info on the <td> tags cose every item in the foreach iteration throws something this:
item = {ver = "Loma", prod = 5}
how can i make that 2 values look like an array, or is there a way to separate them in order to place them in the correct tag?

I solved the issue by creating a class:
public class dataTab
{
public string ver { get; set; }
public int prod { get; set; }
}
then is just change the loop for this:
lista2.Add(new dataTab
{
ver = item.FirstOrDefault().vereda.DESCRIPCION,
prod = item.Count()
});
so it could be post in the HTML like this:
foreach (var item in ViewBag.veredasEncu)
{
<tr>
<td>#item.ver</td>
<td>#item.prod</td>
</tr>
}
However im not sure that creating a class for every type of list that i need to create is a good idea. IF someone have an idea on how to fill the list in a more generic way please respond this.

I have the following in a controller :
outputmodel.Add(new SP_RESULTS.RS_Plans()
{
id = Convert.ToDecimal(SPOutput["id"]),
name = Convert.ToString(SPOutput["name"]),
code = Convert.ToString(SPOutput["code"]),
from = Convert.ToDateTime(SPOutput["from"]),
to = Convert.ToDateTime(SPOutput["to"]),
days = Convert.ToDecimal(SPOutput["days"]),
type_id = convert.YoString(SPOutput["type_id"]),
package = Convert.ToString(SPOutput["package"]),
day = Convert.ToDecimal(SPOutput["day"]),
charge = SPOutput["charge"] as decimal?,
type = Convert.ToString(SPOutput["type"]),
percentage= SPOutput["percentage"] as decimal?,
taxes = Convert.ToDecimal(SPOutput["taxes"]),
order = Convert.ToDecimal(SPOutput["order"]),
level = SPOutput["level"] as decimal?,
Column15 = Convert.ToDecimal(SPOutput[15]),
type_order = (SPOutput["type_order"]) as decimal?,
adults = SPOutput["adults"] as decimal?,
});
var order = outputmodel.OrderBy(c => c.from);
ViewData["RS_Output"] = order;
grabbing output from an MS SQL stored procedure and storing in a viewdata (ordered by the FROM date).
My HTML has the following line to start to build the table
#foreach (var item in ViewData["RS_Output"] as Enumerable<app.Models.SP_RESULTS.RS_Plans>)
{
//basic <tr> <td> </td> </tr> table setup, using #item.variablename to pull info from the viewdata.
}
The output I am trying to achieve is for every TYPE under CODE, where the from date => current date, list the room type /package name etc.
and the output I am getting is
what I am trying to get is
What I think I need is a foreach after the current foreach, but I cannot for the life of me figure it out in my head.
I've changed the
var order line in my controller to now read
var order = outputmodel.OrderBy(c => c.rate);
..and I've put the HTML table create code in an if loop
#foreach (var item in ViewData["RS_Output"] as Enumerable<app.Models.SP_RESULTS.RS_Plans>)
{
if (item.to >= DateTime.now)
{
//basic <tr> <td> </td> </tr> table setup, using #item.variablename to pull info from the viewdata.
}
}
.. but, as I say, I am stumped.
I think I need another foreach within the newly created if loop, but I cannot figure out how.
#foreach (var item in ViewData["RS_Output"] as Enumerable<app.Models.SP_RESULTS.RS_Plans>)
{
if (item.to >= DateTime.now)
{
//other table headers/data
<tr>
<td>
#item.type
</td>
</tr>
<tr>
<td>
Room Type
</td>
<td>
Package / Service
</td>
<td>
Availablility
</td>
<td>
Charge
</td>
<td>
PAX
</td>
<td>
Level
</td>
</tr>
<tr>
==> #foreach (subitem = item.type)
==> {
==> foreach (item.type)
==> {
<td>
#item.type_id
</td>
<td>
#item.package
</td>
<td>
#item.Column15
</td>
<td>
#item.charge
</td>
<td>
#item.adults
</td>
<td>
#item.level
</td>
==> }
==> }
</tr>
}
}
can someone please advise?
thanks
UPDATE:
Hi, what I found worked was, if I create a variable called string previous_type =" " , and another called  decimal previous_id =0 ,  then, in the view, I can amend with
if (item.to >= item.checkdate)
{
if ((previous_id != item.id) && (previous_type != item.type.ToString()) )
{
//some more code
if (item.type.ToString().Equals(previous_type) == false)
{
previous_type = item.type.ToString();
previous_date_from = item.date_from;
}
//etc
}
Thanks everyone for their help

OK, I think what you want is to first group the data, then show a table which then shows 'sub-tables' for each type of accomodation?
if so, then yes you can do this with nested foreach loops, but you'd still be better off strongly typing your view and doing the grouping stuff in the controller (or possibly better in some sort of service layer so it can be more easily tested/re-used)... but to get you started, something like this:
Models:
//Raw data
public class DataRowModel
{
public int Id { get; set; }
public string Class{ get;set;}
public string Description { get; set; }
public DateTime BookingDate { get; set; }
}
//Grouped data
public class GroupedDataRowModel
{
public string Class { get; set; }
public IEnumerable<DataRowModel> Rows { get; set; }
}
//View model
public class DataRowsViewModel
{
public IEnumerable<GroupedDataRowModel> Results { get; set; }
}
Controller Action:
public ActionResult TestData()
{
var PretendDatabaseCall = new List<DataRowModel>
{
new DataRowModel{
Id =1,
BookingDate =new DateTime(2017,1,1),
Description ="Booking 1",
Class="Room"
},
new DataRowModel{
Id =2,
BookingDate =new DateTime(2017,2,1),
Description ="Booking 2",
Class="Room"
},
new DataRowModel{
Id =3,
BookingDate =new DateTime(2017,3,1),
Description ="Booking 3",
Class="Suite"
},
new DataRowModel{
Id =4,
BookingDate =new DateTime(2017,4,1),
Description ="Booking 4",
Class="Room"
},
};
//We can now get the data from the database. We want to group by class so we can
//get a summary of items by class rather than a big flat list. Most LINQ to SQL implementations
//(e.g. Entity Framework) when working with Raw entities could convert this to SQL so the SQL server
//does the grouping, but if not it can happen in memory (get all records, then standard LINQ does it on
//the complete list)
var dataGroupedByClass = PretendDatabaseCall
//Minor Edit: apply filtering here not in the view!
.Where(x=>x.BookingDate >= Datetime.Now)
//Group by class.
.GroupBy(x => x.Class)
//for each class, get the records.
.Select(grpItem => new GroupedDataRowModel()
{
//'key' is the thing grouped by (class)
Class = grpItem.Key,
//grpItem has all the rows within it accessible still.
Rows = grpItem.Select(thisRow => thisRow)
});
var model = new DataRowsViewModel
{
Results = dataGroupedByClass
};
return View("~/Views/Home/TestData.cshtml", model);
}
And View:
#* Strongly typed view. saves any casting back and forth.*#
#model SimpleWeb.Models.DataRowsViewModel
#{
ViewBag.Title = "TestData";
}
<h2>TestData</h2>
<table>
<thead></thead>
<tbody>
#foreach (var groupEntry in Model.Results)
{
#*Add single row with just the class...*#
<tr><td>#groupEntry.Class</td></tr>
#*Header row for each class of booking*#
<tr>
<td>Id</td>
<td>Description</td>
<td>Date</td>
</tr>
foreach (var row in groupEntry.Rows)
{
#*add new row for each actual row*#
<tr>
<td>
#row.Id
</td>
<td>
#row.Description
</td>
<td>
#row.BookingDate
</td>
</tr>
}
}
</tbody>
</table>
This produces Data like I think you want:
Room
Id Description Date
1 Booking 1 01/01/2017 00:00:00
2 Booking 2 01/02/2017 00:00:00
4 Booking 4 01/04/2017 00:00:00
Suite
Id Description Date
3 Booking 3 01/03/2017 00:00:00
Obviously you want the 'Room' and 'Suite' parts to contain more information, but this should hopefully help get you started?

I have been trying to scrape some data off a website. The source has differentiated all the headers of tables to that of the actual contents by different class names. Because I want to scrape all the table information, I got all the headers into one array and contents into another array. But the problem is that when I am trying to write the array contents into a file, I can write a header but second array contains contents from all the table and I cannot mark where contents of first table ends.
Because htmlagilitypack scrapes all the tags of specified Nodes, I get all the contents. First let me show the code to make it clear:
<tr class=tableHeader>
<th width=16%>Caught</th>
<th width=16%><p>Normal Range</p></th>
</tr>
<TR class=content><TD><i>Bluegill</i></TD>
<TD>trap net</TD>
<TD align=CENTER>4.05</TD>
<TD align=CENTER> 7.9 - 37.7</TD>
<TD align=CENTER>0.26</TD>
<TD align=CENTER> 0.1 - 0.2</TD>
</TR>
<TR class=content><TD><i></i></TD>
<TD>Gill net</TD>
<TD align=CENTER>1.50</TD>
<TD align=CENTER>N/A</TD>
<TD align=CENTER>0.07</TD>
<TD align=CENTER>N/A</TD>
</TR>
<tr class=tableHeader>
<th>0-5</th>
<th>6-8</th>
<th>9-11</th>
<th>12-14</th>
<th>15-19</th>
<th>20-24</th>
<th>25-29</th>
<th>30+</th>
<th>Total</th>
</tr>
<TR class=content><TD><i>bluegill</i></TD>
<TD align=CENTER>19</TD>
<TD align=CENTER>65</TD>
<TD align=CENTER>0</TD>
<TD align=CENTER>0</TD>
<TD align=CENTER>0</TD>
<TD align=CENTER>0</TD>
<TD align=CENTER>0</TD>
<TD align=CENTER>0</TD>
<TD align=CENTER>84</TD>
</TR>
Below is my code to save the headers and contents into array and try to display it exactly like in the website.
int count =0;
foreach (var trTag4Pale in trTags4Pale)
{
string trText4Pale = trTag4Pale.InnerText;
paleLake[count] = trText4Pale;
if (trTags4Small != null)
{
int counter = 0;
foreach (var trTag4Small in trTags4Small)
{
string trText4Small = trTag4Small.InnerText;
smallText[counter] = trText4Small;
counter++;
}
}
File.AppendAllText(path,paleLake[count]+Environment.Newline+smallText[count]+Environment.Newline);
}
As you see, When I try to append the contents of the array to a file, it lines in the first header, and contents of all the table. But I only want contents of the first table and would repeat the process to get the content of the second table and so forth.
If I could get the contents between tr tag tableHeader, the arrays for the content would contain every contents for all the tables in different arrays. I don't know how to do this.

This might not be the best approach but I made it work somehow. It might be useful resource for somebody someday. So below is the code that worked for me. I append the data stored in the list into an excel sheet. As I have all the data I need for each tr tag with each class, I can manipulate the data I want:
var trTags4Header = document.DocumentNode.SelectNodes("//tr[#class='tableheader']");
if (trTags4Header != null)
{
//Create a list to store td values
List<string> tableList1 = new List<string>();
int row = 2;
foreach (var item in trTags4Header)
{
//Get only next siblings which matches the calss name as "content"
var found = item.SelectNodes("followin-sibling::*").TakeWhile(tag => tag.Name == "tr" && tag.Attributes["class"].Value == "content");
//store the nodes selected in an array (this is the selection of nodes I wanted which has td information I want.
HtmlNode[] nextItem = found.ToArray();
foreach (var node in nextItem)
{
//Gets individual td values within tr class='content' Notice .//td- this starts looking from the present node instead of the root nodes.
var tdValues = node.SelectNodes(".//td").TakeWhile(tdTag => tdTag.Name == "td");
int column = 1;
//Stores each td values into the list which is why I have control over the data to where I want to store, I am storing them in one excel worksheet.
foreach (var tdText in tdValues)
{
tableList1.Add(tdText.InnerText);
ws1.Cells[row, column] = tdText.InnerText;
column++;
}
row++;
}
}
//Display the content in a listbox
listBox1.DataSource = tableList1;
}
Please suggest a better solution if you come across this or leave your feedback. Thanks

From this XHTML source:
<div class = "page">
<h1>UNIQUE NAME</h1>
<table>
<tbody>
<tr>
<td>DATA TO EXTRACT 1</td>
</tr>
<tr>
<td />
<td />
<td />
<td />
<td />
<td>DATA TO EXTRACT 2</td>
</tr>
</tbody>
</table>
etc...
There are multiple instances of UNIQUE NAME with a similar set of child elements.
I need to locate the UNIQUE NAME element and extract all values (DATA TO EXTRACT) within each of the child element tags. In addition, I need to keep a count of where each value is located. For example DATA TO EXTRACT 1 would be at tr 1, td 1. DATA TO EXTRACT 2 would be at tr 2, td 6.
I am new to linq to xml and I was wondering whether someone could point me in the right direction with regards to a strategy. I have managed to figure out how to get to the UNIQUE name element with the following code:
var choice1 = (from category in _data.Descendants("div")
where category.Element("h1").Value == "UNIQUE NAME"
select category).DescendantNodes();
This returns a set of the values, which I'm sure I could loop through but I'm sure there must be a more elegant way of achieving this goal.
Many thanks!

Here’s one way of doing it using LINQ:
var choice1 =
from category in _data.Descendants("div")
where category.Element("h1").Value == "UNIQUE NAME"
from row in category.Descendants("tr").Select((element, index) => new { element, index })
from col in row.element.Elements("td").Select((element, index) => new { element, index })
where !string.IsNullOrEmpty(col.element.Value)
select new
{
RowIndex = row.index + 1, // one-based index
ColIndex = col.index + 1,
Value = col.element.Value,
};
An example of how to use your results:
foreach (var v in choice1)
Console.WriteLine(string.Format(
"RowIndex = {0}, ColIndex = {1}, Value = \"{2}\".",
v.RowIndex, v.ColIndex, v.Value));
…which would output:
RowIndex = 1, ColIndex = 1, Value = "DATA TO EXTRACT 1".
RowIndex = 2, ColIndex = 6, Value = "DATA TO EXTRACT 2".