I would like to create a program that gets data from the play.eslgaming.com website and puts this into a database.
I already found out that the data I need is just in the raw code of the webpages. Like following:
<div id="round-1" class="ui-tabs-panel ui-widget-content ui-corner-bottom">
<span class="teamstatus">Team: Evil Squirrels <small><i>(Attacker)</i></small></span>
<table class="wot_stats">
<tbody class="matchloser">
<tr>
<th>Tank</th>
<th>Player</th>
<th>Kills</th>
<th>Dmg dealt</th>
<th>Remain. HP</th>
<th>Blocked dmg</th>
<th>Time alive</th>
<th>Shots fired</th>
<th>Accuracy</th>
</tr>
<tr>
<td>
<div class="tank">
<img src="http://static-ptl-eu.gcdn.co/static/2.34.2/encyclopedia/tankopedia/vehicle/contour/ussr-object_140.png" class="icon-img">
<em>U.S.S.R. Object 140</em>
</div>
</td>
<td>
<div class="playername">
<a target="_blank" style="cursor: pointer;" href="/worldoftanks/europe/wot/open/bronze-series/player/7176461">conchita_</a>
<em>Conchita_</em>
</div>
</td>
<td>2</td>
<td>2781</td>
<td>Destroyed</td>
<td>240</td>
<td>07m 34s</td>
<td>18</td>
<td>50%</td>
And so on for each player each round 2 teams.
But the problem now is that I don't know how I get this information the easiest. As well I would like to put them into a database so I can make totals and monitor everything. I can program a bit C# so maybe there is a way to use this code.
I would like to know how I get the needed data from the webpages.
I would probably use the AngleSharp project to do the scraping: https://github.com/AngleSharp/AngleSharp
Related
Still a little bit new to MVC but trying to setup a modal that receives dynamic data depending on where the table data is in the foreach loop.
We have a partial that we are using to generate our table code and are passing the viewmodel we need to this partial. Now within the partial we want the links on a certain column to display a modal corresponding to that row when you click on them. So basically hand off that row's model data and display the more detailed data in the modal. For example, here we are displaying the count of the companies effected but in the modal I want to be able to display the names of them. So I need to pass that on somehow.
Table Code :
#model IEnumerable<Tier2BugViewModel>
<table id="bug" class="table table-striped table-bordered table-hover" width="100%">
<thead>
<tr>
<th></th>
<th data-class="expand">Bug</th>
<th data-hide="phone">Companies Effected</th>
</tr>
</thead>
<tbody>
#{
foreach (var row in Model)
{
<tr>
<td>
<a href="#" class="btn btn-default" data-toggle="modal" data-target="#bugEditModal">
Edit
</a>
</td>
<td>#row.Bug.Bug</td>
<td>
<a href="#" data-toggle="modal" data-target="#companiesEffectedModal">
#row.CompaniesAffected.Count
</a>
</td>
</tr>
}
}
</tbody>
</table>
Modal Code :
<!-- Companies Effected Modal -->
<div class="modal fade" id="companiesEffectedModal" tabindex="-1" role="dialog" aria-labelledby="companiesEffectedModalLabel" aria-hidden="true">
#Html.Partial("_CompaniesEffectedModal")
</div><!-- /.modal -->
After some research it feels like we are going to need some sort of jQuery script to do this, but I can't put my finger on it or figure out if there is just an easier way to do this. I guess we will eventually be doing this for the Edit modal as well. Thanks in advance.
I am working on a sidenav that is built on .NET MenuItems like so:
<asp:MenuItem value="19" Text="Profile" Selectable="false"></asp:MenuItem>
<asp:MenuItem value="0" Text="Overview" ToolTip="Overview" Selected="true"></asp:MenuItem>
<asp:MenuItem value="2" Text="My Info & Email Subscriptions" ToolTip="My Info & Email Subscriptions"></asp:MenuItem>
In HTML, the output produces a series of nested tables around each MenuItem which looks like this:
<div id="_links" class="span-3">
<table id="FormUserControl__tabMenu" cellpadding="0" cellspacing="0" border="0" style="clear:left;">
<tbody>
<tr id="FormUserControl__tabMenun0">
<td>
<table cellpadding="0" cellspacing="0" border="0" width="100%">
<tbody>
<tr>
<td style="white-space:nowrap;width:100%;">
<a style="text-decoration:none;">
<div id="FormUserControl__tabMenu_ctl00__tabMenuItemPanel" class="myAccountHeading ">
Profile
</div>
</a>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr onmouseover="Menu_HoverStatic(this)" onmouseout="Menu_Unhover(this)" onkeyup="Menu_Key(this)" title="Overview" id="FormUserControl__tabMenun1">
<td>
<table cellpadding="0" cellspacing="0" border="0" width="100%">
<tbody>
<tr>
<td style="white-space:nowrap;width:100%;"><a href="javascript:__doPostBack('FormUserControl$_tabMenu','0')" style="text-decoration:none;">
<div id="FormUserControl__tabMenu_ctl01__tabMenuItemPanel" class="sideNav">
Overview
</div>
</td>
<tr>
<tbody>
</td>
</tr>
</tbody>
</table>
</div>
How can I add to add accessibility role and aria-level to these innermost divs? The goal is to achieve accessibility compliance. For example:
<div role="heading" aria-level="[2]">Profile</div>
I have looked through MSDN documentation and it looks like there isn't a way to add those attributes within the intial MenuItem declaration.
I also tried adding role and aria-level attributes within CSS, which I know is hacky, but I figured since content can be set, it was worth trying. That doesn't work.
I could readily do this in JavaScript, but I really want to avoid involving that, it's a last resort and I do know how to do that.
Is there a way to change the MenuItem output to involve role and aria-level? Or, is there a way to have it output a header instead of a div nested within two tables?
Many thanks!
I'm working on a project that is in ASP.NET WebForms on .NET 2.0 and am running into a problem I can't seem to find a cause for. I have the following code in my aspx file:
<table>
<tr>
<td>
<asp:HyperLink ID="supportLink" runat="server">Customer Support Docket</asp:HyperLink>
</td>
</tr>
<tr>
<td>
<asp:HyperLink ID="entitlementsLink" runat="server">Edit Entitlements</asp:HyperLink>
</td>
</tr>
</table>
And when I run locally in debug mode, it works great. The links appear and the HTML is rendered properly as such:
Local Rendered HTML
<table>
<tr>
<td>
<a id="ctl00_ContentMain_supportLink" href="ViewCustomer.aspx?customerid=659">Customer Support Docket</a>
</td>
</tr>
<tr>
<td>
<a id="ctl00_ContentMain_entitlementsLink" href="EditEntitlements.aspx?customerid=659">Edit Entitlements</a>
</td>
</tr>
</table>
However, when I run it on my server in production, I get the following HTML generated:
Server Rendered HTML
<table>
<tr>
<td>
<a id="ctl00_ContentMain_supportLink">Customer Support Docket</a>
</td>
</tr>
<tr>
<td>
<a id="ctl00_ContentMain_entitlementsLink">Edit Entitlements</a>
</td>
</tr>
</table>
As you can see, the href is missing from both. In my code behind I am adding the URLs since I was having a hard time getting them to render properly in the aspx file:
Code Behind File
protected void Page_Load(object sender, EventArgs e)
{
supportLink.NavigateUrl = string.Format("ViewCustomer.aspx?customerid={0}",
Request.QueryString["customerid"]);
entitlementsLink.NavigateUrl = string.Format("EditEntitlements.aspx?customerid={0}",
Request.QueryString["customerid"]);
}
At this point I'm not sure what I can do. My plan was to re-write this site in the next few weeks in MVC4 but seeing as this is an internal site and not generating money it is low on my priorities list and won't get much more attention for awhile, which means I need to fix this issue.
The server environment is IIS 6 on Windows Server 2003 and the HTML renders this way in both IE9 as well as Chrome 24.0.1312.57. I have cleared the cache in both browsers to no avail and I know that the page is getting updated after a publish because other features that I added with the same change are actually working in production.
EDIT
Rezalas was wondering where I got the customerid from. It is from the query string. To show that a blank query string wouldn't cause this here is the local and production HTML with no query string:
Local Rendered With No Query String
<table>
<tr>
<td>
<a id="ctl00_ContentMain_supportLink" href="ViewCustomer.aspx?customerid=">Customer Support Docket</a>
</td>
</tr>
<tr>
<td>
<a id="ctl00_ContentMain_entitlementsLink" href="EditEntitlements.aspx?customerid=">Edit Entitlements</a>
</td>
</tr>
</table>
Production HTML With No Query String
<table>
<tr>
<td>
<a id="ctl00_ContentMain_supportLink">Customer Support Docket</a>
</td>
</tr>
<tr>
<td>
<a id="ctl00_ContentMain_entitlementsLink">Edit Entitlements</a>
</td>
</tr>
</table>
For clarification, where is the customerid value coming from? Are you certain that the value when ran on the server is being properly pulled? I say this because it looks as though the ID isn't being provided, and as such the string isn't being built and the navigate url is being excluded because it doesn't have a value.
I want to extract the movie name of each row in the IMDb`s Boxoffice table..
example html table row:
<tr class="chart_even_row">
<td style="text-align: right;">
<b>1</b>
</td>
<td>
<img border="0" src="http://ia.media-imdb.com/images/M/MV5BMjA4NDg3NzYxMF5BMl5BanBnXkFtZTcwNTgyNzkyNw##._V1._SY30_SX23_.jpg" width="20" height="30">
</td>
<td>
<a href="/title/tt1392170/" >The Hunger Games</a> (2012)
</td>
<td style="text-align: right; padding-right: 20px;">$155M
</td>
<td style="text-align: right;">
$155M
</td>
<td style="text-align: center;">
1
</td>
</tr>
The value I want to extract is "The Hunger Games"..
I need a C# code that would achieve this for me..
NOTE: I want to do this via REGEX
Thanks in advance,
Rashad.
Screen scraping the IMDB is complicated, fragile, and forbidden. The IMDB provides plain-text data files you can use instead at http://www.imdb.com/interfaces
Update
Allow me to reiterate: screen scraping and data mining IMDB.com is in violation of their terms of use.
Regarding Regex: see this answer.
So if you're dead-set on doing this in violation of the IMDB's terms of use, the HTML Agility Pack is probably the best way to go.
try to copy paste the code in single html file. if you have too many pages to fetch then try to write code that will read them through html agility pack.
You can find html agility pack here http://htmlagilitypack.codeplex.com/
Here i am a dragging a table(1) row and dropping it in another table(2) using asp.net mvc similar to this one http://www.redips.net/javascript/drag-and-drop-table-row/ and the values are from database .It is working fine and what i need is the html table values (i.e the values inside the td) of the table(2).how can i get those values and save it in dataset in asp.net MVC.Here is my code
<div id="drag">
<table class="tbl1">
<tr>
<th colspan="2">
Table 1
</th>
</tr>
<% foreach (var item1 in Model)
{ %>
<tr class="rl">
<td class="rowhandler">
<div class="drag row">
<%= Html.Encode(item1.Id) %>
</div>
</td>
<td class="rowhandler">
<div class="drag row">
<%= Html.Encode(item1.Title) %>
</div>
</td>
</tr>
<% } %>
<tr style="display: none;">
<td colspan="2">
<span>Message line</span>
</td>
</tr>
</table>
<table id="tab" runat="server">
<tr>
<th colspan="2" class="mark">
Table 2
</th>
</tr>
<tr class="rd">
<td>
</td>
<td>
</td>
</tr>
<tr style="display: none;">
<td colspan="2">
<span>Message line</span>
</td>
</tr>
</table>
</div>
Any suggesion?I am new to MVC
Use jQuery ajax to save the data in server. You can make a call to a controller action from your jquery with your data and it can save the data to your tables.
var mydata=""; /// read your table cells values and store it in this string
$.post("yourcontroller/action/", { data : mydata} ,function(response){
//do something with the response from action
});
As Brian already suggested, traversing the DOM maybe a bad idea. A better way would be to store the table2 data in a javascript array (whenever a drop event occurs add to the array, or remove from the array when cells are dragged off the table). Then have an ajax call to submit the whole array in one go.
If I am understanding this correctly when the page loads you have the data for both tables in the database. You can just keep a working set of the changes being made to the table and update those records accordingly. Add logic to the "drop" event in your JS to store the item's ID and it's new position.
This saves you from having to update unaffected records and also from traversing the entirety of both tables on submit.