So i want to retrieve the amount of results from a google search. I dont use the Google search API because it doesnt give the right numbers, i want the exact same that are one the webpage when you google someting (thats actually the whole point of the game). I tried Python (with no experience) but failed misaberly. Then I search a bit around the Internet and this what i came up with:
(found code but it was in javascript, and im also to dumb for that)
Edit: Heres the link to the JavaScript r google search result count retrieve
And apparently some guy wanted the same but in java, so if someone could "translate" this(easiest (legal) way to programmatically get the google search result count?) I would be really happy
using System.Collections;
using System.IO;
using UnityEngine;
using UnityEngine.Networking;
using UnityEngine.Windows;
using File = System.IO.File;
public class SearchGoogle : MonoBehaviour
{
public string keyword;
private string searchwebsite;
string fileName = "MyFile.html";
void Start()
{
searchwebsite = "https://www.google.com/search?q=" + keyword;
Debug.Log(searchwebsite);
StartCoroutine(GetText());
Debug.Log("Coroutine wird gecallt");
}
IEnumerator GetText()
{
UnityWebRequest www = UnityWebRequest.Get(searchwebsite);
yield return www.SendWebRequest();
if (www.isNetworkError || www.isHttpError)
{
Debug.Log("Fehler:(");
Debug.Log(www.error);
}
else
{
Debug.Log("CErfolg");
// Show results as text
Debug.Log(www.downloadHandler.text);
if (File.Exists(fileName))
{
Debug.Log(fileName + " already exists.");
}
var sr = File.CreateText(fileName);
sr.WriteLine(www.downloadHandler.text);
sr.Close();
//Printing it in a file to open it in my browser, not nessecary in the final build
// Or retrieve results as binary data
byte[] results = www.downloadHandler.data;
}
}
}
But this code just returns the scource code of the website, and I actually dont quite understand it. Does anyone have a solution for this (i just want the amount of results, nothing else)? I just think it would be possible to do, but if it's not, correct me please. Thanks in advace.
Edit:(The div in html is called "result-stats"
Related
I'm using C# and Xamarin forms to create a phone app that (when a button is pressed) will pull specific html data from a website in and save it into a text file (that the program can read from again later). I started with the tutorial in this video: https://www.youtube.com/watch?v=zvp7wvbyceo if you want to see what I started out with, and here's the code I have so far made using this video https://www.youtube.com/watch?v=wwPx8QJn9Kk, in the the "AboutViewModel.cs" file created in the video:
Image link because this is a new account i guess and i cant embed images or something
Paste of the code itself (but the image gives you a better look at everything):
private Task WebScraper()
{
HtmlAgilityPack.HtmlWeb web = new HtmlAgilityPack.HtmlWeb();
HtmlAgilityPack.HtmlDocument doc = web.Load("https://www.flightview.com/airport/DAB-Daytona_Beach-FL/");
foreach (var item in doc.DocumentNode.SelectNodes("//td[#class='c1']"))
{
var itemstring = item;
File.WriteAllText("AirportData.txt", itemstring);
}
return Task.CompletedTask;
}
public ICommand OpenWebCommand { get; }
public ICommand WebScraperCommand { get; }
}
}
The only error i'm getting right now is "Cannot convert 'HtmlAgilityPack.HtmlNode' to 'string'" Which i'm working on fixing but I don't think this is the best solution so anything you have is useful. Thanks :)
HtmlNode is an object, not a simple string. You probably want to use the OuterHtml property, but consult the docs to see if that is the right fit for your use case
string output = string.Empty;
foreach (var item in doc.DocumentNode.SelectNodes("//td[#class='c1']"))
{
output += item.OuterHtml;
}
File.WriteAllText("AirportData.txt", output);
note that you need to specify a path to a writable folder, the root folder of the app is not writable. See https://learn.microsoft.com/en-us/xamarin/xamarin-forms/data-cloud/data/files?tabs=windows
Hi I am having an issue with my word application. I am trying to add a field side by side in the footnotes. The issue I am having is a merge conflict and I think this is because the range is the same and over writhing the other field. I am trying add them side by side in one line based on the end point of the previous field.
I have tried collapsing the range to the end but I can't get this to work. Any help would be much appreciated as I am newish to using the VSTO tools and tbh I find them not very good.
public static void insertHtmlIntoFootnoteResult (Field field, List<ct>
list)
{
for(var c in ct){
//I am trying to go to the end here
field.Result.Collapse(WdCollapseDirection.wdCollapseEnd);
//How do I create a new field and insert it here based of the the
//last fields ending position?
string guid = Guid.NewGuid().ToString();
var filename = Path.GetTempPath() + "temp" + guid + ".html";
using (StreamWriter s = File.CreateText(filename))
{
s.Write("I am test");
s.Close();
}
field.Result.InsertFile(filename);
File.Delete(filename);
}
} // _insertHtmlIntoRange
Hey so I found an answer eventually using the blog post by flowers
https://gist.github.com/FlorianWolters/6257233
Done by changing the insertempty() method to insert a Word.WdFieldType.wdFieldIf field.
Hope this helps for futre people!
So I am starting to learn how to use XML data within a app and decided to use some free data to do this however I cannot for the life of me get it working this is my code so far. (I have done a few apps with static data before but hey apps are designed to use the web right? :p)
public partial class MainPage : PhoneApplicationPage
{
List<XmlItem> xmlItems = new List<XmlItem>();
// Constructor
public MainPage()
{
InitializeComponent();
LoadXmlItems("http://hatrafficinfo.dft.gov.uk/feeds/datex/England/CurrentRoadworks/content.xml");
test();
}
public void test()
{
foreach (XmlItem item in xmlItems)
{
testing.Text = item.Title;
}
}
public void LoadXmlItems(string xmlUrl)
{
WebClient client = new WebClient();
client.OpenReadCompleted += (sender, e) =>
{
if (e.Error != null)
return;
Stream str = e.Result;
XDocument xdoc = XDocument.Load(str);
***xmlItems = (from item in xdoc.Descendants("situation id")
select new XmlItem()
{
Title = item.Element("impactOnTraffic").Value,
Description = item.Element("trafficRestrictionType").Value
}).ToList();***
// close
str.Close();
// add results to the list
xmlItems.Clear();
foreach (XmlItem item in xmlItems)
{
xmlItems.Add(item);
}
};
client.OpenReadAsync(new Uri(xmlUrl, UriKind.Absolute));
}
}
I am basically trying to learn how to do this at the moment as I am intrigued how to actually do it (I know there are many ways but ATM this way seems the easiest) I just don't get what the error is ATM. (The bit in * is where it says the error is)
I also know the display function ATM is not great (As it will only show the last item) but for testing this will do for now.
To some this may seem easy, as a learner its not so easy for me just yet.
The error in picture form:
(It seems I cant post images :/)
Thanks in advance for the help
Edit:
Answer below fixed the error :D
However still nothing is coming up. I "think" it's because of the XML layout and the amount of descendants it has (Cant work out what I need to do being a noob at XML and pulling it from the web as a data source)
Maybe I am starting too complicated :/
Still any help/tips on how to pull some elements from the feed (As there all in Descendants) correctly and store them would be great :D
Edit2:
I have it working (In a crude way) but still :D
Thanks Adam Maras!
The last issue was the double listing. (Adding it to a list, to then add it to another list was causing a null exception) Just using the 1 list within the method solved this issue, (Probably not the best way of doing it but it works for now) and allowed for me to add the results to a listbox until I spend some time working out how to use ListBox.ItemTemplate & DataTemplate to make it look more appealing. (Seems easy enough I say now...)
Thanks Again!!!
from item in xdoc.Descendants("situation id")
// ^
XML tag names can't contain spaces. Looking at the XML, you probably just want "situation" to match the <situation> elements.
After looking at your edit and further reviewing the XML, I figured out what the problem is. If you look at the root element of the document:
<d2LogicalModel xmlns="http://datex2.eu/schema/1_0/1_0" modelBaseVersion="1.0">
You'll see that it has a default namespace applied. The easiest solution to your problem will be to first get the namespsace from the root element:
var ns = xdoc.Root.Name.Namespace;
And then apply it wherever you're using a string to identify an element or attribute name:
from item in xdoc.Descendants(ns + "situation")
// ...
item.Element(ns + "impactOnTraffic").Value
item.Element(ns + "trafficRestrictionType").Value
One more thing: <impactOnTraffic> and <trafficRestrictionType> aren't direct children of the <situation> element, so you'll need to change that code as well:
Title = items.Descendants(ns + "impactOnTraffic").Single().Value,
Description = item.Descendants(ns + "trafficRestrictionType").Single().Value
I would like to integrate flowplayer in ASP mvc 3 application.
I'm using SQL Server 2008. Database contains path to files as well which are stored in folder. I wrote library which is able to convert video to flv. Now i would like to use this video in page.
Does anyone have knowledge how to implement that?
I found the flowplayer sample code.
http://flowplayer.org/docs/
But I'm looking for codebehind solution (C#) how to implement something like that (how to return path to video).
If anyone have some code I'll be grateful for help.
Please take a look.
public string FindMusicByID(int musicID)
{
var pathh = from plik in _data.musicMusicTables
where plik.musicMusicID == musicID
select new PathToFile { PathFile = plik.musicMusicPath };
return pathh.ToString();
}
This is the linq part code which i'm using. FindMusicByID(int musicID) return path to .flv file. I'm looking for how to send output filepath to webpage. I would like to use this path in flowplayer.
<div class="flowplayer" data-engine="flash"> <video src="some path"></video>
Fox example this is my ouptut path d:\file.flv
I suppose that it's necessary to send output to JavaScript.
How can i insert path into flowplayer by ID?
I'm not sure what you are asking for, but if you want to know how to return .flv file for your player you just need Action that return FileStremResult using this mimetype: video/x-flv
Something along the lines of:
public string FindMusicByID(int musicID)
{
var pathh= from plik in _data.musicMusicTables
where plik.musicMusicID == musicID
select new PathToFile { PathFile = plik.musicMusicPath };
var cd = new System.Net.Mime.ContentDisposition {
FileName = "filename",
Inline = false
};
Response.AppendHeader("Content-Disposition", cd.ToString());
return File(path, asset.AssetType.MimeType);
}
I need to create a html parser, that given a blog url, it returns a list, with all the posts in the page.
I.e. if a page has 10 posts, it
should return a list of 10 divs,
where each div contains h1 and
a p
I can't use its rss feed, because I need to know exactly how it looks like for the user, if it has any ad, image etc and in contrast some blogs have just a summary of its content and the feed has it all, and vice-versa.
Anyway, I've made one that download its feed, and search the html for similar content, it works very well for some blogs, but not for others.
I don't think I can make a parser that works for 100% of the blogs it parses, but I want to make the best possible.
What should be the best approach? Look for tags that have its id attribute equal "post", "content"? Look for p tags? etc etc etc...
Thanks in advance for any help!
I don't think you will be successful on that. You might be able to parse one blog, but if the blog engine changes stuff, it won't work any more. I also don't think you'll be able to write a generic parser. You might even be partially successful, but it's going to be an ethereal success, because everything is so error prone on this context. If you need content, you should go with RSS. If you need to store (simply store) how it looks, you can also do that. But parsing by the way it looks? I don't see concrete success on that.
"Best possible" turns out to be "best reasonable," and you get to define what is reasonable. You can get a very large number of blogs by looking at how common blogging tools (WordPress, LiveJournal, etc.) generate their pages, and code specially for each one.
The general case turns out to be a very hard problem because every blogging tool has its own format. You might be able to infer things using "standard" identifiers like "post", "content", etc., but it's doubtful.
You'll also have difficulty with ads. A lot of ads are generated with JavaScript. So downloading the page will give you just the JavaScript code rather than the HTML that gets generated. If you really want to identify the ads, you'll have to identify the JavaScript code that generates them. Or, your program will have to execute the JavaScript to create the final DOM. And then you're faced with a problem similar to that above: figuring out if some particular bit of HTML is an ad.
There are heuristic methods that are somewhat successful. Check out Identifying a Page's Primary Content for answers to a similar question.
Use the HTML Agility pack. It is an HTML parser made for this.
I just did something like this for our company's blog which uses wordpress. This is good for us because our wordress blog hasn't changed in years, but the others are right in that if your html changes a lot, parsing becomes a cumbersome solution.
Here is what I recommend:
Using Nuget install RestSharp and HtmlAgilityPack. Then download fizzler and include those references in your project (http://code.google.com/p/fizzler/downloads/list).
Here is some sample code I used to implement the blog's search on my site.
using System;
using System.Collections.Generic;
using Fizzler.Systems.HtmlAgilityPack;
using RestSharp;
using RestSharp.Contrib;
namespace BlogSearch
{
public class BlogSearcher
{
const string Site = "http://yourblog.com";
public static List<SearchResult> Get(string searchTerms, int count=10)
{
var searchResults = new List<SearchResult>();
var client = new RestSharp.RestClient(Site);
//note 10 is the page size for the search results
var pages = (int)Math.Ceiling((double)count/10);
for (int page = 1; page <= pages; page++)
{
var request = new RestSharp.RestRequest
{
Method = Method.GET,
//the part after .com/
Resource = "page/" + page
};
//Your search params here
request.AddParameter("s", HttpUtility.UrlEncode(searchTerms));
var res = client.Execute(request);
searchResults.AddRange(ParseHtml(res.Content));
}
return searchResults;
}
public static List<SearchResult> ParseHtml(string html)
{
var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(html);
var results = doc.DocumentNode.QuerySelectorAll("#content-main > div");
var searchResults = new List<SearchResult>();
foreach(var node in results)
{
bool add = false;
var sr = new SearchResult();
var a = node.QuerySelector(".posttitle > h2 > a");
if (a != null)
{
add = true;
sr.Title = a.InnerText;
sr.Link = a.Attributes["href"].Value;
}
var p = node.QuerySelector(".entry > p");
if (p != null)
{
add = true;
sr.Exceprt = p.InnerText;
}
if(add)
searchResults.Add(sr);
}
return searchResults;
}
}
public class SearchResult
{
public string Title { get; set; }
public string Link { get; set; }
public string Exceprt { get; set; }
}
}
Good luck,
Eric