We Keep Coding

JINT - Unable to "console.log"

I am new to JINT, and trying to just do some basic tests to kind of learn the ropes. My first attempt was to just store some javascript in my database, load it, and execute it in a unit test. So that looks essentially like this....
[Fact]
public void can_use_jint_engine() {
using (var database = DocumentStore()) {
using (var session = database.OpenSession()) {
var source = session.Load<Statistic>("statistics/1");
// join the list of strings into a single script
var script = String.Join("\n", source.Scripting);
// this will create the script
// console.log("this is a test from jint.");
//
var engine = new Jint.Engine();
// attempt to execute the script
engine.Execute(script);
}
}
}
And it doesn't work, I get this error, which makes absolutely no sense to me, and I cannot find any documentation on.
Jint.Runtime.JavaScriptExceptionconsole is not defined at
Jint.Engine.Execute(Program program) at
Jint.Engine.Execute(String source) at
SampleProject.Installers.Instanced.__testing_installer.can_use_jint_engine()
in _testing_installer.cs: line 318
Can anyone assist in shedding some light on this? I'm pretty confused at this point.

With JavaScript there are three entities - we care about. The host (browser, your application etc), the engine (JINT in this case) and the script ("console.log(...)") in this case.
JavaScript defines a bunch of functions and object as part of the language, but console is not one of them. By convention, browsers define a console object that can be used in the manner you describe. However, since your app is not a browser (and JINT does not do this by itself), there's no console object defined in your namespace (globals).
What you need to do is add a console object that will be accessible in JINT. You can find how to do this in the docs, but here's a simple example of how to add a log function to the engine so it can be used from the JS code (example taken from github).
var engine = new Engine()
.SetValue("log", new Action<object>(Console.WriteLine))
;
engine.Execute(#"
function hello() {
log('Hello World');
};
hello();
");

How to click on a link using Webkit Browser?

I want to click on link after navigating to a website
webKitBrowser1.Navigate("http://www.somesite.com");
How to click on a link on this website assuming that the link's id is lnkId ?
Go to Google
In the default browser control that comes with Visual Studio, I can do that using the code below :
foreach (HtmlElement el in webBrowser1.Document.GetElementTagName("a")) {
if (el.GetAttribute("id") == "lnkId") {
el.InvokeMember("click");
}
}
What is the equivalent of the code above when I'm using WebkitDotNet control?

As the WebKit doesn't provide a Click() event (see here for details), you cannot do that in the above way. But a small trick may work as an equivalent of the original winforms way as below:
foreach (Node el in webKitBrowser1.Document.GetElementsByTagName("a"))
{
if (((Element) el).GetAttribute("id") == "lnkId")
{
string urlString = ((Element) el).Attributes["href"].NodeValue;
webKitBrowser1.Navigate(urlString);
}
}
Here what I am doing is casting the WebKit.DOM.Node object to its subclass WebKit.DOM.Element to get its Attributes. Then providing href to the NamedNodeMap, i.e. Attributes as the NodeName, you can easily extract the NodeValue, which is the target url in this case. You can then simply invoke the Navigate(urlString) method on the WebKitBrowser instance to replicate the click event.

I don't work with Windows and all my experience is on Webkit GTK. Following comments are based on that experience.
I am not sure which webkit .NET version you are using. Looks like there are multiple implementations. Assuming you are using the one mentioned by Wasif, you can evaluate javascript as mentioned in the example https://code.google.com/p/open-webkit-sharp/source/browse/JavaScriptExample/Form1.cs.
Actually if implementation is supporting javascript execution then you can do most, if not all the DOM operations. The API functions are usually same as javascript functions and most of the time call exact same functions internally despite of origination. Communication between your application and javascript can be little challenging, but if you can read alert messages, that also can be solved. It looks like this library does support alert handling mechanism. A tool I wrote at https://github.com/nhrdl/notesMD will show some examples of achieving this communication though it uses GTK version and is written in python.
Incidentally if you know the id of the element, then Document.GetElementById will save you the loop.

webKitBrowser1.StringByEvaluatingJavaScriptFromString("var inpt = document.createElement(\"input\"); inpt.setAttribute(\"type\", \"submit\"); inpt.setAttribute(\"id\", \"nut\"); inpt.setAttribute(\"type\", \"submit\"); inpt.setAttribute(\"name\", \"tmp\"); inpt.setAttribute(\"value\", \"tmp\"); var element = document.getElementById(\"lnk\"); element.appendChild(inpt);");
webKitBrowser1.StringByEvaluatingJavaScriptFromString("document.getElementById('nut').click();");

Webbrowser control is not showing Html but shows webpage

I am automating a task using webbrowser control , the site display pages using frames.
My issue is i get to a point , where i can see the webpage loaded properly on the webbrowser control ,but when it gets into the code and i see the html i see nothing.
I have seen other examples here too , but all of those do no return all the browser html.
What i get by using this:
HtmlWindow frame = webBrowser1.Document.Window.Frames[1];
string str = frame.Document.Body.OuterHtml;
Is just :
The main frame tag with attributes like SRC tag etc, is there any way how to handle this?Because as i can see the webpage completely loaded why do i not see the html?AS when i do that on the internet explorer i do see the pages source once loaded why not here?
ADDITIONAL INFO
There are two frames on the page :
i use this to as above:
HtmlWindow frame = webBrowser1.Document.Window.Frames[0];
string str = frame.Document.Body.OuterHtml;
And i get the correct HTMl for the first frame but for the second one i only see:
<FRAMESET frameSpacing=1 border=1 borderColor=#ffffff frameBorder=0 rows=29,*><FRAME title="Edit Search" marginHeight=0 src="http://web2.westlaw.com/result/dctopnavigation.aspx?rs=WLW12.01&ss=CXT&cnt=DOC&fcl=True&cfid=1&method=TNC&service=Search&fn=_top&sskey=CLID_SSSA49266105122&db=AK-CS&fmqv=s&srch=TRUE&origin=Search&vr=2.0&cxt=RL&rlt=CLID_QRYRLT803076105122&query=%22LAND+USE%22&mt=Westlaw&rlti=1&n=1&rp=%2fsearch%2fdefault.wl&rltdb=CLID_DB72585895122&eq=search&scxt=WL&sv=Split" frameBorder=0 name=TopNav marginWidth=0 scrolling=no><FRAME title="Main Document" marginHeight=0 src="http://web2.westlaw.com/result/dccontent.aspx?rs=WLW12.01&ss=CXT&cnt=DOC&fcl=True&cfid=1&method=TNC&service=Search&fn=_top&sskey=CLID_SSSA49266105122&db=AK-CS&fmqv=s&srch=TRUE&origin=Search&vr=2.0&cxt=RL&rlt=CLID_QRYRLT803076105122&query=%22LAND+USE%22&mt=Westlaw&rlti=1&n=1&rp=%2fsearch%2fdefault.wl&rltdb=CLID_DB72585895122&eq=search&scxt=WL&sv=Split" frameBorder=0 borderColor=#ffffff name=content marginWidth=0><NOFRAMES></NOFRAMES></FRAMESET>
UPDATE
The two url of the frames are as follows :
Frame1 whose html i see
http://web2.westlaw.com/nav/NavBar.aspx?RS=WLW12.01&VR=2.0&SV=Split&FN=_top&MT=Westlaw&MST=
Frame2 whose html i do not see:
http://web2.westlaw.com/result/result.aspx?RP=/Search/default.wl&action=Search&CFID=1&DB=AK%2DCS&EQ=search&fmqv=s&Method=TNC&origin=Search&Query=%22LAND+USE%22&RLT=CLID%5FQRYRLT302424536122&RLTDB=CLID%5FDB6558157526122&Service=Search&SRCH=TRUE&SSKey=CLID%5FSSSA648523536122&RS=WLW12.01&VR=2.0&SV=Split&FN=_top&MT=Westlaw&MST=
And the properties of the second frame whose html i do not get are in the picture below:
Thank you

I paid for the solution of the question above and it works 100 %.
What i did was use this function below and it returned me the count to the tag i was seeking which i could not find :S.. Use this to call the function listed below:
FillFrame(webBrowser1.Document.Window.Frames);
private void FillFrame(HtmlWindowCollection hwc)
{
if (hwc == null) return;
foreach (HtmlWindow hw in hwc)
{
HtmlElement getSpanid = hw.Document.GetElementById("mDisplayCiteList_ctl00_mResultCountLabel");
if (getSpanid != null)
{
doccount = getSpanid.InnerText.Replace("Documents", "").Replace("Document", "").Trim();
break;
}
if (hw.Frames.Count > 0) FillFrame(hw.Frames);
}
}
Hope it helps people .
Thank you

For taking html you have to do it that way:
WebClient client = new WebClient();
string html = client.DownloadString(#"http://stackoverflow.com");
That's an example of course, you can change the address.
By the way, you need using System.Net;

This works just fine...gets BODY element with all inner elements:
Somewhere in your Form code:
wb.Url = new Uri("http://stackoverflow.com");
wb.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(wbDocumentCompleted);
And here is wbDocumentCompleted:
void wb1DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
var yourBodyHtml = wb.Document.Body.OuterHtml;
}
wb is System.Windows.Forms.WebBrowser
UPDATE:
The same as for the document, I think that your second frame is not loaded at the time you check for it's content...You can try solutions from this link. You will have to wait for your frames to be loaded in order to see its content.

The most likely reason is that frame index 0 has the same domain name as the main/parent page, while the frame index 1 has a different domain name. Am I correct?
This creates a cross-frame security issue, and the WB control just leaves you high and dry and doesn't tell you what on earth went wrong, and just leaves your objects, properties and data empty (will say "No Variables" in the watch window when you try to expand the object).
The only thing you can access in this situation is pretty much the URL and iFrame properties, but nothing inside the iFrame.
Of course, there are ways to overcome teh cross-frame security issues - but they are not built into the WebBrowser control, and they are external solutions, depending on which WB control you are using (as in, .NET version or pre .NET version).
Let me know if I have correctly identified your problem, and if so, if you would like me to tell you about the solution tailored to your setup & instance of the WB control.
UPDATE: I have noticed that you're doing a .getElementByTagName("HTML")(0).outerHTML to get the HTML, all you need to do is call this on the document object, or the .body object and that should do it. MyDoc.Body.innerHTML should get the the content you want. Also, notice that there are additional iFrames inside these documents, in case that is of relevance. Can you give us the main document URL that has these two URL's in it so we / I can replicate what you're doing here? Also, not sure why you are using DomElement but you should just cast it to the native object it wants to be cast to, either a IHTMLDocument2 or the object you see in the watch window, which I think is IHTMLFrameElement (if i recall correctly, but you will know what i mean once you see it). If you are trying to use an XML object, this could be the reason why you aren't able to get the HTML content, change the object declaration and casting if there is one, and give it a go & let us know :). Now I'm curious too :).

how can I use js/coffee to screen scrape an asp page?

I've got a website that I'd like to pull data from and it's really stuck in the stone ages. There's no web service, no API and it's very much an ASP/Session/table-based-layout page. Pretty fugly.
I'd like to just screen scrape it and use js (coffeescript) to automate that. I wonder if this is possible. I could do this with C# and linqpad but then I'm stuck parsing the tables (and sub-tables and sub-sub-tables) with regex. Plus if I do it with js or coffeescript I'll get much more comfortable with those languages and I'll be able to use jQuery for pulling elements out of the DOM.
I see two possibilities here:
use C# and find a library that will do things like Jquery but in C# code
use coffeescript (js) and use jquery to find the elements that I'm looking for in the page
I'd also like to automate the page a bit (get next set of results). This is strictly for personal use -- I'm not pulling results of someone's search to use in my business. I just want to make a crappy search engine do what I want.

I wrote a class that allows you to supply a bunch of urls and a code block to scrape pages inside a chrome extension. You can find the github repo here: https://github.com/jkarmel/Executor. It could use some more testing and I need to work on the documentation, but it looks like it might be what you are looking for.
Here is how you would use it to get the all the links from a few different pages:
/*
* background.js by Jeremy Karmel.
*/
URLS = ['http://www.apple.com/',
'http://www.google.com/',
'http://www.facebook.com/',
'http://www.stanford.edu'];
//Function will be provided to exector to collect information
var getLinks = function() {
var links = [];
var numLinks = $('a');
$links.each(function(i, val) {links.push(val.href)});
var request = {data: links, url: window.location.href};
chrome.extension.sendRequest(request);
}
var main = function() {
var specForUsersTopics = {
urls : URLS,
code : getLinks,
callback : function(results) {
for (var url in results) {
console.log(url + ' has ' + results[url].length + ' links.');
var links = results[url];
for (var i = 0; i < links.length; i++)
console.log(' ' + links[i]);
}
console.log('all done!!!!');
}
};
var exec = Executor(specForUsersTopics);
exec.start();
}
main();
So basically the code to collect the links would be supplied to the executor instance and then you would do whatever you wanted with the results in the callback. It can deal with longish lists of url (~1000) and it will work on more than one at a time (default == 5). It doesn't handle errors in the code block very well right now, so be sure to test the code you are supplying.

I'm liking Curtain A) "use C# and find a library..."
"HTML Agility Pack" might be just what you're looking for:
http://htmlagilitypack.codeplex.com/

You can do it easily with Node.js, jsdom, and jQuery. See this tutorial (in JavaScript).

How to send JavaScript code to IE using C# (.Net 3.5), run it, then get a string return value from the JS code?

We are developing an application which needs to interact with the active document in IE.
Context: The app is a C#, .Net 3.5 desktop app. The goal is to highlight specific text elements in the web page on user request. For this we need to retrieve and interpret web page elements (the need for the return value) then act on them through another JS call. The operations that must be made in the web page are not all done at the same time so we must get some kind of "snapshot" of the interesting text elements (we do this on the Mac version of our app by returning a string containing an XML representation of those elements).
In .Net we used IHTMLDocument2's execScript method successfully to run some JavaScript inside the active IE document, but we can't seem to find a way to get a return value from the call. Based on the doc execScript returns an execution success/failure constant which is not what we need.
In essence what we need to do is to load some JavaScript from a text file into a string, then send it to IE for execution. Then we need to get a string back from the called script.
Any hints on what objects to use? How to proceed to get this functionality?
Thanks in advance!
My colleague found the solution, based on what Alun Harford said:
string jsToRun = "function myTest() { return document.title; } myTest();";
mshtml.IHTMLDocument2 myIHTMLDocument2 = GetSelectedIEWindow();
IE ie = IE.AttachToIE(Find.ByUrl(myIHTMLDocument2.url));
string jsReturn = ie.Eval(jsToRun);
jsReturn then contains the string value returned from myTest() in JavaScript. Note that there is no return before the myTest() function call in the script!

Have a look at the WatiN codebase. In there, IE.Eval does exactly what you're looking for.

If you are providing the html and script yourself you can do the following:
execute the javascript function
let the js function place the result in an html element
wait till the function is done running
retrieve the html element using document.getElementById
and retrieve the value
I'm not sure if there's a easier way to do this.

Well it is nasty but it can be done.
Try this:
[Guid("626FC520-A41E-11CF-A731-00A0C9082637"), InterfaceType(ComInterfaceType.InterfaceIsDual)]
interface IHTMLDocument
{
void Script([Out, MarshalAs(UnmanagedType.Interface)] out object ppScript);
}
public object RunScript(InternetExplorer ie, string scriptText)
{
IHTMLDocument doc = (IHTMLDocument)ie.Document;
object scriptObj;
doc.Script(out scriptObj);
Type t = scriptObj.GetType();
return t.InvokeMember("eval", System.Reflection.BindingFlags.InvokeMethod, null, scriptObj, new object[] { scriptText });
}
That will return your value in the object (just cast to what ever type you expected). Of course .NET 4 makes this even easier ;)