We Keep Coding

Using MutationObserver in GeckoFx with C#?

I am using GeckoFx to perform a login to a specific website. This website edits the page with new information should the login fail (or require additional authentication, such as a ReCaptcha). Unfortunately, it is vital that I have access an event when the page is updated. I have tried numerous approaches mainly
A continual check if the uri is still the same upon each login attempt and a subsequent check on the specific element in question (to see if the display: none property was changed. (This resulted in an infinite loop as it seems GeckoFx updates the page in a nonblocking way, causing the program to go into an infinite loop)
Sleeping for ~5 seconds between login requests and using the aforementioned uri check. All this did (predictably, I was grasping at straws) was freeze the browser for 5 seconds and still fail to update the page
Searching the GeckoFx codebase for a specific event when the page is updated similar to the DocumentCompleted event (no such luck).
The most common approach I have read about (and one that makes the most sense) is to use a MutationObserver. It seems that all of the answers across the internet involve injecting Javascript in order to perform the requisite task. Seeing as all of my programming background has not touched web development whatsoever, I'm trying to stick to what I know.
Here is my approach so far, unfortunately, it is not much.
public class GeckoTestWebLogin
{
private readonly string _user;
private readonly string _pass;
public GeckoWebBrowser Gweb;
public Uri LoginUri { get; } = new Uri("https://website.com/login/");
public bool LoginCompleted { get; private set; } = false;
public bool Loaded { get; private set; } = false;
public GeckoTestWebLogin(string user, string pass)
{
_user = user;
_pass = pass;
Xpcom.EnableProfileMonitoring = false;
Xpcom.Initialize("Firefox");
//this code is for testing purposes, it will be removed upon project completion
CookieManager.RemoveAll();
Gweb = new GeckoWebBrowser();
Gweb.DocumentCompleted += DocLoaded;
//right about here is where I get lost, where can I set a callback method for the observer to report back to? Is this even how it works?
MutationObserver mutationObserver = new MutationObserver(Gweb.Window.DomWindow, (nsISupports)Gweb.Document.DomObject);
}
private void TestObservedEvent(string parms, object[] objs)
{
MessageBox.Show("The page was changed # " + DateTime.Now);
}
public void DocLoaded(object obj, GeckoDocumentCompletedEventArgs e)
{
Loaded = true;
if (Gweb.Url != LoginUri) return;
AttemptLogin();
}
private void AttemptLogin()
{
GeckoElementCollection elements = Gweb.Document.GetElementsByTagName("input");
foreach (GeckoHtmlElement element in elements)
{
switch (element.Id)
{
case "username":
element.SetAttribute("value", _user);
break;
case "password":
element.SetAttribute("value", _pass);
break;
case "importantchangedinfo":
GeckoHtmlElement authcodeModal =
(GeckoHtmlElement)
Gweb.Document.GetElementsByClassName("login_modal").First();
if (authcodeModal.Attributes["style"].NodeValue != "display: none")
{
InputForm form = new InputForm { InputDescription = "Captcha Required!" };
form.ShowDialog();
elements.FirstOrDefault(x => x.Id == "captchabox")?.SetAttribute("value", form.Input);
}
break;
}
}
elements.FirstOrDefault(x => x.Id == "Login")?.Click();
}
public void Login()
{
//this will cause the DocLoaded event to fire after completion
Gweb.Navigate(LoginUri.ToString());
}
}
As stated in the above code in the comments, I am completely lost at
MutationObserver mutationObserver = new MutationObserver(Gweb.Window.DomWindow, (nsISupports)Gweb.Document.DomObject);
I can't seem to find anything in GeckoFx's source for MutationObserver that would allow me to set a callback/event/whathaveyou. Is my approach the correct way to go about things or am I left with no options other than to inject Javascript into the page?
Much appreciated, thank you in advance.
Here is my attempt at option 2 in Tom's answer:
(Added into GeckoTestWebLogin)
public void DocLoaded(object obj, GeckoDocumentCompletedEventArgs e)
{
Loaded = true;
if (Gweb.Url != LoginUri) return;
MutationEventListener mutationListener = new MutationEventListener();
mutationListener.OnDomMutation += TestObservedEvent;
nsIDOMEventTarget target = Xpcom.QueryInterface<nsIDOMEventTarget>(/*Lost here*/);
using (nsAString modified = new nsAString("DOMSubtreeModified"))
target.AddEventListener(modified, mutationListener, true, false, 0);
AttemptLogin();
}
MutationEventListener.cs:
public delegate void OnDomMutation(/*DomMutationArgs args*/);
public class MutationEventListener : nsIDOMEventListener
{
public event OnDomMutation OnDomMutation;
public void HandleEvent(nsIDOMEvent domEvent)
{
OnDomMutation?.Invoke(/*new DomMutationArgs(domEvent, this)*/);
}
}

I don't think Geckofx's webidl compiler is currently advanced enough to generate the callback constructor.
Option 1. - Enhance MutationObserver source.
You could modify MutationObserver source manually to add the necessary constructor callback. Then recompile Geckofx. (I haven't look to see how difficult this is)
Option 2. - Use old style Mutation events.
public class DOMSubtreeModifiedEventListener : nsIDOMEventListener
{
... // Implement HandleEvent
}
Then something like (maybe in DocumentCompleted event handler):
_domSubtreeModifiedEventListener = new DOMSubtreeModifiedEventListener(this);
var target = Xpcom.QueryInterface<nsIDOMEventTarget>(body);
using (nsAString subtreeModified = new nsAString("DOMSubtreeModified"))
target.AddEventListener(subtreeModified, _domSubtreeModifiedEventListener, true, false, 0);
Option 3. - Use Idle + Check.
Add an winforms Application.idle event handler - and examine the document, to know when its ready.
Option 4. - Inject a javascript callback.
(As you have already mentioned) - This example is waiting until after a resize is done.
basically inject: "<body onresize=fireResizedEventAfterDelay()>" : then inject something like this:
string fireResizedEventAfterDelayScript = "<script>\n" +
"var resizeListner;" +
"var msDelay = 20;" +
"function fireResizedEventAfterDelay() {" +
"clearTimeout(resizeListner);" +
"resizeListner = setTimeout(function() { document.dispatchEvent (new MessageEvent('resized')); }, msDelay);" +
"}\n" +
"</script>\n";
Then in the C#:
browser.AddMessageEventListener("resized", (s) => runafterImDone())

Does getting current class and method name reduce performance?

I have a logging class which creates an instance of log4net and I can call it from anywhere in my code. I also have a method to find out the class and method name of caller. here is my method:
private static string CurrentMethod()
{
StackTrace stackTrace = new StackTrace();
MethodBase method = stackTrace.GetFrame(2).GetMethod();
string cn = method.ReflectedType.Name;
string mn = method.Name;
string output = "[" + cn + "." + mn + "]";
return output;
}
i found these articles: link1, Link2, Link3, Link4 and so many others, but none of them discuss about the efficiency and performance.
now I have two questions:
1- can I use this method in a big project(about 1000 requests in one second)? I mean how much does this effect the project's efficiency and performance?
2- Os there a better way to write the above method?
Also here is part of my logging class. I guess it can help:
public static class Tools_Log
{
private static ILog logger;
public static ILog GetLogger()
{
return logger ?? (logger = CreateLogger());
}
private static ILog CreateLogger()
{
//Some log4net initialization
return LogManager.GetLogger("WebService");
}
private static string CurrentMethod()
{
StackTrace stackTrace = new StackTrace();
MethodBase method = stackTrace.GetFrame(2).GetMethod();
string cn = method.ReflectedType.Name;
string mn = method.Name;
string output = "[" + cn + "." + mn + "]";
return output;
}
public static string MessageForLogFile(string message, string exeption, double time)
{
string currentMethod;
string executiontime;
string consumer;
string body;
string output;
currentMethod = CurrentMethod();
executiontime = ExecutionTime(time);
body = BodyForLog(message, exeption);
consumer = Consumer();
output = OutPut(currentMethod, executiontime, consumer, body);
return output;
}
}
i call the log class like this:
Tools_Log.GetLogger().Info(Tools_Log.MessageForLogFile("some text", "some text", execution time));
Thanks.

Yes, reflection is always a slower process. If you want the name of the method you can get it quite easily with the CallerMemberNameAttribute. Add a string parameter with an empty default value, and apply that attribute to it, and the compiler will send the name for you.
Also, if you're looking for fast logging, take a look at ETW instead of Log4Net.
Here: https://msdn.microsoft.com/en-us/library/windows/desktop/bb968803(v=vs.85).aspx
And Here: http://blogs.msdn.com/b/vancem/archive/2012/07/09/logging-your-own-etw-events-in-c-system-diagnostics-tracing-eventsource.aspx
EDIT:
As for your question in the comments, unfortunately, you can't get the name of the method 2 levels up, at least, not directly. CallerMemberName is basically just syntax sugar, it tells the compiler to take the name of the calling member, and place it in the parameter, so it works on a single level. However, since it relies on default parameters to do so, if you send the parameter manually, the CallerMemberName functionality doesn't apply, so you can do something like this:
public void UserMethod()
{
IntermediaryMethod();
}
public void IntermediaryMethod([CallerMemberName] caller = "")
{
LogMethod(caller)
}
public void LogMethod([CallerMemberName] caller = "")
{
// Log...
}
If you pass a value by yourself, it won't be overridden, so doing something like this will allow you to get the name of the caller from 2 levels up, while retaining the functionality for a single level.

Performance must be measured and this is done using profilers.
BTW, maybe instead of using StackTrace class, you can use caller information attributes introduced in .NET 4.5.
While [CallerMemberName] attribute doesn't provide class name, [CallerFilePath] provides the full path to the source code file where the caller member is defined (it might be even better for debugging/logging purposes since you know where to investigate a possible bug or error).

Design pattern for dynamic C# object

I have a queue that processes objects in a while loop. They are added asynchronously somewhere.. like this:
myqueue.pushback(String value);
And they are processed like this:
while(true)
{
String path = queue.pop();
if(process(path))
{
Console.WriteLine("Good!");
}
else
{
queue.pushback(path);
}
}
Now, the thing is that I'd like to modify this to support a TTL-like (time to live) flag, so the file path would be added o more than n times.
How could I do this, while keeping the bool process(String path) function signature? I don't want to modify that.
I thought about holding a map, or a list that counts how many times the process function returned false for a path and drop the path from the list at the n-th return of false. I wonder how can this be done more dynamically, and preferably I'd like the TTL to automatically decrement itself at each new addition to the process. I hope I am not talking trash.
Maybe using something like this
class JobData
{
public string path;
public short ttl;
public static implicit operator String(JobData jobData) {jobData.ttl--; return jobData.path;}
}

I like the idea of a JobData class, but there's already an answer demonstrating that, and the fact that you're working with file paths give you another possible advantage. Certain characters are not valid in file paths, and so you could choose one to use as a delimiter. The advantage here is that the queue type remains a string, and so you would not have to modify any of your existing asynchronous code. You can see a list of reserved path characters here:
http://en.wikipedia.org/wiki/Filename#Reserved_characters_and_words
For our purposes, I'll use the percent (%) character. Then you can modify your code as follows, and nothing else needs to change:
const int startingTTL = 100;
const string delimiter = "%";
while(true)
{
String[] path = queue.pop().Split(delimiter.ToCharArray());
int ttl = path.Length > 1?--int.Parse(path[1]):startingTTL;
if(process(path[0]))
{
Console.WriteLine("Good!");
}
else if (ttl > 0)
{
queue.pushback(string.Format("{0}{1}{2}", path[0], delimiter,ttl));
}
else
{
Console.WriteLine("TTL expired for path: {0}" path[0]);
}
}
Again, from a pure architecture standpoint, a class with two properties is a better design... but from a practical standpoint, YAGNI: this option means you can avoid going back and changing other asynchronous code that pushes into the queue. That code still only needs to know about the strings, and will work with this unmodified.
One more thing. I want to point out that this is a fairly tight loop, prone to running away with a cpu core. Additionally, if this is the .Net queue type and your tight loop gets ahead of your asynchronous produces to empty the queue, you'll throw an exception, which would break out of the while(true) block. You can solve both issues with code like this:
while(true)
{
try
{
String[] path = queue.pop().Split(delimiter.ToCharArray());
int ttl = path.Length > 1?--int.Parse(path[1]):startingTTL;
if(process(path[0]))
{
Console.WriteLine("Good!");
}
else if (ttl > 0)
{
queue.pushback(string.Format("{0}{1}{2}", path[0], delimiter,ttl));
}
else
{
Console.WriteLine("TTL expired for path: {0}" path[0]);
}
}
catch(InvalidOperationException ex)
{
//Queue.Dequeue throws InvalidOperation if the queue is empty... sleep for a bit before trying again
Thread.Sleep(100);
}
}

If the constraint is that bool process(String path) cannot be touched/changed then put the functionality into myqueue. You can keep its public signatures of void pushback(string path) and string pop(), but internally you can track your TTL. You can either wrap the string paths in a JobData-like class that gets added to the internal queue, or you can have a secondary Dictionary keyed by path. Perhaps even something as simple as saving the last poped path and if the subsequent push is the same path you can assume it was a rejected/failed item. Also, in your pop method you can even discard a path that has been rejected too many time and internally fetch the next path so the calling code is blissfully unaware of the issue.

You could abstract/encapsulate the functionality of the "job manager". Hide the queue and implementation from the caller so you can do whatever you want without the callers caring. Something like this:
public static class JobManager
{
private static Queue<JobData> _queue;
static JobManager() { Task.Factory.StartNew(() => { StartProcessing(); }); }
public static void AddJob(string value)
{
//TODO: validate
_queue.Enqueue(new JobData(value));
}
private static StartProcessing()
{
while (true)
{
if (_queue.Count > 0)
{
JobData data = _queue.Dequeue();
if (!process(data.Path))
{
data.TTL--;
if (data.TTL > 0)
_queue.Enqueue(data);
}
}
else
{
Thread.Sleep(1000);
}
}
}
private class JobData
{
public string Path { get; set; }
public short TTL { get; set; }
public JobData(string value)
{
this.Path = value;
this.TTL = DEFAULT_TTL;
}
}
}
Then your processing loop can handle the TTL value.
Edit - Added a simple processing loop. This code isn't thread safe, but should hopefully give you an idea.

Encapsulation with multiple classes C#

So my question is over basic encapsulation. I know I am setting up my getters and setters right (I actually have a question later about this) but I have multiple classes. I have another class and I understand that I am making pieces of my code view-able to my outside class by making certain things public. So I think I set up my first code file right. (Some background, I have a class that is connecting to a database and then another that is encapsulating all the data. The first code section posted is the encapsulating part, I then post my three methods I was messing up on.)
I feel okay with the getting and setting, I feel a little unsure of my constructor. I feel like I put my variables in the parameter list so that I put values in them from my outside class? Right? or should I be putting public forms of my private variables in my other code file and then passing those into my constructor in that same file?
/ this my first code file
using System;
public class clsEncapsulate
{
private int mID;
private string mName;
private string mClassification;
private DateTime mConvocationDate;
private string mLocation;
public int ID
{
get
{
return mID;
}
set
{
mID = value;
}
}
public string Name
{
get
{
return mName;
}
set
{
mName = value;
}
}
public string Classification
{
get
{
return mName;
}
set
{
mName = value;
}
}
private DateTime ConvocationDate
{
get
{
return mConvocationDate;
}
set
{
mConvocationDate = value;
}
}
private string Location
{
get
{
return mLocation;
}
set
{
mLocation = value;
}
}
public clsEncapsulate(int id, string name, string classification, DateTime convocationDate, string location)
{
bool running = false;
while(running == false)
{
ID = mID;
Name = mName;
Classification = mClassification;
ConvocationDate = mConvocationDate;
Location = mLocation;
running = true;
}
}
}
In my second code file I am just going to put the methods that I am having trouble with.
private void refreshbox()
{
string formattedConvocation;
string formattedDateTime;
string formattedConvocationName;
lstConvocations.Items.Clear();
foreach (clsEncapsulate currentConvocation in mConvocationAL)
{
formattedConvocationName = currentConvocation.Name;
formattedConvocationName = truncateString(formattedConvocationName, 30);
formattedConvocation = formattedConvocationName.PadRight(33);
formattedConvocation += currentConvocation.Classification.PadRight(17);
formattedDateTime = currentConvocation.ConvocationDate.ToShortDateString().PadRight(10)
+ currentConvocation.ConvocationDate.ToShortTimeString().PadLeft(8);
formattedConvocation += formattedDateTime;
lstConvocations.Items.Add(formattedConvocation);
}
}
Alright, so in order for my second code file to manipulate the variables in the first code file, I need to expose them to this method. I didn't know if I should be putting my public variables in the constructor, or if I should be declaring them somewhere in my first code file. I was very unsure of how to expose these variables to this method. I've fiddle around with it but my book doesn't address this situation exactly and I was having trouble figuring it out.
If someone does answer this question please break down why you're going to put what you're going to put! I want to understand why, say, I put my public variables in one place, and not another. Or why I declare an object of my encapsulate class in one place and not another. I was trying to declare an encapsulate object in my method so it would give this method access to the variables, but it wasn't working! Please tell me what I was doing wrong or if you want me to post more of my code.
Below are the two other methods I was messing up on.
/ second method from my second code file I was messing up on:
private void displayProperties(int index)
{
if (index == -1)
{
return;
}
clsEncapsulate selectedValue = (clsEncapsulate)mConvocationAL[index];
txtConvocationName.Text = selectedValue.Name;
txtConvocationClassification.Text = selectedValue.Classification;
txtConvocationDate.Text = selectedValue.ConvocationDate.ToShortDateString();
txtConvocationTime.Text = selectedValue.ConvocationDate.ToShortTimeString();
txtConvocationLocation.Text = selectedValue.Location;
txtID.Text = selectedValue.ID.ToString();
}
/ last method I was messing up on:
private void readConvocations(string filterConstraint, string sortField, string sortOrder)
{
OleDbConnection connection = null;
OleDbDataReader reader = null;
try
{
connection = new OleDbConnection();
connection.ConnectionString = mConnectionString;
connection.Open();
string statement = "SELECT ID, Name, Classification, Location, Date FROM Convocations ";
if(filterConstraint != "")
{
statement += "WHERE Name LIKE " + toSQL(filterConstraint, true) + " ";
}
string statement2 = statement;
statement = string.Concat(new string[]
{
statement2, "ORDER BY ", sortField, " ", sortOrder
});
OleDbCommand oleDbCommand = new OleDbCommand(statement, connection);
reader = oleDbCommand.ExecuteReader();
mConvocationAL.Clear();
while(reader.Read())
{
clsEncapsulteconvocation = new clsEncapsulate();
convocation.ID = (int)reader["ID"];
convocation.Name = (string)reader["Name"];
convocation.Classification = (string)reader["Classification"];
convocation.Location = (string)reader["Location"];
convocation.ConvocationDate = (DateTime)reader["Date"];
mConvocationAL.Add(convocation);
}
}
finally
{
if (reader != null)
{
reader.Close();
}
if (connection != null)
{
connection.Close();
}
}
}
Tell me if you need me to elaborate more to help you understand my situation. I am new at learning vocabulary and want to understand this! Thank you for helping. :)

The code you provided is one public object and a bunch of private methods so its difficult to get the overall picture of how your code it working together but there are a few principles you can apply to make your code better structured, now and in the future.
Have a read about SOLID (http://en.wikipedia.org/wiki/SOLID_(object-oriented_design)). The S and the D would apply well to your example.
Also you mentioned construtors and private properties. Try looking into Imutable types. That means once the object is created you cannot change it. For your clsEncapsulate class that would mean making your fields read only and remove the public setters.
Good luck.

Massive list in c# & wpf

I have a list consisting of all the US Zip codes, each with 3 elements. Thus the list is ~45,000 x 3 strings. What is the best way to load this, essentially the most efficient/optimized? Right now I have a foreach loop running it, and every time it gets to the loading point it hangs. Is there a better approach?
Edit
The usage of this is for the user to be able to type in a zip code and have the city and state displayed in two other text boxes. Right now I have it set to check as the user types, an after the dirt number is entered it freezes up, I believe at the ZipCodes codes = new ZipCodes()
This is the code I'm currently using. I left one of the zipCode.Add statements in, but deleted the other 44,999.
struct ZipCode
{
private String cvZipCode;
private String cvCity;
private String cvState;
public string ZipCodeID { get { return cvZipCode; } }
public string City { get { return cvCity; } }
public string State { get { return cvState; } }
public ZipCode(string zipCode, string city, string state)
{
cvZipCode = zipCode;
cvCity = city;
cvState = state;
}
public override string ToString()
{
return City.ToString() + ", " + State.ToString();
}
}
class ZipCodes
{
private List<ZipCode> zipCodes = new List<ZipCode>();
public ZipCodes()
{
zipCodes.Add(new ZipCode("97475","SPRINGFIELD","OR"));
}
public IEnumerable<ZipCode> GetLocation()
{
return zipCodes;
}
public IEnumerable<ZipCode> GetLocationZipCode(string zipCode)
{
return zipCodes;
}
public IEnumerable<ZipCode> GetLocationCities(string city)
{
return zipCodes;
}
public IEnumerable<ZipCode> GetLocationStates(string state)
{
return zipCodes;
}
}
private void LocateZipCode(TextBox source, TextBox destination, TextBox destination2 = null)
{
ZipCodes zips = new ZipCodes();
string tempZipCode;
List<ZipCode> zipCodes = new List<ZipCode>();
try
{
if (source.Text.Length == 5)
{
tempZipCode = source.Text.Substring(0, 5);
dataWorker.RunWorkerAsync();
destination.Text = zipCodes.Find(searchZipCode => searchZipCode.ZipCodeID == tempZipCode).City.ToString();
if (destination2.Text != null)
{
destination2.Text = zipCodes.Find(searchZipCode => searchZipCode.ZipCodeID == tempZipCode).State.ToString();
}
}
else destination2.Text = "";
}
catch (NullReferenceException)
{
destination.Text = "Invalid Zip Code";
if (destination2 != null)
{
destination2.Text = "";
}
}
}

There are several options that depend on your use case and target client machines.
Use paged controls. Use existing paged control variants (eg. telerik) which support paging. This way you will deal with smaller subset of the data available.
Use search/filter controls. Force users to enter partial data to reduce the size of the data you need to show.
Using observable collection will cause performance problems as framework provided class does not support bulk load. Make your own observable collection which supports bulk loading (which does not raise collection changed event on every element you add). On a list of 5-10.000 members I've seen loading times reduced from 3s to 0.03s.
Use async operations when loading data from db. This way UI stays responsive and you have a chance to inform users about the current operation. This improves the perceived performance immensely.

Instead of loading all of the items, try loading on demand. For instance, when user enters the first three letters then query the list and return only matching items. Many controls exists for this purpose both in silverlight and ajax.

Thanks for all the responses, I really do appreciate them. A couple I didn't really understand, but I know that's my own lack of knowledge in certain areas of c#. In researching them though, I did stumble across a different solution that had worked beautifully, using a Dictionary<T> instead of a List. Even without using a BackgroundWorker, it loads on app start-up in about 5 seconds. I had heard of Dictionary<T> before, but until now had never had a cause to use/research it, so this was doubly beneficial to me. Thanks again for all the assistance!