I am working on a .NET Application which is rendering data from a source (say TCP) to excel using the Excel-DNA Library.
I have given example of LtpObserver which I have created by implementing IExcelObservable. I am creating the Observer using a Excel Function Call (Refer Code) --> "GetLtp".
'Data' referred here is containing a List of LtpObservers and Ltp value.
THE PROBLEM -
When I call the formula in Excel using a symbol say 'X', it creates the Observer (verified via Logging), and the data starts updating. Constructor of observer is called, as well as, callback is received on Subscribe() Method. But, when I call the same formula for 'X in another cell, no new Observer is created, neither a call on subscribe is received.
And upon deletion, of formula from just Cell#1, the Dispose is not called. But upon deleting from both Cell#1 and Cell#2 Dispose is called.
So is there just one observer for every unique symbol in the Excel Workbook? In that case does keeping a list of Observers and updating each of them on OnNext() makes sense? And what about the case when 2 or more formulas are used together in a single cell? How does this work Internally
CODE
class LtpObserver : IExcelObservable
{
private List<IExcelObserver> _observerList;
private string _symbol;
private Timer _timer;
public LtpObserver(string symbol)
{
Trace.TraceInformation("Constructor Called. New Ltp Observer Loaded for Symbol : " + symbol);
_symbol = symbol;
_observerList = new List<IExcelObserver>();
_timer = new Timer();
_timer.AutoReset = true;
_timer.Interval = FeedTimerConstant.ltpFrequency;
_timer.Elapsed += _timer_Elapsed;
}
void _timer_Elapsed(object sender, ElapsedEventArgs e)
{
if (Data != null))
{
_timer.Stop();
Data.LtpObserver = _observerList;
timer_tick(Data.Ltp);
}
else
{
timer_tick(0);
}
}
public IDisposable Subscribe(IExcelObserver observer)
{
Trace.TraceInformation("LTP Subscription for : " + observer.ToString() + " Symbol : " + _symbol);
_observerList.Add(observer);
if (Data != null)
{
Data.LtpObserver = _observerList;
timer_tick(Data.Ltp);
}
else
{
timer_tick(0);
_timer.Start();
}
return new ActionDisposable(() => {
Trace.TraceInformation("Un Subscribed for LTP Price :" + _symbol);
_observerList.Remove(observer);
});
}
void timer_tick(object _now)
{
foreach (var obs in _observerList)
obs.OnNext(_now);
}
}
-----------------------------------------------------------------------------------------------------------
[ExcelFunction(Name = "GetLtp", IsVolatile = true)]
public static object GetLtp(string symbol)
{
if (String.IsNullOrEmpty(symbol))
{
return INVALID_SYMBOL;
}
return ExcelAsyncUtil.Observe("GetLtp", symbol, () => new LtpObserver(symbol)
);
}
-------------------------------------------------------------------------------------------------------------
public class Data {
public decimal Ltp { get; set; }
public List<IExcelObserver> LtpObserver { get; set; }
}
The behaviour you describe is by design. The first two parameters passed to ExcelAsyncUtil.Observe are normally a string with the function name and an object or array of objects that provide a unique identifier for the IObservable stream.
So the part you describe in "THE PROBLEM" is exactly how it is intended to work. For every unique symbol you have a stream of values, and you can use the same values in multiple places on the sheet without a problem - they will be listening to the same underlying stream. If you have multiple different symbols active, you have multiple streams that you update with whatever mechanism, using the OnNext() calls.
In your code you need not make provision for multiple IObservers that Subscribe to your IObservable. But this detail is specific to the way Excel-DNA will call your IObservable. Excel-DNA will only ever Subscribe once to your IObservable. However, if you wanted to use the same IObservable class in other contexts or in another application, this might not hold an you might get multiple Subscribe calls.
(It would also make more sense to rename LtpObserver to LtpObservable.)
You could add extra information to that list to create separate topics according to the calling cell. This code might be something like this:
(I've removed the IsVolatile=true as that doesn't make sense for a streaming function wrapper like this)
[ExcelFunction(Name = "GetLtp")]
public static object GetLtp(string symbol)
{
if (String.IsNullOrEmpty(symbol))
{
return INVALID_SYMBOL;
}
var callerReference = XlCall.Excel(XlCall.xlfCaller);
var identifiers = new object[] { symbol, callerReference };
return ExcelAsyncUtil.Observe("GetLtp", identifiers, () => new LtpObserver(symbol, callerReference)
);
}
But now again you have a unique LtpObserver for the combination of symbol and calling cell. And again each such IObservable will only be subscribed to once by Excel-DNA.
Internally this is implemented using Excel's RTD mechanism. Excel can associate one or more RTD topics with a cell. When the topic value changes, Excel will invalidate the relevant cells, and recalculate them. There is some optimisation inside Excel so that this scales well.
Related
I am using GeckoFx to perform a login to a specific website. This website edits the page with new information should the login fail (or require additional authentication, such as a ReCaptcha). Unfortunately, it is vital that I have access an event when the page is updated. I have tried numerous approaches mainly
A continual check if the uri is still the same upon each login attempt and a subsequent check on the specific element in question (to see if the display: none property was changed. (This resulted in an infinite loop as it seems GeckoFx updates the page in a nonblocking way, causing the program to go into an infinite loop)
Sleeping for ~5 seconds between login requests and using the aforementioned uri check. All this did (predictably, I was grasping at straws) was freeze the browser for 5 seconds and still fail to update the page
Searching the GeckoFx codebase for a specific event when the page is updated similar to the DocumentCompleted event (no such luck).
The most common approach I have read about (and one that makes the most sense) is to use a MutationObserver. It seems that all of the answers across the internet involve injecting Javascript in order to perform the requisite task. Seeing as all of my programming background has not touched web development whatsoever, I'm trying to stick to what I know.
Here is my approach so far, unfortunately, it is not much.
public class GeckoTestWebLogin
{
private readonly string _user;
private readonly string _pass;
public GeckoWebBrowser Gweb;
public Uri LoginUri { get; } = new Uri("https://website.com/login/");
public bool LoginCompleted { get; private set; } = false;
public bool Loaded { get; private set; } = false;
public GeckoTestWebLogin(string user, string pass)
{
_user = user;
_pass = pass;
Xpcom.EnableProfileMonitoring = false;
Xpcom.Initialize("Firefox");
//this code is for testing purposes, it will be removed upon project completion
CookieManager.RemoveAll();
Gweb = new GeckoWebBrowser();
Gweb.DocumentCompleted += DocLoaded;
//right about here is where I get lost, where can I set a callback method for the observer to report back to? Is this even how it works?
MutationObserver mutationObserver = new MutationObserver(Gweb.Window.DomWindow, (nsISupports)Gweb.Document.DomObject);
}
private void TestObservedEvent(string parms, object[] objs)
{
MessageBox.Show("The page was changed # " + DateTime.Now);
}
public void DocLoaded(object obj, GeckoDocumentCompletedEventArgs e)
{
Loaded = true;
if (Gweb.Url != LoginUri) return;
AttemptLogin();
}
private void AttemptLogin()
{
GeckoElementCollection elements = Gweb.Document.GetElementsByTagName("input");
foreach (GeckoHtmlElement element in elements)
{
switch (element.Id)
{
case "username":
element.SetAttribute("value", _user);
break;
case "password":
element.SetAttribute("value", _pass);
break;
case "importantchangedinfo":
GeckoHtmlElement authcodeModal =
(GeckoHtmlElement)
Gweb.Document.GetElementsByClassName("login_modal").First();
if (authcodeModal.Attributes["style"].NodeValue != "display: none")
{
InputForm form = new InputForm { InputDescription = "Captcha Required!" };
form.ShowDialog();
elements.FirstOrDefault(x => x.Id == "captchabox")?.SetAttribute("value", form.Input);
}
break;
}
}
elements.FirstOrDefault(x => x.Id == "Login")?.Click();
}
public void Login()
{
//this will cause the DocLoaded event to fire after completion
Gweb.Navigate(LoginUri.ToString());
}
}
As stated in the above code in the comments, I am completely lost at
MutationObserver mutationObserver = new MutationObserver(Gweb.Window.DomWindow, (nsISupports)Gweb.Document.DomObject);
I can't seem to find anything in GeckoFx's source for MutationObserver that would allow me to set a callback/event/whathaveyou. Is my approach the correct way to go about things or am I left with no options other than to inject Javascript into the page?
Much appreciated, thank you in advance.
Here is my attempt at option 2 in Tom's answer:
(Added into GeckoTestWebLogin)
public void DocLoaded(object obj, GeckoDocumentCompletedEventArgs e)
{
Loaded = true;
if (Gweb.Url != LoginUri) return;
MutationEventListener mutationListener = new MutationEventListener();
mutationListener.OnDomMutation += TestObservedEvent;
nsIDOMEventTarget target = Xpcom.QueryInterface<nsIDOMEventTarget>(/*Lost here*/);
using (nsAString modified = new nsAString("DOMSubtreeModified"))
target.AddEventListener(modified, mutationListener, true, false, 0);
AttemptLogin();
}
MutationEventListener.cs:
public delegate void OnDomMutation(/*DomMutationArgs args*/);
public class MutationEventListener : nsIDOMEventListener
{
public event OnDomMutation OnDomMutation;
public void HandleEvent(nsIDOMEvent domEvent)
{
OnDomMutation?.Invoke(/*new DomMutationArgs(domEvent, this)*/);
}
}
I don't think Geckofx's webidl compiler is currently advanced enough to generate the callback constructor.
Option 1. - Enhance MutationObserver source.
You could modify MutationObserver source manually to add the necessary constructor callback. Then recompile Geckofx. (I haven't look to see how difficult this is)
Option 2. - Use old style Mutation events.
public class DOMSubtreeModifiedEventListener : nsIDOMEventListener
{
... // Implement HandleEvent
}
Then something like (maybe in DocumentCompleted event handler):
_domSubtreeModifiedEventListener = new DOMSubtreeModifiedEventListener(this);
var target = Xpcom.QueryInterface<nsIDOMEventTarget>(body);
using (nsAString subtreeModified = new nsAString("DOMSubtreeModified"))
target.AddEventListener(subtreeModified, _domSubtreeModifiedEventListener, true, false, 0);
Option 3. - Use Idle + Check.
Add an winforms Application.idle event handler - and examine the document, to know when its ready.
Option 4. - Inject a javascript callback.
(As you have already mentioned) - This example is waiting until after a resize is done.
basically inject: "<body onresize=fireResizedEventAfterDelay()>" : then inject something like this:
string fireResizedEventAfterDelayScript = "<script>\n" +
"var resizeListner;" +
"var msDelay = 20;" +
"function fireResizedEventAfterDelay() {" +
"clearTimeout(resizeListner);" +
"resizeListner = setTimeout(function() { document.dispatchEvent (new MessageEvent('resized')); }, msDelay);" +
"}\n" +
"</script>\n";
Then in the C#:
browser.AddMessageEventListener("resized", (s) => runafterImDone())
I'm writing a Windows Service that will execute different data import logic, from different data source to eventually write it to a single target, a MS CRM instance. Right now, the only thing I think will be problematic, is the writing to CRM part. The concurent reading of data from different (sometimes same) data source shouldn't really be an issue (I may be wrong on this...) So I came up with a way to make sure there are no concurent writes (create or updates) to CRM.
Here's the general design for the moment:
What happens when the service starts:
Timers = new List<System.Timers.Timer>();
CrmTransactionQueue.Lock = new object { }; //Static class. The object for locking purposes...
System.Threading.Thread.Sleep(20000); //for debugging purpose so I can attach to process before everything kicks in...
//retrieve all types that are extending BaseSyncStrategy..
var strategyTypes = Assembly.GetExecutingAssembly().GetTypes().Where(x => x.BaseType == typeof(BaseSyncStrategy));
foreach (Type strategyType in strategyTypes)
{
//create a instance of that type....
var strategy = (BaseSyncStrategy)Activator.CreateInstance(strategyType);
//create a timer for each of these, they will have different intervals...
System.Timers.Timer t = new System.Timers.Timer
{
Interval = strategy.Interval * 1000,
AutoReset = false,
Enabled = true
};
Timers.Add(t);
t.Elapsed += (sender, e) => TimerElapsed(sender, e, strategy);
t.Start();
}
What happens when the timers' interval are expired:
private void TimerElapsed(object sender, ElapsedEventArgs e, BaseSyncStrategy strategy)
{
//get timer back
var timer = (Timer)sender;
try
{
strategy.Execute();
}
catch (Exception ex)
{
Logger.WriteEntry(EventLogEntryType.Error, $"Error executing strategy {strategy.GetType().Name}: ", ex);
}
timer.Start();
}
And within all the Execute methods of objects extending BaseSyncStrategy, each time I want to update or create something in the target CRM instance, I do this:
XrmServiceContext XrmCtx = new XrmServiceContext();
//....
//code that fetches data from foreign sources and creates CRM entities...
//....
Action<XrmServiceContext> action = (XrmServiceContext ctx) =>
{
//write those created/updated objects
//ctx lets me query entities and write back to CRM...
};
CrmTransactionQueue.Execute(action, XrmCtx);
And the simple code to make sure (I think) no concurent writes to CRM happen:
public static class CrmTransactionQueue
{
public static object Lock { get; set; }
public static void Execute(Action<XrmServiceContext> transaction, XrmServiceContext Ctx)
{
lock (Lock)
{
transaction.Invoke(Ctx);
}
}
}
Is this sound design or there's a better way to do this ?
I am currently using the Change Notifications in Active Directory Domain Services in .NET as described in this blog. This will return all events that happen on an selected object (or in the subtree of that object). I now want to filter the list of events for creation and deletion (and maybe undeletion) events.
I would like to tell the ChangeNotifier class to only observe create-/delete-/undelete-events. The other solution is to receive all events and filter them on my side. I know that in case of the deletion of an object, the atribute list that is returned will contain the attribute isDeleted with the value True. But is there a way to see if the event represents the creation of an object? In my tests the value for usnchanged is always usncreated+1 in case of userobjects and both are equal for OUs, but can this be assured in high-frequency ADs? It is also possible to compare the changed and modified timestamp. And how can I tell if an object has been undeleted?
Just for the record, here is the main part of the code from the blog:
public class ChangeNotifier : IDisposable
{
static void Main(string[] args)
{
using (LdapConnection connect = CreateConnection("localhost"))
{
using (ChangeNotifier notifier = new ChangeNotifier(connect))
{
//register some objects for notifications (limit 5)
notifier.Register("dc=dunnry,dc=net", SearchScope.OneLevel);
notifier.Register("cn=testuser1,ou=users,dc=dunnry,dc=net", SearchScope.Base);
notifier.ObjectChanged += new EventHandler<ObjectChangedEventArgs>(notifier_ObjectChanged);
Console.WriteLine("Waiting for changes...");
Console.WriteLine();
Console.ReadLine();
}
}
}
static void notifier_ObjectChanged(object sender, ObjectChangedEventArgs e)
{
Console.WriteLine(e.Result.DistinguishedName);
foreach (string attrib in e.Result.Attributes.AttributeNames)
{
foreach (var item in e.Result.Attributes[attrib].GetValues(typeof(string)))
{
Console.WriteLine("\t{0}: {1}", attrib, item);
}
}
Console.WriteLine();
Console.WriteLine("====================");
Console.WriteLine();
}
LdapConnection _connection;
HashSet<IAsyncResult> _results = new HashSet<IAsyncResult>();
public ChangeNotifier(LdapConnection connection)
{
_connection = connection;
_connection.AutoBind = true;
}
public void Register(string dn, SearchScope scope)
{
SearchRequest request = new SearchRequest(
dn, //root the search here
"(objectClass=*)", //very inclusive
scope, //any scope works
null //we are interested in all attributes
);
//register our search
request.Controls.Add(new DirectoryNotificationControl());
//we will send this async and register our callback
//note how we would like to have partial results
IAsyncResult result = _connection.BeginSendRequest(
request,
TimeSpan.FromDays(1), //set timeout to a day...
PartialResultProcessing.ReturnPartialResultsAndNotifyCallback,
Notify,
request
);
//store the hash for disposal later
_results.Add(result);
}
private void Notify(IAsyncResult result)
{
//since our search is long running, we don't want to use EndSendRequest
PartialResultsCollection prc = _connection.GetPartialResults(result);
foreach (SearchResultEntry entry in prc)
{
OnObjectChanged(new ObjectChangedEventArgs(entry));
}
}
private void OnObjectChanged(ObjectChangedEventArgs args)
{
if (ObjectChanged != null)
{
ObjectChanged(this, args);
}
}
public event EventHandler<ObjectChangedEventArgs> ObjectChanged;
#region IDisposable Members
public void Dispose()
{
foreach (var result in _results)
{
//end each async search
_connection.Abort(result);
}
}
#endregion
}
public class ObjectChangedEventArgs : EventArgs
{
public ObjectChangedEventArgs(SearchResultEntry entry)
{
Result = entry;
}
public SearchResultEntry Result { get; set; }
}
I participated in a design review about five years back on a project that started out using AD change notification. Very similar questions to yours were asked. I can share what I remember, and don't think things have change much since then. We ended up switching to DirSync.
It didn't seem possible to get just creates & deletes from AD change notifications. We found change notification resulted enough events monitoring a large directory that notification processing could bottleneck and fall behind. This API is not designed for scale, but as I recall the performance/latency were not the primary reason we switched.
Yes, the usn relationship for new objects generally holds, although I think there are multi-dc scenarios where you can get usncreated == usnchanged for a new user, but we didn't test that extensively, because...
The important thing for us was that change notification only gives you reliable object creation detection under the unrealistic assumption that your machine is up 100% of the time! In production systems there are always some case where you need to reboot and catch up or re-synchronize, and we switched to DirSync because it has a robust way to handle those scenarios.
In our case it could block email to a new user for an indeterminate time if an object create were missed. That obviously wouldn't be good, we needed to be sure. For AD change notifications, getting that resync right that would have some more work and hard to test. But for DirSync, its more natural, and there's a fast-path resume mechanism that usually avoids resync. For safety I think we triggered a full re-synchronize every day.
DirSync is not as real-time as change notification, but its possible to get ~30-second average latency by issuing the DirSync query once a minute.
I am using fiddlercore to capture session information to run a compare on the data in a particular response. One of the things I am noticing that I don't understand is that I am getting session information from the second environment into the List collection I have for the first.
public class ManageCompares
{
public static string _test2BaseURL = "https://test2/";
public static string _dev1BaseURL = "http://dev1/";
private void RunCompares(string email, string handler, Reporting report)
{
ManageProcess.ShutDownProcess("iexplore");
RunExports exportTest2 = new RunExports();
RunExports exportDev1 = new RunExports();
string password = "d";
List<Session> oAllSessions_Test2 = exportTest2.RunExportGeneration
(email, password, _test2BaseURL, handler);
ManageProcess.ShutDownProcess("iexplore");
List<Session> oAllSessions_Dev1 = exportDev1.RunExportGeneration
(email, password, _dev1BaseURL, handler);
exportTest2.ExtractResponse(oAllSessions_Test2, handler, report);
//report.SetEnvironment2Body(ManageExports.ExtractResponse
// (oAllSessions_Dev1, handler, report, report.Environment2));
if (report.Test2ResponseCode != 500 && report.Dev1ResponseCode != 500)
{
bool matches = CompareExports.CompareExportResults
(report.Environment1Body, report.Environment2Body);
if (matches)
{
Console.ForegroundColor = ConsoleColor.Green;
Console.WriteLine("Exports matched");
Console.ResetColor();
}
else
{
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine("Export does not match");
Console.ResetColor();
report.GenerateReportFiles();
}
}
else
{
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine
("A exception was returned. Please review the log file.");
Console.ResetColor();
}
}
}
public class RunExports
{
public List<Session> RunExportGeneration
(string email, string password, string baseUrl,
string handlersUrlwithParams)
{
IWebDriver driver = new InternetExplorerDriver();
FiddlerApplication.Startup(8877, FiddlerCoreStartupFlags.Default);
List<Session> oAllSessions = new List<Session>();
LoginPage login = new LoginPage(driver);
FiddlerApplication.AfterSessionComplete += delegate(Session oS)
{
Monitor.Enter(oAllSessions);
oAllSessions.Add(oS);
Monitor.Exit(oAllSessions);
};
try
{
driver.Navigate().GoToUrl(baseUrl);
login.LoginToView(email, password);
driver.Navigate().GoToUrl(baseUrl + handlersUrlwithParams);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
finally
{
FiddlerApplication.Shutdown();
driver.Quit();
}
return oAllSessions;
}
}
List oAllSessions_Test2 and List oAllSessions_Dev1 are my two collections. When I debug the capture I typically see 15 rows in the oAllSessions_Test2 collection. Then after capturing oAllSessions_Dev1 I see the count has jumped up 14 or 15 and when I look at what is contained by the colleciton some of the Dev1 captures are now in there. oAllSessions_Dev1 has just the sessions I am expecting. I am guessing there must be a pointer someplace I am not expecting but I am stumped at this point how to clear it up. The other thing that I am noticing is that the session counter continues to increment while the application is cycling through the various cases.
I am also using Selenium WebDriver and IE to initiate the browser session but I don't think that is particularly relevant to this particular issue.
So what am I missing here?
Do this :
SessionStateHandler tAction = oS =>
{
Monitor.Enter(oAllSessions);
oAllSessions.Add(oS);
Monitor.Exit(oAllSessions);
};
FiddlerApplication.AfterSessionComplete += tAction;
......
//at the end before your return statement:
FiddlerApplication.AfterSessionComplete -= tAction;
So here is what's going on.
this: FiddlerApplication.Startup(8877, FiddlerCoreStartupFlags.Default);
(this specifically) FiddlerApplication
is holding onto references in an external application (fiddler) and administrating them for you. When you += and add a delegate FiddlerApplication.AfterSessionComplete += tAction;, the fiddler application was adding this to the list of methods it calls when the AfterSession event fires.
Because it's singleton (You are only dealing with one Fiddler application instance in your code), every time you do a += it adds it to the same list. This list in the FiddlerApplication doesn't get recreated every time you call your method. It's the same one that you called the first time, so even though your delegate falls out of it's scope declaration space (like local objects normally do in a method), the FiddlerApplication EventList maintains a reference to it and fires that delegate each time (and every other one).
So.....
In your method you create List<Session> oAllSessions = new List<Session>(); and access it in your delegate. This local variable is now passed back to the calling method, List<Session> oAllSessions_Test2 = exportTest2.RunExportGeneration.... and the delegate FiddlerApplication calls is the exact same list. So each time you call that method and the AfterSessionComplete fires, it updates the list even after it's been returned to the calling function.
To stop it you have to add: FiddlerApplication.AfterSessionComplete -= tAction; which tells the system "Hey, don't push updates to this method anymore. It's done receiving notifications."
In .NET languages class objects are passed by reference in almost all cases. If you assign an object reference to a variable, that variable will still refer to the original object. Changes to the original will be visible through all references. This is similar to how things used to happen in C/C++ when you stored pointers or references to structures in memory. The implementation details are different, but the results are the same.
Here's an example of what happens when you store list instances that might be changed later:
class ListTest
{
List<string> l = new List<string>();
public List<string> GetList() { return l; }
public void Add(string v) { l.Add(v); }
}
class Program
{
static void Main(string[] args)
{
ListTest t = new ListTest();
t.Add("a"); t.Add("b"); t.Add("c"); t.Add("d");
List<string> x1 = t.GetList();
List<string> x2 = t.GetList().ToList();
t.Add("e"); t.Add("f"); t.Add("g"); t.Add("h");
List<string> y1 = t.GetList();
List<string> y2 = t.GetList().ToList();
Console.WriteLine("{0}, {1}", x1.Count, y1.Count);
Console.WriteLine("{0}", string.Join(", ", x1));
Console.WriteLine("{0}", string.Join(", ", y1));
Console.WriteLine();
Console.WriteLine("{0}, {1}", x2.Count, y2.Count);
Console.WriteLine("{0}", string.Join(", ", x2));
Console.WriteLine("{0}", string.Join(", ", y2));
}
}
When you run that you get one set of identical results, because x1 and y1 are references to the same object. The second set of results are different because the call to ToList creates a new List<string> instance to hold the results.
This might be what is happening with the list that is returned by your Fiddler code.
--
Update after code review
What it looks like is that the delegate you're assigning to the AfterSessionComplete event is causing the code to treat oAllSessions as a static object bound to the event handler delegate. Various side-effects like this happen when you start playing around with code that generates closures and so on.
I would suggest changing the code to use a class method rather than an inline method - shift your oAllSessions variable and the code you're assigning to AfterSessionComplete out into the body of the class. That will at least establish whether the inline delegate is the cause of the problem.
I'm building a T4 template that will help people construct Azure queues in a consistent and simple manner. I'd like to make this self-documenting, and somewhat consistent.
First I made the queue name at the top of the file, the queue names have to be in lowercase so I added ToLower()
The public constructor uses the built-in StorageClient API's to access the connection strings. I've seen many different approaches to this, and would like to get something that works in almost all situations. (ideas? do share)
I dislike the unneeded HTTP requests to check if the queues have been created so I made is a static bool . I didn't implement a Lock(monitorObject) since I don't think one is needed.
Instead of using a string and parsing it with commas (like most MSDN documentation) I'm serializing the object when passing it into the queue.
For further optimization I'm using a JSON serializer extension method to get the most out of the 8k limit. Not sure if an encoding will help optimize this any more
Added retry logic to handle certain scenarios that occur with the queue (see html link)
Q: Is "DataContext" appropriate name for this class?
Q: Is it a poor practice to name the Queue Action Name in the manner I have done?
What additional changes do you think I should make?
public class AgentQueueDataContext
{
// Queue names must always be in lowercase
// Is named like a const, but isn't one because .ToLower won't compile...
static string AGENT_QUEUE_ACTION_NAME = "AgentQueueActions".ToLower();
static bool QueuesWereCreated { get; set; }
DataModel.SecretDataSource secDataSource = null;
CloudStorageAccount cloudStorageAccount = null;
CloudQueueClient cloudQueueClient = null;
CloudQueue queueAgentQueueActions = null;
static AgentQueueDataContext()
{
QueuesWereCreated = false;
}
public AgentQueueDataContext() : this(false)
{
}
public AgentQueueDataContext(bool CreateQueues)
{
// This pattern of setting up queues is from:
// ttp://convective.wordpress.com/2009/11/15/queues-azure-storage-client-v1-0/
//
this.cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
this.cloudQueueClient = cloudStorageAccount.CreateCloudQueueClient();
this.secDataSource = new DataModel.SecretDataSource();
queueAgentQueueActions = cloudQueueClient.GetQueueReference(AGENT_QUEUE_ACTION_NAME);
if (QueuesWereCreated == false || CreateQueues)
{
queueAgentQueueActions.CreateIfNotExist();
QueuesWereCreated = true;
}
}
// This is the method that will be spawned using ThreadStart
public void CheckQueue()
{
while (true)
{
try
{
CloudQueueMessage msg = queueAgentQueueActions.GetMessage();
bool DoRetryDelayLogic = false;
if (msg != null)
{
// Deserialize using JSON (allows more data to be stored)
AgentQueueEntry actionableMessage = msg.AsString.FromJSONString<AgentQueueEntry>();
switch (actionableMessage.ActionType)
{
case AgentQueueActionEnum.EnrollNew:
{
// Add to
break;
}
case AgentQueueActionEnum.LinkToSite:
{
// Link within Agent itself
// Link within Site
break;
}
case AgentQueueActionEnum.DisableKey:
{
// Disable key in site
// Disable key in AgentTable (update modification time)
break;
}
default:
{
break;
}
}
//
// Only delete the message if the requested agent has been missing for
// at least 10 minutes
//
if (DoRetryDelayLogic)
{
if (msg.InsertionTime != null)
if (msg.InsertionTime < DateTime.UtcNow + new TimeSpan(0, 10, 10))
continue;
// ToDo: Log error: AgentID xxx has not been found in table for xxx minutes.
// It is likely the result of a the registratoin host crashing.
// Data is still consistent. Deleting queued message.
}
//
// If execution made it to this point, then we are either fully processed, or
// there is sufficent reason to discard the message.
//
try
{
queueAgentQueueActions.DeleteMessage(msg);
}
catch (StorageClientException ex)
{
// As of July 2010, this is the best way to detect this class of exception
// Description: ttp://blog.smarx.com/posts/deleting-windows-azure-queue-messages-handling-exceptions
if (ex.ExtendedErrorInformation.ErrorCode == "MessageNotFound")
{
// pop receipt must be invalid
// ignore or log (so we can tune the visibility timeout)
}
else
{
// not the error we were expecting
throw;
}
}
}
else
{
// allow control to fall to the bottom, where the sleep timer is...
}
}
catch (Exception e)
{
// Justification: Thread must not fail.
//Todo: Log this exception
// allow control to fall to the bottom, where the sleep timer is...
// Rationale: not doing so may cause queue thrashing on a specific corrupt entry
}
// todo: Thread.Sleep() is bad
// Replace with something better...
Thread.Sleep(9000);
}
Q: Is "DataContext" appropriate name for this class?
In .NET we have a lot of DataContext classes, so in the sense that you want names to appropriately communicate what the class does, I think XyzQueueDataContext properly communicates what the class does - although you can't query from it.
If you want to stay more aligned to accepted pattern languages, Patterns of Enterprise Application Architecture calls any class that encapsulates access to an external system for a Gateway, while more specifically you may want to use the term Channel in the language of Enterprise Integration Patterns - that's what I would do.
Q: Is it a poor practice to name the Queue Action Name in the manner I have done?
Well, it certainly tightly couples the queue name to the class. This means that if you later decide that you want to decouple those, you can't.
As a general comment I think this class might benefit from trying to do less. Using the queue is not the same thing as managing it, so instead of having all of that queue management code there, I'd suggest injecting a CloudQueue into the instance. Here's how I implement my AzureChannel constructor:
private readonly CloudQueue queue;
public AzureChannel(CloudQueue queue)
{
if (queue == null)
{
throw new ArgumentNullException("queue");
}
this.queue = queue;
}
This better fits the Single Responsibility Principle and you can now implement queue management in its own (reusable) class.