We Keep Coding

Is there a way to avoid using side effects to process this data

I have an application I'm writing that runs script plugins to automate what a user used to have to do manually through a serial terminal. So, I am basically implementing the serial terminal's functionality in code. One of the functions of the terminal was to send a command which kicked off the terminal receiving continuously streamed data from a device until the user pressed space bar, which would then stop the streaming of the data. While the data was streaming, the user would then set some values in another application on some other devices and watch the data streamed in the terminal change.
Now, the streamed data can take different shapes, depending on the particular command that's sent. For instance, one response may look like:
---RESPONSE HEADER---
HERE: 1
ARE: 2 SOME:3
VALUES: 4
---RESPONSE HEADER---
HERE: 5
ARE: 6 SOME:7
VALUES: 8
....
another may look like:
here are some values
in cols and rows
....
So, my idea is to have a different parser based on the command I send. So, I have done the following:
public class Terminal
{
private SerialPort port;
private IResponseHandler pollingResponseHandler;
private object locker = new object();
private List<Response1Clazz> response1;
private List<Response2Clazz> response2;
//setter omited for brevity
//get snapshot of data at any point in time while response is polling.
public List<Response1Clazz> Response1 {get { lock (locker) return new List<Response1Clazz>(response1); }
//setter omited for brevity
public List<Response2Clazz> Response2 {get { lock (locker) return new List<Response1Clazz>(response2); }
public Terminal()
{
port = new SerialPort(){/*initialize data*/}; //open port etc etc
}
void StartResponse1Polling()
{
Response1 = new List<Response1Clazz>();
Parser<List<Response1Clazz>> parser = new KeyValueParser(Response1); //parser is of type T
pollingResponseHandler = new PollingResponseHandler(parser);
//write command to start polling response 1 in a task
}
void StartResponse2Polling()
{
Response2 = new List<Response2Clazz>();
Parser<List<Response2Clazz>> parser = new RowColumnParser(Response2); //parser is of type T
pollingResponseHandler = new PollingResponseHandler(parser); // this accepts a parser of type T
//write command to start polling response 2
}
OnSerialDataReceived(object sender, Args a)
{
lock(locker){
//do some processing yada yada
//we pass in the serial data to the handler, which in turn delegates to the parser.
pollingResponseHandler.Handle(processedSerialData);
}
}
}
the caller of the class would then be something like
public class Plugin : BasePlugin
{
public override void PluginMain()
{
Terminal terminal = new Terminal();
terminal.StartResponse1Polling();
//update some other data;
Response1Clazz response = terminal.Response1;
//process response
//update more data
response = terminal.Response1;
//process response
//terminal1.StopPolling();
}
}
My question is quite general, but I'm wondering if this is the best way to handle the situation. Right now I am required to pass in an object/List that I want modified, and it's modified via a side effect. For some reason this feels a little ugly because there is really no indication in code that this is what is happening. I am purely doing it because the "Start" method is the location that knows which parser to create and which data to update. Maybe this is Kosher, but I figured it is worth asking if there is another/better way. Or at least a better way to indicate that the "Handle" method produces side effects.
Thanks!

I don't see problems in modifying List<>s that are received as a parameter. It isn't the most beautiful thing in the world but it is quite common. Sadly C# doesn't have a const modifier for parameters (compare this with C/C++, where unless you declare a parameter to be const, it is ok for the method to modify it). You only have to give the parameter a self-explaining name (like outputList), and put a comment on the method (you know, an xml-comment block, like /// <param name="outputList">This list will receive...</param>).
To give a more complete response, I would need to see the whole code. You have omitted an example of Parser and an example of Handler.
Instead I see a problem with your lock in { lock (locker) return new List<Response1Clazz>(response1); }. And it seems to be non-sense, considering that you then do Response1 = new List<Response1Clazz>();, but Response1 only has a getter.

Collection stacking ConcurrentQueue with multithreading

Quite a few questions/answers on this topic (only listing a couple that I found. There were many more).
C# Parallel - Adding items to the collection being iterated over, or equivalent?
ConcurrentQueue with multithreading
Thanks to many of them I've come up with what I'm hoping is a possible solution for my problem. I may also be overthinking it. I have an api that needs to write to a text file for logging purposes. Now the api is called N+ times and during each call, it needs to log the request. What I don't want to do is to stop the request from having to wait on the log to be recorded before returning the requested data. Now, the logs cannot just be dropped so it must also stack up on each request if the file is currently in use, using ReaderWriterLock for this. Then when the file isn't locked, I want to write the stacked logs.
I have come up with this in the hopes that it would satisfy the requirements but I think it will still cause a wait.
var wid = WindowsIdentity.GetCurrent().Token;
//add new log items
logs.Enqueue(helpers.createNewLog(requests));
string op;
while (logs.TryDequeue(out op))
{
using (WindowsIdentity.Impersonate(wid))
{
//write to text file, location on shared drive
var wrote = writers.WriteLog(op);
//item cannot be written since file locked, add back to queue to try again
if (!wrote)
{
logs.Enqueue(op);
}
}
}
Logs is a global like so
private static ConcurrentQueue<string> logs = new ConcurrentQueue<string>();
I feel like something isn't right but I'm struggling with what it is and which would be the best way in order for the requirements to be meet and still work in a web farm.

In my opinion, you should use a BlockingCollection instead of the ConcurrentQueue, here is an example of how you can use it as a Producer-Consumer is the same thing you are trying to do.
Now with ASP.Net you can insert modules to intercept every request, if you want to save a log, I suggest you register a module instead of going with your approach. On your Global.asax.cs you have a Register method
public class MvcApplication : System.Web.HttpApplication
{
public static void Register()
{
//registering an HttpModule
HttpApplication.RegisterModule(typeof(LogModule));
}
....
}
public class LogModule: IHttpModule
{
public void Dispose()
{
}
public void Init(HttpApplication context)
{
context.LogRequest += LogEvent;
}
private void LogEvent(object src, EventArgs args)
{
if (HttpContext.Current.CurrentNotification == RequestNotification.LogRequest)
{
if ((MvcHandler)HttpContext.Current.Handler != null)
{
Debug.WriteLine("This was logged!");
//Save the information to your file
}
}
}
}
Hope this helps

Filter Change Notifications in Active Directory: Create, Delete, Undelete

I am currently using the Change Notifications in Active Directory Domain Services in .NET as described in this blog. This will return all events that happen on an selected object (or in the subtree of that object). I now want to filter the list of events for creation and deletion (and maybe undeletion) events.
I would like to tell the ChangeNotifier class to only observe create-/delete-/undelete-events. The other solution is to receive all events and filter them on my side. I know that in case of the deletion of an object, the atribute list that is returned will contain the attribute isDeleted with the value True. But is there a way to see if the event represents the creation of an object? In my tests the value for usnchanged is always usncreated+1 in case of userobjects and both are equal for OUs, but can this be assured in high-frequency ADs? It is also possible to compare the changed and modified timestamp. And how can I tell if an object has been undeleted?
Just for the record, here is the main part of the code from the blog:
public class ChangeNotifier : IDisposable
{
static void Main(string[] args)
{
using (LdapConnection connect = CreateConnection("localhost"))
{
using (ChangeNotifier notifier = new ChangeNotifier(connect))
{
//register some objects for notifications (limit 5)
notifier.Register("dc=dunnry,dc=net", SearchScope.OneLevel);
notifier.Register("cn=testuser1,ou=users,dc=dunnry,dc=net", SearchScope.Base);
notifier.ObjectChanged += new EventHandler<ObjectChangedEventArgs>(notifier_ObjectChanged);
Console.WriteLine("Waiting for changes...");
Console.WriteLine();
Console.ReadLine();
}
}
}
static void notifier_ObjectChanged(object sender, ObjectChangedEventArgs e)
{
Console.WriteLine(e.Result.DistinguishedName);
foreach (string attrib in e.Result.Attributes.AttributeNames)
{
foreach (var item in e.Result.Attributes[attrib].GetValues(typeof(string)))
{
Console.WriteLine("\t{0}: {1}", attrib, item);
}
}
Console.WriteLine();
Console.WriteLine("====================");
Console.WriteLine();
}
LdapConnection _connection;
HashSet<IAsyncResult> _results = new HashSet<IAsyncResult>();
public ChangeNotifier(LdapConnection connection)
{
_connection = connection;
_connection.AutoBind = true;
}
public void Register(string dn, SearchScope scope)
{
SearchRequest request = new SearchRequest(
dn, //root the search here
"(objectClass=*)", //very inclusive
scope, //any scope works
null //we are interested in all attributes
);
//register our search
request.Controls.Add(new DirectoryNotificationControl());
//we will send this async and register our callback
//note how we would like to have partial results
IAsyncResult result = _connection.BeginSendRequest(
request,
TimeSpan.FromDays(1), //set timeout to a day...
PartialResultProcessing.ReturnPartialResultsAndNotifyCallback,
Notify,
request
);
//store the hash for disposal later
_results.Add(result);
}
private void Notify(IAsyncResult result)
{
//since our search is long running, we don't want to use EndSendRequest
PartialResultsCollection prc = _connection.GetPartialResults(result);
foreach (SearchResultEntry entry in prc)
{
OnObjectChanged(new ObjectChangedEventArgs(entry));
}
}
private void OnObjectChanged(ObjectChangedEventArgs args)
{
if (ObjectChanged != null)
{
ObjectChanged(this, args);
}
}
public event EventHandler<ObjectChangedEventArgs> ObjectChanged;
#region IDisposable Members
public void Dispose()
{
foreach (var result in _results)
{
//end each async search
_connection.Abort(result);
}
}
#endregion
}
public class ObjectChangedEventArgs : EventArgs
{
public ObjectChangedEventArgs(SearchResultEntry entry)
{
Result = entry;
}
public SearchResultEntry Result { get; set; }
}

I participated in a design review about five years back on a project that started out using AD change notification. Very similar questions to yours were asked. I can share what I remember, and don't think things have change much since then. We ended up switching to DirSync.
It didn't seem possible to get just creates & deletes from AD change notifications. We found change notification resulted enough events monitoring a large directory that notification processing could bottleneck and fall behind. This API is not designed for scale, but as I recall the performance/latency were not the primary reason we switched.
Yes, the usn relationship for new objects generally holds, although I think there are multi-dc scenarios where you can get usncreated == usnchanged for a new user, but we didn't test that extensively, because...
The important thing for us was that change notification only gives you reliable object creation detection under the unrealistic assumption that your machine is up 100% of the time! In production systems there are always some case where you need to reboot and catch up or re-synchronize, and we switched to DirSync because it has a robust way to handle those scenarios.
In our case it could block email to a new user for an indeterminate time if an object create were missed. That obviously wouldn't be good, we needed to be sure. For AD change notifications, getting that resync right that would have some more work and hard to test. But for DirSync, its more natural, and there's a fast-path resume mechanism that usually avoids resync. For safety I think we triggered a full re-synchronize every day.
DirSync is not as real-time as change notification, but its possible to get ~30-second average latency by issuing the DirSync query once a minute.

Awesomnium Post Parameters

currently i am working with Awsomnium 1.7 in the C# environment.
I'm just using the Core and trying to define custom post parameters.
Now, i googled a lot and i even posted at the awsomnium forums, but there was no answer.
I understand the concept, but the recent changes just dropped the suggested mechanic and examples.
What i found:
http://support.awesomium.com/kb/general-use/how-do-i-send-form-values-post-data
The problem with this is, that the WebView Class does not contain "OnResourceRequest" Event anymore.
So far, i have implemented the IResourceInterceptor and have the "OnRequest"-Function overwritten
public ResourceResponse OnRequest(ResourceRequest request)
is the signature, but i have no chance to reach in there in order to add request headers.
Anyone here any idea? I tried to look in the documentation, but i didn't find anything on that.....

You need to attach your IResourceInterceptor to WebCore, not WebView. Here's a working example:
Resource interceptor:
public class CustomResourceInterceptor : ResourceInterceptor
{
protected override ResourceResponse OnRequest(ResourceRequest request)
{
request.Method = "POST";
var bytes = "Appending some text to the request";
request.AppendUploadBytes(bytes, (uint) bytes.Length);
request.AppendExtraHeader("custom-header", "this is a custom header");
return null;
}
}
Main application:
public MainWindow()
{
WebCore.Started += WebCoreOnStarted;
InitializeComponent();
}
private void WebCoreOnStarted(object sender, CoreStartEventArgs coreStartEventArgs)
{
var interceptor = new CustomResourceInterceptor();
WebCore.ResourceInterceptor = interceptor;
//webView is a WebControl on my UI, but you should be able to create your own WebView off WebCore
webView.Source = new Uri("http://www.google.com");
}

HotN's answer above is good; in fact, it's what I based my answer on. However, I spent a week searching for this information and putting together something that will work. (The answer above has a couple of issues which, at the very least, make it unworkable with v1.7 of Awesomium.) What I was looking for was something that would work right out of the box.
And here is that solution. It needs improvement, but it suits my needs at the moment. I hope this helps someone else.
// CRI.CustomResourceInterceptor
//
// Author: Garison E Piatt
// Contact: {removed}
// Created: 11/17/14
// Version: 1.0.0
//
// Apparently, when Awesomium was first created, the programmers did not understand that someone would
// eventually want to post data from the application. So they made it incredibly difficult to upload
// POST parameters to the remote web site. We have to jump through hoops to get that done.
//
// This module provides that hoop-jumping in a simple-to-understand fashion. We hope. It overrides
// the current resource interceptor (if any), replacing both the OnRequest and OnFilterNavigation
// methods (we aren't using the latter yet).
//
// It also provides settable parameters. Once this module is attached to the WebCore, it is *always*
// attached; therefore, we can simply change the parameters before posting to the web site.
//
// File uploads are currently unhandled, and, once handled, will probably only upload one file. We
// will deal with that issue later.
//
// To incoroprate this into your application, follow these steps:
// 1. Add this file to your project. You know how to do that.
// 2. Edit your MainWindow.cs file.
// a. At the top, add:
// using CRI;
// b. inside the main class declaration, near the top, add:
// private CustomResourceInterceptor cri;
// c. In the MainWindow method, add:
// WebCore.Started += OnWebCoreOnStarted;
// cri = new CustomResourceInterceptor();
// and (set *before* you set the Source value for the Web Control):
// cri.Enabled = true;
// cri.Parameters = String.Format("login={0}&password={1}", login, pw);
// (Choose your own parameters, but format them like a GET query.)
// d. Add the following method:
// private void OnWebCoreOnStarted(object sender, CoreStartEventArgs coreStartEventArgs) {
// WebCore.ResourceInterceptor = cri;
// }
// 3. Compile your application. It should work.
using System;
using System.Runtime.InteropServices;
using System.Text;
using Awesomium.Core;
using Awesomium.Windows.Controls;
namespace CRI {
//* CustomResourceInterceptor
// This object replaces the standard Resource Interceptor (if any; we still don't know) with something
// that allows posting data to the remote web site. It overrides both the OnRequest and OnFilterNavigation
// methods. Public variables allow for run-time configuration.
public class CustomResourceInterceptor : IResourceInterceptor {
// Since the default interceptor remains overridden for the remainder of the session, we need to disable
// the methods herein unless we are actually using them. Note that both methods are disabled by default.
public bool RequestEnabled = false;
public bool FilterEnabled = false;
// These are the parameters we send to the remote site. They are empty by default; another safeguard
// against sending POST data unnecessarily. Currently, both values allow for only one string. POST
// variables can be combined (by the caller) into one string, but this limits us to only one file
// upload at a time. Someday, we will have to fix that. And make it backward-compatible.
public String Parameters = null;
public String FilePath = null;
/** OnRequest
** This ovverrides the default OnRequest method of the standard resource interceptor. It receives
** the resource request object as a parameter.
**
** It first checks whether or not it is enabled, and returns NULL if not. Next it sees if any
** parameters are defined. If so, it converst them to a byte stream and appends them to the request.
** Currently, files are not handled, but we hope to add that someday.
*/
public ResourceResponse OnRequest(ResourceRequest request) {
// We do nothing at all if we aren't enabled. This is a stopgap that prevents us from sending
// POST data with every request.
if (RequestEnabled == false) return null;
// If the Parameters are defined, convert them to a byte stream and append them to the request.
if (Parameters != null) {
var str = Encoding.Default.GetBytes(Parameters);
var bytes = Encoding.UTF8.GetString(str);
request.AppendUploadBytes(bytes, (uint)bytes.Length);
}
// If either the parameters or file path are defined, this is a POST request. Someday, we'll
// figure out how to get Awesomium to understand Multipart Form data.
if (Parameters != null || FilePath != null) {
request.Method = "POST";
request.AppendExtraHeader("Content-Type", "application/x-www-form-urlencoded"); //"multipart/form-data");
}
// Once the data has been appended to the page request, we need to disable this process. Otherwise,
// it will keep adding the data to every request, including those that come from the web site.
RequestEnabled = false;
Parameters = null;
FilePath = null;
return null;
}
/** OnFilterNavigation
** Not currently used, but needed to keep VisualStudio happy.
*/
public bool OnFilterNavigation(NavigationRequest request) {
return false;
}
}
}

Adding a Http-Header in a HttpModule and read it out from a Page

I've tried to write my own HttpModule (IHttpModule) that adds a Header like that:
public class MyModule: IHttpModule
{
public void Init(HttpApplication c)
{
c.BeginRequest += delegate{c.Response.AddHeader("MyHeader", "MyValue");};
}
public void Dispose(){}
}
and tried to read in a aspx page like that:
var x = Request.ServerVariables["MyHeader"];
but it didn't work. Any idea why?

You're adding something to the headers that will be sent from the server to the client and trying to read it from the headers already received by the server from the client. These are two completely different collections.
If you are using this to communicate between the module and the page, you may find it preferable to add something to the HttpContext.Items, this allows for all sorts of objects to be passed, and doesn't pollute the headers with stuff that aren't relevant there, nor require sessions, so it is a good way to communicate between code operating on the same request.

add it like this , use event "EndRequest"
void application_EndRequest(object sender, EventArgs e)
{
HttpResponse response = HttpContext.Current.Response;
response.AddHeader("Author", "Sam Lin");
}