Quite a few questions/answers on this topic (only listing a couple that I found. There were many more).
C# Parallel - Adding items to the collection being iterated over, or equivalent?
ConcurrentQueue with multithreading
Thanks to many of them I've come up with what I'm hoping is a possible solution for my problem. I may also be overthinking it. I have an api that needs to write to a text file for logging purposes. Now the api is called N+ times and during each call, it needs to log the request. What I don't want to do is to stop the request from having to wait on the log to be recorded before returning the requested data. Now, the logs cannot just be dropped so it must also stack up on each request if the file is currently in use, using ReaderWriterLock for this. Then when the file isn't locked, I want to write the stacked logs.
I have come up with this in the hopes that it would satisfy the requirements but I think it will still cause a wait.
var wid = WindowsIdentity.GetCurrent().Token;
//add new log items
logs.Enqueue(helpers.createNewLog(requests));
string op;
while (logs.TryDequeue(out op))
{
using (WindowsIdentity.Impersonate(wid))
{
//write to text file, location on shared drive
var wrote = writers.WriteLog(op);
//item cannot be written since file locked, add back to queue to try again
if (!wrote)
{
logs.Enqueue(op);
}
}
}
Logs is a global like so
private static ConcurrentQueue<string> logs = new ConcurrentQueue<string>();
I feel like something isn't right but I'm struggling with what it is and which would be the best way in order for the requirements to be meet and still work in a web farm.
In my opinion, you should use a BlockingCollection instead of the ConcurrentQueue, here is an example of how you can use it as a Producer-Consumer is the same thing you are trying to do.
Now with ASP.Net you can insert modules to intercept every request, if you want to save a log, I suggest you register a module instead of going with your approach. On your Global.asax.cs you have a Register method
public class MvcApplication : System.Web.HttpApplication
{
public static void Register()
{
//registering an HttpModule
HttpApplication.RegisterModule(typeof(LogModule));
}
....
}
public class LogModule: IHttpModule
{
public void Dispose()
{
}
public void Init(HttpApplication context)
{
context.LogRequest += LogEvent;
}
private void LogEvent(object src, EventArgs args)
{
if (HttpContext.Current.CurrentNotification == RequestNotification.LogRequest)
{
if ((MvcHandler)HttpContext.Current.Handler != null)
{
Debug.WriteLine("This was logged!");
//Save the information to your file
}
}
}
}
Hope this helps
Related
I just worked my way through this MS Learn Tutorial regarding SignalR in Blazor.
At the end of the tutorial, you get a program that can have multiple clients hooked up to a "ChatHub" to send and receive messages, like a "Townsquare-Chatroom"
While testing I realized, that if you send some messages and afterward create a new client, the new client does not display the previously send messages. This is because every client stores its received messages locally as shown here:
#code{
// ...
private List<string> messages = new();
// ...
}
I decided to implement such a feature.
To do so, I created ChatLog.cs which is supposed to log the messages for all clients instead of saving them inside of each individual client:
public class ChatLog
{
private List<string> _messages= new List<string>();
public List<string> Messages
{
get { return _messages; }
set
{
_messages = value;
}
}
}
Of course, I also had to make some changes inside of index.razor to make things work:
I added a new service in program.cs as singleton
==> Program.cs
// ...
builder.Services.AddSingleton<ChatLog>();
// ...
and injected ChatLog into my index.razor
==> Index.razor
// ...
#inject ChatLog ChatLogger
// ...
I changed the code in index.razor #code to add the messages to ChatLog.Messages instead of the "local" messages-List
protected override async Task OnInitializedAsync()
{
// Change
if(ChatLogger.Messages is null)
{
ChatLogger.Messages = new();
}
hubConnection = new HubConnectionBuilder()
.WithUrl(NavManager.ToAbsoluteUri("/chathub"))
.WithAutomaticReconnect()
.Build();
hubConnection.On<string, string>("ReceiveMessage", (user, message) =>
{
var formattedMessage = $"{user}: {message}";
// Change
ChatLogger.Messages.Add(formattedMessage);
InvokeAsync(StateHasChanged);
});
await hubConnection.StartAsync();
}
Now I run into a new problem.
Since the event
hubConnection.On<string, string>...
is called by every client, and all new messages get added into ChatLog.Messages X-times (x == amount of active clients).
I just can't think of a way to avoid this problem and only log every message exactly once.
Can someone help me?
Thanks in advance and sorry for the long explanation. Maybe someone can also help shorten it?
EDIT
To clarify the problem: Since the messages get added to the messages List inside of the event (as shown above), every instance (or every tab of the website) adds the message, resulting in multiple (and unwanted) adds.
E.g.
Two clients
Message "Hello" was sent once but added twice
Message "Ciao" was sent twice but added four times
From what I can gather this is more a learning exercise than something you're actually planning on using in a production environment, so we can ignore the fact that this isn't really a very robust implementation.
In any case, a simply solution would be to have the sender of the message store it in the messagelog, instead of storing it upon reception.
Taking from the tutorial you followed:
using Microsoft.AspNetCore.SignalR;
namespace BlazorServerSignalRApp.Server.Hubs
{
public class ChatHub : Hub
{
public async Task SendMessage(string user, string message)
{
// STORE YOUR MESSAGE IN YOUR MESSAGE LOG HERE
await Clients.All.SendAsync("ReceiveMessage", user, message);
}
}
}
You should be able to inject your MessageLog service into the ChatHub in order to access it from there. (If I'm understanding your project structure correctly)
I am currently using the Change Notifications in Active Directory Domain Services in .NET as described in this blog. This will return all events that happen on an selected object (or in the subtree of that object). I now want to filter the list of events for creation and deletion (and maybe undeletion) events.
I would like to tell the ChangeNotifier class to only observe create-/delete-/undelete-events. The other solution is to receive all events and filter them on my side. I know that in case of the deletion of an object, the atribute list that is returned will contain the attribute isDeleted with the value True. But is there a way to see if the event represents the creation of an object? In my tests the value for usnchanged is always usncreated+1 in case of userobjects and both are equal for OUs, but can this be assured in high-frequency ADs? It is also possible to compare the changed and modified timestamp. And how can I tell if an object has been undeleted?
Just for the record, here is the main part of the code from the blog:
public class ChangeNotifier : IDisposable
{
static void Main(string[] args)
{
using (LdapConnection connect = CreateConnection("localhost"))
{
using (ChangeNotifier notifier = new ChangeNotifier(connect))
{
//register some objects for notifications (limit 5)
notifier.Register("dc=dunnry,dc=net", SearchScope.OneLevel);
notifier.Register("cn=testuser1,ou=users,dc=dunnry,dc=net", SearchScope.Base);
notifier.ObjectChanged += new EventHandler<ObjectChangedEventArgs>(notifier_ObjectChanged);
Console.WriteLine("Waiting for changes...");
Console.WriteLine();
Console.ReadLine();
}
}
}
static void notifier_ObjectChanged(object sender, ObjectChangedEventArgs e)
{
Console.WriteLine(e.Result.DistinguishedName);
foreach (string attrib in e.Result.Attributes.AttributeNames)
{
foreach (var item in e.Result.Attributes[attrib].GetValues(typeof(string)))
{
Console.WriteLine("\t{0}: {1}", attrib, item);
}
}
Console.WriteLine();
Console.WriteLine("====================");
Console.WriteLine();
}
LdapConnection _connection;
HashSet<IAsyncResult> _results = new HashSet<IAsyncResult>();
public ChangeNotifier(LdapConnection connection)
{
_connection = connection;
_connection.AutoBind = true;
}
public void Register(string dn, SearchScope scope)
{
SearchRequest request = new SearchRequest(
dn, //root the search here
"(objectClass=*)", //very inclusive
scope, //any scope works
null //we are interested in all attributes
);
//register our search
request.Controls.Add(new DirectoryNotificationControl());
//we will send this async and register our callback
//note how we would like to have partial results
IAsyncResult result = _connection.BeginSendRequest(
request,
TimeSpan.FromDays(1), //set timeout to a day...
PartialResultProcessing.ReturnPartialResultsAndNotifyCallback,
Notify,
request
);
//store the hash for disposal later
_results.Add(result);
}
private void Notify(IAsyncResult result)
{
//since our search is long running, we don't want to use EndSendRequest
PartialResultsCollection prc = _connection.GetPartialResults(result);
foreach (SearchResultEntry entry in prc)
{
OnObjectChanged(new ObjectChangedEventArgs(entry));
}
}
private void OnObjectChanged(ObjectChangedEventArgs args)
{
if (ObjectChanged != null)
{
ObjectChanged(this, args);
}
}
public event EventHandler<ObjectChangedEventArgs> ObjectChanged;
#region IDisposable Members
public void Dispose()
{
foreach (var result in _results)
{
//end each async search
_connection.Abort(result);
}
}
#endregion
}
public class ObjectChangedEventArgs : EventArgs
{
public ObjectChangedEventArgs(SearchResultEntry entry)
{
Result = entry;
}
public SearchResultEntry Result { get; set; }
}
I participated in a design review about five years back on a project that started out using AD change notification. Very similar questions to yours were asked. I can share what I remember, and don't think things have change much since then. We ended up switching to DirSync.
It didn't seem possible to get just creates & deletes from AD change notifications. We found change notification resulted enough events monitoring a large directory that notification processing could bottleneck and fall behind. This API is not designed for scale, but as I recall the performance/latency were not the primary reason we switched.
Yes, the usn relationship for new objects generally holds, although I think there are multi-dc scenarios where you can get usncreated == usnchanged for a new user, but we didn't test that extensively, because...
The important thing for us was that change notification only gives you reliable object creation detection under the unrealistic assumption that your machine is up 100% of the time! In production systems there are always some case where you need to reboot and catch up or re-synchronize, and we switched to DirSync because it has a robust way to handle those scenarios.
In our case it could block email to a new user for an indeterminate time if an object create were missed. That obviously wouldn't be good, we needed to be sure. For AD change notifications, getting that resync right that would have some more work and hard to test. But for DirSync, its more natural, and there's a fast-path resume mechanism that usually avoids resync. For safety I think we triggered a full re-synchronize every day.
DirSync is not as real-time as change notification, but its possible to get ~30-second average latency by issuing the DirSync query once a minute.
I'm trying to expose an observable sequence that gives observers all existing records in a database table plus any future items. For the sake of argument, lets say it's log entries. Therefore, I'd have something like this:
public class LogService
{
private readonly Subject<LogEntry> entries;
public LogService()
{
this.entries = new Subject<LogEntry>();
this.entries
.Buffer(...)
.Subscribe(async x => WriteLogEntriesToDatabaseAsync(x));
}
public IObservable<LogEntry> Entries
{
get { return this.entries; }
}
public IObservable<LogEntry> AllLogEntries
{
get
{
// how the heck?
}
}
public void Log(string message)
{
this.entries.OnNext(new LogEntry(message));
}
private async Task<IEnumerable<LogEntry>> GetLogEntriesAsync()
{
// reads existing entries from DB table and returns them
}
private async Task WriteLogEntriesToDatabaseAsync(IList<LogEntry> entries)
{
// writes entries to the database
}
}
My initial thought for the implementation of AllLogEntries was something like this:
return Observable.Create<LogEntry>(
async observer =>
{
var existingEntries = await this.GetLogEntriesAsync();
foreach (var existingEntry in existingEntries)
{
observer.OnNext(existingEntry);
}
return this.entries.Subscribe(observer);
});
But the problem with this is that there could log entries that have been buffered and not yet written to the database. Hence, those entries will be missed because they are not in the database and have already passed through the entries observable.
My next thought was to separate the buffered entries from the non-buffered and use the buffered when implementing AllLogEntries:
return Observable.Create<LogEntry>(
async observer =>
{
var existingEntries = await this.GetLogEntriesAsync();
foreach (var existingEntry in existingEntries)
{
observer.OnNext(existingEntry);
}
return this.bufferedEntries
.SelectMany(x => x)
.Subscribe(observer);
});
There are two problems with this:
It means clients of AllLogEntries also have to wait for the buffer timespan to pass before they receive their log entries. I want them to see log entries instantaneously.
There is still a race condition in that log entries could be written to the database between the point at which I finish reading the existing ones and the point at which I return the future entries.
So my question is: how would I actually go about achieving my requirements here with no possibility of race conditions, and avoiding any major performance penalties?
To do this via the client code, you will probably have to implement a solution using polling and then look for differences between calls. Possibly combining a solution with
Observable.Interval() : http://rxwiki.wikidot.com/101samples#toc28 , and
Observable.DistinctUntilChanged()
will give you sufficient solution.
Alternatively, I'd suggest you try to find a solution where the clients are notified when the DB/table is updated. In a web application, you could use something like SignalR to do this.
For example: http://techbrij.com/database-change-notifications-asp-net-signalr-sqldependency
If its not a web-application, a similar update mechanism via sockets may work.
See these links (these came from the accepted answer of SignalR polling database for updates):
http://xsockets.net/api/net-c#snippet61
https://github.com/codeplanner/XSocketsPollingLegacyDB
I am new to developing in .NET and C#, but have been a long-time developer, working with C, C++, Java, PHP, etc.
I have an MVC3 extension class for my data models that refers to the database. It is set as "private static" in the class, but I think that it is not keeping up with database changes. In other words, when I change data in the controllers, those changes aren't "noticed" in the db because it is static. Currently, I am creating and disposing of the variable for each use, to compensate.
My questions are:
Am I correct that a static db variable could behave that way?
Is it necessary to dispose of the dynamic variable in the static class, or will garbage collection still take care of it automatically?
Here is a relevant snippet of the class:
namespace PBA.Models {
using System;
using System.Text.RegularExpressions;
using PBA.Models;
using PBA.Controllers;
public static class Extensions {
private static PbaDbEntities db = null;
public static PbaDbEntities GetDb() {
// TODO: find out about static memory/disposal, etc.
//
if (db != null) {
db.Dispose();
}
db = new PbaDbEntities();
return db;
}
public static string GetCheckpointState(this Activity activity, long memberProjectId) {
GetDb(); // TODO: Do I need to do this each time, or will a one-time setting work?
string state = CheckpointController.CHECKPOINT_STATUS_NOT_STARTED;
try {
var sub = db.ActivitySubmissions.
Where(s => s.activityId == activity.activityId).
Where(s => s.memberProjectId == memberProjectId).
OrderByDescending(s => s.submitted).
First();
if (sub != null) {
state = sub.checkpointStatusId;
}
}
catch (Exception e) {
// omitted for brevity
}
return state;
}
}
}
Your code will fail horribly in production.
DataContexts are not thread-safe; you must not share a context between requests.
Never put mutable objects in static fields in multi-threaded applications.
Ignoring exceptions that way is a terrible idea, if you don't want to handle exceptions just don't try/catch, or catch & rethrow. Think about it like this, after you've buried the exception, your program is in an invalid state, b/c something you have no control over error'd out. Now, b/c you've buried the exception, your program can continue to operate but it's in a bad state.
If your code makes it to production, 3.5 yrs from now some jr. programmer is going to get involved in some middle of the night firestorm because all of a sudden the website is broken, even though it used to work. It will be completely impossible to track down where the exception is happening so, this poor guy is going to spend 48 straight hours adding logging code all over the place to track down the problem. He will find that some DBA somewhere decided to rename the column MemberProjectId to MemberProjectIdentifier, which caused your linq to blow up.
Think of the children, handle exceptions, don't bury them.
btw - yes, i have been that guy that has to figure out these types of mistakes.
It seems like you need to read about mvc3 and entity framework before writing coding and asking in here for help on something that's coded full of bad practices.
Answering your questions:
1- no
2- makes no sense as the answer to 1
Do it right, here are some useful documentation: http://msdn.microsoft.com/en-us/library/ie/gg416514(v=vs.98).aspx
EDIT: Adding some explicit fix
You could access your dbcontext from an static class, something like this:
var context = DbProvider.CurrentDb;
The idea is to access your db from here always: from your extension methods and from your controller actions.
Then, the implementation of the DbProvider.CurrentDb will be something like this:
public static classDbProvider {
public static void Initialize(){
HttpContext.Current.ApplicationInstance.BeginRequest += CreateDb;
HttpConetxt.Current.ApplicationInstance.EndRequest += DisposeDb;
}
private static void CreateDb(object sender, EventArgs e) {
HttpContext.Items.Add("CurrentDb", new PbaDbEntities(););
}
private static void DisposeDb(object sender, EventArgs e)
{
Current.Dispose();
HttpContext.Items.Remove("CurrentDb");
}
public static PbaDbEntities CurrentDb{
get {
return (PbaDbEntities)HttpContext.Current.Items["CurrentDb"];
}
}
}
As you can see, it will create a new Db per each request and it will be available ONLY in that request. In that way, your db will be disposed at the end of each request. This pattern is called Open-Session-in-View.
Finally, you need to initialize the DbProvider calling the method
Initialize() in your Global.asax file, in the event Application_start.
Hope it helps.
I don't have any idea of the context here-- if db is simply a connection-like object or not, but it appears you are throwing away and recreating whatever it is unnecessarily.
Best to create a property (for whatever your doing) so to be consistent.
private static Thing _thing;
private static Thing thing{
get{
if(_thing==null){
_thing=new Thing();
}
return _thing;
}
}
I've tried to write my own HttpModule (IHttpModule) that adds a Header like that:
public class MyModule: IHttpModule
{
public void Init(HttpApplication c)
{
c.BeginRequest += delegate{c.Response.AddHeader("MyHeader", "MyValue");};
}
public void Dispose(){}
}
and tried to read in a aspx page like that:
var x = Request.ServerVariables["MyHeader"];
but it didn't work. Any idea why?
You're adding something to the headers that will be sent from the server to the client and trying to read it from the headers already received by the server from the client. These are two completely different collections.
If you are using this to communicate between the module and the page, you may find it preferable to add something to the HttpContext.Items, this allows for all sorts of objects to be passed, and doesn't pollute the headers with stuff that aren't relevant there, nor require sessions, so it is a good way to communicate between code operating on the same request.
add it like this , use event "EndRequest"
void application_EndRequest(object sender, EventArgs e)
{
HttpResponse response = HttpContext.Current.Response;
response.AddHeader("Author", "Sam Lin");
}