We Keep Coding

Realm how to close and read

I am having an issue where I am encountering the following error: Realms.Exceptions.RealmClosedException
when I try to read my list of realm objects after I retrieved them. The closing of realm is my desired behavior as in my logs I saw ~2K instances of crashes reporting Realm OOM exceptions so I decided to experiment with wrapping my call to realm in a using statement as so:
List<RealmDomain.User> users;
using(var realm = Realm.GetInstance(config))
{
users = realm.All<RealmDomain.User>().ToList();
}
return users;
I then try to work with this data as follows (and that is when the exception is raised)
allUsers.FirstOrDefault(x => x.PortalUserId == id.ToString());. allUsers in this case is the variable holding the data returned by the last block of code. Thus I am wondering what is the correct way to properly handle disposing of realm in order to ensure we don't run into OOM exceptions with it, and how to properly read the data from said source even after it has been closed?
Edit: The block which returns the users is inside of my UserRepository, the application implements the UnitOfWork pattern (to access our realm db) which is then accessed via DI.
Edit2: Another follow up question would be: should I only wrap my calls to realm in a Using statement if it's only a CUD operation (Create, Update, Delete) whilst reads should not be wrapped in that and let the GC handle disposing of the realm instance as needed later?

I think that actually there are multiple points in your question.
Disposing
The way that you dispose your realm really depends on whether you are on the main thread or on a background thread.
If you are on a main thread then you should not dispose your realm, and it would make sense to either call Realm.GetInstance() when you need it, or to initialize the realm as a singleton, like in the following snippet:
public class MyViewModel
{
private readonly Realm realm;
public MyViewModel()
{
this.realm = Realm.GetInstance(); //or this.realm = RealmProvider.Singleton
}
}
If you are on a background thread then you definitely need to dispose of the realm. In this case the using statement is probably the easiest thing to do:
public async Task OnBackgroundThread()
{
using(var realm = Realm.GetInstance()) //From C#8 you don't even need the braces
{
//do work with realm
}
}
You need to dispose of realm on background threads otherwise you will cause the increase of the size of the realm on the disk. You can read more about why in this answer.
ToList()
Using ToList() on realm.All() is probably not a good idea, as you will load all the objects in memory losing the "laziness" of access. If what you need to do is find a certain object you can use a query like:
realm.All<User>().Where(x => x.PortalUserId == id.ToString()).First();
Repository
Realm does not really work well with the repository pattern. If you really want to use the repository pattern, then you will lose some of the advantages of working with realm, like the fact that the objects and collections are live and auto-updating. Besides, this is all made even more complicated by the use of background threads because of what I have said before.

HttpContext.Current.Session null after making method async

I had a method like below
book.Bindbook();
I made it async as follow
new Task(book.Bindbook).Start();
Now this method uses HttpContext.Current.Session which is now returning null.
Here is code that returns null
public static Bookmanager CartManager
{
//Gets the value from the session variable.
get
{
try
{
if (HttpContext.Current.Session["BookData"] == null)
{
Bookmanager bookmgr= new Bookmanager ();
Book book = new Book(SessionManager.CurrentUser);
bookmgr.SetCurrentCart(book);
HttpContext.Current.Session["BookData"] = bookmgr;
}
else if (((Bookmanager)HttpContext.Current.Session["BookData"]).GetCurrentCart() == null)
{
Book book = new Book(SessionManager.CurrentUser);
((Bookmanager)HttpContext.Current.Session["BookData"]).SetCurrentCart(book);
}
}
catch(Exception ex)
{
//throw ex;
}
return ((Bookmanager)HttpContext.Current.Session["BookData"]);
}
//Sets the value of the session variable.
set
{
HttpContext.Current.Session["BookData"] = value;
}
}

There's a lot of potential problems with your solution which have lead to this problem. I'll try to break it down into pieces to explain what's going on.
new Task(book.Bindbook).Start() doesn't always run where you think it does
This method of creating an asynchronous operation is subtly dangerous as it's not easy to know how the task will be executed. When you call this constructor, the Task will capture the TaskScheduler.Current value as the mechanism it will use to schedule it's own execution. This means that your task's execution is invisibly tied to the context it's in.
Typically, you want to use Task.Run(Action) instead of creating a new Task instance and then calling Start, as this always runs on the value of TaskScheduler.Default, which is usually the .NET thread pool and is generally what you want to do when running a background task.
HttpContext is not thread-safe
The HttpContext class was never intended to be called from multiple threads safely. It's Current value tied to the thread which is processing the request and is not available on other threads. You should not pass it to other threads. Generally-speaking you should reduce the surface-area of HttpContext in your applications to a bare minimum. It's nearly impossible to mock for testing purposes and has several subtle limitations (such as you are finding) which make it challenging to work with.
Instead, surface the Current value as early as possible in your code and keep a reference to the objects you actually need to work with (like the session).
Static properties are usually harmful
Having a static property on an object either means that there are exactly one of these things for the entirety of the AppDomain (such as TaskScheduler.Default) where they represent some cross-cutting concern that can be configured, or that there is some hidden context manipulating the value behind the scenes. The former case is rare, but can be acceptable in some cases, but the second is pretty harmful. HttpContext.Current is an example of a value that should not be static (and future version of ASP.NET do away with it entirely). It makes code hard to reason about, nearly impossible to test and introduces subtle bugs (like this one) which can't easily be dealt with.
Fundamentally, this is the biggest problem here and the root cause of your pain. If this property were exposed as an instance property and the instance was scoped to the request context, you would have none of your issues. Once you're working with an object whose lifetime is the same as your request, all your critical state becomes local and easy to reason about.

Use ConfigureAwait(true) to allow to continue on the original context.
var task = new Task(() => book.Bindbook()).ConfigureAwait(true);
task.Start();

HttpContext is bound to thread, that's why it is null.
I think better solution will be to pass all needed data through parameters to other thread and not sharing HttpContext.

Why are Subjects not recommended in .NET Reactive Extensions?

I am currently getting to grips with the Reactive Extensions framework for .NET and I am working my way through the various introduction resources I've found (mainly http://www.introtorx.com)
Our application involves a number of hardware interfaces that detect network frames, these will be my IObservables, I then have a variety of components that will consume those frames or perform some manner of transform on the data and produce a new type of frame. There will also be other components that need to display every n'th frame for example.
I am convinced that Rx is going to be useful for our application, however I am struggling with the implementation details for the IObserver interface.
Most (if not all) of the resources I have been reading have said that I should not implement the IObservable interface myself but use one of the provided functions or classes.
From my research it appears that creating a Subject<IBaseFrame> would provide me what I need, I would have my single thread that reads data from the hardware interface and then calls the OnNext function of my Subject<IBaseFrame> instance. The different IObserver components would then receive their notifications from that Subject.
My confusion is coming from the advice give in the appendix of this tutorial where it says:
Avoid the use of the subject types. Rx is effectively a functional programming paradigm. Using subjects means we are now managing state, which is potentially mutating. Dealing with both mutating state and asynchronous programming at the same time is very hard to get right. Furthermore, many of the operators (extension methods) have been carefully written to ensure that correct and consistent lifetime of subscriptions and sequences is maintained; when you introduce subjects, you can break this. Future releases may also see significant performance degradation if you explicitly use subjects.
My application is quite performance critical, I am obviously going to test the performance of using the Rx patterns before it goes in to production code; however I am worried that I am doing something that is against the spirit of the Rx framework by using the Subject class and that a future version of the framework is going to hurt performance.
Is there a better way of doing what I want? The hardware polling thread is going to be running continuously whether there are any observers or not (the HW buffer will back up otherwise), so this is a very hot sequence. I need to then pass the received frames out to multiple observers.
Any advice would be greatly appreciated.

Ok,
If we ignore my dogmatic ways and ignore "subjects are good/bad" all together. Let us look at the problem space.
I bet you either have 1 of 2 styles of system you need to ingrate to.
The system raises an event or a call back when a message arrives
You need to poll the system to see if there are any message to process
For option 1, easy, we just wrap it with the appropriate FromEvent method and we are done. To the Pub!
For option 2, we now need to consider how we poll this and how to do this effciently. Also when we get the value, how do we publish it?
I would imagine that you would want a dedicated thread for polling. You wouldn't want some other coder hammering the ThreadPool/TaskPool and leaving you in a ThreadPool starvation situation. Alternatively you don't want the hassle of context switching (I guess). So assume we have our own thread, we will probably have some sort of While/Sleep loop that we sit in to poll. When the check finds some messages we publish them. Well all of this sounds perfect for Observable.Create. Now we probably cant use a While loop as that wont allow us to ever return a Disposable to allow cancellation. Luckily you have read the whole book so are savvy with Recursive scheduling!
I imagine something like this could work. #NotTested
public class MessageListener
{
private readonly IObservable<IMessage> _messages;
private readonly IScheduler _scheduler;
public MessageListener()
{
_scheduler = new EventLoopScheduler();
var messages = ListenToMessages()
.SubscribeOn(_scheduler)
.Publish();
_messages = messages;
messages.Connect();
}
public IObservable<IMessage> Messages
{
get {return _messages;}
}
private IObservable<IMessage> ListenToMessages()
{
return Observable.Create<IMessage>(o=>
{
return _scheduler.Schedule(recurse=>
{
try
{
var messages = GetMessages();
foreach (var msg in messages)
{
o.OnNext(msg);
}
recurse();
}
catch (Exception ex)
{
o.OnError(ex);
}
});
});
}
private IEnumerable<IMessage> GetMessages()
{
//Do some work here that gets messages from a queue,
// file system, database or other system that cant push
// new data at us.
//
//This may return an empty result when no new data is found.
}
}
The reason I really don't like Subjects, is that is usually a case of the developer not really having a clear design on the problem. Hack in a subject, poke it here there and everywhere, and then let the poor support dev guess at WTF was going on. When you use the Create/Generate etc methods you are localizing the effects on the sequence. You can see it all in one method and you know no-one else is throwing in a nasty side effect. If I see a subject fields I now have to go looking for all the places in a class it is being used. If some MFer exposes one publicly, then all bets are off, who knows how this sequence is being used!
Async/Concurrency/Rx is hard. You don't need to make it harder by allowing side effects and causality programming to spin your head even more.

In general you should avoid using Subject, however for the thing you are doing here I think they work quite well. I asked a similar question when I came across the "avoid subjects" message in Rx tutorials.
To quote Dave Sexton (of Rxx)
"Subjects are the stateful components of Rx. They are useful for when
you need to create an event-like observable as a field or a local
variable."
I tend to use them as the entry point into Rx. So if I have some code that needs to say 'something happened' (like you have), I would use a Subject and call OnNext. Then expose that as an IObservable for others to subscribe to (you can use AsObservable() on your subject to make sure nobody can cast to a Subject and mess things up).
You could also achieve this with a .NET event and use FromEventPattern, but if I'm only going to turn the event into an IObservable anyway, I don't see the benefit of having an event instead of a Subject (which might mean I'm missing something here)
However, what you should avoid quite strongly is subscribing to an IObservable with a Subject, i.e. don't pass a Subject into the IObservable.Subscribe method.

Often when you're managing a Subject, you're actually just reimplementing features already in Rx, and probably in not as robust, simple and extensible a way.
When you're trying to adapt some asynchronous data flow into Rx (or create an asynchronous data flow from one that's not currently asynchronous), the most common cases are usually:
The source of data is an event: As Lee says, this is the simplest case: use FromEvent and head to the pub.
The source of data is from a synchronous operation and you want polled updates, (eg a webservice or database call): In this case you could use Lee's suggested approach, or for simple cases, you could use something like Observable.Interval.Select(_ => <db fetch>). You may want to use DistinctUntilChanged() to prevent publishing updates when nothing has changed in the source data.
The source of data is some kind of asynchronous api that calls your callback: In this case, use Observable.Create to hook up your callback to call OnNext/OnError/OnComplete on the observer.
The source of data is a call that blocks until new data is available (eg some synchronous socket read operations): In this case, you can use Observable.Create to wrap the imperative code that reads from the socket and publishes to the Observer.OnNext when data is read. This may be similar to what you're doing with the Subject.
Using Observable.Create vs creating a class that manages a Subject is fairly equivalent to using the yield keyword vs creating a whole class that implements IEnumerator. Of course, you can write an IEnumerator to be as clean and as good a citizen as the yield code, but which one is better encapsulated and feels a neater design? The same is true for Observable.Create vs managing Subjects.
Observable.Create gives you a clean pattern for lazy setup and clean teardown. How do you achieve this with a class wrapping a Subject? You need some kind of Start method... how do you know when to call it? Or do you just always start it, even when no one is listening? And when you're done, how do you get it to stop reading from the socket/polling the database, etc? You have to have some kind of Stop method, and you have to still have access not just to the IObservable you're subscribed to, but the class that created the Subject in the first place.
With Observable.Create, it's all wrapped up in one place. The body of Observable.Create is not run until someone subscribes, so if no one subscribes, you never use your resource. And Observable.Create returns a Disposable that can cleanly shutdown your resource/callbacks, etc - this is called when the Observer unsubscribes. The lifetimes of the resources you're using to generate the Observable are neatly tied to the lifetime of the Observable itself.

The quoted block text pretty much explains why you shouldn't be using Subject<T>, but to put it simpler, you are combining the functions of observer and observable, while injecting some sort of state in between (whether you're encapsulating or extending).
This is where you run into trouble; these responsibilities should be separate and distinct from each other.
That said, in your specific case, I'd recommend that you break your concerns into smaller parts.
First, you have your thread that is hot, and always monitoring the hardware for signals to raise notifications for. How would you do this normally? Events. So let's start with that.
Let's define the EventArgs that your event will fire.
// The event args that has the information.
public class BaseFrameEventArgs : EventArgs
{
public BaseFrameEventArgs(IBaseFrame baseFrame)
{
// Validate parameters.
if (baseFrame == null) throw new ArgumentNullException("IBaseFrame");
// Set values.
BaseFrame = baseFrame;
}
// Poor man's immutability.
public IBaseFrame BaseFrame { get; private set; }
}
Now, the class that will fire the event. Note, this could be a static class (since you always have a thread running monitoring the hardware buffer), or something you call on-demand which subscribes to that. You'll have to modify this as appropriate.
public class BaseFrameMonitor
{
// You want to make this access thread safe
public event EventHandler<BaseFrameEventArgs> HardwareEvent;
public BaseFrameMonitor()
{
// Create/subscribe to your thread that
// drains hardware signals.
}
}
So now you have a class that exposes an event. Observables work well with events. So much so that there's first-class support for converting streams of events (think of an event stream as multiple firings of an event) into IObservable<T> implementations if you follow the standard event pattern, through the static FromEventPattern method on the Observable class.
With the source of your events, and the FromEventPattern method, we can create an IObservable<EventPattern<BaseFrameEventArgs>> easily (the EventPattern<TEventArgs> class embodies what you'd see in a .NET event, notably, an instance derived from EventArgs and an object representing the sender), like so:
// The event source.
// Or you might not need this if your class is static and exposes
// the event as a static event.
var source = new BaseFrameMonitor();
// Create the observable. It's going to be hot
// as the events are hot.
IObservable<EventPattern<BaseFrameEventArgs>> observable = Observable.
FromEventPattern<BaseFrameEventArgs>(
h => source.HardwareEvent += h,
h => source.HardwareEvent -= h);
Of course, you want an IObservable<IBaseFrame>, but that's easy, using the Select extension method on the Observable class to create a projection (just like you would in LINQ, and we can wrap all of this up in an easy-to-use method):
public IObservable<IBaseFrame> CreateHardwareObservable()
{
// The event source.
// Or you might not need this if your class is static and exposes
// the event as a static event.
var source = new BaseFrameMonitor();
// Create the observable. It's going to be hot
// as the events are hot.
IObservable<EventPattern<BaseFrameEventArgs>> observable = Observable.
FromEventPattern<BaseFrameEventArgs>(
h => source.HardwareEvent += h,
h => source.HardwareEvent -= h);
// Return the observable, but projected.
return observable.Select(i => i.EventArgs.BaseFrame);
}

It is bad to generalize that Subjects are not good to use for a public interface.
While it is certainly true, that this is not the way a reactive programming approach should look like, it is definitively a good improvement/refactoring option for your classic code.
If you have a normal property with an public set accessor and you want to notify about changes, there speaks nothing against replacing it with a BehaviorSubject.
INPC or additional other events are just not that clean and it personally wears me off.
For this purpose you can and should use BehaviorSubjects as public properties instead of normal properties and ditch INPC or other events.
Additionally the Subject-interface makes the users of your interface more aware about the functionality of your properties and are more likely to subscribe instead of just getting the value.
It is the best to use if you want others to listen/subscribe to changes of a property.

Is it possible to implement scoped lock in C#?

A common pattern in C++ is to create a class that wraps a lock - the lock is either implicitly taken when object is created, or taken explicitly afterwards. When object goes out of scope, dtor automatically releases the lock.
Is it possible to do this in C#? As far as I understand there are no guarantees on when dtor in C# will run after object goes out of scope.
Clarification:
Any lock in general, spinlock, ReaderWriterLock, whatever.
Calling Dispose myself defeats the purpose of the pattern - to have the lock released as soon as we exit scope - no matter if we called return in the middle, threw exception or whatnot.
Also, as far as I understand using will still only queue object for GC, not destroy it immediately...

To amplify Timothy's answer, the lock statement does create a scoped lock using a monitor. Essentially, this translates into something like this:
lock(_lockKey)
{
// Code under lock
}
// is equivalent to this
Monitor.Enter(_lockKey)
try
{
// Code under lock
}
finally
{
Monitor.Exit(_lockKey)
}
In C# you rarely use the dtor for this kind of pattern (see the using statement/IDisposable). One thing you may notice in the code is that if an async exception happens between the Monitor.Enter and the try, it looks like the monitor will not be released. The JIT actually makes a special guarantee that if a Monitor.Enter immediately precedes a try block the async exception will not happen until the try block thus ensuring the release.

Your understanding regarding using is incorrect, this is a way to have scoped actions happen in a deterministic fashion (no queuing to the GC takes place).
C# supplies the lock keyword which provides an exclusive lock and if you want to have different types (e.g. Read/Write) you'll have to use the using statement.
P.S. This thread may interest you.

It's true that you don't know exactly when the dtor is going to run... but, if you implement the IDisposable interface, and then use either a 'using' block or call 'Dispose()' yourself, you will have a place to put your code.
Question: When you say "lock", do you mean a thread lock so that only one thread at a time can use the object? As in:
lock (_myLockKey) { ... }
Please clarify.

For completeness there is another way to achieve a similar RAII effect without using using and IDisposable. In C# using is usually clearer (see also here for some more thoughts), but in other languages (e.g. Java), or even in C# if using is not appropriate for some reason, it's useful to know.
It's an idiom called "Execute Around" and the idea is that you call a method that does the pre and post stuff (e.g. locking/unlocking your threads, or setting up and committing/ closing your DB connection etc), and you pass into that method a delegate that will implement the operations you want to occur in between.
e.g.:
funkyObj.InOut( delegate{ System.Console.WriteLine( "middle bit" ); } );
Depending on what the InOut method does, the output might be something like:
first bit
middle bit
last bit
As I say, this answer is for completeness only, the previous suggestions of using with IDisposable, as well as the lock keyword, are going to be better 99% of the time.
It's a shame that, while .Net has gone further than many other modern OO languages in this regards (I'm looking at you, Java), it still places the responsibility for RAII to work on the client code (ie the code that uses using), whereas in C++ the destructor will always run at the end of the scope.

Why would you want a scoped lock in the first place? Suppose you have the following code:
lock(obj) {
... some logic goes here
}
If exception has happened inside try inserted in place of lock, this is often means that you have a corrupted state now and other threads will continue to work with corrupted state. It is better to let the program hang to signal about the problem.
Another problem is that try incurs some performance penalty, but this is usually much lesser problem if at all.
Jeffrey Richter specifically advises not to use lock statement.

I've been really bothered by the fact that using is up to the developer to remember to do - at best you get a warning, which most people never bother to promote to an error. So, I've been toying with an idea like this - it forces the client to at least TRY to do things correctly. Fortunately and unfortunately, it's a closure, so the client could still keep a copy of the resource, and try to use it again later - but this code at least tries to push the client in the right direction...
public class MyLockedResource : IDisposable
{
private MyLockedResource()
{
Console.WriteLine("initialize");
}
public void Dispose()
{
Console.WriteLine("dispose");
}
public delegate void RAII(MyLockedResource resource);
static public void Use(RAII raii)
{
using (MyLockedResource resource = new MyLockedResource())
{
raii(resource);
}
}
public void test()
{
Console.WriteLine("test");
}
}
Good usage:
MyLockedResource.Use(delegate(MyLockedResource resource)
{
resource.test();
});
Bad usage! (Unfortunately, this can't be prevented...)
MyLockedResource res = null;
MyLockedResource.Use(delegate(MyLockedResource resource)
{
resource.test();
res = resource;
res.test();
});
res.test();

How should I make my hashtable cache implementation thread safe when it gets cleared?

I have a class used to cache access to a database resource. It looks something like this:
//gets registered as a singleton
class DataCacher<T>
{
IDictionary<string, T> items = GetDataFromDb();
//Get is called all the time from zillions of threads
internal T Get(string key)
{
return items[key];
}
IDictionary<string, T> GetDataFromDb() { ...expensive slow SQL access... }
//this gets called every 5 minutes
internal void Reset()
{
items.Clear();
}
}
I've simplified this code somewhat, but the gist of it is that there is a potential concurrency issue, in that while the items are being cleared, if Get is called things may go awry.
Now I can just bung lock blocks into Get and Reset, but I'm worried that the locks on the Get will reduce performance of the site, as Get is called by every request thread in the web app many many times.
I can do something with a doubly checked locks I think, but I suspect there is a cleaner way to do this using something smarter than the lock{} block. What to do?
edit: Sorry all, I didn't make this explicit earlier, but the items.Clear() implementation I am using is not in fact a straight dictionary. Its a wrapper around a ResourceProvider which requires that the dictionary implementation calls .ReleaseAllResources() on each of the items as they get removed. This means that calling code doesn't want to run against an old version that is in the midst of disposal. Given this, is the Interlocked.Exchange method the correct one?

I would start by testing it with just a lock; locks are very cheap when not contested. However - a simpler scheme is to rely on the atomic nature of reference updates:
public void Clear() {
var tmp = GetDataFromDb(); // or new Dictionary<...> for an empty one
items = tmp; // this is atomic; subsequent get/set will use this one
}
You might also want to make items a volatile field, just to be sure it isn't held in the registers anywhere.
This still has the problem that anyone expecting there to be a given key may get disappointed (via an exception), but that is a separate issue.
The more granular option might be a ReaderWriterLockSlim.

One option is to completely replace the IDictionary instance instead of Clearing it. You can do this in a thread-safe way using the Exchange method on the Interlocked class.

See if the database will tell you what data has change. You could use
Trigger to write changes to a history table
Query Notifications (SqlServer and Oracle has these, other must do as well)
Etc
So you don’t have to reload all the data based on a timer.
Failing this.
I would make the Clear method create a new IDictionary by calling GetDataFromDB(), then once the data has been loaded set the “items” field to point to the new Dictionary. (The garbage collector will clean up the old dictionary once no threads are accessing it.)
I don’t think you care if some threads
get “old” results while reloading the
data – (if you do then you will just
have to block all threads on a lock –
painful!)
If you need all thread to swap over to the new dictionary at the same time, then you need to declare the “items” field to be volatile and use the Exchange method on the Interlocked class. However it is unlikely you need this in real life.