We Keep Coding

Handling Tens of Thousands of Events C# Elegantly from TableDependency

I have a project where a part of it monitors changes made to an SQL database. I am using the SQL Table Dependency NuGet Package to monitor changes so I can update the UI when changes are made.
The issue I have is that there is a function in my program that can add 50-99k rows to a table in the database. The event gets triggered as many times as there are rows added. This is problematic because I do not want to update the UI 99k times. I want to update it at most once or twice. How I am handling right now is when I detect that 5 events are triggered within a certain timespan I DeInit the TableDependency, then a delayed task reenables it after a few seconds and also triggers a UI update at the end so it won't miss anything while it was disabled temporarily.
I also tried using a static bool for rate limiting, instead of DeIniting and ReIniting the TableDependency, but it takes 30-90s sometimes because the event handler cannot reasonably keep up with them all. I think the DeInit works better because removing the callbacks from the eventhandler appears to clear it from events. I could not find a way to clear the event handler from the massive queue of events otherwise.
I tried delving into Reactive Extensions and using the Throttle function. This worked OK except for the fact that the first event received would not trigger. It would wait until the events died off to trigger (I realize this is by design). This makes the program feel unresponsive for awhile because when the events are triggered SQL has already added all the rows so all it really needs is that first event and last event to update at most.
The reason I am trying to find an alternative is because TableDependency sometimes (no idea how to replicate this yet) is orphaning Trigger scripts in SQL on the table with invalid ids and it causes fatal exceptions to occur when the DB Instance (I am using EF6 Core) runs SaveChanges(). I theorize running the DeInit and Init functions frequently is at best not helping the issue and at worst the direct cause of it. So I am trying to find some way to avoid frequently DeIniting and ReIniting the TableDependency but also have my UI updates feel responsive and not have bad performance.
DeInit function:
private static void DeInitDependency(TableType tableType)
{
if(tableType == TableType.Event)
{
eventTableDependency.Stop();
eventTableDependency.Dispose();
eventTableDependency.OnChanged -= SqlDependencyEventTable_OnChanged;
eventTableDependency.OnError -= DepEvent_OnError;
eventTableDependency.OnStatusChanged -= DepEvent_OnStatusChanged;
eventChangeTrackingStarted = false;
}
else if (tableType == TableType.Location)
{
locationTableDependency.Stop();
locationTableDependency.Dispose();
locationTableDependency.OnChanged -= SqlDependencyLocationTable_OnChanged;
locationTableDependency.OnError -= DepLocation_OnError;
locationTableDependency.OnStatusChanged -= DepLocation_OnStatusChanged;
locationChangeTrackingStarted = false;
}
}
Init/Reinit Function:
public static void InitDependency(TableType tableType)
{
try
{
//Set Connection String to SQL
string dbConnectionString = "";
dbConnectionString = sqlCore.generateConnectionString();
if(tableType == TableType.Event)
{
//Create Dependency and Connect
eventTableDependency = new SqlTableDependency<NextGenGui.Models.Event>(dbConnectionString, executeUserPermissionCheck: false);
eventTableDependency.OnChanged += SqlDependencyEventTable_OnChanged;
eventTableDependency.OnError += DepEvent_OnError;
eventTableDependency.OnStatusChanged += DepEvent_OnStatusChanged;
eventTableDependency.Start();
eventChangeTrackingStarted = true;
Debug.WriteLine("Event SQL TRACKING STARTED!");
}
else if(tableType == TableType.Location)
{
locationTableDependency = new SqlTableDependency<Models.Location>(dbConnectionString, executeUserPermissionCheck: false);
locationTableDependency.OnChanged += SqlDependencyLocationTable_OnChanged;
locationTableDependency.OnError += DepLocation_OnError;
locationTableDependency.OnStatusChanged += DepLocation_OnStatusChanged;
locationTableDependency.Start();
locationChangeTrackingStarted = true;
Debug.WriteLine("Location SQL TRACKING STARTED!");
}
}catch (Exception ex)
{
Debug.WriteLine(ex);
if(ex.Message.Contains("Service broker"))
{
InitSQLBrokerSetting();
}
}
}

It sounds like you need one event per business-level operation, instead of one event per table update. If that's the case, then you're going to have to look at implementing your own solution. Topics like SignalR and ServiceBus are good starting points to look into. This stream of business operations is a useful thing to implement anyway, for auditing and caching.
It's worth pointing out that you don't have to completely replace the SQL Table Dependency in one go. You can start with just the tables that are causing problems from the bulk operations.

You effectively need to debounce the event signaling. Rather that removing the event handler (which means you won't know what rows have changed during that period) could your handler be changed to simply make marks in memory state based on what the current UI might be caring about? For instance if the UI is displaying one or more key records that you care about, the handler knows what IDs are relevant and if any of those rows are touched the markers are set, in which a periodic check looks at the markers and refreshes the view. This might be a single row the user is viewing, or a set of row IDs based on something like search results, etc.
If instead the UI reflects a summary of all data state and any row change would impact it, then perhaps consider an in-memory representation that can be updated by the event handler and periodically checked and refresh the view if necessary.
Ultimately it depends on what the view is currently displaying and the relationship with regards to the data update monitoring. Libraries like SignalR are typically employed for more server operation to relevant clients signaling where actions invoked by one client can be relayed to other clients to update data or refresh their view. When it comes to something from the database you would probably want to implement some manner of filtering and processing to monitor when relevant changes have come in to raise a signal for a stale view check to pick up on and refresh.

(C#) BackgroundWorker() ProgressChanged not working

I have a WPF application that consist of two threads simulating an enterprise producting and selling items in 52 weeks (only one transaction is allowed per week). I need to use a background worker as well so that I can display the data in a listview. As of right now, my UI freezes when clicking on simulate but I can see that the output is still working in the debugging terminal. I have tried everything that I can think of and to be honest, I have had the help of my teacher and even he couldn't find a working solution.
What is freezing my UI when I call Simulate() ?
When my code is different and my UI isn't freezing, my listview never updates because it seems that DataProgress() doesn't work — e.UserStart is never iterating.
Simulate button calls :
private void Simulate(object sender, RoutedEventArgs e)
{
// Declare BackgroundWorker
Data = new ObservableCollection<Operations>();
worker = new BackgroundWorker();
worker.WorkerReportsProgress = true;
worker.WorkerSupportsCancellation = true;
worker.RunWorkerAsync(52);
worker.DoWork += ShowData;
worker.ProgressChanged += DataProgress;
worker.RunWorkerCompleted += DataToDB;
Production = new Production(qtyProduction, timeExecProd);
Sales = new Sales(qtySales, timeExecSales);
Thread prod = new Thread(Production.Product);
prod.Start();
Thread.Sleep(100);
Thread sales = new Thread(Sales.Sell);
sales.Start();
}
DoWork : ShowData() :
Console.WriteLine("Simulation started | Initial stock : 500");
Production = new Production(qtyProduction, timeExecProd);
Sales = new Sales(qtySales, timeExecSales);
while (Factory.Week < max) // max = 52
{
if (worker.CancellationPending) // also this isn't reacting to worker.CancelAsync();
e.Cancel = true;
// My teacher tried to call my threads from here, but it breaks the purpose of having
// two threads as he was just calling 52 times two functions back to back and therefore
// wasn't "randomizing" the transactions.
int progressPercentage = Convert.ToInt32(((double)(Factory.Week) / max) * 100);
(sender as BackgroundWorker).ReportProgress(progressPercentage, Factory.Week);
}
ProgressChanged : DataProgress() :
if (e.UserState != null) // While using debugger, it looks like this is called over & over
{
Data.Add(new Operations()
{
id = rnd.Next(1,999),
name = Factory.name,
qtyStock = Factory.Stock,
averageStock = Factory.AverageStock,
week = Factory.Week
});
listview.ItemsSource = Data;
}
RunWorkerCompleted : DataToDB() :
// Outputs "Work done" for now.
In case you want to know what happens when I call my threads, it looks like this :
Sell() :
while (Factory.Week <= 52)
{
lock (obj)
{
// some math function callings¸
Factory.Week++;
}
Thread.Sleep(timeExecSales);
}
Should I use a third thread just for updating my listview? I don't see how as I need it to be synced with my static variables. This is my first project for learning multithreading... I'm kind of clueless and flabbergasted that even my teacher can't help.

On the one hand, there isnt enough context in the code posted to get a full picture to answer your questions accurately. We can, however, deduce what is going wrong just from the code you have posted.
First, lets try to answer your two questions. We can likely infer the following:
This code here:
if (e.UserState != null)
{
Data.Add(new Operations()
{
id = rnd.Next(1,999),
name = Factory.name,
qtyStock = Factory.Stock,
averageStock = Factory.AverageStock,
week = Factory.Week
});
listview.ItemsSource = Data;
}
You are using a Windows Forms background thread object to try and update a WPF GUI object which should only be done on the main GUI thread. There is also the obvious no-no of never updating GUI objects from non-UI threads. Using BackgroundWorker also has its own issues with threading (foreground/background), contexts and execution, as it relies on the Dispatcher and SynchronizationContexts to get the job done.
Then there is the curiosity of setting the binding over and over in this line:
listview.ItemsSource = Data;
Let's put a pin in that for a moment...
There is, as the other commenter pointer out already, no exit strategy in your while loop:
while (Factory.Week < max) // max = 52
{
if (worker.CancellationPending) // also this isn't reacting to worker.CancelAsync();
e.Cancel = true;
// My teacher tried to call my threads from here, but it breaks the purpose of having
// two threads as he was just calling 52 times two functions back to back and therefore
// wasn't "randomizing" the transactions.
int progressPercentage = Convert.ToInt32(((double)(Factory.Week) / max) * 100);
(sender as BackgroundWorker).ReportProgress(progressPercentage, Factory.Week);
}
But thats not the bigger problem... in addition to the misuse/misunderstanding of when/how many/how to use threading, there doesnt seem to be any kind of thread synchronization of any kind. There is no way to predict or track thread execution of lifetime in this way.
At this point the question is technically more or less answered, but I feel like this will just leave you more frustrated and no better off than you started. So maybe a quick crash course in basic design might help straighten out this mess, something your teacher should have done.
Assuming you are pursuing software development, and since you have chosen WPF here as your "breadboard" so to speak, you will likely come across terms such as MVC (model view controller) or MVVM (model view view-model). You will also likely come across design principles such as SOLID, separation of concerns, and grouping things into services.
Your code here is a perfect example of why all of these frameworks and principles exist. Lets look at some of the problems you have encountered and how to fix them:
You have threading code (logic and services - controller [loosely speaking]) mixed in with presentation code (listview update - view) and collection update (your observable collection - model). Thats one reason (of many) you are having such a difficult time coding, fixing and maintaining the problem at hand. To clean it up, separate it out (separation of concerns). You might even move each operation into its own class with an interface/API to that class (service/ micro-service).
Not everything needs to be solved with threads. But for now, lets learn to crawl, then walk before we run. Before you start learning about async/await or the TPL (task parallel library) lets go old school. Get a good book... something even from 20 years ago is find... go old school, and learn how to use the ThreadPool and kernel synchronization objects such as mutexes, events, etc and how to signal between threads. Once you master that, then learn about TPL and async/await.
Dont cross the streams. Dont mix WinForms, WPF and I even saw Console.WriteLine.
Learn about Data Binding, and in particular how it works in WPF. ObservableCollection is your friend, bind your ItemsSource to it once, then update the ObservableCollection and leave the GUI object alone.
Hopefully this will help you straighten out the code and get things running.
Good luck!

Setup an event loop for a UWP app

How do I setup an event loop (main loop) in a UWP app?
My goal is to have an app that has a main page with a continuously updating calculation, that then continuously updates an image on the page. While that is constantly happening, the user can click a button to change the calculation behavior, or to go to a second page and edit related data.
The user can then click back to the main page and see the image and calculation restart and continuously update. (The calculation is complex, so it should go as fast as possible and use up as much app time as possible).
If there is a way to accomplish this without an event loop I would like to know that also, but so far I have not found a way.
With an event loop, I can simply update the calculation and UI every time through the loop. But I have not found any way to do so. This question asked nearly the same thing, but was never directly answered and had a different use case anyway.
The closest I have found to a solution is to grab the CoreDispatcher from the CoreWindow and use the RunIdleAsync() method to create a loop:
public MainPage()
{
this.InitializeComponent();
Windows.UI.Core.CoreWindow appwindow = Windows.UI.Core.CoreWindow.GetForCurrentThread();
Windows.UI.Core.CoreDispatcher appdispatcher = appwindow.Dispatcher;
//create a continuously running idle task (the app loop)
appdispatcher.RunIdleAsync( (dummyt) =>
{
//do the event loop here
.
.
.
if (appdispatcher.ShouldYield()) //necessary to prevent blocking UI
{
appdispatcher.ProcessEvents(Windows.UI.Core.CoreProcessEventsOption.ProcessAllIfPresent);
}
});
}
The main problem with this is that you can't switch between pages (you get a system exception from dispatching events within an already dispatched event).
Second, this is very messy and requires maintaining extra state in the event loop. Besides, why should I have to go through these contortions just to have some calculations happening while the app is waiting for user input?
Is there a way to do this (besides switching to a C++ DirectX app)?

I don't know about setting up your own event loop, but there is no reason to do so.
What you are talking about sounds like a great case for Tasks. You would start a calculation Task whenever your user did something, having it report its progress via standard C# events if you need mid-operation updates. Those updates would modify properties in your view model which the binding system would then pick up.
You could also make your calculation code cancellable so changes can abort a previous calculation.
All of this involves pretty standard UWP concepts; no need for a special event loop. That you are even considering that makes me think you need to study MVVM and multi-threading/tasks; you are still thinking in a very "Win-Forms" kind of way.

If we're talking about some event loop, or stream, .Net has a great library named Rx, or Reactive Extensions, which may be helpful for you. You can set up a simple flow, something like this:
var source = Observable
// fire event every second
.Interval(TimeSpan.FromSeconds(1), Scheduler.DispatcherScheduler)
// add timestamp for event
.Timestamp()
// gather system information to calculate
.Select(GetSystemInfo);
Note that the events right now are on UI thread, as you need to access the controls. Now you have two options: use Rx for background processing too or use TPL Dataflow' TransformBlock for processing your system information into new image (it can be Observer and Observable at a time). After that you need to get back to UI thread.
First option:
var source = Observable
// fire event every second
.Interval(TimeSpan.FromSeconds(1), DispatcherScheduler.Current)
// add timestamp for event
.Timestamp()
// gather system information to calculate
.Select(GetSystemInfo)
// expensive calculations are done in background
.Select(x => x.ObserveOn(DefaultScheduler.Instance))
.Select(x => Expensive(x))
.Select(x => x.ObserveOn(DispatcherScheduler.Current))
.Select(x => UpdateUI(x));
You probably should split this chain into several observers and observables, still the idea is the same, more information here: Rx Design Guidelines.
Second option:
var action = new TransformBlock<SystemInfo, ImageDelta>(CalculateDelta,
new ExecutionDataflowBlockOptions
{
// we can start as many item processing as processor count
MaxDegreeOfParallelism = Environment.ProcessorCount,
});
IDisposable subscription = source.Subscribe(action.AsObserver());
var uiObserver = action.AsObservable()
.Select(x => x.ObserveOn(DispatcherScheduler.Current))
.Select(x => UpdateUI(x));
I want to note that UWP and MVVM pattern do provide a possibility to work with binding between UI and ObservableCollection, which will help you to notify user in most natural way.

Reactive-ui binding with async viewmodelupdate crashes

I have a question about Reactive ui, its bindings, and how it handles ui updates. I always assumed that using ReactiveUi would take care of all ui updates on the ui thread. But I recently found out this isn't always the case.
In short the question is: How can I use reactiveui to two-way-model-bind a viewmodel and a view, and assure that updating the ViewModel doesn't crash when run on a different thread than the ui-thread? Without having to manually subscribe to changes and update explicitely on the uiThread, as that defeats the purpose of reactiveui, as well as making it harder to encapsulate all logic in the PCL.
Below I've provided a very simple (Android) project using Xamaring and Reactiveui, to do the following:
Button with the text 'Hello World'
Clicking on it appends 'a' to the button's text.
I let the Activity implement IViewFor, and I use a ViewModel deriving from ReactiveObject, containing the text that I want to change.
I bind the Activity's button.Text to the ViewModel.Text, to let reactiveui deal with all changes and ui updates.
Finally, I add a function to the button's onclick to append 'a' to the ViewModel.
The issue I have is the following:
button.Click += delegate
{
this.ViewModel.Text += "a"; // does not crash
Task.Run(() => { this.ViewModel.Text += "a"; }); // crash
};
Directly appending 'a' is not an issue. However, adding 'a' on a different thread results in the well-known Java exception: Exception: Only the original thread that created a view hierarchy can touch its views.
I understand the exception and where it's coming from. In fact, if I were to append the 'a' on a different thread, I already had it working with simply not binding the text. But rather by subscribing to changes, and using the RunOnUiThread-method to make changes to the ui. But this scenario kind of defeats the purpose of using ReactiveUi. I really like the clean coding way of the simple statement 'this.Bind(ViewModel, x => x.Text, x => x.button.Text);', but if this has to run on the uiThread, I can't see how to make it work.
And naturally this is the bare mininum to show the problem. The actual problem as to why I bring this up is because I want to use the 'GetAndFetchLatest'-method from akavache. It gets data asynchroniously and caches it, and executes a function (being updating the ViewModel). If the data is already in the cache, it will execute the ViewModel-update with the cached result AND do the computationlogic in a different thread, and then call the same function again once it's done (resulting in the crash, because that's on a different thread, updates the ViewModel, which results in the crash).
Note that even though explicitely using RunOnUiThread works, I really don't want (can't even) to call this within the ViewModel. Because I have a more complex piece of code in which a button simply tells the ViewModel to go fetch data and update itself. If I were required to do this on the uiThread (i.e. after I got data back, I update the ViewModel), then I can't bind iOS to the same ViewModel anymore.
And lastly, here's the entire code to make it crash. I've seen the Task.Run-part sometimes work, but if you add some more tasks and keep updating the ViewModel in them, it's bound to crash eventually on the UI-thread.
public class MainActivity : Activity, IViewFor<MainActivity.RandomViewModel>
{
public RandomViewModel ViewModel { get; set; }
private Button button;
protected override void OnCreate(Bundle bundle)
{
base.OnCreate(bundle);
SetContentView(Resource.Layout.Main);
this.button = FindViewById<Button>(Resource.Id.MyButton);
this.ViewModel = new RandomViewModel { Text = "hello world" };
this.Bind(ViewModel, x => x.Text, x => x.button.Text);
button.Click += delegate
{
this.ViewModel.Text += "a"; // does not crash
Task.Run(() => { this.ViewModel.Text += "a"; }); // crash
};
}
public class RandomViewModel : ReactiveObject
{
private string text;
public string Text
{
get
{
return text;
}
set
{
this.RaiseAndSetIfChanged(ref text, value);
}
}
}
object IViewFor.ViewModel
{
get
{
return ViewModel;
}
set
{
ViewModel = value as RandomViewModel;
}
}
}

This has been already discussed here and there, and the short answer is "as designed, for performance reasons".
I'm personally not really convinced by the later (performance is usually a bad driver when designing an API), but I'll try to explain why I think this design is correct anyway:
When binding an object to a view, you usually expect the view to come and peak (read) at your object properties, and it's doing so from the UI thread.
Once you acknowledge that, the only sane (as in thread-safe and guaranteed to work) way to modify this object (which is being peaked into from the UI thread) is to do so also from the UI thread.
Modifications from other threads may work, but only within specific conditions, that devs usually don't care about (up until they get UI artifacts, in which case they ... perform a refresh...).
For instance if you're using INPC, and your property values are immutable (e.g. string), and your view won't feel bad about observing a value change before it receives the notification of it (simple controls probably are ok with it, grids with filtering/sorting capabilities are probably not ok, unless they completely deep-copy their source).
You should design your ViewModel with the fact that it lives in the UI context in mind.
With Rx, that means having .OnserverOn(/* ui sheduler */) right before ViewModel modification code.

Updating UI in realtime

I have to create a WPF UI, which subsribes to real time Fx Rate(Currency + rate) updates and displays them in a grid (roughly 1000 updates per second, which means each row in the grid could get updated upto 1000 times per second).The grid would have atleast 50 rows at any point in time.
Towards this, I have created a Viewmodel which subscribes to the update events, and store those updates inside a concurrent dictionary with key as symbol and value as a RateViewModel object. Then I have another observable collection which has all those rateviewmodel objects, and bind that to a grid.
Code:
public class MyViewModel
{
private readonly IRatesService ratesService;
private readonly ConcurrentDictionary<string, RateViewModel> rateDictionary;
private object _locker = new object();
public MyViewModel(IRatesService ratesService)
{
this.ratesService = ratesService;
this.ratesService.OnUpdate += OnUpdate;
rateDictionary = new ConcurrentDictionary<string, RateViewModel>();
RateViewModels = new ObservableCollection<RateViewModel>();
}
private void OnUpdate(object sender, RateUpdateEventArgs e)
{
RateViewModel exisistingRate;
if (!rateDictionary.TryGetValue(e.Update.Currency, out exisistingRate))
{
exisistingRate = new RateViewModel(new Rate(e.Update.Currency, e.Update.Rate));
rateDictionary.TryAdd(e.Update.Currency, exisistingRate);
return;
}
lock (_locker)
{
exisistingRate.UpdateRate(e.Update.Rate);
}
Application.Current.Dispatcher.BeginInvoke(new Action(() => SearchAndUpdate(exisistingRate)));
}
public ObservableCollection<RateViewModel> RateViewModels { get; set; }
private void SearchAndUpdate(RateViewModel rateViewModel)
{
//Equals is based on Currency
if (!RateViewModels.Contains(rateViewModel))
{
RateViewModels.Add(rateViewModel);
return;
}
var index = RateViewModels.IndexOf(rateViewModel);
RateViewModels[index] = rateViewModel;
}
}
I have 4 questions over this:
Is there a way I can eliminate the ObservableCollection, as it's leading to 2 different datastructures storing the same items - but still have my updates relayed to the UI?
I have used Concurrent Dictionary, which leads to locking the whole update operation. Is there any other clever way of handling this rather than locking the whole dicitionary or for that matter any datastructure?
My UpdateRate method also locks - all my properties on my RateviewModel is read only except the price, as this is getting updated. Is there a way to make this atomic, please note that the price is coming in as a double.
Is there a way I can optimize the SearchAndUpdate method, this is kind of related to 1st. At the moment I believe it's an O(n)operation.
Using .NET 4.0 and have omitted INPC for brevity.
*EDIT:*Could you please help me in rewriting this in a better manner taking all the 4 points into account? Psuedocode will do.
Thanks,
-Mike

1) I wouldn't worry about 50 extra refs floating around
2) Yes, lockless data structures are doable. Interlocked Is your friend here and they are pretty much all one offs. ReaderWriterLock is another good option if you aren't changing what items are in your dictionary often.
3) Generally, if you are dealing with more data more data than the UI can handle you are going to want to do the updates in the background, only fire INPC on the UI thread, and more importantly have a facility to drop UI updates (while still updating the backing field). Basic approach is going to be something like:
Do an Interlocked.Exchange on the backing field
Use Interlocked.CompareExchange to set a private field to 1, if this returns 1 exit becuase there is still a pending UI update
If Interlocked.CompareExchange returned 0, invoke to the UI and fire your property changed event and update you throttling field to 0 (technically there is more you need to do if you care about non x86)
4) SearchAndUpdate Seems superfluous... UpdateRate should be bubbling to the UI and you only need to Invoke to the UI thread if you need to add or remove an item to the observable collection.
Update: here is a sample implementation... things are little more complicated because you are using doubles which don't get atomicity for free on 32 bit CPUs.
class MyViewModel : INotifyPropertyChanged
{
private System.Windows.Threading.Dispatcher dispatcher;
public MyViewModel(System.Windows.Threading.Dispatcher dispatcher)
{
this.dispatcher = dispatcher;
}
int myPropertyUpdating; //needs to be marked volatile if you care about non x86
double myProperty;
double MyPropery
{
get
{
// Hack for Missing Interlocked.Read for doubles
// if you are compiled for 64 bit you should be able to just do a read
var retv = Interlocked.CompareExchange(ref myProperty, myProperty, -myProperty);
return retv;
}
set
{
if (myProperty != value)
{
// if you are compiled for 64 bit you can just do an assignment here
Interlocked.Exchange(ref myProperty, value);
if (Interlocked.Exchange(ref myPropertyUpdating, 1) == 0)
{
dispatcher.BeginInvoke(() =>
{
try
{
PropertyChanged(this, new PropertyChangedEventArgs("MyProperty"));
}
finally
{
myPropertyUpdating = 0;
Thread.MemoryBarrier(); // This will flush the store buffer which is the technically correct thing to do... but I've never had problems with out it
}
}, null);
}
}
}
}
public event PropertyChangedEventHandler PropertyChanged = delegate {};
}

Mike -
I would approach this a little differently. You really dont need an Observable Collection unless new Fx rows are being added. Observable Collection as you know only gives you built-in change notification in that scenario. If you have a list of 50 rows (for example) and the Fx object (which represents each individual row) is updated 1000 times a second - then you can very well use the INotifyPropertyChanged on the Fx Properties on the Object and let that mechanism update the UI as they change. My line of thought is - this is a simpler approach for UI updates rather than move them from one collection to another
Now with regards to your second point - 1000 updates in a second (to an existing FX object) - which technically is unreadable from a UI perspective - the approach I have taken is freeze and thaw - which means you essentially intercept the InotifyPropertyChanged (as its firing to the UI) and keep it frequency based - so for example - every 1 sec - whatever my status of all FX objects is (refresh the UI). Now within that second - whatever updates happen to the FX properties - they keep overwriting on themselves - and the latest/correct value when the 1 second interval happens - is shown to UI. That way - data being shown to UI is always correct and relevant when its displayed to UI.

There are a couple of factors to take into account, especially if the number of displayed rates will change dynamically. I'm assuming the 1000 updates/sec are coming from a thread other than the UI thread.
The first is that you will need to marshall the updates to the UI thread - done for you for updates to an existing ViewModel, not done for you for new/deleted ViewModels. With a 1000 updates a second you probably want to control the granularity of the marshalling to the UI thread and the context switching that this entails. Ian Griffiths wrote a great blog series on this.
The second is that if you want your UI to remain responsive you probably want to avoid as many gen 2 garbage collections as possible which means minimising the pressure on the GC. This might be an issue in your case as you create a new Rate object update for each update.
Once you start to have a few screens that do the same thing you'll want to find a way to abstract this updating behaviour out into a common component. Other wise you'll be sprinkling threading code through your ViewModels which is error prone.
I've created an open source project, ReactiveTables, which addresses these three concerns and adds a couple of other features such as being able to filter, sort, join your model collections. Also there are demos showing how to use it with virtual grids to get the best performance. Maybe this can help you out/inspire you.