We Keep Coding

Synchronising threads between C++/CLI and C#

Say I have the following C++/CLI class:
public ref class ManagedDLAContainer {
private:
DLAContainer* native_dla_container;
public:
ManagedDLAContainer() : native_dla_container(new DLAContainer()) {}
~ManagedDLAContainer() { delete native_dla_container; }
KeyValuePair<int,int> GetMRAParticle() {
std::pair<int,int> mra_p = native_dla_container->mra_particle();
KeyValuePair<int,int>^ mra_kvp = gcnew
KeyValuePair<int,int>(mra_p.first, mra_p.second);
return *mra_kvp;
}
size_t Size() {
return native_dla_container->size();
}
void Generate(size_t _n) {
native_dla_container->generate(_n);
}
};
where DLAContainer is an unmanaged, native C++ class. The method generate of this class does computationally intensive calculations involving building up a system of particles, whilst mra_particle returns a std::pair<int,int> representing the most-recently-added particle to the DLAContainer. This C++/CLI code is packaged in a class library which is then used by a C# WPF project.
The WPF project has the following class:
public partial class MainWindow : Window {
private static readonly object locker = new object();
private readonly ManagedDLAContainer dla;
private KeyValuePair<int,int> mra_pair;
private readonly AggregateSystemManager aggregate_manager;
public MainWindow() {
InitializeComponent();
dla = new ManagedDLAContainer();
mra_pair = new KeyValuePair<int,int>();
aggregate_manager = new AggregateSystemManager();
// a Model3DGroup which is part of the GUI
WorldModels.Children.Add(aggregate_manager.AggregateSystemModel());
}
private void AggregateUpdateListener(uint _particle_slider_val){
while (dla_2d.Size() < _particle_slider_val) {
KeyValuePair<int,int> agg_kvp = dla.GetMRAParticle();
if (agg_kvp.Equals(mra_pair) {
// no updates to aggregate
}
else {
mra_pair = agg_kvp;
Point3D position = new Point3D(agg_kvp.Key, agg_kvp.Value,0);
aggregate_manager.AddParticle(position);
Dispatcher.Invoke(() => { aggregate_manager.Update(); } );
}
}
}
private void GenerateAggregate() {
lock(locker) {
uint particle_slider_val = 0;
Dispatcher.Invoke(() => {
particle_slider_val = (uint)particles_slider.Value;
});
// start AggregateUpdateListener in new task
Task.Factory.StartNew(() => AggregateUpdateListener(particle_slider_val));
// generate the aggregate
dla.Generate(particle_slider_val);
}
}
private void GenerateButtonHandler(object sender, RoutedEventArgs e) {
// start GenerateAggregate method in new task
Task.Factory.StartNew(() => GenerateAggregate());
}
}
Explanation of Program Flow
The user sets the number of particles to generate using the particle_slider GUI element, they then click the generate button.
The method GenerateAggregate is run in a new task using Task.Factory.StartNew, this function then runs the AggregateUpdateListener in a separate task and finally calls Generate to produce the particle system.
AggregateUpdateListener continuously runs whilst Generate is running and checks for updates to the most-recently-added particle and renders the new particles to the interface using the AggregateManager class as necessary.
The Problem
Whilst this program is mostly successful, occasionally particles generated using ManagedDLAContainer::Generate(size_t) are missed by the AggregateUpdateListener method resulting in gaps in the particle system shown in the interface.
The issue here, I believe, is that the two processes (the generation of the particle system and the checking-rendering process) are not running in a correctly synchronised way. I need to somehow get it so that when a particle is added to the system an event is fired which allows AggrgegateUpdateListener to then perform the rendering and then hand the control back to generation.
I am unsure of how to do this however, as my Generate function will run non-stop in the background until the particle system has been generated completely up to the number of particles required - and this process is executed via the native C++ code behind the scenes which has no knowledge whatsoever of my C# project. It is for this reason that I think using something like AutoResetEvent is not applicable in this case, but if it is then please do let me know how!
The only solution I can come up with at the moment (which has nothing to do with correctly synchronising the processes) is to iterate over the final particle system of the GUI and do comparison checks with the particle system container of the C++ code (which will always be correct) and fill in any missing gaps in the former when a miss is detected from comparison with the latter. But this is a nasty "solution" and I'd much rather have it correctly running in real-time.
If any further information is required, let me know.

You could try using a named Semaphore both in C++ and C# however it's probably a bit heavy as it's meant for synchronisation between processes.
Otherwise, following Hans' comment you could create a BlockingCollection in the managed C++ part and expose it to the C# project. You would then need to consume all the particles in the ManagedDLAContainer and enqueue them in the blocking queue.
In the C# GUI I'd recommend you have a timer every 200/250ms which when it fires dequeues all the available particles in the queue and then updates the GUI. Make sure to bound this by some max number of updates so that you don't get stuck constantly pulling items off the queue (if the native code is faster than the the C# code).

Xamarin iOS memory leaks everywhere

We've been using Xamarin iOS for the last 8 months and developed a non-trivial enterprise app with many screens, features, nested controls. We've done our own MVVM arch, cross platform BLL & DAL as "recommended". We share code between Android and even our BLL/DAL is used on our web product.
All is good except now in release phase of project we discover irreparable memory leaks everywhere in the Xamarin iOS-based app. We've followed all the "guidelines" to resolve this but the reality is that C# GC and Obj-C ARC appear to be incompatible garbage collection mechanisms in the current way they overlay each other in monotouch platform.
The reality we've found is that hard cycles between native objects and managed objects WILL occur and FREQUENTLY for any non-trivial app. It's extremely easy for this to happen anywhere you use lambdas or gesture recognizers for example. Add in the complexity of MVVM and it's almost a guarantee. Miss just one of these situations and entire graphs of objects will never get collected. These graphs will lure other objects in and grow like a cancer, eventually resulting in a prompt and merciless extermination by iOS.
Xamarin's answer is an uninterested deferral of the issue and an unrealistic expectation that "devs should avoid these situations". Careful consideration of this reveals this as an admission that Garbage Collection is essentially broken in Xamarin.
The realization for me now is that you don't really get "garbage collection" in Xamarin iOS in the traditional c# .NET sense. You need employ "garbage maintanence" patterns actually get the GC moving and doing its job, and even then it'll never be perfect - NON DETERMINISTIC.
My company has invested a fortune trying to stop our app from crashing and/or running out of memory. We've basically had to explicitly and recursively dispose every damn thing in sight and implement garbage maintanence patterns into the app, just to stop the crashes and have a viable product we can sell. Our customers are supportive and tolerant, but we know this cannot hold forever. We are hoping Xamarin have a dedicated team working on this issue and get it nailed once and for all. Doesn't look like it, unfortunately.
Question is, is our experience the exception or the rule for non-trivial enterprise-class apps written in Xamarin?
UPDATE
See answer for DisposeEx method and solution.

I have shipped a non-trivial app written with Xamarin. Many others have as well.
"Garbage collection" isn't magic. If you create a reference that is attached to the root of your object graph and never detach it, it will not be collected. That's not only true of Xamarin, but of C# on .NET, Java, etc.
button.Click += (sender, e) => { ... } is an anti-pattern, because you don't have a reference to the lambda and you can never remove the event handler from the Click event. Similarly, you have to be careful that you understand what you're doing when you create references between managed and unmanaged objects.
As for "We've done our own MVVM arch", there are high profile MVVM libraries (MvvmCross, ReactiveUI, and MVVM Light Toolkit), all of which take reference/leak issues very seriously.

I used the below extension methods to solve these memory leak issues. Think of Ender's Game final battle scene, the DisposeEx method is like that laser and it disassociates all views and their connected objects and disposes them recursively and in a way that shouldn't crash your app.
Just call DisposeEx() on UIViewController's main view when you no longer need that view controller. If some nested UIView has special things to dispose, or you dont want it disposed, implement ISpecialDisposable.SpecialDispose which is called in place of IDisposable.Dispose.
NOTE: this assumes no UIImage instances are shared in your app. If they are, modify DisposeEx to intelligently dispose.
public static void DisposeEx(this UIView view) {
const bool enableLogging = false;
try {
if (view.IsDisposedOrNull())
return;
var viewDescription = string.Empty;
if (enableLogging) {
viewDescription = view.Description;
SystemLog.Debug("Destroying " + viewDescription);
}
var disposeView = true;
var disconnectFromSuperView = true;
var disposeSubviews = true;
var removeGestureRecognizers = false; // WARNING: enable at your own risk, may causes crashes
var removeConstraints = true;
var removeLayerAnimations = true;
var associatedViewsToDispose = new List<UIView>();
var otherDisposables = new List<IDisposable>();
if (view is UIActivityIndicatorView) {
var aiv = (UIActivityIndicatorView)view;
if (aiv.IsAnimating) {
aiv.StopAnimating();
}
} else if (view is UITableView) {
var tableView = (UITableView)view;
if (tableView.DataSource != null) {
otherDisposables.Add(tableView.DataSource);
}
if (tableView.BackgroundView != null) {
associatedViewsToDispose.Add(tableView.BackgroundView);
}
tableView.Source = null;
tableView.Delegate = null;
tableView.DataSource = null;
tableView.WeakDelegate = null;
tableView.WeakDataSource = null;
associatedViewsToDispose.AddRange(tableView.VisibleCells ?? new UITableViewCell[0]);
} else if (view is UITableViewCell) {
var tableViewCell = (UITableViewCell)view;
disposeView = false;
disconnectFromSuperView = false;
if (tableViewCell.ImageView != null) {
associatedViewsToDispose.Add(tableViewCell.ImageView);
}
} else if (view is UICollectionView) {
var collectionView = (UICollectionView)view;
disposeView = false;
if (collectionView.DataSource != null) {
otherDisposables.Add(collectionView.DataSource);
}
if (!collectionView.BackgroundView.IsDisposedOrNull()) {
associatedViewsToDispose.Add(collectionView.BackgroundView);
}
//associatedViewsToDispose.AddRange(collectionView.VisibleCells ?? new UICollectionViewCell[0]);
collectionView.Source = null;
collectionView.Delegate = null;
collectionView.DataSource = null;
collectionView.WeakDelegate = null;
collectionView.WeakDataSource = null;
} else if (view is UICollectionViewCell) {
var collectionViewCell = (UICollectionViewCell)view;
disposeView = false;
disconnectFromSuperView = false;
if (collectionViewCell.BackgroundView != null) {
associatedViewsToDispose.Add(collectionViewCell.BackgroundView);
}
} else if (view is UIWebView) {
var webView = (UIWebView)view;
if (webView.IsLoading)
webView.StopLoading();
webView.LoadHtmlString(string.Empty, null); // clear display
webView.Delegate = null;
webView.WeakDelegate = null;
} else if (view is UIImageView) {
var imageView = (UIImageView)view;
if (imageView.Image != null) {
otherDisposables.Add(imageView.Image);
imageView.Image = null;
}
} else if (view is UIScrollView) {
var scrollView = (UIScrollView)view;
// Comment out extension method
//scrollView.UnsetZoomableContentView();
}
var gestures = view.GestureRecognizers;
if (removeGestureRecognizers && gestures != null) {
foreach(var gr in gestures) {
view.RemoveGestureRecognizer(gr);
gr.Dispose();
}
}
if (removeLayerAnimations && view.Layer != null) {
view.Layer.RemoveAllAnimations();
}
if (disconnectFromSuperView && view.Superview != null) {
view.RemoveFromSuperview();
}
var constraints = view.Constraints;
if (constraints != null && constraints.Any() && constraints.All(c => c.Handle != IntPtr.Zero)) {
view.RemoveConstraints(constraints);
foreach(var constraint in constraints) {
constraint.Dispose();
}
}
foreach(var otherDisposable in otherDisposables) {
otherDisposable.Dispose();
}
foreach(var otherView in associatedViewsToDispose) {
otherView.DisposeEx();
}
var subViews = view.Subviews;
if (disposeSubviews && subViews != null) {
subViews.ForEach(DisposeEx);
}
if (view is ISpecialDisposable) {
((ISpecialDisposable)view).SpecialDispose();
} else if (disposeView) {
if (view.Handle != IntPtr.Zero)
view.Dispose();
}
if (enableLogging) {
SystemLog.Debug("Destroyed {0}", viewDescription);
}
} catch (Exception error) {
SystemLog.Exception(error);
}
}
public static void RemoveAndDisposeChildSubViews(this UIView view) {
if (view == null)
return;
if (view.Handle == IntPtr.Zero)
return;
if (view.Subviews == null)
return;
view.Subviews.ForEach(RemoveFromSuperviewAndDispose);
}
public static void RemoveFromSuperviewAndDispose(this UIView view) {
view.RemoveFromSuperview();
view.DisposeEx();
}
public static bool IsDisposedOrNull(this UIView view) {
if (view == null)
return true;
if (view.Handle == IntPtr.Zero)
return true;;
return false;
}
public interface ISpecialDisposable {
void SpecialDispose();
}

Couldn't be agree more with the OP that "Garbage Collection is essentially broken in Xamarin".
Here's an example shows why you have to always use a DisposeEx() method as suggested.
The following code leaks memory:
Create a class the inherits UITableViewController
public class Test3Controller : UITableViewController
{
public Test3Controller () : base (UITableViewStyle.Grouped)
{
}
}
Call the following code from somewhere
var controller = new Test3Controller ();
controller.Dispose ();
controller = null;
GC.Collect (GC.MaxGeneration, GCCollectionMode.Forced);
Using Instruments you will see that there are ~ 274 persistent objects with 252 KB never collected.
Only way to fix this is add DisposeEx or similar functionality to the Dispose() function and call Dispose manually to ensure disposing == true.
Summary: Creating a UITableViewController derived class and then disposing/nulling will always cause the heap to grow.

iOS and Xamarin have a slightly troubled relationship. iOS uses reference counts to manage and dispose of its memory. The reference count of an object gets incremented and decremented when references are added and removed. When the reference count goes to 0, the object is deleted and the memory freed. Automatic Reference Counting in Objective C and Swift help with this, but it’s still difficult to get 100% right and dangling pointers and memory leaks can be a pain when developing using native iOS languages.
When coding in Xamarin for iOS, we have to bear reference counts in mind as we will be working with iOS native memory objects. In order to communicate with the iOS operating system, Xamarin creates what are known as Peers which manage the reference counts for us. There are two types of Peers – Framework Peers and User Peers. Framework Peers are managed wrappers around well-known iOS objects. Framework Peers are stateless and therefore hold no strong references to the underlying iOS objects and can be cleaned up by the garbage collectors when required – and don’t cause memory leaks.
User Peers are custom managed objects that are derived from Framework Peers. User Peers contain state and are therefore kept alive by the Xamarin framework even if your code has no references to them – e.g.
public class MyViewController : UIViewController
{
public string Id { get; set; }
}
We can create a new MyViewController, add it to the view tree, then cast a UIViewController to a MyViewController. There may be no references to this MyViewController, so Xamarin needs to ‘root’ this object to keep this it alive whilst the underlying UIViewController is alive, otherwise we will lose the state information.
The problem is that if we have two User Peers that reference each other then this creates a reference cycle that cannot be automatically broken – and this situation happens often!
Consider this case:-
public class MyViewController : UIViewController
{
public override void ViewDidAppear(bool animated)
{
base.ViewDidAppear (animated);
MyButton.TouchUpInside =+ DoSomething;
}
void DoSomething (object sender, EventArgs e) { ... }
}
Xamarin creates two User Peers that reference each other – one for MyViewController and another for MyButton (because we have an event handler). So, this will create a reference cycle that will not be cleared up by the garbage collector. In order to have this cleared up, we must unsubscribe the event handler, and this is usually done in the ViewDidDisappear handler – e.g.
public override void ViewDidDisappear(bool animated)
{
ProcessButton.TouchUpInside -= DoSomething;
base.ViewDidDisappear (animated);
}
Always unsubscribe to your iOS event handlers.
How to diagnose these memory leaks
A good way to diagnose these memory problems is to add some code in debug to the Finalisers of the classes derived from iOS wrapper classes – such as UIViewControllers. (Although only put this in your debug builds and not in release builds because it’s reasonably slow.
public partial class MyViewController : UIViewController
{
#if DEBUG
static int _counter;
#endif
protected MyViewController (IntPtr handle) : base (handle)
{
#if DEBUG
Interlocked.Increment (ref _counter);
Debug.WriteLine ("MyViewController Instances {0}.", _counter);
#endif
}
#if DEBUG
~MyViewController()
{
Debug.WriteLine ("ViewController deleted, {0} instances left.",
Interlocked.Decrement(ref _counter));
}
#endif
}
So, Xamarin’s memory management is not broken in iOS, but you do have to be aware of these ‘gotchas’ which are specific to running on iOS.
There is an excellent page by Thomas Bandt called Xamarin.iOS Memory Pitfalls that goes into this in more detail and also provides some very useful hints and tips.

I noticed in your DisposeEx method you dispose of the collection view source and table view source before you kill the visible cells of that collection. I noticed when debugging that the visible cells property gets set to an empty array therefore, when you start to dispose visible cells, they no longer "exist" hence it becomes an array of zero elements.
Another thing I noticed is that you will run into inconsistency exceptions if you don't remove the parameter view from its super view, I've noticed especially with setting the layout of the collection view.
Other than that I've had to implement something similar on our side.

How to return data from void function?

So, around a week ago I asked a question about activex and UDP. Here it is:
C# UDP Socket client and server
Now, I created two applications, one (the sender) to send pre-defined strings via UDP. The other is activex component that is called from a webpage, and it's thread is working in the background. Once an UDP message arrives, then it's doing it's stuff (writing in database, writing in log.txt, and so on).
The last thing i need is to return data (it's yet to be said if it will be string or something else). However, the method in the activex which is called must be a void, because if it's made to be string, the threading wont work, and only the first message will arrive.
My question is, how to do that? How to return data from a void function? For example, the web app now is calling the activex DLL like this:
ClassLibrary1.Class1 activex = new ClassLibrary1.Class1();
activex.StartThread();
And the StartThread() calls the listening thread and it's working in the background, and once UDP msg arrives, its doing some stuff like i said above.
How can i return value with the threads (events) and the web app will catch it and use it?
Thanks a lot.

You can use events (which implement the Observable pattern) to alert any listener that a new message has arrived:
public class NewMessageArgs : EventArgs
{
public string Message { get; private set; }
public NewMessageArgs(string message)
{
Message = message;
}
}
public class ActiveXComponent
{
public event EventHandler<NewMessageArgs> OnMessage;
public void StartThread()
{
while (true)
{
//do stuff
//raise "message received" event
if (OnMessage != null)
OnMessage(this, new NewMessageArgs("hi"));
}
}
}
You can then listen to these events like so:
ActiveXComponent activex = new ActiveXComponent();
activex.OnMessage += ProcessMessage;
activex.StartThread();
public void ProcessMessage(object sender, NewMessageArgs args)
{
var msg = args.Message;
//process
}

Basically you have to store some data in a spot where you can access it from both places (from the thread, and from the place where you started the thread). So you have a couple of options from the top of my head.
Store it in a database
Create a specific object (whatever type you need), and store it in a place where it is accessible from both places. For example, a singleton. A simpler better solution is to create a property on your ClassLibrary.Class1 class: set it from within the Class1-class, and get it from the place where you created an instance of your Class1-class.
Add an event to your Class1-class which fires when it is finished doing its job. And add some data to the EventArgs.
I'm assuming here you get notified when your thread is done doing whatever it is doing.
Edit: added events

The threading function can change the fields values of the class and you can access those fields, also your thread can fire events that other classes can subcribe to and then act on it.
Class1
{
private string value;
public string Value{get{return value;} set{value=value; FireTheEvent();}}
}

C# - Separate UI from business code

I want to know what's the best practice for the following case :
I'm developing an application that need to be used on three different screen size : 240x320 ; 800x600 ; 1280x1024 (and only these three) in different languages (using localization).
So what I've done is to create a library by screen size, each implementing an interface defined in the project that calls these screens.
First question is, all my resources files are duplicated and that is many files to keep up to date (or with duplicate labels). Is there a simple way to change this?
Second question is, am I using the good approach or exists it a better way to do what I'm trying to do? By applying a design pattern maybe?
Sample code
The caller code :
public int X { get { return 800; } }
public int Y { get { return 600; } }
public interface IScreenForm
{
Form FormComponent { get; }
}
public interface IFormA : IScreenForm
{
TextBox UserTextBox { get; } // For use
}
public void LoadForm<T>()
where T:IScreenForm
{
if (!typeof(T).IsInterface)
{
throw new ArgumentException(typeof(T).Name + " is not an interface");
}
Assembly screenAssembly;
string screenResolutionDll = string.Format("{0}_{1}_screens.dll", this.X, this.Y);
screenAssembly = Assembly.LoadFrom(screenResolutionDll);
Type formType = screenAssembly.GetTypes()
.FirstOrDefault(t => t.GetInterfaces().Where(i => i.Name == typeof(T).Name).Count() > 0);
if (formType != null)
{
Form form = (Form)formType.GetConstructor(new Type[] { }).Invoke(null);
if (form != null)
{
form.Show();
}
else
{
throw new ArgumentException("Form doesn't provide a new() ctor");
}
}
else
{
throw new ArgumentException("Any form doesn't implement the interface : " + typeof(T).Name);
}
}
Screen DLL :
public partial class MyFirstForm : Form, caller.IFormA
{
public Form1()
{
InitializeComponent();
}
/* caller.IFormA interface implementation */
}

Arnaud, from my own experience, from reading, and from talking to experienced developers: When it comes to supporting multiple screen sizes and with localization, there are no magic bullets. For localization put all your strings in resource files, or even in data files.
As far as the screen sizes, I would not try to be too clever. Yes, make sure that none of your business / non-GUI logic gets duplicated, but duplicating resource files, forms, etc - I would not worry about it. Some GUI frameworks like Qt and GTK auto-resize and auto-position GUI-widgets (e.g. Qt has 'spacer' widgets). It works OK most of the time, however I still prefer explicit control. With GUI programming, there are often unexpected glitches, having three independent sets of GUI components will allow you to address them, should they arise. Some example sources of problems:
1) Font sizing.
2) Windows accessibility settings.
3) Some national languages have longer average words than others, the long words have trouble fitting in the real estate available.
If I were in your shoes, I would look for example how browsers handle this (e.g. mobile vs. desktop version) and I would try to find some practical advice on the Web (like here at SO). I doubt that books on design patterns will help much with this. Disclaimer: I am a design-pattern skeptic.

It is hard to advice without getting the whole picture but for some tips that might by handy read this
http://msdn.microsoft.com/en-us/library/ff648753.aspx
about the SmartClient software Factory.
It comes with architecural guidance and solutions for issue you often see in this kind of apps

Reactive Framework (RX) and dealing with events Asynchronously

So I'm just playing around with RX and learning it. I started playing with Events, and wanted to know how to subscribe to events, and process the results in batches asynchronously. Allow me to explain with code:
Simple class that raises events:
public class EventRaisingClass
{
public event EventHandler<SomeEventArgs> EventOccured;
//some other code that raises event...
}
public class SomeEventArgs : EventArgs
{
public SomeEventArgs(int data)
{
this.SomeArg = data;
}
public int SomeArg { get; private set; }
}
Then my Main:
public static void Main(string[] args)
{
var eventRaiser = new EventRaisingClass();
IObservable<IEvent<SomeEventArgs>> observable =
Observable.FromEvent<SomeEventArgs>(e => eventRaiser.EventOccured += e, e => eventRaiser.EventOccured -= e);
IObservable<IList<IEvent<SomeEventArgs>>> bufferedEvents = observable.BufferWithCount(100);
//how can I subscribte to bufferedEvents so that the subscription code gets called Async?
bufferedEvents.Subscribe(list => /*do something with list of event args*/); //this happens synchrounously...
}
As you can see in my comments, when you just call subscribe like that, all the subscription code happens synchronously. Is there a way out of the box using RX to have the Subscribe be called on different threads whenever there's a new batch of events to work on?

bufferedEvents.ObserveOn(Scheduler.TaskPool).Subscribe(...
SubscribeOn is to specify the schedule on which so-called "subscription side effects" are happening. For example, your observable can open a file each time somebody subscribes.
ObserveOn is to specify the schedule on which the call to the observer will happen every time when there is a new value. In practice, it is used more often than SubscribeOn.

I believe you're looking for SubscribeOn or ObserveOn, passing an IScheduler. There are several schedulers built-in under System.Concurrency; some of them use whatever thread is current, and others use specific threads.
This video has more info on the scheduler concept.
The Rx team also recently released a hands-on labs document which is the closest thing to a tutorial right now.