We Keep Coding

Count + List with Futures not working

I'm having problems with Futures in Nhibernate 3 and can't realize what's wrong.
The following code (without Futures), works as expected:
SessionHandler.DoInTransaction(transaction =>
{
var criteria = SessionHandler.Session.CreateCriteria<T>();
var clonedCriteria = (ICriteria)criteria.Clone();
var count = criteria
.SetProjection(Projections.RowCount())
.UniqueResult<Int32>();
var result = clonedCriteria
.SetMaxResults(PageSize)
.SetFirstResult(page * PageSize)
.List<T>();
ItemList = result;
TotalResults = count;
RecalculatePageCount();
});
SessionHandler just stores a Session for this context, and DoInTransaction is a convenience method:
public void DoInTransaction(Action<ITransaction> action)
{
using (var transaction = Session.BeginTransaction())
{
action(transaction);
transaction.Commit();
}
}
Now, the following code causes GenericAdoException:
SessionHandler.DoInTransaction(transaction =>
{
var criteria = SessionHandler.Session.CreateCriteria<T>();
var clonedCriteria = (ICriteria)criteria.Clone();
var count = criteria
.SetProjection(Projections.RowCount())
.FutureValue<Int32>();
var result = clonedCriteria
.SetMaxResults(PageSize)
.SetFirstResult(page * PageSize)
.Future<T>();
ItemList = result;
TotalResults = count.Value;
RecalculatePageCount();
});
I'm using PostgreSQL 9.2, Npgsql 2.0.11.0 and NHibernate 3.3.1.4000. If that matters, I use Fluent NHibernate for my mappings
Thank you for any advice.
EDIT:
After more research, I found that this error only occurrs after I add an item. At starting, I'm loading data in my form, and it works just fine. I get the exception when I reload the data in my form after adding an item. But it is pretty strange. The item is added correctly. The code for adding or updating items looks like this:
if (IsEditing)
{
SessionHandler.DoInTransaction(tx => SessionHandler.Session.Update(CurrentItem));
}
else
{
SessionHandler.DoInTransaction(tx => SessionHandler.Session.Save(CurrentItem));
}
What is strange is that I (sometimes, I think) get this exception when I'm raising the PropertyChanged event. I noticed that sometimes the InnerException is different. Sounds like a threading problem, but it is strange that it works without futures. But I'm not using threads for loading the data, just for adding items (hmm, but maybe, because I notify when my items are added, and I load the items in answer to "that message", I think that load would be executed in another thread)
EDIT 2:
The error seems pretty random. Sometimes I get it, sometimes not :S

I think I've found the issue.
This may sound stupid, but I think it makes sense. I had to swap these lines:
ItemList = result;
TotalResults = count.Value;
So they resulted
TotalResults = count.Value;
ItemList = result;
The problem was, basically, multithreading (I think I didn't mentioned it pretty much in my question, but the randomness of the errors were a bit suspicious). So first, I'll tell you some background so the solution is clearer:
When a new element is added to the database, a message is (globally) sent, so everyone 'interested' can update its elements to reflect the changes. As I'm using MVVM Light, I do it like this:
Messenger.Default.Send(new DataReloadSuggested<MyEntityType>(theUpdatedId));
I was using Tasks to add the elements, so when I clicked on the 'Add' button, something like this was executed:
Task.Factory.StartNew(CommitCurrentItem);
And, inside CommitCurrentItem, I added the item to the database and notified the program that the list was updated (sending the message as mentioned above).
My main form registered to that messages, like this:
Messenger.Default.Register<DataReloadSuggested<T>>(this, true, unused => LoadPage(CurrentPage));
Even though the LoadPage function was not creating a different thread manually (at that moment), it was executed in the same thread as the CommitCurrentItem. A Task is not guaranteed to run in a different thread, but in this case it was. LoadPage just called the code that was in the question. My code is guaranteed to raise the PropertyChanged event (from the INotifyPropertyChanged interface) in the UI thread, so when I set the ItemList property (of type IEnumerable), the UI was notified so it shows the new list. So, the UI retrieved the new value of ItemList while (in the other thread), the line TotalResults = count.Value; was executing.
In this case, I guess the query is not executed against the database until I retrieve the first value (the first item in the list or the RowCount).
Remember that ISession is not thread safe, so this situation was unreliable: the UI thread and the other thread was using the same session at the same time. In my code I'm not sharing the sessions between the ViewModels, and each ViewModel uses the Session with only one thread at the same time, in order to prevent this kind of situations.
So, the final solution was to force the execution of the query in the same thread I was working, so I simply called count.Value before setting ItemList to result.

Enumerator problem, Any way to avoid two loops?

I have a third party api, which has a class that returns an enumerator for different items in the class.
I need to remove an item in that enumerator, so I cannot use "for each". Only option I can think of is to get the count by iterating over the enum and then run a normal for loop to remove the items.
Anyone know of a way to avoid the two loops?
Thanks
[update] sorry for the confusion but Andrey below in comments is right.
Here is some pseudo code out of my head that won't work and for which I am looking a solution which won't involve two loops but I guess it's not possible:
for each (myProperty in MyProperty)
{
if (checking some criteria here)
MyProperty.Remove(myProperty)
}
MyProperty is the third party class that implements the enumerator and the remove method.

Common pattern is to do something like this:
List<Item> forDeletion = new List<Item>();
foreach (Item i in somelist)
if (condition for deletion) forDeletion.Add(i);
foreach (Item i in forDeletion)
somelist.Remove(i); //or how do you delete items

Loop through it once and create a second array which contains the items which should not be deleted.

If you know it's a collection, you can go with reverted for:
for (int i = items.Count - 1; i >= 0; i--)
{
items.RemoveAt(i);
}
Otherwise, you'll have to do two loops.

You can create something like this:
public IEnumerable<item> GetMyList()
{
foreach (var x in thirdParty )
{
if (x == ignore)
continue;
yield return x;
}
}

I need to remove an item in that enumerator
As long as this is a single item that's not a problem. The rule is that you cannot continue to iterate after modifying the collection. Thus:
foreach (var item in collection) {
if (item.Equals(toRemove) {
collection.Remove(toRemove);
break; // <== stop iterating!!
}
}

It is not possible to remove an item from an Enumerator. What you can do is to copy or filter(or both) the content of the whole enumeration sequence.
You can achieve this by using linq and do smth like this:
YourEnumerationReturningFunction().Where(item => yourRemovalCriteria);

Can you elaborate on the API and the API calls you are using?
If you receive an IEnumerator<T> or IEnumerable<T> you cannot remove any item from the sequence behind the enumerator because there is no method to do so. And you should of course not rely on down casting an received object because the implementation may change. (Actually a well designed API should not expose mutable objects holding internal state at all.)
If you receive IList<T> or something similar you can just use a normal for loop from back to front and remove the items as needed because there is no iterator which state could be corrupted. (Here the rule about exposing mutable state should apply again - modifying the returned collection should not change any state.)

IEnumerator.Count() will decide at run-time what it needs to do - enumerate to count or reflect to see it's a collection and call .Count that way.
I like SJoerd's suggestion but I worry about how many items we may be talking about.

Why not something like ..
// you don't want 2 and 3
IEnumerable<int> fromAPI = Enumerable.Range(0, 10);
IEnumerable<int> result = fromAPI.Except(new[] { 2, 3 });

A clean, readable way to do this is as follows (I'm guessing at the third-party container's API here since you haven't specified it.)
foreach(var delItem in ThirdPartyContainer.Items
.Where(item=>ShouldIDeleteThis(item))
//or: .Where(ShouldIDeleteThis)
.ToArray()) {
ThirdPartyContainer.Remove(delItem);
}
The call to .ToArray() ensures that all items to be deleted have been greedily cached before the foreach iteration begins.
Behind the scenes this involves an array and an extra iteration over that, but that's generally very cheap, and the advantage of this method over the other answers to this question is that it works on plain enumerables and does not involve tricky mutable state issues that are hard to read and easy to get wrong.
By contrast, iterating in reverse, while not rocket science, is much more prone to off-by-one errors and harder to read; and it also relies on internals of the collection such as not changing order in between deletions (e.g. better not be a binary heap, say). Manually adding items that should be deleted to a temporary list is just unnecessary code - that's what .ToArray() will do just fine :-).

an enumerator always has a private field pointing to the real collection.
you can get it via reflection.modify it.
have fun.

Sorting List vs. ObservableCollection

I have found out that I am "doing WPF wrong" and frustratingly must overhaul alot of my code.
How could I convert the following:
public static class SortName
{
public static int Compare(Person a, Person b)
{
return a.Name.CompareTo(b.Name);
}
}
and I call it like:
list.Sort(SortName.Compare);
to the format required for ObservableCollection. And how would I call it. So far i've tried this following based on what I read here
class ObservableCollectionSortName<T> : ObservableCollection<T>
{
public int Compare (Person a, Person b)
{
return a.Name.CompareTo(b.Name);
}
}

The observable collection doesn't implement sorting, for the simple reason that every time an item moves from one location in the list to another the collection raises an event. That would be great for watching animations of the sort algorithm in action, but it would sort of suck for, you know, sorting.
There are two ways to do this; they're very similar, and both start by sorting the items outside their observable collection, e.g. if _Fruits is an ObservableCollection<Fruit>, and you've defined an IComparer<Fruit> for the sort, you'd do:
var sorted = _Fruits.OrderBy(x => x, new FruitComparer());
That creates a new IEnumerable<Fruit> that, when you iterate over it, will have the objects in the new order. There are two things you can do with this.
One is to create a new collection, replace the old one, and force any items control(s) in the UI to rebind to it:
_Fruits = new ObservableCollection<Fruit>(sorted);
OnPropertyChanged("Fruits");
(This assumes that your class implements INotifyPropertyChanged in the usual way.)
The other is to create a new sorted list, and then use it to move the items in your collection:
int i = 0;
foreach (Fruit f in sorted)
{
_Fruits.MoveItem(_Fruits.IndexOf(f), i);
i++;
}
The second approach is something I'd only try if I had a really serious commitment to not rebinding the items controls, because it's going to raise a ton of collection-changed events.

If the main reason for sorting your observable collection is to display the data in a sorted list to the user and not for performance reasons (i.e. faster access) then a CollectionViewSource can be used. Bea Stollnitz has a good description on her blog of how to use the CollectionViewSource to implement sorting and grouping of collections for the UI.
This may help as it means you will not have to implement sorting on your observable collection and worry about the performance hits indicated by Robert of sending the INotifyCollectionChanged. However it will allow for the displaying of items in a sorted order in the UI.

C# Dictionary Loop Enhancment

I have a dictionary with around 1 milions items. I am constantly looping throw the dictionnary :
public void DoAllJobs()
{
foreach (KeyValuePair<uint, BusinessObject> p in _dictionnary)
{
if(p.Value.MustDoJob)
p.Value.DoJob();
}
}
The execution is a bit long, around 600 ms, I would like to deacrese it. Here is the contraints :
MustDoJob values mostly stay the same beetween two calls to DoAllJobs()
60-70% of the MustDoJob values == false
From time to times MustDoJob change for 200 000 pairs.
Some p.Value.DoJob() can not be computed at the same time (COM object call)
Here, I do not need the key part of the _dictionnary objet but I really do need it somewhere else
I wanted to do the following :
Parallelizes but I am not sure is going to be effective due to 4.
Sorts the dictionnary since 1. and 2. (and stop want I find the first MustDoJob == false) but I am wondering what 3. would result in
I did not implement any of the previous ideas since it could be a lot of job and I would like to investigate others options before. So...any ideas ?

What I would suggest is that your business object could raise an event to indicate that it needs to do a job when MustDoJob becomes true and you can subscribe to that event and store references to those objects in a simple list and then process the contents of that list when the DoAllJobs() method is called

My first suggestion would be to use just the values from the dictionary:
foreach (BusinessObject> value in _dictionnary.Values)
{
if(value.MustDoJob)
{
value.DoJob();
}
}
With LINQ this could be even easier:
foreach (BusinessObject value in _dictionnary.Values.Where(v => v.MustDoJob))
{
value.DoJob();
}
That makes it clearer. However, it's not clear what else is actually causing you a problem. How quickly do you need to be able to iterate over the dictionary? I expect it's already pretty nippy... is anything actually wrong with this brute force approach? What's the impact of it taking 600ms to iterate over the collection? Is that 600ms when nothing needs to do any work?
One thing to note: you can't change the contents of the dictionary while you're iterating over it - whether in this thread or another. That means not adding, removing or replacing key/value pairs. It's okay for the contents of a BusinessObject to change, but the dictionary relationship between the key and the object can't change. If you want to minimise the time during which you can't modify the dictionary, you can take a copy of the list of references to objects which need work doing, and then iterate over that:
foreach (BusinessObject value in _dictionnary.Values
.Where(v => v.MustDoJob)
.ToList())
{
value.DoJob();
}

Try using a profiler first. 4 makes me curious - 600ms may not be that much if the COM object uses most of the time, and then it is either paralellize or live with it.
I would get sure first - with a profiler run - that you dont target the totally wrong issue here.

Having established that the loop really is the problem (see TomTom's answer), I would maintain a list of the items on which MustDoJob is true -- e.g., when MustDoJob is set, add it to the list, and when you process and clear the flag, remove it from the list. (This might be done directly by the code manipulating the flag, or by raising an event when the flag changes; depends on what you need.) Then you loop through the list (which is only going to be 60-70% of the length), not the dictionary. The list might contain the object itself or just its key in the dictionary, although it will be more efficient if it holds the object itself as you avoid the dictionary lookup. It does depend on how frequently you're queuing 200k of them, and how time-critical the queuing vs. the execution is.
But again: Step 1 is make sure you're solving the right problem.

The use of a dictionary to me implies that the intention is to find items by a key, rather than visit every item. On the other hand, 600ms for looping through a million items is respectable.
Perhaps alter your logic so that you can simply pick the relevant items satisfying the condition directly out of the dictionary.

Use a List of KeyValuePairs instead. This means you can iterate over it super-quickly by doing
List<KeyValuePair<string,object>> list = ...;
int totalItems = list.Count;
for (int x = 0; x < totalItems; x++)
{
// whatever you plan to do with them, you have access to both KEY and VALUE.
}
I know this post is old, but I was looking for a way to iterate over a dictionary without the increased overhead of the Enumerator being created (GC and all), or generally a faster way to iterate over it.

How would you obtain the first and last items in a Queue?

Say I have a rolling collection of values where I specify the size of the collection and any time a new value is added, any old values beyond this specified size are dropped off. Obviously (and I've tested this) the best type of collection to use for this behavior is a Queue:
myQueue.Enqueue(newValue)
If myQueue.Count > specifiedSize Then myQueue.Dequeue()
However, what if I want to calculate the difference between the first and last items in the Queue? Obviously I can't access the items by index. But to switch from a Queue to something implementing IList seems like overkill, as does writing a new Queue-like class. Right now I've got:
Dim firstValue As Integer = myQueue.Peek()
Dim lastValue As Integer = myQueue.ToArray()(myQueue.Count - 1)
Dim diff As Integer = lastValue - firstValue
That call to ToArray() bothers me, but a superior alternative isn't coming to me. Any suggestions?

One thing you could do is have a temporary variable that stores the value that was just enqueued because that will be the last value and so the variable can be accessed to get that value.

Seems to me if you need quick access to the first item in the list, then you're using the wrong data structure. Switch a LinkedList instead, which conveniently has First and Last properties.
Be sure you only add and remove items to the linked list using AddLast and RemoveFirst to maintain the Queue property. To prevent yourself from inadvertantly violating the Queue property, consider creating a wrapper class around the linked list and exposing only the properties you need from your queue.

public class LastQ<T> : Queue<T>
{
public T Last { get; private set; }
public new void Enqueue(T item)
{
Last = item;
base.Enqueue(item);
}
}
Edit:
Obviously this basic class should be more robust to do things like protect the Last property on an empty queue. But this should be enough for the basic idea.

You could use a deque (double-ended queue).
I don't think there is one built into System.Collections(.Generic) but here's some info on the data structure. If you implemented something like this you could just use PeekLeft() and PeekRight() to get the first and last values.
Of course, it will be up to you whether or not implementing your own deque is preferable to dealing with the unsexiness of ToArray(). :)
http://www.codeproject.com/KB/recipes/deque.aspx

Your best bet would be to keep track of the last value added to the Queue, then use the myQueue.Peek() function to see the "first" (meaning next) item in the list without removing it.