I'm having problems with Futures in Nhibernate 3 and can't realize what's wrong.
The following code (without Futures), works as expected:
SessionHandler.DoInTransaction(transaction =>
{
var criteria = SessionHandler.Session.CreateCriteria<T>();
var clonedCriteria = (ICriteria)criteria.Clone();
var count = criteria
.SetProjection(Projections.RowCount())
.UniqueResult<Int32>();
var result = clonedCriteria
.SetMaxResults(PageSize)
.SetFirstResult(page * PageSize)
.List<T>();
ItemList = result;
TotalResults = count;
RecalculatePageCount();
});
SessionHandler just stores a Session for this context, and DoInTransaction is a convenience method:
public void DoInTransaction(Action<ITransaction> action)
{
using (var transaction = Session.BeginTransaction())
{
action(transaction);
transaction.Commit();
}
}
Now, the following code causes GenericAdoException:
SessionHandler.DoInTransaction(transaction =>
{
var criteria = SessionHandler.Session.CreateCriteria<T>();
var clonedCriteria = (ICriteria)criteria.Clone();
var count = criteria
.SetProjection(Projections.RowCount())
.FutureValue<Int32>();
var result = clonedCriteria
.SetMaxResults(PageSize)
.SetFirstResult(page * PageSize)
.Future<T>();
ItemList = result;
TotalResults = count.Value;
RecalculatePageCount();
});
I'm using PostgreSQL 9.2, Npgsql 2.0.11.0 and NHibernate 3.3.1.4000. If that matters, I use Fluent NHibernate for my mappings
Thank you for any advice.
EDIT:
After more research, I found that this error only occurrs after I add an item. At starting, I'm loading data in my form, and it works just fine. I get the exception when I reload the data in my form after adding an item. But it is pretty strange. The item is added correctly. The code for adding or updating items looks like this:
if (IsEditing)
{
SessionHandler.DoInTransaction(tx => SessionHandler.Session.Update(CurrentItem));
}
else
{
SessionHandler.DoInTransaction(tx => SessionHandler.Session.Save(CurrentItem));
}
What is strange is that I (sometimes, I think) get this exception when I'm raising the PropertyChanged event. I noticed that sometimes the InnerException is different. Sounds like a threading problem, but it is strange that it works without futures. But I'm not using threads for loading the data, just for adding items (hmm, but maybe, because I notify when my items are added, and I load the items in answer to "that message", I think that load would be executed in another thread)
EDIT 2:
The error seems pretty random. Sometimes I get it, sometimes not :S
I think I've found the issue.
This may sound stupid, but I think it makes sense. I had to swap these lines:
ItemList = result;
TotalResults = count.Value;
So they resulted
TotalResults = count.Value;
ItemList = result;
The problem was, basically, multithreading (I think I didn't mentioned it pretty much in my question, but the randomness of the errors were a bit suspicious). So first, I'll tell you some background so the solution is clearer:
When a new element is added to the database, a message is (globally) sent, so everyone 'interested' can update its elements to reflect the changes. As I'm using MVVM Light, I do it like this:
Messenger.Default.Send(new DataReloadSuggested<MyEntityType>(theUpdatedId));
I was using Tasks to add the elements, so when I clicked on the 'Add' button, something like this was executed:
Task.Factory.StartNew(CommitCurrentItem);
And, inside CommitCurrentItem, I added the item to the database and notified the program that the list was updated (sending the message as mentioned above).
My main form registered to that messages, like this:
Messenger.Default.Register<DataReloadSuggested<T>>(this, true, unused => LoadPage(CurrentPage));
Even though the LoadPage function was not creating a different thread manually (at that moment), it was executed in the same thread as the CommitCurrentItem. A Task is not guaranteed to run in a different thread, but in this case it was. LoadPage just called the code that was in the question. My code is guaranteed to raise the PropertyChanged event (from the INotifyPropertyChanged interface) in the UI thread, so when I set the ItemList property (of type IEnumerable), the UI was notified so it shows the new list. So, the UI retrieved the new value of ItemList while (in the other thread), the line TotalResults = count.Value; was executing.
In this case, I guess the query is not executed against the database until I retrieve the first value (the first item in the list or the RowCount).
Remember that ISession is not thread safe, so this situation was unreliable: the UI thread and the other thread was using the same session at the same time. In my code I'm not sharing the sessions between the ViewModels, and each ViewModel uses the Session with only one thread at the same time, in order to prevent this kind of situations.
So, the final solution was to force the execution of the query in the same thread I was working, so I simply called count.Value before setting ItemList to result.
Related
As a reaction from Shane Neuville I'm changing my question. But yes. It may be more complicated then I first thought it was.
I have a ReactiveList<ServiceEntryModel> Services; that will be populated when the view have appeared. With every entry in the list I have used the function called Services.ItemsAdded.Subscribe() where the RxCell will be populated with data that at the end of the whole list should be ordered in a specific way. As of now I have have two separate ReactiveCommands.
1.ReactiveCommand<Unit, List<ServiceEntryModel>> RefreshServices;
2.ReactiveCommand<Unit, List<ServiceEntryModel>> OrderByCommand;
The second command should be called when the whole list is populated from the Services.ItemsAdded.Subscribe method. How can I check/know when every entry is done?
I have tried something with counters and ObservablePropertyHelpers, but without any luck so far.
Here is my ViewModel part that should be relevant
Services.ItemsAdded.Subscribe(async entryItem =>
{
ServiceEntityStatusDto serviceEntityStatus = await GetServiceEntryStatus(entryItem);
if (serviceEntityStatus != null)
{
entryItem.AvailabilityStatus = serviceEntityStatus.AvailabilityStatus;
entryItem.PerformanceStatus = serviceEntityStatus.PerformanceStatus;
}
counterServices++;
//await OrderServices.Execute(); //TODO infinite loop ?
});
The part is commented is a infinite loop, because the Services.ItemsAdded will be continue to be called.
The user can pull to refresh and the first command will be called again, so right when that is done. The seconds should order the whole list again.
Is this enough?
RefreshServices
.SelectMany(_ => OrderServices.Execute())
.Subscribe();
Or the InvokeCommand syntax
RefreshServices
.InvokeCommand(OrderServices);
The distinction being that with InvokeCommand OrderServices will not execute if it's already executing which might not fit your use case. For example maybe RefreshServices can execute again while OrderServices is still running.
Note: This is a follow on question from a previous one I asked here.
Just to summarise quickly, my previous problem was with how to databind a BlockingCollection to a control in WPF - which was solved by using the CollectionViewSource.
However, I have thought about my use case a bit more and realised that simply using a BlockingCollection isn't going to work for me. I want the following behaviour;
One source of "work items", submitted to a common pool
Multiple "processors" of these work items
Items which are still "pending" and those which are being "processed" should both show up in the same view for databinding.
For example;
8 work items are submitted simultaneously, and the max level of concurrency is 4. Four of the work items should be moved into the "Processing" state, while the other four remain in "Pending". As each item in the "Processing" state completes, another item from the "Pending" state is picked up for processing. Once an item is finished processing, it is removed from the pool of work items. This is all visible to the user in real time.
The problem I had with the previous approach was that as an item was picked up for processing, it would disappear from the view because it had been "consumed" by the call to GetConsumingEnumerable. What I really wanted was for items to be safely picked out of the "pending" pool for processing, but still remain in the view so that status updates (via INotifyPropertyChanged) could be visible in the UI.
I have addressed the problem of items disappearing from the view by actually using two concurrent collections instead, then wrapping them up as a single CompositeCollection (which I bind to instead of using the ICollectionView)
I have implemented this behaviour as below;
this.currentWorkItems = new ObservableConcurrentCollection<WorkItem>();
this.pendingWorkItems = new ObservableConcurrentCollection<WorkItem>();
this.compositeCollection = new CompositeCollection
{
new CollectionContainer { Collection = this.currentWorkItems},
new CollectionContainer { Collection = this.pendingWorkItems },
};
for (int i = 0; i < workConcurrencyFactor; i++)
{
Task.Factory.StartNew(this.ProcessWorkItems);
}
Then my Add method;
public void Add(WorkItem workItem)
{
this.pendingWorkItems.TryAdd(workItem);
}
Finally, the ProcessWorkItems method;
private void ProcessWorkItems()
{
while (true)
{
Thread.Sleep(100);
WorkItem workItem;
if (this.pendingWorkItems.TryTake(out workItem))
{
this.currentWorkItems.TryAdd(workItem);
workItem.Status = "Simulating First Step";
Thread.Sleep(1000);
workItem.Status = "Simulating Second Step";
Thread.Sleep(1000);
// Finished processing
this.currentWorkItems.TryTake(out workItem);
}
}
}
Note, I'm using ObservableConcurrentCollection from here.
This works OK, but I feel like I'm missing something here, or that I might be incurring totally unnecessary overhead by having multiple tasks sleeping and waking constantly when nothing else is really happening. Also, I feel like I'm abusing the second ObservableConcurrentCollection somewhat, by essentially just using it as a holding area for items that I'm working on, but that I still want to be visible.
Is there a better approach to this problem? What is the standard pattern for concurrent consumers to process a collection "in place", whilst avoiding multiple consumers grabbing the same item?
As Patryk already suggested this is a good example for TPL Dataflow - we do something similar (just with several steps in the pipeline including batching and transforming) here:
Create your Dataflow block to process the tasks and a collection to hold all of them:
var actionBlock = new ActionBlock<WorkItem>(item => ProcessWorkItem(item),
new ExecutionDataflowBlockOptions() { MaxDegreeOfParallelism = threadCount });
var allItems = new ConcurrentDictionary<int, WorkItem>(); // or whatever fits
Then in the the Add method:
public void Add(WorkItem workItem)
{
allItems.Add(workItem.Id, workItem);
actionBlock.Post(workItem);
}
And at the end of ProcessWorkItem do a allItems.Remove(workItem.Id).
P.S.: The dataflow blocks are pretty fast also - we do several hundred Post calls per second here...
I was wondering if following scenario could be fixed by using RX?
I have a REST service call that has an input parameter distance and loads data, this data is than inserted in the ObservableCollection of the ViewModel so that the View will show the data...
Pseudo code like this:
public async Task<int> LoadData(int distance)
{
this.ListOnUI.Clear();
var dataList = await Task.Run(() => _dataService.GetListAsync(distance));
foreach(var dataItem in dataList)
{
this.ListOnUI.Add(dataItem);
}
return dataList.Count;
}
Now this small code snippet is wrapped inside a method, that returns the count of the dataList.
What I do with that count, check if the amount returned is at least 20, if not I recall the method with a larger distance.
So what is wrong with this setup...
Each time the method is called the UI list is cleared
The user sometimes has to wait long until we reach 20 items
While we haven't reached 20 items, the UI will act weird with the clearing of the list
So my gut feeling is telling me this could be solved by using RX somehow, so that we 'chunk' load/add the UI list.
But my knowledge of RX is not good enough to solve it... so any ideas?
REMARK: When we call the LoadData service we are getting a JSON string that is then mapped to a collection of DataItems, so if we not clear the UI ObservableCollection and would just Add them with each itteration... we would get the same item multiple times in the list because it are newly constructed objects ( although with the same data ).
Is there any Key inside the data objects? If so you could check in your foreach wether the object is already contained and only add it if it's not. That way you wouldn't have to clear it (together with all side effects).
If there is no key you could create one by hashing the title + distance or whatever data fields you have that could together uniquely identify your data item and use that for the check.
Don't know wether there is a better way with reactive extensions but it should solve your case at least.
Modified to calculate a list delta each time. For contains to work correctly you just need to implement Equals appropriately on the items returned form GetListAsync. Perhaps by a contrived key comparison as SB Dev suggested. Not sure there's much Rx can bring to the table in the context of the question.
public async Task<int> LoadData(int distance)
{
int count = 0;
IList<object> dataList = null;
while (count < 20)
{
dataList = await Task.Run(() => _dataService.GetListAsync(distance));
count = dataList.Count;
var newItems = dataList.Except(ListOnUI).ToList();
var removedItems = ListOnUI.Except(dataList).ToList();
removedItems.ForEach(x => ListOnUI.Remove(x));
newItems.ForEach(ListOnUI.Add);
}
return count;
}
Assuming you are using an ObservableCollection for your list, see Sort ObservableCollection - what is the best approach? for sorting.
Thanks to the suggested answers, it got me thinking about using a Sortable Observable collection and just adding the items as they come in!
I've used the example explained by Andrea here: http://www.xamlplayground.org/post/2010/04/27/Keeping-an-ObservableCollection-sorted-with-a-method-override.aspx
But used the Binary search option noted in the comments of the blog post!
To be sure we don't stop the code when we find items already in the list, I just commented out the Throw Exception.
For this to work I only needed to implement IComparable.
I am writing a WinForms application. I am pulling data from my database, performing some actions on that data set and then plan to save it back to the database. I am using LINQ to SQL to perform the query to the database because I am only concerned with 1 table in our database so I didn't want to implement an entire ORM for this.
I have it pulling the dataset from the DB. However, the dataset is rather large. So currently what I am trying to do is separate the dataset into 4 relatively equal sized lists (List<object>).
Then I have a separate background worker to run through each of those lists, perform the action and report its progress while doing so. I have it planned to consolidate those sections into one big list once all 4 background workers have finished processing their section.
But I keep getting an error while the background workers are processing their unique list. Do the objects maintain their tie to the DataContext for the LINQ to SQL even though they have been converted to List objects? Any ideas how to fix this? I have minimal experience with multi-threading so if I am going at this completely wrong, please tell me.
Thanks guys. If you need any code snippets or any other information just ask.
Edit: Oops. I completely forgot to give the error message. In the DataContext designer.cs it gives the error An item with the same key has already been added. on the SendPropertyChanging function.
private void Setup(){
List<MyObject> quarter1 = _listFromDB.Take(5000).ToList();
bgw1.RunWorkerAsync();
}
private void bgw1_DoWork(object sender, DoWorkEventArgs e){
e.Result = functionToExecute(bgw1, quarter1);
}
private List<MyObject> functionToExecute(BackgroundWorker caller, List<MyObject> myList)
{
int progress = 0;
foreach (MyObject obj in myList)
{
string newString1 = createString();
obj.strText = newString;
//report progress here
caller.ReportProgress(progress++);
}
return myList;
}
This same function is called by all four workers and is given a different list for myList based on which worker is called the function.
Because a real answer has yet to be posted, I'll give it a shot.
Given that you haven't shown any LINQ-to-SQL code (no usage of DataContext) - I'll take an educated guess that the DataContext is shared between the threads, for example:
using (MyDataContext context = new MyDataContext())
{
// this is just some random query, that has not been listed - ToList()
// thus query execution is defered. listFromDB = IQueryable<>
var listFromDB = context.SomeTable.Where(st => st.Something == true);
System.Threading.Tasks.Task.Factory.StartNew(() =>
{
var list1 = listFromDB.Take(5000).ToList(); // runs the SQL query
// call some function on list1
});
System.Threading.Tasks.Task.Factory.StartNew(() =>
{
var list2 = listFromDB.Take(5000).ToList(); // runs the SQL query
// call some function on list2
});
}
Now the error you got - An item with the same key has already been added. - was because the DataContext object is not thread safe! A lot of stuff happens in the background - DataContext has to load objects from SQL, track their states, etc. This background work is what throws the error (because each thread is running the query, the DataContext gets accessed).
At least this is my own personal experience. Having come across the same error while sharing the DataContext between multiple threads. You only have two options in this scenario:
1) Before starting the threads, call .ToList() on the query, making listFromDB not an IQueryable<>, but an actual List<>. This means that the query has already ran and the threads operate on an actual List, not on the DataContext.
2) Move the DataContext definition into each thread. Because the DataContext is no longer shared, no more errors.
The third option would be to re-write the scenario into something else, like you did (for example, make everything sequential on a single background thread)...
First of all, I don't really see why you'd need multiple worker threads at all. (are theses lists in seperate databases / tables / servers? Do you really want to show 4 progress bars if you have 4 lists or are you somehow merging these progress reportings into one weird progress bar:D
Also, you're trying to speed up processing updates to your databases, but you don't send linq to sql any SAVES, so you're not really batching transactions, you'll just save everything at the end in one big transaction, is that really what you're aiming for? the progress bar will just stop at 100% and then spend a lot of time on the SQL side.
Just create one background thread and process everything synchronously, but batch a save transaction every couple of rows (i'd suggest something like every 1000 rows, but you should experiment with this) , it'll be fast, even with millions of rows,
If you really need this multithreaded solution:
The "another blabla with the same key has been added" error suggests that you are adding the same item to multiple "mylists", or adding the same item to the same list twice, otherwise how would there be any errors at all?
Using Parallel LINQ (PLINQ), you can take benefit of multiple CPU cores for processing your data. But if your application is going to run on single-core CPU, then splitting data into peaces wouldn't give you performance benefits instead it will incur some context-change overhead.
Hope it Helps
everyone! I searched the best I could and did not find exactly the help I was looking for.
Problem
AutoCompleteTextbox FREEZES and "eats" characters while query is performed
Asking for
Mimic Google Instant functionality
Background
First things first: C#, WPF, .NET 4.0
Ok, now that's out of the way, I'm trying to find the best way to implement a dynamic AutoComplete Textbox, which queries a database for results after each letter typed.
The following code gets executed when the AutoCompleteTextBox's TextChanged event is fired:
public void Execute(object sender, object parameter)
{
//removed some unnecessary code for the sake of being concise
var autoCompleteBox = sender as AutoCompleteTextBox;
var e = parameter as SearchTextEventArgs;
var result = SearchUnderlyings(e.SearchText);
autoCompleteBox.ItemsSource = result;
}
Now, let's say that SearchUnderlyings(e.SearchText) takes an average of 600-1100ms - during that time, the textbox is frozen and it "eats" any keys pressed. This is an annoying problem I've been having. For some reason, the LINQ in SearchUnderlyings(e.SearchText) is running in the GUI thread. I tried delegating this to a background thread, but still same result.
Ideally, I would like the textbox to work the way Google Instant does - but I don't want to be "killing" threads before the server/query can return a result.
Anyone have experience or can offer some guidance which will allow me to query as I type without freezing the GUI or killing the server?
Thank you guys!
This line:
var result = SearchUnderlyings(e.SearchText);
Runs synchronously, locking the UI thread. The way to cure this would be to switch to an asynchronous pattern, where you start the query, and then do something when it finishes.
This article demonstrates it pretty nicely, and shows some solutions - http://www.codeproject.com/KB/cs/AsyncMethodInvocation.aspx
What is probably killing you is setting the binding source over and over again (which is why running the query on a background thread doesn't make a difference).
You might consider the algorithm as a whole. Depending on your data, you could wait until the user enters the first three characters and then do one large query against the database. Bind the item source once. Each character typed afterwards just performs a filter against your data that is already cached on the client. That way you are not hitting the database over and over (which is going to be terribly expensive).
Or consider just bringing back three or so results from the DB to keep your service serialization time down.
So, we kind of hacked something quick. By making the calls to SearchUnderlyings(e.SearchText) asynchronous, my GUI thread is no longer blocked and the textbox is no longer "eating" key presses. By adding the lastQueryTag == _lastQuery check, we are trying to ensure some thread-safety, allowing only the most recent query to set the ItemsSource.
Perhaps not the most ideal or elegant solution. I am still open to further critiques and suggestions. Thank you!
private long _lastQuery = DateTime.Now.Ticks;
public void Execute(object sender, object parameter)
{
var autoCompleteBox = sender as AutoCompleteTextBox;
var e = parameter as SearchTextEventArgs;
// removed unecessary code for clarity
long lastQueryTag = _lastQuery = DateTime.Now.Ticks;
Task.Factory.StartNew(() =>
{
var result = SearchUnderlyings(e.SearchText);
System.Windows.Application.Current.Dispatch(() =>
{
if (lastQueryTag == _lastQuery)
autoCompleteBox.ItemsSource = result;
});
});
}