We Keep Coding

Sorted and indexed WinForms ListBox items

There is Client app with ListBox containing Records sorted by Record Time attribute. On application start Client loads Records from server and start listening to server updates. When server inform Client about new Record Client add Record to ListBox (theoretically with Time before last showed Recort Time). When Server inform about update or delete Client find message by ID and update or delete it (Theoretically when Time of Record is changed order of Records in ListBox must be changed).
I guess somethink like sorted dictionary with value time comparator is required.
public partial class RecordsForm : Form
private System.Windows.Forms.ListBox recordsListBox;
peivate SortedDictionary<long, Record> recordsDictionary;
public RecordsForm()
{
InitializeComponent();
// this is not working because comparer must compare keys
recordsDictionary = new SortedDictionary<long, Record>(new RecordComparer());
var recordsBbinding = new BindingSource();
recordsBbinding.DataSource = recordsDictionary;
recordsListBox.DataSource = recordsBbinding;
}
public HandleCreateUpdate(Record record)
{
recordsDictionary[record.Id] = record;
}
public HandleDelete(Record record)
{
if (recordsDictionary.ContainsKey(record.Id))
{
recordsDictionary.Remove(record.Id);
}
}
}
class Record {
public long Id { get; set; }
public DateTime Time { get; set; }
public String Title { get; set; }
}
class RecordComparer : Comparer<Record>
{
public override int Compare(Record left, Record right)
{
return left.Time.CompareTo(right.Time);
}
}
Or exists another pattern used for something like this?
EDIT: Added screen
List of Records is always sorted by Time desc. I want to synchronize this list across clients. When one edit/add/delete record other clients may reflects changes without reloading entire list or iterating over all items.

Index was outside the bounds of the array error after a long idle time

I have class like this that reads the data from database once and keeps it in memory.
public class PlanetService
{
private List<Planet> planets = new List<Planet>();
public List<Planet> FindAll()
{
if (!planets.Any())
{
planets.AddRange(GetAllPlanetsFromDataBase());
}
return planets;
}
public List<Planet> FindByName(string name)
{
if (!planets.Any())
{
planets.AddRange(GetAllPlanetsFromDataBase());
}
return planets.Where(x => x.Name == name).ToList();
}
}
public class Planet
{
public Guid Id { get; set; }
public string Name { get; set; }
}
public class Lookup
{
public Lookup(string value, string label)
{
Value = value;
Label = label;
}
public string Value { get; set; }
public string Label { get; set; }
}
This is being used as follows
var lookupList = planetService.FindAll().Select(x => new Lookup(x.Id.ToString(), x.Name)).ToList();
This works as expected initially. But every now and then, usually after a long idle time, the following error occurs
[IndexOutOfRangeException: Index was outside the bounds of the
array.] System.Collections.Generic.Enumerator.MoveNext()
System.Linq.WhereSelectListIterator'2.MoveNext()
System.Collections.Generic.List'1..ctor(IEnumerable'1 collection)
System.Linq.Enumerable.ToList(IEnumerable'1 source)
There is always planets in the database. And even if not it should return an empty Lookup list and shouldn't be throwing an IndexOutOfRangeException, right?
Once this error occurs, subsequent page refreshes keeps showing this error and won't go away for sometime.
Since other database calls are working I assume this is not connection related.
This happens in production only, I am unable to recreate the issue in the dev environment yet.

This exception in my experience tends to come when you try to Serialize massive amounts of data, usually massive single columns from the database.
Are any of the columns you're retrieving from the database containing abnormally large amounts of data?
We had a REST API which had this same exception where the cause was a column in the DB which had grown to GB's in size due to a self referencing bug.

Using Contains() in a Realm query

Let's say we have a realm results taken with
RealmDb.All<Entry>();
Then I want to do some search over those results using not yet supported techniques, like StartsWith on a function return or on a property which is not mapped in realm etc, so I get a subset
IEnumerable<Entry> subset = bgHaystack;
var results = subset.Where(entry => entry.Content.ToLower().StartsWith(needle));
To get somehow these as part of RealmResults, I extract the entry ids like this:
List<int> Ids = new List<int>();
foreach (Entry entry in entries)
{
Ids.Add(entry.Id);
}
return Ids;
and finally I want to return a subset of RealmResults (not IEnumerable) of only those Entries that contain those ids, how can I do that? IDE says the Contains method is not supported.
Can I use some kind of predicate or a comparer for that?
Entry is my model class
using System.ComponentModel.DataAnnotations.Schema;
using Realms;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System;
namespace Data.Models
{
[Table("entry")]
public class Entry : RealmObject
{
public class EntryType
{
public const byte Word = 1;
public const byte Phrase = 2;
public const byte Text = 3;
};
[Key]
[PrimaryKey]
[Column("entry_id")]
public int Id { get; set; }
[Column("user_id")]
public int UserId { get; set; }
[Column("source_id")]
public int SourceId { get; set; }
[Indexed]
[Column("type")]
public byte Type { get; set; }
[Column("rate")]
public int Rate { get; set; }
[Column("created_at")]
public string CreatedAt { get; set; }
[Column("updated_at")]
public string UpdatedAt { get; set; }
[NotMapped]
public Phrase Phrase { get; set; }
[NotMapped]
public Word Word { get; set; }
[NotMapped]
public Text Text { get; set; }
[NotMapped]
public IList<Translation> Translations { get; }
[NotMapped]
public string Content
{
get {
switch (Type)
{
case EntryType.Phrase:
return Phrase?.Content;
case EntryType.Word:
return Word?.Content;
case EntryType.Text:
return Text?.Content;
}
return "";
}
}
}
}

According to the documentation, Realm .NET supports LINQ, so that's promising. In your specific example, you indicate that StartsWith isn't supported, but I see that on the above page, specifically here.
Now, your example makes clear that Entry is a RealmObject, so it's not clear where you'd possibly get a RealmResult from (nor does their documentation on that page mention a RealmResult). Specifically, the home page indicates that you're really only going to ever work with Realm, RealmObject and Transaction, so I'm going to just assume that you meant that you'll need a resulting RealmObject per their examples.
The way you presently have your data object set up, you're rather stuck calling it like you are (though if I could make a recommendation to simplify it a little bit:
var entries = RealmDb.All<Entry>().ToList();
var results = entries.Where(entry => entry.Content.ToLower().StartsWith(needle));
var ids = results.Select(a => a.Id).ToList();
Now, your big issue with just combining the filter predicate in line 2 with the end of line 1: Content itself is marked with a [NotMapped] attribute. Per the documentation again:
As a general rule, you can only create predicates with conditions that
rely on data in Realm. Imagine a class
class Person : RealmObject
{
// Persisted properties
public string FirstName { get; set; }
public string LastName { get; set; }
// Non-persisted property
public string FullName => FirstName + " " + LastName;
}
Given this class, you can create queries with conditions that apply to
the FirstName and LastName properties but not to the FullName
property. Likewise, properties with the [Ignored] attribute cannot be
used.
Because you're using [NotMapped], I've got to believe that's going to behave similarly to [Ignored] and further, because it's just a computed value, it's not something that Realm is going to be able to process as part of the query - it simply doesn't know it because you didn't map it to the information Realm is storing. Rather, you'll have to compute the Content property when you've actually got the instances of your Entry objects to enumerate through.
Similarly, I expect you'll have issues pulling values from Phrase, Word and Text since they're also not mapped, and thus not stored in the record within Realm (unless you're populating those in code you didn't post before executing your Where filter).
As such, you might instead consider storing separate records as a PhraseEntry, WordEntry, and TextEntry so you can indeed perform exactly that filter and execute it on Realm. What if you instead used the following?
public class Entry : RealmObject
{
[Key]
[PrimaryKey]
[Column("entry_id")]
public int Id { get; set; }
[Column("user_id")]
public int UserId { get; set; }
[Column("source_id")]
public int SourceId { get; set; }
[Column("rate")]
public int Rate { get; set; }
[Column("created_at")]
public string CreatedAt { get; set; }
[Column("updated_at")]
public string UpdatedAt { get; set; }
[Column("content")]
public string Content { get; set; }
[NotMapped]
public IList<Translation> Translations { get; }
}
[Table("wordEntry")]
public class WordEntry : Entry
{
}
[Table("phraseEntry")]
public class PhraseEntry : Entry
{
}
[Table("textEntry")]
public class TextEntry : Entry
{
}
And now, you can offload the filtering to Realm:
var wordEntries = RealmDb.All<WordEntry>.Where(entry =>
entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var phraseEntries = RealmDb.All<PhraseEntry>.Where(entry => entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var textEntries = RealmDb.All<TextEntry>.Where(entry => entry.Content.StartsWith(needle, StringComparison.OrdinalIgnoreCase)).ToList();
var entries = new List<Entry>();
entries.AddRange(wordEntries);
entries.AddRange(phraseEntries);
entries.AddRange(textEntries);
var ids = entries.Select(entry => entry.Id).ToList();
It's not quite as brief as storing it all in one table, but I'm not immediately seeing any Realm documentation that indicates support for executing the same query against multiple tables simultaneously, so at least this would allow you to leave the filtering to the database and work against a more limited subset of values locally.
Finally, so we have all that and I missed your final question up top. You indicate that you want to return a subset of your entries based on some collection of ids you create. In the logic you provide, you're retrieving all the Id properties in all your results, so there's really no further subset to pull.
That said, let's assume you have a separate list of ids that for whatever complicated reason, you were only able to derive after retrieving the list of Entry types from above (themselves all PhraseEntry, WordEntry or TextEntry objects).
At this point, since you've already pulled all the values from Realm and have them locally, just execute another Where statement against them. Because a List implements IEnumerable, you can thus execute the LINQ locally without any of the Realm restrictions:
var myLimitedIdSet = new List<int>()
{
10, 15, 20, 25 //Really complicated logic to narrow these down locally
};
var resultingEntries = entries.Where(entry => myLimitedIdSet.Contains(entry.Id)).ToList();
And you're set. You'll have only those entries that match the IDs listed in myLimitedIdSet.
Edit to address comment
You see this error because of the detail provided at the top of this page in the documentation. Specifically (and adapting to your code):
The first statement gives you a new instance of Entry of a class that implements IQueryable... This is standard LINQ implementation - you get an object representing the query. The query doesn't do anything until you made a further call that needs to iterate or count the results.
Your error is then derived by taking the result from RealmDb.All<Entry>() and trying to cast it to an IEnumerable<Entry> to operate against it as though you have local data. Until you call ToList() onRealmDb.All` you simply have a LINQ representation of what the call will be, not the data itself. As such, when you further refine your results with a Where statement, you're actually adding that to a narrowed version of the IQueryable statement, which will also fail because you lack the appropriate mapping in the Realm dataset.
To skip the optimization I provided above, the following should resolve your issue here:
var bgHaystack = realm.All<Entry>().ToList(); //Now you have local data
var results = bgHaystack.Where(entry => entry.Content.ToLower().StartsWith(needle));
Unfortunately, given your provided code, I don't expect that you'll see any matches here unless needle is an empty string. Not only is your Content property not part of the Realm data and you thus cannot filter on it within Realm, but neither are your Phrase, Word or Text properties mapped either. As a result, you will only ever see an empty string when getting your Content value.
You can further refine the results variable above to yield only those instances with a provided ID as you see fit with normal LINQ (as again, you'll have pulled the data from Realm in the first line).
var limitedIds = new List<int>{10, 20, 30};
var resultsLimitedById = results.Select(a => limitedIds.Contains(a.Id)).ToList();
I've updated my examples above to reflect the use of ToList() in the appropriate places as well.

Wpf - How to control observable collection updates

In the parent there is a Observable Collection PendingPayment that has a list of all pending payments of sales with a column amount paid.
Then the user can select a particular sale and open it in new child window.
The thing thats going wrong is if the user just edits the text box paid amount in child window and closes the window without saving the new paid amount to database,the observable collection containing Amount paid column in the parent window gets updated.
What I want is it the collection to get updated only when the values are updated in the database.

This can be achieved by creating a copy of your sale object when the user select it in the list, and then using this copy as the view model of your child view.
You will then be able to set the new values in the original object from your list only once the save button has been clicked and the database update succeed.
An other way to proceed if you need to edit only few of the object properties would be to create and editor object and use it as the child window's view model.
Something like this :
public class Sale
{
public int PaidAmount { get; set; }
public int Some { get; set; }
public int More { get; set; }
public int Properties { get; set; }
}
public class SaleEditor
{
private Sale _sale;
public int PaidAmount { get; set; }
public SaleEditor(Sale sale)
{
_sale = sale;
PaidAmount = sale.PaidAmount;
}
public void Save()
{
// update your data here
_sale.PaidAmount = PaidAmount;
}
}
If you need your original object to update the database, then the save method could first update the object and the revert the changes if DB update failed :
public void Save()
{
var oldAmount = _sale.PaidAmount;
_sale.PaidAmount = PaidAmount;
if (!SalesDB.Update(_sale))
_sale.PaidAmount = oldAmount;
// you could also read back the value from DB
}

Whenever possible (I've never see a reason why it cannot),for listing purpose use proxy or flatted objects, you can implement this using projections query. Then user select an item from a list and the only thing you need to grab is a key to load the full object with its required object graph as the use case might dictate.
Here is a sample implementation using Entity Framework and c# lambda expressions:
Using anonymous object:
var anonymousListProjection = DbContext.PendingPayments.Select( pp=>
new { pp.Order, pp.Amount})
Using a hardcoded proxy:
var hardcodedListProjection = DbContext.PendingPayments.Select( pp=>
new PendingPaymentProxy { Order = pp.Order, Amount = pp.Amount})
//To return an observable:
var observableColl = new ObservableCollection<PendingPaymentProxy>
(hardcodedListProjection.Tolist());
public class PendingPaymentProxy
{
public string Order { get; set; }
public decimal Amount{ get; set; }
}
Apart from avoiding possibles performance problems due to unintentional loading real objects, this way you only have to worry for your list when the user do save in the detail view.

Code-First Entity Framework w/ Stored Procedure returning results from complex Full-text Searches

I am looking for design advice for the following scenario:
I have a code-first EF5 MVC application. I am building a full-text search function which will incorporate multiple weighted columns from many tables. As I cannot create view with an index from these tables (some of them contain text / binary columns), I have created a stored procedure which will output the ID of my object (eg. PersonID) and the rank associated with that object based on the search terms.
My current approach is to create a helper class for executing full text searches which call the stored procedure(s) and load all the objects from the context based on the returned IDs.
My questions are:
Does my approach seem sensible / follow reasonable best practice?
Has anyone else done something similar with any lessons learned?
Is there a way to do this more efficiently (i.e. have the results of the stored procedure return/map to the entities directly without an additional look-up required?)
UPDATE
Moved my detailed implementation from an edit of the question into its own answer to be more in line with what is recommended frequently # meta.stackexchange.com

Seeing as you can't use SQL methods like containstable with entityframework code first which the rest of your application could be using you could be 'forced' to do something with a storedprocedure like your describe. Whether it's best practice I don't know. However it it gets the job done I don't see why it wouldn't be sensible.
Yes - I have and still am working on a project build around EF codefirst where I had to do a fairly complex search that included several search parameters marked as 'must have' and several values marked as 'nice to have' and in from that return a weighted result.
Depending on the complexity of the result set I don't think you need to do a second roundtrip to the database and I will show you a way I have been doing it below.
Bear in mind that below is simply an example:
public List<Person> GetPeople(params string[] p)
{
var people = new List<Person>();
using (var db = new DataContext())
{
var context = ((IObjectContextAdapter)db).ObjectContext;
db.Database.Connection.Open();
var command = db.Database.Connection.CreateCommand();
command.CommandText = "SomeStoredProcedureReturningWeightedResultSetOfPeople";
command.CommandType = System.Data.CommandType.StoredProcedure;
//Add parameters to command object
people = context.Translate<Person>(command.ExecuteReader()).ToList();
}
return people;
}
Even though the storedprocedure will have a column for the weight value it won't get mapped when you translate it.
You could potentially derive a class from Person that includes the weight value if you needed it.

Posting this as an answer rather than an edit to my question:
Taking some of the insight provided by #Drauka's (and google) here is what I did for my initial iteration.
Created the stored procedure to do the full text searching. It was really too complex to be done in EF even if supported (as one example some of my entities are related via business logic and I wanted to group them returning as a single result). The stored procedure maps to a DTO with the entity id's and a Rank.
I modified this blogger's snippet / code to make the call to the stored procedure, and populate my DTO: http://www.lucbos.net/2012/03/calling-stored-procedure-with-entity.html
I populate my results object with totals and paging information from the results of the stored procedure and then just load the entities for the current page of results:
int[] projectIDs = new int[Settings.Default.ResultsPerPage];
foreach (ProjectFTS_DTO dto in
RankedSearchResults
.Skip(Settings.Default.ResultsPerPage * (pageNum - 1))
.Take(Settings.Default.ResultsPerPage)) {
projectIDs[index] = dto.ProjectID;
index++;
}
IEnumerable<Project> projects = _repository.Projects
.Where(o=>projectIDs.Contains(o.ProjectID));
Full Implementation:
As this question receives a lot of views I thought it may be worth while to post more details of my final solution for others help or possible improvement.
The complete solution looks like:
DatabaseExtensions class:
public static class DatabaseExtensions {
public static IEnumerable<TResult> ExecuteStoredProcedure<TResult>(
this Database database,
IStoredProcedure<TResult> procedure,
string spName) {
var parameters = CreateSqlParametersFromProperties(procedure);
var format = CreateSPCommand<TResult>(parameters, spName);
return database.SqlQuery<TResult>(format, parameters.Cast<object>().ToArray());
}
private static List<SqlParameter> CreateSqlParametersFromProperties<TResult>
(IStoredProcedure<TResult> procedure) {
var procedureType = procedure.GetType();
var propertiesOfProcedure = procedureType.GetProperties(BindingFlags.Public | BindingFlags.Instance);
var parameters =
propertiesOfProcedure.Select(propertyInfo => new SqlParameter(
string.Format("#{0}",
(object) propertyInfo.Name),
propertyInfo.GetValue(procedure, new object[] {})))
.ToList();
return parameters;
}
private static string CreateSPCommand<TResult>(List<SqlParameter> parameters, string spName)
{
var name = typeof(TResult).Name;
string queryString = string.Format("{0}", spName);
parameters.ForEach(x => queryString = string.Format("{0} {1},", queryString, x.ParameterName));
return queryString.TrimEnd(',');
}
public interface IStoredProcedure<TResult> {
}
}
Class to hold stored proc inputs:
class AdvancedFTS :
DatabaseExtensions.IStoredProcedure<AdvancedFTSDTO> {
public string SearchText { get; set; }
public int MinRank { get; set; }
public bool IncludeTitle { get; set; }
public bool IncludeDescription { get; set; }
public int StartYear { get; set; }
public int EndYear { get; set; }
public string FilterTags { get; set; }
}
Results object:
public class ResultsFTSDTO {
public int ID { get; set; }
public decimal weightRank { get; set; }
}
Finally calling the stored procedure:
public List<ResultsFTSDTO> getAdvancedFTSResults(
string searchText, int minRank,
bool IncludeTitle,
bool IncludeDescription,
int StartYear,
int EndYear,
string FilterTags) {
AdvancedFTS sp = new AdvancedFTS() {
SearchText = searchText,
MinRank = minRank,
IncludeTitle=IncludeTitle,
IncludeDescription=IncludeDescription,
StartYear=StartYear,
EndYear = EndYear,
FilterTags=FilterTags
};
IEnumerable<ResultsFTSDTO> resultSet = _context.Database.ExecuteStoredProcedure(sp, "ResultsAdvancedFTS");
return resultSet.ToList();
}