We Keep Coding

OrderBy items inside GroupJoin queryable by string list

results = await query
.GroupJoin(_invitations.GetAll().AsNoTracking()
.Where(i => i.GroupId == groupId),
user => user.Id,
invitation => invitation.UserId,
(a, s) => new { User = a, Invitation = s})
.SelectMany(
ai => ai.Invitation.DefaultIfEmpty(),
(a, s) => new { Users = a.User, Invitations = s }
)
.Select(i => i.Users)
.Skip(skip)
.Take(take)
.ToListAsync();
Howdy. I have quite simple group join but I can't figure out how to sort invitations inside this group join. Problem is, I don't want to make simple orderBy(i => i.creationDate) but I want to do something like (but on queryable):
var list = new List<string> {"Fall","Mid","Spring"};
return _db.MiaLog1A.Where(m => m.Campus == selectedCampus)
.AsEnumerable()
.OrderBy(m => m.StudentName)
.ThenBy(m=> list.IndexOf(m.Term));
I need to order them by specific strings. Because I have a scenario where newest invitation isn't what I need. I want accepted invitation status first. Is there a way to do this?
I've tried things like that:
.OrderBy(i => sortOrderList.IndexOf(i.InvitationsStatus) in many places but it just throws.
Thank you in advance.

A pattern of
.OrderBy(m => m.StudentName)
.ThenBy(m => m.X == "y" ? 0 : (m.X == "x" ? 1:2))
Should translate if you want to sort by "y","x","z"
But I note that both examples you given; (fall, mid, spring) and (approved vs pending / not invited) are sorted in that order anyway.. you don't need to sort by the list index of these, you can just sort the strings ascending

Can this query about finding missing keys be improved? (either SQL or LINQ)

I am developing a ASP.NET MVC website and is looking a way to improve this routine. It can be improved either at LINQ level or SQL Server level. I hope at best we can do it within one query call.
Here is the tables involved and some example data:
We have no constraint that every Key has to have each LanguageId value, and indeed the business logic does not allow such contraint. However, at application level, we want to warn the admin that a key is missing a/some language values. So I have this class and query:
public class LocalizationKeyWithMissingCodes
{
public string Key { get; set; }
public IEnumerable<string> MissingCodes { get; set; }
}
This method get the Key list, as well as any missing codes (for example, if we have en + jp + ch language codes, and the key only has values for en + ch, the list will contains jp):
public IEnumerable<LocalizationKeyWithMissingCodes> GetAllKeysWithMissingCodes()
{
var languageList = Utils.ResolveDependency<ILanguageRepository>().GetActive();
var languageIdList = languageList.Select(q => q.Id);
var languageIdDictionary = languageList.ToDictionary(q => q.Id);
var keyList = this.GetActive()
.Select(q => q.Key)
.Distinct();
var result = new List<LocalizationKeyWithMissingCodes>();
foreach (var key in keyList)
{
// Get missing codes
var existingCodes = this.Get(q => q.Active && q.Key == key)
.Select(q => q.LanguageId);
// ToList to make sure it is processed at application
var missingLangId = languageList.Where(q => !existingCodes.Contains(q.Id))
.ToList();
result.Add(new LocalizationKeyWithMissingCodes()
{
Key = key,
MissingCodes = missingLangId
.Select(q => languageIdDictionary[q.Id].Code),
});
}
result = result.OrderByDescending(q => q.MissingCodes.Count() > 0)
.ThenBy(q => q.Key)
.ToList();
return result;
}
I think my current solution is not good, because it make a query call for each key. Is there a way to improve it, by either making it faster, or pack within one query call?
EDIT: This is the final query of the answer:
public IQueryable<LocalizationKeyWithMissingCodes> GetAllKeysWithMissingCodes()
{
var languageList = Utils.ResolveDependency<ILanguageRepository>().GetActive();
var localizationList = this.GetActive();
return localizationList
.GroupBy(q => q.Key, (key, items) => new LocalizationKeyWithMissingCodes()
{
Key = key,
MissingCodes = languageList
.GroupJoin(
items,
lang => lang.Id,
loc => loc.LanguageId,
(lang, loc) => loc.Any() ? null : lang)
.Where(q => q != null)
.Select(q => q.Code)
}).OrderByDescending(q => q.MissingCodes.Count() > 0) // Show the missing keys on the top
.ThenBy(q => q.Key);
}

Another possibility, using LINQ:
public IEnumerable<LocalizationKeyWithMissingCodes> GetAllKeysWithMissingCodes(
List<Language> languages,
List<Localization> localizations)
{
return localizations
.GroupBy(x => x.Key, (key, items) => new LocalizationKeyWithMissingCodes
{
Key = key,
MissingCodes = languages
.GroupJoin( // check if there is one or more match for each language
items,
x => x.Id,
y => y.LanguageId,
(x, ys) => ys.Any() ? null : x)
.Where(x => x != null) // eliminate all languages with a match
.Select(x => x.Code) // grab the code
})
.Where(x => x.MissingCodes.Any()); // eliminate all complete keys
}

Here is the SQL logic to identify the keys that are missing "complete" language assignments:
SELECT
all.[Key],
all.LanguageId
FROM
(
SELECT
loc.[Key],
lang.LanguageId
FROM
Language lang
FULL OUTER JOIN
Localization loc
ON (1 = 1)
WHERE
lang.Active = 1
) all
LEFT JOIN
Localization loc
ON (loc.[Key] = all.[Key])
AND (loc.LanguageId = all.LanguageId)
WHERE
loc.[Key] IS NULL;
To see all keys (instead of filtering):
SELECT
all.[Key],
all.LanguageId,
CASE WHEN loc.[Key] IS NULL THEN 1 ELSE 0 END AS Flagged
FROM
(
SELECT
loc.[Key],
lang.LanguageId
FROM
Language lang
FULL OUTER JOIN
Localization loc
ON (1 = 1)
WHERE
lang.Active = 1
) all
LEFT JOIN
Localization loc
ON (loc.[Key] = all.[Key])
AND (loc.LanguageId = all.LanguageId);

your code seems to be doing a lot of database query and materialization..
in terms of LINQ, the single query would look like this..
we take the cartesian product of language and localization tables to get all combinations of (key, code) and then subtract the (key, code) tuples that exist in the relationship. this gives us the (key, code) combination that don't exist.
var result = context.Languages.Join(context.Localizations, lang => true,
loc => true, (lang, loc) => new { Key = loc.Key, Code = lang.Code })
.Except(context.Languages.Join(context.Localizations, lang => lang.Id,
loc => loc.LanguageId, (lang, loc) => new { Key = loc.Key, Code = lang.Code }))
.GroupBy(r => r.Key).Select(r => new LocalizationKeyWithMissingCodes
{
Key = r.Key,
MissingCodes = r.Select(kc => kc.Code).ToList()
})
.ToList()
.OrderByDescending(lkmc => lkmc.MissingCodes.Count())
.ThenBy(lkmc => lkmc.Key).ToList();
p.s. i typed this LINQ query on the go, so let me know if it has syntax issues..
the gist of the query is that we take a cartesian product and subtract matching rows.

EF linq Find in list where objectA.prop1 is any of objectB.prop2

I have some question about a linq query I cannot manage to do. It's matter of finding objects
Consider the List (ListA), where i want to select all rows where the value of field F20 is in a list (ListB). For each of these items, i will then replace the "searched value" by it's "replacement value".
//StringReplacer(SearchValue,ReplacementValue)
List<StringReplacer> ListB = new List<StringReplacer>();
ListB.Add(new StringReplacer("JAN", "24/01/2015"));
ListB.Add(new StringReplacer("FEB", "24/02/2015"));
ListB.Add(new StringReplacer("MAR", "24/03/2015"));
ListB.Add(new StringReplacer("APR", "24/04/2015"));
List<XLS2015> ListA = new List<XLS2015>();
ListA = db.XLS2015.Where(y => (ListB.Any(y.F20.Any(tt => tt.SearchValue.Contains)))).ToList();
ListA = db.XLS2015.Where(y => (ListB.Any(y.F20.Contains(tt => tt.Source)))).ToList();
I could do that with foreach but i think a linq statement would achieve that better.
I already achieved to do it when ListBisList`:
ListA = ListA.Where(c => (ListB.Any(c.F20.ToString().Contains))).ToList();
but i can't figure out how to deal with an attribute of the object...
As an extra question, Instead of searching values (Any(), Contains()), is there any function that may "SearchAndReplace" in one shot ?

You can only select on DB side, and replace on .NET side. You can still do it in single LINQ query, but it will only partialy be executed on SQL.
You can try this:
// To make it easier to create sql query put all search string in array
string[] search = ListB.Select(b => b.Search).ToArray();
List<XLS2015> ListA = db.XLS2015
.Where(a => search.Contains(a.F20))
.AsEnumerable() // execute on server at this point, and do rest in C#
.Join(ListB, a => a.F20, b => b.Search, (a, b) => new { A = a, B = b })
.Select(x => {
// do replacement
x.A.F20 = x.A.F20.Replace(x.B.Search, x.B.Replacement);
return x.A;
})
.ToList();

Difficulty with LINQ Query writing custom sort logic

I have files like Avinash_Create.sql, Avinash_Insert.sql, Avinash_Update.sql , Avinash_Delete.sql.
I need to iterate over the files list and group them based on the name and order by create, insert, update, and delete files.
I am finding it difficult difficult to accomplish. This is what I have so far:
var userGroups = shortfilenames.GroupBy(s => s.Substring(0, s.IndexOf('_')))
.Select(g => g.OrderBy(x => x.Substring(x.IndexOf('_')).Contains("CREATE"))
.ThenBy(x => x.Substring(x.IndexOf('_')).Contains("INSERT"))
.ThenBy(x => x.Substring(x.IndexOf('_')).Contains("UPDATE"))
.ThenBy(x => x.Substring(x.IndexOf('_')).Contains("DELETE")));
The above query is grouping by name 'Avinash' but not working for custom ordering. Please help.
Update:
Please see updated query , still it is not sorting properly

Uses Split to extract the relevant sections of the string. Converts the result to upper case, which seems to be missing from your attempt. Additionally makes it a bit shorter by using an array to hold your custom sort order and then Array.IndexOf to get a sort order from it, rather than multiple OrderBy/ThenBy.
var ordering = new [] {"CREATE", "INSERT", "UPDATE", "DELETE"};
var results = shortfilenames.GroupBy(s => s.Split('_')[0])
.Select(g => g.OrderBy(x => Array.IndexOf(ordering, x.Split('_')[1].Split('.')[0].ToUpper())));

This seems to be what what you want:
var shortfilenames = new List<string>(){"Avinash_Create.sql" , "Avinash_Insert.sql" , "Avinash_Update.sql" , "Avinash_Delete.sql"};
var userGroups = shortfilenames
.Select(fn =>
{
string fileName = Path.GetFileNameWithoutExtension(fn);
string[] nameAndAction = fileName.Split('_');
return new
{
extension = Path.GetExtension(fn),
fileName,
name = nameAndAction[0],
action = nameAndAction[1]
};
})
.GroupBy(x => x.name)
.Select(g => g.OrderByDescending(x => x.action.Equals("CREATE", StringComparison.InvariantCultureIgnoreCase))
.ThenByDescending(x => x.action.Equals("INSERT", StringComparison.InvariantCultureIgnoreCase))
.ThenByDescending(x => x.action.Equals("UPDATE", StringComparison.InvariantCultureIgnoreCase))
.ThenByDescending(x => x.action.Equals("DELETE", StringComparison.InvariantCultureIgnoreCase))
.ToList());
foreach (var ug in userGroups)
foreach (var x in ug)
Console.WriteLine("{0} {1}", x.name, x.action);
prints out:
Avinash Create
Avinash Insert
Avinash Update
Avinash Delete
Presumes that the file-names always contain the underscore.

Rx how to group by a key a complex object and later do SelectMany without "stopping" the stream?

This is related to my other question here. James World presented a solution as follows:
// idStream is an IObservable<int> of the input stream of IDs
// alarmInterval is a Func<int, TimeSpan> that gets the interval given the ID
var idAlarmStream = idStream
.GroupByUntil(key => key, grp => grp.Throttle(alarmInterval(grp.Key)))
.SelectMany(grp => grp.IgnoreElements().Concat(Observable.Return(grp.Key)));
<edit 2:
Question: How do I start the timers immediately without waiting for the first events to arrive? That's the root problem in my question, I guess. For that end, I planned on sending off dummy objects with the IDs I know should be there. But as I write in following, I ended up with some other problems. Nevertheless, I'd think solving that too would be interesting.
Forwards with the other interesting parts then! Now, if I'd like to group a complex object like the following and group by the key as follows (won't compile)
var idAlarmStream = idStream
.Select(i => new { Id = i, IsTest = true })
.GroupByUntil(key => key.Id, grp => grp.Throttle(alarmInterval(grp.Key)))
.SelectMany(grp => grp.IgnoreElements().Concat(Observable.Return(grp.Key)));
then I get into trouble. I'm unable to modify the part about SelectMany, Concat and Observable.Return so that the query would work as before. For instance, if I make query as
var idAlarmStream = idStream
.Select(i => new { Id = i, IsTest = true })
.GroupByUntil(key => key.Id, grp => grp.Throttle(alarmInterval(grp.Key)))
.SelectMany(grp => grp.IgnoreElements().Concat(Observable.Return(grp.Key.First())))
.Subscribe(i => Console.WriteLine(i.Id + "-" + i.IsTest);
Then two events are needed before an output can be observed in the Subscribe. It's the effect of the call to First, I gather. Furthermore, I woul like to use the complex object attributes in the call to alarmInterval too.
Can someone offer an explanation what's going on, perhaps even a solution? The problem in going with unmodified solution is that the grouping doesn't look Ids alone for the key value, but also the IsTest field.
<edit: As a note, the problem probably could be solved firsly by creating an explicit class or struct and then that implements a custom IEquatable and secondly then using James' code as-is so that grouping would happen by IDs alone. It feels like hack though.

Also, if you want to count the number of times you've seen an item before the alarm goes off you can do it like this, taking advantage of the counter overload in Select.
var idAlarmStream = idStream
.Select(i => new { Id = i, IsTest = true })
.GroupByUntil(key => key.Id, grp => grp.Throttle(alarmInterval(grp.Key))
.SelectMany(grp => grp.Select((count, alarm) => new { count, alarm }).TakeLast(1));
Note, this will be 0 for the first (seed) item - which is probably what you want anyway.

You are creating an anonymous type in your Select. Lets call it A1. I will assume your idStream is an IObservable. Since this is the Key in the GroupByUntil you do not need to worry about key comparison - int equality is fine.
The GroupByUntil is an IObservable<IGroupedObservable<int, A1>>.
The SelectMany as written is trying to be an IObservable<A1>. You need to just Concat(Observable.Return(grp.Key)) here - but the the type of the Key and the type of the Group elements must match or the SelectMany won't work. So the key would have to be an A1 too. Anonymous types use structural equality and the return type would be stream of A1 - but you can't declare that as a public return type.
If you just want the Id, you should add a .Select(x => x.Id) after the Throttle:
var idAlarmStream = idStream
.Select(i => new { Id = i, IsTest = true })
.GroupByUntil(key => key.Id, grp => grp.Throttle(alarmInterval(grp.Key)
.Select(x => x.Id))
.SelectMany(grp => grp.IgnoreElements().Concat(Observable.Return(grp.Key)));
If you want A1 instead - you'll need to create a concrete type that implements Equality.
EDIT
I've not tested it, but you could also flatten it more simply like this, I think this is easier! It is outputing A1 though, so you'll have to deal with that if you need to return the stream somewhere.
var idAlarmStream = idStream
.Select(i => new { Id = i, IsTest = true })
.GroupByUntil(key => key.Id, grp => grp.Throttle(alarmInterval(grp.Key))
.SelectMany(grp => grp.TakeLast(1));