I have the following query:
var countA=await _importContext.table1.CountAsync(ssc => ssc.ImportId == importId)
var countB=await _importContext.table2.CountAsync(ssc => ssc.ImportId == importId)
var countC=await _importContext.table3.CountAsync(ssc => ssc.ImportId == importId)
var countD=await _importContext.table4.CountAsync(ssc => ssc.ImportId == importId)
There are 9 more count from different tables. Is there a way to summarize the query in terms of optimizing & removing redundancy?
I tried wrapping up the queries like:
var result = new
{
countA = context.table1.Count(),
countB = context.table2.Count(),
.....
};
but this takes more time than the first one above.
You can't really optimise it as you seem to need the counts from all of the tables. Your second method of getting the data still calls the database the same amount of times as the first but also creates an object containing all of the counts so is likely to take longer.
The only thing you can really do to make it faster is to get the data in parallel but this might be overkill. I would just go with your first option unless it's really slow.
You can create such query via gouping by constant and Concat operator:
Helper class:
public class TableResult
{
public string Name { get; set; }
public int Count { get; set; }
}
Query:
var query = _importContext.table1.Where(ssc.ImportId == importId).GroupBy(x => 1).Select(g => new TableResult { Name = "table1", Count = g.Count() })
.Concat(_importContext.table2.Where(ssc.ImportId == importId).GroupBy(x => 1).Select(g => new TableResult { Name = "table2", Count = g.Count() }))
.Concat(_importContext.table3.Where(ssc.ImportId == importId).GroupBy(x => 1).Select(g => new TableResult { Name = "table3", Count = g.Count() }))
.Concat(_importContext.table4.Where(ssc.ImportId == importId).GroupBy(x => 1).Select(g => new TableResult { Name = "table4", Count = g.Count() }));
var result = await query.ToListAsync();
Related
Currently I have this LINQ query which calculate the totalcount of a parent table (CafeTables) from child tables (CafeTableDetails). These code works. But somehow, I believe these code can the shorten.
var selectedTable = db.CafeTables.Where(c => c.TableNo.Equals(userName)).SingleOrDefault();
var selectedTableDetailsRaw = db.CafeTableDetails.
Where(cd => cd.CafeTableId == selectedTable.Id);
selectedTable.TotalOrders = selectedTableDetailsRaw.Count();
I think you can try to use linq JOIN and GroupBy to make it.
var result = db.CafeTables.Where(c => c.TableNo == userName)
.Join(db.CafeTableDetails.GroupBy(x=>x.CafeTableId)
.Select(g => new { CafeTableId = g.Key, cnt = g.Count() }),
st => st.Id,
cd => cd.CafeTableId,
(st,cd) => new
{
st.Id,
cd.cnt
//..... your expect property
});
I have this document, a post :
{Content:"blabla",Tags:["test","toto"], CreatedOn:"2019-05-01 01:02:01"}
I want to have a page that displays themost used tags since the last 30 days.
So far I tried to create an index like this
public class Toss_TagPerDay : AbstractIndexCreationTask<TossEntity, TagByDayIndex>
{
public Toss_TagPerDay()
{
Map = tosses => from toss in tosses
from tag in toss.Tags
select new TagByDayIndex()
{
Tag = tag,
CreatedOn = toss.CreatedOn.Date,
Count = 1
};
Reduce = results => from result in results
group result by new { result.Tag, result.CreatedOn }
into g
select new TagByDayIndex()
{
Tag = g.Key.Tag,
CreatedOn = g.Key.CreatedOn,
Count = g.Sum(i => i.Count)
};
}
}
And I query it like that
await _session
.Query<TagByDayIndex, Toss_TagPerDay>()
.Where(i => i.CreatedOn >= firstDay)
.GroupBy(i => i.Tag)
.OrderByDescending(g => g.Sum(i => i.Count))
.Take(50)
.Select(t => new BestTagsResult()
{
CountLastMonth = t.Count(),
Tag = t.Key
})
.ToListAsync()
But this gives me the error
Message: System.NotSupportedException : Could not understand expression: from index 'Toss/TagPerDay'.Where(i => (Convert(i.CreatedOn, DateTimeOffset) >= value(Toss.Server.Models.Tosses.BestTagsQueryHandler+<>c__DisplayClass3_0).firstDay)).GroupBy(i => i.Tag).OrderByDescending(g => g.Sum(i => i.Count)).Take(50).Select(t => new BestTagsResult() {CountLastMonth = t.Count(), Tag = t.Key})
---- System.NotSupportedException : GroupBy method is only supported in dynamic map-reduce queries
Any idea how can I make this work ? I could query for all the index data from the past 30 days and do the groupby / order / take in memory but this could make my app load a lot of data.
The results from the map-reduce index you created will give you the number of tags per day. You want to have the most popular ones from the last 30 days so you need to do the following query:
var tagCountPerDay = session
.Query<TagByDayIndex, Toss_TagPerDay>()
.Where(i => i.CreatedOn >= DateTime.Now.AddDays(-30))
.ToList();
Then you can the the client side grouping by Tag:
var mostUsedTags = tagCountPerDay.GroupBy(x => x.Tag)
.Select(t => new BestTagsResult()
{
CountLastMonth = t.Count(),
Tag = t.Key
})
.OrderByDescending(g => g.CountLastMonth)
.ToList();
#Kuepper
Based on your index definition. You can handle that by the following index:
public class TrendingSongs : AbstractIndexCreationTask<TrackPlayedEvent, TrendingSongs.Result>
{
public TrendingSongs()
{
Map = events => from e in events
where e.TypeOfTrack == TrackSubtype.song && e.Percentage >= 80 && !e.Tags.Contains(Podcast.Tags.FraKaare)
select new Result
{
TrackId = e.TrackId,
Count = 1,
Timestamp = new DateTime(e.TimestampStart.Year, e.TimestampStart.Month, e.TimestampStart.Day)
};
Reduce = results => from r in results
group r by new {r.TrackId, r.Timestamp}
into g
select new Result
{
TrackId = g.Key.TrackId,
Count = g.Sum(x => x.Count),
Timestamp = g.Key.Timestamp
};
}
}
and the query using facets:
from index TrendingSongs where Timestamp between $then and $now select facet(TrackId, sum(Count))
The reason for the error is that you can't use 'GroupBy' in a query made on an index.
'GroupBy' can be used when performing a 'dynamic query',
i.e. a query that is made on a collection, without specifying an index.
See:
https://ravendb.net/docs/article-page/4.1/Csharp/client-api/session/querying/how-to-perform-group-by-query
I solved a similar problem, by using AdditionalSources that uses dynamic values.
Then I update the index every morning to increase the Earliest Timestamp. await IndexCreation.CreateIndexesAsync(new AbstractIndexCreationTask[] {new TrendingSongs()}, _store);
I still have to try it in production, but my tests so far look like it's a lot faster than the alternatives. It does feel pretty hacky though and I'm surprised RavenDB does not offer a better solution.
public class TrendingSongs : AbstractIndexCreationTask<TrackPlayedEvent, TrendingSongs.Result>
{
public DateTime Earliest = DateTime.UtcNow.AddDays(-16);
public TrendingSongs()
{
Map = events => from e in events
where e.TypeOfTrack == TrackSubtype.song && e.Percentage >= 80 && !e.Tags.Contains(Podcast.Tags.FraKaare)
&& e.TimestampStart > new DateTime(TrendingHelpers.Year, TrendingHelpers.Month, TrendingHelpers.Day)
select new Result
{
TrackId = e.TrackId,
Count = 1
};
Reduce = results => from r in results
group r by new {r.TrackId}
into g
select new Result
{
TrackId = g.Key.TrackId,
Count = g.Sum(x => x.Count)
};
AdditionalSources = new Dictionary<string, string>
{
{
"TrendingHelpers",
#"namespace Helpers
{
public static class TrendingHelpers
{
public static int Day = "+Earliest.Day+#";
public static int Month = "+Earliest.Month+#";
public static int Year = "+Earliest.Year+#";
}
}"
}
};
}
}
i'm new to linq and i've tried this function which should check for duplicates. what i want to do is to check through my list of buildingobjects to check if any buildingobject contains an objectID identical to another buildingObject in the list. finally i want to use the GUID of the buildingObject which had a duplicate, and print it to my log for the user to see.
public class FMBuildingObject
{
public Int64 ObjectId { get; set; }
public string GUID { get; set; }
}
the building object is bigger, but this is the values i'm using.
next i'm trying to use the buildingobjects, find duplicates and then print the GUID out. however i can't figure out how to access that GUID.
var query =
buildingObjects
.GroupBy(x => new { x })
.Select(group => new { Name = group.Key, Count = group.Count() })
.OrderByDescending(x => x.Count);
foreach (var q in query)
{
var updateLog = new LogServiceModel()
{
LogType = LogTypes.Warning, Parameters = { {?GUID?}}, LogTitle = "You have used two different classifications on a same Buildingobject in {0}. "
};
logService.Create(updateLog);
}
This will return a List<string> containing the GUID's of the objects which has duplicate ObjectId's:
var result = buildingObjects
.GroupBy(b => b.ObjectId)
.Where(g => g.Count() > 1)
.SelectMany(g => g.Select(b => b.GUID)).ToList();
You can get duplicate objects by:
var query = buildingObjects.GroupBy(x => new { x.ObjectId, x.GUID })
.Where(g => g.Count() > 1)
.Select(group => new { Name = group.Key, group.Key.GUID, group.Key.ObjectId }); //I don't know what is Name and why it's equal to Key
Then inside your foreach loop:
foreach (var q in query)
{
var updateLog = new LogServiceModel()
{
LogType = LogTypes.Warning, Parameters = q.GUID, LogTitle = $"You have used two different classifications on a same Buildingobject, Id: {q.ObjectId}. "
};
logService.Create(updateLog);
}
Or you can simply do:
var query = buildingObjects.GroupBy(x => new { x.ObjectId, x.GUID })
.Where(g => g.Count() > 1)
.Select(group => new LogServiceModel()
{
LogType = LogTypes.Warning,
Parameters = group.Key.GUID,
LogTitle = $"You have used two different classifications on a same Buildingobject, Id: {group.Key.ObjectId}."
});
I have documents stored in cosmos db, I have multiple documents for a same "stationkey"(partition key), in this example stationkey "ABC" has more than one documents with "yymm" has "2018-02" & "2018-01" e.t.c,
query that i am trying is get all "avg" & "dd" fields along with "yymm" for the given stationkey and yymm filter combination
I am trying to query using C#, I am trying to get "avg", "dd" & "yymm" fields from "data" array, the query that I have written is giving entire "data" array.
var weatherQuery = this.docClient.CreateDocumentQuery<WeatherStation>(docUri, queryOptions)
.Where(wq => wq.stationName == stationKey && lstYearMonthFilter.Contains(wq.yearMonth))
.Select(s => s.data);
what is the best way to query specific fields in from a document array?
So you got the data in s => s.data. To get only the avg from the array you have to do another projection as following:
.Select (s => s.data.Select ( a => a.avg ))
Modifying my answer as you say you don't find 'Select' on 'data'.
Define a class MyDocument as such:
public class Datum
{
[JsonProperty("dd")]
public string dd;
[JsonProperty("max")]
public int max;
[JsonProperty("min")]
public int min;
[JsonProperty("avg")]
public int avg;
}
public class MyDocument : Document
{
[JsonProperty("id")]
public string id;
[JsonProperty("data")]
public Datum[] data;
}
modify your code accordingly
IDocumentQuery<MyDocument> query = client.CreateDocumentQuery<MyDocument>(UriFactory.CreateDocumentCollectionUri(_database, _collection),
new FeedOptions { MaxItemCount = -1, EnableCrossPartitionQuery = true, MaxDegreeOfParallelism = 199, MaxBufferedItemCount = 100000})
.Where(predicate)
.AsDocumentQuery();
while (query.HasMoreResults)
{
FeedResponse<MyDocument> feedResponse = await query.ExecuteNextAsync<MyDocument>();
Console.WriteLine (feedResponse.Select(x => x.data.Select(y => y.avg)));
}
HTH
You can select only specific fields from the array items using a double-nested anonymous class - see the altered SelectMany below. This will return yymm with every Datum, so may not be as efficient as just selecting the entire array - definitely measure the RU/s in both cases.
var weatherQuery = this.docClient.CreateDocumentQuery<WeatherStation>(docUri, queryOptions)
.Where(wq => wq.stationName == stationKey && lstYearMonthFilter.Contains(wq.yearMonth))
.SelectMany(x => x.data.Select(y => new { x.yymm, data = new[] { new { y.dd, y.avg } } }))
.AsDocumentQuery();
var results = new List<WeatherStation>();
while (weatherQuery.HasMoreResults)
{
results.AddRange(await weatherQuery.ExecuteNextAsync<WeatherStation>());
}
var groupedResults = results
.GroupBy(x => x.yymm)
.Select(x => new { x.First().yymm, data = x.SelectMany(y => y.data).ToArray() })
.Select(x => new WeatherStation() { yymm = x.yymm, data = x.data });
I have a table having TeamName and CurrentStatus fields. I am making a linq query to get for each team and for each status the count of records:
var teamStatusCounts = models.GroupBy(x => new { x.CurrentStatus, x.TeamName })
.Select(g => new { g.Key, Count = g.Count() });
The results of this query returns all the counts except where count is 0. I need to get the rows where there is no record for a specific team and a specific status (where count = 0).
You could have a separate collection for team name and statuses you are expecting and add the missing ones to the result set
//assuming allTeamNamesAndStatuses is a cross joing of all 'CurrentStatus' and 'TeamNames'
var teamStatusCounts = models.GroupBy(x => new { x.CurrentStatus, x.TeamName })
.Select(g => new { g.Key, Count = g.Count() })
.ToList();
var missingTeamsAndStatuses = allTeamNamesAndStatuses
.Where(a=>
!teamStatusCounts.Any(b=>
b.Key.CurrentStatus == a.CurrentStatus
&& b.Key.TeamName == a.TeamName))
.Select(a=>new {
Key = new { a.CurrentStatus, a.TeamName },
Count = 0
});
teamStatusCounts.AddRange(emptyGroups);
I've created a fiddle demonstrating the answer as well
I would select the team names and status first:
var teams = models.Select(x => x.TeamName).Distinct().ToList();
var status = models.Select(x => x.CurrentStatus).Distinct().ToList();
You can skip this if you know the list entries already.
Then you can select for each team and each state the number of models:
var teamStatusCounts = teams.SelectMany(team => states.Select(state =>
new
{
TeamName = team,
CurrentStatus = state,
Count = models.Count(model =>
model.TeamName == team && model.CurrentStatus == state)
}));