I have a fairly unique situation, I have never needed to this before anyways. I have a Linq query that returns data from a database using EF4.1. I want to create multiple similar (same signature) anonymous (or even named if necessary) results from each query result.
Here's the code i'm using now:
var data = getMyData().Select(x =>
new
{
GoalName = x.GoalType.Name,
Start = x.StartDate,
End = x.EndDate,
x.StartValue,
x.CheckIns
}).ToList();
var r1 = data.Select(x =>
new
{
title = x.GoalName,
start = x.Start.ToString(),
end = x.End.ToString(),
className = "hidden",
type = "goal"
});
var r2 = data.Select(x =>
new
{
title = string.Format("Start: {0:0.##}", x.StartValue),
start = x.Start.ToString(),
end = x.Start.ToString(),
className = "",
type = ""
});
var r3 = data.Select(x =>
new
{
title = "End",
start = x.End.ToString(),
end = x.End.ToString(),
className = "",
type = ""
});
var r4 = data.SelectMany(x => x.CheckIns)
.Select(y =>
new
{
title = y.CheckInValue.Value.ToString(),
start = y.CheckInDateTime.ToString(),
end = y.CheckInDateTime.ToString(),
className = "",
type = ""
});
var result = r1.Union(r2).Union(r3).Union(r4);
Now maybe this is as good a way as any, but I can't help feeling that i'm missing something.
Is there a better solution?
What you have is actually OK I think.
But StevenzNPaul's suggestion not that bad, here's how you can use the let keyword to store the different projections, then select the results individually (for brevity, I did not project all the fields, but you get the point):
var query = from x in data
let result1 = new {title = x.GoalName, start = x.Start}
let result2 = new {title = string.Format("Start: {0:0.##}", x.StartValue), start = x.Start}
let result3 = new {title = "End", start = x.End}
let checkins = x.CheckIns.Select(checkin => new { title = "...", start = checkin.Start })
from result in new[] { result1, result2, result3 }.Concat(checkins)
select result;
Obviously, whether this is better is a matter of preference. Also, this will result in a different ordering, which may or may not be a problem for you.
You can create an iterator using yield which also has the advantage of being evaluated lazily (doesn't require the ToList()). I created a typed class Result to hold the query results
private IEnumerable<Result> PerformQuery()
{
var results= getMyData().Select(x => new {GoalName = x.GoalType.Name,
Start = x.StartDate, End = x.EndDate, x.StartValue, x.CheckIns});
foreach (var result in results)
{
yield return new Result() { Title = result.GoalName, Start = result.Start.ToString(), End = result.End.ToString(), ClassName = "Hidden", Type = "Goal" };
yield return new Result() { Title = String.Format("Start: {0:0.##}",result.StartValue), Start = result.Start.ToString(), End = result.Start.ToString() }
yield return new Result() { Title = "End", Start = result.End.ToString(), End = result.End.ToString() };
foreach (var checkIn in result.CheckIns)
yield return new Result() { Title = checkIn.CheckInValue.Value.ToString(), Start = checkIn.CheckInDateTime.ToString(), End = checkIn.CheckInDateTime.ToString() };
}
}
try using let keyword it will work for you.
Related
Query to scroll at the matching records from the query
this is the query of nest in C# to get all the records from nest C# find many questions which can solve it by using different method linq method but i want to do this this way any suggestions help would be appreciated
string[] MERCHANTNO = MerchantId.Split(",");
var mustClause = new List<QueryContainer>();
var filterClause = new List<QueryContainer>();
var filters = new List<QueryContainer>();
filters.Add(new TermsQuery{
Field = new Field("MERCHANTNO"),
Terms = MERCHANTNO,
});
Logger.LogInformation(clsName, funcName, "Filter Clause is:", filters);
var SearchRequest = new SearchRequest<AcquirerDTO>(idxName) {
Size = 10000,
SearchType = Elasticsearch.Net.SearchType.QueryThenFetch,
Scroll = "5m",
Query = new BoolQuery { Must = filters }
};
var searchResponse = await _elasticClient.SearchAsync<AcquirerDTO>( SearchRequest );
The code for Scroll all the Records you have in ElasticSearch is
Filter
filters.Add(new TermsQuery {
Field = new Field("MERCHANTNO"), >>> Value needs to be searched
Terms = MERCHANTNO,
});
Date Range Filter
filterClause.Add(new DateRangeQuery {
Boost = 1.1,
Field = new Field("filedate"),
GreaterThanOrEqualTo = DateMath.Anchored(yesterday),
LessThanOrEqualTo = DateMath.Anchored(Today),
Format = "yyyy-MM-dd",
TimeZone = "+01:00"
});
Search Request for scrolling
var SearchRequest = new SearchRequest<AcquirerDTO>(idxName) {
From = 0,
Scroll = scrollTimeoutMinutes,
Size = scrollPageSize,
Query = new BoolQuery
{
Must = filters,
Filter = filterClause
}
};
var searchResponse = await _elasticClient.SearchAsync<AcquirerDTO>(SearchRequest);
if (searchResponse.ApiCall.ResponseBodyInBytes != null) {
var requestJson = System.Text.Encoding.UTF8.GetString(searchResponse.ApiCall.RequestBodyInBytes);
var JsonFormatQuery = JsonConvert.SerializeObject(JsonConvert.DeserializeObject(requestJson), Formatting.Indented);
}
This is the code for Scrolling all the results in kibana
List<AcquirerDTO> results = new List<AcquirerDTO>();
if (searchResponse.Documents.Any())
results.AddRange(searchResponse.Documents);
string scrollid = searchResponse.ScrollId;
bool isScrollSetHasData = true;
while (isScrollSetHasData)
{
ISearchResponse<AcquirerDTO> loopingResponse = _elasticClient.Scroll<AcquirerDTO>(scrollTimeoutMinutes, scrollid);
if (loopingResponse.IsValid)
{
results.AddRange(loopingResponse.Documents);
scrollid = loopingResponse.ScrollId;
}
isScrollSetHasData = loopingResponse.Documents.Any();
}
var records = results;
I am writing x-unit testing.
I want to check if list contains the newly added value.
I tried the below code but i am not getting the result
_record.Add(new PortfolioCompanyLinkModel { Id = 3, PortfolioCompanyId = 1, URL = "www.historiclreports2.com", LinkName = "Historical Reports", ToBeDeleted = false, IsExternalLink = false, LinkId = 1 });
_record.Add(new PortfolioCompanyLinkModel { Id = 4, PortfolioCompanyId = 1, URL = "www.SalesForce.com", LinkName = "SalesForce", ToBeDeleted = false, IsExternalLink = false, LinkId = 2 });
var repo = new PortfolioCompanyLinkRepository(dbContext, obj.HttpContextAccessor);
var update = await repo.GetOne(2);
_record.Add(new PortfolioCompanyLinkModel { Id = update.Id, PortfolioCompanyId = update.PortfolioCompanyId, URL = "www.historiclreportstest.com", LinkName = update.Link.Name, ToBeDeleted = false, IsExternalLink = true, LinkId = update.LinkId });
var delete = await repo.GetOne(1);
_record.Add(new PortfolioCompanyLinkModel { Id = delete.Id, PortfolioCompanyId = delete.PortfolioCompanyId, URL = delete.URL, LinkName = delete.Link.Name, ToBeDeleted = true, IsExternalLink = delete.Link.IsExternalLink, LinkId = delete.LinkId });
await repo.AddUpdateDelete(_record);
await repo.SaveAsync();
var actual = await repo.GetAll();
Assert.Collection(actual, item => Assert.Contains("www.historiclreports2.com", item.URL));
How do i check if actual has www.historiclreports2.com as URL ?
In order to do this, you want to single out the record you desire to assert on. This can be done multiple ways, but the way I have do this is always as follows:
var recordInQuestion = actual.Where(x => x.URL
.Equals("www.historiclreports2.com"))
.SingleOrDefault();
Assert.NotNull(recordInQuestion);
Other options are:
Assert.Single(actual.Where(x => x.URL.Equals("www.historiclreports2.com")));
Assert.True(actual.Any(x => x.URL.Equals("www.historiclreports2.com")))
Using the first option with the SingleOrDefault() in which the record is assigned to a variable allows you to easily perform multiple asserts on that object's data.
Does anyone know how to do multiple aggregations with nest?
I have found quite a few examples unfortunately none of them work.
Here's what I have:
Vehicles fields = new Vehicles();
//create a terms query
var query = new TermsQuery
{
IsVerbatim = true,
Field = "VehicleOwnerId",
Terms = new string[] { 25 },
};
var aggregations = new Dictionary<string, IAggregationContainer>
{
{ "years", new AggregationContainer
{
Terms = new TermsAggregation(nameof(fields.Year))
{
Field = new Field(nameof(fields.Year))
}
}
}
//,
//{ "makes", new AggregationContainer
// {
// Terms = new TermsAggregation("Make")
// {
// Field = new Field(nameof(fields.Make))
// }
// }
//}
};
//create the search request
var searchRequest = new SearchRequest
{
Query = query,
From = 0,
Size = 100,
Aggregations = aggregations
};
var result = client.SearchAsync<InventoryLiveView>(searchRequest).Result;
var years = result.Aggregations.Terms("years");
Dictionary<string, long> yearCounts = new Dictionary<string, long>();
foreach (var item in years.Buckets)
{
yearCounts.Add(item.Key, item.DocCount ?? 0);
}
If I just execute the code like this it works. Years returns the aggregates as expected. If I try to add another field (like the one commented out above) it fails and I get zero records.
How can I get multiple aggregates in one query? I see examples of it all over, but none of the examples I've tried seem to work and most seem to be outdated (including some in the Nest documentation).
I have also tried this approach which is pretty close to the documentation.
//create the search request
var searchRequest = new SearchRequest
{
Query = query,
From = 0,
Size = 100,
//Aggregations = aggregations
Aggregations = new AggregationDictionary
{
{
"childAgg", new ChildrenAggregation("childAgg", typeof(Vehicles ))
{
Aggregations = new AggregationDictionary
{
{"years", new TermsAggregation(nameof(fields.VehicleYear))},
{"makes", new TermsAggregation(nameof(fields.VehicleMakeName))},
{"models", new TermsAggregation(nameof(fields.VehicleModelName))},
}
}
}
}
};
var result = client.SearchAsync<Vehicles>(searchRequest).Result;
This just produces a null reference exception.
I guess I'll never have too worry about getting to proud as a programmer :)
It's too often that the solution to the problem makes me feel stupid when it reveals itself.
So my issue was that the field I was trying to use in the aggregation was text and couldn't be used. I switched everything to the ID fields and multiple aggregations work as expected.
So this version of the code works like a champ:
Vehicle fields = new Vehicle ();
//create a terms query
var query = new TermsQuery
{
IsVerbatim = true,
Field = "VehicleOwnerId",
Terms = new string[] { "30" },
};
string[] Fields = new[]
{
nameof(fields.Year),
nameof(fields.MakeId),
nameof(fields.ModelId)
};
var aggregations = new Dictionary<string, IAggregationContainer>();
foreach (string sField in Fields)
{
var termsAggregation = new TermsAggregation(sField)
{
Field = sField
};
aggregations.Add(sField, new AggregationContainer { Terms = termsAggregation });
}
//create the search request
var searchRequest = new SearchRequest
{
Query = query,
From = 0,
Size = 10,
Aggregations = aggregations
};
var result = client.SearchAsync<InventoryLiveView>(searchRequest).Result;
var years = result.Aggregations.Terms(nameof(fields.Year));
Dictionary<string, long> yearCounts = new Dictionary<string, long>();
foreach (var item in years.Buckets)
{
yearCounts.Add(item.Key, item.DocCount ?? 0);
}
The exact error from elasticsearch, which I saw using postman was:
Fielddata is disabled on text fields by default. Set fielddata=true on [MakeName] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
Here is my example using SearchDescriptors. My only problem is how to serialize returned results into a proper Key Value list. Is Looping through a fields list the best way to return results.
SearchDescriptor<Advert> agghDescriptor = new SearchDescriptor<Advert>();
agghDescriptor.Aggregations(ag => ag.Terms("make", a => a.Field(f => f.Make)) &&
ag.Terms("region", a => a.Field(f => f.Region)) &&
ag.Terms("city", a => a.Field(f => f.City)) &&
ag.Terms("category", a => a.Field(f => f.Category)) &&
ag.Terms("application", a => a.Field(f => f.Application)) &&
ag.Terms("portalId", a => a.Field(f => f.PortalId)) &&
ag.Terms("isActiveAuctionAdvert", a => a.Field(f => f.IsActiveAuctionAdvert)) &&
ag.Terms("isBargainAccount", a => a.Field(f => f.IsBargainAccount)) &&
ag.Terms("condition", a => a.Field(f => f.Condition))
);
agghDescriptor.Size(0);
var json2 = _client.RequestResponseSerializer.SerializeToString(agghDescriptor);
var aggregationResult = _client.Search<Advert>(agghDescriptor);
List<string> fields = new List<string>();
fields.Add("make");
fields.Add("category");
fields.Add("region");
List<Aggregation> aggregations = new List<Aggregation>();
foreach (var field in fields)
{
var aggrs = aggregationResult.Aggregations.Terms(field);
List<AggregateItem> aggregateItems = new List<AggregateItem>();
foreach (var item in aggrs.Buckets)
{
aggregateItems.Add(new AggregateItem()
{
Count = item.DocCount ?? 0,
Key = item.Key
});
}
aggregations.Add(new Aggregation()
{
Name = field,
Aggregates = aggregateItems
});
}
I have two functions, returning expression translated to EF.:
public static Expression<Func<TripLanguage, TripViewModel>> ToSearchModel(ILookup<int, TagViewModel> tags)
{
return tripLanguage => new TripViewModel()
{
From = tripLanguage.From,
To = tripLanguage.To,
Annotation = tripLanguage.Description.Truncate(Strings.TRUNCATE_ANOTATION),
Level = tripLanguage.Trip.Level,
BicycleType = tripLanguage.Trip.BicycleType,
UrlId = tripLanguage.UrlId,
Distance = tripLanguage.Trip.Distance,
Tags = tags[tripLanguage.TripId], //This is only different and in function args of course
MainImage = tripLanguage.Trip.Images.OrderBy(s => s.Date).Select(i => new ImageViewModel
{
Filename = i.Filename,
Id = i.Id,
Title = i.Title
}).Take(1)
};
}
public static Expression<Func<TripLanguage, TripViewModel>> ToSearchModel()
{
return tripLanguage => new TripViewModel()
{
From = tripLanguage.From,
To = tripLanguage.To,
Annotation = tripLanguage.Description.Truncate(Strings.TRUNCATE_ANOTATION),
Level = tripLanguage.Trip.Level,
BicycleType = tripLanguage.Trip.BicycleType,
UrlId = tripLanguage.UrlId,
Distance = tripLanguage.Trip.Distance,
MainImage = tripLanguage.Trip.Images.OrderBy(s => s.Date).Select(i => new ImageViewModel
{
Filename = i.Filename,
Id = i.Id,
Title = i.Title
}).Take(1)
};
}
The only different is Tags collection. Is possible something like call method, without arguments and add Tags attribute which exclude duplicate code? Or Use some inheritance expression?
Thank you for your time.
Change the first method to work with null tags, and provide a null default for it:
public static Expression<Func<TripLanguage,TripViewModel>> ToSearchModel(ILookup<int, TagViewModel> tags = null) {
return tripLanguage => new TripViewModel() {
From = tripLanguage.From,
To = tripLanguage.To,
Annotation = tripLanguage.Description.Truncate(Strings.TRUNCATE_ANOTATION),
Level = tripLanguage.Trip.Level,
BicycleType = tripLanguage.Trip.BicycleType,
UrlId = tripLanguage.UrlId,
Distance = tripLanguage.Trip.Distance,
Tags = tags?[tripLanguage.TripId], // <<== Note the question mark
MainImage = tripLanguage.Trip.Images.OrderBy(s => s.Date).Select(i => new ImageViewModel
{
Filename = i.Filename,
Id = i.Id,
Title = i.Title
}).Take(1)
};
}
I had a quick question. Can I do all of this logic inside the select statement?
var entries = atisDAO.GetPME(xl, null);
response.Data.Detectors = new List<DetectorDetails>(entries.Select(pme => new DetectorDetails {ID = pme.PlaceNum.ToString()}));
if(response.Data.Detectors.Any())
{
response.Data.Detectors.ForEach(d =>{
int id;
if(int.TryParse(d.ID, out id))
{
var summaries = atisDAO.GetSummaryEntries(id);
if (summaries.Any())
{
var count = summaries.Sum(summary => summary.TODCount + summary.BFICount + summary.ViolationCount);
var detectionDate = summaries.Max(summary => summary.ReadDate);
d.Count = count.ToString();
d.DetectionTime = new DateTimeZone {
ReadDate = detectionDate.ToString(DATE_FORMAT)
, ReadTime = detectionDate.ToString(TIME_FORMAT)
};
}
}
});
}
It feels wrong to do a select, then loop through the list and modify the items I just selected. Can I do all of this inside the select statement?
Thanks for any tips.
Cheers,
~ck in San Diego
Sure, why not? What's stopping you from changing the new DetectorDetails, with the code from the ForEach, in the Select statement?
Does this help get you to where you're going? I can't be sure that the datatypes all match and will compile as-is, but it's a stab at putting all that logic in your .Select(). Surely it can be enhanced to be better! Feel free to edit this answer to make it work better.
response.Data.Detectors = atisDAO.GetPME(xl, null).Select(pme =>
new DetectorDetails{
ID = pme.PlaceNum.ToString(),
Count = atisDAO.GetSummaryEntries(int.Parse(pme.PlaceNum.ToString())).Count(), //some work needed here to ensure pme.PlaceNum is actually an number
DetectionTime = new DateTimeZone{
ReadDate = summaries.Max(summary => summary.ReadDate).ToString(DATE_FORMAT),
ReadTime = summaries.Max(summary => summary.ReadDate).ToString(TIME_FORMAT)
}
}
);
I figured this out. I needed a return statement inside my projection.
var entries = atisDAO.GetPME(xl, null);
response.Data.Detectors = new List<DetectorDetails>(entries.Select(pme =>{
var details = new DetectorDetails { ID = pme.PlaceNum.ToString()};
var summaries = atisDAO.GetSummaryEntries(pme.PlaceNum);
if (summaries.Any())
{
var count = summaries.Sum(summary => summary.TODCount + summary.BFICount + summary.ViolationCount);
var detectionDate = summaries.Max(summary => summary.ReadDate);
details.Count = count.ToString();
details.DetectionTime = new DateTimeZone {
ReadDate = detectionDate.ToString(DATE_FORMAT)
, ReadTime = detectionDate.ToString(TIME_FORMAT)
};
}
return details;
}));