I want to search a document in a way that I filter it by exact values for the "from" and "to" fields. I use DateRangeQuery objects that I pass to the bool query instance.
This filter works fine when its only one date being filtered, but when both are active it returns no documents at all. Code bellow
DateRangeQuery fromDateRangeQuery = new DateRangeQuery()
{
Name = "from_query",
Field = "from",
GreaterThanOrEqualTo = dateFrom,
LessThanOrEqualTo = dateFrom,
};
queryContainers.Add(fromDateRangeQuery);
DateRangeQuery ToDateRangeQuery = new DateRangeQuery()
{
Name = "to_query",
Field = "to",
GreaterThanOrEqualTo = dateTo,
LessThanOrEqualTo = dateTo,
};
queryContainers.Add(ToDateRangeQuery);
//more terms filters not related to the 2 fields
var searchRequest = new SearchRequest("0___aggregate_");
searchRequest.SearchType = SearchType.QueryThenFetch;
searchRequest.From = 0;
searchRequest.Size = 10000;
searchRequest.Query = boolQuery;
var searchResponse = Get().SearchAsync<AggregationHolder>(searchRequest);
searchResponse.Wait(60000);
var status = searchResponse.Result;
Both dateFrom and dateTo are regular date time object. What am I doing wrong, why cant I combine these 2 filters?
Maybe, the problem is your date format. Could you please try like below?
ElasticClient elasticClient = new ElasticClient(new ConnectionSettings(
new Uri(connString))
.RequestTimeout(TimeSpan.FromMinutes(5))
.DefaultIndex(defaultIndexName)
);
DateRangeQuery queryFrom = new DateRangeQuery()
{
Name = "from_query",
Field = "from",
GreaterThanOrEqualTo = "05/12/2020 01:03:15",
LessThanOrEqualTo = "05/12/2020 01:03:16",
Format = "dd/MM/yyyy HH:mm:ss"
};
DateRangeQuery queryTo = new DateRangeQuery()
{
Name = "to_query",
Field = "to",
GreaterThanOrEqualTo = "05/12/2020 11:20:32",
LessThanOrEqualTo = "05/12/2020 11:20:33",
Format = "dd/MM/yyyy HH:mm:ss"
};
QueryContainer queryMain = queryFrom & queryTo;
ISearchResponse<AggregationHolder> searchResponse = elasticClient.Search<AggregationHolder>(s => s
.RequestConfiguration(r => r.DisableDirectStreaming())
.From(0)
.Size(100)
.Query(q2 => q2
.Bool(b => b
.Should(queryMain))
)
);
Related
Query to scroll at the matching records from the query
this is the query of nest in C# to get all the records from nest C# find many questions which can solve it by using different method linq method but i want to do this this way any suggestions help would be appreciated
string[] MERCHANTNO = MerchantId.Split(",");
var mustClause = new List<QueryContainer>();
var filterClause = new List<QueryContainer>();
var filters = new List<QueryContainer>();
filters.Add(new TermsQuery{
Field = new Field("MERCHANTNO"),
Terms = MERCHANTNO,
});
Logger.LogInformation(clsName, funcName, "Filter Clause is:", filters);
var SearchRequest = new SearchRequest<AcquirerDTO>(idxName) {
Size = 10000,
SearchType = Elasticsearch.Net.SearchType.QueryThenFetch,
Scroll = "5m",
Query = new BoolQuery { Must = filters }
};
var searchResponse = await _elasticClient.SearchAsync<AcquirerDTO>( SearchRequest );
The code for Scroll all the Records you have in ElasticSearch is
Filter
filters.Add(new TermsQuery {
Field = new Field("MERCHANTNO"), >>> Value needs to be searched
Terms = MERCHANTNO,
});
Date Range Filter
filterClause.Add(new DateRangeQuery {
Boost = 1.1,
Field = new Field("filedate"),
GreaterThanOrEqualTo = DateMath.Anchored(yesterday),
LessThanOrEqualTo = DateMath.Anchored(Today),
Format = "yyyy-MM-dd",
TimeZone = "+01:00"
});
Search Request for scrolling
var SearchRequest = new SearchRequest<AcquirerDTO>(idxName) {
From = 0,
Scroll = scrollTimeoutMinutes,
Size = scrollPageSize,
Query = new BoolQuery
{
Must = filters,
Filter = filterClause
}
};
var searchResponse = await _elasticClient.SearchAsync<AcquirerDTO>(SearchRequest);
if (searchResponse.ApiCall.ResponseBodyInBytes != null) {
var requestJson = System.Text.Encoding.UTF8.GetString(searchResponse.ApiCall.RequestBodyInBytes);
var JsonFormatQuery = JsonConvert.SerializeObject(JsonConvert.DeserializeObject(requestJson), Formatting.Indented);
}
This is the code for Scrolling all the results in kibana
List<AcquirerDTO> results = new List<AcquirerDTO>();
if (searchResponse.Documents.Any())
results.AddRange(searchResponse.Documents);
string scrollid = searchResponse.ScrollId;
bool isScrollSetHasData = true;
while (isScrollSetHasData)
{
ISearchResponse<AcquirerDTO> loopingResponse = _elasticClient.Scroll<AcquirerDTO>(scrollTimeoutMinutes, scrollid);
if (loopingResponse.IsValid)
{
results.AddRange(loopingResponse.Documents);
scrollid = loopingResponse.ScrollId;
}
isScrollSetHasData = loopingResponse.Documents.Any();
}
var records = results;
I want to sum of all Volume & TotalTimes Column after Group BY from DataTable Alias as Temp_DT. Remember Volume & All TotalTimes Column may have null or blank value.
Can anyone help me to get the code through Linq Method
Sample Temp_DT:
Result should be in this format
I have tried the below code, but Temp_DT result is Null.
var Temp_DT = tempdt.AsEnumerable().Select(x =>
new
{
UID = x.Field<string>("UID"),
EMPNAME = x.Field<string>("Emp Name"),
EMPROLE = x.Field<string>("Role"),
SUPID = x.Field<string>("Sup ID"),
SUPNAME = x.Field<string>("Sup Name"),
DESIGNATION = x.Field<string>("Designation"),
VOLUME = x.Field<int>("Volume"),
LOGINTIME = x.Field<TimeSpan>("Login Time"),
BREAKTIME = x.Field<TimeSpan>("Break Time"),
HANDLETIME = x.Field<TimeSpan>("Handle Time"),
ACTIVETIME = x.Field<TimeSpan>("Active Time"),
HOLDTIME = x.Field<TimeSpan>("Hold Time"),
ACWTIME = x.Field<TimeSpan>("ACW"),
IDLETIME = x.Field<TimeSpan>("Idle"),
PRODUCTIVE = x.Field<TimeSpan>("Productive"),
NONPRODUCTIVE = x.Field<TimeSpan>("Non Productive"),
USERERROR = x.Field<TimeSpan>("User Error Time")
}).GroupBy(s => new { s.UID, s.EMPNAME, s.EMPROLE, s.SUPID, s.SUPNAME, s.DESIGNATION })
.Select(g => new
{
g.Key.UID,
g.Key.EMPNAME,
g.Key.EMPROLE,
g.Key.SUPID,
g.Key.SUPNAME,
g.Key.DESIGNATION,
VOLUME = g.Sum(x => Convert.ToInt16(x.VOLUME)),
LOGINTIME = new TimeSpan(g.Sum(x => x.LOGINTIME.Ticks)),
BREAKTIME = new TimeSpan(g.Sum(x => x.BREAKTIME.Ticks)),
HANDLETIME = new TimeSpan(g.Sum(x => x.HANDLETIME.Ticks)),
ACTIVETIME = new TimeSpan(g.Sum(x => x.ACTIVETIME.Ticks)),
HOLDTIME = new TimeSpan(g.Sum(x => x.HOLDTIME.Ticks)),
ACWTIME = new TimeSpan(g.Sum(x => x.ACWTIME.Ticks)),
IDLETIME = new TimeSpan(g.Sum(x => x.IDLETIME.Ticks)),
PRODUCTIVE = new TimeSpan(g.Sum(x => x.PRODUCTIVE.Ticks)),
NONPRODUCTIVE = new TimeSpan(g.Sum(x => x.NONPRODUCTIVE.Ticks)),
USERERROR = new TimeSpan(g.Sum(x => x.USERERROR.Ticks)),
});
You have to Parse the TimeSpan and Sum by using Ticks, like below.
var Temp_DT = tempdt.AsEnumerable().Select(x =>
new
{
UID = x["UID"],
EMPNAME = x["Emp Name"],
EMPROLE = x["Role"],
SUPID = x["Sup ID"],
SUPNAME = x["Sup Name"],
DESIGNATION = x["Designation"],
VOLUME = x["Volume"],
ERRORTIME = x["Error_Time"],
ACWTIME = x["ACW"],
BREAKTIME = x["Break Time"],
IDLETIME = x["Idle"],
NONPRODUCTIVE = x["Non Productive"],
}).GroupBy(s => new { s.UID, s.EMPNAME, s.EMPROLE, s.SUPID, s.SUPNAME, s.DESIGNATION })
.Select(g => {
var grouped = g.ToList();
return new
{
UID = g.Key.UID,
EMPNAME = g.Key.EMPNAME,
EMPROLE = g.Key.EMPROLE,
SUPID = g.Key.SUPID,
SUPNAME = g.Key.SUPNAME,
DESIGNATION = g.Key.DESIGNATION,
VOLUME = grouped.Sum(x => Convert.ToInt16(x.VOLUME)),
Error_Time = new TimeSpan(grouped.Select(x => ConvertTimeSpan(x.ERRORTIME.ToString())).ToList().Sum(r=> r.Ticks)),
ACW = new TimeSpan(grouped.Select(x => ConvertTimeSpan(x.ACWTIME.ToString())).ToList().Sum(r=> r.Ticks)),
Break = new TimeSpan(grouped.Select(x => ConvertTimeSpan(x.BREAKTIME.ToString())).ToList().Sum(r=> r.Ticks)),
Idle = new TimeSpan(grouped.Select(x => ConvertTimeSpan(x.IDLETIME.ToString())).ToList().Sum(r=> r.Ticks)),
NonProductive = new TimeSpan(grouped.Select(x => ConvertTimeSpan(x.NONPRODUCTIVE.ToString())).ToList().Sum(r=> r.Ticks))
};
}).ToList();
The Conversion method is,
private static TimeSpan ConvertTimeSpan(string str)
{
TimeSpan outputValue;
TimeSpan.TryParse(str, out outputValue);
return outputValue;
}
C# Fiddle with sample data.
i have 'created date' and 'closed date' in my file and i'm converting it in json so i have that dates in json.
in my method i have two parameter like from date and to date and i want to count particular column data of my file between from date and to date.so how can we write code to fetch it using linq.
i tried this...
public JsonResult StatusDerails(DateTime from,DateTime to)
{
string csvurl = WebConfigurationManager.AppSettings["csvfileurl"];
var lines = System.IO.File.ReadAllLines(csvurl).Skip(1);
List<Product> prdt = new List<Product>();
foreach (string line in lines)
{
Product c1 = new Product();
var split = line.Split(',');
c1.ID = Int32.Parse(split[0]);
c1.Area_Path = split[1];
c1.IterationPath = split[2];
c1.State = split[3];
c1.Reason = split[4];
c1.Priority = Int32.Parse(split[5]);
c1.Severity = split[6];
c1.Tags = split[7];
c1.Title = split[8];
c1.CreatedDate = split[9];
c1.CreatedBy = split[10];
c1.ResolvedDate = split[11];
c1.ResolvedBy = split[12];
c1.ClosedDate = split[13];
c1.AssignedTo = split[14];
prdt.Add(c1);
}
//var list = prdt.GroupBy(a=>a.AreaPath).Select(a=>new UIproduct() {
var productName = prdt.Select(a => a.Area_Path).Distinct();
List<StatusDetail> statusdetail = new List<StatusDetail>();
foreach (var Name in productName)
{
StatusDetail sd = new StatusDetail();
sd.CarryOver = prdt.Where(a => a.CreatedDate >= from.Date.ToString() && a.ClosedDate <= to.Date.ToShortDateString
}
return Json(statusdetail, JsonRequestBehavior.AllowGet);
}
The comparison of DateTime as string will not be a good option and that wont gives you the exact result, So I recommend you to change the type of CreatedDate and ClosedDate to DateTime. and compare two DateTime values in linq. I think instead of splitting json for creating object of certain types you can use json converters.
Fix for your scenario:
c1.CreatedDate = DateTime.Parse(split[9]);
c1.ClosedDate = DateTime.Parse(split[13]);
Don't forget to change the type in the class, Now its fine to use the linq as like the following:
sd.CarryOver = prdt.Where(a => a.CreatedDate >= from.Date && a.ClosedDate <= to.Date);
I'm trying to convert the string separated by commas which has 7 values of:
2014-21-2,1207.81,1209.87,1202.84,1203.79,1862300,1203.79
To another model which is:
return lines[1].Split(',').Select(i => new StockModel
{
StockDate = DateTime.ParseExact(i.ToString(), "yyyy-MM-dd", null),
StockOpen = float.Parse(i.ToString()),
StockHigh = float.Parse(i.ToString()),
StockLow = float.Parse(i.ToString()),
StockClose = float.Parse(i.ToString()),
StockVolume = Convert.ToInt32(i.ToString()),
StockAdjustedClose = float.Parse(i.ToString()),
StockSymbol = stockSymbol
}).SingleOrDefault();
However I get errors such as: Additional information: Input string was not in a correct format. http://s17.postimg.org/ro4k3tzct/Screenshot_1.png
If I do it manually like: DateTime date = DateTime.Parse(lines[1].Split(',')[0]), it works fine.
Whatever value I'm trying to put into the new Model, I get errors such as this one.
OK, I see the problem. You shouldn't use Select here. Try following instead:
var i = lines[1].Split(',');
return new StockModel()
{
StockDate = DateTime.ParseExact(i[0].ToString(), "yyyy-MM-dd", null),
StockOpen = float.Parse(i[1].ToString()),
StockHigh = float.Parse(i[2].ToString()),
StockLow = float.Parse(i[3].ToString()),
StockClose = float.Parse(i[4].ToString()),
StockVolume = Convert.ToInt32(i[5].ToString()),
StockAdjustedClose = float.Parse(i[6].ToString()),
StockSymbol = stockSymbol
};
i am writing code for search page and i have to pass some filters to the action and depending on those input I have to generate hyper links, hence i am using Url.Action function to generate links.
below is my code
#Url.Action("Index","Search",new SkillKindleWeb.ViewModels.Search.SearchRawInput()
{
CategoryIds = Model.Request.CategoryIds,
SubCategoryIds = Model.Request.SubCategoryIds,
StartDate = Model.Request.StartDate,
EndDate = Model.Request.EndDate,
StartPrice = Model.Request.StartPrice,
LocationGroupIds = Model.Request.LocationGroupIds,
LocationIds = Model.Request.LocationIds,
EndPrice = Model.Request.EndPrice,
City = Model.Request.City,
PageNo = 1,
SearchQuery = Model.Request.SearchQuery,
Segment1 = Model.Request.Segment1,
Segment2 = Model.Request.Segment2,
TargetAge = Model.Request.TargetAge
})
and it is generating url like this
http://someDomain.com/ncr/classes?CategoryIds=System.Collections.Generic.List%601%5BSystem.Int32%5D&StartDate=03%2F30%2F2013%2000%3A00%3A00&StartPrice=0&EndPrice=140000&PageNo=2
My expected Url was
http://SomeDomain.com/ncr/classes?CategoryIds=9&StartDate=3/30/2013&StartPrice=0&EndPrice=140000
What about converting it to string representation yourself like that:
#Url.Action("Index","Search",new SkillKindleWeb.ViewModels.Search.SearchRawInput()
{
CategoryIds = string.Join(",", Model.Request.CategoryIds),
SubCategoryIds = string.Join(",", Model.Request.SubCategoryIds),
StartDate = Model.Request.StartDate.ToShortDateString(),
EndDate = Model.Request.EndDate.ToShortDateString(),
StartPrice = Model.Request.StartPrice,
LocationGroupIds = Model.Request.LocationGroupIds,
LocationIds = Model.Request.LocationIds,
EndPrice = Model.Request.EndPrice,
City = Model.Request.City,
PageNo = 1,
SearchQuery = Model.Request.SearchQuery,
Segment1 = Model.Request.Segment1,
Segment2 = Model.Request.Segment2,
TargetAge = Model.Request.TargetAge
})
That is what a viewmodel should be for. That you convert and format all the values you need in the way the view expects it.
Notice that I added a ToShortDateString() to your dates as well, since it seems you are not interested in the time part.