We Keep Coding

ElasticSearch Filter and match using NEST query

I'm currently working on implementing ES in .NET using NEST.
problem:
I have four docs in my ES index and I want to get the records which contain a word and date greater than today's date.
class P{
public int id;
public string text;
public DateTime date;
}
for eg: these are the four records I have.(these are the values passed to ES)
id:1, text: this is post1, date= DateTime.Now;
id:1, text: this is another post, date= DateTime.Now;
id:2, text: This is post from ES, date= DateTime.Now;
id:3, text: this is post from ES FUTURE, date= DateTime.Now.AddDays(5);
this is my version of code for the same:
var result =
client.Search<P>(s => s
.Query(p => p.Term(q => q.text, "ES"))
.Query(b => b.Bool(a => a.Filter(t=>t.Range(r=>r.Field("date").GreaterThan(2016-08-04))))) );
Now I want to write a query using Lamba query to find all the docs which contain the word ES and the date greater than today's date.(expected output should be 4)
Can someone help me in solving this?

Sort List by date values

I have the following list -
List<string> finalMessageContent
where
finalMessageContent[0] = "<div class="mHr" id="mFID">
<div id="postedDate">11/12/2015 11:12:16</div>
</div>" // etc etc
I am trying to sort the list by a particular value located in the entires - postedDate tag.
Firstly I have create an new object and then serialized it to make the html elements able to be parsed -
string[][] newfinalMessageContent = finalMessageContent.Select(x => new string[] { x }).ToArray();
string json = JsonConvert.SerializeObject(newfinalMessageContent);
JArray markerData = JArray.Parse(json);
And then used Linq to try and sort using OrderByDescending -
var items = markerData.OrderByDescending(x => x["postedDate"].ToString()).ToList();
However this is failing when trying to parse the entry with -
Accessed JArray values with invalid key value: "postedDate". Array position index expected.
Perhaps linq is not the way to go here however it seemed like the most optimised, where am I going wrong?

First, i would not use string methods, regex or a JSON-parser to parse HTML. I would use HtmlAgilityPack. Then you could provide such a method:
private static DateTime? ExtractPostedDate(string inputHtml, string controlID = "postedDate")
{
var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(inputHtml);
HtmlNode div = doc.GetElementbyId(controlID);
DateTime? result = null;
DateTime value;
if (div != null && DateTime.TryParse(div.InnerText.Trim(), DateTimeFormatInfo.InvariantInfo, DateTimeStyles.None, out value))
result = value;
return result;
}
and following LINQ query:
finalMessageContent = finalMessageContent
.Select(s => new { String = s, Date = ExtractPostedDate(s) })
.Where(x => x.Date.HasValue)
.OrderByDescending(x => x.Date.Value)
.Select(x => x.String)
.ToList();

Don't know if I get your question right.
But did you know that you can parse HTML with XPath?
foreach (var row in doc.DocumentNode.SelectNodes("//div[#id="postedDate"]"))
{
Console.WriteLine(row.InnerText);
}
this is just an example from the top of my head you might have to double-check the XPath query depending on your document. You can also consider converting it to array or parsing the date and do other transformations with it.
Like I said this is just from the top of my head. Or if the html is not so compley consider to extract the dates with an RegEx but this would be a topic for another question.
HTH

Json Serializer serializes JSON typed strings. Example here to json
To parse HTML I suggest using HtmlAgility https://htmlagilitypack.codeplex.com/
Like this:
HtmlAgilityPack.HtmlDocument htmlparsed = new HtmlAgilityPack.HtmlDocument();
htmlParsed.LoadHtml(finalMessageContent[0]);
List<HtmlNode> OrderedDivs = htmlParsed.DocumentNode.Descendants("div").
Where(a => a.Attributes.Any(af => af.Value == "postedDate")).
OrderByDescending(d => DateTime.Parse(d.InnerText)); //unsafe parsing

Linq-to-xml to get child nodes

I am having trouble identifying how to use linq-to-xml to extract total price and individual prices from the xml below (e.g I want to get the fare price and also sum of all prices). Any help would be much appreciated especially with using the method syntax of linq-to-xml
I use the following code to get the data loaded into an xDocument and work with the xmlResponse object to parse the response.
var xmlResponse = from element in xdoc.Descendants()
select element;
and get data like
xmlResponse.SingleOrDefault(x => x.Name.LocalName == "Registration")
Below is a subset of thwe xml response :-
<StateList>
<State>
<SourceJobID>J999999999999</SourceJobID>
<TargetJobState>Complete</TargetJobState>
<TargetJobID>11111111</TargetJobID>
<TargetSystem>TESTSYSTEM</TargetSystem>
<VehicleDetails>
<Registration>TESTREGISRATION</Registration>
<Plate>11111111111</Plate>
<CO2Rating>160</CO2Rating>
<Badge>1111111</Badge>
<Description>TEST DESCRIPTION</Description>
</VehicleDetails>
<CompleteDetails>
<CompletedOn>2015-09-15T13:39:11+01:00</CompletedOn>
<JobDistance>0</JobDistance>
<WaitingTime />
<CO2Usage>0</CO2Usage>
<ChargeList>
<Charge>
<Name>Airport Pickup</Name>
<Currency>GBP</Currency>
<Price>0.00</Price>
</Charge>
<Charge>
<Name>Fare</Name>
<Currency>GBP</Currency>
<Price>0.00</Price>
</Charge>
<Charge>
<Name>Extra Stops</Name>
<Currency>GBP</Currency>
<Price>0.00</Price>
</Charge>
</ChargeList>
</CompleteDetails>
</State>

Assuming you only have a single state like in your example, you could do something like the following:
decimal fare = decimal.Parse(xml.Descendants("Charge").Single(x => x.Element("Name").Value == "Fare").Element("Price").Value);
decimal total = xml.Descendants("Charge").Sum(x => decimal.Parse(x.Element("Price").Value));
Although if you have a series of elements in your list you will have to modify that.
EDIT: If, as you say in the comments, you would like to sum only certain charges:
// Valid names of charges to sum.
string[] names = { "Airport Pickup", "Fare" };
// Iterate over every state.
foreach (var state in xml.Descendants("State"))
{
// Get all charge elements in the current state whose names are contained in 'names' - then convert their 'Price' element to decimal and sum them.
decimal stateTotal = state.Descendants("Charge").Where(x => names.Contains(x.Element("Name").Value)).Sum(x => decimal.Parse(x.Element("Price").Value));
}

if(doc.Descendants("Charge").Any())
{
var FarePrice = doc.Descendants("Charge")
.Where(x => x.Descendants("Name").First().Value.Equals("Fare")).First().Element("Price").Value;
var Sum = doc.Descendants("Charge")
.Select(x => Convert.ToDouble(x.Descendants("Price").First().Value))
.Sum();
Console.WriteLine("Fare price:{0} Sum:{1}",FarePrice,Sum);
}
It returns 35 as sum for 10 and 25 inputs.
Fiddle here : https://dotnetfiddle.net/cuHXBn

String was not recognized as a valid DateTime - Whats wrong?

I have been getting an annoying littler error and cannot for the life of me figure out why it is being cause. I have an xml file where i am storing data, as shown below.
- <EmployeeFinance>
<EmployeeEmploy_Id>5584</EmployeeEmploy_Id>
<EmpPersonal_Id>30358</EmpPersonal_Id>
<No_DaysWorked>30</No_DaysWorked>
<Date_Appointment>17/02/2012</Date_Appointment>
<Date_Employment>02/05/1984</Date_Employment>
<Date_Termination>01/01/0001</Date_Termination>
<Payperiod_StartDate>01/01/2013</Payperiod_StartDate>
<Payperiod_EndDate>31/01/2013</Payperiod_EndDate>
<BatchNumber>38</BatchNumber>
<PAYE_ToDate_Computed>0</PAYE_ToDate_Computed>
<Income_Tax_RateID>0</Income_Tax_RateID>
<NIS_RateID>0</NIS_RateID>
<NIS_weeks_worked>0</NIS_weeks_worked>
</EmployeeFinance>
If you look at the date nodes, Payperiod_StartDate,Payperiod_EndDate, Date_Appointment etc. They all have the same format. Now in my C# code, when i write my query to select from the xml file i get the String was not recognized as a valid DateTime error. WHen i comment out all the other dates and leave start_date, it works. They are the same format , i cant see what i am doing wrong. Please help me.
var context = new SSPModel.sspEntities();
XElement xelement = XElement.Load(GlobalClass.GlobalUrl);
XDocument doc = XDocument.Load(GlobalClass.GlobalUrl);
var query = from nm in xelement.Elements("EmployeeFinance")
select new EmployeeEmploy
{
Employee_Personal_InfoEmp_id = (int)nm.Element("EmpPersonal_Id"),
Substantive_designation = (int)nm.Element("Position_Id"),
Grade_Id = (int)nm.Element("Grade_Id"),
PositionTotal_PtBasic = (double)nm.Element("Sum_AllPosition"),//part of basic
GradeTotal_PtBasic = (double)nm.Element("Sum_AllGrade"), //part of basic
Housing_Allowance = (double)nm.Element("Housing"),
Base_Pay = (double)nm.Element("Base_Pay"),
startDate = (DateTime)nm.Element("Payperiod_StartDate"),
endDate = (DateTime)nm.Element("Payperiod_EndDate"),
Date_of_Appointment = (DateTime)nm.Element("Date_Appointment"),
Date_of_Employment = (DateTime)nm.Element("Date_Employment"),
Termination_date_actual = (DateTime)nm.Element("Date_Termination"),
Base_Pay_Currency = (string)nm.Element("Currency"),
Exchange_rate = (double)nm.Element("Exchange_Rate")
};
var x = query.ToList();
foreach (var xy in x) {
Debug.WriteLine(xy.endDate);
}

Because 17/02/2012 is not a valid date, however, 02/17/2012 is. The date will be parsed as mm/dd/yyyy. One option is to use DateTime.ParseExact to parse a date with the dd as the first set of numbers. e.g.
var startDate = DateTime.ParseExact("17/02/2012", "dd/MM/yyyy", null);

The debugger will show you that nm.Element("Payperiod_EndDate").ToString() gives you a string that includes the xml tags for that element. Try the following instead:
startDate = DateTime.ParseExact(nm.Element("Payperiod_EndDate").Value, "dd/MM/yyyy", null)

How to query with the datetime value using LINQ to XML?

I am developing window phone 7 application in silverlight. I am new to the silverlight. I am also new to LINQ to XML. In my application the user select the date & submit some transaction details into the application. The details gets stored in XML File. I am using the custom date control in my application for the date selection as follows
private void DatePicker_ValueChanged(object sender, DateTimeValueChangedEventArgs e)
{
AppObj = Application.Current as App;
AppObj.date = (DateTime)EntryDate.Value;
}
Then the value of AppObj.date gets stored in the XML file. Sometimes I use the DateTime.Now to store the date in the XML File. Now I want to generate the report of submitted transaction details by querying through LINQ to XML. I want to generate the report for today's date, current week & current month. For today's date report I am using the following code
public void GetTransactionObjects(String strXMLFile, DateTime VDateTime)
{
XDocument doc = null;
XMLFileManager XMLDocObj = new XMLFileManager();
doc = XMLDocObj.LoadXMLFile(strXMLFile);
var vTransaction = from s in doc.Descendants("Transaction")
.Where(x => x.Element("Current_Date").Value == VDateTime.ToShortDateString())
select new Transaction(s);
this.Clear();
AddRange(vTransaction);
}
The Transaction class contains the following constructor.
public Transaction(XElement xElement)
{
Transaction_ID = Convert.ToInt32(xElement.Element("Transaction_ID").Value.ToString());
TransactionType_ID = Convert.ToInt32(xElement.Element("TransactionType_ID").Value.ToString());
Alphabet_ID = Convert.ToInt32(xElement.Element("Alphabet_ID").Value.ToString());
ID = Convert.ToInt32(xElement.Element("ID").Value.ToString());
SubCategory_ID = Convert.ToInt32(xElement.Element("SubCategory_ID").Value.ToString());
Item_ID = Convert.ToInt32(xElement.Element("Item_ID").Value.ToString());
Currency_ID = Convert.ToInt32(xElement.Element("Currency_ID").Value.ToString());
InputTypeMethod_ID = Convert.ToInt32(xElement.Element("InputTypeMethod_ID").Value.ToString());
Principle = Convert.ToInt32(xElement.Element("InputTypeMethod_ID").Value.ToString());
Interest = Convert.ToInt32(xElement.Element("Interest").Value.ToString());
ROI = Convert.ToInt32(xElement.Element("InputTypeMethod_ID").Value.ToString());
Amount = Convert.ToInt32(xElement.Element("InputTypeMethod_ID").Value.ToString());
Current_Date = Convert.ToDateTime(xElement.Element("Current_Date").Value.ToString());
}
In the XML File the value gets stored for date & time. The value gets stored as follows
<Transactions>
<Transaction>
<Transaction_ID>0</Transaction_ID>
<TransactionType_ID>0</TransactionType_ID>
<Alphabet_ID>3</Alphabet_ID>
<ID>0</ID>
<SubCategory_ID>0</SubCategory_ID>
<Item_ID>0</Item_ID>
<Currency_ID>3</Currency_ID>
<InputTypeMethod_ID>0</InputTypeMethod_ID>
<Principle>0</Principle>
<Interest>0</Interest>
<ROI>0</ROI>
<Amount>5000</Amount>
<Current_Date>2010-12-31T18:08:23.433+05:30</Current_Date>
</Transaction>
</Transactions>
Look at the node
<Current_Date>2010-12-31T18:08:23.433+05:30</Current_Date>
The date format is yyyy-mm-dd.
Now how should I write the following query to get all the submitted transaction details for today's date ?
var vTransaction = from s in doc.Descendants("Transaction")
.Where(x => x.Element("Current_Date").Value == VDateTime.ToShortDateString())
select new Transaction(s);
Similarly how should I write the query to get all the transaction details for the current week & current month? Can you please provide me any code or link through which I can resolve the above issue ? If I am doing anything wrong then please guide me.

Don't use Convert.ToDateTime or ToShortDateString etc with LINQ to XML. Use the conversions which already exist in XAttribute and XElement. For example:
DateTime today = DateTime.Today;
var query = doc.Descendants("Transaction")
.Where(x => ((DateTime) x.Element("Current_Date")).Date == today)
.Select(x => new Transaction(s));
(You should use the conversion operator in your Transaction constructor too.)