We Keep Coding

New class selected by LINQ query, how to transfer to another model?

I'm really not understanding this as I've only dabbled in MVC and C#. I apologize if my terminology is wrong or confusing, I will do my best to answer questions. I have a couple models like so:
public class DataSharingModels
{
public string ReferenceID { get; set; }
public NBTC NBTCGroup { get; set; }
public Contractors ContractorsGroup { get; set; }
public Coordinators CoordinatorsGroup { get; set; }
public NGO NGOGroup { get; set; }
public Public PublicGroup { get; set; }
public SelectList FA_RA_List { get; set; }
}
public class NBTC
{
public String NBTC_FA_Centroid { get; set; }
public String NBTC_FA_Bound { get; set; }
public String NBTC_RA_Centroid { get; set; }
//more properties...
}
The DataSharingModels class contains the public NBTC NBTCGroup property. It is not public List<NBTC> NBTCGroup because there will only be one produced per instance of the controller being hit.
Now in my controller, I have a LINQ statement that selects a new NBTC class:
var nbtcVals = (from ds in db.SharingPermissions
where ds.FocalRefID.ToString() == ReferenceID
&& ds.ShareGroup == "NBTC"
select new NBTC
{
NBTC_FA_Centroid = ds.CIP_FA_Centroid,
NBTC_FA_Bound = ds.CIP_FA_Boundary,
NBTC_RA_Centroid = ds.CIP_RA_Centroid,
//more properties...
});
Where I'm going wrong is I would like to add that to my DataSharingModels model. I thought the nbtcVals type would be NBTC, but it's IQueryable<##.Models.NBTC>. I understand I could do this, but it seems redundant:
DataSharingModels dsm = new DataSharingModels();
if (nbtcVals.Any())
{
foreach (var i in nbtcVals)
{
dsm.NBTCGroup.NBTC_FA_Centroid = i.NBTC_FA_Centroid;
dsm.NBTCGroup.NBTC_FA_Boundary = i.NBTC_FA_Bound;
dsm.NBTCGroup.NBTC_RA_Centroid = i.NBTC_RA_Centroid;
//more properties...
}
}
What is a more direct way to do this? There must be one. I supposed I could also return an anonymous type in the LINQ query and then assign each property in the foreach like dsm.NBTCGroup.NBTC_RA_Centroid = i.NBTC_RA_Centroid but that seems the same as the other way.

var nbtcgroup = (from ds in db.SharingPermissions
where ds.FocalRefID.ToString() == ReferenceID
&& ds.ShareGroup == "NBTC"
select new NBTC
{
NBTC_FA_Centroid = ds.CIP_FA_Centroid,
NBTC_FA_Bound = ds.CIP_FA_Boundary,
NBTC_RA_Centroid = ds.CIP_RA_Centroid,
//more properties...
})
.OrderByDescending(n => n.Id) // or some other property that could identify sorting
.FirstOrDefault();
This one has a translation to SQL (LIMIT or TOP depending on backend).

NEST Api SearchAfter return null in NEST but works in Kibana

We are using elastic search just for document search in our application so we don't have any one expert in it. I was able to use TermQuery, SimpleQueryStringQuery and MatchPhraseQuery successfully. But I found out in documentation that using From & Size for pagination is not good for production and Search After is recommended.
But my implementation return null. It is confusing for me what should be in <Project> parameter as shown in Nest API Object Initializer Syntax in docs here.
My code looks like this:
var request = new SearchRequest<ElasticSearchJsonObject._Source>
{
//Sort = new List<ISort>
//{
// new SortField { Field = Field<ElasticSearchJsonObject>(p=>)}
//},
SearchAfter = new List<object> {
},
Size = 20,
Query = query
};
Reality is I don't understand this. Over here ElasticSearchJsonObject._Source is the class to map returned results.
My documents are simple text documents and I only want documents sorted according to score so document Id is not relevant.
There was already a question like this on SO but I can't find it somehow.
Update
After looking at answer I updated my code and though query obtained does work. It return result in kibana but not in NEST.
This is the new updated code:
var request = new SearchRequest<ElasticSearchJsonObject.Rootobject>
{
Sort = new List<ISort>
{
new SortField { Field = "_id", Order = SortOrder.Descending}
},
SearchAfter = new List<object> {
"0fc3ccb625f5d95b973ce1462b9f7"
},
Size = 1,
Query = query
};
Over here I am using size=1 just for test as well as hard code _id value in SearchAfter.
The query generated by NEST is:
{
"size": 1,
"sort": [
{
"_id": {
"order": "desc"
}
}
],
"search_after": [
"0fc3ccb625f5d95b973ce1462b9f7"
],
"query": {
"match": {
"content": {
"query": "lahore",
"fuzziness": "AUTO",
"prefix_length": 3,
"max_expansions": 10
}
}
}
}
The response from the ES does say successful but no results are returned.
Results do return in Kibana
Query status is successful
But...
Total returned is 0 in NEST
Sort value is null in kibana I used TrackScores = true to solve this issue
Here is the debug information:
Valid NEST response built from a successful low level call on POST: /extract/_source/_search?typed_keys=true
# Audit trail of this API call:
- [1] HealthyResponse: Node: http://localhost:9200/ Took: 00:00:00.1002662
# Request:
{"size":1,"sort":[{"_id":{"order":"desc"}}],"search_after":["0fc3ccb625f5d95b973ce1462b9f7"],"query":{"match":{"content":{"query":"lahore","fuzziness":"AUTO","prefix_length":3,"max_expansions":10}}}}
# Response:
{"took":3,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}
So please tell me where I am wrong and what can be the problem and how to solve it.
Update 2:
Code in Controller:
Connection String:
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(node);
settings.DisableDirectStreaming();
settings.DefaultIndex("extract");
var client = new ElasticClient(settings);
Query:
var query = (dynamic)null;
query = new MatchQuery
{
Field = "content",
Query = content,
Fuzziness = Fuzziness.Auto,
PrefixLength = 3,
MaxExpansions = 10
};
Query Builder
var request = new SearchRequest<ElasticSearchJsonObject.Rootobject>
{
Sort = new List<ISort>
{
new SortField { Field = "_id", Order = SortOrder.Descending}
},
SearchAfter = new List<object> {
documentid //sent as parameter
},
Size = 1, //for testing 1 other wise 10
TrackScores = true,
Query = query
};
JSON Query
I use this code to get query I posted above. This query is then passed to kibana with GET <my index name>/_Search and there it works
var stream = new System.IO.MemoryStream();
client.SourceSerializer.Serialize(request, stream);
var jsonQuery = System.Text.Encoding.UTF8.GetString(stream.ToArray());
ES Response
string responseJson = "";
ElasticSearchJsonObject.Rootobject response = new ElasticSearchJsonObject.Rootobject();
var res = client.Search<object>(request);
if (res.ApiCall.ResponseBodyInBytes != null)
{
responseJson = System.Text.Encoding.UTF8.GetString(res.ApiCall.ResponseBodyInBytes);
try
{
response = JsonConvert.DeserializeObject<ElasticSearchJsonObject.Rootobject>(responseJson);
}
catch (Exception)
{
var model1 = new LoginSignUpViewModel();
return PartialView("_NoResultPage", model1);
}
}
This is where things go wrong. Above debug information was captured from response
ElasticSearchJsonObject
Some how I think problem might be here somewhere? The class is generated by taking response from NEST in Search request.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
namespace ESAPI
{
public class ElasticSearchJsonObject
{
public class Rootobject
{
public int took { get; set; }
public bool timed_out { get; set; }
public _Shards _shards { get; set; }
public Hits hits { get; set; }
}
public class _Shards
{
public int total { get; set; }
public int successful { get; set; }
public int skipped { get; set; }
public int failed { get; set; }
}
public class Hits
{
public int total { get; set; }
public float max_score { get; set; }
public Hit[] hits { get; set; }
}
public class Hit
{
public string _index { get; set; }
public string _type { get; set; }
public string _id { get; set; }
public float _score { get; set; }
public _Source _source { get; set; }
}
public class _Source
{
public string content { get; set; }
public Meta meta { get; set; }
public File file { get; set; }
public Path path { get; set; }
}
public class Meta
{
public string title { get; set; }
public Raw raw { get; set; }
}
public class Raw
{
public string XParsedBy { get; set; }
public string Originator { get; set; }
public string dctitle { get; set; }
public string ContentEncoding { get; set; }
public string ContentTypeHint { get; set; }
public string resourceName { get; set; }
public string ProgId { get; set; }
public string title { get; set; }
public string ContentType { get; set; }
public string Generator { get; set; }
}
public class File
{
public string extension { get; set; }
public string content_type { get; set; }
public DateTime last_modified { get; set; }
public DateTime indexing_date { get; set; }
public int filesize { get; set; }
public string filename { get; set; }
public string url { get; set; }
}
public class Path
{
public string root { get; set; }
public string _virtual { get; set; }
public string real { get; set; }
}
}
}
I am sure this can be used to get response.
Please note that in case of simple search this code works:
so for this query below my code is working:
var request = new SearchRequest
{
From = 0,
Size = 20,
Query = query
};

Using from/size is not recommended for deep pagination because of the amount of documents that need to be fetched from all shards for a deep page, only to be discarded when finally returning an overall ordered result set. This operation is inherent to the distributed nature of Elasticsearch, and is common to many distributed systems in relation to deep pagination.
With search_after, you can paginate forward through documents in a stateless fashion and it requires
the documents returned from the first search response are sorted (documents are sorted by _score by default)
passing the values for the sort fields of the last document in the hits from one search request as the values for "search_after": [] for the next request.
In the Search After Usage documentation, a search request is made with sort on NumberOfCommits descending, then by Name descending. The values to use for each of these sort fields are passed in SearchAfter(...) and are the values of Project.First.NumberOfCommits and Project.First.Name properties, respectively. This tells Elasticsearch to return documents that have values for the sort fields that correspond to the sort constraints for each field, and relate to the values supplied in the request. For example, sort descending on NumberOfCommits with a supplied value of 775 means that Elasticsearch should only consider documents with a value less than 775 (and to do this for all sort fields and supplied values).
If you ever need to dig further into any NEST documentation, click the "EDIT" link on the page:
which will take you to the github repository of the documentation, with the original asciidoc markdown for the page:
Within that page will be a link back to the original NEST source code from which the asciidoc was generated. In this case, the original file is SearchAfterUsageTests.cs in the 6.x branch

Properties of referenced class not showing during internal assignment

I have below ItemCountAmount.cs class.
public class ItemCountAmount
{
public int Count { get; set; }
public string Amount { get; set; }
}
and I am referencing it in one more class ChartData.cs as below:
public class ChartData
{
public ChartData()
{
Purchase = Stocks = Sales = new ItemCountAmount();
}
public string period { get; set; }
public ItemCountAmount Purchase { get; set; }
public ItemCountAmount Stocks { get; set; }
public ItemCountAmount Sales { get; set; }
}
Now when I try to assign the values for the ChartData properties, it isn't recognizing or available. Compile time exception will occur in VS.
var model = new ChartData
{
period = yearMonth,
Purchase.Count = await purchase.CountAsync(x => x.order_date.Month == mnth && x.order_date.Year == yr),
}
But Purchase.Count or Purchase.Amount isn't identified here where in if I try it as
var model=new ChartData();
model.Purchase.Count = .....;
model.Purchase.Amount = .....;
.......
it works fine, I mean, property Count and Amount is identified. I am still not getting why the first method is not able to identify the properties from ItemCountAmount.cs class which is referenced in ChartData. Any specific reasons for this?

I don't believe the C# object initialiser supports that syntax, however, you can just do this. I am not sure whether this would result in two constructions and assignemnts to Purchase though, you could break point that yourself and see.
var model = new ChartData
{
Purchase = new ItemCountAmount
{
count = await purchase.CountAsync(x => x.order_date.Month == mnth && x.order_date.Year == yr)
}
}

MongoDB Unable to determine the serialization information for the expression error

My data is having following structure
public enum ParamType
{
Integer=1,
String=2,
Boolean=3,
Double=4
}
public class Gateway
{
public int _id { get; set; }
public string SerialNumber { get; set; }
public List<Device> Devices { get; set; }
}
public class Device
{
public string DeviceName { get; set; }
public List<Parameter> Parameters { get; set; }
}
public class Parameter
{
public string ParamName { get; set; }
public ParamType ParamType { get; set; }
public string Value { get; set; }
}
I filled 10 document objects of Gateway in a MongoDB database.
Now I want to query all those gateways which contains a device having Parameter with ParamName as "Target Temperature" and whose Value > 15.
I created following queries
var parameterQuery = Query.And(Query<Parameter>.EQ(p => p.ParamName, "Target Temperature"), Query<Parameter>.GT(p => int.Parse(p.Value), 15));
var deviceQuery = Query<Device>.ElemMatch(d => d.Parameters, builder => parameterQuery);
var finalQuery = Query<Gateway>.ElemMatch(g => g.Devices, builder => deviceQuery);
But when I run this, it is giving an exception
Unable to determine the serialization information for the expression: (Parameter p) => Int32.Parse(p.Value)
Please suggest where I am wrong.

As the error suggests, you can't use Int32.Parse inside your query. This lambda expression is used to get out the name of the property and it doesn't understand what Int32.Parse is.
If you are querying a string, you need to use a string value for comparison:
var parameterQuery = Query.And(Query<Parameter>.EQ(p => p.ParamName, "Target Temperature"), Query<Parameter>.GT(p => p.Value, "15"));
However, that's probably not what you want to do since you're using GT. To be treated as a number for this comparison you need the value to actually be an int in mongo, so you would need to change the type of your property:
public class Parameter
{
public string ParamName { get; set; }
public ParamType ParamType { get; set; }
public int Value { get; set; }
}
var parameterQuery = Query.And(Query<Parameter>.EQ(p => p.ParamName, "Target Temperature"), Query<Parameter>.GT(p => p.Value, 15));

Summary
I ran into this when I was modifying a list. It appears that Linq First/FirstOrDefault was not handled well by MongoDB for me. I changed to be an array index var update = Builders<Movie>.Update.Set(movie => movie.Movies[0].MovieName, "Star Wars: A New Hope"); Note: This is in Asp.Net 5 using MongoDB.Driver 2.2.0.
Full Example
public static void TypedUpdateExample() {
var client = new MongoClient("mongodb://localhost:27017");
var database = client.GetDatabase("test");
var collection = database.GetCollection<Movie>("samples");
//Create some sample data
var movies = new Movie {
Name = "TJ",
Movies = new List<MovieData>
{
new MovieData {
MovieName = "Star Wars: The force awakens"
}
}
};
collection.InsertOne(movies);
//create a filter to retreive the sample data
var filter = Builders<Movie>.Filter.Eq("_id", movies.Id);
//var update = Builders<Movie>.Update.Set("name", "A Different Name");
//TODO:LP:TSTUDE:Check for empty movies
var update = Builders<Movie>.Update.Set(movie => movie.Movies[0].MovieName, "Star Wars: A New Hope");
collection.UpdateOne(filter, update);
}
public class Movie {
[BsonId]
public ObjectId Id { get; set; }
public string Name { get; set; }
public List<MovieData> Movies { get; set; }
}
public class MovieData {
[BsonId]
public ObjectId Id { get; set; }
public string MovieName { get; set; }
}

DeviceUniqueId & Azure Mobile Service

Problem
I have Windows Phone 8 app. From the app users can upload photos, then add entry to the table for processing using Azure Mobile Service. One of the columns in a table is DeviceId.
This part works fine.
In another view folks could load list of entries they already have added. Unfortunately for some users this doesnt seem to work as it returns no entries (I can find their entries in database).
Please help as I cant figure out why it works for some users and not others.
Here is how I get DeviceUniqueId:
byte[] myDeviceID = (byte[])Microsoft.Phone.Info.DeviceExtendedProperties.GetValue("DeviceUniqueId");
string DeviceIDAsString = Convert.ToBase64String(myDeviceID);
Here is how I add new entries:
TimelapseBatch.DeviceId (first param in constructor is string)
var item = new TimelapseBatch(DeviceInfo.DeviceUniqueId, /*other properties go here*/);
await App.MobileService.GetTable<TimelapseBatch>().InsertAsync(item);
How I query the service
private readonly IMobileServiceTable<TimelapseBatch> batchesTable = App.MobileService.GetTable<TimelapseBatch>();
...
batches = await batchesTable.Where(t => t.DeviceId == DeviceInfo.DeviceUniqueId).ToListAsync();
var processed = batches.Where(t => t.StatusCode == (int)BatchStatus.Processed).OrderByDescending(t => t.Processed).Select(t => new Timelapse()
{
Page = new Uri(t.PlayerUrl),
Thumbnail = new Uri(t.ThumbnailUrl)
}).ToList();
Apparently after all of this processed doesnt contain any entries even though according to the database it should.
I did verify that when the same users (even those that experience issues) create multiple entries, DeviceId in the database seems to be the same.
Please help, why could it work for some users but not others???
UPDATE
So when I use DeviceUniqueId from customers who experience issues I can reproduce the problem.
Also, if I do it this way:
batches = await batchesTable.Take(200).ToListAsync();
batches = batches.Where(t => t.DeviceId == DeviceInfo.DeviceUniqueId).ToList();
It populates batches as expected. However if I do this (below), batches are empty (I specified 200 for testing purposes as there are fewer entries).
batches = await batchesTable.Take(200).Where(t => t.DeviceId == DeviceInfo.DeviceUniqueId).ToListAsync();
UPDATE 2
It seems like comparison fails if DeviceId contains "+" sign. If I do
t.DeviceId.Substring(0,X) == DeviceInfo.DeviceUniqueId.Substring(0,X)
Where X is position of + sign, then it works as expected. Omg. Now I'm desperately trying to find a workaround.
UPDATE 3
Type definition:
public enum BatchStatus
{
QueuedUp = 0,
Processed = 1,
Error = 2,
}
public class TimelapseBatch
{
public TimelapseBatch()
{
TimelapseId = String.Empty;
DeviceId = String.Empty;
ContainerUrl = String.Empty;
VideoMp4Url = String.Empty;
VideoWebmUrl = String.Empty;
ThumbnailUrl = String.Empty;
Uploaded = new DateTime(2000, 1, 1);
Processed = new DateTime(2000, 1, 1);
PlayerUrl = String.Empty;
}
public TimelapseBatch(string deviceId, string timelapseId, string containerUrl, int photosCount, int fps):this()
{
DeviceId = deviceId;
ContainerUrl = containerUrl;
TimelapseId = timelapseId;
PhotosCount = photosCount;
Fps = fps;
StatusCode = (int) BatchStatus.QueuedUp;
Uploaded = DateTime.Now;
}
public int Id { get; set; }
public int StatusCode { get; set; }
public string TimelapseId { get; set; }
public string DeviceId { get; set; }
public string ContainerUrl { get; set; }
public int PhotosCount { get; set; }
public int Fps { get; set; }
public DateTime Uploaded { get; set; }
public string VideoMp4Url { get; set; }
public string VideoWebmUrl { get; set; }
public string ThumbnailUrl { get; set; }
public double VideoSize { get; set; }
public DateTime Processed { get; set; }
public string PlayerUrl { get; set; }
}

And now the solution:
batches = await batchesTable.Where(t => t.DeviceId == DeviceInfo.DeviceUniqueId.Replace("+", "%2B") || t.DeviceId == DeviceInfo.DeviceUniqueId).ToListAsync();
I had to manually escape + sign as %2B. Added extra condition in case Azure team will fix it the bug.