We Keep Coding

Issues with Dapper query syntax

little assistance here with my query using dapper, been getting error Message = "ORA-00936: missing expression\n" on my query. I would like to know what am I missing here?
public class LocationDto
{
public int LocationId { get; set; }
public int RouteId { get; set; }
public string StartTime { get; set; }
public string Location { get; set; }
}
// Query Below
using (OracleConnection connection = new OracleConnection(ConfigurationManager.ConnectionStrings["ConnectionString"].ConnectionString))
{
try {
var x = connection.QueryAsync<LocationDto>("Select ROUTE_ID as RouteId, SCHEDULE_STOP as Location, START_TIME as StartTime From SCHEDULE WHERE ROUTE_ID = #Id", new { input.RouteId }).Result.ToList();
}
catch (Exception ex)
{
}
}

Firstly with Oracle queries you need to use : instead of # to denote a parameter placeholder.
Then secondly this code:
new { input.RouteId }
will produce an object with a property called RouteId.
This RouteId does not match the name of the parameter in your query, which is Id. They need to match, otherwise there's no way for the database to bind the parameter to the correct input value.
You can either
change the Sql query:
WHERE ROUTE_ID = :RouteId
OR
change the C#:
new { Id = input.RouteId }
and use :Id in the SQL.
It doesn't really matter which, the important thing is that the names match.

NEST Api SearchAfter return null in NEST but works in Kibana

We are using elastic search just for document search in our application so we don't have any one expert in it. I was able to use TermQuery, SimpleQueryStringQuery and MatchPhraseQuery successfully. But I found out in documentation that using From & Size for pagination is not good for production and Search After is recommended.
But my implementation return null. It is confusing for me what should be in <Project> parameter as shown in Nest API Object Initializer Syntax in docs here.
My code looks like this:
var request = new SearchRequest<ElasticSearchJsonObject._Source>
{
//Sort = new List<ISort>
//{
// new SortField { Field = Field<ElasticSearchJsonObject>(p=>)}
//},
SearchAfter = new List<object> {
},
Size = 20,
Query = query
};
Reality is I don't understand this. Over here ElasticSearchJsonObject._Source is the class to map returned results.
My documents are simple text documents and I only want documents sorted according to score so document Id is not relevant.
There was already a question like this on SO but I can't find it somehow.
Update
After looking at answer I updated my code and though query obtained does work. It return result in kibana but not in NEST.
This is the new updated code:
var request = new SearchRequest<ElasticSearchJsonObject.Rootobject>
{
Sort = new List<ISort>
{
new SortField { Field = "_id", Order = SortOrder.Descending}
},
SearchAfter = new List<object> {
"0fc3ccb625f5d95b973ce1462b9f7"
},
Size = 1,
Query = query
};
Over here I am using size=1 just for test as well as hard code _id value in SearchAfter.
The query generated by NEST is:
{
"size": 1,
"sort": [
{
"_id": {
"order": "desc"
}
}
],
"search_after": [
"0fc3ccb625f5d95b973ce1462b9f7"
],
"query": {
"match": {
"content": {
"query": "lahore",
"fuzziness": "AUTO",
"prefix_length": 3,
"max_expansions": 10
}
}
}
}
The response from the ES does say successful but no results are returned.
Results do return in Kibana
Query status is successful
But...
Total returned is 0 in NEST
Sort value is null in kibana I used TrackScores = true to solve this issue
Here is the debug information:
Valid NEST response built from a successful low level call on POST: /extract/_source/_search?typed_keys=true
# Audit trail of this API call:
- [1] HealthyResponse: Node: http://localhost:9200/ Took: 00:00:00.1002662
# Request:
{"size":1,"sort":[{"_id":{"order":"desc"}}],"search_after":["0fc3ccb625f5d95b973ce1462b9f7"],"query":{"match":{"content":{"query":"lahore","fuzziness":"AUTO","prefix_length":3,"max_expansions":10}}}}
# Response:
{"took":3,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}
So please tell me where I am wrong and what can be the problem and how to solve it.
Update 2:
Code in Controller:
Connection String:
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(node);
settings.DisableDirectStreaming();
settings.DefaultIndex("extract");
var client = new ElasticClient(settings);
Query:
var query = (dynamic)null;
query = new MatchQuery
{
Field = "content",
Query = content,
Fuzziness = Fuzziness.Auto,
PrefixLength = 3,
MaxExpansions = 10
};
Query Builder
var request = new SearchRequest<ElasticSearchJsonObject.Rootobject>
{
Sort = new List<ISort>
{
new SortField { Field = "_id", Order = SortOrder.Descending}
},
SearchAfter = new List<object> {
documentid //sent as parameter
},
Size = 1, //for testing 1 other wise 10
TrackScores = true,
Query = query
};
JSON Query
I use this code to get query I posted above. This query is then passed to kibana with GET <my index name>/_Search and there it works
var stream = new System.IO.MemoryStream();
client.SourceSerializer.Serialize(request, stream);
var jsonQuery = System.Text.Encoding.UTF8.GetString(stream.ToArray());
ES Response
string responseJson = "";
ElasticSearchJsonObject.Rootobject response = new ElasticSearchJsonObject.Rootobject();
var res = client.Search<object>(request);
if (res.ApiCall.ResponseBodyInBytes != null)
{
responseJson = System.Text.Encoding.UTF8.GetString(res.ApiCall.ResponseBodyInBytes);
try
{
response = JsonConvert.DeserializeObject<ElasticSearchJsonObject.Rootobject>(responseJson);
}
catch (Exception)
{
var model1 = new LoginSignUpViewModel();
return PartialView("_NoResultPage", model1);
}
}
This is where things go wrong. Above debug information was captured from response
ElasticSearchJsonObject
Some how I think problem might be here somewhere? The class is generated by taking response from NEST in Search request.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
namespace ESAPI
{
public class ElasticSearchJsonObject
{
public class Rootobject
{
public int took { get; set; }
public bool timed_out { get; set; }
public _Shards _shards { get; set; }
public Hits hits { get; set; }
}
public class _Shards
{
public int total { get; set; }
public int successful { get; set; }
public int skipped { get; set; }
public int failed { get; set; }
}
public class Hits
{
public int total { get; set; }
public float max_score { get; set; }
public Hit[] hits { get; set; }
}
public class Hit
{
public string _index { get; set; }
public string _type { get; set; }
public string _id { get; set; }
public float _score { get; set; }
public _Source _source { get; set; }
}
public class _Source
{
public string content { get; set; }
public Meta meta { get; set; }
public File file { get; set; }
public Path path { get; set; }
}
public class Meta
{
public string title { get; set; }
public Raw raw { get; set; }
}
public class Raw
{
public string XParsedBy { get; set; }
public string Originator { get; set; }
public string dctitle { get; set; }
public string ContentEncoding { get; set; }
public string ContentTypeHint { get; set; }
public string resourceName { get; set; }
public string ProgId { get; set; }
public string title { get; set; }
public string ContentType { get; set; }
public string Generator { get; set; }
}
public class File
{
public string extension { get; set; }
public string content_type { get; set; }
public DateTime last_modified { get; set; }
public DateTime indexing_date { get; set; }
public int filesize { get; set; }
public string filename { get; set; }
public string url { get; set; }
}
public class Path
{
public string root { get; set; }
public string _virtual { get; set; }
public string real { get; set; }
}
}
}
I am sure this can be used to get response.
Please note that in case of simple search this code works:
so for this query below my code is working:
var request = new SearchRequest
{
From = 0,
Size = 20,
Query = query
};

Using from/size is not recommended for deep pagination because of the amount of documents that need to be fetched from all shards for a deep page, only to be discarded when finally returning an overall ordered result set. This operation is inherent to the distributed nature of Elasticsearch, and is common to many distributed systems in relation to deep pagination.
With search_after, you can paginate forward through documents in a stateless fashion and it requires
the documents returned from the first search response are sorted (documents are sorted by _score by default)
passing the values for the sort fields of the last document in the hits from one search request as the values for "search_after": [] for the next request.
In the Search After Usage documentation, a search request is made with sort on NumberOfCommits descending, then by Name descending. The values to use for each of these sort fields are passed in SearchAfter(...) and are the values of Project.First.NumberOfCommits and Project.First.Name properties, respectively. This tells Elasticsearch to return documents that have values for the sort fields that correspond to the sort constraints for each field, and relate to the values supplied in the request. For example, sort descending on NumberOfCommits with a supplied value of 775 means that Elasticsearch should only consider documents with a value less than 775 (and to do this for all sort fields and supplied values).
If you ever need to dig further into any NEST documentation, click the "EDIT" link on the page:
which will take you to the github repository of the documentation, with the original asciidoc markdown for the page:
Within that page will be a link back to the original NEST source code from which the asciidoc was generated. In this case, the original file is SearchAfterUsageTests.cs in the 6.x branch

How to query LIST by passing parameter?

I am using Entity Framework 5.0 and I created my database from model. The below is the screenshot of the edmx diagram.
I am working towards to a below structure of data:
On given Client ID give me list of Theader which belongs to that ClientID and its TReports so I modeled my models as below:
public class TReportHeaderModel
{
public int ID { get; set; }
public int ClientID { get; set; }
public string THeaderTitle { get; set; }
public int RowNumber { get; set; }
public IList<TReportModel> TReports { get; set; }
}
public class TReportModel
{
public int ID { get; set; }
public string TReportName { get; set; }
public string URL { get; set; }
public int RowNumber { get; set; }
}
So when I query to get Theaders and its each report for given clientID:
I am listing the headers first for given clientID:
public IList<TReportHeaderModel> GetHeadersByClient(int ClientID)
{
using (var connection = new TReportEntitiesConnection())
{
var clientHeaders= (from st in connection.THeaders
where ClientID == st.ClientID
select new TReportHeaderModel
{
ID=st.ID,
THeaderTitle=st.THeaderTitle,
RowNumber=st.RowNumber
}).ToList();
return (clientHeaders);
}
}
And then to get the list of reports for each title and this is where I am stuck--->
public IList<TReportModel> GetChildReportsByHeader(int THeaderID)
{
using (var connection = new TReportEntitiesConnection())
{
// ....
}
}
Instead of separating it by get the headers by client first and then get the report by header id, is it possible to combine it in one method? sorry for the confusing explanation but I am new to LINQ Query so please understand.
The below is the ideal structure for the UI implemetation:
Client ID =2
Header 1
TReportName
URL
Header 2
TReportName
URL

is it possible to combine it in one method?
If I understand you correctly, this is what you're looking for:
using (var connection = new TReportEntitiesConnection())
{
var clientHeaders = (
from st in connection.THeaders
where ClientID == st.ClientID
select new TReportHeaderModel
{
ID=st.ID,
THeaderTitle = st.THeaderTitle,
RowNumber = st.RowNumber,
Reports = from r in st.TReports
select new TReportModel
{
ID = r.ID,
TReportName = r.TReportName,
URL = r.URL,
RowNumber = r.RowNumber,
}
}
).ToList();
}
return clientHeaders;
Note that for this to work, TReportHeaderModel.TReports should be IEnumerable<TReportModel>.

Normally I would suggest you separate the methods for getting your data and transforming your data into DTOs like this (And usually I have the connection defined at the class level, not at the method level because I will reuse the connection many times, and I prefer keeping my data accesses as lazy as possible):
TReportEntitiesConnection conn = new TReportEntitiesConnection();
Then I will create extension methods like so:
public static class MyExtensions
{
public IQueryable<THeader> ByClientId(this IQuerable<THeader> conn, int ClientID)
{
return conn
.Include(h=>h.Reports)
.Where(h=>h.ClientID==ClientID);
}
public TReportHeaderModel ToDto(this THeader t)
{
return new TReportHeaderModel
{
ID=t.ID,
ClientID=t.ClientID,
THeaderTitle=t.THeaderTitle,
RowNumber=t.RowNumber,
Reports=t.Reports.ToDto()
};
}
public TReportModel ToDto(this TReport r)
{
return new TReportModel
{
ID=r.ID,
TReportName=r.TReportName,
URL=r.URL,
RowNumber=r.RowNumber
};
}
public IEnumerable<TReportHeaderModel> ToDto(this IEnumerable<THeader> h)
{
return h.Select(x=>x.ToDto());
}
public IEnumerable<TReportModel> ToDto(this IEnumerable<TReport> r)
{
return r.Select(x=>x.ToDto());
}
}
Then you can use it like so:
var result=conn.THeaders.ByClientId(200).ToDto();
If you prefer not having your connection at the module level, that is easy too:
using(var connection = new TReportEntitiesConnection())
{
var result=connection.THeaders.ByClientId(200).ToDto();
}
(or use AutoMapper and skip all the manual Dto conversions)

How to convert string array property to string to save in the sql server as comma seperated row mongo db to sql server C#

I have a collection in mongo db "computers",I want to retrieve the collect and save it into sql server using Entity framework. The problem is that the collection have an array object.I want to convert it to string and save. please help, Thank you. SQL server skips the string [] credentials and inserts rest.
Mongo db collection
{
"_id": ObjectId('57852fcsdsdsdsd2662a0ce400'),
"machineName": "AAADESKTOP",
"updated": ISODate('2017-05-17T15:09:39.399Z'),
"__v": 111,
"credentials": [
"####################",
"####################",
"#################",
"#####################"
],
"clientVersion": "2.12.2",
"lastActivity": ISODate('2017-05-17T16:30:50.165Z'),
"secret": "####################"
}
**C# code**
var client = new MongoClient();
var db = client.GetDatabase("Computerdata");
var collection = db.GetCollection<Computers>("computers");
var data = collection.Find(new BsonDocument()).ToListAsync().Result;
foreach (var item in data)
{
EntityFramewrokContext.Computers.Add(item);
EntityFramewrokContext.SaveChanges();
}
public class Computers
{
[BsonId]
[BsonRepresentation(BsonType.ObjectId)]
[Key]
public string Id { get; set; }
public string machineName { get; set; }
[BsonDateTimeOptions(Representation = BsonType.DateTime)]
public DateTime updated { get; set; }
public int version { get; set; }
public string[] credentials { get; set; }
public string clientVersion { get; set; }
[BsonDateTimeOptions(Representation = BsonType.DateTime)]
public DateTime lastActivity { get; set; }
public string secretKey { get; set; }
}

If you have a table called Computer in SQL Server (That you want to write these records to), then you should also create a table called ComputerCredential that has the following columns:
CredentialId int
ComputerId int
Credential nvarchar(200)
Then for each credential in the array you should create a ComputerCredential entity.
If you desperately want to do it the way you asked. Then:
var creds = String.Join(",", computer.credentials);
That will take the array of credentials and make them into a single comma separated string.
When you retrieve the values you can do this:
var creds = commaSepCredentials.Split(',');
To retrieve the array. But don't do this, do the first option and learn about relations and joins in SQL Server. It's the power in relational databases.

Windows azure table access taking too long

I am using windows azure table storage. My problem is that the access of entities for a given user from table is taking long time. The code which i am using to access the table is as follows :
public CloudTableQuery<T> GetEntites(string username)
{
try
{
CloudTableQuery<T> entries =
(from e in ServiceContext.CreateQuery<T>(TableName)
where e.PartitionKey == username
select e).AsTableServiceQuery();
return entries;
}
catch (Exception)
{ return null; }
}
The total entities in the table are currently only 100 or so. For example : The query seems to take as long as 40 seconds to return 25 entities for a given user. Please suggest if there is any scope for improvement in the code for faster performance ?

The real problem here is you are returning queries instead of entities, and you are likely executing a query again for each returned entity (query). Define your entity with something like:
public class UserEntity
{
public string UserName { get; set; }
public string Name { get; set; }
public string Email { get; set; }
}
Then define your table storage entity with something like:
public class StoredUserEntity : TableServiceEntity
{
public StoredUserEntity (string username)
{
this.PartitionKey = username;
this.RowKey = Guid.NewGuid().ToString();
this.Timestamp = DateTime.Now;
}
public string Email { get; set; }
public string Name { get; set; }
}
Then your query should actually return a list of UserEntities:
public List<UserEntity> GetUserData(string username)
{
var q = _tableContext.CreateQuery<StoredUserEntity>(_tableName);
return
q.ToList().Select(x => new UserEntity {UserName = x.PartitionKey, Email = x.Email, Name = x.Name}).Where(x => x.UserName == username).ToList();
}

Try using Fiddler to see what's going on. It could be that you are experiencing a few retries that could slow things down. The API won't tell you that, but with Fiddler you can see exactly the requests being made, any responses (along with errors) and how many of them are being made.