While creating index, get "Query_shard_exception" in elasticsearch - c#

I am trying to send data to Elasticsearch from .Net by using NEST library.
My ES version is 6.7 and NEST version 6.7 too.
So while creating index I get exception type query_shard_exception No mapping found for [#timestamp] in order to sort on
I created single node and tried send my message.
var node = new Uri("http://localhost:9200/");
var mysettings = new ConnectionSettings(node);
var client = new ElasticClient(mysettings);
var tweet = new {
Id = 1,
FirstName = "John",
LastName = "Grank",
PostDate = new DateTime(2019,7,12),
Message = "Trying out NEST, so far so good?"
}
var response = client.Index(tweet,idx=> idx.Index(mytweetindex));
Do you have any idea what I did wrong? Thanks.

Related

How to find a document with Bson type in mongo in c#

I want to find a role details with specified username in MongoDb with Drivers in C#.
I don't want to use any builders or linq methods.
I tried this to insert a Bson document and it worked.
var client = new MongoClient("mongodb://localhost:27017");
var database = client.GetDatabase("test");
var collec = database.GetCollection<BsonDocument>("user");
var documnt = new BsonDocument{
{"username", txtUsername.Text},
{"password", txtPassword.Password}
};
var check_count = collec.CountDocuments(documnt);
But when I tried this code to find a role with username its not working:
var client = new MongoClient("mongodb://localhost:27017");
var database = client.GetDatabase("test");
var collec = database.GetCollection<BsonDocument>("user");
var documnt = new BsonDocument{
{"username", "admin"},
{"role", 1}
};
var role = collec.Find(documnt);
txtTestRole.Text = role.ToString();
I got this as output:
enter image description here
You have to materialize the query.
As you query for single record, you can use IAsyncCursorSourceExtensions.SingleOrDefault Method.
Or you can refer to IFindFluent (Extension methods section) to select the method that is best suited to you.
Next, from returned BsonDocument to select the specific field for the displayed value.
var user = collec.Find(documnt)
.SingleOrDefault();
if (user == null)
{
// Handle user not exist
return;
}
txtTestRole.Text = user["role"].ToString();

Bulk collection inside the right Index path in ElasticSearch using NEST in a .NET Core application

I am trying to bulk a collection of elements inside an index of ElasticSearch using NEST inside a .NET Core application.
Currently what I have is working, and the elements are saved, but Is not saved where I try to do
My client creation:
protected ElasticClient GetClient()
{
var node = new Uri("http://localhost:9200/");
var settings = new ConnectionSettings(node)
.DefaultIndex("TestIndex")
.PrettyJson(true);
return new ElasticClient(settings);
}
Here is how I create the descriptor for bulk all the data
protected BulkDescriptor GenerateBulkDescriptor<T>(IEnumerable<T> elements, string indexName) where T: class, IIndexable
{
var bulkIndexer = new BulkDescriptor();
foreach (var element in elements)
bulkIndexer.Index<T>(i => i
.Document(element)
.Id(element.Id)
.Index(indexName));
return bulkIndexer;
}
Finally, once I have this, here is how I index the data
var descriptor = GenerateBulkDescriptor(indexedElements, "indexed_elements");
var response = GetClient().Bulk(descriptor);
But, If I see how It's stored in the Elastic index using this, that is what I have:
How can I know if is created under TestIndex index? Because as far as I can see, there is just one index created
Thank you a lot in advance
When defining the index operations on the BulkDescriptor, you are explicitly setting the index to use for each operation
foreach (var element in elements)
bulkIndexer.Index<T>(i => i
.Document(element)
.Id(element.Id)
.Index(indexName));
where indexName is "indexed_elements". This is why all documents are indexed into this index and you do not see any in "TestIndex".
The Bulk API allows multiple operations to be defined, which may include indexing documents into different indices. When the index is specified directly on an operation, that will be the index used. If all index operations on a Bulk API call are to take place against the same index, you can omit the index on each operation and instead, specify the index to use on the Bulk API call directly
var defaultIndex = "default_index";
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var settings = new ConnectionSettings(pool)
.DefaultIndex(defaultIndex);
var client = new ElasticClient(settings);
var people = new []
{
new Person { Id = 1, Name = "Paul" },
new Person { Id = 2, Name = "John" },
new Person { Id = 3, Name = "George" },
new Person { Id = 4, Name = "Ringo" },
};
var bulkResponse = client.Bulk(b => b
.Index("people")
.IndexMany(people)
);
which sends the following request
POST http://localhost:9200/people/_bulk
{"index":{"_id":"1","_type":"person"}}
{"id":1,"name":"Paul"}
{"index":{"_id":"2","_type":"person"}}
{"id":2,"name":"John"}
{"index":{"_id":"3","_type":"person"}}
{"id":3,"name":"George"}
{"index":{"_id":"4","_type":"person"}}
{"id":4,"name":"Ringo"}
Note that the URI is /people/bulk and that each JSON object representing an operation does not contain an "_index".
If you omit the .Index() on Bulk API call, it will use the DefaultIndex configured on ConnectionSettings:
var bulkResponse = client.Bulk(b => b
.IndexMany(people)
);
which yields
POST http://localhost:9200/_bulk
{"index":{"_id":"1","_index":"default_index","_type":"person"}}
{"id":1,"name":"Paul"}
{"index":{"_id":"2","_index":"default_index","_type":"person"}}
{"id":2,"name":"John"}
{"index":{"_id":"3","_index":"default_index","_type":"person"}}
{"id":3,"name":"George"}
{"index":{"_id":"4","_index":"default_index","_type":"person"}}
{"id":4,"name":"Ringo"}
You can also specify a default index to use for a given POCO type on ConnectionSettings with DefaultMappingFor<T>(), where T is your POCO type.
After som tests and attemps, I have found a solution.
First of all, it was a problem with the index configured, once I set it in lower case, the index was working fine indexing data inside.
Then, I had the problem of index data in a specific "path" inside the same index, finalyy I found the Type solution from NEST, taking also advantage of the DefaultMappingFor suggested by Russ in the previous answer.
Client definition:
var node = new Uri(_elasticSearchConfiguration.Node);
var settings = new ConnectionSettings(node)
.DefaultMappingFor<IndexedElement>(m => m
.IndexName(_elasticSearchConfiguration.Index)
.TypeName(nameof(IndexedElement).ToLower()))
.PrettyJson(true)
.DisableDirectStreaming();
var client = new ElasticClient(settings);
Then, the BulkDescriptior creation:
var bulkIndexer = new BulkDescriptor();
foreach (var element in elements)
bulkIndexer.Index<IndexedElement>(i => i
.Document(element)
.Type(nameof(IndexedElement).ToLower()))
.Id(element.Id)
);
And finally, data bulk:
client.Bulk(bulkIndexer);
Now, If I perform a call to the index, I can see this
{
"testindex": {
"aliases": {},
"mappings": {
"indexedelement": {
[...]
}
Thank you Russ for your help and for who have had a look to the post.
UPDATE
Finally, it seems that the unique problem was regarding the default index, that it must be lowercase, so, specify the type with the name of the POCO itself is not neccesary, like #RussCam has truly detected in comments above. After changing thedefault index to lowercase, all the different possibilities worked fine.
Thank you all again

RetrieveEntityRequest for multiple entities at once?

I'm using the RetrieveEntityRequest to get an entity's attributes' metadata:
RetrieveEntityRequest entityRequest = new RetrieveEntityRequest
{
EntityFilters = EntityFilters.Attributes,
LogicalName = joinedEntityName.Value,
};
RetrieveEntityResponse joinedEntityMetadata = (RetrieveEntityResponse)_service.Execute(entityRequest);
Now, consider I need to execute this request for multiple entities. Is it possible to do this in one execution (maybe not with RetrieveEntityRequest), instead of one request for each entity?
You can't do it with RetrieveEntityRequest. However, you can do a RetrieveMetadataChangesRequest to get what you want. It's misleadingly named for your purposes, but if you don't provide a ClientVersionStamp property, it will simply retrieve everything you've specified in the Query property.
Here's a simple example where you'd retrieve the metadata for account and contact, and only retrieve the LogicalName and DisplayName properties:
var customFilterExpression = new[]
{
new MetadataConditionExpression("LogicalName", MetadataConditionOperator.Equals, "account"),
new MetadataConditionExpression("LogicalName", MetadataConditionOperator.Equals, "contact")
};
var customFilter = new MetadataFilterExpression(LogicalOperator.Or);
customFilter.Conditions.AddRange(customFilterExpression);
var entityProperties = new MetadataPropertiesExpression
{
AllProperties = false
};
entityProperties.PropertyNames.AddRange("LogicalName", "DisplayName");
var request = new RetrieveMetadataChangesRequest
{
Query = new EntityQueryExpression
{
Properties = entityProperties,
Criteria = customFilter,
}
};
This method also has the benefit of only retrieving what specific properties you need, which makes the request faster and the payload smaller. It's specifically designed for mobile where you want to only retrieve the Metadata you need, and what has changed since the last time you retrieved it, but it works nicely in a lot of scenarios.
You have to use RetrieveAllEntitiesRequest. Sample below:
RetrieveAllEntitiesRequest retrieveAllEntityRequest = new RetrieveAllEntitiesRequest
{
RetrieveAsIfPublished = true,
EntityFilters = EntityFilters.Attributes
};
RetrieveAllEntitiesResponse retrieveAllEntityResponse = (RetrieveAllEntitiesResponse)serviceProxy.Execute(retrieveAllEntityRequest);
CRM SDK has all or one-by-one approach only.
You have to keep your list of entities ready & issue the RetrieveEntityRequest for each item.

How to search for "Lot Numbered Assembly/Bill of Materials" items by external ID using SuiteTalk

WSDL 2014.2. C# in Visual Studio 2013.
I can get "Inventory Item" items using this code:
var recordRefs = new List<RecordRef>();
foreach (string externalId in ExternalIds)
{
recordRefs.Add(new RecordRef
{
externalId = externalId,
type = RecordType.inventoryItem,
typeSpecified = true
});
}
var request = new ItemSearchBasic
{
externalId = new SearchMultiSelectField
{
#operator = SearchMultiSelectFieldOperator.anyOf,
operatorSpecified = true,
searchValue = recordRefs.ToArray()
}
};
SearchResult response = SuiteTalkConnection.Service.search(request);
However, if I change type = RecordType.inventoryItem to type = RecordType.lotNumberedAssemblyItem then I don't get results. I am specifying correct values as external IDs.
I have tried all sorts of variations on the above, including ItemSearch instead of ItemSearchBasic but I never get results.
It doesn't help that I can find zero NetSuite documentation on searching for specific item types.
I think answer to this question would also work in this case. Try using type as "assemblyItem" only and use an additional searchFilter "isLotItem" and set it to true.

NEST not indexing

Forgive me if this is redundant or I'm missing something simple, but I'm playing around with ElasticSearch (and NEST in particular) to see if it would be a good add for our b2b ecommerce site's search function.
I grabbed the latest NuGet of NEST and then tried to serialize and add something to an index. Here is a snippet of the approach I was using
var localhost = new Uri("http://localhost/9200");
var setting = new ConnectionSettings(localhost).SetDefaultIndex("cpi_catalog");
var client = new ElasticClient(setting);
client.MapFromAttributes<Item>();
var testitem = new Item()
{
Description = "test",
Id = 9999999,
Manufacturer_Id = 5,
Quantity_Per_Unit = 1,
Quantity_Unit_Id = "EA",
SKU = "AVE29845",
Subtitle = "test",
Title = "test"
};
var status = client.Index(testitem);
However, it seems that testitem is never indexed at all, when I do a GET for /cpi_catalog/items/9999999 I get the following:
{"_index":"cpi_catalog","_type":"items","_id":"9999999","exists":false}
What seemingly simple thing am I missing here?
EDIT: When debugging, I get back a Nest.IndexResponse with all fields NULL besides status.OK which is false
Seems like the uri is has a typo:
var localhost = new Uri("http://localhost/9200");
should probably be:
var localhost = new Uri("http://localhost:9200");

Categories