When we delete an item form the cache and then immediate do a search the search does not reflect the recent deletion.
The search happens immediately after the index deletion or update.
How can I fix this? I don't see an aws configuration setting that will ensure the index is updated fully before returning.
I was able to get it fixed by adding the "?refresh=wait_for" on the query string.
like DELETE /invoices/306?refresh=wait_for
This is a link to the ElasticSearch version of this problem here is a link to the OpenSearch docs. Pay attention to the refresh paramater.
and
Related
I'm very new to the Azure Search Service and I'm following along with the guides that I found on the web. In one of these guides they have a method such as this:
static void Main(string[] args)
{
IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
IConfigurationRoot configuration = builder.Build();
SearchServiceClient serviceClient = CreateSearchServiceClient(configuration);
var test = serviceClient.Indexes.GetClient("testindex");
Console.WriteLine("{0}", "Deleting index...\n");
DeleteHotelsIndexIfExists(serviceClient);
Console.WriteLine("{0}", "Creating index...\n");
CreateHotelsIndex(serviceClient);
ISearchIndexClient indexClient = serviceClient.Indexes.GetClient("hotels");
Console.WriteLine("{0}", "Uploading documents...\n");
UploadDocuments(indexClient);
ISearchIndexClient indexClientForQueries = CreateSearchIndexClient(configuration);
RunQueries(indexClientForQueries);
Console.WriteLine("{0}", "Complete. Press any key to end application...\n");
Console.ReadKey();
}
So the idea above is to delete an index if it exists, then create a new index. Then generate and upload documents and then finally run a search. Now this all makes sense to me how it works, but one thing that troubles me is the deleting the index part. Essentially what would appear to me is that they are suggesting to always delete the index and then recreate it. Which makes me concerned.
If this is a live index, then by me deleting it even for a split of a second, wouldn't that mean that services calling search on this index will fail? The other thing that I'm concerned about is that in most cases my data will be 90% the same, I'll have some updates and newer records, maybe few deleted. But if my database has a million records, it just seems foolish to delete it all and then create it all over again.
Is there a better approach. Is there a way for me to just update the documents in the index as opposed to deleting it?
So I guess this is a 2-part question. 1. If I delete the index, will it stop the searching capability while the new one is being built? 2. Is there a better approach than just deleting the index. If there is a way to update, how do you handle scenarios where document structure has changed, for example a new field is added.
That sample is meant to demonstrate how to call Azure Search via the .NET SDK. It should not be taken as an example of best practices.
To answer your specific questions:
Deleting the index will immediately cause all requests targeting that index to fail. You should not delete a live index in production. Also, there is not yet a backup/restore capability for indexes, so even if an index is not live, you shouldn't delete it unless you can restore the data from elsewhere.
If you need to update documents without updating the index schema, you can achieve this with the "merge" action of the Index API. There is a tutorial that covers this here. If you need to add a new field, you can do this without deleting and re-creating the index -- just use one of the Indexes.CreateOrUpdate methods of the .NET SDK. By default this operation does not cause any index downtime. However, if you need to delete, rename, or change the type of an existing field, or enable new behaviors on a field (e.g. -- make it facetable, sortable, or part of a suggester), you'll need to create a new index. For this reason, it is recommended that your application implement an extra layer of indirection over index names so that you can swap and re-build indexes when needed.
So I am moving from a Lucene based engine over to Azure Search....
All is going well except for when my Indexer runs, it removes all the documents from the Index and doesnt repopulate it with anything.
If i delete and recreate the Index it shows again but when the Indexer runs on the Hour (got it runnning hourly) it deletes all 4k + documents that were in there.
Is this a option I am missing when I create the Indexer
Steps that i use to create index
Add new Datasource => Sql Server
Creat Index (removing where necessary fields not to be indexed)
Add Indexer
Initial run is all good, then when the indexer runs on the hour it clears the Document list
So the image below shows after the second index is run (on the hour)
Update : What I have noticed is I have run a query and it returns what is expected. result wise but the indexer still shows no documents
Based on the screenshot, it looks like the indexer is configured with a change detection policy (Integrated change tracking or high water mark), so the second indexer run simply finds no new documents to index. Therefore, you see "0/0 documents succeeded" in the portal.
To emphasize, indexer never deletes documents unless specifically requested with a soft-delete policy.
I'm a newbie with redis.
Currently i'm using redis to stored posts
urn:post:1
urn:post:3
urn:post:5
...
i stored posts that posted by user in list like this
urn:user:1:posts => [1,3,4,5]
urn:user:2:posts => [2,3,5,6,7,8,9]
...
i also stored list of latest posts in list
urn:post:lastest => [9,8,7]
My question is if i remove one post, ex: del urn:post:1
How to remove that post's id in all list (or set) has it
Thank a lot
As you use NoSQL updating of "foreign keys" is up to you. I would use http://redis.io/commands/lrem
But note that you have to remove the data in correct order. I mean that it's necessary to remove those references where some of your queries can find reference to the content. It means that I would remove article id from posts and latest and than remove a post itself.
In my Sitecore content tree there are few thousands of items, and I just want to alter few items programmatically. Instead of rebuilding the entire lucene index which is taking a big time, I want to update index entries for each item I'm altering in real time. I tried
item.Database.Indexes.UpdateItem(item);
but it is obsolete and ask me to use SearchManager.
Can anyone guide me how to update index entries for a given item?
PS: I'm altering items from desktop application, not the website.
Try to execute one of the HistoryEngine.RegisterItem... methods, e.g:
item.Database.Engines.HistoryEngine.RegisterItemSaved(item, new ItemChanges(item));
item.Database.Engines.HistoryEngine.RegisterItemCreated(item);
item.Database.Engines.HistoryEngine.RegisterItemMoved(item, oldParentId);
Well actually there is no Update operation on indexes, so feel free to do delete/add
in SQL Server 2008, when setting up a full text search, it gives me an option to choose a time/day for it to re-populate the index, can I do this from code instead like C# code? if so how would I do it? Would I need to execute SQL code from C# to refresh the index?
Also, say if I DONT re-populate the index and add some new records, by using the FTS query will the new records still come up? But slower? Or will they not come up at all unless i reindex?
Edit: beware that there are two ways of keeping an FT index current:
to "Update" it which requires keeping track of changes to individual records (this is done automatically for you, given the proper "Change Tracking" setting, but this does incur a slight overhead).
to "re-popluate" it i.e. to create the complete index anew.
The former method is +/- transparent to continued service for the users of the underlying catalog, the latter implies a partial loss of service while the index is being repopulated (although maybe not in 2008?)
It is a bit unclear what type of update or re-population the question refers to, so the responses below are generic.
1) Yes! Full-text index re-population (or update) can be done programmatically using a plain TSQL query which looks like the following. Such queries can well be invoked from C# code, using OLEDB or even ODBC.
ALTER FULLTEXT INDEX ON myCatalog START FULL POPULATION; -- for full (re-)population
ALTER FULLTEXT INDEX ON myCatalog START UPDATE POPULATION; -- for update
2) Yes, the FTS index will reflect changes to the database, in close to real-time, provided that it is setup accordingly.
You need to set the FT index for "Change tracking" so that it would be able to maintain the index current. This is the necessary condition. With the list of changed and new records in had (from the change tracking), SQL may either update the FT index "on the fly" (keeping this index slightly behind realtime re. the updates to the database; the delay is because the updates to FT index are a bit buffered, and also because this type of update is typically low priority with regards to CPU usage on the server), or this can be done "manually" (the manually is in quotes because "explicitly" may be more appropriate, for such manual updates may well be scheduled and hence happen "automatically".)
See this Microsoft technet article for more info. (Also check the links at the bottom of this article for an overview of FT indexes etc.)