ElasticSearch NEST Delete all document - c#

I am using ElastciSearch 2.3.0
I am trying to delete documents from the ElasticSearch using .net and NEST for specific index.
I only want to delete all documents and not the _mapping
DeleteByQueryRequest r = new DeleteByQueryRequest(new IndexName() { Name = indexName });
r.QueryOnQueryString = "*";
var response = client.DeleteByQuery(r);
I am try to do this by using above code but it is not working.
Please suggest on what's wrong with the above code or how this can be done.
Thanks for your help in advance.

Don't use delete by query it was made a plugin since elastic 2.0 for a good reason. You can get out of memory exceptions easy.
You should delete the whole index and recreate the mappings
static void Main(string[] args)
{
ElasticClient db = new ElasticClient(new Uri("http://localhost.fiddler:9200"));
db.IndexMany(Enumerable.Range(0, 100).Select(i => new Data { Id = i, Name = "Name" + i }), "test_index");
var mappings = db.GetMapping<Data>();
var delete = db.DeleteIndex(new DeleteIndexRequest("test_index"));
var indexMapping = mappings.IndexTypeMappings["test_index"].ToDictionary(k => k.Key, v => (ITypeMapping)v.Value);
db.CreateIndex(new CreateIndexRequest("test_index")
{
Mappings = new Mappings(indexMapping)
});
Console.ReadLine();
}
class Data
{
public int Id { get; set; }
public string Name { get; set; }
}
Raw copy of index
var res = db.LowLevel.IndicesGetMapping<JObject>("test_index");
var delete = db.DeleteIndex(new DeleteIndexRequest("test_index"));
var mappings = res.Body["test_index"].ToString();
var create = db.LowLevel.IndicesCreate<JObject>("test_index", mappings);
If you really need to install the plug-in
sudo bin/plugin install delete-by-query

It worked.
Thanks a lot.
var res = db.LowLevel.IndicesGetMapping<JObject>("test_index");
var delete = db.DeleteIndex(new DeleteIndexRequest("test_index"));
var mappings = res.Body["test_index"].ToString();
var create = db.LowLevel.IndicesCreate<JObject>("test_index", mappings);

Related

Trouble Fetching Value in Variable

Basically here's my code which I'm having trouble with. Insanely new to mongoDB and would love to understand how to get values out of a JSON string that is returns in the variable 'line'.
public string get_data()
{
var client = new MongoClient();
var db = client.GetDatabase("test");
var collection = db.GetCollection<BsonDocument>("metacorp");
var cursor = collection.Find("{'movie_name' : 'Hemin'}").ToCursor();
var line = "";
foreach (var document in cursor.ToEnumerable())
{
using (var stringWriter = new StringWriter())
using (var jsonWriter = new JsonWriter(stringWriter))
{
var context = BsonSerializationContext.CreateRoot(jsonWriter);
collection.DocumentSerializer.Serialize(context, document);
line = stringWriter.ToString();
}
}
var js = new JavaScriptSerializer();
var d = js.Deserialize<dynamic>(line);
var a = d["movie_name"];
return line;
}
This is the output I get if I return line:
{ "_id" : ObjectId("58746dcafead398e4d7233f5"), "movie_name" : "Hemin"
}
I want to be able to fetch the value 'Hemin' into 'a'.
I know this is not what you're asking for but since you're using the c# driver then I would recommend the following. Assumes you have a c# class corresponding to metacorp collection or at least a serializer that handles it. Hope it helps.
var client = new MongoClient();
var db = client.GetDatabase("test");
var collection = db.GetCollection<MetaCorp>("metacorp");
var m = collection.SingleOrDefault(x => x.Movie_Name == "Hemin"); // Assuming 0 or 1 with that name. Use Where otherwise
var movieName = "Not found";
if(m!= null)
movieName = m.Movie_Name;
You could have your dto class for movie ans just fetch the data from collection:
public class Movie
{
public ObjectId Id { get; set; }
public string movie_name { get; set;}
}
...
var client = new MongoClient();
var db = client.GetDatabase("test");
var collection = db.GetCollection<BsonDocument>("metacorp");
var movies = collection.Find(x=>x.movie_name == "Hemin").ToEnumerable();

MongoDb Driver 2.0 C# Filter and Aggregate

I'm playing around with the new driver of mongodb 2.0, and looking for adding some facetted searchs (Temporary move ,before using elastic search).
Here is some method where I created to build the agreggation. I guess that it should work.
As parameter I passed also a filterdefinition in the method.
But I don't find how to limit my agreggation to the filter.
Any Idea ???
private void UpdateFacets(SearchResponse response, FilterDefinition<MediaItem> filter, ObjectId dataTableId)
{
response.FacetGroups =new List<SearchFacetGroup>();
SearchFacetGroup group = new SearchFacetGroup()
{
Code = "CAMERAMODEL",
Display = "Camera model",
IsOptional = false
};
using (IDataAccessor da = NodeManager.Instance.GetDataAccessor(dataTableId))
{
var collection = da.GetCollection<MediaItem>();
var list = collection.Aggregate()
.Group(x => ((ImageMetaData) x.MetaData).Exif.CameraModel, g => new { Model = g.Key, Count = g.Count() })
.ToListAsync().Result;
foreach (var l in list)
{
group.Facets.Add(new SearchFacetContainer()
{
Code = l.Model,
Display = l.Model,
Hits = l.Count,
IsSelected = false
});
}
}
response.FacetGroups.Add(group);
}
I haven't used facet, but with Mongo driver Aggregate has .Match operation that accepts a filterdefinition.
collection1.Aggregate().Match(filter)

Generate lambda expression from string

I have a need to store lambda expressions in a database and execute them dynamically. The following snippet works well as is,
float result = dataRowArray.AsEnumerable().Sum(r => r.Field<float>("ColumnA") * r.Field<float>("ColumnB"));
but I'd like to store the following part in a string...
r => r.Field<float>("ColumnA") * r.Field<float>("ColumnB")
... and inject it dynamically as in:
float result = MySession.Current.dr.AsEnumerable().Sum(storedLambdaString);
I've done a ton of Googling but I'm coming up short. Any suggestions?
I suggest store the query in the database and execute using LINQ like below.
using (eEntities db = new eEntities())
{
string query = "select name,age,salary from employees"
var employees = db.Database.SqlQuery<Employee>(query).
}
Please note: Employee is here my custom class having properties same as queried.
Theres a nuget package that may help you "System.Linq.Dynamic". This sample code works...
public class TestRow
{
public Dictionary<string,float> Field { get; set; }
public TestRow()
{
this.Field = new Dictionary<string, float>();
this.Field.Add("ColumnA", 2);
this.Field.Add("ColumnB", 3);
}
}
class Program
{
static void Main(string[] args)
{
var testRow = new TestRow();
string expressionString = "r.Field[\"ColumnA\"] * r.Field[\"ColumnB\"]";
var parameter = Expression.Parameter(typeof(TestRow),"r");
var lambdaET = System.Linq.Dynamic.DynamicExpression.ParseLambda(new[] { parameter }, typeof(float), expressionString);
var result = lambdaET.Compile().DynamicInvoke(testRow);
Console.WriteLine(result);
Console.ReadLine();
}
}

Elastic search with Nest

I am working on below code, and what I want to do is query by object itself.
For example: I have a search form, that populates objects fields such as below. Then what I want to do is to search Elastic search based on whatever user filled the form with.
ie: below, I want to query the index by searchItem object. How can I do it easily?
class Program
{
static void Main(string[] args)
{
var p = new Program();
var item1 = new Announcement() {Id=1, Title = "john", ContentText = "lorem", Bar = false, Num = 99, Foo = "hellow"};
//p.Index(item1, "add");
var searchItem = new Announcement() {Title="john",Num=99};
ElasticClient.Search<Announcement>();
Console.Read();
}
public void Index(Announcement announcement, String operation)
{
var uriString = "http://localhost:9200";
var searchBoxUri = new Uri(uriString);
var settings = new ConnectionSettings(searchBoxUri);
settings.SetDefaultIndex("test");
var client = new ElasticClient(settings);
if (operation.Equals("delete"))
{
client.DeleteById("test", "announcement", announcement.Id);
}
else
{
client.Index(announcement, "test", "announcement", announcement.Id);
}
}
private static ElasticClient ElasticClient
{
get
{
try
{
var uriString = "http://localhost:9200";
var searchBoxUri = new Uri(uriString);
var settings = new ConnectionSettings(searchBoxUri);
settings.SetDefaultIndex("test");
return new ElasticClient(settings);
}
catch (Exception)
{
throw;
}
}
}
}
You can't :)
NEST cannot infer how to best query only based on a partially filled POCO. Should it OR or AND should it do a nested term query or a term query wrapped in a has_child? You catch my drift.
Nest does have a slick feature called conditionless queries that allow you the write out to entire query like so:
ElasticClient.Search<Announcement>(s=>s
.Query(q=>
q.Term(p=>p.Title, searchItem.Title)
&& q.Term(p=>p.Num, searchItem.Num)
//Many more queries use () to group all you want
)
)
When NEST sees that the argument passed to Term is null or empty it simply wont render that part of the query.
Read more here on how this feature works http://nest.azurewebsites.net/concepts/writing-queries.html

How can I write a LINQ to SQL query to update tags?

I have an image site where users can tag photos much like you can tag a question on Stackoverflow.
I have the following tables:
Images [ID, URL, etc]
Tags [ID, TagName]
ImageTag [TagID, ImageID]
I want to write a method with the signature:
public void UpdateImageTags(int imageId, IEnumerable<string> currentTags)
This method will do the following:
Create any new Tags in currentTags that don't already exist in the Tags table.
Get the old ImageTag's for an image.
Delete any ImageTag's that no longer exist in the currentTags
Add any ImageTag's that are new between the currentTags and oldTags.
Here is my attempt at that method:
public void UpdateImageTags(int imageId, IEnumerable<string> currentTags)
{
using (var db = new ImagesDataContext())
{
var oldTags = db.ImageTags.Where(it => it.ImageId == imageId).Select(it => it.Tag.TagName);
var added = currentTags.Except(oldTags);
var removed = oldTags.Except(currentTags);
// Add any new tags that need created
foreach (var tag in added)
{
if (!db.Tags.Any(t => t.TagName == tag))
{
db.Tags.InsertOnSubmit(new Tag { TagName = tag });
}
}
db.SubmitChanges();
// Delete any ImageTags that need deleted.
var deletedImageTags = db.ImageTags.Where(it => removed.Contains(it.Tag.TagName));
db.ImageTags.DeleteAllOnSubmit(deletedImageTags);
// Add any ImageTags that need added.
var addedImageTags = db.Tags.Where(t => added.Contains(t.TagName)).Select(t => new ImageTag { ImageId = imageId, TagId = t.TagId });
db.ImageTags.InsertAllOnSubmit(addedImageTags);
db.SubmitChanges();
}
}
However, this fails on the line:
db.ImageTags.DeleteAllOnSubmit(deletedImageTags);
With the error:
Local sequence cannot be used in LINQ to SQL implementations of query
operators except the Contains operator.
Is there an easier way I can handle the operation of adding new tags, deleting old ImageTags, adding new ImageTags in LINQ to SQL?
Seems like this would be easiest
public void UpdateImageTags(int imageId, IEnumerable<string> currentTags)
{
using (var db = new ImagesDataContext())
{
var image = db.Images.Where(it => it.ImageId == imageId).First()
image.Tags.Clear();
foreach(string s in currentTags)
{
image.Tags.Add(new Tag() { TagName = s});
}
db.SubmitChanges();
}
}
This might have to be modified slightly for LinqtoSQL. EF is what i have been using most recently. Also this is dependent on Lazy loading being enabled. If it is not, you will have to force the include of the image tags.
Here is a helper method to deal with many-to-many relationships:
public static void UpdateReferences<FK, FKV>(
this EntitySet<FK> refs,
Expression<Func<FK, FKV>> fkexpr,
IEnumerable<FKV> values)
where FK : class
where FKV : class
{
Func<FK, FKV> fkvalue = fkexpr.Compile();
var fkmaker = MakeMaker(fkexpr);
var fkdelete = MakeDeleter(fkexpr);
var fks = refs.Select(fkvalue).ToList();
var added = values.Except(fks);
var removed = fks.Except(values);
foreach (var add in added)
{
refs.Add(fkmaker(add));
}
foreach (var r in removed)
{
var res = refs.Single(x => fkvalue(x) == r);
refs.Remove(res);
fkdelete(res);
}
}
static Func<FKV, FK> MakeMaker<FKV, FK>(Expression<Func<FK, FKV>> fkexpr)
{
var me = fkexpr.Body as MemberExpression;
var par = Expression.Parameter(typeof(FKV), "fkv");
var maker = Expression.Lambda(
Expression.MemberInit(Expression.New(typeof(FK)),
Expression.Bind(me.Member, par)), par);
var cmaker = maker.Compile() as Func<FKV, FK>;
return cmaker;
}
static Action<FK> MakeDeleter<FK, FKV>(Expression<Func<FK, FKV>> fkexpr)
{
var me = fkexpr.Body as MemberExpression;
var pi = me.Member as PropertyInfo;
var assoc = Attribute.GetCustomAttribute(pi, typeof(AssociationAttribute))
as AssociationAttribute;
if (assoc == null || !assoc.DeleteOnNull)
{
throw new ArgumentException("DeleteOnNull must be set to true");
}
var par = Expression.Parameter(typeof(FK), "fk");
var maker = Expression.Lambda(
Expression.Call(par, pi.GetSetMethod(),
Expression.Convert(Expression.Constant(null), typeof(FKV))), par);
var cmaker = maker.Compile() as Action<FK>;
return cmaker;
}
Usage:
IEnumerable<Tag> values = ...;
Image e = ...;
e.ImageTags.UpdateReferences(x => x.Tag, tags);

Categories