I have a search bar that executes a lucene query on the "description" field, but it doesn't return results when with apostrophes. For example, I have a product where the description is Herter's® EZ-Load 200lb Feeder - 99018. When I search for "Herter", I get results, but I get no results if I search for "Herter's" or "Herters". This is my search code:
var query = Request.QueryString["q"];
var search = HttpContext.Current.Server.UrlDecode(query);
var rewardProductLookup = new RewardCatalogDataHelper();
RewardProductSearchCriteria criteria = new RewardProductSearchCriteria()
{
keywords = search,
pageSize = 1000,
sortDirection = "desc"
};
IEnumerable<SkinnyItem> foundProducts = rewardProductLookup.FindByKeywordQuery(criteria);
public IEnumerable<SkinnyItem> FindByKeywordQuery(RewardProductSearchCriteria query)
{
var luceneIndexDataContext = new LuceneDataContext("rewardproducts", _dbName);
string fieldToQuery = "rpdescription";
bool sortDirection = query.sortDirection.ToLower().Equals("desc");
MultiPhraseQuery multiPhraseQuery = new MultiPhraseQuery();
var keywords = query.keywords.ToLower().Split(',');
foreach (var keyword in keywords)
{
if (!String.IsNullOrEmpty(keyword))
{
var term = new Term(fieldToQuery, keyword);
multiPhraseQuery.Add(term);
}
}
var booleanQuery = new BooleanQuery();
booleanQuery.Add(multiPhraseQuery, BooleanClause.Occur.MUST);
return
luceneIndexDataContext.BooleanQuerySearch(booleanQuery, fieldToQuery, sortDirection)
.Where(i => i.Fields["eligibleforpurchase"] == "1");
}
The problem here is analysis. You haven't specified the analyzer being used in this case, so I'll assume it's StandardAnalyzer.
When analyzed, the term "Herter's" will be translated to "herter". However, no analyzer is being applied in your FindByKeywordQuery method, so looking for "herter" works, but "herter's" doesn't.
One solution would be to use the QueryParser, in stead of manually constructing a MultiPhraseQuery. The QueryParser will handle tokenizing, lowercasing, and such. Something like:
QueryParser parser = new QueryParser(VERSION, "text", new StandardAnalyzer(VERSION));
Query query = parser.Parse("\"" + query.keywords + "\"");
The single quote is the delimiter for text fields in a query.
Select * FROM Product where Description = 'foo'
You will need to escape or double any single quote your query. try this in the loop.
foreach (var keyword in keywords)
{
if (!String.IsNullOrEmpty(keyword))
{
var term = new Term(fieldToQuery, keyword);
term = term.Replace("'", "''");
multiPhraseQuery.Add(term);
}
}
You could also create an extension method
[DebuggerStepThrough]
public static string SanitizeSQL(this string value)
{
return value.Replace("'", "''").Replace("\\", "\\\\");
}
in which case you could then you could do this in the loop
foreach (var keyword in keywords)
{
if (!String.IsNullOrEmpty(keyword))
{
var term = new Term(fieldToQuery, keyword.SanitizeSQL());
multiPhraseQuery.Add(term);
}
}
Hope this helps.
Related
This is a function to work with lists in string interpolation. It takes a List and an inner Func, and it appends the string result of the inner Func called for each member of the list, with a separator.
So the following builds a valid start of an Insert statement...
static void Main(string[] args)
{
var tableName = "customers";
var cols = new List<dynamic>
{
new { Name = "surname"},
new { Name = "firstname"},
new { Name = "dateOfBirth"}
};
Func<List<dynamic>, Func<dynamic, string>, string, string> ForEach = (list, func, separator) =>
{
var bldr = new StringBuilder();
var first = true;
foreach (var obj in list)
{
if (!first)
bldr.Append(separator);
first = false;
bldr.Append(func(obj));
}
return bldr.ToString();
};
var InsertStatement = $"Insert into { tableName } ( {ForEach(cols, col => col.Name, ", ")} )";
Console.WriteLine(InsertStatement);
Console.ReadLine();
}
Outputs...
Insert into customers ( surname, firstname, dateOfBirth )
It works for dynamic. How do I make it work for any type? The outer Func shouldn't care about the Type in the list, it just passes it through to the inner Func.
The .NET framework already gives you a generic function to achieve what you are trying to do String.Join and you can combine it with a LINQ Select statement, which will allow you to use a lambda on a generic type to select the property that you want to print. You can view the source code of these methods if you are interested as they are open source.
using System;
using System.Collections.Generic;
using System.Linq;
public class MyType
{
public string Name { get; set; }
}
public class Program
{
public static void Main()
{
var tableName = "customers";
var cols = new List<MyType>
{
new MyType { Name = "surname"},
new MyType { Name = "firstname"},
new MyType { Name = "dateOfBirth"}
};
var InsertStatement = $"Insert into { tableName } ( {String.Join(", ", cols.Select(col => col.Name))} )";
Console.WriteLine(InsertStatement);
}
}
Replace dynamic with object, or TValue with a type constraint stipulating it must be a class (where TValue : class), and call obj.ToString() instead of just obj
However, this doesn't guarantee it would "work with any type" - for that you need to know that those types all follow a contract to output the desired column name as their string representation. To get more specificity, require that your accepted types must implement some interface eg IColumnName and put that interface into the type constraint instead
You can create the text easily like this:
var query = $"INSERT INTO {tableName}({string.Join(",", cols.Select(x=>x.Name))})";
However, if for learning purpose you are going to handle the case using a generic method, you can create a generic function like the following and then easily use a for loop and strip additional separator using TrimEnd, or as a better option, like String.Join implementation of .NET Framework get enumerator like this:
string Join<TItem>(
IEnumerable<TItem> items, Func<TItem, string> itemTextSelecor, string separator)
{
var en = items.GetEnumerator();
if (!en.MoveNext())
return String.Empty;
var builder = new StringBuilder();
if (en.Current != null)
builder.Append(itemTextSelecor(en.Current));
while (en.MoveNext())
{
builder.Append(separator);
if (en.Current != null)
builder.Append(itemTextSelecor(en.Current));
}
return builder.ToString();
}
And use it this way:
var tableName = "customers";
var cols = new[]
{
new { Name = "surname"},
new { Name = "firstname"},
new { Name = "dateOfBirth"}
};
var InsertStatement = $"INSERT INTO {tableName} ({Join(cols, col => col.Name, ", ")})"
+ $"VALUES({Join(cols, col => $"#{col.Name}", ", ")})";
var kusto = string.Format("let MyData = CompanyMydata" +
" | where ID == 'Z123' | top 1 by dateTimeUtc desc");
var reader = client.ExecuteQuery(kusto);
while (reader.Read())
{
//how can i return coming result as list of string or json string?
}
In while loop I can able to return single column value one by one by using
string state = reader.GetString(1); but i want to return complete result instead of one by one column value.
so that i can do JsonConvert.DeserializeObject<T>(resultString); to specific class.
assuming you're using the client libraries mentioned here, you should be able to do something like the following:
static void Main(string[] args)
{
var kcsb = new KustoConnectionStringBuilder("https://help.kusto.windows.net").WithAadUserPromptAuthentication();
var databaseName = "Samples";
using (var queryProvider = KustoClientFactory.CreateCslQueryProvider(kcsb))
{
var clientRequestProperties = new ClientRequestProperties() { ClientRequestId = "Sample;" + Guid.NewGuid() };
var query = "StormEvents | summarize count(), max(EndTime) by State";
var result = queryProvider.ExecuteQuery<MyType>( // focus on this part
databaseName,
query,
clientRequestProperties)
.ToList();
foreach (var row in result)
{
Console.WriteLine($"State = {row.State}, Count = {row.Count}, MaxEndTime = {row.MaxEndTime}");
}
}
}
class MyType
{
public string State;
public long Count;
public DateTime MaxEndTime;
}
I am trying to create a replace function, which can replace multiple strings within the 'body' fieldname of my linq queries.
This is my linq function:
public string GetReplacedText(string body)
{
var data = from c in db.StoryTbls
where c.Body == body
select new
{
c.Body
};
Regex rgx2 = new Regex("<P align=right><A href>RS</A></P>");
Regex rgx3 = new Regex("<A href>ML</A>");
string res = rgx2.Replace(body, "");
res = rgx3.Replace(body,"");
return res;
}
I am calling the above function, into the below 'httpresponse' method.
public HttpResponseMessage getData()
{
if (User.IsInRole("admin"))
{
Regex rgx2 = new Regex("<A href></A>");
var join = (from s in db.StoryTbls
join c in db.META_Cat on s.Theme equals c.metaIndex
where s.ACTIVE == true
&& s.PUB_ID == 250
&& c.Categories == "RM"
orderby s.ACTIVEDATE descending
select new
{
s.TITLE,
s.Body,
s.ACTIVEDATE,
c.Categories
}).AsEnumerable().Select(c => new NewObj
{
Title = c.TITLE,
Body = GetReplacedText(c.Body),
ActiveDate = c.ACTIVEDATE,
Categories = c.Categories
}).Take(50).ToList();
var data = join.ToList();
if (!data.Any())
{
var message = string.Format("No data found");
return Request.CreateErrorResponse(HttpStatusCode.NotFound, message);
}
However, when I call the API, the body content data is not replaced. Hence the function not working.
Please advise, as to where I may be going wrong.
Many thanks
You need to take the output from your first call to Replace and pass it in to your second call to Replace, like so:
string res = rgx2.Replace(body, "");
res = rgx3.Replace(res, "");
I'm querying a SharePoint 2013 Term Store via the SharePoint Client Object Model in order to get a TermCollection.
I'd like to bind the results to a WPF TreeView control. Any idea how I can turn the TermCollection into something that the TreeView will understand?
public static TermCollection GetTaxonomyTerms(string webUrl, string libraryTitle, string fieldTitle)
{
var context = new ClientContext(webUrl);
var web = context.Web;
var list = web.Lists.GetByTitle(libraryTitle);
var fields = list.Fields;
var field = context.CastTo<TaxonomyField>(fields.GetByInternalNameOrTitle(fieldTitle));
context.Load(field);
var termStores = TaxonomySession.GetTaxonomySession(context).TermStores;
context.Load(termStores);
context.ExecuteQuery(); // TODO: Can this ExecuteQuery be avoided by using a LoadQuery statement?
var termStore = termStores.Where(t => t.Id == field.SspId).FirstOrDefault();
var termSet = termStore.GetTermSet(field.TermSetId);
var terms = termSet.GetAllTerms(); //TODO: Do we need a version that returns a paged set of terms? or queries the server again when a node is expanded?
context.Load(terms);
context.ExecuteQuery();
return terms;
}
I ended up writing my own code (please let me know if there's an easier way to do this).
My 'Term' object below is just a simple POCO with Name and Terms.
var terms = SharePointHelper.GetTaxonomyTerms(webUrl, libraryTitle, fieldTitle);
var term = terms.AsRootTreeViewTerm();
....
}
public static Term AsRootTreeViewTerm(this SP.TermCollection spTerms)
{
var root = new Term();
foreach (SP.Term spTerm in spTerms)
{
List<string> names = spTerm.PathOfTerm.Split(';').ToList();
var term = BuildTerm(root.Terms, names);
if (!root.Terms.Contains(term))
root.Terms.Add(term);
}
return root;
}
static Term BuildTerm(IList<Term> terms, List<string> names)
{
Term term = terms.Where(x => x.Name == names.First())
.DefaultIfEmpty(new Term() { Name = names.First() })
.First();
names.Remove(names.First());
if (names.Count > 0)
{
Term child = BuildTerm(term.Terms, names);
if (!term.Terms.Contains(child))
term.Terms.Add(child);
}
return term;
}
I have an image site where users can tag photos much like you can tag a question on Stackoverflow.
I have the following tables:
Images [ID, URL, etc]
Tags [ID, TagName]
ImageTag [TagID, ImageID]
I want to write a method with the signature:
public void UpdateImageTags(int imageId, IEnumerable<string> currentTags)
This method will do the following:
Create any new Tags in currentTags that don't already exist in the Tags table.
Get the old ImageTag's for an image.
Delete any ImageTag's that no longer exist in the currentTags
Add any ImageTag's that are new between the currentTags and oldTags.
Here is my attempt at that method:
public void UpdateImageTags(int imageId, IEnumerable<string> currentTags)
{
using (var db = new ImagesDataContext())
{
var oldTags = db.ImageTags.Where(it => it.ImageId == imageId).Select(it => it.Tag.TagName);
var added = currentTags.Except(oldTags);
var removed = oldTags.Except(currentTags);
// Add any new tags that need created
foreach (var tag in added)
{
if (!db.Tags.Any(t => t.TagName == tag))
{
db.Tags.InsertOnSubmit(new Tag { TagName = tag });
}
}
db.SubmitChanges();
// Delete any ImageTags that need deleted.
var deletedImageTags = db.ImageTags.Where(it => removed.Contains(it.Tag.TagName));
db.ImageTags.DeleteAllOnSubmit(deletedImageTags);
// Add any ImageTags that need added.
var addedImageTags = db.Tags.Where(t => added.Contains(t.TagName)).Select(t => new ImageTag { ImageId = imageId, TagId = t.TagId });
db.ImageTags.InsertAllOnSubmit(addedImageTags);
db.SubmitChanges();
}
}
However, this fails on the line:
db.ImageTags.DeleteAllOnSubmit(deletedImageTags);
With the error:
Local sequence cannot be used in LINQ to SQL implementations of query
operators except the Contains operator.
Is there an easier way I can handle the operation of adding new tags, deleting old ImageTags, adding new ImageTags in LINQ to SQL?
Seems like this would be easiest
public void UpdateImageTags(int imageId, IEnumerable<string> currentTags)
{
using (var db = new ImagesDataContext())
{
var image = db.Images.Where(it => it.ImageId == imageId).First()
image.Tags.Clear();
foreach(string s in currentTags)
{
image.Tags.Add(new Tag() { TagName = s});
}
db.SubmitChanges();
}
}
This might have to be modified slightly for LinqtoSQL. EF is what i have been using most recently. Also this is dependent on Lazy loading being enabled. If it is not, you will have to force the include of the image tags.
Here is a helper method to deal with many-to-many relationships:
public static void UpdateReferences<FK, FKV>(
this EntitySet<FK> refs,
Expression<Func<FK, FKV>> fkexpr,
IEnumerable<FKV> values)
where FK : class
where FKV : class
{
Func<FK, FKV> fkvalue = fkexpr.Compile();
var fkmaker = MakeMaker(fkexpr);
var fkdelete = MakeDeleter(fkexpr);
var fks = refs.Select(fkvalue).ToList();
var added = values.Except(fks);
var removed = fks.Except(values);
foreach (var add in added)
{
refs.Add(fkmaker(add));
}
foreach (var r in removed)
{
var res = refs.Single(x => fkvalue(x) == r);
refs.Remove(res);
fkdelete(res);
}
}
static Func<FKV, FK> MakeMaker<FKV, FK>(Expression<Func<FK, FKV>> fkexpr)
{
var me = fkexpr.Body as MemberExpression;
var par = Expression.Parameter(typeof(FKV), "fkv");
var maker = Expression.Lambda(
Expression.MemberInit(Expression.New(typeof(FK)),
Expression.Bind(me.Member, par)), par);
var cmaker = maker.Compile() as Func<FKV, FK>;
return cmaker;
}
static Action<FK> MakeDeleter<FK, FKV>(Expression<Func<FK, FKV>> fkexpr)
{
var me = fkexpr.Body as MemberExpression;
var pi = me.Member as PropertyInfo;
var assoc = Attribute.GetCustomAttribute(pi, typeof(AssociationAttribute))
as AssociationAttribute;
if (assoc == null || !assoc.DeleteOnNull)
{
throw new ArgumentException("DeleteOnNull must be set to true");
}
var par = Expression.Parameter(typeof(FK), "fk");
var maker = Expression.Lambda(
Expression.Call(par, pi.GetSetMethod(),
Expression.Convert(Expression.Constant(null), typeof(FKV))), par);
var cmaker = maker.Compile() as Action<FK>;
return cmaker;
}
Usage:
IEnumerable<Tag> values = ...;
Image e = ...;
e.ImageTags.UpdateReferences(x => x.Tag, tags);