Code breaks after update to Lucene.net 4.8.0-beta00001 - c#

I have just started using Lucene.net for a project. I have based my code on the code provided here: https://github.com/synhershko/LuceneNetDemo by Itamar Syn-Hershko. After I updated to the newest NuGet, the code breaks in a couple of places. What do I need to change?
First problem:
searcherManager.ExecuteSearch(searcher =>
{
var topDocs = searcher.Search(query, 10);
_totalHits = topDocs.TotalHits;
foreach (var result in topDocs.ScoreDocs)
{
var doc = searcher.Doc(result.Doc);
l.Add(new SearchResult
{
Name = doc.GetField("name")?.StringValue,
Description = doc.GetField("description")?.StringValue,
Url = doc.GetField("url")?.StringValue,
// Results are automatically sorted by relevance
Score = result.Score,
});
}
}, exception => { Console.WriteLine(exception.ToString()); });
The errormessage:
'SearcherManager' does not contain a definition for 'ExecuteSearch' and no extension method 'ExecuteSearch' accepting a first argument of type 'SearcherManager' could be found (are you missing a using directive or an assembly reference?)
Second problem:
public class HtmlStripAnalyzerWrapper : Analyzer
{
private readonly Analyzer _wrappedAnalyzer;
public HtmlStripAnalyzerWrapper(Analyzer wrappedAnalyzer)
{
_wrappedAnalyzer = wrappedAnalyzer;
}
public override TokenStreamComponents CreateComponents(string fieldName, TextReader reader)
{
return _wrappedAnalyzer.CreateComponents(fieldName, new HTMLStripCharFilter(reader));
}
}
The errormessage:
'HtmlStripAnalyzerWrapper.CreateComponents(string, TextReader)': cannot change access modifiers when overriding 'protected internal' inherited member 'Analyzer.CreateComponents(string, TextReader)'
And
Cannot access protected member 'Analyzer.CreateComponents(string, TextReader)' via a qualifier of type 'Analyzer'; the qualifier must be of type 'HtmlStripAnalyzerWrapper' (or derived from it)

There is an update to the demo at: https://github.com/NightOwl888/LuceneNetDemo
First Problem:
The API was inadvertently removed because it was not properly marked and does not exist in Lucene 4.8.0. However, it is only a supplemental API to SearcherManager.Acquire() and SearcherManager.Release(). You can see its usage in the SearcherManager documentation of Lucene 4.8.0.
var searcher = searcherManager.Acquire();
try
{
var topDocs = searcher.Search(query, 10);
_totalHits = topDocs.TotalHits;
foreach (var result in topDocs.ScoreDocs)
{
var doc = searcher.Doc(result.Doc);
l.Add(new SearchResult
{
Name = doc.GetField("name")?.GetStringValue(),
Description = doc.GetField("description")?.GetStringValue(),
Url = doc.GetField("url")?.GetStringValue(),
// Results are automatically sorted by relevance
Score = result.Score,
});
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
finally
{
searcherManager.Release(searcher);
searcher = null; // Never use searcher after this point!
}
We are considering whether to bring back the original ExecuteSearch() API, or create a new one that can be used with a using block for a more .NET-friendly experience. See an example of the second option in pull request 207. Feedback welcome.
Certainly, an API that swallows exceptions by default is less than ideal.
Second Problem:
Accessibility of API members was also corrected to match Lucene. CharFilters were not intended to be used in conjunction with pre-built Analyzers for performance reasons. Instead, you must build up an Analyzer from pre-built tokenizers and filters.
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.CharFilters;
using Lucene.Net.Analysis.Core;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.Util;
using System.IO;
namespace LuceneNetDemo.Analyzers
{
class HtmlStripAnalyzer : Analyzer
{
private readonly LuceneVersion matchVersion;
public HtmlStripAnalyzer(LuceneVersion matchVersion)
{
this.matchVersion = matchVersion;
}
protected override TokenStreamComponents CreateComponents(string fieldName, TextReader reader)
{
StandardTokenizer standardTokenizer = new StandardTokenizer(matchVersion, reader);
TokenStream stream = new StandardFilter(matchVersion, standardTokenizer);
stream = new LowerCaseFilter(matchVersion, stream);
stream = new StopFilter(matchVersion, stream, StopAnalyzer.ENGLISH_STOP_WORDS_SET);
return new TokenStreamComponents(standardTokenizer, stream);
}
protected override TextReader InitReader(string fieldName, TextReader reader)
{
return base.InitReader(fieldName, new HTMLStripCharFilter(reader));
}
}
}
Usage:
analyzer = new PerFieldAnalyzerWrapper(new HtmlStripAnalyzer(LuceneVersion.LUCENE_48),
new Dictionary<string, Analyzer>
{
{"owner", new LowercaseKeywordAnalyzer()},
{"name", new RepositoryNamesAnalyzer()},
});

Related

Type or namespace definition, or end-of-file expected problem on compiling

I'm creating an expo sender application which is web based, but I have a problem compiling the project. It keeps me saying
"Type or namespace definition, or end-of-file expected"
My code looks like this and I've got it from here
using MobileServices.Client;
using MobileServices.Models;
using System;
using System.Collections.Generic;
using System.Linq;
var expoSDKClient = new PushApiClient();
var pushTicketReq = new PushTicketRequest()
{
PushTo = new List<string>() { "..." },
PushBadgeCount = 7,
PushBody = ""
};
var result = expoSDKClient.PushSendAsync(pushTicketReq).GetAwaiter().GetResult();
if (result?.PushTicketErrors?.Count() > 0)
{
foreach (var error in result.PushTicketErrors)
{
Console.WriteLine($"Error: {error.ErrorCode} - {error.ErrorMessage}");
}
}
var pushReceiptResult = expoSDKClient.PushGetReceiptsAsync(pushReceiptReq).GetAwaiter().GetResult();
if (pushReceiptResult?.ErrorInformations?.Count() > 0)
{
foreach (var error in result.ErrorInformations)
{
Console.WriteLine($"Error: {error.ErrorCode} - {error.ErrorMessage}");
}
}
foreach (var pushReceipt in pushReceiptResult.PushTicketReceipts)
{
Console.WriteLine($"TicketId & Delivery Status: {pushReceipt.Key} {pushReceipt.Value.DeliveryStatus} {pushReceipt.Value.DeliveryMessage}");
}
This is my first C# application that I'm building and I have searched the error but with no result. Can you please help me how to solve this. I know that is something simple but I really need help because I'm stuck.
You need to declare both a namespace and a class like this, and then put it in a function.
Every piece of code in c# needs to be declared inside a type (that could be a class, or a struct or an interface etc.
All types in turn are declared inside interfaces, which are logical groupings of types and other interfaces.
All (or most of it) of your code modeling behavior, should then be declared inside functions.
using MobileServices.Client;
using MobileServices.Models;
using System;
using System.Collections.Generic;
using System.Linq;
namespace mynamespace {
public class ExpoClient {
public void DoPushSend() {
var expoSDKClient = new PushApiClient();
var pushTicketReq = new PushTicketRequest()
{
PushTo = new List<string>() { "..." },
PushBadgeCount = 7,
PushBody = ""
};
var result = expoSDKClient.PushSendAsync(pushTicketReq).GetAwaiter().GetResult();
if (result?.PushTicketErrors?.Count() > 0)
{
foreach (var error in result.PushTicketErrors)
{
Console.WriteLine($"Error: {error.ErrorCode} - {error.ErrorMessage}");
}
}
var pushReceiptResult = expoSDKClient.PushGetReceiptsAsync(pushReceiptReq).GetAwaiter().GetResult();
if (pushReceiptResult?.ErrorInformations?.Count() > 0) {
foreach (var error in result.ErrorInformations)
{
Console.WriteLine($"Error: {error.ErrorCode} - {error.ErrorMessage}");
}
}
foreach (var pushReceipt in pushReceiptResult.PushTicketReceipts)
{
Console.WriteLine($"TicketId & Delivery Status: {pushReceipt.Key} {pushReceipt.Value.DeliveryStatus} {pushReceipt.Value.DeliveryMessage}");
}
}
}
}
Be careful, the code in the repository is a snippet of the whole, possibly needing to be broken down to more functions. My fix only intends to make your code compile.
It looks like the code sample you're using is written in C# 9, which is currently in preview. This version introduces a new feature called top-level statements, which allows you to write code without enclosing it in a method of a class. You're probably using an earlier version, which expects the entry point to be in a static method named Main.

loading loose XAML drawing securely

A C# Windows application would like to load vector drawings that are stored in loose XAML files without allowing arbitrary code execution.
I am already loading such drawings from resources in linked assemblies over which I have control. However, I would like to also support loading loose XAML files. I imagine you can use XAML access control to limit the objects that can be instantiated in such XAML? Ideally, I would limit the loader to instantiating only the drawing primitives that are in the files we know about. It's ok that it would reject a file that has new drawing primitives in it that we have not whitelisted.
Is this a standard thing already supported by an API? Because I could not find it. Otherwise, does anyone have an example or beginnings of an example? This is for a free open source project and any help getting started would probably cut down the research I need to do by a lot.
The following seems to do a pretty decent job of white listing specific types in a XAML load:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Reflection;
using System.Windows.Controls;
using System.Windows.Media;
using System.Xaml;
using System.Xml;
namespace TestXamlLoading
{
internal class SchemaContext : XamlSchemaContext
{
// map from XAML element name to required namespace (currently always the same)
private static readonly Dictionary<string, string> AllowedTypes = new Dictionary<string, string>();
static SchemaContext()
{
// questionable: <Image> is used in some drawing XAML, should review it
foreach (string name in new[]
{
"Canvas", "Compound", "Ellipse", "GradientStop", "GradientStopCollection", "Group", "Line",
"LinearGradientBrush", "MatrixTransform", "Path", "PathGeometry", "Polygon",
"RadialGradientBrush", "Rectangle", "RotateTransform", "ScaleTransform", "SkewTransform", "TextBlock",
"TransformGroup", "TranslateTransform"
})
{
AllowedTypes[name] = "http://schemas.microsoft.com/winfx/2006/xaml/presentation";
}
}
public SchemaContext(IEnumerable<Assembly> referenceAssemblies, XamlSchemaContextSettings settings) : base(
referenceAssemblies, settings)
{
// no code
}
protected override XamlType GetXamlType(string xamlNamespace, string name, params XamlType[] typeArguments)
{
if (!AllowedTypes.TryGetValue(name, out string requiredNamespace) || xamlNamespace != requiredNamespace)
{
throw new Exception($"disallowed instantiation of '{xamlNamespace}' '{name}' from XAML");
}
return base.GetXamlType(xamlNamespace, name, typeArguments);
}
}
internal class Program
{
[STAThreadAttribute]
private static void Main(string[] args)
{
bool shouldFail = TestLoad("..\\..\\..\\badfile.xaml");
Debug.Assert(!shouldFail);
bool shouldSucceed = TestLoad("..\\..\\..\\goodfile.xaml");
Debug.Assert(shouldSucceed);
}
private static bool TestLoad(string path)
{
Stream inputStream = new FileStream(path, FileMode.Open);
XmlReader xmlReader = new XmlTextReader(inputStream);
Assembly[] referenceAssemblies =
{
// these are two separate assemblies which contain all the types we allow
Assembly.GetAssembly(typeof(Canvas)),
Assembly.GetAssembly(typeof(TransformGroup))
};
XamlSchemaContextSettings settings = new XamlSchemaContextSettings();
XamlSchemaContext schemaContext = new SchemaContext(referenceAssemblies, settings);
try
{
XamlReader reader = new XamlXmlReader(xmlReader, schemaContext);
Canvas canvas = (Canvas) System.Windows.Markup.XamlReader.Load(reader);
}
catch (Exception e)
{
Debug.WriteLine(e);
return false;
}
return true;
}
}
}

Problem with the Serializer for a Xslt2.0 transformation with Saxon

This is my first try to program a Xslt2.0 transformation with SaxonHE 9.9 in C#, so the problem here is when I create the serilizer I get the error that the class Saxon.Api.Serializer contains no constractor with 0 arguments.
I know what this error means, but not why it occurs, cause each example that I see creates the serializer like this.. This question sounds a bit stupid, but I cannot find a answer to get it work.
using Saxon.Api;
namespace XY
{
class Program
{
static void Main(string[] args)
{
String SourceFilename = "./test/test.xml";
String StylesheetFilename = "./scripte/xml-to-html.xsl";
String OutputFilename = "./Output/test.html";
using (FileStream streamXml = File.OpenRead(SourceFilename))
{
using (FileStream streamXsl = File.OpenRead(StylesheetFilename))
{
Processor processor = new Processor();
DocumentBuilder builder = processor.NewDocumentBuilder();
Uri uri = new Uri("urn:test");
builder.BaseUri = uri;
XdmNode input = builder.Build(streamXml);
XsltTransformer transformer = processor.NewXsltCompiler().Compile(streamXsl).Load();
transformer.InitialContextNode = input;
Serializer serializer = new Serializer();
serializer.SetOutputFile(OutputFilename);
transformer.Run(serializer);
}
}
Console.WriteLine("test.html created successfully");
}
}
}
EDIT
using System;
using Saxon.Api;
using System.IO;
using System.Reflection;
namespace XY
{
class Program
{
static void Main(string[] args)
{
string currentDirectory = Directory.GetCurrentDirectory();
String SourceFilename = ".\\test\\test.xml";
String StylesheetFilename = ".\\scripte\\xml-to-html.xsl";
String OutputFilename = ".\\Output\\result.html";
if (StylesheetFilename.StartsWith(".\\"))
{
StylesheetFilename = System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) + "\\" + StylesheetFilename;
}
if (SourceFilename.StartsWith(".\\"))
{
SourceFilename = System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) + "\\" + SourceFilename;
}
var uri_source = new System.Uri(SourceFilename);
var uri_xsl = new System.Uri(StylesheetFilename);
Processor processor = new Processor();
XdmNode input = processor.NewDocumentBuilder().Build(uri_source);
processor.SetProperty("http://saxon.sf.net/feature/preferJaxpParser", "true");
XsltCompiler compiler = processor.NewXsltCompiler();
XsltExecutable executable = compiler.Compile(uri_xsl);
XsltTransformer transformer = executable.Load();
transformer.InitialContextNode = input;
Serializer serializer = processor.NewSerializer();
System.IO.StreamWriter stream = new StreamWriter(OutputFilename);
serializer.SetOutputWriter(stream);
transformer.Run(serializer);
stream.Close();
}
}
}
I change also some other thinks and now it works, thanks for the answers.
I'll log a bug on the fact that there are sample apps and/or documentation that use the "new Serializer()" form.
We dropped this from the Java product in 9.8 because it caused constant trouble that the Serializer doesn't (necessarily) have access to all the configuration options (held in the Processor); also using a factory method Processor.newSerializer() potentially allows us to to create a subclass of Serializer, so it's more flexible. We then followed this pattern on .NET in the 9.9 release, partly for the same reasons, and partly because the .NET API has now been rewritten as a very thin layer on top of the Java API, which helps us to maintain commonality, and simplifies testing.
We try hard to maintain backwards compatibility in the main product APIs but it's not a requirement that overrides all others; if we feel that we got something badly wrong, then we fix it. As some people say to justify the policy, "the future is longer than the past".
LATER
We have done some checking and we think the 9.9 documentation and sample applications are correct; you must be using an older version. If I'm wrong, please identify the specific location where you found incorrect information.
In 9.9 you can (or really need to) create a Serializer with the various overloads of processor.NewSerializer (see http://saxonica.com/html/documentation/dotnetdoc/Saxon/Api/Processor.html#NewSerializer(Stream))..
Here is my solution for the problem:
using System;
using System.IO;
using Saxon.Api;
namespace Project1
{
public static class ClassMain
{
public static string TransformXml(string xmlData, string xslData)
{
var xsltProcessor = new Processor();
var documentBuilder = xsltProcessor.NewDocumentBuilder();
documentBuilder.BaseUri = new Uri("file://");
var xdmNode = documentBuilder.Build(new StringReader(xmlData));
var xsltCompiler = xsltProcessor.NewXsltCompiler();
var xsltExecutable = xsltCompiler.Compile(new StringReader(xslData));
var xsltTransformer = xsltExecutable.Load();
xsltTransformer.InitialContextNode = xdmNode;
var results = new XdmDestination();
xsltTransformer.Run(results);
return results.XdmNode.OuterXml;
}
public static void Main()
{
var xmlData = File.ReadAllText("a.xml");
var xslData = File.ReadAllText("a.xsl");
var data = TransformXml(xmlData, xslData);
Console.WriteLine(data);
Console.ReadKey();
}
}
}

Roslyn: Compile expression with non-public fields from external assembly

I am working on evaluating of user expressions from debugger. I want to compile expression in method context, and then inject IL-code with debugger.
Is it possible to compile expression, which contains non-public class/class-fields from external assembly to IL-code with Roslyn?
I've got 'MyNamespace.dll' with public class 'Test' and private method 'PrivateMethod', and I want to call it from Roslyn compilation.
I am trying to do it with next code:
public class TestCompilationOptions
{
public void Test()
{
var filePath = Path.Combine(Directory.GetCurrentDirectory(), "Output.dll");
Console.WriteLine("Preparing syntax tree");
string expressionString = #"
using System;
class XXX
{
public static void Main()
{
Console.WriteLine(MyNamespace.Test.PrivateMethod(2));
}
}";
//SyntaxTree targetTree = SyntaxFactory.ParseSyntaxTree(expressionString);
SyntaxTree targetTree = CSharpSyntaxTree.ParseText(expressionString);
Console.WriteLine("Preparing metadata references");
Assembly[] assemblys = new Assembly[4];
assemblys[0] = typeof(MyNamespace.Test).Assembly;
assemblys[1] = typeof(Console).Assembly;
assemblys[2] = typeof(object).Assembly;
assemblys[3] = Assembly.LoadFile(Path.Combine(Directory.GetCurrentDirectory(), "System.Runtime.dll"));
MetadataReference[] metadataReferences = MetadataFromAssembly(assemblys);
Console.WriteLine("Preparing default namespaces");
IEnumerable<string> DefaultNamespaces = new[] {"System", "System.Runtime"};
Console.WriteLine("Preparing compilation options");
CSharpCompilationOptions ReleaseDll = new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary, optimizationLevel: OptimizationLevel.Release);
CSharpCompilationOptions cOptions = ReleaseDll.WithUsings(DefaultNamespaces);
//.WithMetadataImportOptions(MetadataImportOptions.All);
Console.WriteLine("Getting compilation");
CSharpCompilation compilation = CSharpCompilation.Create("Output.dll", new SyntaxTree[] {targetTree}, metadataReferences, cOptions);
Console.WriteLine("Emitting compilation");
using (var dll = new FileStream(filePath, FileMode.Create, FileAccess.Write))
{
var emitRes = compilation.Emit(dll);
if (!emitRes.Success)
{
Console.WriteLine("Emited unsuccessfully!");
foreach (var d in emitRes.Diagnostics)
Console.WriteLine(d.ToString());
return;
}
}
}
public unsafe MetadataReference[] MetadataFromAssembly(Assembly[] assemblys)
{
MetadataReference[] result = new MetadataReference[assemblys.Length];
byte *b;
int length;
for (int i = 0; i < assemblys.Length; i++)
{
if (assemblys[i].TryGetRawMetadata(out b, out length))
{
var moduleMetadata = ModuleMetadata.CreateFromMetadata((IntPtr) b, length);
var assemblyMetadata = AssemblyMetadata.Create(moduleMetadata);
result[i] = assemblyMetadata.GetReference();
}
else
{
return null;
}
}
return result;
}
And got following error:
(8,44): error CS0117: 'Test' does not contain a definition for 'privateMember'
I've made 'WithMetadataImportOptions' and 'MetadataImportOptions' public inside Roslyn and uncomennted line
//.WithMetadataImportOptions(MetadataImportOptions.All);
And then got following error:
(8,44): error CS0122: 'Test.privateMember' is inaccessible due to its protection level
So may be it could be done using some Roslyn API?
P.S.
I know, that I can get non-public fields symbols using System.Reflection, but how do I compile the expression then?
If a member is private, you can't access it with normal code in another class. Nothing to do with Roslyn in particular.
If you really do actually need to access a private member in a different class, and you fully understand why it may not be a good idea, the code that accesses it must do so using reflection.

MSMQ empty object on message body

Ok, so I'm very VERY new to MSMQ and I'm already confused.
I have created a private queue and added a few messages to it, all good so far. BUT when I retrieve the messages back from the queue the message body contains a empty object of the type I added. By this I don't mean that the body is null, it does have a reference to a type of the object that I added, but it's not instantiated so all the properties are in their null or default state.
This is the code I use to add to the queue:
using (var mQueue = new MessageQueue(QueueName))
{
var msg = new Message(observation)
{
Priority = MessagePriority.Normal,
UseJournalQueue = true,
AcknowledgeType = AcknowledgeTypes.FullReceive,
};
mQueue.Send(msg);
}
And this is the code that dequeues the messages:
using (var mQueue = new MessageQueue(QueueName))
{
mQueue.MessageReadPropertyFilter.SetAll();
((XmlMessageFormatter)mQueue.Formatter).TargetTypes =
new[] { typeof(Observation) };
var msg = mQueue.Receive(new TimeSpan(0, 0, 5));
var observation = (Observation)msg.Body;
return observation;
}
The Message constructor uses XML serialization to serialize your "observation" object. You'll need to make sure that this works properly. XML serialization will only deal with public members of the class, it is not going to serialize private members. Your object may well look "empty" after it is deserialized again.
Here's some test code to verify that it works properly:
using System;
using System.IO;
using System.Xml.Serialization;
class Program {
static void Main(string[] args) {
var ser = new XmlSerializer(typeof(Observation));
var sw = new StringWriter();
var obj = new Observation();
ser.Serialize(sw, obj);
Console.WriteLine(sw.ToString());
var sr = new StringReader(sw.ToString());
var obj2 = (Observation)ser.Deserialize(sr);
// Compare obj to obj2 here
//...
Console.ReadLine();
}
}
public class Observation {
// etc...
}
Also, make sure that your custom Message object has public setters on each property!

Categories