I am asking this question, because I didn't find yet any posts that are C# related and there might be some build in methods for that I couldn't find. If there are, please tell me so and I can close this question.
Basically I have the common situation:
User types a function w.r.t. one or two variables into some TextBlock
I take this string analyse it
As a return I would like to have a delegate to a method that will take one or two inputs (the variables) and return the function value according to what the user typed in.
Now, I could probably think (and I would like to do this on my own, because I want to use my brain) of an algorithm of analysing the string step by step to actually find out, what has to be calculated first and in what way. E.g. First scan for parentheses, look for the expression within a group of parantheses and calculate that according to more general functions etc.
But in the end I would like to "create" a method of this analysis to be easily used as a normal delegate with a couple of arguments that will return the correct function value.
Are there any methods included in C# for that already, or would I have to go and program everything by myself?
As a remark: I don't want to use anybody else' library, only .NET libraries are acceptable for me.
Edit: After Matt pointed out expression trees, I found this thread which is a good example to my problem.
Edit2: The example pointed out does only include simple functions and will not be useful if I want to include more complex functions such as trigonometric ones or exponentials.
What you are describing is a parser. There are a number of different ways of implementing them, although generally speaking, for complex grammars, a "parser generator" is often used.
A parser generator will take a description of the grammar and convert it into code that will parse text that conforms to the grammar into some form of internal representation that can be manipulated by the program, e.g. a parse tree.
Since you indicate you want to avoid third-party libraries, I'll assume that the use of a parser generator is similarly excluded, which leaves you with implementing your own parser (which fortunately is quite an interesting exercise).
The Wikipedia page on Recursive descent parsers will be particularly useful. I suggest reading through it and perhaps adapting the example code therein to your particular use case. I have done this myself a number of times for different grammars with this as a starting point, so can attest to its usefulness.
The output from such a parser will be a "parse tree". And you then have a number of possibilities for how you convert this into an executable delegate. One option is to implement an Evaluate() method on your parse tree nodes, which will take a set of variables and return the result of evaluating the user's expression. As others have mentioned, your parse tree could leverage .NET's Expression trees, or you can go down the route of emitting IL directly (permitting you to produce a compiled .NET assembly from the user's expression for later use as required).
You might want to look at expression trees.
Check out NCalc for some examples of how to do this. You don't need to use the library, but reading the source is pretty educational.
I found a very helpful pdf explaining the parsing in C# 2.0. This link leads to a very good tutorial on parsers used in C# and also applies that later on to an arithmetic expression.
As this directly helps and answers to my question, I posted this as an answer, rather than as a comment or edit.
Related
I am wondering if are there any examples (googling I haven't found any) of TAB auto-complete solutions for Command Line Interface (console), that use ANTLR4 grammars for predicting the next term (like in a REPL model).
I've written a PL/SQL grammar for an open source database, and now I would like to implement a command line interface to the database that provides the user the feature of completing the statements according to the grammar, or eventually discover the proper database object name to use (eg. a table name, a trigger name, the name of a column, etc.).
Thanks for pointing me to the right direction.
Actually it is possible! (Of course, based on the complexity of your grammar.) Problem with auto-completion and ANTLR is that you do not have complete expression and you want to parse it. If you would have complete expression, it wont be any big problem to know what kind of element is at what place and to know what can be used at such a place. But you do not have complete expression and you cannot parse the incomplete one. So what you need to do is to wrap the input into some wrapper/helper that will complete the expression to create a parse-able one. Notice that nothing that is added only to complete the expression is important to you - you will only ask for members up to last really written character.
So:
A) Create the wrapper that will change this (excel formula) '=If(' into '=If()'
B) Parse the wrapped input
C) Realize that you are in the IF function at the first parameter
D) Return all that can go into that place.
It actually works, I have completed intellisense editor for several simple languages. There is much more infrastructure than this, but the basic idea is as I wrote it. Only be careful, writing the wrapper is not easy if not impossible if the grammar is really complex. In that case look at Papa Carlo project. http://lakhin.com/projects/papa-carlo/
As already mentioned auto completion is based on the follow set at a given position, simply because this is what we defined in the grammar to be valid language. But that's only a small part of the task. What you need is context (as Sam Harwell wrote: it's a semantic process, not a syntactic one). And this information is independent of the parser. And since a parser is made to parse valid input (and during auto completion you have most of the time invalid input), it's not the right tool for this task.
Knowing what token can follow at a given position is useful to control the entire process (e.g. you don't want to show suggestions if only a string can appear), but is most of the time not what you actually want to suggest (except for keywords). If an ID is possible at the current position, it doesn't tell you what ID is actually allowed (a variable name? a namespace? etc.). So what you need is essentially 3 things:
A symbol table that provides you with all possible names sorted by scope. Creating this depends heavily on the parsed language. But this is a task where a parser is very helpful. You may want to cache this info as it is time consuming to run this analysis step.
Determine in which scope you are when invoking auto completion. You could use a parser as well here (maybe in conjunction with step 1).
Determine what type of symbol(s) you want to show. Many people think this is where a parser can give you all necessary information (the follow set). But as mentioned above that's not true (keywords aside).
In my blog post Universal Code Completion using ANTLR3 I especially addressed the 3rd step. There I don't use a parser, but simulate one, only that I don't stop when a parser would, but when the caret position is reached (so it is essential that the input must be valid syntax up to that point). After reaching the caret the collection process starts, which not only collects terminal nodes (for keywords) but looks at the rule names to learn what needs to be collected too. Using specific rule names is my way there to put context into the grammar, so when the collection code finds a rule table_ref it knows that it doesn't need to go further down the rule chain (to the ultimate ID token), but instead can use this information to provide a list of tables as suggestion.
With ANTLR4 things might become even simpler. I haven't used it myself yet, but the parser interpreter could be a big help here, as it essentially doing what I do manually in my implementation (with the ANTLR3 backend).
This is probably pretty hard to do.
Fundamentally you want to use some parser to predict "what comes next" to display as auto-completion. This has to at least predict what the FIRST token is at the point where the user's input stops.
For ANTLR, I think this will be very difficult. The reason is that ANTLR generates essentially procedural, recursive descent parsers. So at runtime, when you need to figure out what FIRST tokens are, you have to inspect the procedural source code of the generated parser. That way lies madness.
This blog entry claims to achieve autocompletion by collecting error reports rather than inspecting the parser code. Its sort of an interesting idea, but I do not understand how his method really works, and I cannot see how it would offer all possible FIRST tokens; it might acquire some of them. This SO answer confirms my intuition.
Sam Harwell discusses how he has tackled this; he is one of the ANTLR4 implementers and if anybody can make this work, he can. It wouldn't surprise me if he reached inside ANTLR to extract the information he needs; as an ANTLR implementer he would certainly know where to tap in. You are not likely to be so well positioned. Even so, he doesn't really describe what he did in detail. Good luck replicating. You might ask him what he really did.
What you want is a parsing engine for which that FIRST token information is either directly available (the parser generator could produce it) or computable based on the parser state. This is actually possible to do with bottom up parsers such as LALR(k); you can build an algorithm that walks the state tables and computes this information. (We do this with our DMS Software Reengineering Toolkit for its GLR parser precisely to produce syntax error reports that say "missing token, could be any of these [set]")
I have seen Attributes and Reflection and now i know how to create and use reflection to see meta data of attribute but is it possible to make a standalone tool that can analyse a cs file and extract attributes used ?
What am I trying to do?
Basically I am working on a tool which takes C# code as input. Next step is to see what Attributes are used in that source code. Extract Intrinsic and Custom Attributes.
Probem?
this makes sense if you are using reflection in same project in which your attributes are defined, however I do not know in what direction I should move to write a separate tool that can give you above extracted statics and meta data of attributes.
Some say I should use Regex to extract the attributes in files where as other say I need to use Irony - .NET Language Implementation Kit
Furthermore
above work will result me to have an application that will be used for attributes(annotation) based Design Pattern Recovery from Source Code. I have less idea if Regex would come to rescue or i need something like Reflection. As Reflection is deals with runtime. I do not have to deal with run time. just static files analysis
If I properly understood your problem, you really need to parse your code. Regex won't help you, as beside parsing attributes you will need to parse class hierarchy. Reflection might do the trick, but you won't be able to show to the user the results. So, the best idea is to use any parser to get an expression tree from the source, and than investigate it.
If you don't know which parser to choose - I'd recommend Rosalyn, as it should be easiest for parsing C# code (it is designed especially for it). You can find an example for parsing here:
http://blog.filipekberg.se/2011/10/20/using-roslyn-to-parse-c-code-files/
I think it should be really powerful and useful for your task
Apparently I don't have enough reputation to comment, so I'm gonna have to say this as an answer.
Reflection deals with Runtime Type Information. It is a mechanism for finding out things about a type that you the programmer don't already know about (perhaps someone else is providing you a code library, and forgot to document it). Reflection will give you any information you need about the public contract of a class, including methods, properties, fields, attributes, and interfaces/classes inherited.
What you need however is a parser. A parser is a standard programming concept that processes files and extracts specific information. You are looking for information in code files, which are not runtime types yet, which means reflection has no information on them yet, however you have your eyes, since they're still code files. In the event your eyes are not sufficient (as I suspect their not if you asked the question) you need to write a parser. Extracting specific information from a cs file is pretty simple. And the regex for an attribute is: \[.+\]
This question sounds trivial but let me explain my scenario.
I am working in an object oriented programming language (C#) and most of the actual execution code is procedural, i.e. series of statements, sometimes branches and loops. Fairly standard.
Now I am presented with a task to deal with a textual format (PGN, but it could be anything other like VCard or some custom format). At least for me, the "standard" way to work with it would be to use a mix of:
regular expressions
if / switch statements
for-loops
storing regexp matches into some custom structure and / or outputting it to some result format
However, I don't like this procedural approach at all - regular expressions are prone to errors, the code is usually quite hard to understand and debug, it usually tends to have quite a high cyclomatic complexity etc.
Simply put, I'd like it to be declarative but I don't know what tools or libraries to use.
I remember that when I saw demos of the "M" language I thought that that was exactly I was looking for. There was a simple way to declare syntax of my textual format, the tool would then automatically parse input string into an in-memory representation of the textual DSL, I think that it was also possible to transform the format into another etc.
I have been also in touch with the people behind JetBrains MPS which is another tool for working with DSLs but my scenario doesn't seem to be a perfect match for what they are trying to provide.
So if anyone has any idea about how to elegantly deal with textual formats in otherwise procedural code base, I'd be happy to learn about the options.
Check out my open source project meta#. I think it sounds like exactly what you're looking for.
I am researching ways, tools and techniques to parse code files in order to support syntax highlighting and intellisence in an editor written in c#.
Does anyone have any ideas/patterns & practices/tools/techiques for that.
EDIT: A nice source of info for anyone interested:
Parsing beyond Context-free grammars
ISBN 978-3-642-14845-3
My favourite parser for C# is Irony: http://irony.codeplex.com/ - i have used it a couple of times with great success
Here is a wikipedia page listing many more: http://en.wikipedia.org/wiki/Compiler-compiler
There are two basic aproaches:
1) Parse the entire solution and everything it references so you understand all the types involved in the code
2) Parse locally and do your best to guess what types etc are.
The trouble with (2) is that you have to guess, and in some circumstances you just can't tell from a code snippet exactly what everything is. But if you're happy with the sort oif syntax highlighting shown on (e.g.) Stack Overflow, then this approach is easy and quite effective.
To do (1) then you need to do one of (in decreasing order of difficulty):
Parse all the source code. Not possible if you reference 3rd party assemblies.
Use reflection on the compiled code to garner type information you can use when parsing the source.
Use the host IDE's (if avaiable - so not applicable in your case!) code element interfaces to provide the information you need
You could take a look at how http://www.icsharpcode.net/ did it. They wrote a book doing just that, Dissecting a C# Application: Inside SharpDevelop, it even has a chapter called
Implement a parser to provide syntax
highlighting and auto-completion as
users type
I often type string in c# when actually I want to type String.
I know that string is an alias of String and I am really just being pedantic but i wish to outlaw string to force me to write String.
Can this be done in ether visual studio intellesence or in resharper and how?
I've not seen it done before, but you may be able to achieve this with an Intellisense extension. A good place to start would be to look at the source for this extension on CodePlex.
Would be good to hear if you have any success with this.
I have always read in "Best Coding Practices" for C# to prefer string, int, float ,double to String, Int32, Single, Double. I think it is mostly to make C# look less like VB.NET, and more like C, but it works for me.
Also, you can go the other way, and add the following on top of every file
using S = System.String;
..
S msg = #"I don't like string.";
you may laugh at this, but I have found it invaluable when I have two similar source codes with different underlying data types. I usually have using num=System.Single; or using num=System.Double; and the rest of the code is identical, so I can copy and paste from one file to the other and maintain both single precision and double precision library in sync.
I think ReSharper can do this!
Here is an extract from the documentation:
ReSharper 5 provides Structural Search and Replace to find custom code constructs and replace them with other code constructs. What's even more exciting is that it's able to continuously monitor your solution for your search patterns, highlight code that matches them, and provide quick-fixes to replace the code according to your replace patterns. That essentially means that you can extend ReSharper's own 900+ code inspections with your custom inspections. For example, if you're migrating to a newer version of a framework, you can create search patterns to find usages of its older API and replace patterns to introduce an updated API.
cheers,
Chris