We Keep Coding

How to include keywords and aliases in Roslyn recommended symbols?

I am using Roslyn to create a C# scripting control with IntelliSense.
I am generally very happy with the results I am getting, however, the recommended symbols don't include keywords such as for and if et cetera and also don't contain type aliases such as int, when it includes Int32.
More specifically, I am using Microsoft.CodeAnalysis.Recommendations, that is:
Recommender.GetRecommendedSymbolsAtPositionAsync(mySemanticModel, scriptPosition, myAdhocWorkspace);
My SemanticModel object is obtained from a C# compilation which always has a reference to mscorlib.dll at the very least.
At all positions in my script, the recommended completions are always correct. However, I would argue that they are incomplete if they are missing keywords such as if, else and for etc.
I can see that it would be easy for me to include common type aliases in my IntelliSense manually. That is, if Int32 is a possible completion, then I could manually add int.
However, it is less obvious when an if statement or a for statement or even is/as would be appropriate in the given scope.
Is there a way to include these keywords when getting the recommended symbols this way?
Is there also a way to automatically include type aliases?

It seems that Recommender.GetRecommendedSymbolsAtPositionAsync provides only symbols completion. That mean, Methods, Types etc (ISymbol implementations).
If you want keywords or snippets completion, you can use Microsoft.CodeAnalysis.Completion.CompletionService
void CompletionExample()
{
var code = #"using System;
namespace NewConsoleApp
{
class NewClass
{
void Method()
{
fo // I want to get 'for' completion for this
}
}
}";
var completionIndex = code.LastIndexOf("fo") + 2;
// Assume you have a method that create a workspace for you
var workspace = CreateWorkspace("newSln", "newProj", code);
var doc = workspace.CurrentSolution.Projects.First().Documents.First();
var service = CompletionService.GetService(doc);
var completionItems = service.GetCompletionsAsync(doc, completionIndex).Result.Items;
foreach (var result in completionItems)
{
Console.WriteLine(result.DisplayText);
Console.WriteLine(string.Join(",", result.Tags));
Console.WriteLine();
}
}
You can play around to figure it out how to customize it for your needs (rules, filters).
Notice that each result comes from a specific completion provider (item.Properties["Provider"]) and you can create a custom CompletionProvider (at least you should be able).
You can also take a look at C# for VS code (that powered with OmniSharp) to see how they did the work.

basic scripting language usable from C#

For a project, I require that my users are able to provide a simple function as a string, and I want to be able to evaluate it from my program.
Let's say the function signature is always int f(int a, int b)
I want to make it so that my users can provide implementations of f written in a basic language.
For instance:
return a*b;
if(a ==1) return b else return 3
And in my program, I want to be able to parse the user input function to make it a real callable function from my C# code. I only require basic features like basic math stuff and conditionals (not even loops).
Is there a basic "language" that exists that would allow me to interpret such functions defined as input and transform them in callable code?
For instance I think that using Lua for this overkill?
Ideally I want something open-source, with minimal codebase and that is widely known to be of good quality. Does such a thing exist? Ideally it should also have implementations in several languages, like C# and Java.
I know the temptation to write my own parser is high but I feel like there must be an out-of-the-box solution for my problem

If you only need math functions you can just have take in a string an eval it in a data table then return the results.
using System.Data;
//string mathExp = "3 * (2+4)"
public string ShiestyEval(string mathExp)
{
DataTable dt = new DataTable();
var v = dt.Compute(mathExp,"");
return v;
}
To get it to be callable from your application, you need to provide an interface in which they can eval there methods, or if you wanna do some shady hack and generate an assembly though Il, and load it you can, it's a little vague on how you want to use it
you can also checkout JSharp

I don't think there is a simple way to do it. But there are some solutions that are easy to use. I really like Irony had a lot of fun with it. But it's a bit abandoned. You can use ANTLR but this one is not simple. You might be able to use Roslyn Api to do what you want it all depends. I am not sure how good of a sandbox do you need but you can use C# scripts
This will allow you to do something like this
ScriptState state = CSharpScript.Run("int Times(int x) { return x * x; }");
var fn = state.CreateDelegate>("Times");
var value = fn(5);
Console.Write(value);

Is there a way to implement custom language features in C#?

I've been puzzling about this for a while and I've looked around a bit, unable to find any discussion about the subject.
Lets assume I wanted to implement a trivial example, like a new looping construct: do..until
Written very similarly to do..while
do {
//Things happen here
} until (i == 15)
This could be transformed into valid csharp by doing so:
do {
//Things happen here
} while (!(i == 15))
This is obviously a simple example, but is there any way to add something of this nature? Ideally as a Visual Studio extension to enable syntax highlighting etc.

Microsoft proposes Rolsyn API as an implementation of C# compiler with public API. It contains individual APIs for each of compiler pipeline stages: syntax analysis, symbol creation, binding, MSIL emission. You can provide your own implementation of syntax parser or extend existing one in order to get C# compiler w/ any features you would like.
Roslyn CTP
Let's extend C# language using Roslyn! In my example I'm replacing do-until statement w/ corresponding do-while:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Roslyn.Compilers.CSharp;
namespace RoslynTest
{
class Program
{
static void Main(string[] args)
{
var code = #"
using System;
class Program {
public void My() {
var i = 5;
do {
Console.WriteLine(""hello world"");
i++;
}
until (i > 10);
}
}
";
//Parsing input code into a SynaxTree object.
var syntaxTree = SyntaxTree.ParseCompilationUnit(code);
var syntaxRoot = syntaxTree.GetRoot();
//Here we will keep all nodes to replace
var replaceDictionary = new Dictionary<DoStatementSyntax, DoStatementSyntax>();
//Looking for do-until statements in all descendant nodes
foreach (var doStatement in syntaxRoot.DescendantNodes().OfType<DoStatementSyntax>())
{
//Until token is treated as an identifier by C# compiler. It doesn't know that in our case it is a keyword.
var untilNode = doStatement.Condition.ChildNodes().OfType<IdentifierNameSyntax>().FirstOrDefault((_node =>
{
return _node.Identifier.ValueText == "until";
}));
//Condition is treated as an argument list
var conditionNode = doStatement.Condition.ChildNodes().OfType<ArgumentListSyntax>().FirstOrDefault();
if (untilNode != null && conditionNode != null)
{
//Let's replace identifier w/ correct while keyword and condition
var whileNode = Syntax.ParseToken("while");
var condition = Syntax.ParseExpression("(!" + conditionNode.GetFullText() + ")");
var newDoStatement = doStatement.WithWhileKeyword(whileNode).WithCondition(condition);
//Accumulating all replacements
replaceDictionary.Add(doStatement, newDoStatement);
}
}
syntaxRoot = syntaxRoot.ReplaceNodes(replaceDictionary.Keys, (node1, node2) => replaceDictionary[node1]);
//Output preprocessed code
Console.WriteLine(syntaxRoot.GetFullText());
}
}
}
///////////
//OUTPUT://
///////////
// using System;
// class Program {
// public void My() {
// var i = 5;
// do {
// Console.WriteLine("hello world");
// i++;
// }
//while(!(i > 10));
// }
// }
Now we can compile updated syntax tree using Roslyn API or save syntaxRoot.GetFullText() to text file and pass it to csc.exe.

The big missing piece is hooking into the pipeline, otherwise you're not much further along than what .Emit provided. Don't misunderstand, Roslyn brings alot of great things, but for those of us who want to implement preprocessors and meta programming, it seems for now that was not on the plate. You can implement "code suggestions" or what they call "issues"/"actions" as an extension, but this is basically a one off transformation of code that acts as a suggested inline replacement and is not the way you would implement a new language feature. This is something you could always do with extensions, but Roslyn makes the code analysis/transformation tremendously easier:
From what I've read of comments from Roslyn developers on the codeplex forums, providing hooks into the pipeline has not been an initial goal. All of the new C# language features they've provided in C# 6 preview involved modifying Roslyn itself. So you'd essentially need to fork Roslyn. They have documentation on how to build Roslyn and test it with Visual Studio. This would be a heavy handed way to fork Roslyn and have Visual Studio use it. I say heavy-handed because now anyone who wants to use your new language features must replace the default compiler with yours. You could see where this would begin to get messy.
Building Roslyn and replacing Visual Studio 2015 Preview's compiler with your own build
Another approach would be to build a compiler that acts as a proxy to Roslyn. There are standard APIs for building compilers that VS can leverage. It's not a trivial task though. You'd read in the code files, call upon the Roslyn APIs to transform the syntax trees and emit the results.
The other challenge with the proxy approach is going to be getting intellisense to play nicely with any new language features you implement. You'd probably have to have your "new" variant of C#, use a different file extension, and implement all the APIs that Visual Studio requires for intellisense to work.
Lastly, consider the C# ecosystem, and what an extensible compiler would mean. Let's say Roslyn did support these hooks, and it was as easy as providing a Nuget package or a VS extension to support a new language feature. All of your C# leveraging the new Do-Until feature is essentially invalid C#, and will not compile without the use of your custom extension. If you go far enough down this road with enough people implementing new features, very quickly you will find incompatible language features. Maybe someone implements a preprocessor macro syntax, but it can't be used along side someone else's new syntax because they happened to use similar syntax to delineate the beginning of the macro. If you leverage alot of open source projects and find yourself digging into their code, you would encounter alot of strange syntax that would require you side track and research the particular language extensions that project is leveraging. It could be madness. I don't mean to sound like a naysayer, as I have alot of ideas for language features and am very interested in this, but one should consider the implications of this, and how maintainable it would be. Imagine if you got hired to work somewhere and they had implemented all kinds of new syntax that you had to learn, and without those features having been vetted the same way C#'s features have, you can bet some of them would be not well designed/implemented.

You can check www.metaprogramming.ninja (I am the developer), it provides an easy way to accomplish language extensions (I provide examples for constructors, properties, even js-style functions) as well as full-blown grammar based DSLs.
The project is open source as well. You can find documentations, examples, etc at github.
Hope it helps.

You can't create your own syntactic abstractions in C#, so the best you can do is to create your own higher-order function. You could create an Action extension method:
public static void DoUntil(this Action act, Func<bool> condition)
{
do
{
act();
} while (!condition());
}
Which you can use as:
int i = 1;
new Action(() => { Console.WriteLine(i); i++; }).DoUntil(() => i == 15);
although it's questionable whether this is preferable to using a do..while directly.

I found the easiest way to extend the C# language is to use the T4 text processor to preprocess my source. The T4 Script would read my C# and then call a Roslyn based parser, which would generate a new source with custom generated code.
During build time, all my T4 scripts would be executed, thus effectively working as an extended preprocessor.
In your case, the none-compliant C# code could be entered as follows:
#if ExtendedCSharp
do
#endif
{
Console.WriteLine("hello world");
i++;
}
#if ExtendedCSharp
until (i > 10);
#endif
This would allow syntax checking the rest of your (C# compliant) code during development of your program.

No there is no way to achieve what you'are talking about.
Cause what you're asking about is defining new language construct, so new lexical analysis, language parser, semantic analyzer, compilation and optimization of generated IL.
What you can do in such cases is use of some macros/functions.
public bool Until(int val, int check)
{
return !(val == check);
}
and use it like
do {
//Things happen here
} while (Until(i, 15))

Is there .net magic to get parameter values by name in console application?

I've been developing .net console applications using C# and have always just dictated what order parameters must be inserted in so that args[0] is always start date and args[1] is always end date, for example.
however I would like to move over to using named parameters so that any combination of parameters can be sent in any order, such as the typical "-sd" would prefix a start date.
I know I could parse through the args[] looking for "-" and then read the name and look the next position for the accompanying value, but before doing that wanted to see if there was any kind of baked in handling for this rather standard practice.
is there something like this out there already that could do as such:
DateTime startDate = (DateTime)((ConsoleParameters)args[])["sd"]
I'm using C# and .Net 4

There is nothing built into the core framework.
A lot of people think NDesk.Options is useful for this sort of thing. Check out this example (taken directly from the provided link):
string data = null;
bool help = false;
int verbose = 0;
var p = new OptionSet () {
{ "file=", v => data = v },
{ "v|verbose", v => { ++verbose } },
{ "h|?|help", v => help = v != null },
};
List<string> extra = p.Parse (args);

Yes, the "magic" is that this is a common problem and it has been adequately solved. So I recommend using an already written library to handle parsing command line arguments.
CommandLineParser has been great for me. It is reasonably documented and flexible enough for every type of command line argument I've wanted to handle. Plus, it assists with usage documentation.
I will say that I'm not the biggest fan of making a specific class that has to be adorned with attributes to use this library, but it's a minor point considering that it solves my problem. And in reality forcing that attributed class pushes me to keep that class separate from where my app actually retrieves it's settings from and that always seems to be a better design.

You can use NDesk.Options.

There is no such a thing as named parameters. "-sd" is just a choice for a specific application. It can be "/sd" as well. Or "sd=". Or whatever you want.
Since there are no named parameters, there is nothing inside .NET Framework which let you use the "-sd" syntax.
But you can quite easily build your own method to get a set of "named parameters" for your app.
Edit: or, even better, you can use an existing library, like suggested in other answers.
Edit: reading the answer by #Sander Rijken, I see that I was wrong: there were still an implementation of "-sd" syntax in .NET 4.0 before the release. But since it was dropped before the final release, the only ways are still to create your own method or to use an existing library.

Is there a way I can dynamically define a Predicate body from a string containing the code?

This is probably a stupid question, but here goes. I would like to be able to dynamically construct a predicate < T > from a string parsed from a database VARCHAR column, or any string, for that matter. For example, say the column in the database contained the following string:
return e.SomeStringProperty.Contains("foo");
These code/string values would be stored in the database knowing what the possible properties of the generic "e" is, and knowing that they had to return a boolean. Then, in a magical, wonderful, fantasy world, the code could execute without knowing what the predicate was, like:
string predicateCode = GetCodeFromDatabase();
var allItems = new List<SomeObject>{....};
var filteredItems = allItems.FindAll(delegate(SomeObject e) { predicateCode });
or Lambda-ized:
var filteredItems = allItems.FindAll(e => [predicateCode]);
I know it can probably never be this simple, but is there a way, maybe using Reflection.Emit, to create the delegate code dynamically from text and give it to the FindAll < T > (or any other anonymous/extension) method?

The C# and VB compilers are available from within the .NET Framework:
C# CodeDom Provider
Be aware though, that this way you end up with a separate assembly (which can only be unloaded if it's in a separate AppDomain). This approach is only feasible if you can compile all the predicates you are going to need at once. Otherwise there is too much overhead involved.
System.Reflection.Emit is a great API for dynamically emitting code for the CLR. It is, however, a bit cumbersome to use and you must learn CIL.
LINQ expression trees are an easy to use back-end (compilation to CIL) but you would have to write your own parser.
I suggest you have a look at one of the "dynamic languages" that run on the CLR (or DLR) such as IronPython. It's the most efficient way to implement this feature, if you ask me.

Check out the Dynamic Linq project it does all this and more!
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
Great for simple stuff like user selected orderby's or where clauses

It is possible using emit, but you'd be building your own parser.
EDIT
I remember that in ScottGu's PDC keynote, he showed a feature using the CLI version of the .net framework that resembled Ruby's eval, but I can't find a URL that can corroborate this. I'm making this a commnity wiki so that anyone who has a good link can add it.

I stepped off the dynamic linq because it's limited in ways I want to search a collection, unless you prove me wrong.
My filter needs to be: in a list of orders, filter the list so that I have only the orders with in the collection of items in that order, an item with the name "coca cola".
So that will result to a method of: orders.Findall(o => o.Items.Exists(i => i.Name == "coca cola"))
In dynamic linq I didn't find any way to do that, so I started with CodeDomProvicer.
I created a new Type with a method which contains my dynamically built FindAll Method:
public static IList Filter(list, searchString)
{
// this will by dynamically built code
return orders.Findall(o => o.Items.Exists(i => i.Name == "coca cola"));
}
when I try to build this assembly:
CompilerResults results = provider.CompileAssemblyFromSource(parameters, sb.ToString());
I'm getting the error:
Invalid expression term ">"
Why isn't the compiler able to compile the predicate?