I've been looking quite a bit at Mr. Skeet's blog on how to re-implement LINQ.
In particular, he states that the code:
var list = (from person in people
where person.FirstName.StartsWith("J")
orderby person.Age
select person.LastName)
.ToList();
is translated to methods that are extension methods that are provided by the LINQ library:
people.Where(person => person.FirstName.StartsWith("J"))
.OrderBy(person => person.Age)
.Select(person => person.LastName)
BY THE COMPILER.
My question is, how does one impress the bigwigs enough with a library to cause them to allow the language to change to support the library? Or were those words already reserved before LINQ came along?
Grab the Mono C# Compiler - it's open source and you can do whatever language modifications you want and which .net supports, e.g., use enums as generic constraints, create methods that return references to value types (public ref int Max(ref int x, ref int y) { if (x>y) return ref x; else return ref y; }), that have protected or internal visibility etc.
Of course, you are then creating an incompatible derivate of C#, but if you push it hard enough then people might like it.
The other option: Start a blog, come up with some really good use cases for it, possibly a sample implementation in a .net language or using a customized compiler, show what problem it solves and why this would be a big win that justifies the cost that goes into specifying, designing, developing, testing and documenting of the feature.
I highly doubt the compiler developers would implement syntax extensions for just any library. If you really wanted language-level integration, you could develop a pre-processor that transforms your custom syntax into valid C#. That's essentially what the compiler does with LINQ anyway (as you pointed out in your question).
Of course, you would lose things like auto-complete and syntax highlighting in Visual Studio, but that could be fixed with an extension.
Some languages allow to extend their syntax and semantics. The closest to C# is Nemerle (and it even supports a safe subset of C#, which you can, in turn, extend as you like), but the same can be done with almost any Lisp, for example. So, if you're using a language powerful enough, you don't need to "impress" anyone - any library can add new functionality to a language itself.
There were rumors that the next C# will provide some rudimentary metaprogramming support as well, but I could not find any specifics.
It is possible but it will be hard and they'll probably reimplement the idea to better fit their language constructs. The new async feature is similar to a library called AsyncEnumerator for example but they are building everything to better suit the language. The keywords for LINQ were not reserved in advance but they are contextual keywords meaning that there can be identifiers that match this keywords out of the LINQ context. When the compiler detects a LINQ construct it goes into LINQ mode where these keywords are actual reserved words.
Related
I know it might not be worth it but just for education purposes I want to know if there is a way to inject your own keywords to .NET languages.
For example I thought it's good to have C++ asm keyword in C#.
Remember I'm not talking about how to implement asm keyword but a general way to add keyword to C#.
My imagined code :
asm{
mov ax,1
add ax,4
}
So is there a way to achieve this ?
The answers which cover implementing keyword{ } suits enough for this question.
This isn't possible at the moment. However, there's a Microsoft project in development called Roslyn that can be summarised as "the compiler as a service." It allows you, amongst other things, to extend or modify the behaviour of the compiler through an API.
When Roslyn becomes available, I believe this should be something that (with caution!) is quite doable.
You can use whatever tools you would like to pre-process your code before sending it to the C# compiler. For example, you might use VS macros to do the pre-processing, mapping a given syntax that you invented into something that does compile into C# code, possibly generating an error if there is a problem. If VS macros aren't powerful enough for you then you can always use your own IDE that does whatever you code it to do to the text before sending it to the compiler.
There is no built in support in the compiler for specifying your own keywords/syntax; you would need to handle it entirely independent of the compiler.
Unfortunately this is not possible. You can't extend or alter the languages in any way.
You could in some obscure way use PostSharp to read and parse strings and transform them to custom code at compile time (a pre processor). But you would not get very happy with that, as it is very error prone and you won't get any kind of intellisense or code completion for your magic strings.
According to MSDN keywords are predefined and cannot be altered. So you can't add any, because you would need to tell the compiler how to handle them. Insofar, no you can't.
I think of C# language compiler as a self contained black box capable of understanding text of a certain syntax and producing compiled code. On the other hand .NET framework is a massive library that contains functionality written partly by C# and partly by C++. So .NET framework depends on C# language, not the other way around.
But I cannot fit this into how LINQ works. LINQ queries are text of a particular syntax that C# compiler can understand. But to build by own LINQ provider I need to work with interfaces like IQueryable and IQueryProvider both of which are defined in System.Linq namespace of the framework.
Does that mean a functionality C# language offers is dependent on a part of .NET framework? Does C# language know about .NET framework?
.NET Framework contains of many pieces. One of the most important is CLR — Common Language Runtime. All .NET languages depend on it, C# included, because they produce IL-code which cannot be executed by machine processor. Instead, CLR executes it.
And there is also Base Class Library, BCL, which is available to use for every .NET language: C#, VB.NET, Managed C++, F#, IronRuby, you name it. I doubt it was written in C#. It doesn't depend on any features of those languages, because classes and OOP are built in CLR.
So, yes, C# language knows about .NET framework, it absolutely must know about it. Think about IEnumerable: to compile foreach into GetEnumerator(), and MoveNext() calls, C# compiler has to know that, well, IEnumerable exists. And is somewhat special.
Or think about attributes! C# compiler has the intrinsic knowledge about what methods Attribute interface provides.
But CLR itself doesn't know anything about C#. At all.
LINQ queries are text of a particular syntax that C# compiler can understand.
Well, query expressions are - but the compiler doesn't really "understand" them. It just translates them in a pretty mechanical manner. For example, take this query:
var query = from foo in bar
where foo.X > 10
select foo.Y;
That is translated into:
var query = bar.Where(foo => foo.X > 10)
.Select(foo => foo.Y);
The compiler doesn't know anything about what Where and Select mean here. They don't even have to be methods - if you had appropriate fields or properties of delegate types, the compiler would be fine with it. Basically, if the second form will compile, so will the query expression.
Most LINQ providers use extension methods to provide these methods (Where, Select, SelectMany etc). Again, they're just part of the C# language - the compiler doesn't know or care what the extension methods do.
For more details about how query expressions are translated, see part 41 of my Edulinq blog series. You may find the rest of my Edulinq series informative, too - it's basically a series of blog posts in which I reimplement LINQ to Objects, one method at a time. Again, this demonstrates that the C# compiler doesn't rely on the LINQ implementation being in the System.Linq namespace, or anything like that.
I am trying to design a library in F#. The library should be friendly for use from both F# and C#.
And this is where I'm stuck a little bit. I can make it F# friendly, or I can make it C# friendly, but the problem is how to make it friendly for both.
Here is an example. Imagine I have the following function in F#:
let compose (f: 'T -> 'TResult) (a : 'TResult -> unit) = f >> a
This is perfectly usable from F#:
let useComposeInFsharp() =
let composite = compose (fun item -> item.ToString) (fun item -> printfn "%A" item)
composite "foo"
composite "bar"
In C#, the compose function has the following signature:
FSharpFunc<T, Unit> compose<T, TResult>(FSharpFunc<T, TResult> f, FSharpFunc<TResult, Unit> a);
But of course, I don't want FSharpFunc in the signature, what I want is Func and Action instead, like this:
Action<T> compose2<T, TResult>(Func<T, TResult> f, Action<TResult> a);
To achieve this, I can create compose2 function like this:
let compose2 (f: Func<'T, 'TResult>) (a : Action<'TResult> ) =
new Action<'T>(f.Invoke >> a.Invoke)
Now, this is perfectly usable in C#:
void UseCompose2FromCs()
{
compose2((string s) => s.ToUpper(), Console.WriteLine);
}
But now we have a problem using compose2 from F#! Now I have to wrap all standard F# funs into Func and Action, like this:
let useCompose2InFsharp() =
let f = Func<_,_>(fun item -> item.ToString())
let a = Action<_>(fun item -> printfn "%A" item)
let composite2 = compose2 f a
composite2.Invoke "foo"
composite2.Invoke "bar"
The question: How can we achieve first-class experience for the library written in F# for both F# and C# users?
So far, I couldn't come up with anything better than these two approaches:
Two separate assemblies: one targeted to F# users, and the second to C# users.
One assembly but different namespaces: one for F# users, and the second for C# users.
For the first approach, I would do something like this:
Create a F# project, call it FooBarFs and compile it into FooBarFs.dll.
Target the library purely to F# users.
Hide everything unnecessary from the .fsi files.
Create another F# project, call if FooBarCs and compile it into FooFar.dll
Reuse the first F# project at the source level.
Create .fsi file which hides everything from that project.
Create .fsi file which exposes the library in C# way, using C# idioms for name, namespaces, etc.
Create wrappers that delegate to the core library, doing the conversion where necessary.
I think the second approach with the namespaces can be confusing to the users, but then you have one assembly.
The question: None of these are ideal, perhaps I am missing some kind of compiler flag/switch/attribute
or some kind of trick and there is a better way of doing this?
The question: has anyone else tried to achieve something similar and if so how did you do it?
EDIT: to clarify, the question is not only about functions and delegates but the overall experience of a C# user with an F# library. This includes namespaces, naming conventions, idioms and suchlike that are native to C#. Basically, a C# user shouldn't be able to detect that the library was authored in F#. And vice versa, an F# user should feel like dealing with a C# library.
EDIT 2:
I can see from the answers and comments so far that my question lacks the necessary depth,
perhaps mostly due to use of only one example where interoperability issues between F# and C#
arise, the issue of function values. I think this is the most obvious example and so this
led me to use it to ask the question, but by the same token gave the impression that this is
the only issue I am concerned with.
Let me provide more concrete examples. I have read through the most excellent
F# Component Design Guidelines
document (many thanks #gradbot for this!). The guidelines in the document, if used, do address
some of the issues but not all.
The document is split into two main parts: 1) guidelines for targeting F# users; and 2) guidelines for
targeting C# users. Nowhere does it even attempt to pretend that it is possible to have a uniform
approach, which exactly echoes my question: we can target F#, we can target C#, but what is the
practical solution for targeting both?
To remind, the goal is to have a library authored in F#, and which can be used idiomatically from
both F# and C# languages.
The keyword here is idiomatic. The issue is not the general interoperability where it is just possible
to use libraries in different languages.
Now to the examples, which I take straight from
F# Component Design Guidelines.
Modules+functions (F#) vs Namespaces+Types+functions
F#: Do use namespaces or modules to contain your types and modules.
The idiomatic use is to place functions in modules, e.g.:
// library
module Foo
let bar() = ...
let zoo() = ...
// Use from F#
open Foo
bar()
zoo()
C#: Do use namespaces, types and members as the primary organizational structure for your
components (as opposed to modules), for vanilla .NET APIs.
This is incompatible with the F# guideline, and the example would need
to be re-written to fit the C# users:
[<AbstractClass; Sealed>]
type Foo =
static member bar() = ...
static member zoo() = ...
By doing so though, we break the idiomatic use from F# because
we can no longer use bar and zoo without prefixing it with Foo.
Use of tuples
F#: Do use tuples when appropriate for return values.
C#: Avoid using tuples as return values in vanilla .NET APIs.
Async
F#: Do use Async for async programming at F# API boundaries.
C#: Do expose asynchronous operations using either the .NET asynchronous programming model
(BeginFoo, EndFoo), or as methods returning .NET tasks (Task), rather than as F# Async
objects.
Use of Option
F#: Consider using option values for return types instead of raising exceptions (for F#-facing code).
Consider using the TryGetValue pattern instead of returning F# option values (option) in vanilla
.NET APIs, and prefer method overloading over taking F# option values as arguments.
Discriminated unions
F#: Do use discriminated unions as an alternative to class hierarchies for creating tree-structured data
C#: no specific guidelines for this, but the concept of discriminated unions is foreign to C#
Curried functions
F#: curried functions are idiomatic for F#
C#: Do not use currying of parameters in vanilla .NET APIs.
Checking for null values
F#: this is not idiomatic for F#
C#: Consider checking for null values on vanilla .NET API boundaries.
Use of F# types list, map, set, etc
F#: it is idiomatic to use these in F#
C#: Consider using the .NET collection interface types IEnumerable and IDictionary
for parameters and return values in vanilla .NET APIs. (i.e. do not use F# list, map, set)
Function types (the obvious one)
F#: use of F# functions as values is idiomatic for F#, obviously
C#: Do use .NET delegate types in preference to F# function types in vanilla .NET APIs.
I think these should be sufficient to demonstrate the nature of my question.
Incidentally, the guidelines also have a partial answer:
... a common implementation strategy when developing higher-order
methods for vanilla .NET libraries is to author all the implementation using F# function types, and
then create the public API using delegates as a thin façade atop the actual F# implementation.
To summarise.
There is one definite answer: there are no compiler tricks that I missed.
As per the guidelines doc, it seems that authoring for F# first and then creating
a facade wrapper for .NET is a reasonable strategy.
The question then remains regarding the practical implementation of this:
Separate assemblies? or
Different namespaces?
If my interpretation is correct, Tomas suggests that using separate namespaces should
be sufficient, and should be an acceptable solution.
I think I will agree with that given that the choice of namespaces is such that it
does not surprise or confuse the .NET/C# users, which means that the namespace
for them should probably look like it is the primary namespace for them. The
F# users will have to take the burden of choosing F#-specific namespace.
For example:
FSharp.Foo.Bar -> namespace for F# facing the library
Foo.Bar -> namespace for .NET wrapper, idiomatic for C#
Daniel already explained how to define a C#-friendly version of the F# function that you wrote, so I'll add some higher-level comments. First of all, you should read the F# Component Design Guidelines (referenced already by gradbot). This is a document that explains how to design F# and .NET libraries using F# and it should answer many of your questions.
When using F#, there are basically two kinds of libraries you can write:
F# library is designed to be used only from F#, so it's public interface is written in a functional style (using F# function types, tuples, discriminated unions etc.)
.NET library is designed to be used from any .NET language (including C# and F#) and it typically follows .NET object-oriented style. This means that you'll expose most of the functionality as classes with method (and sometimes extension methods or static methods, but mostly the code should be written in the OO design).
In your question, you're asking how to expose function composition as a .NET library, but I think that functions like your compose are too low level concepts from the .NET library point of view. You can expose them as methods working with Func and Action, but that probably isn't how you would design a normal .NET library in the first place (perhaps you'd use the Builder pattern instead or something like that).
In some cases (i.e. when designing numerical libraries that do not really fit well with the .NET library style), it makes a good sense to design a library that mixes both F# and .NET styles in a single library. The best way to do this is to have normal F# (or normal .NET) API and then provide wrappers for natural use in the other style. The wrappers can be in a separate namespace (like MyLibrary.FSharp and MyLibrary).
In your example, you could leave the F# implementation in MyLibrary.FSharp and then add .NET (C#-friendly) wrappers (similar to code that Daniel posted) in the MyLibrary namespace as static method of some class. But again, .NET library would probably have more specific API than function composition.
You only have to wrap function values (partially-applied functions, etc) with Func or Action, the rest are converted automatically. For example:
type A(arg) =
member x.Invoke(f: Func<_,_>) = f.Invoke(arg)
let a = A(1)
a.Invoke(fun i -> i + 1)
So it makes sense to use Func/Action where applicable. Does this eliminate your concerns? I think your proposed solutions are overly-complicated. You can write your entire library in F# and use it pain-free from F# and C# (I do it all the time).
Also, F# is more flexible than C# in terms of interoperability so it's generally best to follow traditional .NET style when this is a concern.
EDIT
The work required to make two public interfaces in separate namespaces, I think, is only warranted when they are complementary or the F# functionality is not usable from C# (such as inlined functions, which depend on F#-specific metadata).
Taking your points in turn:
Module + let bindings and constructor-less type + static members appear exactly the same in C#, so go with modules if you can. You can use CompiledNameAttribute to give members C#-friendly names.
I may be wrong, but my guess is that the Component Guidelines were written prior to System.Tuple being added to the framework. (In earlier versions F# defined it's own tuple type.) It's since become more acceptable to use Tuple in a public interface for trivial types.
This is where I think you have do things the C# way because F# plays well with Task but C# doesn't play well with Async. You can use async internally then call Async.StartAsTask before returning from a public method.
Embrace of null may be the single biggest drawback when developing an API for use from C#. In the past, I tried all kinds of tricks to avoid considering null in internal F# code but, in the end, it was best to mark types with public constructors with [<AllowNullLiteral>] and check args for null. It's no worse than C# in this respect.
Discriminated unions are generally compiled to class hierarchies but always have a relatively friendly representation in C#. I would say, mark them with [<AllowNullLiteral>] and use them.
Curried functions produce function values, which shouldn't be used.
I found it was better to embrace null than to depend on it being caught at the public interface and ignore it internally. YMMV.
It makes a lot of sense to use list/map/set internally. They can all be exposed through the public interface as IEnumerable<_>. Also, seq, dict, and Seq.readonly are frequently useful.
See #6.
Which strategy you take depends on the type and size of your library but, in my experience, finding the sweet spot between F# and C# required less work—in the long run—than creating separate APIs.
Although it probably would be an overkill, you could consider writing an application using Mono.Cecil (it has awesome support on the mailing list) that would automate the conversion on the IL level. For example, you implement your assembly in F#, using the F#-style public API, then the tool would generate a C#-friendly wrapper over it.
For instance, in F# you would obviously use option<'T> (None, specifically) instead of using null like in C#. Writing a wrapper generator for this scenario should be fairly easy: the wrapper method would invoke the original method: if it's return value was Some x, then return x, otherwise return null.
You would need to handle the case when T is a value type, i.e. non-nullable; you would have to wrap the return value of the wrapper method into Nullable<T>, which makes it a bit painful.
Again, I'm quite certain that it would pay off to write such a tool in your scenario, maybe except if you'll be working on this such library (usable seamlessly from F# and C# both) regularly. In any case, I think it would be an interesting experiment, one that I might even explore sometime.
Draft F# Component Design Guidelines
(August 2010)
Overview This document looks at some of the issues related to F# component design and coding. In particular, it covers:
Guidelines for designing “vanilla” .NET libraries for use from any .NET language.
Guidelines for F#-to-F# libraries and F# implementation code.
Suggestions on coding conventions for F# implementation code
Is there a way, and how would I go about implementing my own keyword such as in, and as (etc), to be used in my code?
Here is what I had in mind. I want to (just for my own personal reasons, I guess) add a few keywords of my own, one of which would be the "was" keyword:
if(Control was Clicked)
{
// etc etc
}
No.
The closest you could get would be an extension method:
Control.WasClicked()
No, you can't add to the C# language, short of writing your own compiler.
However, your "was" keyword makes me think you might be looking for a way to declaratively handle events. Microsoft have a library called "Reactive Extensions for .NET" (Rx) that is an extension to LINQ that allows you to deal with events in a declarative fashion.
No, you cannot add keywords to C# - at least not without writing a compiler for yourself. If what you want to do is simple, however, perhaps you could do it using a custom preprocessor. You would lose some syntax highlighting and error checking in Visual Studio, though.
One language for the CLR, that is designed to be extensible like that, is Boo.
This is a question for anyone who has the pleasure to work in both Java and C#.
Do you find that you have to make a mental context switch of some kind when you move from one to the other?
I'm working in both at the moment and because the syntax and libraries are so similar and yet subtly different I'm finding it frustrating when i move from one to the other.
This is more so than I've experienced moving between any other programming languages.
Does anyone have any tips for making your brain work differently for languages that are so similar?
Yes, I have to make a mental context switch - because LINQ isn't available in Java :(
Beyond that, there are little things like foreach (X x in y) vs for (X x : y) which often trip me up, but not a huge amount otherwise.
As for a tip to make your brain work differently: don't give into the temptation to use the naming conventions of the other language. I find that using camel-cased method names in Java is a good nudge to my brain, for example.
EDIT: The differences in terms of generics are important and can occasionally be brain-warping, too.
I find that most difficult switches are around generics (which C# does better), delegates (which Java sort of hacks with anonymous inner classes) and the events (which Java doesn't have)
I find that making sure you are using the idioms of the language (i.e. using delegates instead of creating an interface in C#) will help snap your mind into gear.
The langauge differences as others have mentioned is the most obvious and problematic. I too find the library differences maddening at times and really wasting a lot of time.
The "details" can really get you as well, such as understanding how the memory model works or the way your runtime performs optimizations. Having intimate knowledge of the runtime environment's memory model, garbage collection techniques, threading model, etc can create significant changes in the way you think and develop software.
I cannot intelligently compare Java vs C#'s details at that level, but I can say that many things I would do in Java, I am uncomfortable or unsure if I can do in C# because I don't understand its low-level details. This affects code I write for everything from GUI-worker interaction to dealing with memory management.
I have found the best way to cope with the differences in Java and C# is to simply think of them as complete different languages--avoid the trap of "C# and Java are basicaly the same language with different class names".
The difference in how string equality comparisons work between Java and C# is one thing that needs to be kept in mind, particularly when moving from working in C# to working in Java.
In C# you can do a value comparison on two string instances with the == operator:
// C# string value comparison example
string string1 = GetStringValue1();
string string2 = GetStringValue2();
// Check to see whether the string values are equal
if (string1 == string2)
{
// Do something...
}
In Java, the == operator does a reference (not value) comparison on strings, so you need to use the equals() method:
// Java string value comparison example
String string1 = getStringValue1();
String string2 = getStringValue2();
// Check to see whether the string values are equal
if (string1.equals(string2))
{
// Do something...
}
Bottom line: When comparing strings in Java, don't forget that operator == doesn't do a value equality comparison, like it does when comparing strings in C#. (The same concept also applies to operator !=.)
They are similar enough that the differences trip me up. I learned Java in college and starting using C# when I got my first programming job.
C# in .Net 1.1 seemed a lot more similar, but now in C# 3.0, I use delegates, generics and especially API constructs like Dictionaries, way more than I ever did in Java. I've used the functional aspects of C# more and more as I work my way through learning F# (now writing in that or OCaml really forces me to change my way of thinking).