I wrote some queries in C# using LINQ. After a while, I started using Haskell a little bit, which is a functional programming language (a not so popular one), and for me it seems that both of them are almost the same thing. But I am unsure about this. Please, if someone has used them more than me, could they tell me if they are almost the same thing regarding principles in programming ?
Also, could LINQ be considered functional programming ?
Thanks.
for me it seems that both of them are almost the same thing. But I am unsure about this. Please, if someone has used them more than me, could they tell me if they are almost the same thing regarding principles in programming ?
Yes, the design of LINQ query comprehensions was heavily influenced by the design of Haskell. Haskell expert Erik Meijer was on the C# language design committee when we designed LINQ; his insights were very valuable. (I joined the design team at the end of this process, so unfortunately I did not get to participate in all the interesting twists and turns the design went through over the years; it started being much more traditional OO than it ended up!)
If you've recently done a serious exploration into Haskell then you're probably becoming familiar with the idea of a monad. The LINQ syntax is designed specifically to make operations on the sequence monad feel natural, but in fact the implementation is more general; what C# calls "SelectMany" is a slightly modified form of the "Bind" operation on an arbitrary monad. You can actually use query comprehension with any monad, as my colleague Wes describes here, but doing so looks pretty weird and I recommend against it in production code.
Also, could LINQ be considered functional programming ?
Yes, LINQ is heavily influenced by ideas from functional programming. It is designed to treat functions as first-class objects, to emphasize calculation over side effects, and so on.
It may be worthwhile to take a look at Erik Meijer Lecture. Here's the description for it.
We kick off C9 Lectures with a journey
into the world of Functional
Programming with functional language
purist and high priest of the lambda
calculus, Dr. Erik Meijer (you can
thank Erik for many of the functional
constructs that have shown up in
languages like C# and VB.NET. When you
use LINQ, thank Erik in addition to
Anders).
It is true that C# has gained some aspects of functional programming. First and foremost is the lambda statement, implemented as an anonymous delegate. You now no longer need to define a method as belonging to an object, and you can define a method in a similar fashion as any other variable. This allows for many functional-type constructs:
var mult = (a,b)=>a*b;
var square = (a)=>Math.Pow(a,2);
var multandsquare = (a,b)=>square(mult(a,b));
//None of the above give a lick about what a and b really are, until...
multandsquare(5,3); //== 225
The basic paradigm of functional programming - that basically anything you can tell a computer to do can be told in terms of higher-order functions - can be applied to C# programs, especially now that C# actually has higher-order functions. The compiler will force you to have at least one class with at least one public main method (that's just how OO works), but from that point you can define pretty much everything only in terms of functions (def: a subclass of "methods" that take N parameters and produce 1 output without "side effects") instantiated as lambdas.
Linq does have some functional structure. Basically its method-chain paradigm is an example of monadic processing, which is how sequences of operations are structured through operation encapsulation in functional languages (and now in C#). Calling a Linq method on an IEnumerable returns another IEnumerable, which is really a different concrete class which contains a handle to the source Enumerable and some lambda to perform. You can replicate it in a functional language very simply; it's a tuple of the source (itself a tuple of the current element and everything else) and a function to perform that transforms the source tuple into the result, one element at a time (which in Linq, as it would be implemented in a functional language, is a nesting of some operation into a defined "rollup" function). Call a method that must produce an actual answer (a concrete typed result), and all these functions are evaluated sufficiently to produce the desired result.
Also could LINQ be considered
functional programming?
LINQ implies a functional style in that it resembles, for instance, a SQL select, which is read-only in principle. However, because the LINQ clauses in .NET are implemented by plain-old CLR routines, there is nothing to prevent LINQ expressions from modifying state.
Haskell, on the other hand, is a "pure" functional language, so expressions cannot modify global state. Although it is possible to perform IO and graphics operations, these are carried out using a very different idiom from anything in .NET — including F#, which lets you drop into a procedural style more or less transparently.
Related
In most programming languages that have control-flow keywords, each control-flow keyword has one clear and unambiguous meaning. And then there's Ruby, in which the meaning of return is contextual: it does two very different things depending on whether or not you're inside of some (but not all!) types of anonymous functions. I'm not familiar with any other language with similar semantics.
That being true, if I have some Ruby code that's reasonably well written, and I want to translate it into another language, (let's say C#, because it's pretty popular around here,) how would I handle the "special" returns in order to preserve the original behavior?
The closest thing I can think of is to turn a "special" return into a throw new RubyNestedReturnException(<return value here>) and put all invocations in a try/catch block that catches RubyNestedReturnException and returns the value inside. This would probably work, but it would be very messy. Is there any better way to do it? (And no, "just turn all invocations that use a proc with "special" returns into a return statement" is not a good answer, since some invocations involve a proc that gets passed in as an argument from outside the function that invokes them.)
C# has no direct equivalent. The try..catch hack may work on the surface, but you'll be better served translating the code to the language of your choice.
I mean no self respecting programmer would ever write a plain for i=1 to 10 (or equivalent) in Python, no matter what other languages say. It's simply not the "Pythonic" way. C# has more traditional roots (C++), so writing C#-like code should be relatively straight forward.
If you are interested, both C (setjmp/longjmp) and Java (break label;) have something similar, and both are (generally) viewed as code smell generators. I suggest avoiding it altogether and rewriting the code using more traditional control flow operations.
As a follow up to this question What are the advantages of built-in immutability of F# over C#?--am I correct in assuming that the F# compiler can make certain optimizations knowing that it's dealing with largely immutable code? I mean even if a developer writes "Functional C#" the compiler wouldn't know all of the immutability that the developer had tried to code in so that it couldn't make the same optimizations, right?
In general would the compiler of a functional language be able to make optimizations that would not be possible with an imperative language--even one written with as much immutability as possible?
Am I correct in assuming that the F# compiler can make certain
optimizations knowing that it's dealing with largely immutable code?
Unfortunately not. To a compiler writer, there's a huge difference between "largely immutable" and "immutable". Even guaranteed immutability is not that important to the optimizer; the main thing that it buys you is you can write a very aggressive inliner.
In general would the compiler of a functional language be able to make optimizations that would not be possible with an imperative language--even one written with as much immutability as possible?
Yes, but it's mostly a question of being able to apply the classic optimizations more easily, in more places. For example, immutability makes it much easier to apply common-subexpression elimination because immutability can guarantee you that contents of certain memory cells are not changed.
On the other hand, if your functional language is not just immutable but pure (no side effects like I/O), then you enable a new class of optimizations that involve rewriting source-level expressions to more efficient expressions. One of the most important and more interesting to read about is short-cut deforestation, which is a way to avoid allocating memory space for intermediate results. A good example to read about is stream fusion.
If you are compiling a statically typed, functional language for high performance, here are some of the main points of emphasis:
Use memory effectively. When you can, work with "unboxed" values, avoiding allocation and an extra level of indirection to the heap. Stream fusion in particular and other deforestation techniques are all very effective because they eliminate allocations.
Have a super-fast allocator, and amortize heap-exhaustion checks over multiple allocations.
Inline functions effectively. Especially, inline small functions across module boundaries.
Represent first-class functions efficiently, usually through closure conversion. Handle partially applied functions efficiently.
Don't overlook the classic scalar and loop optimizations. They made a huge difference to compilers like TIL and Objective Caml.
If you have a lazy functional language like Haskell or Clean, there are also a lot of specialized things to do with thunks.
Footnotes:
One interesting option you get with total immutability is more ability to execute very fine-grained parallelism. The end of this story has yet to be told.
Writing a good compiler for F# is harder than writing a typical compiler (if there is such a thing) because F# is so heavily constrained: it must do the functional things well, but it must also work effectively within the .NET framework, which was not designed with functional languages in mind. We owe a tip of the hat to Don Syme and his team for doing such a great job on a heavily constrained problem.
No.
The F# compiler makes no attempt to analyze the referential transparency of a method or lambda. The .NET BCL is simply not designed for this.
The F# language specification does reserve the keyword 'pure', so manually marking a method as pure may be possible in vNext, allowing more aggressive graph reduction of lambda-expressions.
However, if you use the either record or algebraic types, F# will create default comparison and equality operators, and provide copy semantics. Amongst many other benefits (pattern-matching, closed-world assumption) this reduces a significant burden!
Yes, if you don't consider F#, but consider Haskell for instance. The fact that there are no side effects really opens up a lot of possibilities for optimization.
For instance consider in a C like language:
int factorial(int n) {
if (n <= 0) return 1;
return n* factorial(n-1);
}
int factorialuser(int m) {
return factorial(m) * factorial(m);
}
If a corresponding method was written in Haskell, there would be no second call to factorial when you call factorialuser. It might be possible to do this in C#, but I doubt the current compilers do it, even for a simple example as this. As things get more complicated, it would be hard for C# compilers to optimize to the level Haskell can do.
Note, F# is not really a "pure" functional language, currently. So, I brought in Haskell (which is great!).
Unfortunately, because F# is only mostly pure there aren't really that many opportunities for aggressive optimization. In fact, there are some places where F# "pessimizes" code compared to C# (e.g. making defensive copies of structs to prevent observable mutation). On the bright side, the compiler does a good job overall despite this, providing comparable performace to C# in most places nonetheless while simultaneously making programs easier to reason about.
I would say largely 'no'.
The main 'optimization' advantages you get from immutability or referential transparency are things like the ability to do 'common subexpression elimination' when you see code like ...f(x)...f(x).... But such analysis is hard to do without very precise information, and since F# runs on the .Net runtime and .Net has no way to mark methods as pure (effect-free), it requires a ton of built-in information and analysis to even try to do any of this.
On the other hand, in a language like Haskell (which mostly means 'Haskell', as there are few languages 'like Haskell' that anyone has heard of or uses :)) that is lazy and pure, the analysis is simpler (everything is pure, go nuts).
That said, such 'optimizations' can often interact badly with other useful aspects of the system (performance predictability, debugging, ...).
There are often stories of "a sufficiently smart compiler could do X", but my opinion is that the "sufficiently smart compiler" is, and always will be, a myth. If you want fast code, then write fast code; the compiler is not going to save you. If you want common subexpression elimination, then create a local variable (do it yourself).
This is mostly my opinion, and you're welcome to downvote or disagree (indeed I've heard 'multicore' suggested as a rising reason that potentially 'optimization may get sexy again', which sounds plausible on the face of it). But if you're ever hopeful about any compiler doing any non-trivial optimization (that is not supported by annotations in the source code), then be prepared to wait a long, long time for your hopes to be fulfilled.
Don't get me wrong - immutability is good, and is likely to help you write 'fast' code in many situations. But not because the compiler optimizes it - rather, because the code is easy to write, debug, get correct, parallelize, profile, and decide which are the most important bottlenecks to spend time on (possibly rewriting them mutably). If you want efficient code, use a development process that let you develop, test, and profile quickly.
Additional optimizations for functional languages are sometimes possible, but not necessarily because of immutability. Internally, many compilers will convert code into an SSA (single static assignment) form, where each local variable inside a function can only be assigned once. This can be done for both imperative and functional languages. For instance:
x := x + 1
y := x + 4
can become
x_1 := x_0 + 1
y := x_1 + 4
where x_0 and x_1 are different variable names. This vastly simplifies many transformations, since you can move bits of code around without worrying about what value they have at specific points in the program. This doesn't work for values stored in memory though (i.e., globals, heap values, arrays, etc). Again, this is done for both functional and imperative languages.
One benefit most functional languages provide is a strong type system. This allows the compiler to make assumptions that it wouldn't be able to otherwise. For instance, if you have two references of different types, the compiler knows that they cannot alias (point to the same thing). This is not an assumption a C compiler could ever make.
Let me state up front that I have an infantile understanding of Monads. I have read the various threads on Monads here and have done a few hours of study on the concept. I hardly feel comfortable with the term, but I think it is safe to say that I generally understand what a Monad is/does.
I'm a C# developer who is looking to improve the way I work. What would help me further in my Monaducation is see a real world application of a Monad in C# (i.e. via a linq SelectMany() or somesuch) that is clearly an improvement over other ways of solving the same sort of problem in oldskool C#.
Has anyone seen such a beast?
Here is one such scenario: you want to author a parsing library (a nice example of an embedded DSL), and you discover that the best ones are monadic parser combinator libraries. So you write it leveraging LINQ syntax sugars to author C# code that has the same structure as the grammar of the language you're parsing, and you get the benefits of an awesome programming model for on-the-fly semantic analysis and error-recovery. See this blog for a description.
Find Pythagorean triples:
var r = from a in Enumerable.Range(1, 25)
from b in Enumerable.Range(a, 25-a)
from c in Enumerable.Range(b, 25-b)
where a*a + b*b == c*c
select new [] { a, b, c };
Here is one such scenario: you want to write code that makes sequential async calls (e.g. IO) without holding threads, but you don't want to write the hopeless tangle of spaghetti that the async programming model (BeginFoo/EndFoo) forces you into. So you can use a monad and LINQ sugars and write code that looks straight-line but it releases/switches threads throughout. See this blog for a short description.
check out http://memoirsofaprogrammer.blogspot.com
One example is simplifying null checks using the Maybe monad as shown in this article.
LINQ is used in many solutions (and often requested in questions) here on StackOverflow. Review questions with the LINQ tag and you will see real world usage.
Programming with monads is declarative, describing what you want at a high level rather than the low-level details of how to generate it.
See the exercises at the end of Brian Beckman's state-monad talk on Channel 9.
I recently blogged about refactoring a typical imperative real-world C# code (a function in NuGet) to functional, monadic style (more concretely, using the Maybe monad). I did my best to do it in little steps, explaining the rational behind step, so I think it helps in understanding how monads are useful.
Is there any C#/F# performance comparison available on web to show proper usage of new F# language?
Natural F# code (e.g. functional/immutable) is slower than natural (imperative/mutable object-oriented) C# code. However, this kind of F# is much shorter than usual C# code.
Obviously, there is a trade-off.
On the other hand, you can, in most cases, achieve performance of F# code equal to performance of C# code. This will usually require coding in imperative or mutable object-oriented style, profile and remove bottlenecks. You use that same tools that you would otherwise use in C#: e.g. .Net reflector and a profiler.
That having said, it pays to be aware of some high-productivity constructs in F# that decrease performance. In my experience I have seen the following cases:
references (vs. class instance variables), only in code executed billions of times
F# comparison (<=) vs. System.Collections.Generic.Comparer, for example in binary search or sort
tail calls -- only in certain cases that cannot be optimized by the compiler or .Net runtime. As noted in the comments, depends on the .Net runtime.
F# sequences are twice slower than LINQ. This is due to references and the use of functions in F# library to implement translation of seq<_>. This is easily fixable, as you might replace the Seq module, by one with same signatures that uses Linq, PLinq or DryadLinq.
Tuples, F# tuple is a class sorted on the heap. In some case, e.g. a int*int tuple it might pay to use a struct.
Allocations, it's worth remembering that a closure is a class, created with the new operator, which remembers the accessed variables. It might be worth to "lift" the closure out, or replaced it with a function that explicitly takes the accessed variables as arguments.
Try using inline to improve performance, especially for generic code.
My experience is to code in F# first and optimize only the parts that matter. In certain cases, it might be easier to write the slow functions in C# rather that to try to tweak F#. However, from programmer efficiency point of view makes sense to start/prototype in F# then profile, disassemble and optimize.
Bottom line is, your F# code might end-up slower than C# because of program design decisions, but ultimately efficiency can be obtained.
See these questions that I asked recently:
Is a program F# any more efficient (execution-wise) than C#?
How can I use functional programming in the real world?
Is it possible that F# will be optimized more than other .Net languages in the future?
Here are a few links on (or related to) this topic:
http://cs.hubfs.net/forums/thread/3207.aspx
http://strangelights.com/blog/archive/2007/06/17/1588.aspx
http://khigia.wordpress.com/2008/03/30/ocaml-vs-f-for-big-integer-surprising-performance-test/
http://cs.hubfs.net/blogs/f_team/archive/2006/08/15/506.aspx
http://blogs.msdn.com/jomo_fisher/
What I seem to remember from another post on Robert Pickering's blog (or was it Scott Hanselman?) that in the end, because both are sitting on the same framework, you can get the same performance from both, but you sometimes have to 'twist' the natural expression of the language to do so. In the example I recall, he had to twist F# to get comparable performance with C#...
I know that in C# 3.0 you can do some functional programming magic with Linq and lambda expression and all that stuff. However, is it really possible to go completely "pure" functional in C#? By "pure" I mean having methods that are pure (always gives the same output for the same input) and completely free of side-effects. How do we get around the fact that we do not even have immutable integer type in C#?
If you want to program in a pure functional way, there is nothing stopping you.
On the other hand, if you have some program, there is no magic flag you can flip to force the program to behave in a pure functional way.
For ints (immutable)
If you use an int as a parameter, it is passed by value. Any changes are not propogated to the caller.
If you use an int declared in one method's scope in a closure within the method, than that int variable is shared. In this case, one must either pledge not to modify the int (programmer enforced), or simply not use an int in this way.
And if you truly need an immutable int, have you seen the readonly keyword?
Have you looked into F#? It seems much more along the lines of what you are talking about. C# just really isn't designed with functional programming in mind, and therefore won't really give you any of the benefits that are normally associated with a functional language.
Unfortunately no - C# is not a pure functional language and it does not intend to be. What has happened is that the C# team has seen that there are benefits to adding certain functionally-styled constructs and syntax to the language.
Functional purity is better found in other places (Lisp derivatives like Common Lisp and Scheme are good places to start).