We Keep Coding

Improving a big hierarchy tree

I have been working on a project where I need to interface and show a hierarchy from a big database (~100.000 folders, even more files), but it takes 5 minutes to be able to load the data from the database to the project, and I want to reduce it.
My implementation used a tree to represent this data (because it made sense at the beginning), using recursion to navigate down the tree to place new folders and items (which right now are not being used, since I tried to add them but it took so long to populate the tree I just removed them). Are there better structures to hold such hierarchy? Otherwise, is there a better way to transverse the tree (since it seems that most of the time is spent climbing down and up the tree, due to recursion and the fact the database is rather deep)?
I am using C#, but I believe an agnostic answer would be good enough.

You don't need to load everything because you can not see them all in your monitor anyway, you can use:
Lazy loading, load only if user want to, for example if user double click a certain folder
Using Virtualization User Interface or showing only the visible folder
For user interface virtualization example you can read an watch the Video:
Code Canvas

When to use mvvmcross data bindings?

I'm pretty new to mvvmcross, and have started to databind pretty much everything I display on the screen. Although it's usually not a lot of extra code, there is a bit of extra effort involved, and it certainly makes some things more cumbersome (say, when you have some conditional rules about screen layout that depend on the data itself).
There are quite a few cases in an app I'm implementing where data really is mostly static. Like, for example, when showing the times a restaurant is open at. That data simply won't change, and it's known at the time we initialize the screen. Because I'm somewhat new to the pattern I've been kind of just blindly binding stuff everywhere.
Now that I've done this quite a bit, I'm reflecting on how silly it is in certain situation to do the databinding at all, for what is essentially "read only" data that won't change.
I'm thinking I already know the answer here, but wanted to see what people think, overall. Are there reasons that are not obvious, where it's "better" to just data bind everything?

Data binding everything is beneficial because it keeps your code consistent regardless of whether it is a one-way or two-way binding. There is no performance penalty for having a one-way binding.
You can also use data binding for hiding/showing, enabling/disabling, etc. elements in your layout. You can create ValueConverters to manipulate any property of your View.
The more logic you move out of your View and into your ViewModel, the more portable your code will be. Data binding is the bridge between your View and ViewModel.
Given all that, ultimately the only true answer is "It depends." Each app is different, and you'll need to determine what is the best way for your situation.
Hope this helps.

Build separate pages or not

I have about 20 grid views that I have to create. All of them are pretty standard across the board. Just take IEnumerable T and display it in a grid view, that's it.
I would prefer to create one aspx page and have the grid view be dynamically generated by using ITemplate. And I guess for the data source use IEnumerable Object.
Are there significant performance considerations between doing it the way I'd like to do it or would it be better to go ahead and build the 20 or more grid views on separate aspx pages?
An example of a concern I have is taking List T and casting to IEnumerable T where T is type Object.

Build just the one and performance test. It will be easier to apply lessons learned.
If the data is long, turn buffering off to improve time to first byte.

Having one generic view page is preferable - where it is possible which it sounds like it is in your case.
Secondly no performance hit going from List to IEnumerable as IEnumerable is a behaviour which List has.
However you will get a performance hit building a List if you don't already have it - it is much better to ensure that you are passing IEnumerable from the LINQ statements directly as it is only realized when used - which can have major benefit with long lists and when using sorting or filtering (because you can modify the IEnumerable before it is realized)
As with anything related to performance build it and profile it to see if performance is an issue. No amount of opinion, however well informed is a substitute for profiling and only optimising when necessary, always avoid premature optimisation.

20 grid views, Ok
Just make sure you disable the ViewState of the controls that do not require it.
That will considerably reduce you page size & in turn reduce the page load time.

If there is nothing custom and you have to only show default data for all 20 tables/lists then i think you should use one page

How do I avoid 'Binding to Large CLR Objects'?

This guide on optimizing DataBinding says:
There is a significant performance impact when you data bind to a single CLR object with thousands of properties. You can minimize this impact by dividing the single object into multiple CLR objects with fewer properties.
What does this mean? I am still trying to get familiar with DataBinding, but my analogy here is that properties are like SQL table fields, and objects are rows. This advice then translates to "to avoid problems with a large number of fields, use less fields and create more rows". As this doesn't make any sense to me, possibly my understanding of databinding is completely askew?
Does this advice actually apply? I am unsure if it is specific to .NET 4/WPF, while i am using 3.5 and a custom WinForms based control library (DevExpress)
As an aside: am I correct in thinking DataBinding uses reflection when an using IList style datasource?
This is not just a academic question. I am currently trying to speed up loading a XtraGridView (DevExpress Control) with ~100,000 objects with 50 properties or so.

This advice then translates to "to avoid problems with a large number of fields, use less fields and create more rows"
I think it should translate to "use less fields and create smaller tables" (i.e. with less fields). And the original advice should read "[...]dividing the single class into multiple classes", with fewer properties. As you correctly noted, it wouldn't make sense to create more "rows"...
Anyway, if you do have a class that exposes hundreds or thousands of properties, you have a far more serious problem than binding performance... This is a serious design flaw that you should fix after reading some OO principles.
Does this advice actually apply? I am unsure if it is specific to .NET 4/WPF, while i am using 3.5 and a custom WinForms based control library (DevExpress)
Well, the page you mentioned is about WPF, but I think the idea of binding to smaller objects can apply to WinForms too (because the more properties need to be watched, the slower it will be)
As an aside: am I correct in thinking DataBinding uses reflection when an using IList style datasource?
You're partially correct... it actually uses TypeDescriptor, which in turns uses reflection to examine regular CLR objects. But this mechanism is much more flexible than reflection: a type can implement ICustomTypeDescriptor to provide its own description, list of members, etc (DataTable is one example of such a type)

You are solving the wrong problem. It will take a typical user well over a week to find back what she is looking for when she's got 5 million fields to search through. The speed of your UI becomes irrelevant. Only a machine can do a better job finding the data back.
You've got one. Help the user narrow down what she is searching for by letting her enter search terms so that the total query result doesn't contain more than, say, a hundred rows. The dbase engine helps you make that fast. And it automatically solves your grid perf problem.

Keeping your main form class short best practice

I have a main form at present it has a tab control and 3 data grids (DevExpress xtragrid's). Along with the normal buttons combo boxes... I would say 2/3 rds of the methods in the main form are related to customizing the grids or their relevant event handlers to handle data input. This is making the main forms code become larger and larger.
What is an ok sort of code length for a main form?
How should I move around the code if necesary? I am currently thinking about creating a user control for each grid and dumping it's methods in there.

I build a fair number of apps at my shop and try to avoid, as a general rule, to clog up main forms with a bunch of control-specific code. Rather, I'll encapsulate behaviors and state setup into some commonly reusable user controls and stick that stuff in the user controls' files instead.
I don't have a magic number I shoot for in the main form, instead I'll use the 'Why would I put this here?' test. If I can't come up with a good reason as to why I'm thinking of putting the code in the main form, I'll avoid it. Otherwise, as you've mentioned, the main form starts growing and it becomes a real pain to manage everything.
I like to put my glue code (event handler stuff, etc.) separate from the main form itself.
At a minimum, I'll utilize some regions to separate the code out into logically grouped chunks. Granted, many folks hate the #region/#endregion constructs, but I've got the keystrokes pretty much all memorized so it isn't an issue for me. I like to use them simply because it organizes things nicely and collapses down well in VS.
In a nutshell, I don't put anything in the main form unless I convince myself it belongs there. There are a bunch of good patterns out there that, when employed, help to avoid the big heaping pile that otherwise tends to develop. I looked back at one file I had early on in my career and the darn thing was 10K lines long... absolutely ridiculous!
Anyway, that is my two cents.
Have a good one!

As with any class, having more than about 150 lines is a sign that something has gone horribly wrong. The same OO principles apply to classes relating to UI as everywhere else in your application.
The class should have a single responsibility.

A number is hard to come up with. In general, I agree with the previous two posters, it's all about responsibilities.
Ask yourself, what does the code to customize behavior for grid 1 have to do with grid 2?
What is the responsibility of the main form? Recently I have been subscribing to MVP design patterns (MVC is fine as well). In this pattern, you main form is the presenter, or ui layer. It's responsibility should be to present data and accept user input.
Without seeing your code, I can only guesstimate as to what is the best course of action. But I agree with your feelings that each grid and it's customization code should live in a different control. Perhaps the responsibility of the main form should be to merely pass the data to the correct control and pass requests from the control back to the controller/presenter. It is the responsibility of each control to understand the data passed to it, and display it accordingly.
Attached is an example MVP implementation.
http://www.c-sharpcorner.com/UploadFile/rmcochran/PassiveView01262008091652AM/PassiveView.aspx