I am having one windows form which is dynamic form,used in many forms.
In this form i have to bind data from dataset to datagridview.But takes 3-4 minutes to bind data from dataset to datagridview.Result from query executed is placed in dataset.And then this dataset is then bind to datagridview as
dgSearch.DataSource=ds.tables[0];
but this statement takes time to execute.As my dataset has more than 100k rows. How to bind such large amount of data to datagridview?
I have a C# windows application.
First, the idea of binding that much data in an application suggests either a) bad architecture or b) bad business requirements. People cannot process 100,000 rows at one time. If this is a report, then the grid is not your best way of handling this.
If you are forced to head this direction, I would consider some form of paging and only show a portion of the "gridded" data at any one time. http://www.codeproject.com/Articles/16303/DataGrid-Paging-C-Windows-Forms
Heading this direction, you can control the amount of data that is bound at any one time. You can also implement sort on different columns so the user can refine what he or she is looking at. But you will dramatically speed up binding, as you are not binding 100K+ rows at a time.
If you have large amount of records, the best way to show the records is that you have paging in your gridview.
Effective paging is important for applications that handle large number of records to build scalable applications.
Here is the good article for that. It shows you how to do it on ASP.NET web forms, but when you find out the way of that you can do it on windows forms application.
Also there is another question may help you with that. What is the fastest way to load a big data set into a GridView? It's for windows forms application. You may read this too for solving your problem.
Hope it helps.
Related
I am reading a long text file with currently round about 900.000 lines (log files). I am then filling a DataTable object with the data and until then everything is fine. But when assigning the huge DataTable object to the DataGridView.DataSource it takes ~ 10 minutes until the application is responsive again and the DataGridView shows the data. The same happens if I load the data to the DataGridView directly without a DataTable object. Is there a better way to work with this huge amount of data and a DataGridView?
Yes; you need to enable "virtual mode". This isn't entirely trivial, as you need to provide the code to provide cell values on-demand (rather than filling everything in advance), but it isn't horrendous either. Here's a complete walkthrough in the Microsoft docs.
However, from a UX perspective, maybe a better solution is to make it such that you don't need to display nearly a million lines in a grid. That isn't a useful user experience, in most cases.
I'm trying to build a product catalog application in ASP.NET and C# that will allow a user to select product attributes from a series of drop-down menus, with a list of relevant products appearing in a gridview.
On page load, the options for each of the drop-downs are queried from the database, as well as the entire product catalog for the gridview. Currently this catalog stands at over 6000 items, but we're looking at perhaps five or six times that when the application goes live.
The query that pulls this catalog runs in less than a second when executed in SQL Server Management Studio, but takes upwards of ten seconds to render on the web page. We've refined the query as much as we know how: pulling only the columns that will show in our gridview (as opposed to saying select * from ...) and adding the with (nolock) command to the query to pull data without waiting for updates, but it's still too slow.
I've looked into SqlCacheDependency, but all the directions I can find assume I'm using a SqlDataSource object. I can't do this because every time the user makes a selection from the menu, a new query is constructed and sent to the database to refine the list of displayed products.
I'm out of my depth here, so I'm hoping someone can offer some insight. Please let me know if you need further information, and I'll update as I can.
EDIT: FYI, paging is not an option here. The people I'm building this for are standing firm on that point. The best I can do is wrap the gridview in a div with overflow: auto set in the CSS.
The tables I'm dealing with aren't going to update more than once every few months, if that; is there any way to cache this information client-side and work with it that way?
Most of your solution will come in a few forms (none of which have to do with a Gridview):
Good indexes. Create good indexes for the tables that pull this data; good indexes are defined as:
Indexes that store as little information as actually needed to display the product. The smaller the amount of data stored, the greater amount of data can be stored per 8K page in SQL Server.
Covering indexes: Your SQL Query should match exactly what you need (not SELECT *) and your index should be built to cover that query (hence why it's called a 'covering index')
Good table structure: this goes along with the index. The fewer joins needed to pull the information, the faster you can pull it.
Paging. You shouldn't ever pull all 6000+ objects at once -- what user can view 6000 objects at once? Even if a theoretical superhuman could process that much data; that's never going to be your median usecase. Pull 50 or so at a time (if you really even need that many) or structure your site such that you're always pulling what's relevant to the user, instead of everything (keep in mind this is not a trivial problem to solve)
The beautiful part of paging is that your clients don't even need to know you've implemented paging. One such technique is called "Infinite Scrolling". With it, you can go ahead and fetch the next N rows while the customer is scrolling to them.
If, as you're saying paging really is not an option (although I really doubt it ; please explain why you think it is, and I'm pretty sure someone will find a solution), there's really no way to speed up this kind of operation.
As you noticed, it's not the query that's taking long, it's the data transfer. Copying the data from one memory space (sql) to another (your application) is not that fast, and displaying this data is orders of magnitude slower.
Edit: why are your clients "firm on that point" ? Why do they think it's not possible otherwise ? Why do they think it's the best solution ?
There are many options to show a big largeset of data on a grid but third parties software.
Try to use jquery/javascript grids with ajax calls. It will help you to render on client a large amount of rows. Even you can use the cache to not query many times the database.
Those are a good grids that will help your to show thousands of rows on a web browser:
http://www.trirand.com/blog/
https://github.com/mleibman/SlickGrid
http://demos.telerik.com/aspnet-ajax/grid/examples/overview/defaultcs.aspx
http://w2ui.com/web/blog/7/JavaScript-Grid-with-One-Million-Records
I Hope it helps.
You can load all the rows into a Datatable on the client using a Background thread when the application (Web page) starts. Then only use the Datatable to populate your Grids etc....So you do not have to hit SQL again until you need to read / write different data. (All the other answers cover the other options)
Can anybody explain the which is best way writing code in my scenario
I have data base with some tables, I have to show data from the database in pages (contains 20 rows in each page)
I am doing my app is - I am retrieving the data from data by 20 rows each and every time for showing in a each page using SP, every page click I connected to the database and showing data in to front end page.
My idea is - to get complete data in to datatable in one single query execution (single iteration) and store the data in session and use it in every page click.
Is this way is correct or not?
And which is best way to get the data of every page click .
If amount of retrieved data is not big, then yes - getting everything at once is a way to go. On the other hand if its hundreds of thousands of records then your current approach - server-side paging is a much better alternative.
This is a bad approach, because you are storing per user, which means that every logged in user that goes to this page will potentially have their own copy of all the data.
Do real custom paging, as opposed to built-in paging, which still gets the whole data set and then shows one page at a time, so if you need to show 10 records on one page, then you only ask your data store (i.e. database) for 10 records and nothing more. It is more chatty than one single call, but you will have more efficient use of memory and not having that first time loading lag of the entire data set.
Just display the data in a Gridview, Set Datasource as a SQLdatasoure and on the Gridview properties set AllowPaging = "True" and PageSize = "20". This will do it. On SQLDatasoure set EnableCaching ="True" if you don't want requests made for every page.
If money is not an issue, I strongly recommend you looking into either DevExpress or Telerik control suite along with an ORM tool that supports IQueryable.
DevExpress and Telerik grid control has paging feature out-of-box. Binding the grid to IQueryable will sure help with performance.
I was looking for some advice on what options are possible to create an interactive table similar to the image below with C#4.0 WPF.
Essentially I want some fixed data displayed on the left and based on user input the remaining data recalcs based based on selections
I was thinking a datatable has about 95% of the functionality I need but I can't find many resources or info on adding controls to make it interactive or if what I'm considering is a dead-end.
I've currently got new datatables being generated based on selections elsewhere and displayed in Winforms DataGridView but it makes it difficult to compare so I really wanted to make it more interactive. For WPF I'd hoped there would be something nicer.
My questions are, can a datatable do this? Are there any better alternatives?
Additional Info:
The data doesn't come from a database, or a single source.
It doesn't need to be saved anywhere either
The columns and rows are not fixed
A DataGrid can of course do this for you, similar to WinForms DataGridView.
For the backend you have many choices but to accommodate runtime flexibility the good old DataTable might still be a good choice.
Probably link it through a CollectionViewSource
I have 3 form's in my project
When I open form 3, I fill Combo box with data (from DataBase)
it take's time......
How I can fill this Combo box only one time when the program is open ?
(in the first form - form1)
thank's in advance
There are a million ways to do this, and your question is pretty vague. is it the same data in all three combo boxes? Regardless, you want to load the data and store the lists in memory when you application first initializes. There are a lot of good, and a lot of bad ways to do this. Then when each form comes up, check to see if the list in memory is filled, if it is, bind to that list. (If not, of course, fill the list from the database, and then bind to it).
The overall concept is to preload the data, and then always check your memory persistence before going to the database.
Edit
To quickly list a good and bad way of storing these values in memory before I turn in for the night. I'll try to expand on this in the morning.
The best way would be to create a memory repository layer in your application, and have your business objects poll it before heading to the database, but there is some complexity in using this sort of model (mainly dealing with concurrency issues.)
The worst way would be to just declare some global collections of data somewhere, and pull them directly into your UI.