When using a Class to get one row of Data from the Database what is best to use:
A DataSet?
A Reader and do what store the data in a Structure?
What else?
Thanks for your time, Nathan
A DataReader is always your best choice--provided that it is compatible with your usage. DataReaders are very fast, efficient, and lightweight--but they carry the requirement that you maintain an active/open db connection for their lifecycle, this means they can't be marshalled across AppDomains (or across webservices, etc).
DataSets are actually populated by DataReaders--they are eager-loaded (all data is populated before any is accessed) and are therefore less performant, but they have the added benefit of being serializable (they're essentially just a DTO) and that means they're easy to carry across AppDomains or webservices.
The difference is sometimes summed up by saying "DataReaders are ideal for ADO.NET ONLINE (implying that it's fine to keep the db connection open) whereas DataSets are ideal for ADO.NET OFFLINE (where the consumer can't necessarily connect directly to the database).
DataAdapter (which fills a DataSet) uses a DataReader to do so.
So, DataReader is always more lightweight and easier to use than a DataAdapter. DataSets and DataTables always have a huge overhead in terms of memory usage. Makes no difference if you are fetching a single row, but makes a huge difference for bigger result sets.
If you are fetching a fixed number of items, in MS SQL Server, output variables from a stored proc (or parameterized command) usually perform best.
if you use a reader you must have a open connection to your database generally a DataReader is used for fetch a combo or dataGrid, but if you want to stock your data in memory and you close our data base connexion you must use Datatable
Note : excuse my english level
If you just want read-only access to the data, then go with a raw DataReader; it's the fasted and most lightweight data access method.
However, if you intend to alter the data and save back to the database, then I would recommend using a DataAdapter and a DataSet (even a typed DataSet) because the DataSet class takes care of tracking changes, additions and deletions to the set which makes saves much easier. Additionally, if you have multiple tables in the dataset, you can model the referential constraints between them in the dataset.
Related
I often use the DataTable class in my .NET WCF services, since many of our SPs require TVPs. As far as I know, DataTables are the only way of passing TVPs to SPs.
It just occurred to me that similarly to how tables, in which information is stored according to rows and columns are useful, that the DataTable class may be useful beyond just as a means of interfacing with SQL Server TVPs.
Actually... thinking about this, I have previously written code that iterated over a DataTable's rows, building up an HTML string. However the main reason we used a DataTable as because the same table could be passed to SQL Server as a TVP.
Looking at the docs: https://msdn.microsoft.com/en-us/library/system.data.datatable%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396, it looks like you can effectively create relational object models using DataTables.
Would using DataTables be an effective way of caching data retrieved from a SQL Server in a service?
Another potential use-case that comes to mind... Would there be any benefit of using a DataTable for a collection instead of List<MyType>?
Datatables are slower than Lists/Enumerables, and its better to use dataAdapter while reading data if you really care about performance.
But Datatables can be really useful as a item source for grids, where you want to just publish whole table data on the UI and no need to specify each column individually as in the case of List.
I have three layer application in which all database related operation are performed in database layer.
for some queries huge data is fetch in sqldatareader (around 10 Millions rows with 32 columns), my question is how i can pass this big data to presentation layer where i am generating some reports.
after analyzing i have below options please share your inputs on the same.
Pass sqlDatareader itself
which is actually not the idea as i have to keep the connection open all the time.
use datatable
load sqldatareader into datatable and pass it as return statement.
this sounds good but i am not sure that this is the proper approach, in this case i would like to know does it will affect overall performance of the application or not. ?
use list as custom object.
Its winform based application installed on single machine only and i am using .net framework 3.5
your inputs and feedback are greatly appreciated.
Is it really necessary to pass so many data at one moment? Wouldn't be paging better? It would solve you many problems. Anyway, who wants to see 10 millions of data on a single page? Who would want to wait 5 minutes until the page is loaded with all the data?
If you have three-layer architecture, you would probably want to use some business objects instead of objects for direct communication with a database. Presentation layer should know nothing or very little about the database used. So in your db layer, take loaded data, store it in business objects and pass these objects to a presentation layer.
** passing sqldatareader will cost you lots of network usage and its not at all recommended to keep the connection open for such long time.
** Using datatable is a preferable option but however it depends on what kind of operations you want to perform on this huge data. By default, datatable can hold up to 16,777,216 rows
out of these two, second one is more preferable and gives good performance
You shouldn't pass an SQLDataReader or a DataTable on to another application layer, that will expose your DAL and DataBase model implementation and will damage the seperation of application layers.
That being said, SQLDataReader keeps an open connection to the DataBase. Therefore, you should read all the rows you need and close it as early as you can.
Designwise, you should represent the data using your own classes.
Anyway, are you sure you need that many rows in the presentation layer? sounds peculiar to me.
I'm trying to learn to use SQLite, but I'm very frustrated and confused. I've gotten as far as finding System.Data.SQLite, which is apparently the thing to use for SQLite in C#.
The website has no documentation whatsoever. The "original website", which is apparently obsolete from 2010 onwards, has no documentation either. I could find a few blog tutorials, but from what I can tell their method of operation is basically:
Initialize a database connection.
Feed SQL statements into the connection.
Take out stuff that comes out of the connection.
Close connection.
I don't want to write SQL statements in my C# code, they're ugly and I get no assistance from the IDE because I have to put the SQL code in strings.
Can't I just:
Create a DataSet.
Tell the DataSet that it should correspond to the SQLite database MyDB.sqlite.
Manipulate the DataSet using its member functions.
Not worry about SQLite because the DataSet automatically keeps itself in sync with the SQLite database on disc.
I know that I can fill a DataSet with the contents of a database, but if I want access to the entire database I will have to fill the DataSet with all of its contents. If my database is 1 GB, I have just used up 1 GB of RAM (not to mention the time needed to write all of it at once).
Can't I simply take a SQLite database connection and pretend it's just an ordinary DataSet (that perhaps needs to be asked occasionally if it's done syncing yet)?
The answer to the question is no.
No you cannot simply take a SQLite connection pretend it's just a DataSet.
If you don't want to code SQL statements then consider Entity Framework.
Using SQLite Embedded Database with Entity Framework and Linq-to-SQL
You shouldn't treat a DataSet as a database. It's just a result of a query.
You query the database to get a subset of data (you never want ALL the data from your DB) and this subset is used to populate your DataSet.
You are required to synchronize your changes manually because DataSet doesn't know which updates should be a part of which transaction. This is your system knowledge.
The DataSet is an in memory cache and will only synchronize to the underlying data store when the developer allows it. You could put a timer wrapper around in and do it on a schedule but you still need to keep the Dataset and data store synchronized manually.
Storing 1GB+ of data is really not recommended as the memory usage would be very high and the performance very low. You also don't want to be sending that amount of data over a network or god forbid an internet connection.
Why would you want to keep 1GB of data in memory?
I keep reading that SqlDataReaders are much faster than SqlDataAdapters because of their fast-forward, read-only, one-row-at-a-time connected nature, and that they are specifically faster than SqlDataAdapters when to populate a DataTable object (SqlDataAdapter.Fill(dataTable)).
However, here and there somebody will mention "it probably won't make a difference what you use because SqlDataAdapter uses a data reader internally to fill its table." If this is true, how exactly can the adapter be so much slower if it's communicating with the database by using an internal data reader anyway?
I know I could set up some tests and profile the performance of each one, but what I'd really like is for someone to shed some light on the alleged performance discrepancies if we're essentially dealing with the same process either way.
I understand that you'd typically use a reader to create a list of strongly-typed POCOs unlike the data adapter that just fills a table. However, my question is strictly about the details of the performance difference between the two and not O/RM concerns...
If you are using a DataReader, you can react to some information when reading the first row and even disregard the rest of the reading.
If you are using a DataAdapter, you have to first load the entire table and then read the first row in order to react to that same information.
I just saw this topic: Datatable vs Dataset
but it didn't solve my doubt .. Let me explain better, I was doing connection with database and needed to show the results in a GridView. (I used RecordSet when I worked with VB6 while ago and DataSet is pretty similar to it so was much easier to use DataSet.)
Then a guy told me DataSet wasn't the best method to do ..
So, should I 'learn' DataReader or keep using DataSet ? DataTable ?
What are the pros/cons ?
That is essentially: "which is better: a bucket or a hose?"
A DataSet is the bucket here; it allows you to carry around a disconnected set of data and work with it - but you will incur the cost of carrying the bucket (so best to keep it to a size you are comfortable with).
A data-reader is the hose: it provides one-way/once-only access to data as it flies past you; you don't have to carry all of the available water at once, but it needs to be connected to the tap/database.
And in the same way that you can fill a bucket with a hose, you can fill the DataSet with the data-reader.
The point I'm trying to make is that they do different things...
I don't personally use DataSet very often - but some people love them. I do, however, make use of data-readers for BLOB access etc.
It depends on your needs. One of the most important differences is that a DataReader will retain an open connection to your database until you're done with it while a DataSet will be an in-memory object. If you bind a control to a DataReader then it's still open. In addition, a DataReader is a forward only approach to reading data that can't be manipulated. With a DataSet you can move back and forth and manipulate the data as you see fit.
Some additional features: DataSets can be serialized and represented in XML and, therefore, easily passed around to other tiers. DataReaders can't be serialized.
On the other hand if you have a large amount of rows to read from the database that you hand off to some process for a business rule a DataReader may make more sense rather than loading a DataSet with all the rows, taking up memory and possibly affecting scalability.
Here's a link that's a little dated but still useful: Contrasting the ADO.NET DataReader and DataSet.
Further to Marc's point: you can use a DataSet with no database at all.
You can fill it from an XML file, or just from a program. Fill it with rows from one database, then turn around and write it out to a different database.
A DataSet is a totally in-memory representation of a relational schema. Whether or not you ever use it with an actual relational database is up to you.
Different needs, different solutions.
As you said, dataset is most similar to VB6 Recordset. That is, pull down the data you need, pass it around, do with it what you will. Oh, and then eventually get rid of it when you're done.
Datareader is more limited, but it gives MUCH better performance when all you need is to read through the data once. For instance, if you're filling a grid yourself - i.e. pull the data, run through it, for each row populate the grid, then throw out the data - datareader is much better than dataset. On the other hand, dont even try using datareader if you have any intention of updating the data...
So, yes, learn it - but only use it when appropriate. Dataset gives you much more flexibility.
DataReader vs Dataset
1) - DataReader is designed in the connection-oriented architecture
- DataSet is designed in the disconnected architecture
2) - DataReader gives forward-only access to the data
- DataSet gives scrollable navigation to the data
3) - DataReader is read-only we can’t make changes to the data present under it
- DataSet is updatable we can make changes to the data present under it and send those changes back to the data source
4) - DataReader does not provide options like searching and sorting of data
- DataSet provides options like searching and sorting of data
To answer your second question - Yes, you should learn about DataReaders. If anything, so you understand how to use them.
I think you're better of in this situation using DataSets - since you're doing data binding and all (I'm thinking CPU cycles vs Human effort).
As to which one will give a better performance. It very much depends on your situation. For example, if you're editing the data you're binding and batching up the changes then you will be better off with DataSets
DataReader is used to retrieve read-only and forward-only data from a database. It read only one row at a time and read only forward, cannot read backward/random. DataReader cannot update/manipulate data back to database. It retrieve data from single table. As it is connected architecture, data is available as long as the connection exists.
DataSet is in-memory tables. It is disconnected architecture, automatically opens the connection and retrieve the data into memory, closes the connection when done. It fetches all the data at a time from the datasource to its memory. DataSet helps to fetch data from multiple tables and it can fetch back/forth/randomly.DataSet can update/insert/manipulate data.