We Keep Coding

Differences between SqlDataReader or Dataset to query data from table

I have a table with millions of rows of data, and I would like to know which is the best way to query my data - using .ExecuteReader() or using a Dataset.
Using SqlDataReader like this:
myReader = cmd.ExecuteReader();
And after fill a list with the result
Or using DataSet
using (SqlDataAdapter da = new SqlDataAdapter(cmd))
{
a.Fill(ds);
}
Which are the best method?

The two objects are to be used in contexts fundamentally different.
A DataReader instance returned by the ExecuteReader doesn't return anything until you loop over it using the Read() method. It is a connected object that has a pointer to a current record on the backend database. You read the content of the record using the various GetXXXXX methods provided by the reader or simply using the indexer. When you have done with the current record you orderly pass to the following one using the Read() method. No way to go back or jump to record N + 100.
A DataSet instead is a disconnected object. It uses internally a DataReader to fill its local memory buffer with all the records returned by the command text query. It is handy if you need to work randomly on the data returned or show them on video or print them. But of course, waiting to have millions of records returned by the internal reader could be time consuming and the consuming of the local memory probably will kill your process before the end.
So, which is the best? Neither, if you have millions of records in your table, you need to put in place an appropriate WHERE condition to reduce the amount of records returned. Said that, it depends on what you need to do with the returned data. To display them on a grid probably you could use a DataSet. Instead a DataReader is better if you need to execute operations on the records one by one.

The question is what you want to fill
If you want to fill a DataSet/DataTable use DataAdapter.Fill(ds)
If you want to fill a list/array use a DataReader and a loop
The DataAdapter also uses a DataReader behind the scenes, but it loops all records. You could add a different logic to loop only part of the resultset.
"I have a table with million of rows": You should almost never need to return so many records. So don't filter in memory but in the database.

Both Are good Methods. But if you use SqlDataReader than you have to close it. its is must. Otherwise you will not able to execute any other query until SqlDataReader is not closed.

Handling a large ADODB record set in data table?

I am loading a large result set of about 3 million rows (ADODB record set) into a data table. Its taking too long to even load the result set into a data table. I want to find out a way of extracting only part of a result set and then loading it into a DataTable. Alternatively, is there a way to directly read the recordset directly instead of loading it into a data table and then reading it ?
This is the code I use to fill my DataTable -
OleDbDataAdapter oleDA = new OleDbDataAdapter();
DataTable dt = new DataTable();
oleDA.Fill(dt, myADODBRecordset);

Here are some options to consider:
Get only the rows and columns you really need to work with.
Get some data and let the user ask for the next set of rows when they want to.
Write optimized SQL queries
Don't use a DataTable unless you have to because it contains more metadata information that other list-type objects.
Consider using a managed .NET provider.

Why load so large data into memory? A large data transaction must comsume large resource, so optimize your code, or use EF.

Returned types from stored procedure execution

I have a SP I want to execute and save the groos result aside (in a class field).
Later on I want to acquire the values of some columns for some rows from this result.
What returned types are possible? Which one is the most sutiable for my goal?
I know there are DataSet, DataReader, resultSet. what else?
What is the main difference between them ?

If you want to store the results and use them later (as you have written), you may use the heavy data sets or fill the lightweight lists with custom container types via the data reader.
Or in case you want to consume the results immediately, go on with the data reader.
Result set is the old VB6 class AFAIK or the current Java interface.

The traditional way to get data is by using the the classes in System.Data.SqlClient namespace. You can use the DataReader which is a read only forward type of cursor, fast and efficient when you just want to read a recordset. DataReader is bindable but you read it one record at the time and therefore don't have the options of going back, for instance. If the recordset is very big the reader is also good because it stores just one record at the time in memory.
You can use the DataAdapter and get a DataSet and then you have a complete control of all the data within the DataSet-class. It is heavier on the system but very powerful when you need to work with the data in you application. You can also use DataSet if the query returns more than one recordset.
So it really depends on what you need to do with the data after getting it from the database. If you just need to read it into something else, use DataReader otherwise DataSet.

ADO.NET Question: When to use DataReader, DataAdapter

i just wondering, what things i have to consider when using DataReader and DataAdapter in fetching data from the database and whats the difference between this two other the datareader needs open connection and the datadapter does not... In our projects, were using DataReader in ALL our DAL, we never use dataadapter. So I wondering what scenario would it been better to use DataAdapter + Datatable combo than using DataReader. Thanks in advance.

DataReader : This is best used when you just want to fetch data in readony mode , populate your business entity and close the reader. This is really fast.
Say suppose , you are having a customer class and you want to have fully initilized object with all your customer properties filled like( Name,Address etc..)
You will use DataReader here and just populate the entity and close reader.
You cannot do update with datareader.
DataAdapter : You can Read/Update the data with dataadapters but it is less faster when reading the data then Datareader.
You can update the data with DataAdapter but with reader you won't

I almost always favor the DataReader when doing ADO.NET stuff as well; the reason being, it does not force you to store the data on the client any longer than you must.
That's also somewhat the answer to when to use a DataAdapter to a DataSet/Table; when you want to store the data on the client, perhaps to work with it somehow - iterating back and forth through it, or operating on it as a set, as opposed to simply outputting the values into a grid, where the Reader, IMO, is a better option.

DataReader allow you to process each record and throw it away, which is good when you want to process a lot of data records with no relation to each other. For example, you might use DataReader when you want to calculate some complex statistic value from every records in the database, or to save a lot of data records into a local file.
DataAdapter is something else, it is capable to let you have data records in the memory. That allows you to make the GUI to browse data, editing data, etc.. It is more general but will not work well with large data set.

You only want to use DataAdapters when you use DataSets.
An Adapter has the 2 main methods Fill() and Updater() to read a Dataset from and write it to the Database.
Note that Fill() will open a Connnection, use a DataReader to get all records and then close the Connetion.
Without Datasets and DataTables you don't have a use for DataAdapters.
So the real question is: What kind of storage classes do you want to use in your DAL? DataSets are viable and simple but it's an aging technology (no longer improved).
Maybe you should look around for an ORM (Object Relational Mapping) library. But that will replace your DataReader/Adapter question with a much more complicated choice.

I never use DataReader.
Since I strongly layer my application, my DAL is responsible for talking to the database and my BLL is responsible for building objects, there's no way for the BLL to close the DataReader when it's done. Instead the BLL requests a DataSet/DataTable from the DAL, which the DAL fulfills. It does this by performing a Fill (to TomTom's point > look at the stack trace and yes, you will see a DataReader in there). The BLL then does what it likes with the result set.

what things i have to consider when
using DataReader and DataAdapter
DataReader: Good low level interface. PRetty much the ONLY interface - if you load data into higher up structures, the actual load is always done using a DataReader.
DataAdapter / DataSet: stuff not used by people who like structured p rograms and nice code and do not just happen to write a reporting applcation. Use an ORM instead - NHipernate (good), Linq2SQL (bad), Entity Framework (bad) or one of the other better abstractions.

I guess this question just to talk about proc and cons ,and being off side the code
*Data Reader is much faster than DataAdapter in fetching Data but you have to know what's exactly Disconnected mode
*DataReader or Connected mode and DataAdapter Disconnected mode are being used in the same scenarios but some times Disconnected mode is better in case of you're a way of your data
*But the Disconnectiod mode is provided with rich APIs like DataAdapter ,DataView ,DataTable and DataSet. The powerful thing is you simply provide your DataAdapter with SELECT,INSERT,UPDATE,DELETE Command ,Attach you Data from single table or multiple tables ,with one line of code Adapter.Fill(DataTable) or Adapter.Fill(DataSet) ,and the same way with Updating Data Adapter.Update(DataTable)
*Updating Hierarchical Data in Disconnected mode is far better than working in connected mode which has to use extra code and extra logic to maintain ,in Disconnected mode you have the the ability to update Only Inserted Rows or Updated Rows Or Deleted Rows beside updating operation is wrapped inside Dot Net Transaction
Adapter.Update(DataTable.Select("","",DataViewRowState.Added))
*in disconnected mode you got the ability get the versions of every single row in your data ,beside that you can the Changes to your Data DataTable.GetChanges()
*Disconnected mode provide you with stronglyTypedDataSet ,so you get your data definition schema and the relations ,you can get parent and child rows
*Disconnected mode provide method for Getting Rows by PrimaryKey also getting rows with specific criteria DataTable.Select("FilterExpression","SortOrder",DataRowViewState)
*you can make calculation over DataTable and don't disturb your server with calculations like select productID,ProductName,Price,Quantity,price*quantity as Total, you can easily add column with specific criteria (price*quantity)
*you can make aggregations or your snatched DataTable ,DataTable.Compute("Sum(price)","price>250")
*in Disconnected mode you have CommandBuilder which it creates the sqlcommands for you,but its only working with single table

which Data object should i use

i have a query that return only one row (always) and i want to convert this row to class object (lets say obi)
i have a feeling that using data table to this kind of query is to much
but i dont realy know which other data object to use
data reader?
is there a way to execute sql command to data row ?

DataReader is the best choice here - DataAdapters and DataSets may be overkill for a single row, although, that said, if performance is not critical then keeping-it-simple isn't a bad thing. You don't need to go from DataReader -> DataRow -> your object, just read the values off of the DataReader and you're done.

A datareader lets you query individual fields. If you want the row as a single object, I believe the DataTable/DataRowView family of objects is in fact the way to go.

You might seriously consider taking a look at Linq-to-Sql or Linq-to-Entities.
The appeal of these frameworks is they provide automatic serialization of your database data into objects, abstract away many of the mundane details of connection management, and have better compile-time support by providing strongly-typed properties which you can use without string keys or column ordinals.
When using Linq, the difference between retrieving a single row vs. retrieving multiple rows often only involves appending .Single() or .First() to your query.
At any rate, if you already use or are willing to learn one of these frameworks, you may see the bulk and difficulty of data access code reduce substantially.
With respect to DataReader vs. DataSet/DataTable, it is correct that it takes more cycles to allocate and populate a data table; however, I highly doubt you will notice the difference unless creating an extremely high volume of database calls.
In case it is helpful, here are documentation examples of data access using data readers and data sets.
DataReader
DataSet