I can store my SQL result set into a ADODB record set or ADO.NET recordset. In .NET
I can use a .NET connection manager, get .NET DataSet, extract the DataTable from
this DataSet and use it. The problem is that I am getting about 30-40K rows in memory.
I don't want to do that. I want to get a SqlDataReader into "some kind of .NET recordset"
so that I can read row by row instead of all at once. How do I do this in ? I want to use C# and .NET only.
Thanks.
.NET has several types of collections List, HashSet ....
All of those will hold the records in memory
DataTable has many features and memory overhead
Create a class with proper properties
List<MyClass> MyClasses = new List<MyClass>();
String strSQL = "select * from Table01";
SqlCommand cmd = new SqlCommand { Connection = Cn, CommandText = strSQL };
try
{
Cn.Open();
SqlDataReader sdr = cmd.ExecuteReader();
while (sdr.Read())
{
MyClasses.Add(new MyClass(sdr.GetValue(0));
}
sdr.close();
}
catch (Exception Ex) {}
finally
{
Cn.Close();
}
30-40K is not all that big unless your row is really big.
I have used collections over 1 million.
If you want to process your data row by row and not load it all into memory then why don't you code a custom transformation: Here is something to give you an idea: http://technet.microsoft.com/en-us/library/ms136027.aspx
Use your existing connection managers for the source and destination and put a script component in between for your custom row by row processing.
This will process data row by row with the advantage that it uses all of SSIS native ability to shift only the optimal amount of rows into memory buffers (and back out again when not needed)
SqlDataReader sdr;
var strSQL = "select * from Table01";
var cmd = new SqlCommand { Connection = Cn, CommandText = strSQL };
try
{
sdr = cmd.ExecuteReader();
sdr.Read();
if (sdr.HasRows)
{
var field = sdr.GetValue(0);
// and so on....
}
dtrContID.Close();
}
catch (Exception err)
{
// .......
}
Related
Would you please show me how to retrieve data from SQL Server to C# (Windows Forms application)?
Consider I have a textbox and I need to fill it with data from SQL Server WHERE 'emp_id = something' for example, how can I do it without a DataGridView?
Or take this another example:
SELECT sum(column) FROM table_name
How to get the value of the above command (also without a DataGridView)?
There are multiple ways to achieve this. You can use DataReader or DataSet \ DataTable. These are connected and disconnected architectures respectively. You can also use ExecuteScalar if you want to retrieve just one value.
Recommendations:
Enclose SqlConnection (and any other IDisposable object) in using block. My code uses try-catch block.
Always use parameterized queries.
Following is some example code with DataReader in case your query returns multiple rows. The code is copied from here.
//Declare the SqlDataReader
SqlDataReader rdr = null;
//Create connection
SqlConnection conn = new SqlConnection("Your connection string");
//Create command
SqlCommand cmd = new SqlCommand("Your sql statement", conn);
try
{
//Open the connection
conn.Open();
// 1. get an instance of the SqlDataReader
rdr = cmd.ExecuteReader();
while(rdr.Read())
{
// get the results of each column
string field1 = (string)rdr["YourField1"];
string field2 = (string)rdr["YourField2"];
}
}
finally
{
// 3. close the reader
if(rdr != null)
{
rdr.Close();
}
// close the connection
if(conn != null)
{
conn.Close();
}
}
In case your query returns single value, you can continue with above code except SqlDataReader. Use int count = cmd.ExecuteScalar();. Please note that ExecuteScalar may return null; so you should take additional precautions.
Filling a Textbox:
using (var sqlConnection = new SqlConnection("your_connectionstring"))
{
sqlConnection.Open();
using (var sqlCommand = sqlConnection.CreateCommand())
{
sqlCommand.CommandText = "select sum(field) from your_table";
object result = sqlCommand.ExecuteScalar();
textBox1.Text = result == null ? "0" : result.ToString();
}
sqlConnection.Close();
}
for reading more than one row you can take a look at SqlCommand.ExecuteReader()
You need to use direct database access such as in the System.Data.SqlClient Namespace which is documented here System.Data.SqlClient Namespace.
Basically, look up creating a SQLConnection and SQLCommand and using them to retrieve the data.
I am new to Visual C# programming language and recently i was trying to make a application that is supposed to insert into a local database of users some data but every times my code runs and the insertion works fine the database does not update.This is the code that i am using
try
{
cn.Open();
SqlCommand insert = new SqlCommand();
insert.CommandText = "insert into Clienti (Nume,Prenume,Parola,Email) values(#Nume,#Prenume,#Parola,#Email)";
insert.Connection = cn;
insert.Parameters.AddWithValue("#Nume", register_nume.Text);
insert.Parameters.AddWithValue("#Prenume", register_prenume.Text);
insert.Parameters.AddWithValue("#Parola", register_password.Text);
insert.Parameters.AddWithValue("#Email", register_email.Text);
insert.ExecuteNonQuery();
SqlDataReader reader = insert.ExecuteReader();
while (reader.Read()) { }
MessageBox.Show("Added succesfully");
}
catch (Exception ex)
{
MessageBox.Show(""+ex);
}
I already tried the property Copy to output and it doesn't seems to work.
I am sorry for any grammar mistakes that i made,I would be grateful for any help.
insert.ExecuteNonQuery();
SqlDataReader reader = insert.ExecuteReader();
while (reader.Read()) { }
You only need the ExecuteNonQuery, it will run the INSERT. You need to use ExecuteReader instead only when you're running a statement that produces result sets (eg. SELECT). So it should be:
insert.ExecuteNonQuery();
Its because you have to now read the database using a select statement, you cant use an INSERT SQL statement to read.
You could add the following immediately after your insert.
using(var selectCmd = new SqlCommand("SELECT Nume,Prenume,Parola,Email FROM Clienti WHERE Nume = #Nume", cn))
{
selectCmd.Parameters.AddWithValue("#Nume", register_nume.Text);
using(SqlDataReader reader = selectCmd.ExecuteReader())
{
while (reader.Read()) { }
}
}
That said if you want to know IF the row was inserted or how many records were inserted ExecuteNonQuery returns the number of rows affected. You could change that part of the code like this:
var recordsAffected = insert.ExecuteNonQuery();
if(recordsAffected > 0)
MessageBox.Show("Added succesfully");
else
MessageBox.Show("Nothing happened");
Although in this particular case it would not make sense because if nothing was inserted it would probably be caused by an Exception.
Some side notes
Always wrap types that implement IDisposable in using blocks (see code above as example). It ensures that resources are always released as soon as you are done with them even if an Exception is thrown.
Never swallow Exceptions! Either recover from one and log it or do not catch it at all. If you swallow it you will never know if/why your code broke.
This part of your code is preparing the SQL command that you are running
cn.Open();
SqlCommand insert = new SqlCommand();
insert.CommandText = "insert into Clienti (Nume,Prenume,Parola,Email) values(#Nume,#Prenume,#Parola,#Email)";
insert.Connection = cn;
insert.Parameters.AddWithValue("#Nume", register_nume.Text);
insert.Parameters.AddWithValue("#Prenume", register_prenume.Text);
insert.Parameters.AddWithValue("#Parola", register_password.Text);
insert.Parameters.AddWithValue("#Email", register_email.Text);
The SQL command that your code is running, is an INSERT statement, which only adds a new record to your table.
This statement runs the command that you setup earlier:
insert.ExecuteNonQuery();
If I understand correctly, here you are trying to read the data from the table again:
SqlDataReader reader = insert.ExecuteReader();
while (reader.Read()) { }
Problem is, your command is NOT set for reading. To read data, you need to use a SELECT statement. Something like this:
insert.CommandText = "SELECT Nume,Prenume,Parola,Email from Clienti" ;
So, to read the data after executing the insert, you should do this:
insert.CommandText = "SELECT Nume,Prenume,Parola,Email from Clienti" ;
SqlDataReader reader = insert.ExecuteReader();
while (reader.Read()) { }
You ar executing 2 times the query:
insert.ExecuteNonQuery();
SqlDataReader reader = insert.ExecuteReader();
Try to get the affected rows
int rows = insert.ExecuteNonQuery();
I would suggest creating a stored procedure in your database and just execute the SP.
It's always better to execute SP from code and leave the SQL programming in the DB.
Let Me first tell you what i am doing right now and what problem i am facing.
Right now i am using SqlDataReader for fetching data from database my function looks like
public List<TOPIC_REPORT> gettopicreports()
{
query = "SELECT * FROM [Topic Reports]";
List<TOPIC_REPORT> rpl = new List<TOPIC_REPORT>();
try
{
con.Open();
com = new SqlCommand(query, con);
sdr = com.ExecuteReader();
while (sdr.Read())
{
rt = new TOPIC_REPORT();
rt.ContentId = sdr.GetString(0);
rt.TimesReported = sdr.GetInt64(1);
rt.IsBanned = sdr.GetInt32(2);
rpl.Add(rt);
}
con.Close();
return rpl;
}
catch (Exception e)
{
con.Close();
throw e;
}
}
Problems with above code
Need to assign values to class variables on by one and problematic with multiple select query.
Need to take care of ResultSet , while loop etc.
Above Problem solution use SqlDataAdapter.
public DataSet getdata()
{
com.CommandText = "GetMasterPageData";
com.Connection = con;
com.CommandType = CommandType.StoredProcedure;
con.Open();
SqlDataAdapter adapter = new SqlDataAdapter(com);
DataSet ds = new DataSet();
adapter.Fill(ds);
con.Close();
return ds;
}
This solves above stated problems but
Decrease my prog readability because my front end person don't know in which order i am sending DataTables in DataSet.
Don't know order of selected values in DataTable.
Front end person need to work on column index which is at end will be problematic.
Suggest me what should i do to solve above stated problems.
You can change your query to return multiple result sets, by combining all your queries. For example:
query = "SELECT * FROM [Topic Reports]; SELECT * FROM [Other table]";
After iterating the first result set with SqlDataReader.Read, use SqlDataReader.NextResultSet() method to get to the second result, then use SqlDataReader.Read to iterate the second set.
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqldatareader.nextresult(v=vs.110).aspx
As a side note, if you do this, then you will have to figure out a way of returning collections of two types of objects from your method. It would probably be better to just use 1 method to get a collection of object A, and a different method to get a collection of object B.
Trying to improve my C# to SQL skills... Currently I am using this bit of code to pull data from our application server. I have two different DBA's telling me two other ways to write this, just trying to figure out if this should be improved on or changed. If so, I would really appreciate some kind of examples.
FYI: This code...
db.con(user.Authority)
...Is essentially a 'new sqlconnection' code.
DataTable dtInfo = new DataTable("SomeInfo");
using (SqlConnection con = db.con(user.Authority))
{
string command = "SOME SQL STATEMENT;";
using (SqlCommand cmd = new SqlCommand(command,con))
{
cmd.CommandType = CommandType.Text;
cmd.Parameters.AddWithValue("#Param", sqlDbType).Value = Param;
con.Open();
cmd.ExecuteNonQuery();
**********
*** OR ***
**********
cmd.CommandType = CommandType.Text;
cmd.Parameters.AddWithValue("#Param", sqlDbType).Value = Param;
using (SqlDataAdapter da = new SqlDataAdapter(cmd))
{
da.Fill(dtInfo );
}
}
}
So, if I'm understanding the provided information, this is my best route?
using (SqlConnection con = db.con(user.Authority))
{
string command = "SELECT [TBL_EMPLOYEE].[ACTIVE_DIRECTORY] FROM [TBL_EMPLOYEE];";
using (SqlCommand cmd = new SqlCommand(command, con))
{
con.Open();
using (SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
MessageBox.Show(reader["ACTIVE_DIRECTORY"].ToString());
}
}
}
And one last thing... This should prevent the need for
cmd.Dispose();
etc...
The code would depend on the specific query. If the query retrieves rows of data (as a SELECT does), then you would go the da.Fill() route. If it's a query that just makes a change to the database (such as INSERT, UPDATE, or DELETE), then you would use ExecuteNonQuery().
I would not use the SqlDataAdapter version. The version that uses the SqlCommand object and the SqlDataReader will perform better, and allows more insight into the actual data being returned.
// Assumes the following sql:
// SELECT foo, bar FROM baz
// error checking left out for simplicity
var list = new List<SomeClass>();
using(var reader = cmd.ExecuteReader()) {
while(reader.Read()) {
list.Add(new SomeClass {
// NOTE: you can see the columns that the c# is referencing
// and compare them to the sql statement being executed
Foo = (string)reader["foo"],
Bar = (string)reader["bar"]
});
}
}
Later as your level of experiance increases you will be able to use other features of the SqlCommand and SqlDataReader classes in order to ensure that the code executes as quickly as possible. If you start using the SqlDataAdapter route, you will eventually have to relearn how to do the exact same things you have already been doing because the SqlCommand and SqlDataReader have operations that do not exist elsewhere in .NET.
ExecuteNonQuery returns the number of rows effected.
A DataTable is not an efficient way to retrieve that number.
int rowsRet = cmd.ExecuteNonQuery();
SqlCommand.ExecuteNonQuery Method
I have never learned this aspect of programming, but is there a way to get each separate result of a excel query(using OleDB) or the likes.
The only way I can think of doing this is to use the INTO keyword in the SQL statement, but this does not work for me (SELECT attribute INTO variable FROM table).
An example would be to use the select statement to retrieve the ID of Clients, and then compare these ID's to clientID's in a client ListArray, and if they match, then the clientTotal orders should be compared.
Could someone prove some reading material and/or some example code for this problem.
Thank you.
This code fetches rows from a sql procedure. Will probably work for you too with some
modifications.
using (var Conn = new SqlConnection(ConnectString))
{
Conn.Open();
try
{
using (var cmd = new SqlCommand("THEPROCEDUREQUERY", Conn))
{
cmd.CommandType = CommandType.StoredProcedure;
SqlDataReader reader = cmd.ExecuteReader();
// Find Id of column in query only once at start
var Col1IdOrd = reader.GetOrdinal("ColumnName");
var Col2IdOrd = reader.GetOrdinal("ColumnName");
// loop through all the rows
while (reader.Read())
{
// get data for each row
var Col1 = reader.GetInt32(ColIdOrd);
var Col2 = reader.GetDouble(Col2IdOrd);
// Do something with data from one row for both columns here
}
}
}
finally
{
Conn.Close();
}