I have tables in SQLite database.
I use C# and System.Data.SQLite DLL to access the database.
My test shows that the following query is very slow:
string sql = "select column1, column2 from table_1 where column1=1 and column2=2"
SQLiteCommand mycommand = new SQLiteCommand(cnn);
mycommand.CommandText = sql;
SQLiteDataReader reader = mycommand.ExecuteReader();
tb.Load(reader);
reader.Close();
But if I first load the whole table into .NET DataTable like below, and then use DataTable.Select(), it's is much faster:
string sql = #"SELECT * FROM '" + data_table_name + "'" + ";\n";
SQLiteCommand mycommand = new SQLiteCommand(cnn);
mycommand.CommandText = sql;
SQLiteDataReader reader = mycommand.ExecuteReader();
tb.Load(reader);
reader.Close();
DataRow[] rows = tb.Select("column1=1 AND column2=2");
The difference is even more significant if have multiple queries after loading the whole table.
This is essentially cache the whole table in .NET. I tried different kind of tables with different sizes and primary keys, all behave the same.
For now, I know we would use such optimization, but wondering if there are other workaround.
Also, I figured out that if the index of a table is string rather than integer, as long as the string is short (tested up to 10 characters string), the direct SQL query is actually faster without caching the table. However, after caching the table, using integer index is faster than string index.
Anybody can explain the behavior?
The company that I work for has large databases, millions of records in a single table. I have written a C# program that migrates tables between remote servers.
I first create all the tables using SMO without copying data and then the data insertion is done after all the tables have been created.
During the record insertion since there are so many records the console window remains blank until all the rows have been inserted. Due to the sheer volumes of data this takes a long time.
What I want now is a way to print n rows updated like in MSSQL import export data wizard.
The insert part is just a simple insert into select * query.
It sounds like you might be using SqlCommands, if so here is a sample
using (SqlConnection connection = new SqlConnection(Connection.ConnectionString) )
{
using(SqlCommand command = new SqlCommand("insert into OldCustomers select * from customers",connection))
{
connection.Open();
var numRows = command.ExecuteNonQuery();
Console.WriteLine("Affected Rows: {0}",numRows);
}
}
You definitely need to look on OUTPUT clause. There are useful examples on MSDN.
using (SqlConnection conn = new SqlConnection(connectionStr) )
{
var sqlCmd = "
CREATE TABLE #tmp (
InsertedId BIGINT
);
INSERT INTO TestTable
OUTPUT Inserted.Id INTO #tmp
VALUES ....
SELECT COUNT(*) FROM #tmp";
using(SqlCommand cmd = new SqlCommand(sqlCmd,conn))
{
conn .Open();
var numRows = command.ExecuteNonQuery();
Console.WriteLine("Affected Rows: {0}",numRows);
}
}
Also I suggest to use stored procedure for such purposes.
I am doing a simple login form using winforms and access 2010 database (.accdb) in C#.
I have the following code and it seems that the connection string is wrong. I have tried searching and found that .Jet is for access 07?? but this doesnt seem to work too.
i am an amateur at databases (code referred from msdn). I am having trouble understand which should i use for this example too.
access table name: haha
ID (PK) | password
-----------------------
1 | testing
System.Data.SqlClient.SqlConnection conn = new System.Data.SqlClient.SqlConnection(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\BC207\test.accdb");
System.Data.SqlClient.SqlCommand comm = new System.Data.SqlClient.SqlCommand();
comm.CommandText = "SELECT HAHA(*) FROM password";
comm.CommandType = CommandType.Text;
comm.Connection = conn;
conn.Open();
Object returnValue = comm.ExecuteScalar();
conn.Close();
MessageBox.Show((string)returnValue);
edited: the table's name is password, and the field that i want to get the value is ID.
SQL statement i wrote it as : SELECT ID FROM password
and yes, only one record in only one field in the table as the primary key.
anyway the problem is that the program hangs upon execution on the first line
-> Keyword not supported: 'provider'.
so i figured that I have a wrong connection string..
For Acces databases (.mdb, .accdb, etc...), you want to use OleDbConnection, not SqlConnection (SQL Server), like this:
conn = new System.Data.OleDb.OleDbConnection(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\BC207\test.accdb")
Edit: as pointed out, for access OleDbConnection should be used, not SqlConnection...
you can use a much more compact way and also be sure connection is closed and disposed in any possible case even when exceptions are thrown, by using the using statements:
your query text was also, probably wrong as others have suggested...
using (var conn = new OleDbConnection(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\BC207\test.accdb"))
using (var comm = conn.CreateCommand())
{
comm.CommandText = "SELECT password FROM HAHA";
comm.CommandType = CommandType.Text;
conn.Open();
var returnValue = comm.ExecuteScalar();
MessageBox.Show(returnValue.ToString());
}
Edit: are you sure the table HAHA only contains one row? Because the ExecuteScalar returns only one value, if you want to get 1 column but from many records you could use a DataReader or a DataSet...
comm.CommandText = "SELECT HAHA(*) FROM password";
It´s wrong.
"SELECT password FROM HAHA"
Your SQL statement should be,
SELECT * from HAHA
OR
SELECT [Password] From HAHA
EDIT:
You should change the ConnectionString.
I am writing a console program in C# and I need to use a database.
I am looking for very basic tutorials on connecting with and using a db from a C# console program. I haven't been able to find anything basic enough yet and I hope people here can help me find the info I need. I've read the material on MSDN, but MSDN assumes basic knowledge about these things that I am still looking for.
I have created a db within VS Express in my project, created tables, and written some starter records into the tables. I'm trying to find out exactly what each of these things is, and how to determine how to apply them in my project:
SQLConnection
SQLConnection class
SQLCommand
SQLDataAdapter
DataSets
Thanks.
Something like:
using System.Data;
using System.Data.SqlClient;
using(SqlConnection connection = new SqlConnection("")){
SqlCommand command = new SqlCommand(#"
insert into
tblFoo (
col1,
col2
) values (
#val1,
#val2
)",
connection
);
SqlParameter param = new SqlParameter("#val1", SqlDbType.NVarChar);
param.Value = "hello";
command.Parameters.Add(param);
param = new SqlParameter("#val2", SqlDbType.NVarChar);
param.Value = "there";
command.Parameters.Add(param);
command.ExecuteNonQuery();
connection.Close();
}
-- Edit:
Though, of course, when you start doing serious things, I recommend an ORM. I use LLBLGen (it costs money, but most definitely worth it).
-- Edit:
SqlConnection
The thing through which you communicate to the database. This will hold the name of the
server, the username, password, and other misc things.
SqlCommand
Something that holds the sql statement you want to send to the server. This may be an 'update' or 'insert' or 'select' or anything. Depending on what it is, you use a different method to execute it, to possible get data back.
SqlDataAdapter
A strange one; it's used specifically to fill a 'DataSet'. It basically does a bit of work for you, adding the information it finds to the set.
DataSet
Not sure how simple you want this. It's just a collection of returned data, in a table-like format, that you can iterate over. It contains DataTables, because some queries can return more than one table. Typically, though, you'll only have one table, and you can bind to it, or whatever.
Create a sqlconnection, Open it, Create a sqlcommand, execute it to get a sqldatareader, voila. You won't need a dataadapter for a simple example.
string connectionString = "...";
using (SqlConnection conn = new SqlConnection(connectionString))
{
conn.Open();
string sql = "select field from mytable";
SqlCommand cmd = new SqlCommand(sql, conn);
SqlDataReader rdr = cmd.ExecuteReader();
while (rdr.Read())
{
Console.WriteLine(rdr[0]);
}
}
There's a tutorial on ADO.NET that covers a lot of the things you're looking for at http://www.csharp-station.com/Tutorials/AdoDotNet/Lesson01.aspx. Lesson 1 is mostly background but lesson 2 and onwards goes over SQL client objects.
Another tutorial at http://www.codeproject.com/KB/database/sql_in_csharp.aspx covers some of the basics (SqlConnection, SqlCommand).
I bought a book called Pragmatic ADO.NET.
Well, there are two ways to interact with a SQL Server database in C#. The first is with LINQ, and the second is with the SqlClient library.
LINQ
Ever since .NET 3.0, we've had access to LINQ, which is a pretty impressive ORM and way to deal with collections and lists. There are two different ways that LINQ can work with a database. They are:
LINQ to SQL
LINQ to Entities
Scott Gu has a pretty good tutorial on LINQ to SQL, as well. I'd recommend LINQ to SQL for just getting started, and you can use a lot of that in LINQ to Entities going forward.
A sample select to grab all customers in New York would be:
var Custs = from c in Customers
where c.State = 'NY'
select c;
foreach(var Cust in Custs)
{
Console.WriteLine(Cust.Name);
}
SqlClient
The traditional C# way to hit a SQL Server database (pre-.NET 3.0) has been via the SqlClient library. Essentially, you create a SqlConnection to open up a connection to the database. If you need help with your connection strings, check out ConnectionStrings.com.
After you've connected to your database, you will use the SqlCommand object to interact with it. The most important property for this object is the CommandText. This accepts SQL as its language, and will run raw SQL statements against the database.
If you're doing an insert/update/delete, you will use the ExecuteNonQuery method of SqlCommand. However, if you're doing a select, you will use ExecuteReader and return a SqlDataReader. You can then iterate through the SqlDataReader to get your results.
The following is the code to grab all customers in New York, again:
using System.Data;
using System.Data.SqlClient;
//...
SqlConnection dbConn = new
SqlConnection("Data Source=localhost;Initial Catalog=MyDB;Integrated Security=SSPI");
SqlCommand dbComm = new SqlCommand();
SqlDataReader dbRead;
dbConn.Open();
dbComm.Connection = dbConn;
dbComm.CommandText = "select name from customers where state = #state";
dbComm.Parameters.Add("#state", System.Data.SqlDbType.VarChar);
dbComm.Parameters["#state"].Value = "NY";
dbRead = dbComm.ExecuteReader();
if(dbRead.HasRows)
{
while(dbRead.Read())
{
Console.WriteLine(dbRead[0].ToString());
}
}
dbRead.Close();
dbConn.Close();
Hopefully this gives you a good intro to what each approach does and how to learn more.
In general, I recommend using the Microsoft Enterprise Library for DB access. I've used it in a few projects, and am very fond of it.
See the Data Access Quickstart provided by Microsoft that should help you get started
Also, I've also grown accustomed to writing Extension Methods for extracting data from DataRows. For example, I can do something like this:
//Create an extension method, Value,
//to extract a certain type from a DataRow,
//supplying a default value to be used if DbNull.Value is encountered
DateTime someDateValue = dr["SomeDatabaseField"].Value(new DateTime());
Hope this helps!
See ADO.NET Sample Application
Examples cover
SqlClient
using System;
using System.Data;
using System.Data.SqlClient;
class Sample
{
public static void Main()
{
SqlConnection nwindConn = new SqlConnection("Data Source=localhost;Integrated Security=SSPI;Initial Catalog=northwind");
SqlCommand catCMD = nwindConn.CreateCommand();
catCMD.CommandText = "SELECT CategoryID, CategoryName FROM Categories";
nwindConn.Open();
SqlDataReader myReader = catCMD.ExecuteReader();
while (myReader.Read())
{
Console.WriteLine("\t{0}\t{1}", myReader.GetInt32(0), myReader.GetString(1));
}
myReader.Close();
nwindConn.Close();
}
}
OleDb
using System;
using System.Data;
using System.Data.OleDb;
class Sample
{
public static void Main()
{
OleDbConnection nwindConn = new OleDbConnection("Provider=SQLOLEDB;Data Source=localhost;Integrated Security=SSPI;Initial Catalog=northwind");
OleDbCommand catCMD = nwindConn.CreateCommand();
catCMD.CommandText = "SELECT CategoryID, CategoryName FROM Categories";
nwindConn.Open();
OleDbDataReader myReader = catCMD.ExecuteReader();
while (myReader.Read())
{
Console.WriteLine("\t{0}\t{1}", myReader.GetInt32(0), myReader.GetString(1));
}
myReader.Close();
nwindConn.Close();
}
}
Odbc
using System;
using System.Data;
using System.Data.Odbc;
class Sample
{
public static void Main()
{
OdbcConnection nwindConn = new OdbcConnection("Driver={SQL Server};Server=localhost;" +
"Trusted_Connection=yes;Database=northwind");
OdbcCommand catCMD = new OdbcCommand("SELECT CategoryID, CategoryName FROM Categories", nwindConn);
nwindConn.Open();
OdbcDataReader myReader = catCMD.ExecuteReader();
while (myReader.Read())
{
Console.WriteLine("\t{0}\t{1}", myReader.GetInt32(0), myReader.GetString(1));
}
myReader.Close();
nwindConn.Close();
}
}
I want to take a backup of my Access database Pragmatically.
And After taking all data in backup i want to delete data from source database.
( So that it will not take much time while querying and filtering through application.)
The source database name is Data.mdb
The destination database name is Backup.mdb
Both are protected by same password.
For these purpose i am writing a query in C# like this.
string conString = "Provider=Microsoft.Jet.OLEDB.4.0 ;Data Source=Backup.mdb;Jet
OLEDB:Database Password=12345";
OleDbConnection dbconn = new OleDbConnection();
OleDbDataAdapter dAdapter = new OleDbDataAdapter();
OleDbCommand dbcommand = new OleDbCommand();
try
{
if (dbconn.State == ConnectionState.Closed)
dbconn.Open();
string selQuery = "INSERT INTO [Bill_Master] SELECT * FROM [MS Access;DATABASE="+
"\\Data.mdb" + "; Jet OLEDB:Database Password=12345;].[Bill_Master]";
dbcommand.CommandText = selQuery;
dbcommand.CommandType = CommandType.Text;
dbcommand.Connection = dbconn;
int result = dbcommand.ExecuteNonQuery();
}
catch(Exception ex) {}
Everything goes fine if i try with without password database file.
I think error in passing password on query statement.
I am trying to execute through access query but it is saying "Invalid argument".
Please is there any other programing logic for doing that.
Thanks
prashant
YuvaDeveloper
Are Data.mdb and Backup.mdb identically in strcuture? If so, I wouldn't bother copying data via SQL but just copy the whole file.
Try remove the space between the ; and Jet …
So the format would be:
INSERT INTO [Bill_Master] SELECT * FROM [MS Access;DATABASE="+
"\\Data.mdb" + ";Jet OLEDB:Database Password=12345;].[Bill_Master]
You can copy and rename Data.mdb, and then truncate all the tables in Data.mdb. Far easier than trying to copy a table at a time..
Don't delete data. This becomes a lot mroe difficult in the future to do analysis or inquiries. If it's taking a long time then review indexing or upszing to SQL Server. The Express edition is free and can handle databases up to 4 Gb.