I am creating a SQL Server database that records vehicle movements, cleans, repairs etc.
I am struggling with how to record the following data.
I need to record each time the vehicle is cleaned. I also need to extract the amount of time that vehicle is cleaned over the past 28 days and make an average
So far I have a master table with the following columns:
Callsign, Registration, Location, Date Last Cleaned.
Then, as a work around I have "vehicle specific" tables which is where I record it each time it is cleaned.
however as I am working the ASP.net, the only way I can find to access the clean dates is by doing a foreach loop through each of the vehicle tables and returning the count of dates in the past 28 days. the problems with this I cannot work out how do this in 1 data connection instead of multiple requests to the server.
Here is a snippet of the code, but you can see that it runs through each of the check box list, and if "true" then will add to the vehicle specific table
string constr = myconstring;
SqlConnection cnn = new SqlConnection(constr);
SqlTransaction transaction;
for (int i = 0; i < checkboxlistAM.Items.Count; i++)
{
string callsignString = checkboxlistAM.Items[i].Text;
if (checkboxlistAM.Items[i].Selected == true)
{
string declare = "declare #morning datetime declare #date datetime declare #counting int ";
string setting = " set #date = DATEADD(d, DATEDIFF(d, 0, getdate()), 0) set #morning = dateadd(hh, 7, #date) set #counting = (select count([made ready]) from["+callsignString+"] where[Made Ready] = #morning) ";
string insertmorning = " if #counting <>1 insert into ["+callsignString+"] ([made ready]) values (#morning) ";
string QueryVehicleSpecificTable = declare + setting + insertmorning;
string QueryMasterTable = "update Shropshire SET[last made Ready AM] = GETDATE() where Callsign = '"+callsignString+"'";
cnn.Open();
transaction = cnn.BeginTransaction();
SqlCommand cmd1 = new SqlCommand(QueryVehicleSpecificTable, cnn);
cmd1.CommandType = CommandType.Text;
SqlCommand cmd2 = new SqlCommand(QueryMasterTable, cnn);
transaction.Commit();
cmd1.ExecuteNonQuery();
cmd2.ExecuteNonQuery();
cnn.Close();
}
else if (checkboxlistAM.Items[i].Selected == false)
{
string declare = "declare #morning datetime declare #date datetime declare #counting int ";
string setting = " set #date = DATEADD(d, DATEDIFF(d, 0, getdate()), 0) set #morning = dateadd(hh, 7, #date) set #counting = (select count([made ready]) from[" + callsignString + "] where[Made Ready] = #morning) ";
string deletemorning = " delete from ["+callsignString+"] where [Made Ready] = #morning";
string queryDeleteRecordfromVehicleSpecific = declare + setting + deletemorning;
string QueryMasterTable = "update Shropshire SET[last made Ready AM] = null where Callsign = '" + callsignString + "'";
cnn.Open();
transaction = cnn.BeginTransaction();
SqlCommand cmd1 = new SqlCommand(QueryMasterTable, cnn);
SqlCommand cmd2 = new SqlCommand(queryDeleteRecordfromVehicleSpecific, cnn);
cmd1.CommandType = CommandType.Text;
cmd2.CommandType = CommandType.Text;
transaction.Commit();
cmd1.ExecuteNonQuery();
cmd2.ExecuteNonQuery();
cnn.Close();
}
}
As you can see this loop has approx. 42 iterations in the morning and afternoon (84 altogether) and opening and closing the connection each time is going to slow everything down.
Does anyone have any ideas of how I can make this better.
Either by making a new "cleaning" table, but then how can I keep track over 28 days instead of forever.
Or by copying each of the "vehicle specific tables" to 1 DataTable and then somehow updating them again after altering them?!?
Any ideas or questions are welcomed.
Thanks in advance
Before I answer your question, I have to point this out:
where Callsign = '"+callsignString+"'"
This is called SQL concatenation, and it allows for SQL injection, which is the #1 security vulnerability in software today. Never do this. Use parametrized SQL queries all the time.
Now to answer your question.
opening and closing the connection each time is going to slow everything down.
That's probably true, but you should obtain proof that it is a performance problem before you decide to optimize it.
Like #PaulF mentions, you don't need to close the connection every time. Move the code that opens and close the connection outside of your loop.
Another technique is to create a stored procedure in your database that would do all this logic in TSQL. That way you can provide only the callsign to the stored proc and it would execute all 84 operations inside a single SQL command. The disadvantage of this approach is that stored procedures are generally a little more expensive to maintain and refactor.
Related
I am developing a simple C# application for retrieving particular records from one table and insert in to another table in MS Access. Also this task ha s to be performed daily using Windows task scheduler.
The selection of records has to be done on date range to retrieve "Today created records".
My application works between [ DateTime.Today.AddDays(-100) to DateTime.Today.AddDays(0)] but, it wont work for DateTime.Today.AddDays(0) to DateTime.Today.AddDays(1)
I have done so far below.
OleDbConnection con = new OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\\Users\\Tom\\Dropbox\\P2002\\p2002.mdb;Persist Security Info=True");
OleDbCommand cmd = con.CreateCommand();
con.Open();
Console.WriteLine("Connected...");
cmd.CommandText = "Insert into New1 ([B ID], [Dat], [Sum]) SELECT BID, Dat, Summe FROM Bestellung Where [Datum] BETWEEN #" + DateTime.Today.AddDays(0) + "# AND #" + DateTime.Today.AddDays(1) + "#" ;
Console.WriteLine(DateTime.Today.AddDays(0));
Console.WriteLine(DateTime.Today.AddDays(1));
cmd.ExecuteNonQuery();
Console.WriteLine("Record Submitted");
con.Close();
Probably your problem is caused by the automatic conversion from the datetime values (Today and Today.AddDays(1)) into a string when you concatenate those values.
When you query MS Access on date fields with string constants, it wants the strings formatted in the "MM/dd/yyyy" way, and thus you should return a string formatted in that mode when you use it to represent your date values. Like so:
... Where [Datum] BETWEEN #" + DateTime.Today.ToString("MM/dd/yyyy") ...
(Note that I removed the Today.AddDays(0) because it makes no sense)
but there is another way a lot more safe because it avoids any possible Sql Injection attack and more flexible because it doesn't force you to concatenate strings and use formatting options. This method is called "Parameterized query"
Here how to rewrite your code to use parameters.
using(OleDbConnection con = new OleDbConnection(.....))
{
OleDbCommand cmd = con.CreateCommand();
con.Open();
Console.WriteLine("Connected...");
cmd.CommandText = #"Insert into New1 ([B ID], [Dat], [Sum])
SELECT BID, Dat, Summe
FROM Bestellung
Where [Datum] BETWEEN #d1 AND #d2";
Console.WriteLine(DateTime.Today.AddDays(0));
Console.WriteLine(DateTime.Today.AddDays(1));
cmd.Parameters.Add("#d1", OleDbType.Date).Value = DateTime.Today;
cmd.Parameters.Add("#d2", OleDbType.Date).Value = DateTime.Today.AddDays(1);
cmd.ExecuteNonQuery();
Console.WriteLine("Record Submitted");
}
I have a query which fetches the information from sql server on datematch.
I have searched a lot about SQL Server date string, I just want to match with the date and get the data from database. Also I am using SQL Server 2005, I want to fetch the date and take the time out of it?
Can anybody help me in that... I am new to C#
Here is my query.
return "select Timein, Timeout from Attendance where E_ID = " + E_ID + " and Date = " + DateTime.Now.ToShortDateString();
use the sql server CONVERT function to convert the input date param to time
Change your query to accommodate any one of the below CONVERT function
SQL query to convert Time format into hh:mm:ss:
select convert(varchar, <<dateparam>>, 108)
SQL query to convert Time format into hh:mi:ss:mmm(24h):
select convert(varchar, <<dateparam>>, 114)
You should always use parameters when querying a database - whether or not SQL injection is possible, it's just plain good practice to use parameters, and it solves some of the thorny how many quotes and which kind do I need here to make it a valid SQL statement questions, too.
So try something like:
string sqlStmt = "SELECT Timein, Timeout FROM dbo.Attendance " +
"WHERE E_ID = #ID AND Date = #Date";
using(SqlConnection conn = new SqlConnection("your-connection-string-here"))
using(SqlCommand cmd = new SqlCommand(sqlStmt, conn))
{
// set up parameters
cmd.Parameters.Add("#ID", SqlDbType.Int).Value = E_ID;
cmd.Parameters.Add("#Date", SqlDbType.DateTime).Value = DateTime.Now.Date;
// open connection, read data, close connection
conn.Open();
using(SqlDataReader rdr = cmd.ExecuteReader())
{
while(rdr.Read())
{
// read your data
}
rdr.Close();
}
conn.Close();
}
I have a SQL SELECT statement which will not be known until runtime, which could contain JOIN's and inner selects. I need to determine the names and data types of each of the columns of the returned result of the statment from within C#. I am inclined to do something like:
string orginalSelectStatement = "SELECT * FROM MyTable";
string selectStatement = string.Format("SELECT TOP 0 * FROM ({0}) s", orginalSelectStatement);
SqlConnection connection = new SqlConnection(#"MyConnectionString");
SqlDataAdapter adapter = new SqlDataAdapter(selectStatement, connection);
DataTable table = new DataTable();
adapter.Fill(table);
foreach (DataColumn column in table.Columns)
{
Console.WriteLine("Name: {0}; Type: {1}", column.ColumnName, column.DataType);
}
Is there a better way to do what I am trying to do? By "better" I mean either a less resource-intensive way of accomplishing the same task or a more sure way of accomplishing the same task (i.e. for all I know the code snippet I just gave will fail in some situations).
SOLUTION:
First of all, my TOP 0 hack is bad, namely for something like this:
SELECT TOP 0 * FROM (SELECT 0 AS A, 1 AS A) S
In other words, in a sub-select, if two things are aliased to the same name, that throws an error. So it is out of the picture. However, for completeness sake, I went ahead and tested it, along with the two proposed solutions: SET FMTONLY ON and GetSchemaTable.
Here are the results (in milliseconds for 1,000 queries, each):
Schema Time: 3130
TOP 0 Time: 2808
FMTONLY ON Time: 2937
My recommendation would be GetSchemaTable since it's more likely to be future-proofed by a removal of the SET FMTONLY ON as valid SQL and it solves the aliasing problem, even though it is slightly slower. However, if you "know" that duplicate column names will never be an issue, then TOP 0 is faster than GetSchemaTable and is more future-proofed than SET FMTONLY ON.
Here is my experimental code:
int schemaTime = 0;
int topTime = 0;
int fmtOnTime = 0;
SqlConnection connection = new SqlConnection(#"MyConnectionString");
connection.Open();
SqlCommand schemaCommand = new SqlCommand("SELECT * FROM MyTable", connection);
SqlCommand topCommand = new SqlCommand("SELECT TOP 0 * FROM (SELECT * FROM MyTable) S", connection);
SqlCommand fmtOnCommand = new SqlCommand("SET FMTONLY ON; SELECT * FROM MyTable", connection);
for (int i = 0; i < 1000; i++)
{
{
DateTime start = DateTime.Now;
using (SqlDataReader reader = schemaCommand.ExecuteReader(CommandBehavior.SchemaOnly))
{
DataTable table = reader.GetSchemaTable();
}
DateTime stop = DateTime.Now;
TimeSpan span = stop - start;
schemaTime += span.Milliseconds;
}
{
DateTime start = DateTime.Now;
DataTable table = new DataTable();
SqlDataAdapter adapter = new SqlDataAdapter(topCommand);
adapter.Fill(table);
DateTime stop = DateTime.Now;
TimeSpan span = stop - start;
topTime += span.Milliseconds;
}
{
DateTime start = DateTime.Now;
DataTable table = new DataTable();
SqlDataAdapter adapter = new SqlDataAdapter(fmtOnCommand);
adapter.Fill(table);
DateTime stop = DateTime.Now;
TimeSpan span = stop - start;
fmtOnTime += span.Milliseconds;
}
}
Console.WriteLine("Schema Time: " + schemaTime);
Console.WriteLine("TOP 0 Time: " + topTime);
Console.WriteLine("FMTONLY ON Time: " + fmtOnTime);
connection.Close();
You could use GetSchemaTable to do what you want.
There is an example of how to use it here.
If using SQL Server, I would try using SET FMTONLY ON
Returns only metadata to the client. Can be used to test the format of
the response without actually running the query.
Apparently on SQL Server 2012, there's a better way. All is specified in the linked MSDN article.
BTW, this technique is what LINQ To SQL uses internally to determine the result set returned by a stored procedure, etc.
Dynamic SQL is always a bit of a minefield, but you could the SET FMTONLY ON on your query - this means the query will only return Metadata, the same as if no results were returned. So:
string selectStatement = string.Format("SET FMTONLY ON; {0}", orginalSelectStatement);
Alternatively, if you aren't tied to ADO, could you not go down the Linq-to-SQL route and generate a data context which will map out all of your database schemas in to code and their relevant types? You could also have a look at some of the Micro ORMs out there, such as Dapper.Net
There are plenty of other ORMs out there too.
I am converting a VB6 windows application to C# using VS2008, V3.5. I have a SQL Server 2000 database I use for data storage and retrieval. One table holds a single record that is of type Int and is used for generating a quote number, ie, 123456. In VB6 I do the following:
OpenDBCon
Set rst = New ADODB.Recordset
rst.Open "SELECT idx From tblQuoteIdx", cn, adOpenDynamic, adLockPessimistic
Select Case rst.BOF
Case False
rst.MoveFirst
Me.txtRef = rst!idx
tID = rst!idx + 1
OKToContinue = True
Case False
'Do something
End Select
If OKToContinue = True Then
Set rst = New ADODB.Recordset
rst.Open "update tblQuoteIdx set idx = '" & tID & "' ", cn, adOpenDynamic,
adLockPessimistic
End If
CloseDBCon
In C# I currently am doing this:
SqlConnection conn = new SqlConnection();
conn.ConnectionString = Vars.connstring;
conn.Open();
SqlCommand sqlComm = new SqlCommand("Select idx from tblQuoteIdx", conn);
Int32 tt = (Int32)sqlComm.ExecuteScalar();
Int32 tt2 = tt++;
SqlCommand sqlComm2 = new SqlCommand("Update tblQuoteIdx set " +
"idx = " + tt2 + "", conn);
sqlComm.ExecuteNonQuery();
conn.Close();
Unfortunately I find now without a cursor lock of "adLockPessimistic" when several people hit the record at the same time multiple instances of the same index number can show up. If anyone could either explain how to use an ADODB.Recordset in C# for this specific purpose and or use record locks so as to lock the db record when needed, OR a better way within the .Net framework and C# principals to accomplish the same thing, I would be greatful. Many thanks in advance.
Noel
I believe you will find this article useful:
Pessimistic locking in ado.net
Basically you will need to wrap everything in a transaction that applies a lock on the records as soon as editing starts. This way no other method will be able to edit the values until the lock is released.
I'm working in Microsoft Visual C# 2008 Express and with SQLite.
I'm querying my database with something like this:
SQLiteCommand cmd = new SQLiteCommand(conn);
cmd.CommandText = "select id from myTable where word = '" + word + "';";
cmd.CommandType = CommandType.Text;
SQLiteDataReader reader = cmd.ExecuteReader();
Then I do something like this:
if (reader.HasRows == true) {
while (reader.Read()) {
// I do stuff here
}
}
What I want to do is count the number of rows before I do "reader.Read()" since the number returned will affect what I want/need to do. I know I can add a count within the while statement, but I really need to know the count before.
Any suggestions?
The DataReader runs lazily, so it doesn't pick up the entirety of the rowset before beginning. This leaves you with two choices:
Iterate through and count
Count in the SQL statement.
Because I'm more of a SQL guy, I'll do the count in the SQL statement:
cmd.CommandText = "select count(id) from myTable where word = '" + word + "';";
cmd.CommandType = CommandType.Text;
int RowCount = 0;
RowCount = Convert.ToInt32(cmd.ExecuteScalar());
cmd.CommandText = "select id from myTable where word = '" + word + "';";
SQLiteDataReader reader = cmd.ExecuteReader();
//...
Note how I counted *, not id in the beginning. This is because count(id) will ignore id's, while count(*) will only ignore completely null rows. If you have no null id's, then use count(id) (it's a tad bit faster, depending on your table size).
Update: Changed to ExecuteScalar, and also count(id) based on comments.
What you request is not feasible -- to quote Igor Tandetnik, my emphasis:
SQLite produces records one by one, on request, every time you call sqlite3_step.
It simply doesn't know how many there are going to be, until on some sqlite3_step
call it discovers there are no more.
(sqlite3_step is the function in SQLite's C API that the C# interface is calling here for each row in the result).
You could rather do a "SELECT COUNT(*) from myTable where word = '" + word + "';" first, before your "real" query -- that will tell you how many rows you're going to get from the real query.
Do a second query:
cmd.CommandText = "select count(id) from myTable where word = '" + word + "';";
cmd.CommandType = CommandType.Text;
SQLiteDataReader reader = cmd.ExecuteReader();
Your reader will then contain a single row with one column containing the number of rows in the result set. The count will have been performed on the server, so it should be nicely quick.
If you are only loading an id column from the database, would it not be easier to simply load into a List<string> and then work from there in memory?
Normally i would do
select count(1) from myTable where word = '" + word + "';";
to get the result as fast as possible. In the case where id is an int then it won't make much difference. If it was something a bit bigger like a string type then you'll notice a difference over a large dataset.
Reasoning about it count(1) will include the null rows. But i'm prepared to be corrected if i'm wrong about that.
Try this,
SQLiteCommand cmd = new SQLiteCommand(conn);
cmd.CommandText = "select id from myTable where word = '" + word + "';";
SQLiteDataReader reader = cmd.ExecuteReader();
while (reader.HasRows)
reader.Read();
int total_rows_in_resultset = reader.StepCount;
total_rows_in_resultset gives you the number of rows in resultset after processing query
remember that if you wanna use the same reader then close this reader and start it again.
Here is my full implementation in a static method.
You should be able to plug this into your class (replace _STR_DB_FILENAME & STR_TABLE_NAME with your database file name and table name).
/// <summary>
/// Returns a count of the number of items in the database.
/// </summary>
/// <returns></returns>
public static int GetNumberOfItemsInDB()
{
// Initialize the count variable to be returned
int count = 0;
// Start connection to db
using (SqliteConnection db =
new SqliteConnection("Filename=" + _STR_DB_FILENAME))
{
// open connection
db.Open();
SqliteCommand queryCommand = new SqliteCommand();
queryCommand.Connection = db;
// Use parameterized query to prevent SQL injection attacks
queryCommand.CommandText = "SELECT COUNT(*) FROM " + _STR_TABLE_NAME;
// Execute command and convert response to Int
count = Convert.ToInt32(queryCommand.ExecuteScalar());
// Close connection
db.Close();
}
// return result(count)
return count;
}
Note: To improve performance, you can replace '' in "SELECT COUNT()…." with the column name of the primary key in your table for much faster performance on larger datasets.
but I really need to know the count before
Why is that ? this is usually not necessary, if you use adequate in-memory data structures (Dataset, List...). There is probably a way to do what you want that doesn't require to count the rows beforehand.
You do have to count with select count... from...
This will make your application slower. However there is an easy way to make your app faster and that way is using parameterized queries.
See here: How do I get around the "'" problem in sqlite and c#?
(So besides speed parameterized queries have 2 other advantages too.)
SQLiteCommand cmd = new SQLiteCommand(conn);
cmd.CommandText = "select id from myTable where word = '" + word + "';";
SQLiteDataReader reader = cmd.ExecuteReader();
while (reader.Read())
{
total_rows_in_resultset++;
}
Surely a better way to get a row count would be something like this:-
SQLiteDataReader reader = SendReturnSQLCommand(dbConnection, "SELECT COUNT(*) AS rec_count FROM table WHERE field = 'some_value';");
if (reader.HasRows) {
reader.Read();
int count = Convert.ToInt32(reader["rec_count"].ToString());
...
}
That way you don't have to iterate over the rows