We Keep Coding

Sql count where condition is not returning correct result

I have a Cars table(sql server) for my parking management application in which each car record is to be assigned unique Slot automatically by app upon checkin.
I'm struggling in implementing it. The concept is that I will keep '1' value in a variable "slotx" and will then figure out using queries if there is already same value present in the 'Slot' column of table or not. If its already there then "slotx" value is incremented by 1 and database is again queried to check if next number i.e. '2' exists in table or not, and this process continues until the "slotx" value doesn't match any record in Cars table, at this point(when same value is not present in table) this value of "slotx" is assigned to 'Slot' column of table for coming vehicle.
But code is not working, I have tried to figure out problem using breakpoints and hopefully correctly spotted that following query is always returning value '1' even if there is not any matching record, hence causing infinite loop.
"Select COUNT(Slot) From Cars Where Slot= #slot;"
Code:
//variables/member declaration
public static int slotx = 1;
private static string searchcarslotauto = "Select COUNT(Slot) From Cars Where Slot= #slot;";
//sqlcommand setup
SqlCommand checkreservedcar = new SqlCommand(searchcarslotauto, con);
checkreservedcar.Parameters.AddWithValue("#slot", slotx);
//CheckIn related
con.Open();
Loop: int isreservereader = (int)checkreservedcar.ExecuteScalar();
if (isreservereader != 0)
{
slotx++;
goto Loop;
}
else
{
insertcarcmd.ExecuteNonQuery();
}
con.Close();
Sample input Data(Table Cars):-
Serial: 1 Reg#: JGI-12-5888 DateTimeIn: 3/5/2018 9:52:00 AM Slot: 1 Subscribed: False;
Expected Result: If there variable "slotx" value doesn't match any record value of 'Slot' column the query is expected to return '0' value, which it doesn't.
Sorry for very lengthy question I couldn't explain it briefly. Kindly someone guide me!
Already tried alternative query to no avail. Count based on condition in SQL Server

Please stop with what you're doing, forget everything you did so far for this particular question and jump to learn a bit about database normalization & identities.
In short, consider this scenario:
Your application gets a request for slot
It pings SQL server and gets info that 2 is free
Another request comes to application
(First request was not written in yet) Application again pings SQL server and gets info that slot 2 is free
Both slot requests are written for slot 2 - effectively, you've lost one record
Bottomline: having application check identities is a very bad idea. That has to be done in transactional way or using IDENTITY columns (better for you for now).

Most likely the problem in your code is in the call AddWithValue. There you are adding a parameter to the command with the initial value of slotx, a 1 I guess. Then in your "Loop" you keep updating the value of slotx, but this is just updating that static variable, not the value you stored in the command parameter. See the accepted answer to this question by Tim Schmelter to see a way to update that parameter for each loop iteration. Another way could be to keep the SqlParameter object returned by AddWithValue and update that one.
A few extra comments:
- Follow the advice by dee zg in another answer to this question and look carefully about using sql transactions to ensure that the same slot not assigned to more than one car.
- Don't use the goto tag, even more when it doesn't do anything that a for/while loop would perfectly accomplish
- Consider using a Numbers table, that way you could use a WHERE NOT EXISTS in your SQL query and solve the problem with just one query, no needing the loop at all. Here a link to an article by Aaron Bertrand talking about those tables.
The code for your loop could look something like this:
con.Open();
while ((int)checkreservedcar.ExecuteScalar() != 0)
{
slotx++;
checkreservedcar .Parameters["#slot"].Value = slotx;
}
insertcarcmd.ExecuteNonQuery();
con.Close();

Oracle Custom Web Interface (ASP.NET, C#) - Getting DBMS_OUTPUT contents

I'm writing a customized, simple Web interface for Oracle DB, using ASP.NET, a Web API project in C#, and Oracle.DataAccess (ODP.NET). This is an educational project which I am designing for an extra project for a college course. There's several reasons for me designing this project, but the upshot is that using Oracle-provided tools (SQL Developer, Enterprise Manage Express, etc.) are not suitable for the task at hand.
I have an API call that can accept a query string, execute it against the DBMS and return the DBMS's output as JSON data, along with some additional return data. This has been sufficient for simple SELECT queries and other basic DDL/DML queries. However, now we're branching into PL/SQL.
For example, the most basic PL/SQL HELLO WORLD program that we'd execute looks like:
BEGIN
DBMS_OUTPUT.PUT_LINE('Hello World');
END;
When I feed this query into my C# API, it does execute successfully. However, I want to be able to retrieve the output of the DBMS_OUTPUT.PUT_LINE call(s).
This question has been addressed before and I have looked into a few of the solutions, and came down on one involving a piece of code which calls the following PL/SQL on the database:
BEGIN
Dbms_output.get_line(:line, :status);
END;
The C# code obviously creates and adds the correct parameter objects to the request before sending it. I plan to call this function repeatedly until a NULL value comes back, indicating the end of output. This data would then be added to the JSON object returned by the API so that the Web interface can display the output. However, this function never returns any lines of output.
My hunch (I'm still learning Oracle myself, so not sure) is that either the server isn't actually outputting the data, or that the buffer is flushed after the PL/SQL anonymous procedure (the Hello World) program finishes.
It was also suggested to add set serveroutput on; to the PL/SQL query but this did not work: it produced the error ORA-00922: missing or invalid option.
Here is the actual C# code being used to retrieve a line of output from the DBMS_OUTPUT buffer:
private string GetDbmsOutputLine(OracleConnection conn)
{
OracleCommand command = new OracleCommand
{
CommandText = "begin dbms_output.get_line(:line, :status); end;",
CommandType = CommandType.Text,
Connection = conn,
};
OracleParameter lineParameter = new OracleParameter("line",
OracleDbType.Varchar2);
lineParameter.Size = 32000;
lineParameter.Direction = ParameterDirection.Output;
command.Parameters.Add(lineParameter);
OracleParameter statusParameter = new OracleParameter("status",
OracleDbType.Int32);
statusParameter.Direction = ParameterDirection.Output;
command.Parameters.Add(statusParameter);
command.ExecuteNonQuery();
if (command.Parameters["line"].Value is DBNull)
return null;
string line = command.Parameters["line"].Value as string;
return line;
}
Edit: I tried manually calling the following procedure prior to executing the user's code: BEGIN DBMS_OUTPUT.ENABLE(32768); END;. This executes without error but after doing so the later calls to DBMS_OUTPUT.GET_LINE still return null.
It looks like what may be happening is that each time I execute a new query to the database, even though it's on the same connection, that the DBMS_OUTPUT buffer is being cleared. I am not sure if this is the case, but it seems to be - nothing else would readily explain the lack of data in the buffer.
Still searching for a way to handle this...
Points to keep in mind:
This is an academic project for student training and development; hence, it is not expected that this mini-application be "production-ready" in any way. Allowing users to execute raw queries posted via the Web obviously leads to all sorts of security risks - which is why this would never be put into an actual production scenario.
I currently open a connection and maintain it throughout a single API call by passing it into each OracleCommand object I create. This, in theory, should mean that the buffer is maintained, but it doesn't appear to be the case. Either the data I write is not making it to the buffer in the first place, or the buffer is flushed each time an OracleCommand object is actually executed against the database connection.

With the caveat that in reality you'd never write code that expects that anyone will ever see data that you attempt to write to the dbms_output...
Within a session, you'd need to call dbms_output.enable that allocates the buffer that is written to by dbms_output. Depending on the Oracle version, you may be able to pass in a null to indicate that you want an unlimited buffer size. In older versions, you'd need to allocate a fixed buffer size (and you'd get an error if you try to write too much data to the buffer). Then you'd call the procedure that calls dbms_output.put[_line]. Finally, you'd be able to call dbms_output.get[_line]. Note that all three things have to happen in the context of a single session. Each session has a separate dbms_output buffer (or no dbms_output buffer).

Why are 700k database rows taking over 15 seconds to load into memory?

Using C# and .NET 3.5 and with either a ADO connection or OLEDB connection, filling a DataTable or DataSet with 700k rows of 3 columns each, takes over 15 seconds.
Executing the actual select statement on the DB takes less than a second. The DB is on a different machine to the one querying it, processing the data. (Perhaps this adds time?)
The data looks like this:
public class Node
{
public DateTime Timestamp;
public float Value;
public string Name;
}
Doing it with a SqlDataReader and calling reader.Read(), then manually putting the data in a new instance of the above class, adding it to a List<Node> also takes over 15 seconds.
Code looks like this:
List<Node> data = new List<Node>();
while (reader.Read())
{
Node n = new Node();
n.Timestamp = (DateTime)reader["Timestamp"];
n.Value = (float)reader["Value"];
n.NodeName = (string)reader["NodeName"];
data.Add(n);
}
I measured this using a StopWatch class in release mode with optimization turned on in project properties.
I get that it has to iterate each record, but I would have expected any machine today to be able to iterate 700k records in a few seconds, but not more.
What could be the reasons this takes over 15 seconds? Am I unreasonable to expect that this should be much faster?
EDIT Doing SqlDataReader.Read() by itself also takes over 15 seconds.

I think the problem lies in the container you're using. The List<> is being dynamically resized a lot. Try the following procedure instead:-
run query with a COUNT clause to get the number of records, only select a single column
List<Node> data = new List<Node>(count from above);
run normal query
fill List<> as above
This will prevent the list from constantly resizing.
Alternatively, to see if this is the problem, replace List<> with LinkedList<> as this doesn't have the resizing issues that List<> does.

It should be the network speed between database and machine you are executing the code at.

What also happens in your loop, is that the values from your query are unboxed. It might be worth it to try the GetString, GetFloat, etc methods, since you have so many records.
List<Node> data = new List<Node>();
while (reader.Read())
{
Node n = new Node();
n.Timestamp = reader.GetDateTime(0); // TODO: Check column numbers
n.Value = reader.GetFloat(1);
n.NodeName = reader.GetString(2);
data.Add(n);
}
No conversions are performed in these methods.
Remarks
No conversions are performed; therefore, the data retrieved
must already be a string, or an exception is generated.

I'm reading a lot of guesses, which could be right, but they are still guesses.
If you run it under the debugger and manually pause it a few times, and each time display the stack, you will be using the random pausing method.
It will tell you exactly what's taking the time and why - no guesswork.
If you want to use a profiler, you need one that samples on wall-clock time.
Otherwise you will have to choose between a) a sampler that gives you line-level inclusive percent, with no visibility into IO time, or b) an instrumenter, that only gives you function-level inclusive percent.
Neither one tells you why the time is being spent, only how much.
Whatever you choose, ignore the temptation to look at self time, which is misleading at best in any app that spends all its time in subfunctions, and it totally ignores IO.

If it is not a code issue then suspect it has to do with your query plan then.
Make sure you are setting the right options before executing the query. and they are of the same state on .NET and MSSQL.
One interesting option that has been found to cause a performance hit before is the ARITHABOIRT being enabled on SQL and off on .NET.
Try adding SET ARITHABORT ON before your query in the command.
Refer to :
Slow performance of SqlDataReader

SQL - Better two queries instead of one big one

I am working on a C# application, which loads data from a MS SQL 2008 or 2008 R2 database. The table looks something like this:
ID | binary_data | Timestamp
I need to get only the last entry and only the binary data. Entries to this table are added irregular from another program, so I have no way of knowing if there is a new entry.
Which version is better (performance etc.) and why?
//Always a query, which might not be needed
public void ProcessData()
{
byte[] data = "query code get latest binary data from db"
}
vs
//Always a smaller check-query, and sometimes two queries
public void ProcessData()
{
DateTime timestapm = "query code get latest timestamp from db"
if(timestamp > old_timestamp)
data = "query code get latest binary data from db"
}
The binary_data field size will be around 30kB. The function "ProcessData" will be called several times per minutes, but sometimes can be called every 1-2 seconds. This is only a small part of a bigger program with lots of threading/database access, so I want to the "lightest" solution. Thanks.

Luckily, you can have both:
SELECT TOP 1 binary_data
FROM myTable
WHERE Timestamp > #last_timestamp
ORDER BY Timestamp DESC
If there is a no record newer than #last_timestamp, no record will be returned and, thus, no data transmission takes place (= fast). If there are new records, the binary data of the newest is returned immediately (= no need for a second query).

I would suggest you perform tests using both methods as the answer would depend on your usages. Simulate some expected behaviour.
I would say though, that you are probably okay to just do the first query. Do what works. Don't prematurely optimise, if the single query is too slow, try your second two-query approach.

Two-step approach is more efficient from overall workload of system point of view:
Get informed that you need to query new data
Query new data
There are several ways to implement this approach. Here are a pair of them.
Using Query Notifications which is built-in functionality of SQL Server supported in .NET.
Using implied method of getting informed of database table update, e.g. one described in this article at SQL Authority blog

I think that the better path is a storedprocedure that keeps the logic inside the database, Something with an output parameter with the data required and a return value like a TRUE/FALSE to signal the presence of new data

What's a good alternative to firing a stored procedure 368 times to update the database?

I'm working on a .NET component that gets a set of data from the database, performs some business logic on that set of data, and then updates single records in the database via a stored procedure that looks something like spUpdateOrderDetailDiscountedItem.
For small sets of data, this isn't a problem, but when I had a very large set of data that required an iteration of 368 stored proc calls to update the records in the database, I realized I had a problem. A senior dev looked at my stored proc code and said it looked fine, but now I'd like to explore a better method for sending "batch" data to the database.
What options do I have for updating the database in batch? Is this possible with stored procs? What other options do I have?
I won't have the option of installing a full-fledged ORM, but any advice is appreciated.
Additional Background Info:
Our current data access model was built 5 years ago and all calls to the db currently get executed via modular/static functions with names like ExecQuery and GetDataTable. I'm not certain that I'm required to stay within that model, but I'd have to provide a very good justification for going outside of our current DAL to get to the DB.
Also worth noting, I'm fairly new when it comes to CRUD operations and the database. I much prefer to play/work in the .NET side of code, but the data has to be stored somewhere, right?
Stored Proc contents:
ALTER PROCEDURE [dbo].[spUpdateOrderDetailDiscountedItem]
-- Add the parameters for the stored procedure here
#OrderDetailID decimal = 0,
#Discount money = 0,
#ExtPrice money = 0,
#LineDiscountTypeID int = 0,
#OrdersID decimal = 0,
#QuantityDiscounted money = 0,
#UpdateOrderHeader int = 0,
#PromoCode varchar(6) = '',
#TotalDiscount money = 0
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
Update OrderDetail
Set Discount = #Discount, ExtPrice = #ExtPrice, LineDiscountTypeID = #LineDiscountTypeID, LineDiscountPercent = #QuantityDiscounted
From OrderDetail with (nolock)
Where OrderDetailID = #OrderDetailID
if #UpdateOrderHeader = -1
Begin
--This code should get code the last time this query is executed, but only then.
exec spUpdateOrdersHeaderForSkuGroupSourceCode #OrdersID, 7, 0, #PromoCode, #TotalDiscount
End

If you are using SQL 2008, then you can use a table-valued parameter to push all of the updates in one s'proc call.
update
Incidentally, we are using this in combination with the merge statement. That way sql server takes care of figuring out if we are inserting new records or updating existing ones. This mechanism is used at several major locations in our web app and handles hundreds of changes at a time. During regular load we will see this proc get called around 50 times a second and it is MUCH faster than any other way we've found... and certainly a LOT cheaper than buying bigger DB servers.

An easy and alternative way I've seen in use is to build a SQL statement consisting of sql_execs calling the sproc with the parameters in the string. Not sure if this is advised or not, but from the .NET perspective, you are only populating one SqlCommand and calling ExecuteNonQuery once...
Note if you choose this then please, please use the StringBuilder! :-)
Update: I much prefer Chris Lively's answer, didn't know about table-valued parameters until now... unfortunately the OP is using 2005.

You can send the full set of data as XML input to the stored procedure. Then you can perform Set operations to modify the database. Set based will beat RBARs on performance almost every single time.

If you are using a version of SQL Server prior to 2008, you can move your code entirely into the stored procedure itself.
There are good and "bad" things about this.
Good
No need to pull the data across a network wire.
Faster if your logic is set based
Scales up
Bad
If you have rules against any logic in the database, this would break your design.
If the logic cannot be set based then you might end up with a different set of performance problems
If you have outside dependencies, this might increase difficulty.
Without details on exactly what operations you are performing on the data it's hard to give a solid recommendation.
UPDATE
Ben asked what I meant in one of my comments about the CLR and SQL Server. Read Using CLR Integration in SQL Server 2005. The basic idea is that you can write .Net code to do your data manipulation and have that code live inside the SQL server itself. This saves you from having to read all of the data across the network and send updates back that way.
The code is callable by your existing proc's and gives you the entire power of .net so that you don't have to do things like cursors. The sql will stay set based while the .net code can perform operations on individual records.
Incidentally, this is how things like heirarchyid were implemented in SQL 2008.
The only real downside is that some DBA's don't like to introduce developer code like this into the database server. So depending on your environment, this may not be an option. However, if it is, then it is a very powerful way to take care of your problem while leaving the data and processing within your database server.

Can you create batched statement with 368 calls to your proc, then at least you will not have 368 round trips. ie pseudo code
var lotsOfCommands = "spUpdateOrderDetailDiscountedItem 1; spUpdateOrderDetailDiscountedItem 2;spUpdateOrderDetailDiscountedItem ... 368'
var new sqlcommand(lotsOfCommands)
command.CommandType = CommandType.Text;
//execute command

I had issues when trying to the same thing (via inserts, updates, whatever). While using an OleDbCommand with parameters, it took a bunch of time to constantly re-create the object and parameters each time I called it. So, I made a property on my object for handling such call and also added the appropriate "parameters" to the function. Then, when I needed to actually call/execute it, I would loop through each parameter in the object, set it to whatever I needed it to be, then execute it. This created SIGNIFICANT performance improvement... Such pseudo-code of my operation:
protected OleDbCommand oSQLInsert = new OleDbCommand();
// the "?" are place-holders for parameters... can be named parameters,
// just for visual purposes
oSQLInsert.CommandText = "insert into MyTable ( fld1, fld2, fld3 ) values ( ?, ?, ? )";
// Now, add the parameters
OleDbParameter NewParm = new OleDbParameter("parmFld1", 0);
oSQLInsert.Parameters.Add( NewParm );
NewParm = new OleDbParameter("parmFld2", "something" );
oSQLInsert.Parameters.Add( NewParm );
NewParm = new OleDbParameter("parmFld3", 0);
oSQLInsert.Parameters.Add( NewParm );
Now, the SQL command, and place-holders for the call are all ready to go... Then, when I'm ready to actuall call it, I would do something like..
oSQLInsert.Parameters[0].Value = 123;
oSQLInsert.Parameters[1].Value = "New Value";
oSQLInsert.Parameters[2].Value = 3;
Then, just execute it. The repetition of 100's of calls could be killed by time by creating your commands over and over...
good luck.

Is this a one-time action (like "just import those 368 new customers once") or do you regularly have to do 368 sproc calls?
If it's a one-time action, just go with the 368 calls.
(if the sproc does much more than just updates and is likely to drag down the performance, run it in the evening or at night or whenever no one's working).
IMO, premature optimization of database calls for one-time actions is not worth the time you spend with it.

Bulk CSV Import
(1) Build data output via string builder as CSV then do a Bulk CSV import:
http://msdn.microsoft.com/en-us/library/ms188365.aspx

Table-valued parameters would be best, but since you're on SQL 05, you can use the SqlBulkCopy class to insert batches of records. In my experience, this is very fast.