We Keep Coding

Duplicate key error

this bugs me for a while.
I have a C# app inserting data to MSSQL database.
it is using entity linq
the column [id] is Primary key, and no auto increase.
existed data like :
id other columns
1001 ......
1002 ......
1003 ......
then i get new data :
ROW1: 1003 .......
ROW2: 1004 .......
ROW3: 1005 .......
the 1003 is existed so surely ROW1 will return "Duplicate key 1003 error"
but, when i try to insert others like
1004,1005 they are NO EXISTED
the program will also return me "Duplicate key 1003 error",
and fail to insert.
then i try on database will sql client, just insert a '1004', it will go through.
I am thinking is this kind of insert buffer,
or like 'none or all' architecture?
then how can i do it?
my code is a loop ,
inserting one row then use dbconn.savechange()

It is an all or nothing. Read the IDs from the Database and manually remove the duplicate entries.
I would recommend checking the min and max values, and read the values back between them. That way you don't have to read all the IDs back.
Because of the overhead, this should be quicker than trying to insert one entry at a time, and checking for an error.

In you are inserting this new Records/Rows as One batch then in case of any error whole batch will be ROLLBACK but if you insert these records per statement as a separate batch then 1003 with throw and error and sql execution will continue executing rest of the sql statements.

Thanks all of you.
I find a Solution, that is
when i find the Duplicate key error come out for the first time.
which means real Duplicate key.
i re-establish database connection by:
dbconn.Dispose();
dbconn = new Entities();
then this will work
thanks a lot!
hope this will help others

Adding a record to access database from C#

I have a Form Windows program in C# that adds a record to the database and can remove it.
In the database i have ID (which is Auto Number), but if i delete a record and if i want to add another record instead, the Auto Number increases and doesn't add the missing numbers.
I mean that if i have 9 records in my Access Database and i want to remove a record, it will be 8, but when i add a new record, i get 10 instead of 9. like this picture:
Is there any solution for that?

If it's an auto number, the database will generate a number greater than the last one used - this is how relational databases are supposed to work. Why would there be solution for this? Imagine deleting 5, what would you want to do then, have the auto number create the next record as 5? If you are displaying an id in your C# app - bad idea - then change this to some other value that you can control as you wish.
However what you are trying to achieve does not make sense.

if i delete a record and if i want to add another record instead, the Auto Number increases and doesn't add the missing numbers.
[...]
Is there any solution for that?
The short answer is "No". Once used, AutoNumber values are typically never re-used, even if the deleted record had the largest AutoNumber value in the table. This is due (at least in in part) to the fact that the Jet/Ace database engine has to be able to manage AutoNumber values in a multi-user environment.
(One exception to the above rule is if the Access database is compacted then the next available AutoNumber value for a table with a sequential AutoNumber field is reset to Max(current_value)+1.)
For more details on how AutoNumber fields work, see my other answer here.

In MS access, there is no any solutions for this. But in case of sql server you can create your own function rather using Identity column.

How to ignore LINQ to SQL on unique key?

I insert data with LINQ to SQL on my DB on a table where there is a unique key on 2 columns.
When I try to insert a row with this unique key already inserted, I get:
Cannot insert duplicate key row in object 'dbo.613_LiveLove' with
unique index 'IX_613_LiveLove'. The duplicate key value is
(35715346455553, paul). The statement has been terminated.
I don't want this error message, just "LINQ to SQL, does not insert it" and continue the process.
Is there a way on doing this? Or need I to use try/catch?

This might help: http://msdn.microsoft.com/en-us/library/bb425822.aspx#linqtosql_topic18
"... any errors detected by the database will cause the submission process to abort and an exception will be raised. All changes to the database will be rolled back as if none of the submissions ever took place. The DataContext will still have a full recording of all changes so it is possible to attempt to rectify the problem and resubmit them by calling SubmitChanges() again..."

Use try catch. Catch the specific exception type, and throw your own exception (you propably want to create your own exception type) including your custom message. At the location in you code you want to continue the process, just catch that exception.

Why does MS Access 2007 not allow a row insert, but then allow it on the next insert attempt?

My insert statement is:
INSERT INTO myTable (inst_id,user_id,app_id,type,accessed_on)
VALUES (3264,2580,'MyApp','Renew',Now);
...where all of the values are formatted correctly. The table has the above fields and one other, a long int auto-increment key field. The foreign keys are 'inst_id', 'user_id', and 'app_id'.
I am getting this error from Access:
...and the following error from VS 2005 when it errors out:
System.Data.OleDb.OleDbException: The changes you requested to the table
were not successful because they would
create duplicate values in the index,
primary key, or relationship. Change
the data in the field or fields that
contain duplicate data, remove the
index, or redefine the index to permit
duplicate entries and try again.
When making this insert query I can look into the database and see that the each of the foreign key values exist in their respective tables and have been for months (for the particular example I am using). These fields are also set so that I can have duplicates, so that is not the issue. Calls of this nature in other tables works great. I do not need to supply the auto-increment key value in the insert query, it adds it for me automatically (like it should).
The weird thing is that if I do this in my code:
try
{
//Execute the query here...
}
catch
{
//Execute the same query again
}
...or if I just try and execute this within Access twice, it works.
Has anyone encountered this before? Again, this type of insert works for other tables, all foreign keys are present in their respective tables, the primary key of this table is set as 'Auto-increment', and all fields (other than the primary key field of course) are set to allow duplicates.
Any ideas?
EDIT: Largest key before inserting: 343085. Largest key after inserting: 343086. The format is:
id: AutoNumber (Field Size=Long Interger, New Values=Increment, Indexed=Yes - No Duplicates)
inst_id: Number (Field Size=Long Interger, Required=Yes, Indexed=Yes - Duplicates OK)
user_id: Number (Field Size=Long Interger, Required=Yes, Indexed=Yes - Duplicates OK)
app_id: Text (Field Size=255, Required=Yes, Indexed=Yes - Duplicates OK)
type: Text (Field Size=50, Required=Yes, Indexed=No)
accessed_on: Date/Time (Default Value=Now(), Required=Yes, Indexed=No)

Going by some old memory here...
Try putting a timestamp field in your table.
I can't remember exactly why that works -- something to do with Access having difficulty identifying records / maybe some kind of locking or indexing quirk. I did some research on that several years ago when it happened to one of my tables.
The key violation the error refers to isn't a missing key in another table, it's a duplicate key in the same table. Sometimes, Access gets it's wires crossed and thinks that the key it's assigning to the new record is already assigned to another record in the table. I don't know what causes that to happen. But by putting a timestamp field in the table, it causes Access to think differently.
It's a frustrating fix, because I don't know why it works. And now I have an otherwise useless timestamp field in my table. But so be it.

MS-Access has been known to barf up spurious errors that have nothing to do with the problem they report. It wouldn't hurt to surround the column called "type" with brackets, [type].
http://office.microsoft.com/en-us/access-help/access-2007-reserved-words-and-symbols-HA010030643.aspx#_Toc272229038

Is the value Now changing between attempts so that there is now no longer a duplicate key error?

INSERT INTO myTable (inst_id,user_id,app_id,type,accessed_on)
VALUES (3264,2580,'MyApp','Renew',Now);
Can you just check this out with accessed_on datatype and Now datatype
Change the value type of DateTime to String while inserting that will be good.
Do let me know if this works for you.
Thanks
rAfee

I believe Jet/ACE will not understand the NOW() method.
And i worked with ACE version, the syntax could not work.
Need to find the other way for direct implementing the syntax.

I know long time ago I had a similuar issue. In my cases I was getting the same error but I didn't have any unique indexes in the table. I finally solved it by reparing and compacting the database.

SqlBulkCopy Error handling / continue on error

I am trying to insert huge amount of data into SQL server. My destination table has an unique index called "Hash".
I would like to replace my SqlDataAdapter implementation with SqlBulkCopy. In SqlDataAapter there is a property called "ContinueUpdateOnError", when set to true adapter.Update(table) will insert all the rows possible and tag the error rows with RowError property.
The question is how can I use SqlBulkCopy to insert data as quickly as possible while keeping track of which rows got inserted and which rows did not (due to the unique index)?
Here is the additional information:
The process is iterative, often set on a schedule to repeat.
The source and destination tables can be huge, sometimes millions of rows.
Even though it is possible to check for the hash values first, it requires two transactions per row (first for selecting the hash from destination table, then perform the insertion). I think in the adapter.update(table)'s case, it is faster to check for the RowError than checking for hash hits per row.

SqlBulkCopy, has very limited error handling facilities, by default it doesn't even check constraints.
However, its fast, really really fast.
If you want to work around the duplicate key issue, and identify which rows are duplicates in a batch. One option is:
start tran
Grab a tablockx on the table select all current "Hash" values and chuck them in a HashSet.
Filter out the duplicates and report.
Insert the data
commit tran
This process will work effectively if you are inserting huge sets and the size of the initial data in the table is not too huge.
Can you please expand your question to include the rest of the context of the problem.
EDIT
Now that I have some more context here is another way you can go about it:
Do the bulk insert into a temp table.
start serializable tran
Select all temp rows that are already in the destination table ... report on them
Insert the data in the temp table into the real table, performing a left join on hash and including all the new rows.
commit the tran
That process is very light on round trips, and considering your specs should end up being really fast;

Slightly different approach than already suggested; Perform the SqlBulkCopy and catch the SqlException thrown:
Violation of PRIMARY KEY constraint 'PK_MyPK'. Cannot insert duplicate
key in object 'dbo.MyTable'. **The duplicate key value is (17)**.
You can then remove all items from your source from ID 17, the first record that was duplicated. I'm making assumptions here that apply to my circumstances and possibly not yours; i.e. that the duplication is caused by the exact same data from a previously failed SqlBulkCopy due to SQL/Network errors during the upload.

Note: This is a recap of Sam's answer with slightly more details
Thanks to Sam for the answer. I have put it in an answer due to comment's space constraints.
Deriving from your answer I see two possible approaches:
Solution 1:
start tran
grab all possible hit "hash" values by doing "select hash in destinationtable where hash in (val1, val2, ...)
filter out duplicates and report
insert data
commit tran
solution 2:
Create temp table to mirror the
schema of destination table
bulk insert into the temp table
start serializable transaction
Get duplicate rows: "select hash from
tempTable where
tempTable.hash=destinationTable.hash"
report on duplicate rows
Insert the data in the temp table
into the destination table: "select * into destinationTable from temptable left join temptable.hash=destinationTable.hash where destinationTable.hash is null"
commit the tran
Since we have two approaches, it comes down to which approach is the most optimized? Both approaches have to retrieve the duplicate rows and report while the second approach requires extra:
temp table creation and delete
one more sql command to move data from temp to destination table
depends on the percentage of hash collision, it also transfers a lot of unnecessary data across the wire
If these are the only solutions, it seems to me that the first approach wins. What do you guys think? Thanks!