We Keep Coding

Why won't Entity Framework Core save a new record to an SQLite table having a foreign key?

I have created a following simple SQLite database for experimenting purposes:
CREATE TABLE "cities"
(
`id` INTEGER PRIMARY KEY AUTOINCREMENT,
`name` TEXT UNIQUE
);
CREATE TABLE `people`
(
`id` INTEGER PRIMARY KEY AUTOINCREMENT,
`city` INTEGER,
`name` TEXT,
`surname` TEXT,
FOREIGN KEY(`city`) REFERENCES `cities`(`id`)
);
Then I have used the SQLite Toolbox by #erikej to generate the model code.
Right after that the following code works just fine:
var context = new SqliteEfcExampleContext();
var city = new Cities {Name = "Praha"};
context.Cities.Add(city);
context.SaveChanges();
This adds the city to the database immediately and this can be seen by an external client.
The following code, however, does nothing:
var context = new SqliteEfcExampleContext();
var city = context.Cities.Single(c => c.Name == "Praha"); // I know this is unsafe
var person = new People {Name = "Jan"};
city.People.Add(person);
context.People.Add(person);
context.SaveChanges();
Even simpler:
var context = new SqliteEfcExampleContext();
var person = new People { Name = "Jan" };
context.People.Add(person);
context.SaveChanges();
works neither. It seems that the mere existence of the foreign key column (even when it is nullable and is not used) makes it impossible to insert a record into the table.
Adding
context.Entry(person).State = EntityState.Added
context.Update(person)
and
context.Dispose()
don't help.
Please tell me what I am doing wrong.
UPDATE:
As soon as I have removed the properties used for the City and People entities to reference each other (to let them work as simple independent tables, the foreign key column is nullable anyway) adding cities has stopped working too.
Also, If I just add a city using context.Cities.Add(city); (calling context.SaveChanges(); after that, but this does not seem to do anything at all in side effects) and try to get it by name later using the same context - it won't find it. I have also tried using context.Local instead of just context.
At the same time, context.Database.ExecuteSqlCommand works just fine and lets me insert a record manually.

Unusual behavior of Entity Framework

I have the following code in C#:
public int AddSynonymBL(String[] syns, String word, User user)
{
int dismissedCounter = 0;
foreach (var item in syns)
{
BusinessLayerStatus.StatusBL res = this.dataAccess.AddSynonymDA(item.Trim().ToLowerInvariant(), word.Trim().ToLowerInvariant(), user);
if (res == BusinessLayerStatus.StatusBL.SynonymNotAdded)
++dismissedCounter;
}
int numberOfFailures = dismissedCounter;
return numberOfFailures;
}
And the following code is for AddSynonymDA method:
internal BusinessLayerStatus.StatusBL AddSynonymDA(string synonym, string word, User user)
{
try
{
Synonym newSyn = new Synonym()
{
Meaning = synonym
};
//The following if means that the searched word does not exist int the Searched table
if (this.context.Users.Where(a => a.Mail.Equals(user.Mail)).FirstOrDefault().Searcheds.Where(b => b.RealWord.Equals(word)).Count() == validNumberForKeyValues)
{
this.context.Users.Where(a => a.Mail.Equals(user.Mail)).FirstOrDefault().Searcheds.Where(b => b.RealWord.Equals(word)).FirstOrDefault().Synonyms.Add(newSyn);
this.context.SaveChanges();
return BusinessLayerStatus.StatusBL.SynonymAdded;
}
else
return BusinessLayerStatus.StatusBL.SynonymNotAdded;
}
catch (Exception ex)
{
ExceptionAction(ex);
return BusinessLayerStatus.StatusBL.SynonymNotAdded;
}
}
I am using Entity Framework. I have a table which contains an Id, a word column. Both of them together have unique key constraint in the database. My main code is as follows:
public static void Main()
{
EngineEntities context = new EngineEntities();
BusinessLogic bl = new BusinessLogic();
String[] s = new String[] { "java", "DB" };
Console.WriteLine(bl.AddSynonymBL(s, "Java", new User() { Mail = "media" }));
}
When I add a value which does not exist in the table everything is fine but when I add a value which already exists in the table, calling this.context.SaveChanges(); in the AddSynonymDA method, always throws an exception which was for the previous first exception which caused the first exception and nothing is added to database even if they do not exist in the database. Why is that?
I get the following error which shows that Java already exists. The problem is that Java is for the first call, as the second call, I have passed DB not Java.
{"Violation of UNIQUE KEY constraint 'IX_Searched'. Cannot insert duplicate key in object 'dbo.Searched'. The duplicate key value is (java, 2).\r\nThe statement has been terminated."}

I suspect that you have not set a column to be an Identity column in your database
In other words when you are inserting an entity you need a column to be automatically incrementing.
The way I do this is for example using SQL server:
ALTER TABLE [User] DROP COLUMN [ID];
ALTER TABLE [User]
ADD [ID] integer identity not null;
If you do not have an ID column already you do not need the first line.
After this, update your EF model in your project by deleting the User table and right clicking and Updating Model from Database and select the table.
So now when you insert new entries in you EF model, the ID column will be automatically incremented and you won't get an error.

you must initially check whether the item exists or not, since you seem to have a unique constraint, then you should utilize the attributes of reference in your code .

How to achieve partial update in Entity Framework 5/6?

I am working on Entity framework with database first approach and I came across below issue.
I have a Customer table with columns col1, col2, col3 ,....,col8. I have created an entity for this table and this table has around 100 records already. Out of above 8 columns, col4 is marked as Non-null.
Class Customer
{
member col1;
member col2;
member col3;
member col4;
.
.
member col8;
}
class Main
{
//main logic to read data from database using EF
Customer obj = object of Customerwith values assigned to col1,col2 and col3 members
obj.col2=some changed value.
DBContext.SaveChanges(); //<- throws an error stating it is expecting value of col4.
}
In my application, I am trying to read the one of the record using the stored procedure using EF and stored procedure only returns col1,col2 and col3.
I am trying to save the modified value of col2 and trying to save back to database using DBContext. But it thows an error stating value of required field col4 is not provided.
FYI: I have gone through couple of forums and question and option to go with disabled verfication on SaveChanges is not feasible for me.
Is there any other way through which I can achieve partial update?

I guess EntityFramework.Utilities satisfies your conditions.
This code:
using (var db = new YourDbContext())
{
db.AttachAndModify(new BlogPost { ID = postId }).Set(x => x.Reads, 10);
db.SaveChanges();
}
will generate single SQL command:
exec sp_executesql N'UPDATE [dbo].[BlogPosts]
SET [Reads] = #0
WHERE ([ID] = #1)
',N'#0 int,#1 int',#0=10,#1=1

disabled verfication on SaveChanges is not feasible for me
Sure it is. You even have to disable validation on Save. But then you can't mark the whole entity as modified, which I think you did. You must mark individual properties as modified:
var mySmallCustomer = someService.GetCustomer(); // from sproc
mySmallCustomer.col2 = "updated";
var myLargeCustomer = new Customer();
context.Customers.Attach(myLargeCustomer);
Entry(myLargeCustomer).CurrentValues.SetValues(mySmallCustomer);
// Here it comes:
Entry(myLargeCustomer).Property(c => c.col2).IsModified = true;
context.Configuration.ValidateOnSaveEnabled = false;
context.SaveChanges();
So you see it's enough to get the "small" customer. From this object you create a stub entity (myLargeCustomer) that is used for updating the one property.

EF ignores DatabaseGeneratedOption.None

I am using EF 6.1.3 Code First, but without migrations as the database already exists. I have an entity SRReports with the following property:
[Key, Required]
[DatabaseGeneratedAttribute(DatabaseGeneratedOption.None)]
public int FRID { get; set; }
Entity Framework ignores the DatabaseGeneratedOption.None and sends TSQL to the server assuming FRID is autogenerated. The entity is assigned a value for FRID but entity framework ignores it and assumes that is autogenerated. (this is confirmed by checking a TRACE of what is sent to the server). The exception message is:
Message=Cannot insert the value NULL into column 'FRID', table 'RevLogon.dbo.SCIREPORTS'; column does not allow nulls. INSERT fails.
The statement has been terminated.
I tried using the fluent API instead of (and in addition to) the annotations.
modelBuilder.Entity<SCIREPORTS>()
.HasKey(e => e.FRID);
modelBuilder.Entity<SCIREPORTS>()
.Property(e => e.FRID).HasDatabaseGeneratedOption(DatabaseGeneratedOption.None);
But then I get the error when I query the table:
SCIREPORTS sr = mydb.SCIREPORTS.Where(s => s.FRID == FRID ).FirstOrDefault();
I get the following error:
System.MissingMethodException was caught
HResult=-2146233069
Message=Method not found: 'System.Data.Entity.ModelConfiguration.Configuration.PrimitivePropertyConfiguration System.Data.Entity.ModelConfiguration.Configuration.PrimitivePropertyConfiguration.HasDatabaseGeneratedOption(System.Nullable`1)'.*
I have uninstalled and reinstalled my EF package but haven't been able to solve this problem.
How can I get EF to insert the record with the ID?
Here is the basic code that inserts the record:
public SRUserComp(string myemail, short mycounter, int myFRID, string myreptype, string mypassword)
{ email = myemail;
Counter = mycounter;
reptype = myreptype;
password = mypassword;
FRID = myFRID; }
public bool CreateSRRecord(DateTime repduedate,
short repnumber)
{BSRModel mydb = new BSRModel();
sr = new SCIREPORTS();
sr.FRID = FRID;
sr.Counter = Counter;
sr.Final = true;
sr.RepDueDate = repduedate;
mydb.SCIREPORTS.Add(sr);
mydb.SaveChanges();
return true; }
After the saveChanges statement I get the following statements from the sql profiler on the server:
INSERT [dbo].[SCIREPORTS]([Counter], [RepDueDate], [RepArrivalDate],
[SciAppDate], [Adminappdate], [legacyDate], [RepNumber], [Final],
[RepReminderID], [Notes], [HaimSaw], [ResAuthApprovDate])
VALUES (#0, #1, NULL, NULL, NULL, NULL, #2, #3, NULL, NULL, NULL, NULL)
SELECT [FRID]
FROM [dbo].[SCIREPORTS]
WHERE ##ROWCOUNT > 0 AND [FRID] = scope_identity()

How to avoid "Violation of UNIQUE KEY constraint" when doing LOTS of concurrent INSERTs

I am performing MANY concurrent SQL INSERT statements which are colliding on a UNIQUE KEY constraint, even though I am also checking for existing records for the given key inside of a single transaction. I am looking for a way to eliminate, or minimize, the amount of collisions I am getting without hurting the performance (too much).
Background:
I am working on an ASP.NET MVC4 WebApi project which receives A LOT of HTTP POST requests to INSERT records. It gets about 5K - 10K requests a second. The project's sole responsibility is de-duplicating and aggregating records. It is very write heavy; it has a relatively small amount of read requests; all of which use a Transaction with IsolationLevel.ReadUncommitted.
Database schema
Here is the DB table:
CREATE TABLE [MySchema].[Records] (
Id BIGINT IDENTITY NOT NULL,
RecordType TINYINT NOT NULL,
UserID BIGINT NOT NULL,
OtherID SMALLINT NULL,
TimestampUtc DATETIMEOFFSET NOT NULL,
CONSTRAINT [UQ_MySchemaRecords_UserIdRecordTypeOtherId] UNIQUE CLUSTERED (
[UserID], [RecordType], [OtherID]
),
CONSTRAINT [PK_MySchemaRecords_Id] PRIMARY KEY NONCLUSTERED (
[Id] ASC
)
)
Repository Code
Here is the code for the Upsert method which is causing the Exception:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Linq;
using Dapper;
namespace MyProject.DataAccess
{
public class MyRepo
{
public void Upsert(MyRecord record)
{
var dbConnectionString = "MyDbConnectionString";
using (var connection = new SqlConnection(dbConnectionString))
{
connection.Open();
using (var transaction = connection.BeginTransaction(IsolationLevel.ReadCommitted))
{
try
{
var existingRecord = FindByByUniqueKey(transaction, record.RecordType, record.UserID, record.OtherID);
if (existingRecord == null)
{
const string sql = #"INSERT INTO [MySchema].[Records]
([UserID], [RecordType], [OtherID], [TimestampUtc])
VALUES (#UserID, #RecordType, #OtherID, #TimestampUtc)
SELECT CAST(SCOPE_IDENTITY() AS BIGINT";
var results = transaction.Connection.Query<long>(sql, record, transaction);
record.Id = results.Single();
}
else if (existingRecord.TimestampUtc <= record.TimestampUtc)
{
// UPDATE
}
transaction.Commit();
}
catch (Exception e)
{
transaction.Rollback();
throw e;
}
}
}
}
// all read-only methods use explicit transactions with IsolationLevel.ReadUncommitted
private static MyRecord FindByByUniqueKey(SqlTransaction transaction, RecordType recordType, long userID, short? otherID)
{
const string sql = #"SELECT * from [MySchema].[Records]
WHERE [UserID] = #UserID
AND [RecordType] = #RecordType
AND [OtherID] = #OtherID";
var paramz = new {
UserID = userID,
RecordType = recordType,
OtherID = otherID
};
var results = transaction.Connection.Query<MyRecord>(sql, paramz, transaction);
return results.SingleOrDefault();
}
}
public class MyRecord
{
public long ID { get; set; }
public RecordType RecordType { get; set; }
public long UserID { get; set; }
public short? OtherID { get; set; }
public DateTimeOffset TimestampUtc { get; set; }
}
public enum RecordType : byte
{
TypeOne = 1,
TypeTwo = 2,
TypeThree = 3
}
}
The Problem
When the server is under heavy enough load, I am seeing many of these Exceptions occurring:
Violation of UNIQUE KEY constraint 'UQ_MySchemaRecords_UserIdRecordTypeOtherId'. Cannot insert duplicate key in object 'MySchema.Records'. The duplicate key value is (1234567890, 1, 123). The statement has been terminated.
This Exception occurs often, as many as 10 times in a minute.
What I have tried
I tried changing the IsolationLevel to Serializable. The Exception occured much less often but still occured. Also, the performance of the code suffered greatly; the system could only handle 2K requests a second. I suspect that this decrease in throughput was actually the cause of the reduced Exceptions so I concluded that this didn't solve my problem.
I have considered using the UPDLOCK Table Hint but I don't fully understand how it cooperates with isolation levels or how to apply it to my code. It does seem like it might be the best solution though, from my current understanding.
I also tried adding the initial SELECT statement (for existing records) to be part of the INSERT statement, like shown here but this attempt still had the same problem.
I tried implementing my Upsert method by using the SQL MERGE statement but this also suffered from the same problem.
My Question(s)
Is there anything I can do to prevent this type of UNIQUE key constraint collisions?
If I should be using the UPDLOCK table hint (or any other table hint for that matter), how would I add that to my code? Would I add it to the INSERT? The SELECT? Both?

Make the validating read take a lock:
FROM SomeTable WITH (UPDLOCK, ROWLOCK, HOLDLOCK)
This serializes accesses on a single key, allowing for concurrency on all others.
HOLDLOCK ( = SERIALIZABLE) protects a range of values. This ensures a row that doesn't exist continues to not exist so the INSERT succeeds.
UPDLOCK ensures any existing row is not changed or deleted by another concurrent transaction so the UPDATE succeeds.
ROWLOCK encourages the engine to take a row-level lock.
These changes may increase the chances of a deadlock.

It may be faster to permit and suppress the errors in your scenario than to attempt to eliminate them. If you're consolidating multiple sources synchronously with overlapping data you will need to create a bottleneck somewhere to manage the race condition.
You could create a singleton manager class that held the unique constraints of the records in a hashset so you would automatically drop duplicates when they're added to the set. Records get added prior to submitting to the DB and removed upon statement completion. That way either the hashset eats the duplicate or the existing record check you do at the top of your try detects the committed duplicate record.

AFAIK, the only solution is to check for duplication before insert. It demands at least one round-trip to DB results in poor performance.
You can do SELECT on a table and hold the lock to prevent other parallel threads to SELECT and getting the same value. Here is the detailed solution: Pessimistic locking in EF code first
PS:
Based on Aron's comment and it's nice work-around, I should say my proposed solution is based on this assumption that you don't want to use buffer or queue.