We Keep Coding

loading XML Data in to an Existing Database with Persisting ID's C#

Hi I'm trying to read from a XML file and pump the data in to a existing Database. I'm in a situation where I need the ID's to stay the same from the XML File and I am using EF6.
Everything is actually working when I run this without the Identity disabled part. however that would mean all the ID's are automatically incremented and not the ones I set in to the DB.
My question is simple How do I using this approach Force the ID I provide to be used in the Database ?
I'm translating between a EDMX objects and my DTO objects so I could read from the XML file without having noise from the EDMX file messing with my XML reading.
Also I know this is not pretty but it is almost a one time scenario and I wanted to eliminate the possibilities of human Error as much as possible as it is expected that I have to scale this approach up to about 50+ other tables with various foreign keys and primary keys. simply too much data for a manual hand written Data transfer.
Mapper.Initialize(cfg =>
{
cfg.CreateMap<Model.Logic.DTO.DTORespondents, Model.Datamodel.Respondents>();
});
using (var ctxTest = new Model.Datamodel.MedicalNavigationTestDBEntities())
{
using (var transaction = ctxTest.Database.BeginTransaction())
{
//EXEC sys.sp_identitycolumnforreplication table_object_id, 1
string[] arrFilenames = Directory.GetFiles(fileNamePath);
List<Model.Datamodel.Respondents> dbRespondentsImportList = new List<Model.Datamodel.Respondents>();
foreach (string filename in arrFilenames)
{
var objRespondent = Program.DeSerializeObject<Model.Logic.DTO.DTORespondents>(Path.Combine(fileNamePath, filename));
dbRespondentsImportList.Add(Mapper.Map<Model.Datamodel.Respondents>(objRespondent));
}
foreach (Model.Datamodel.Respondents objRespondents in dbRespondentsImportList)
{
ctxTest.Respondents.Add(objRespondents);
//ctxTest.Respondents.FirstOrDefault(r => r.CollectUrl.Equals(objRespondents.CollectUrl)).Id = objRespondents.Id;
}
ctxTest.Database.ExecuteSqlCommand(#"declare #object_ids int;
set #object_ids = (select[object_id] from sys.objects WHERE [name] = 'respondents')
select #object_ids
EXEC sys.sp_identitycolumnforreplication #object_ids, 1");
ctxTest.Database.ExecuteSqlCommand("SET IDENTITY_INSERT [dbo].[Respondents] ON");
ctxTest.SaveChanges();
ctxTest.Database.ExecuteSqlCommand("SET IDENTITY_INSERT [dbo].[Respondents] OFF");
ctxTest.Database.ExecuteSqlCommand(#"declare #object_ids int;
set #object_ids = (select[object_id] from sys.objects WHERE [name] = 'respondents')
select #object_ids
EXEC sys.sp_identitycolumnforreplication #object_ids, 1");
transaction.Commit() ;
}
}

SET IDENTITY_INSERT [table] ON not working

I want to insert some records where I specify the Id, for the purpose of migrating data where I would like to maintain the integrity of existing relationships.
To do this I ran this command directly in SSMS on the table:
SET IDENTITY_INSERT [CRMTItem] ON;
Yet, when I insert an item from C# with Id of 1, the Id is still incrementing from around 850.
I deleted the entities from EDMX and updated again from DB but with the same result.
Here is my insert code, where as you can see I am ensuring that the Id is indeed 1 before inserting, yet this just gets ignored..
var crmtItem = new CRMTItem();
crmtItem.Id = adv.PrimaryId;
crmtItem.ProjectTitle = adv.ProjectTitle;
crmtItem.CreatedByUser = (adv.CreatedBy == null) ? (Guid?)null : new Guid(adv.CreatedBy);
crmtItem.Opportunity = (adv.Opportunity == null) ? (Guid?)null : new Guid(adv.Opportunity);
crmtItem.BidNoBid = adv.Bnb;
crmtItem.SPUrl = adv.SPUrl;
crmtItem.BnbId = (adv.BnbId == null) ? (Guid?)null : new Guid(adv.BnbId);
crmtItem.Stage = adv.ProjectStage;
crmtItem.Confidential = adv.Confidential;
crmtItem.OpportunityStatus = adv.OpportunityStatus;
crmtItem.OpportunityNumber = adv.OpportunityNumber;
crmtItem.CRMTNumber = adv.CrmtNumber;
crmtItem.ProjectNumber = adv.ProjectNumber;
crmtItem.Sector = adv.Sector;
crmtItem.Service = adv.Service;
crmtItem.CreatedDate = adv.CreatedDate;
crmtItem.Archive = adv.Archive;
crmtItem.ProjectManager = adv.ProjectManager;
crmtItem.WorkTeam = adv.WorkTeam;
crmtItem.Custodian = adv.Custodian;
db.CRMTItems.Add(crmtItem);
if (adv.PrimaryId == 1 || adv.PrimaryId == 2 || adv.PrimaryId == 3)
{
await db.SaveChangesAsync();
}
I also tried adding this line before inserting the item
db.Database.ExecuteSqlCommand("SET IDENTITY_INSERT [dbo].[CRMTItem] ON");
But it still doesn't work.
Based on another SO question I found, I tried this next:
db.Database.ExecuteSqlCommand("SET IDENTITY_INSERT [dbo].[CRMTItem] ON");
db.CRMTItems.Add(crmtItem);
if (adv.PrimaryId == 1)
{
await db.SaveChangesAsync();
}
db.Database.ExecuteSqlCommand("SET IDENTITY_INSERT [dbo].[CRMTItem] OFF");
transaction.Commit();
And now I get an error
Explicit value must be specified for identity column in table 'CRMTItem' either when IDENTITY_INSERT is set to ON or when a replication user is inserting into a NOT FOR REPLICATION identity column.
Am i missing something? Why must it be so difficult to control my own data? If I can't achieve this, I will be forced to creat a temporary column in my table just to store the id from the original (CDS) table, which is absolutely ridiculous, after all it is MY DATA, why can't I choose the value of the columns!?!?!

When you generate your model from database - Entity Framework will map all identity columns to model properties with StoreGeneratedPattern.Identity. In your case, such property is crmtItem.Id as I understand. When you insert crmItem - Entity Framework will ignore value you set for identity property (if you set any), because it knows this value is provided by database, so it knows if it tries to provide such value in insert statement - database will return an error.
Entity Framework has no knowledge of IDENTITY_INSERT, so it will always behave according to StoreGeneratedPattern metadata of target model property. If it's Identity or Computed - it will not provide value for it in insert, whatever you do. If it's set to None - then it will provide a value (no matter what).
So for your case you need to set this attribute to None in EDMX designer for target property (CRMTItem.Id). Of course after doing that - you will have to always provide this value while inserting.
Another part of the problem, with IDENTITY_INSERT being not respected, you already solved but still worth some explanation. This setting is session-scoped, so when you just execute it in SSMS and then try to insert from your application - it has no effect: SSMS and your application are in different sessions.
When you just do:
db.Database.ExecuteSqlCommand("SET IDENTITY_INSERT [dbo].[CRMTItem] ON");
This still executes in separate session, not in the same your SaveChanges will execute. So to execute both IDENTITY_INSERT and SaveChanges in the same session - you need to wrap them in transaction, as you already figured out.

Raw query to add/alter column of table using Entity Framework without migration

I have restricted access to SQL server database & I have to alter table from my MVC project. I have tried:
var db = MyDbContext.Create();
try
{
var res =
db.Speakers.SqlQuery("ALTER TABLE [dbo].[Speakers] ADD [LastName] [nvarchar](256) NULL");
}
catch (Exception exception)
{
Console.WriteLine(exception);
}
its not throwing exception but not updating table. I have no idea whether Raw query allow alteration or not. But I just gave a try. Can any one tell how can I alter database?

Try this approach:
db.Database
.ExecuteSqlCommand("ALTER TABLE [dbo].[Speakers] ADD [LastName] [nvarchar](256) NULL");

Non-query commands can be sent to the database using the
ExecuteSqlCommand method on Database. For example:
using (var context = new BloggingContext())
{
context.Database.ExecuteSqlCommand(
"UPDATE dbo.Blogs SET Name = 'Another Name' WHERE BlogId = 1");
}
Note that any changes made to data in the database using
ExecuteSqlCommand are opaque to the context until entities are loaded
or reloaded from the database.
Entity Framework Raw SQL Queries
https://msdn.microsoft.com/en-gb/data/jj592907.aspx

How to achieve partial update in Entity Framework 5/6?

I am working on Entity framework with database first approach and I came across below issue.
I have a Customer table with columns col1, col2, col3 ,....,col8. I have created an entity for this table and this table has around 100 records already. Out of above 8 columns, col4 is marked as Non-null.
Class Customer
{
member col1;
member col2;
member col3;
member col4;
.
.
member col8;
}
class Main
{
//main logic to read data from database using EF
Customer obj = object of Customerwith values assigned to col1,col2 and col3 members
obj.col2=some changed value.
DBContext.SaveChanges(); //<- throws an error stating it is expecting value of col4.
}
In my application, I am trying to read the one of the record using the stored procedure using EF and stored procedure only returns col1,col2 and col3.
I am trying to save the modified value of col2 and trying to save back to database using DBContext. But it thows an error stating value of required field col4 is not provided.
FYI: I have gone through couple of forums and question and option to go with disabled verfication on SaveChanges is not feasible for me.
Is there any other way through which I can achieve partial update?

I guess EntityFramework.Utilities satisfies your conditions.
This code:
using (var db = new YourDbContext())
{
db.AttachAndModify(new BlogPost { ID = postId }).Set(x => x.Reads, 10);
db.SaveChanges();
}
will generate single SQL command:
exec sp_executesql N'UPDATE [dbo].[BlogPosts]
SET [Reads] = #0
WHERE ([ID] = #1)
',N'#0 int,#1 int',#0=10,#1=1

disabled verfication on SaveChanges is not feasible for me
Sure it is. You even have to disable validation on Save. But then you can't mark the whole entity as modified, which I think you did. You must mark individual properties as modified:
var mySmallCustomer = someService.GetCustomer(); // from sproc
mySmallCustomer.col2 = "updated";
var myLargeCustomer = new Customer();
context.Customers.Attach(myLargeCustomer);
Entry(myLargeCustomer).CurrentValues.SetValues(mySmallCustomer);
// Here it comes:
Entry(myLargeCustomer).Property(c => c.col2).IsModified = true;
context.Configuration.ValidateOnSaveEnabled = false;
context.SaveChanges();
So you see it's enough to get the "small" customer. From this object you create a stub entity (myLargeCustomer) that is used for updating the one property.

Linq To Sql and identity_insert

I am trying to do record inserts on a table where the Primary Key is an Identity field.
I have tried calling
mycontext.ExecuteCommand("SET identity_insert myTable ON")
but this doesn't do any good.
I get an error saying IDENTITY_INSERT is OFF when I submit changes.
How can I turn it ON from the C# code before I submit changes?
EDIT
I have read that this is because ExecuteCommand's code gets executed in a different session.
EDIT 2
Is there any way I can execute some DDL to remove the Identity Specification from my C# code, do the inserts, and then turn Identity Specification back on?

Another option is to wrap all your Linq2Sql calls in a TransactionScope(). This should force them all to run in the same connection.
using System.Transactions; // Be sure to add a reference to System.Transactions.dll to your project.
// ... in a method somewhere ...
using (System.Transaction.TransactionScope trans = new TransactionScope())
{
using(YourDataContext context = new YourDataContext())
{
context.ExecuteCommand("SET IDENTITY_INSERT MyTable ON");
context.ExecuteCommand("yourInsertCommand");
context.ExecuteCommand("SET IDENTITY_INSERT MyTable OFF");
}
trans.Complete();
}
// ...
Although, if you are trying to do something like:
context.ExecuteCommand("SET IDENTITY_INSERT MyTable ON");
context.MyTable.InsertOnSubmit(myTableObject)
context.SubmitChanges()
context.ExecuteCommand("SET IDENTITY_INSERT MyTable OFF");
you will probably run into other issues, especially if the identity column has the IsDbGenerated attribute set to true. The SQL command generated by Linq2Sql will not know to include the identity column and value.

You need to do all the steps in a single T-SQL code block - which is going to be really hard if not impossible if you want to turn it on, then execute your LINQ-to-SQL query, and then turn it back off :(
The only real solution I see is to package up the entire SQL into a SQL statement and execute that:
SET IDENTITY_INSERT MyTable ON
(do your update here)
SET IDENTITY_INSERT MyTable OFF
and execute that as a single code block using .ExecuteContext()
Marc
PS: for your EDIT#2 : no, unfortunately, there's no (easy) way to remove the identity from a column, and turn it back on. Basicall you'd have to create a new column without the IDENTITY, copy the values over, drop the IDENTITY column and then do the same backwards when you're done - sorry! :-(
PS #2: this really begs the question: what on earth to do need to do an "identity insert" for? On a regular basis, from an app? Granted - you might run into this need once in a while, but I'd always do this separately, in SQL Mgmt Studio - certainly not in my app..... (just curious what your use case / motivation is).

It should be enough to open the connection manually before executing commands. This makes the commands run in the same session:
context.Connection.Open();
context.ExecuteCommand("SET IDENTITY_INSERT MyTable ON");
// make changes
// ...
context.SubmitChanges();

What I did is something like this(Nbuider is used to create entities). I create all rows normally except the identity insert row; which is done in the end. This is test data creation hence transaction was not needed.
using (var entitiesEfContext = new ContextABC())
{
var platforms = Builder<Platform>
.CreateListOfSize(4)
.TheFirst(1)
.With(x => x.Description = "Desc1")
.With(x => x.IsDeleted = false)
.TheNext(1)
.With(x => x.Description = "Desc2")
.With(x => x.IsDeleted = false)
.TheNext(1)
.With(x => x.Description = "Desc3")
.With(x => x.IsDeleted = false)
.TheNext(1)
.With(x => x.Description = "Desc4")
.With(x => x.IsDeleted = false)
.Build();
foreach (var platform in platforms)
{
entitiesEfContext.Platform.AddObject(platform);
}
entitiesEfContext.SaveChanges();
// the identity insert row (o as id in my case)
entitiesEfContext.ExecuteStoreCommand("SET IDENTITY_INSERT Platform ON; INSERT INTO [Platform](Platformid,[Description],[IsDeleted],[Created],[Updated]) VALUES (0,'Desc0' ,0 ,getutcdate(),getutcdate());SET IDENTITY_INSERT Platform Off");
}

I had the same error message. And I resolve it by changing the properties of the Primary key in the table.
MyTable.MyId int IDENTITY(1,1) NOT NULL
The property settings MyId (in the dbml)
Auto Generated Value: True;
Auto-Sync: OnInsert;
Primary Key: True;
Server Data Type: int NOT NULL IDENTITY;
Type: int;

just simple reseed identity key (with custom id) every time you add new record, eg:
using (var desDb = new DesDbContext())
{
// del-table-all --------------------------------------------------------
desDb.Database.ExecuteSqlCommand("DELETE FROM [Products]");
foreach (var desItem in desList) //desList is List of Product
{
// reseed identity key
desDb.Database.ExecuteSqlCommand("DBCC CHECKIDENT('Products', RESEED,"
+ (desItem.ProductID - 1) + ");");
// and record
desDb.Products.Add(desItem);
// save-db
desDb.SaveChanges();
}
}