I insert large amount of records like 20K using bulk insert,it will be working fine when I insert only one entity. But when I used to insert multiple entities like one to many it will be inserting only the parent entity the child entities are not inserted.
My Entities and Code
Customer.cs
public class Customer
{
public Customer()
{
this.AccountCustomers = new HashSet<AccountCustomer>();
}
public int CustomerId{get;set;}
public int CustomerName{get;set;}
public virtual ICollection<AccountCustomer> AccountCustomers { get; set; }
}
AccountCustomer.cs
public partial class AccountCustomer
{
public int CustomerId { get; set; }
public string CustomData { get; set; }
public System.DateTime CreatedDate { get; set; }
public virtual Customer Customer { get; set; }
}
My code:
List<Customer> customerList = CreateCustomer();
for (int index = 0; index < 20000;index++ )
{
Customer customer = new Customer();
customer.CustomerName= "Parthi";
AccountCustomer accountCustomer = new AccountCustomer();
accountCustomer.CustomdData= "customdata";
accountCustomer.CreatedDate = DateTime.UtcNow;
customer.AccountCustomers.Add(accountCustomer);
customerList.Add(customer);
}
private static void AddCustomer(List<Customer> customerList)
{
using (var ctx =new Directdialogs())
{
using (var transactionScope = new TransactionScope())
{
try
{
ctx.BulkInsert(customerList);
ctx.SaveChanges();
transactionScope.Complete();
}
catch(Exception ex)
{
transactionScope.Dispose();
}
}
}
}
This is my code but it will inserted only the customer entity data not insert the account customer data, Is bulk insert not supporting to insert multiple entities anybody knows help me.
Thanks in advance.
The EntityFramework.BulkInsert project you are using (BulkInsert comes from this 3rd party project not EF6) only supports bulk inserting to one table at a time.
If you want the project to be able to insert in to multiple tables at a time to be able to handle child entities you will need to modify the code to do it yourself (If you do write it yourself please share and contribute the code back to the project).
EDIT: However, this may be a lot harder than you think at first glance. You will have no way to know what columns with Identity values (be it a int identity or a uniqueidentifier) where set to during the bulk insert operation reliably, so setting up Foreign Key relationships may be very hard to do. You may need to pre-set any identity values before you insert.
Related
I'm trying to create an entity object that has many to many relationships with other entities. The relationships are indicated as follows.
public class Change {
// Change Form Fields
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int ChangeId { get; set; }
public string ChangeTitle { get; set; }
public string ChangeType { get; set; }
public DateTime DateSubmitted { get; set; }
public DateTime TargetDate { get; set; }
//Many to Many Collections
public virtual ICollection<Change_CriticalBankingApp> Change_CriticalBankingApps { get; set; } = new List<Change_CriticalBankingApp>();
public virtual ICollection<Change_ImpactedBusiness> Change_ImpactedBusinesses { get; set; } = new List<Change_ImpactedBusiness>();
public virtual ICollection<Change_ImpactedService> Change_ImpactedServices { get; set; } = new List<Change_ImpactedService>();
public virtual ICollection<Change_TestStage> Change_TestStages { get; set; } = new List<Change_TestStage>();
public virtual ICollection<Change_TypeOfChange> Change_TypeOfChanges { get; set; } = new List<Change_TypeOfChange>();
And the DbContext set up is as follows
public class ChangeContext : DbContext {
public ChangeContext(DbContextOptions<ChangeContext> options) : base(options) {
Database.Migrate();
}
public DbSet<Change> Change { get; set; }
public DbSet<TestStage> TestStage { get; set; }
public DbSet<TypeOfChange> TypeOfChange { get; set; }
public DbSet<CriticalBankingApp> CriticalBankingApp { get; set; }
public DbSet<ImpactedBusiness> ImpactedBusiness { get; set; }
public DbSet<ImpactedService> ImpactedService { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder) {
modelBuilder.Entity<Change_CriticalBankingApp>().HasKey(t => new { t.ChangeId, t.CriticalBankingAppId });
modelBuilder.Entity<Change_ImpactedBusiness>().HasKey(t => new { t.ChangeId, t.ImpactedBusinessId });
modelBuilder.Entity<Change_ImpactedService>().HasKey(t => new { t.ChangeId, t.ImpactedServiceId });
modelBuilder.Entity<Change_TestStage>().HasKey(t => new { t.ChangeId, t.TestStageId });
modelBuilder.Entity<Change_TypeOfChange>().HasKey(t => new { t.ChangeId, t.TypeOfChangeId });
}
}
Where I start running into problems is I'm not generating an Id using Entity Framework, the primary key is an identity in SQL Server 2012 and I get that back once the insert is completed, as opposed to using a GUID (which I've read pretty much everywhere is super frowned upon in the DBA world).
So what ends up happening is I either try and do the insert and it tries to insert the many to many relationships with changeId in the junction table being null (because it isn't generated yet) or when I try what I have below to do an insert and an update in one post method. It errors out because the ChangeId key value is already being tracked. Here is what I'm attempting below.
Controller method
public IActionResult CreateChange([FromBody] ChangeModel change) {
if (change == null) {
return BadRequest();
}
//Remove many to many from Change to insert without them (as this can't be done until primary key is generated.
List<Change_CriticalBankingAppModel> criticalApps = new List<Change_CriticalBankingAppModel>();
criticalApps.AddRange(change.Change_CriticalBankingApps);
List<Change_ImpactedBusinessModel> impactedBusinesses = new List<Change_ImpactedBusinessModel>();
impactedBusinesses.AddRange(change.Change_ImpactedBusinesses);
List<Change_ImpactedServiceModel> impactedServices = new List<Change_ImpactedServiceModel>();
impactedServices.AddRange(change.Change_ImpactedServices);
List<Change_TestStageModel> testStages = new List<Change_TestStageModel>();
testStages.AddRange(change.Change_TestStages);
List<Change_TypeOfChangeModel> changeTypes = new List<Change_TypeOfChangeModel>();
changeTypes.AddRange(change.Change_TypeOfChanges);
change.Change_CriticalBankingApps.Clear();
change.Change_ImpactedBusinesses.Clear();
change.Change_ImpactedServices.Clear();
change.Change_TestStages.Clear();
change.Change_TypeOfChanges.Clear();
//Map Change model to change entity for inserting
var changeEntity = Mapper.Map<Change>(change);
_changeRepository.AddChange(changeEntity);
if (!_changeRepository.Save()) {
throw new Exception("Creating change failed on save.");
}
var changetoReturn = Mapper.Map<ChangeModel>(changeEntity);
//Iterate through Many to many Lists to add generated changeId
foreach (var criticalApp in criticalApps) {
criticalApp.ChangeId = changetoReturn.ChangeId;
}
foreach (var impactedBusiness in impactedBusinesses) {
impactedBusiness.ChangeId = changetoReturn.ChangeId;
}
foreach (var impactedService in impactedServices) {
impactedService.ChangeId = changetoReturn.ChangeId;
}
foreach (var testStage in testStages) {
testStage.ChangeId = changetoReturn.ChangeId;
}
foreach (var changeType in changeTypes) {
changeType.ChangeId = changetoReturn.ChangeId;
}
//Add many to many lists back to change to update
changetoReturn.Change_CriticalBankingApps = criticalApps;
changetoReturn.Change_ImpactedBusinesses = impactedBusinesses;
changetoReturn.Change_ImpactedServices = impactedServices;
changetoReturn.Change_TestStages = testStages;
changetoReturn.Change_TypeOfChanges = changeTypes;
changeEntity = Mapper.Map<Change>(changetoReturn);
_changeRepository.UpdateChange(changeEntity);
if (!_changeRepository.Save()) {
throw new Exception("Updating change with many to many relationships failed on save.");
}
changetoReturn = Mapper.Map<ChangeModel>(changeEntity);
return CreatedAtRoute("GetChange",
new { changeId = changetoReturn.ChangeId },
changetoReturn);
}
Relevant Repository methods
public Change GetChange(int changeId) {
return _context.Change.FirstOrDefault(c => c.ChangeId == changeId);
}
public void AddChange(Change change) {
_context.Change.Add(change);
}
public void UpdateChange(Change change) {
_context.Change.Update(change);
}
public bool ChangeExists(int changeId) {
return _context.Change.Any(c => c.ChangeId == changeId);
}
I encounter this error on the update attempt.
I understand that if I were to have entity framework generate the guid instead of having the database generate the identity int that I would have a much easier time with this but a requirement for this project is to not use Guid's.
Any help on how to successfully process this would be greatly appreciated.
EDIT: In case it helps, here is the http post I'm using with postman.
{
"changeTitle": "Test",
"changeType": "Test",
"dateSubmitted": "02/12/2018",
"targetDate": "02/12/2018",
"change_CriticalBankingApps": [
{
"criticalBankingAppId" : 1,
"description" : "Very critical"
},
{
"criticalBankingAppId" : 2,
"description" : "Moderately critical"
}
],
"change_impactedBusinesses": [
{
"ImpactedBusinessId" : 1
},
{
"ImpactedBusinessId" : 2
}
]
}
The error you are getting has nothing to do with the guid vs db identity.
You are getting it because you are:
Fetching an entity from the database
Creating new entity (not tracked) from within your controller (the mapper does this)
Try to update the entity that is not tracked by entity framework
The update will try to add the entity to the EF repository, but will fail because it already contains an entity with the given ID.
If you plan to make changes to an entity, you need to make sure entity framework tracks the entity prior to calling the update method.
If EF does not track your entity, it does not know which fields have been updated (if any).
Edit:
If you want to get rid of the error, you could detach your original entity. Make sure you do it prior to mapping the changetoReturn back into your changeEntity.
dbContext.Entry(entity).State = EntityState.Detached;
But since your new entity won't be tracked, I don't think anything will be updated (EF does not know what has been changed).
Edit 2:
Also take a look at this to get your changes back into your original entity.
Change this:
changeEntity = Mapper.Map<Change>(changetoReturn);
Into this:
Mapper.Map(changetoReturn, changeEntity);
Using Automapper to update an existing Entity POCO
add new entities via joint table...that way, entities are tracked both in the joint table and their individual respective tables
Ok, whether this is an elegant solution is up for debate, but I was able to detach the entity state from changeEntity after doing the initial insert as follows
_changeRepository.AddChange(changeEntity);
_changecontext.Entry(changeEntity).State = EntityState.Detached;
Then after reattaching all of the many to many lists back to changeToReturn, I created a new Change entity and added that entity state, and updated on that as follows.
var newChangeEntity = Mapper.Map<Change>(changeToReturn);
_changecontext.Entry(newChangeEntity).State = EntityState.Added;
_changeRepository.UpdateChange(newChangeEntity);
Then I returned this mapped back to a view model.
It seems hacky and perhaps through a deeper understanding of entity framework I'll discover a much better way of going about this but this works for now.
I'm trying to update a one-to-many relationship with EntityFramework, but EF won't save the relationship for some reason. I'm using ASP.Net MVC, but that does not seem to matter in this case as the data is received correctly.
I've tried a lot of possible solutions and some tutorials, unfortunately almost all of them describe a scenario where the connection is made via a foreign key property in the class itself.(I'm aware that EF adds a FK in the database, but i cant access that directly.) My approach seems to be significantly different as none of their solution seems to work for me.
The code below seems to me to be the most promising, but it still doesn't work. the foreign key of the activity object doesn't get updated.
Removing context.Entry(act.ActivityGroup).State = EntityState.Detached; causes a Primary Key collision, as EF tries to insert the ActivityGroup as a new Entity. Marking it as Modified, doesn't do the trick either.
Models:
public class Activity
{
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
[Key]
public Guid ActivityID { get; set; }
public string Name { get; set; }
public ActivityGroup ActivityGroup { get; set; }
}
public class ActivityGroup
{
public int ActivityGroupID { get; set; }
public string GroupName { get; set; }
public string BackgroundColor { get; set; }
}
Method to save Data
public ActionResult SaveActivities(List<Activity> activities)
{
if (ModelState.IsValid)
{
using (TSSDBContext context = new TSSDBContext())
{
foreach (Activity act in activities)
{
if (act.ActivityGroup != null)
{
context.Entry(act.ActivityGroup).State = EntityState.Detached;
}
context.Entry(act).State = (act.ActivityID == null || act.ActivityID == Guid.Empty) ? EntityState.Added : EntityState.Modified;
}
context.SaveChanges();
return new HttpStatusCodeResult(200);
}
}else
{
return new HttpStatusCodeResult(500);
}
}
You could try something like this.
EF context is tracking each entity you don't need manually marking
entities , Modified or Added for each. Read about Entityframework context tracking
Just fetch the entities what you need and decide to insert or update on based on your condition and just Add what should be added and update
Just do a SaveChanges EF will show the magic
This is a basic idea of inserting and updating entities at one shot. If you have concerns about performance i would suggest you to update using AddRange method in EF 6.0
using(var db1 = new Entities1())
{
var activitylists = db.Activity.ToList();
foreach (var item in activitylists )
{
if(item.Id==null)
{
var newActivity= new Activity();
//Your entities
newActivity.Name="Name";
db.Activity.Add(newActivity);
db.Entry<Activity>(item).State = System.Data.Entity.EntityState.Added;
}
else
{
item.Name="new name update";
db.Entry<Activity>(item).State = System.Data.Entity.EntityState.Modified;
}
}
db.SaveChanges();
}
Update : if your getting data from PostRequest , you need to manually mark the entities as modified or added as the context is not aware of what to do with entities
When I insert my objects, they recognize they are one-to-many and the foreign key is correctly placed in the many side table.
When I retrieve my objects, they do not recognize the one-to-many on the one side table so I cannot access the ICollection of the many side objects. Specifically a Null Reference Exception is thrown when trying to access the collection/
In the explanation below, Incident is the one side and Disturbance is the many side. An Incident is associated with many Disturbances, but a Disturbance is a part of only one Incident.
Disclaimer: due to some project constraints and some modules being built on top of other modules we are using Entity Framework in our DAL and have models cross cutting Business/Data. This may factor into the issue. I'm aware this isn't ideal, but this is where we are at and I haven't seen anything that explicitly says you cannot use EF like this.
I have an Incident defined like this:
public class Incident
{
public Incident()
{
}
public Incident(List<Disturbance> sortedDisturbances)
{
StartTime = sortedDisturbances[0].StartTime;
Disturbances = new List<Disturbance>(sortedDisturbances);
}
[Key]
public int IncidentID { get; set; }
public virtual ICollection<Disturbance> Disturbances { get; set; }
[Column(TypeName="datetime2")]
public DateTime? StartTime { get; set; }
}
I had to add a parameterless constructor to deal with errors resulting from Entity Framework trying to use a parameterless constructor in certain areas.
I have a Disturbance defined like this :
public class Disturbance : IComparable<Disturbance>
{
[Key]
public int DisturbanceID { get; set; }
[Column(TypeName = "datetime2")]
public DateTime StartTime { get; set; }
[Column(TypeName = "datetime2")]
public DateTime EndTime { get; set; }
public int CompareTo(Disturbance other)
{
if (this.StartTime < other.StartTime)
return 1;
if (this.StartTime > other.StartTime)
return -1;
return 0;
}
}
I haven't read anything that said implementing an interface would break anything in Entity Framework so I did it.
This is how I add an Incident:
Business Layer:
private void MakeIncident(List<Disturbance> DisturbancesToAggregate)
{
Incident incidentToInsert = new Incident(DisturbancesToAggregate);
_iDAL.InsertIncident(incidentToInsert);
}
Data Layer:
public void InsertIncident(Incident incidentToInsert)
{
using (var context = new InternalContext())
{
context.Incident.Add(incidentToInsert);
context.SaveChanges();
}
}
The problem is that when I access my Incidents:
public IEnumerable<DomainModel.Disturbance> GetProcessedDisturbances()
{
List<DomainModel.Disturbance> processedDisturbances = new List<DomainModel.Disturbance>();
using(var context = new InternalContext())
{
foreach(var i in context.Incident)
{
foreach(var d in i.Disturbances)
{
processedDisturbances.Add(d);
}
}
}
return processedDisturbances;
}
The i.Disturbances Collection causes a Null Reference Exception. Is there something I need to call to force the context to get the Disturbances? Am I doing something blatantly wrong?
My ideas (I don't like any of them and don't want to do any of them):
1. Explicitly put the IncidentID on the Disturbance table (not even sure if this would work)
2. Force a lookup table by adding an ICollection of Incidents to Disturbances (its not a many-to-many relationship and I think this would prevent me from being able to clear all Disturbances from an Incident)
3. Explicitly define the relationship when the model is created. (I don't like the idea of having to do this, plus I think EF is half way there because it is inserting correctly.
Its happening because of lazy loading in EF. We need to Eagerly loading the data. To know more about them, please refer the link below.
https://msdn.microsoft.com/en-in/data/jj574232.aspx
I've defined two tables in SQL: "Inquerito" and "Pergunta", and a third table "Inquerito_Pergunta" to make the many-to-many relationship. In that last table, the primary key is both the primary key of the Inquerito and the Pergunta.
I'am supposed to add as many as "Perguntas" as I want into an "Inquerito" instance. And, it's important to keep the insertion order, so when I'm showing it to the user it's shown in the same order. A "Pergunta" can also have multiple "Inquerito", but the order doesn't matter in that case.
I'm using MVC 4 Entity Framework and my Models are defined like this:
public partial class Inquerito
{
public Inquerito()
{
this.Pergunta = new List<Pergunta>();
}
public System.Guid id { get; set; }
public virtual ICollection<Pergunta> Pergunta { get; set; }
}
public partial class Pergunta
{
public Pergunta()
{
this.Inquerito = new List<Inquerito>();
}
public System.Guid id { get; set; }
public virtual ICollection<Inquerito> Inquerito { get; set; }
}
As you can see I've already changed the default HashSet to a List.
To save all stuff to the database I do:
inquerito.Pergunta.Add(pergunta);
db.Pergunta.Add(pergunta);
db.Inquerito.Add(inquerito);
The problem is the insertion order is lost.
After adding all "Pergunta" that I want I do:
// Grava alterações e desconecta da base de dados.
db.SaveChanges();
Inquerito inquerito1 = db.Inquerito.Find(inquerito.id);
if (inquerito1 != null)
{
foreach (Pergunta p in inquerito1.Pergunta.ToList())
{
System.Diagnostics.Debug.WriteLine("__pergunta: " + p.descricao);
}
}
db = new quest_geralEntities();
inquerito1 = db.Inquerito.Find(inquerito.id);
if (inquerito1 != null)
{
foreach (Pergunta p in inquerito1.Pergunta.ToList())
{
System.Diagnostics.Debug.WriteLine("__pergunta: " + p.descricao);
}
}
So, the first time I print all the "Pergunta" linked to that "Inquerito" everything is shown in the right order (insertion order), but when I update the context, with: "new quest_geralEntities()" when I print it again the insertion order is completely lost.
I've been struggling with this problem for several hours now and I can't find a solution. I hope I've been clear enough to be helped.
Thanks.
If you want to maintain insertion order, I recommend using a field containing an int that increments. You can add an int identity column without it being the primary key and use that column to sort on, maintaining your insertion order.
You can try to change the property type of your relation :
public virtual IList<Inquerito> Inquerito { get; set; }
public virtual IList<Pergunta> Pergunta { get; set; }
You'll need to regenerate the database schema. I'm not sure if it's working on EF, but in NHibernate, it works.
Hope it helps !
I am using Entity Framework 4.3 Code First, and I have problem with updating many-to-many relationships.
I defined the following classes:
public abstract class Entity
{
[Column(Order = 0)]
[Key]
public int Id { get; set; }
[Timestamp]
[Column(Order = 1)]
public byte[] Version { get; set; }
}
public class Video : Entity
{
public string Title { get; set; }
public string Description { get; set; }
public TimeSpan Length { get; set; }
public virtual ICollection<Coworker> Coworkers { get; set; }
}
public class Coworker : Entity
{
public string FirstName { get; set; }
public string LastName { get; set; }
public virtual ICollection<Video> Videos { get; set; }
}
When the database is created, the schema look right:
There is a Videos, Coworkers and VideoCoworkers table too, without
I use repository pattern in an N-Tier application to access database, my Insert and Update method looks like this:
public T Insert(T entity)
{
//Creates database context. When it disposes, it calls context.SaveChanges()
using (var session = new DatabaseSession())
{
session.Context.Set<T>().Add(entity);
}
}
public T Update(T entity)
{
//Creates database context. When it disposes, it calls context.SaveChanges()
using (var session = new DatabaseSession())
{
entity = session.Context.Set<T>().Attach(entity);
session.Context.Entry(entity).State = EntityState.Modified;
}
return entity;
}
When I update an entity, I create the entity object from a DTO, that's why use DbSet.Attach instead of selecting it and updating the properties one-by-one.
When I initialize the database, I add some test data:
Create 3 Coworkers, where I set first and last name. (A, B, C)
Create 3 Videos, where I set title, description and length, and also set some coworkers. First video has A,B, second has B,C and third has A,C.
When I list the Videos from code, I can see that Video.Coworkers collection is filled with good values, and when I query the link table (VideoCoworkers) in SQL Server Management Studio, it also looks good.
My problem is
when I update for example the title of the Video, it works. But when I try to delete from Video2 the existing coworkers (B and C), and try to add coworker A, then the relationship is not updated. It also does not work when I only try to add new coworker, or only try to delete one. I create the entity which is used as the parameter of the Update() method by creating a new Video entity with a new collection of Coworkers (which are selected from the database with Find() method by Id).
What is the correct way to update many-to-many relationships?
But when I try to delete from Video2 the existing coworkers (B and C),
and try to add coworker A, then the relationship is not updated.
Without using a generic repository the correct procedure would be:
using (var session = new DatabaseSession())
{
video2 = session.Context.Set<Video>().Include(v => v.Coworkers)
.Single(v => v.Id == video2Id);
coworkerA = new Coworker { Id = coworkerAId };
session.Context.Set<Coworker>().Attach(coworkerA);
video2.Coworkers.Clear();
video2.Coworkers.Add(coworkerA)
session.Context.SaveChanges();
}
The essential part is that you must load or attach the entity in its original state, change the entity, i.e. remove and add children, and then save the changes. EF's change detection will create the necessary INSERT and DELETE statements for the link table entries. The simple procedure to set the state to Modified you are trying in your generic Update method is suited only for updating scalar properties - like changing the video title - but won't work for updating relationships between entities.
For solve this problem:
attach the entity to context
load the collection(the collection is not loaded, because )
change the state of entity to modified
save changes
So your code for update should be like this:
public Video Update(Video entity)
{
//Creates database context. When it disposes, it calls context.SaveChanges()
using (var session = new DatabaseSession())
{
entity = session.Context.Set<Video>().Attach(entity);
session.Context.Entry(entity).Collection(p => p.Coworkers).Load();
session.Context.Entry(entity).State = EntityState.Modified;
}
return entity;
}
Please refer here to see how to save master detail in asp.net mvc with database first. Hopefully it will give you the idea about the code first. You may also have a look at knokout.js example