We Keep Coding

How do I encrypt data in Entity Framework Code First?

I've been trying and failing to figure out a good approach to encrypting SQL data with the Entity Framework Code First. I must preface this with that I am hosting in Azure and do not have access to native SQL encryption.
Taking a page from SecurEntity, I have fully implemented an approach that utilizes SaveChanges and ObjectMaterialized to handle the encrypting/decrypting of the entities, but in testing I have found this has been far too unreliable to use.
Here is a sample of some of the implementation:
public override int SaveChanges()
{
var pendingEntities = ((IObjectContextAdapter)this).ObjectContext.ObjectStateManager
.GetObjectStateEntries(EntityState.Added | EntityState.Modified)
.Where(en => !en.IsRelationship).ToList();
foreach (var entry in pendingEntities) //Encrypt all pending changes
EncryptEntity(entry.Entity);
int result = base.SaveChanges();
foreach (var entry in pendingEntities) //Decrypt updated entities for continued use
DecryptEntity(entry.Entity);
return result;
}
void ObjectMaterialized(object sender, ObjectMaterializedEventArgs e)
{
DecryptEntity(e.Entity);
}
I've seen other posts that manually encrypt/decrypt via secondary properties, like so:
public Value { get; set; }
[NotMapped]
public DecryptedValue
{
get { return Decrypt(this.Value); }
set { this.Value = Encrypt(value); }
}
This will most definitely work, but I find this approach to be... less than ideal. When using this approach, all the developers have to wade through all the encrypted properties to find which ones they can use.
The most ideal solution would be for me to be able to override the getting/setting of each value at the data-access level. Is there a way to do this? If not, how can I implement data encryption with Entity Framework - Code First so that it will be easy to maintain and work with?

I have good news. The instability I was experiencing with the SaveChanges/ObjectMaterialized approach was due to the fact that DetectChanges() isn't called until the DbContext actually performs the save.
I was able to fix this by calling DetectChanges() before I pulled the Added/Modified records from the ObjectStateManager. This cleared up any object state oddities that were causing inconsistent encryption behavior.
The resulting code being:
public override int SaveChanges()
{
var contextAdapter = ((IObjectContextAdapter)this);
contextAdapter.ObjectContext.DetectChanges();
var pendingEntities = contextAdapter.ObjectContext.ObjectStateManager
.GetObjectStateEntries(EntityState.Added | EntityState.Modified)
.Where(en => !en.IsRelationship).ToList();
foreach (var entry in pendingEntities) //Encrypt all pending changes
EncryptEntity(entry.Entity);
int result = base.SaveChanges();
foreach (var entry in pendingEntities) //Decrypt updated entities for continued use
DecryptEntity(entry.Entity);
return result;
}
EDIT - Added a sample DataContext to see my end-to-end solution for encrypting all entities. Note: You don't have to use a custom attribute for encrypting properties. Source

Enity Framework updating with a child table causing errors

I have the following update function
public void UpdateBatchDefinition(BatchDefinition batchToUpdate)
{
if (batchToUpdate == null)
{
throw new ArgumentNullException("batchToUpdate");
}
BatchDefinition foundDefinition =
this.context.BatchDefinitions.SingleOrDefault(definition => definition.Id == batchToUpdate.Id);
if (foundDefinition != null)
{
if (!string.IsNullOrWhiteSpace(batchToUpdate.Name))
{
foundDefinition.Name = batchToUpdate.Name;
}
if (!string.IsNullOrWhiteSpace(batchToUpdate.Description))
{
foundDefinition.Description = batchToUpdate.Description;
}
if (!string.IsNullOrWhiteSpace(batchToUpdate.LoadType))
{
foundDefinition.LoadType = batchToUpdate.LoadType;
}
if (batchToUpdate.JobId != Guid.Empty)
{
foundDefinition.JobId = batchToUpdate.JobId;
}
foundDefinition.Tables = batchToUpdate.Tables;
this.context.SaveChanges();
}
}
the issue I am having Is when I am trying to update the Tables list. Tables is a List of Table and Table is a Entity of another table
Tables could be added to, removed from or left alone. I need to update that with what ever is being passed in
when I run this right now I get an 'EntityValidationErrors' error, though it wont tell me what the validation issue actually is.
on Inserting I got the same error but was able to fix it using the following
var underlyingContext = this.context as DbContext;
if (underlyingContext != null)
{
foreach (var table in batchDefinition.Tables)
{
// Need to mark the table entity as unchanged or
// else EF will treat it as a new table
underlyingContext.Entry(table).State = EntityState.Unchanged;
}
}
so I tried using that in this update function
var underlyingContext = this.context as DbContext;
if (underlyingContext != null)
{
foreach (var table in foundDefinition.Tables)
{
// Need to mark the table entity as unchanged or
//else EF will treat it as a new table
underlyingContext.Entry(table).State = EntityState.Unchanged;
}
}
foundDefinition.Tables = batchToUpdate.Tables;
and I get the following error instead:
AcceptChanges cannot continue because the object's key values conflict
with another object in the ObjectStateManager. Make sure that the key
values are unique before calling AcceptChanges.
Any thoughts one what I am missing here?

Change end of your update method like this:
foreach (var t in foundDefinition.Tables.ToList())
Context.Tables.Remove(t);
foundDefinition.Tables = batchToUpdate.Tables;
this.context.SaveChanges();
And about your last error, it is said that there are some duplicates in your context. So, EF can't save the context changes into the db (because there are duplicates in the context!)
In fact, I don't know the last error is from add or delete - you didn't mention clearly. So, I don't know the last two code samples are from your add method, or your update method...
However for update, the trick I mentioned here, must solve your problem for update...

Adding and deleting many-to-many using DbContext API

I am using Entity Framework and DbContext API do build my application but I am having trouble working with objects with many-to-many relations. A simplified save-method could look like this
public void MyObj_Save(MyObj myobj)
{
DbContext.Entry(myobj).State = EntityState.Added;
DbContext.SaveChanges();
}
This code works fine, but if MyObj contains a many-to-many relation this is not saved. I know from using the old POCO API, that I needed to attach the related objects to the context but I cannot find a way to do this correctly with the DbContext API - a simplified example below
public void MyObj_Save(MyObj myobj, List<OtherObj> otherObjList)
{
foreach (OtherObj otherObj in otherObjList)
{
DbContext.OtherObj.Attach(otherObj);
myobj.OtherObj.Add(otherObj);
}
DbContext.Entry(myobj).State = EntityState.Added;
DbContext.SaveChanges();
}
I get no error, but the relations are not saved. What to do?

I quote your (important!) comment:
The objects I send to the method are attached and EntityState is
Unchanged. The configuration of my DbContext is, that I have disabled
AutoDetectChangesEnabled...
So, your code would look like this:
DbContext.Configuration.AutoDetectChangesEnabled = false;
DbContext.Entry(myobj).State = EntityState.Unchanged;
foreach (OtherObj otherObj in otherObjList)
DbContext.Entry(otherObj).State = EntityState.Unchanged;
// entering MyObj_Save method here
foreach (OtherObj otherObj in otherObjList)
{
//DbContext.OtherObj.Attach(otherObj); // does not have an effect
myobj.OtherObj.Add(otherObj);
}
DbContext.Entry(myobj).State = EntityState.Added;
DbContext.SaveChanges();
And this indeed doesn't work because EF doesn't notice that you have changed the relationship between myobj and the list of OtherObj in the line myobj.OtherObj.Add(otherObj); because you have disabled automatic change detection. So, no entries will be written into the join table. Only myobj itself will be saved.
You cannot set any state on an entity to put the state manager into a state that the relationship is saved because it is not an entity state which is important here but a relationship state. These are separate entries in the object state manager which are created and maintained by change detection.
I see three solution:
Set DbContext.Configuration.AutoDetectChangesEnabled = true;
Call DetectChanges manually:
//...
DbContext.Entry(myobj).State = EntityState.Added;
DbContext.ChangeTracker.DetectChanges();
DbContext.SaveChanges();
Detach the new myobj from the context before you set it into Added state (this feels very hacky to me):
// entering MyObj_Save method here
DbContext.Entry(myobj).State = EntityState.Detached;
foreach (OtherObj otherObj in otherObjList)
//...
Maybe it is possible - by getting to the ObjectContext through the IObjectContextAdapter - to modify the relationship entries in the object state manager manually but I don't know how.
In my opinion, this procedure to manipulate entity (and relationship) states manually is not the way you are supposed to work with EF. AutoDetectChangesEnabled has been introduced to make working with EF easier and safer and the only recommended situation to disable it is a high performance requirement (for example for bulk inserts). If you disable automatic change detection without need you are running into problems like this which are difficult to detect and it requires advanced knowledge of EF's inner workings to fix those bugs.

public void MyObj_Save(MyObj myobj, List<OtherObj> otherObjList)
{
DbContext.Entry(myobj).State = EntityState.Added;
foreach (OtherObj otherObj in otherObjList)
{
(((System.Data.Entity.Infrastructure.IObjectContextAdapter)DbContext)
.ObjectContext)
.ObjectStateManager
.ChangeRelationshipState(myobj, otherObj,
q => q.OtherObjs, EntityState.Added);
}
DbContext.SaveChanges();
}
Again, it is a simplified and not a real life example!

How to save combined (new+modified) detached entities in Entity Framework?

What is the proper and fast way to save combined new and modified detached POCO entities?
I was thinking about these methods:
private void Method_2(IList<Entity> entities) //detached entities
{
//This method is using SELECT to check if entity exist
using (var context = new ModelContainer())
{
foreach (Entity entity in entities)
{
var foundEntity = context.CreateObjectSet<Entity>().SingleOrDefault(t => t.Id == entity.Id);
context.Detach(foundEntity); //Remove it from ObjectStateManager
if (foundEntity != null)//It is modified entity
{
context.AttachTo("EntitySet", entity); //Attach our entity
context.ObjectStateManager.ChangeObjectState(entity, EntityState.Modified); //We know it exists
}
else//It is new entity
{
context.CreateObjectSet<Entity>().AddObject(entity);
}
}
context.SaveChanges();
}
}
private void Method_1(IList<Entity> entities) //detached entities
{
//This method doesn't select anything from DB, but i have ta call Savechanges after each object
using (var context = new ModelContainer())
{
foreach (Entity entity in entities)
{
try
{
context.AttachTo("EntitySet", entity);
context.ObjectStateManager.ChangeObjectState(entity, EntityState.Modified);
context.SaveChanges();
}
catch (OptimisticConcurrencyException)
{
context.ObjectStateManager.ChangeObjectState(entity, EntityState.Added);
context.SaveChanges();
}
}
}
}
When you are working in detached environment you have to know which entity was added and which is modified - it is your responsibility to keep this information and provide it to ObjectContext.
Well i agree with this statement if you found yourself in situation when you need to use EF code like this in EF definitely something is wrong with you decision. I have chosen wrong tool for this job.

When you are working in detached environment you have to know which entity was added and which is modified - it is your responsibility to keep this information and provide it to ObjectContext.
The very easy way is:
foreach (var entity in entities)
{
if (entity.Id == 0) // 0 = default value: means new entity
{
// Add object
}
else
{
// Attach object and set state to modified
}
}
The example requires that you have some db auto-generated primary key (Id).
Your Method 2 is possible with some modifications. It is not needed to detach entity when you load it. Instead use ApplyCurrentValues. The approach with loading entity first is very usefull when you decide to work with object graphs instead of single entity. But in the case of object graph you have to do synchronization manually. ApplyCurrentValues works only for scalar (non navigation) properties. You can try to futher optimize your method to load needed enitites in single roundtrip to database instead of loading entities one by one.
Your Method 1 is terrible solution. Using exceptions raised on database server to control program flow is bad approach.

I agree with #Ladislav - Method_1 is a bad approach. Let the database raise exceptions which are caught by EF - don't try and swallow these exceptions yourself.
Your on the right track with Method 1.
Here is how i do it - as i also have a detached context (POCO's, no change tracking, ASP.NET MVC).
BLL Interface: (note i have TPT in my model, hence generics. "Post" is abstract)
void Add(Post post);
void Update<TPost>(TPost post) where TPost : Post, new();
The new() constraint is crucial - you'll see why shortly.
I won't show how i do "Add", because it's simple as you think - AddObject(entity);
The "Update" is the tricky part:
public class GenericRepository<T> : IRepository<T> where T : class
{
public void Update<T2>(T2 entity) where T2: class, new()
{
var stub = new T2(); // create stub, now you see why we need new() constraint
object entityKey = null;
// ..snip code to get entity key via attribute on all domain entities
// once we have key, set on stub.
// check if entity is already attached..
ObjectStateEntry entry;
bool attach;
if (CurrentContext.ObjectStateManager.TryGetObjectStateEntry(CurrentContext.CreateEntityKey(CurrentContext.GetEntityName<T>(), stub), out entry))
{
// Re-attach if necessary.
attach = entry.State == EntityState.Detached;
}
else
{
// Attach for first time.
attach = true;
}
if (attach)
CurrentEntitySet.Attach(stub as T);
// Update Model. (override stub values attached to graph)
CurrentContext.ApplyCurrentValues(CurrentContext.GetEntityName<T>(), entity);
}
}
And that works for me.
As for the entity key, i have used attributes on my domain classes. An alternative (which i'm about to move to), is have all my domain entities implement an interface, which specifies that all domain entities must have a property called "EntityKey". Then i'll use that interface on my constraints. Basically, i needed a dynamic way to create stub entities in a generic repository.
I don't personally like the idea of "checking the ID, if its > 0 then it's an update". Because i'm working with ASP.NET MVC, if i (or another developer) forgets to bind the ID to the View, it won't be passed through, so even though it may be an update, because the ID == 0 it will be added.
I like to be explicit about the operations. This way, i can perform Add/Update seperate validation logic.

Perhaps take a look at Self Tracking POCO entities. IMHO they are perfect for any scenario that requires the entity to be separated from the context. It takes care of all the plumbing code for you.

Add a relationship entity in generic/abstract way

I'm trying to write a WCF method that will receive a detached EntityObject from the client and will be able to tell which properties and which relationships were changed for this entity comparing it with what's already in the context.
Of course that if this entity is a new entity or one of its relationships were added/deleted/modified it should also recognize that and act accordingly.
I'm already able to recognize if the entity's relationship is a new one but can't seem to be able to add it correctly. -With every approach I try I get a different exception.
Here is the method I use to update the detached object:
public static void AttachUpdated(this ObjectContext context, EntityObject objectDetached)
{
if (objectDetached.EntityState == EntityState.Detached)
{
object currentEntityInDb = null;
if (context.TryGetObjectByKey(objectDetached.EntityKey, out currentEntityInDb))
{
context.ApplyPropertyChanges(objectDetached.EntityKey.EntitySetName, objectDetached);
//Apply property changes to all referenced entities in context
context.ApplyReferencePropertyChanges((IEntityWithRelationships)objectDetached,
(IEntityWithRelationships)currentEntityInDb); //Custom extensor method
}
else
{
//The entity should be added
//?????
}
}
}
And this is a method I use to update the entity's relationships:
public static void ApplyReferencePropertyChanges(this ObjectContext context,
IEntityWithRelationships newEntity,
IEntityWithRelationships oldEntity)
{
foreach (var oldRelatedEnd in oldEntity.RelationshipManager.GetAllRelatedEnds())
{
var oldRef = oldRelatedEnd as EntityReference;
if (oldRef != null)
{
// this related end is a reference not a collection
var newRef = newEntity.RelationshipManager.GetRelatedEnd(oldRef.RelationshipName, oldRef.TargetRoleName) as EntityReference;
if (newRef.EntityKey != null)
{
oldRef.EntityKey = newRef.EntityKey;
}
else
{
//When oldRed is a 1:Many relationship
//newRef is an EntityReference<TEntity> object
EntityObject entity = newRef.GetType().GetProperty("Value").GetValue(newRef, null) as EntityObject;
oldRef.EntityKey = entity.EntityKey;
}
}
else
{
IRelatedEnd newRelatedEnd = newEntity.RelationshipManager.GetRelatedEnd(oldRelatedEnd.RelationshipName, oldRelatedEnd.TargetRoleName);
foreach (IEntityWithRelationships e in newRelatedEnd)
{
if (!oldRelatedEnd.Contains((e as IEntityWithKey).EntityKey))
{
//this is a new relation and it needs to be added.
//???????
}
else
{
//Find out if relation was modified - and update it if needed
//????????
}
}
IEnumerable entities = oldRelatedEnd as IEnumerable;
}
}
}
How should it be implemented?
Please help :(

Where is your ObjectContext coming from? (I am assuming that this is your Entity Framework database reference)
There may be two problems here:
Firstly, I do not think that ObjectContext is serializable, so if you are sending it to the client and then sending it back, you will get an error.
Secondly, If you are keeping ObjectConext on the server, the server objects are by default per call, not per session, therefore you will be trying to associate your Entity with a new ObjectContext.
In our projects we map Entity framework Objects to data transfer objects in order to send then over WCF. What you are trying to do may be easier (possible?) in the next version of Entity Framework.

I'm not sure what you want to achieve - if you want to save the changes in the server side you can use ADO .Net Data Services. Is that the case?

See Perseus:
Perseus is a small project designed to
explore ways for exchanging graphs of
Entity Framework entities over WCF web
services. The key piece of the project
is EntityBag which stores a graph
of entities along with change tracking
information. Here's hoping no one will
use this to store & transport
something as nasty as Medusa's head.
;-)