I've been trying and failing to figure out a good approach to encrypting SQL data with the Entity Framework Code First. I must preface this with that I am hosting in Azure and do not have access to native SQL encryption.
Taking a page from SecurEntity, I have fully implemented an approach that utilizes SaveChanges and ObjectMaterialized to handle the encrypting/decrypting of the entities, but in testing I have found this has been far too unreliable to use.
Here is a sample of some of the implementation:
public override int SaveChanges()
{
var pendingEntities = ((IObjectContextAdapter)this).ObjectContext.ObjectStateManager
.GetObjectStateEntries(EntityState.Added | EntityState.Modified)
.Where(en => !en.IsRelationship).ToList();
foreach (var entry in pendingEntities) //Encrypt all pending changes
EncryptEntity(entry.Entity);
int result = base.SaveChanges();
foreach (var entry in pendingEntities) //Decrypt updated entities for continued use
DecryptEntity(entry.Entity);
return result;
}
void ObjectMaterialized(object sender, ObjectMaterializedEventArgs e)
{
DecryptEntity(e.Entity);
}
I've seen other posts that manually encrypt/decrypt via secondary properties, like so:
public Value { get; set; }
[NotMapped]
public DecryptedValue
{
get { return Decrypt(this.Value); }
set { this.Value = Encrypt(value); }
}
This will most definitely work, but I find this approach to be... less than ideal. When using this approach, all the developers have to wade through all the encrypted properties to find which ones they can use.
The most ideal solution would be for me to be able to override the getting/setting of each value at the data-access level. Is there a way to do this? If not, how can I implement data encryption with Entity Framework - Code First so that it will be easy to maintain and work with?
I have good news. The instability I was experiencing with the SaveChanges/ObjectMaterialized approach was due to the fact that DetectChanges() isn't called until the DbContext actually performs the save.
I was able to fix this by calling DetectChanges() before I pulled the Added/Modified records from the ObjectStateManager. This cleared up any object state oddities that were causing inconsistent encryption behavior.
The resulting code being:
public override int SaveChanges()
{
var contextAdapter = ((IObjectContextAdapter)this);
contextAdapter.ObjectContext.DetectChanges();
var pendingEntities = contextAdapter.ObjectContext.ObjectStateManager
.GetObjectStateEntries(EntityState.Added | EntityState.Modified)
.Where(en => !en.IsRelationship).ToList();
foreach (var entry in pendingEntities) //Encrypt all pending changes
EncryptEntity(entry.Entity);
int result = base.SaveChanges();
foreach (var entry in pendingEntities) //Decrypt updated entities for continued use
DecryptEntity(entry.Entity);
return result;
}
EDIT - Added a sample DataContext to see my end-to-end solution for encrypting all entities. Note: You don't have to use a custom attribute for encrypting properties. Source
Related
Experiencing an issue about updating mysql DB through EF. It's not the first time I'm dealing with it, so I had some ideas about why isn't my data getting changed. I tried changing an element in goods array; tried editing an object, recieved through LINQ-request (seen some examples of this method); made some attempts on marking element found in the database before editing (like EntityState and Attach()). Nothing of these made any difference, so I tried removing <asp:UpdatePanel> from Site.Master to see what happens (responsive for postback blocking to prevent page shaking on update), but nothing changed (while btnRedeemEdit.IsPostBack having its default value).
Code below is the function I use for updates.
protected void btnRedeemEdit_Click(object sender, EventArgs e)
{
if (!string.IsNullOrEmpty(Request.QueryString["id"]))
{
var db = new GoodContext();
var goods = db.Goods.ToList();
Good theGood = goods.FirstOrDefault(x => x.Id == int.Parse(Request.QueryString["id"]));
//db.Goods.Attach(theGood);//No effect
//db.Entry(theGood).State = System.Data.Entity.EntityState.Modified; //No effect
if (theGood != default)
{
theGood.AmountSold = GetInput().AmountSold;
theGood.APF = GetInput().APF;
theGood.Barcode = GetInput().Barcode;
theGood.Description = GetInput().Description;
theGood.ImagesUrl = GetInput().ImagesUrl;//"https://i.pinimg.com/564x/2d/b7/d8/2db7d8c53b818ce838ad8bf6a4768c71.jpg";
theGood.Name = GetInput().Name;
theGood.OrderPrice = GetInput().OrderPrice;
theGood.Profit = GetInput().Profit;
theGood.RecievedOn = GetInput().RecievedOn;//DateTime.Parse(GetInput().RecievedOn).Date.ToString();
theGood.TotalAmount = GetInput().TotalAmount;
theGood.WeightKg = GetInput().WeightKg;
//SetGoodValues(goods[editIndex],GetInput());//Non-working
db.SaveChanges();
Response.Redirect("/AdminGoods");
}
else Response.Write($"<script>alert('Good on ID does not exist');</script>");
}
else Response.Write($"<script>alert('Unable to change: element selected does not exist');</script>");
}
Notice, that no alerts appear during execution, so object in database can be found.
Are there any more things, that can be responsible for blocking database updates?
A few things to update & check:
Firstly, DbContexts should always be disposed, so in your case wrap the DbContext inside a using statement:
using (var db = new GoodContext())
{
// ...
}
Next, there is no need to load all goods from the DbContext, just use Linq to retrieve the one you want to update:
using (var db = new GoodContext())
{
Good theGood = db.Goods.SingleOrDefault(x => x.Id == int.Parse(Request.QueryString["id"]));
if (theGood is null)
{
Response.Write($"<script>alert('Good on ID does not exist');</script>");
return;
}
}
The plausible suspect is what does "GetInput()" actually do, and have you confirmed that it actually has the changes you want? If GetInput is a method that returns an object containing your changes then it only needs to be called once rather than each time you set a property:
(Inside the using() {} scope...)
var input = GetInput();
theGood.AmountSold = input.AmountSold;
theGood.APF = input.APF;
theGood.Barcode = input.Barcode;
theGood.Description = input.Description;
// ...
db.SaveChanges();
If input has updated values but after calling SaveChanges you aren't seeing updated values in the database then there are two things to check.
1) Check that the database connection string at runtime matches the database that you are checking against. The easiest way to do that is to get the connection string from the DbContext instance's Database.
EF 6:
using (var db = new GoodContext())
{
var connectionString = db.Database.Connection.ConnectionString; // Breakpoint here and inspect.
EF Core: (5/6)
using (var db = new GoodContext())
{
var connectionString = db.Database.GetConnectionString();
Often at runtime the DbContext will be initialized with a connection string from a web.config / .exe.config file that you don't expect so you're checking one database expecting changes while the application is using a different database / server. (More common than you'd expect:)
2) Check that you aren't disabling tracking proxies. By default EF will enable change tracking which is how it knows if/when data has changed for SaveChanges to generate SQL statements. Sometimes developers will encounter performance issues and start looking for ways to speed up EF including disabling change tracking on the DbContext. (A fine option for read-only systems, but a pain for read-write)
EF6 & EF Core: (DbContext initialization)
Configuration.AutoDetectChangesEnabled = false; // If you have this set to false consider removing it.
If you must disable change tracking then you have to explicitly set the EntityState of the entity to Modified before calling SaveChanges():
db.Entry(theGood).State = EntityState.Modified;
db.SaveChanges();
Using change tracking is preferable to using EntityState because with change tracking EF will only generate an UPDATE statement if any values have changed, and only for the values that changed. With EntityState.Modified EF will always generate an UPDATE statement for all non-key fields regardless if any of them had actually changed or not.
I have some code, you can check project github, error contains in UploadContoller method GetExtensionId.
Database diagram:
Code (in this controller I sending files to upload):
[HttpPost]
public ActionResult UploadFiles(HttpPostedFileBase[] files, int? folderid, string description)
{
foreach (HttpPostedFileBase file in files)
{
if (file != null)
{
string fileName = Path.GetFileNameWithoutExtension(file.FileName);
string fileExt = Path.GetExtension(file.FileName)?.Remove(0, 1);
int? extensionid = GetExtensionId(fileExt);
if (CheckFileExist(fileName, fileExt, folderid))
{
fileName = fileName + $" ({DateTime.Now.ToString("dd-MM-yy HH:mm:ss")})";
}
File dbFile = new File();
dbFile.folderid = folderid;
dbFile.displayname = fileName;
dbFile.file_extensionid = extensionid;
dbFile.file_content = GetFileBytes(file);
dbFile.description = description;
db.Files.Add(dbFile);
}
}
db.SaveChanges();
return RedirectToAction("Partial_UnknownErrorToast", "Toast");
}
I want to create Extension in database if it not exist yet. And I do it with GetExtensionId:
private static object locker = new object();
private int? GetExtensionId(string name)
{
int? result = null;
lock (locker)
{
var extItem = db.FileExtensions.FirstOrDefault(m => m.displayname == name);
if (extItem != null) return extItem.file_extensionid;
var fileExtension = new FileExtension()
{
displayname = name
};
db.FileExtensions.Add(fileExtension);
db.SaveChanges();
result = fileExtension.file_extensionid;
}
return result;
}
In the SQL Server database I have unique constraint on displayname column of FileExtension.
Problem starts only if I uploading few files with the same extension and this extension not exist in database yet.
If I remove lock, in GetExtensionId will be Exception about unique constraint.
Maybe, for some reason, next iteration of foreach cycle calls GetExtensionId without waiting? I don't know.
But only if I set lock my code works fine.
If you know why it happens please explain.
This sounds like a simple concurrency race condition. Imagine two requests come in at once; they both check the FirstOrDefault, which correctly says "nope" for both. Then they both try and insert; one wins, one fails because the DB has changed. While EF manages transactions around SaveChanges, that transaction doesn't start from when you query the data initially
The lock appears to work, by preventing them getting into the looking code at the same time, but this is not a reliable solution for this in general, as it only works inside a single process, let alone node.
So: a few option here:
your code could detect the foreign key violation exception and recheck from the start (FirstOrDefault etc), which keeps things simple in the success case (which is going to be the majority of the time) and not horribly expensive in the failure case (just an exception and an extra DB hit) - pragmatic enough
you could move the "select if exists, insert if it doesn't" into a single operation inside the database inside a transaction (ideally serializable isolation level, and/or using the UPDLOCK hint) - this requires writing TSQL yourself, rather than relying on EF, but minimises round trips and avoids writing "detect failure and compensate" code
you could perform the selects and possible inserts inside a transaction via EF - complicated and messy, frankly: don't do this (and it would again need to be serializable isolation level, but now the serializable transaction spans multiple round trips, which can start to impact locking, if at scale)
I've been attempting to refactor some EF6 code to EF Core 1 and have hit a small stumbling block. The code I'm attempting to convert is here:
https://github.com/mehdime/DbContextScope
Everything is mostly fine but DbContextScope.cs in particular is proving tricky, e.g. this method (edited for brevity):
public void RefreshEntitiesInParentScope(IEnumerable entities)
{
foreach (IObjectContextAdapter contextInCurrentScope in
_dbContexts.InitializedDbContexts.Values)
{
var correspondingParentContext =
_parentScope._dbContexts.InitializedDbContexts.Values
.SingleOrDefault(parentContext =>
parentContext.GetType() == contextInCurrentScope.GetType())
as IObjectContextAdapter;
if (correspondingParentContext == null)
continue;
foreach (var toRefresh in entities)
{
ObjectStateEntry stateInCurrentScope;
if (contextInCurrentScope.ObjectContext.ObjectStateManager
.TryGetObjectStateEntry(toRefresh, out stateInCurrentScope))
{
var key = stateInCurrentScope.EntityKey;
ObjectStateEntry stateInParentScope;
if (correspondingParentContext.ObjectContext.ObjectStateManager
.TryGetObjectStateEntry(key, out stateInParentScope))
{
if (stateInParentScope.State == EntityState.Unchanged)
{
correspondingParentContext.ObjectContext.Refresh(
RefreshMode.StoreWins, stateInParentScope.Entity);
}
}
}
}
}
}
Questions.
Firstly, I know I can replace ObjectContext.ObjectStateManager with the new ChangeTracker but want to ensure that the entry I obtain is obtained correctly.How would the following line translate in EF Core?
contextInCurrentScope.ObjectContext.ObjectStateManager
.TryGetObjectStateEntry(toRefresh, out stateInCurrentScope)
Secondly, what is the equivalent of this in EF Core?
correspondingParentContext.ObjectContext.Refresh
Thanks!
P.s. There are many helpful comments in the source at the GitHub repo above.
I think the correct way to get an entity's entry, and consequently it's keys and state, is via:
var entry = contextInCurrentScope.Entry(toRefresh);
var keys = entry.Metadata.GetKeys();
var state = entry.State;
you can also refresh a single entity from the database using the entry as follows:
entry.Reload();
What is the proper and fast way to save combined new and modified detached POCO entities?
I was thinking about these methods:
private void Method_2(IList<Entity> entities) //detached entities
{
//This method is using SELECT to check if entity exist
using (var context = new ModelContainer())
{
foreach (Entity entity in entities)
{
var foundEntity = context.CreateObjectSet<Entity>().SingleOrDefault(t => t.Id == entity.Id);
context.Detach(foundEntity); //Remove it from ObjectStateManager
if (foundEntity != null)//It is modified entity
{
context.AttachTo("EntitySet", entity); //Attach our entity
context.ObjectStateManager.ChangeObjectState(entity, EntityState.Modified); //We know it exists
}
else//It is new entity
{
context.CreateObjectSet<Entity>().AddObject(entity);
}
}
context.SaveChanges();
}
}
private void Method_1(IList<Entity> entities) //detached entities
{
//This method doesn't select anything from DB, but i have ta call Savechanges after each object
using (var context = new ModelContainer())
{
foreach (Entity entity in entities)
{
try
{
context.AttachTo("EntitySet", entity);
context.ObjectStateManager.ChangeObjectState(entity, EntityState.Modified);
context.SaveChanges();
}
catch (OptimisticConcurrencyException)
{
context.ObjectStateManager.ChangeObjectState(entity, EntityState.Added);
context.SaveChanges();
}
}
}
}
When you are working in detached environment you have to know which entity was added and which is modified - it is your responsibility to keep this information and provide it to ObjectContext.
Well i agree with this statement if you found yourself in situation when you need to use EF code like this in EF definitely something is wrong with you decision. I have chosen wrong tool for this job.
When you are working in detached environment you have to know which entity was added and which is modified - it is your responsibility to keep this information and provide it to ObjectContext.
The very easy way is:
foreach (var entity in entities)
{
if (entity.Id == 0) // 0 = default value: means new entity
{
// Add object
}
else
{
// Attach object and set state to modified
}
}
The example requires that you have some db auto-generated primary key (Id).
Your Method 2 is possible with some modifications. It is not needed to detach entity when you load it. Instead use ApplyCurrentValues. The approach with loading entity first is very usefull when you decide to work with object graphs instead of single entity. But in the case of object graph you have to do synchronization manually. ApplyCurrentValues works only for scalar (non navigation) properties. You can try to futher optimize your method to load needed enitites in single roundtrip to database instead of loading entities one by one.
Your Method 1 is terrible solution. Using exceptions raised on database server to control program flow is bad approach.
I agree with #Ladislav - Method_1 is a bad approach. Let the database raise exceptions which are caught by EF - don't try and swallow these exceptions yourself.
Your on the right track with Method 1.
Here is how i do it - as i also have a detached context (POCO's, no change tracking, ASP.NET MVC).
BLL Interface: (note i have TPT in my model, hence generics. "Post" is abstract)
void Add(Post post);
void Update<TPost>(TPost post) where TPost : Post, new();
The new() constraint is crucial - you'll see why shortly.
I won't show how i do "Add", because it's simple as you think - AddObject(entity);
The "Update" is the tricky part:
public class GenericRepository<T> : IRepository<T> where T : class
{
public void Update<T2>(T2 entity) where T2: class, new()
{
var stub = new T2(); // create stub, now you see why we need new() constraint
object entityKey = null;
// ..snip code to get entity key via attribute on all domain entities
// once we have key, set on stub.
// check if entity is already attached..
ObjectStateEntry entry;
bool attach;
if (CurrentContext.ObjectStateManager.TryGetObjectStateEntry(CurrentContext.CreateEntityKey(CurrentContext.GetEntityName<T>(), stub), out entry))
{
// Re-attach if necessary.
attach = entry.State == EntityState.Detached;
}
else
{
// Attach for first time.
attach = true;
}
if (attach)
CurrentEntitySet.Attach(stub as T);
// Update Model. (override stub values attached to graph)
CurrentContext.ApplyCurrentValues(CurrentContext.GetEntityName<T>(), entity);
}
}
And that works for me.
As for the entity key, i have used attributes on my domain classes. An alternative (which i'm about to move to), is have all my domain entities implement an interface, which specifies that all domain entities must have a property called "EntityKey". Then i'll use that interface on my constraints. Basically, i needed a dynamic way to create stub entities in a generic repository.
I don't personally like the idea of "checking the ID, if its > 0 then it's an update". Because i'm working with ASP.NET MVC, if i (or another developer) forgets to bind the ID to the View, it won't be passed through, so even though it may be an update, because the ID == 0 it will be added.
I like to be explicit about the operations. This way, i can perform Add/Update seperate validation logic.
Perhaps take a look at Self Tracking POCO entities. IMHO they are perfect for any scenario that requires the entity to be separated from the context. It takes care of all the plumbing code for you.
I'm trying to write a WCF method that will receive a detached EntityObject from the client and will be able to tell which properties and which relationships were changed for this entity comparing it with what's already in the context.
Of course that if this entity is a new entity or one of its relationships were added/deleted/modified it should also recognize that and act accordingly.
I'm already able to recognize if the entity's relationship is a new one but can't seem to be able to add it correctly. -With every approach I try I get a different exception.
Here is the method I use to update the detached object:
public static void AttachUpdated(this ObjectContext context, EntityObject objectDetached)
{
if (objectDetached.EntityState == EntityState.Detached)
{
object currentEntityInDb = null;
if (context.TryGetObjectByKey(objectDetached.EntityKey, out currentEntityInDb))
{
context.ApplyPropertyChanges(objectDetached.EntityKey.EntitySetName, objectDetached);
//Apply property changes to all referenced entities in context
context.ApplyReferencePropertyChanges((IEntityWithRelationships)objectDetached,
(IEntityWithRelationships)currentEntityInDb); //Custom extensor method
}
else
{
//The entity should be added
//?????
}
}
}
And this is a method I use to update the entity's relationships:
public static void ApplyReferencePropertyChanges(this ObjectContext context,
IEntityWithRelationships newEntity,
IEntityWithRelationships oldEntity)
{
foreach (var oldRelatedEnd in oldEntity.RelationshipManager.GetAllRelatedEnds())
{
var oldRef = oldRelatedEnd as EntityReference;
if (oldRef != null)
{
// this related end is a reference not a collection
var newRef = newEntity.RelationshipManager.GetRelatedEnd(oldRef.RelationshipName, oldRef.TargetRoleName) as EntityReference;
if (newRef.EntityKey != null)
{
oldRef.EntityKey = newRef.EntityKey;
}
else
{
//When oldRed is a 1:Many relationship
//newRef is an EntityReference<TEntity> object
EntityObject entity = newRef.GetType().GetProperty("Value").GetValue(newRef, null) as EntityObject;
oldRef.EntityKey = entity.EntityKey;
}
}
else
{
IRelatedEnd newRelatedEnd = newEntity.RelationshipManager.GetRelatedEnd(oldRelatedEnd.RelationshipName, oldRelatedEnd.TargetRoleName);
foreach (IEntityWithRelationships e in newRelatedEnd)
{
if (!oldRelatedEnd.Contains((e as IEntityWithKey).EntityKey))
{
//this is a new relation and it needs to be added.
//???????
}
else
{
//Find out if relation was modified - and update it if needed
//????????
}
}
IEnumerable entities = oldRelatedEnd as IEnumerable;
}
}
}
How should it be implemented?
Please help :(
Where is your ObjectContext coming from? (I am assuming that this is your Entity Framework database reference)
There may be two problems here:
Firstly, I do not think that ObjectContext is serializable, so if you are sending it to the client and then sending it back, you will get an error.
Secondly, If you are keeping ObjectConext on the server, the server objects are by default per call, not per session, therefore you will be trying to associate your Entity with a new ObjectContext.
In our projects we map Entity framework Objects to data transfer objects in order to send then over WCF. What you are trying to do may be easier (possible?) in the next version of Entity Framework.
I'm not sure what you want to achieve - if you want to save the changes in the server side you can use ADO .Net Data Services. Is that the case?
See Perseus:
Perseus is a small project designed to
explore ways for exchanging graphs of
Entity Framework entities over WCF web
services. The key piece of the project
is EntityBag which stores a graph
of entities along with change tracking
information. Here's hoping no one will
use this to store & transport
something as nasty as Medusa's head.
;-)