We Keep Coding

Entity Framework Code First and SQL Server 2012 Sequences

I was in the middle of implementing a database audit trail whereby CRUD operations performed through my controllers in my Web API project would serialize the old and new poco's and store their values for later retrieval (historical, rollback, etc...).
When I got it all working, I did not like how it made my controllers look during a POST because I ended up having to call SaveChanges() twice, once to get the ID for the inserted entity and then again to commit the audit record which needed to know that ID.
I set out to convert the project (still in its infancy) to use sequences instead of identity columns. This has the added bonus of further abstracting me from SQL Server, though that is not really an issue, but it also allows me to reduce the number of commits and lets me pull that logic out of the controller and stuff it into my service layer which abstracts my controllers from the repositories and lets me do work like this auditing in this "shim" layer.
Once the Sequence object was created and a stored procedure to expose it, I created the following class:
public class SequentialIdProvider : ISequentialIdProvider
{
private readonly IService<SequenceValue> _sequenceValueService;
public SequentialIdProvider(IService<SequenceValue> sequenceValueService)
{
_sequenceValueService = sequenceValueService;
}
public int GetNextId()
{
var value = _sequenceValueService.SelectQuery("GetSequenceIds #numberOfIds", new SqlParameter("numberOfIds", SqlDbType.Int) { Value = 1 }).ToList();
if (value.First() == null)
{
throw new Exception("Unable to retrieve the next id's from the sequence.");
}
return value.First().FirstValue;
}
public IList<int> GetNextIds(int numberOfIds)
{
var values = _sequenceValueService.SelectQuery("GetSequenceIds #numberOfIds", new SqlParameter("numberOfIds", SqlDbType.Int) { Value = numberOfIds }).ToList();
if (values.First() == null)
{
throw new Exception("Unable to retrieve the next id's from the sequence.");
}
var list = new List<int>();
for (var i = values.First().FirstValue; i <= values.First().LastValue; i++)
{
list.Add(i);
}
return list;
}
}
Which simply provides two ways to get IDs, a single and a range.
This all worked great during the first set of unit tests but as soon as I started testing it in a real world scenario, I quickly realized that a single call to GetNextId() would return the same value for the life of that context, until SaveChanges() is called, thus negating any real benefit.
I am not sure if there is a way around this short of creating a second context (not an option) or going old school ADO.NET and making direct SQL calls and use AutoMapper to get to the same net result. Neither of these are appeal to me so I am hoping someone else has an idea.

Don't know if this might help you, but this is how I did my audit log trail using code first.
The following is coded into a class inheriting from DbContext.
in my constructor I have the following
IObjectContextAdapter objectContextAdapter = (this as IObjectContextAdapter);
objectContextAdapter.ObjectContext.SavingChanges += SavingChanges;
This is my saving changes method wired up previously
void SavingChanges(object sender, EventArgs e) {
Debug.Assert(sender != null, "Sender can't be null");
Debug.Assert(sender is ObjectContext, "Sender not instance of ObjectContext");
ObjectContext context = (sender as ObjectContext);
IEnumerable<ObjectStateEntry> modifiedEntities = context.ObjectStateManager.GetObjectStateEntries(EntityState.Modified);
IEnumerable<ObjectStateEntry> addedEntities = context.ObjectStateManager.GetObjectStateEntries(EntityState.Added);
addedEntities.ToList().ForEach(a => {
//Assign ids to objects that don't have
if (a.Entity is IIdentity && (a.Entity as IIdentity).Id == Guid.Empty)
(a.Entity as IIdentity).Id = Guid.NewGuid();
this.Set<AuditLogEntry>().Add(AuditLogEntryFactory(a, _AddedEntry));
});
modifiedEntities.ToList().ForEach(m => {
this.Set<AuditLogEntry>().Add(AuditLogEntryFactory(m, _ModifiedEntry));
});
}
And these are the methods used previosly to build up the audit log details
private AuditLogEntry AuditLogEntryFactory(ObjectStateEntry entry, string entryType) {
AuditLogEntry auditLogEntry = new AuditLogEntry() {
EntryDate = DateTime.Now,
EntryType = entryType,
Id = Guid.NewGuid(),
NewValues = AuditLogEntryNewValues(entry),
Table = entry.EntitySet.Name,
UserId = _UserId
};
if (entryType == _ModifiedEntry) auditLogEntry.OriginalValues = AuditLogEntryOriginalValues(entry);
return auditLogEntry;
}
/// <summary>
/// Creates a string of all modified properties for an entity.
/// </summary>
private string AuditLogEntryOriginalValues(ObjectStateEntry entry) {
StringBuilder stringBuilder = new StringBuilder();
entry.GetModifiedProperties().ToList().ForEach(m => {
stringBuilder.Append(String.Format("{0} = {1},", m, entry.OriginalValues[m]));
});
return stringBuilder.ToString();
}
/// <summary>
/// Creates a string of all modified properties' new values for an entity.
/// </summary>
private string AuditLogEntryNewValues(ObjectStateEntry entry) {
StringBuilder stringBuilder = new StringBuilder();
for (int i = 0; i < entry.CurrentValues.FieldCount; i++) {
stringBuilder.Append(String.Format("{0} = {1},",
entry.CurrentValues.GetName(i), entry.CurrentValues.GetValue(i)));
}
return stringBuilder.ToString();
}
Hopefully this might point you into a direction that might help you solve your problem.

Update Child Collection with EF 4.3 - Losing association in DB

This is driving me nuts. I'm not sure what else to try. This is my latest attempt to update a list of objects within another object using EF 4.3.
The scenario is that a user has added a new Task to an Application that already has one task in its Tasks property. The Application is not attached to the DB context because it was retrieved in a prior logic/DB call. This is the class and property:
public class Application : EntityBase
{
public ObservableCollection<TaskBase> Tasks { // typical get/set code here }
}
This is my attempt to update the list. What happens is that the new Task gets added and the association correctly exists in the DB. However, the first task, that wasn't altered, has its association removed in the DB (its reference to the Application).
This is the Save() method that takes the Application that the user modified:
public void Save(Application newApp)
{
Application appFromContext;
appFromContext = this.Database.Applications
.Include(x => x.Tasks)
.Single(x => x.IdForEf == newApp.IdForEf);
AddTasksToApp(newApp, appFromContext);
this.Database.SaveChanges();
}
And this is the hooey that's apparently necessary to save using EF:
private void AddTasksToApp(Application appNotAssociatedWithContext, Application appFromContext)
{
List<TaskBase> originalTasks = appFromContext.Tasks.ToList();
appFromContext.Tasks.Clear();
foreach (TaskBase taskModified in appNotAssociatedWithContext.Tasks)
{
if (taskModified.IdForEf == 0)
{
appFromContext.Tasks.Add(taskModified);
}
else
{
TaskBase taskBase = originalTasks.Single(x => x.IdForEf == taskModified.IdForEf); // Get original task
this.Database.Entry(taskBase).CurrentValues.SetValues(taskModified); // Update with new
}
}
}
Can anyone see why the first task would be losing its association to the Application in the DB? That first task goes through the else block in the above code.
Next, I'll need to figure out how to delete one or more items, but first things first...

After continual trial and error, this appears to be working, including deleting Tasks. I thought I'd post this in case it helps someone else. I'm also hoping that someone tells me that I'm making this more complicated than it should be. This is tedious and error-prone code to write when saving every object that has a list property.
private void AddTasksToApp(Application appNotAssociatedWithContext, Application appFromContext)
{
foreach (TaskBase taskModified in appNotAssociatedWithContext.Tasks)
{
if (taskModified.IdForEf == 0)
{
appFromContext.Tasks.Add(taskModified);
}
else
{
TaskBase taskBase = appFromContext.Tasks.Single(x => x.IdForEf == taskModified.IdForEf); // Get original task
this.Database.Entry(taskBase).CurrentValues.SetValues(taskModified); // Update with new
}
}
// Delete tasks that no longer exist within the app.
List<TaskBase> tasksToDelete = new List<TaskBase>();
foreach (TaskBase originalTask in appFromContext.Tasks)
{
TaskBase task = appNotAssociatedWithContext.Tasks.Where(x => x.IdForEf == originalTask.IdForEf).FirstOrDefault();
if (task == null)
{
tasksToDelete.Add(originalTask);
}
}
foreach (TaskBase taskToDelete in tasksToDelete)
{
appFromContext.Tasks.Remove(taskToDelete);
this.Database.TaskBases.Remove(taskToDelete);
}
}

Entity Framework - "The relationship between the two objects cannot be defined" error, but I think I'm using the same context

In my ViewModel I have some code like that:
public class OrderViewModel
{
private UserOrder order;
private DeliveryCentre deliveryCentre;
// This is my EF Container
private CatalogueContainer catalogue = new CatalogueContainer();
// do some stuff...
public void Save()
{
if (order == null)
{
order = catalogue.UserOrders.CreateObject();
}
// do some other stuff...
if ((deliveryCentre == null)
|| (deliveryCentre.Id != deliveryCentreId))
{
deliveryCentre = catalogue.DeliveryCentres.First(centre => centre.Id == deliveryCentreId);
//Causes a context error, not sure why...
order.DeliveryCentre= deliveryCentre;
}
catalogue.SaveChanges();
}
So when the delivery centre is new and the order is new, I am hit by the old "The relationship between the two objects cannot be defined because they are attached to different ObjectContext objects" error, which seems a trifle unfair to me - I just can't figure out what I need to do to make them belong more to the same object context. I assume this is due to some fundamental misunderstanding of the behaviour of Entity Framework.

You are not disposing your context. It may be possible that one of the entities order or deliveryCentre is attached to an old context which still holds references to the entities. You can create and dispose your context with an using statement inside of the Save method instead to using it as a member variable:
public void Save()
{
using (var catalogue = new CatalogueContainer())
{
// your code...
}
}
And remove the private catalogue member.

The solution turned out to only be indirectly related to the error message- #Slauma asked about the //do stuff... placeholders and when I commented those out the error disappeared.
It turned out that there was another relationship there, where I was creating the object as this.Item = new Item() rather than using this.Item = catalogue.Items.CreateObject() so it was being created out of context and when it was added to the order, although the order itself was created from the local context, when the Item was added to it this was somehow dirtying up the context but for some reason this only showed up as a problem when I added the next related object.

Is this a good/preferable pattern to Azure Queue construction for a T4 template?

I'm building a T4 template that will help people construct Azure queues in a consistent and simple manner. I'd like to make this self-documenting, and somewhat consistent.
First I made the queue name at the top of the file, the queue names have to be in lowercase so I added ToLower()
The public constructor uses the built-in StorageClient API's to access the connection strings. I've seen many different approaches to this, and would like to get something that works in almost all situations. (ideas? do share)
I dislike the unneeded HTTP requests to check if the queues have been created so I made is a static bool . I didn't implement a Lock(monitorObject) since I don't think one is needed.
Instead of using a string and parsing it with commas (like most MSDN documentation) I'm serializing the object when passing it into the queue.
For further optimization I'm using a JSON serializer extension method to get the most out of the 8k limit. Not sure if an encoding will help optimize this any more
Added retry logic to handle certain scenarios that occur with the queue (see html link)
Q: Is "DataContext" appropriate name for this class?
Q: Is it a poor practice to name the Queue Action Name in the manner I have done?
What additional changes do you think I should make?
public class AgentQueueDataContext
{
// Queue names must always be in lowercase
// Is named like a const, but isn't one because .ToLower won't compile...
static string AGENT_QUEUE_ACTION_NAME = "AgentQueueActions".ToLower();
static bool QueuesWereCreated { get; set; }
DataModel.SecretDataSource secDataSource = null;
CloudStorageAccount cloudStorageAccount = null;
CloudQueueClient cloudQueueClient = null;
CloudQueue queueAgentQueueActions = null;
static AgentQueueDataContext()
{
QueuesWereCreated = false;
}
public AgentQueueDataContext() : this(false)
{
}
public AgentQueueDataContext(bool CreateQueues)
{
// This pattern of setting up queues is from:
// ttp://convective.wordpress.com/2009/11/15/queues-azure-storage-client-v1-0/
//
this.cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
this.cloudQueueClient = cloudStorageAccount.CreateCloudQueueClient();
this.secDataSource = new DataModel.SecretDataSource();
queueAgentQueueActions = cloudQueueClient.GetQueueReference(AGENT_QUEUE_ACTION_NAME);
if (QueuesWereCreated == false || CreateQueues)
{
queueAgentQueueActions.CreateIfNotExist();
QueuesWereCreated = true;
}
}
// This is the method that will be spawned using ThreadStart
public void CheckQueue()
{
while (true)
{
try
{
CloudQueueMessage msg = queueAgentQueueActions.GetMessage();
bool DoRetryDelayLogic = false;
if (msg != null)
{
// Deserialize using JSON (allows more data to be stored)
AgentQueueEntry actionableMessage = msg.AsString.FromJSONString<AgentQueueEntry>();
switch (actionableMessage.ActionType)
{
case AgentQueueActionEnum.EnrollNew:
{
// Add to
break;
}
case AgentQueueActionEnum.LinkToSite:
{
// Link within Agent itself
// Link within Site
break;
}
case AgentQueueActionEnum.DisableKey:
{
// Disable key in site
// Disable key in AgentTable (update modification time)
break;
}
default:
{
break;
}
}
//
// Only delete the message if the requested agent has been missing for
// at least 10 minutes
//
if (DoRetryDelayLogic)
{
if (msg.InsertionTime != null)
if (msg.InsertionTime < DateTime.UtcNow + new TimeSpan(0, 10, 10))
continue;
// ToDo: Log error: AgentID xxx has not been found in table for xxx minutes.
// It is likely the result of a the registratoin host crashing.
// Data is still consistent. Deleting queued message.
}
//
// If execution made it to this point, then we are either fully processed, or
// there is sufficent reason to discard the message.
//
try
{
queueAgentQueueActions.DeleteMessage(msg);
}
catch (StorageClientException ex)
{
// As of July 2010, this is the best way to detect this class of exception
// Description: ttp://blog.smarx.com/posts/deleting-windows-azure-queue-messages-handling-exceptions
if (ex.ExtendedErrorInformation.ErrorCode == "MessageNotFound")
{
// pop receipt must be invalid
// ignore or log (so we can tune the visibility timeout)
}
else
{
// not the error we were expecting
throw;
}
}
}
else
{
// allow control to fall to the bottom, where the sleep timer is...
}
}
catch (Exception e)
{
// Justification: Thread must not fail.
//Todo: Log this exception
// allow control to fall to the bottom, where the sleep timer is...
// Rationale: not doing so may cause queue thrashing on a specific corrupt entry
}
// todo: Thread.Sleep() is bad
// Replace with something better...
Thread.Sleep(9000);
}

Q: Is "DataContext" appropriate name for this class?
In .NET we have a lot of DataContext classes, so in the sense that you want names to appropriately communicate what the class does, I think XyzQueueDataContext properly communicates what the class does - although you can't query from it.
If you want to stay more aligned to accepted pattern languages, Patterns of Enterprise Application Architecture calls any class that encapsulates access to an external system for a Gateway, while more specifically you may want to use the term Channel in the language of Enterprise Integration Patterns - that's what I would do.
Q: Is it a poor practice to name the Queue Action Name in the manner I have done?
Well, it certainly tightly couples the queue name to the class. This means that if you later decide that you want to decouple those, you can't.
As a general comment I think this class might benefit from trying to do less. Using the queue is not the same thing as managing it, so instead of having all of that queue management code there, I'd suggest injecting a CloudQueue into the instance. Here's how I implement my AzureChannel constructor:
private readonly CloudQueue queue;
public AzureChannel(CloudQueue queue)
{
if (queue == null)
{
throw new ArgumentNullException("queue");
}
this.queue = queue;
}
This better fits the Single Responsibility Principle and you can now implement queue management in its own (reusable) class.

What type of method to use for an object Delete or Save?

It may be a subjective question but I want to know what people feel is the best pattern for behavior of an object's save or delete methods?
When you setup a class that will contain Db record information I usualy end up declaring the objects Delete() and Save() methods as void. As these methods usualy take direct action on other properties of the object do we need to return anything else to confirm the action? On fails I just let the error handleing framework or implementig code handle any exception fall out.
I have seen alot of objects that send back a rows affected int or other return, what is the 'expert' opinion on this?
i.e. this:
public void Delete()
{
if (this.objectID == null || this.objectID == -1) { throw new exNotDbObject() }
//Do dB operations
this.objectID = null;
this.TimeRetrievedFromDb = null;
}
vs this:
public int Delete()
{
if (this.objectID == null || this.objectID == -1) { throw new exNotDbObject() }
int retVal = dataLayer.DeleteObj(this.objectID);
return retVal;
}

I like the int Delete() myself. I can then tell how many (if any) records were deleted, and unless there is some problem, the program can continue as expected.
I suppose you could do void Delete() and throw an exception if no records (or multiple ones for that matter) were deleted, but I never liked this approach as that always assumed that I know what the expected outcome of the deletion was.