We Keep Coding

CRM Dynamics 2013 How to Update Multiple Records From External Source Using ExecuteMultipleRequest

I am having a scenario in CRM where I need to update multiple accounts values(text fields and option sets) with values from an external sql database table. How can I go about doing this using the execute multiple request. The requirement is to sync all the account data in CRM with our ERP data which comes from a sql table. This process needs to be automated so i opted to use a windows service that runs daily to update accounts that ar flagged for update in the external sql table. I am struggling to find the best approach for this,I tested this idea in a console application on DEV and here is my solution code below. My question is how can I do this better using ExecuteMultipleRequest request.
public static void UpdateAllCRMAccountsWithEmbraceAccountStatus(IOrganizationService service, CRM_Embrace_IntegrationEntities3 db)
{
List<C_Tarsus_Account_Active_Seven__> crmAccountList = new List<C_Tarsus_Account_Active_Seven__>();
//Here I get the list from Staging table
var crmAccounts = db.C_Tarsus_Account_Active_Seven__.Select(x => x).ToList();
foreach (var dbAccount in crmAccounts)
{
CRMDataObjectFour modelObject = new CRMDataObjectFour()
{
ID = dbAccount.ID,
Account_No = dbAccount.Account_No,
Account_Name = dbAccount.Account_Name,
Account_Status = Int32.Parse(dbAccount.Account_Status.ToString()),
Country = dbAccount.Country,
Terms = dbAccount.Terms
};
}
var officialDatabaseList = crmAccounts;
//Here I query CRM to
foreach (var crmAcc in officialDatabaseList)
{
QueryExpression qe = new QueryExpression();
qe.EntityName = "account";
qe.ColumnSet = new ColumnSet("accountnumber", "new_embraceaccountstatus");
qe.Criteria.AddCondition("statecode", ConditionOperator.Equal, 0);
qe.Criteria.AddCondition("accountnumber", ConditionOperator.NotIn, "List of acconts for example"
);
EntityCollection response = service.RetrieveMultiple(qe);
//Here I update the optionset value
foreach (var acc in response.Entities)
{
if (acc.Attributes["accountnumber"].ToString() == crmAcc.Account_No)
{
if (acc.Contains("new_embraceaccountstatus"))
{
continue;
}
else
{
acc.Attributes["new_embraceaccountstatus"] = new OptionSetValue(Int32.Parse(crmAcc.Account_Status.ToString()));
}
service.Update(acc);
}
}
}
}
I know this might not be the right approach, please advise me how to use ExecuteMultipleRequest or perhaps a different solution altogether.

Here is some helper methods I've used before to handle this:
public static ExecuteMultipleRequest MultipleRequest { get; set; }
private const int BatchSize = 250;
public static long LastBatchTime { get; set; }
private static void Batch(IOrganizationService service, OrganizationRequest request)
{
if (MultipleRequest.Requests.Count == BatchSize)
{
ExecuteBatch(service);
}
MultipleRequest.Requests.Add(request);
}
private static void ExecuteBatch(IOrganizationService service)
{
if (!MultipleRequest.Requests.Any())
{
return;
}
Log("Executing Batch size {0}. Last Batch was executed in {1}",MultipleRequest.Requests.Count, LastBatchTime);
var watch = new System.Diagnostics.Stopwatch();
watch.Start();
var response = (ExecuteMultipleResponse)service.Execute(MultipleRequest);
watch.Stop();
LastBatchTime = watch.ElapsedMilliseconds;
Log("Completed Executing Batch in " + watch.ElapsedMilliseconds);
WriteLogsToConsole();
var errors = new List<string>();
// Display the results returned in the responses.
foreach (var responseItem in response.Responses)
{
// A valid response.
if (responseItem.Fault != null)
{
errors.Add(string.Format(
"Error: Execute Multiple Response Fault. Error Code: {0} Message {1} Trace Text: {2} Error Keys: {3} Error Values: {4} ",
responseItem.Fault.ErrorCode,
responseItem.Fault.Message,
responseItem.Fault.TraceText,
responseItem.Fault.ErrorDetails.Keys,
responseItem.Fault.ErrorDetails.Values));
}
}
MultipleRequest.Requests.Clear();
if (errors.Any())
{
throw new Exception(string.Join(Environment.NewLine, errors));
}
}
You can then call this from your normal logic like so:
public static void UpdateAllCRMAccountsWithEmbraceAccountStatus(IOrganizationService service, CRM_Embrace_IntegrationEntities3 db)
{
List<C_Tarsus_Account_Active_Seven__> crmAccountList = new List<C_Tarsus_Account_Active_Seven__>();
//Here I get the list from Staging table
var crmAccounts = db.C_Tarsus_Account_Active_Seven__.Select(x => x).ToList();
foreach (var dbAccount in crmAccounts)
{
CRMDataObjectFour modelObject = new CRMDataObjectFour()
{
ID = dbAccount.ID,
Account_No = dbAccount.Account_No,
Account_Name = dbAccount.Account_Name,
Account_Status = Int32.Parse(dbAccount.Account_Status.ToString()),
Country = dbAccount.Country,
Terms = dbAccount.Terms
};
}
var officialDatabaseList = crmAccounts;
//Here I query CRM to
foreach (var crmAcc in officialDatabaseList)
{
QueryExpression qe = new QueryExpression();
qe.EntityName = "account";
qe.ColumnSet = new ColumnSet("accountnumber", "new_embraceaccountstatus");
qe.Criteria.AddCondition("statecode", ConditionOperator.Equal, 0);
qe.Criteria.AddCondition("accountnumber", ConditionOperator.NotIn, "List of acconts for example");
EntityCollection response = service.RetrieveMultiple(qe);
//Here I update the optionset value
foreach (var acc in response.Entities)
{
if (acc.Attributes["accountnumber"].ToString() == crmAcc.Account_No)
{
if (acc.Contains("new_embraceaccountstatus"))
{
continue;
}
else
{
acc.Attributes["new_embraceaccountstatus"] = new OptionSetValue(Int32.Parse(crmAcc.Account_Status.ToString()));
}
Batch(service, new UpdateRequest { Target = acc });
}
}
}
// Call ExecuteBatch to ensure that any batched requests, get executed.
ExeucteBatch(service)
}

Because it is 2013, and you need to sync records, you'll need to know if some previous records were already in CRM, because depending on that you'll need to send a bunch of Create's or Update's. I would do it in 2 batches of ExecuteMultiple:
1) One batch to execute a query to find which accounts need to be created / updated in CRM, depending on some matching field there.
2) Another batch which will use the previous one to generate all Create / Update operations in one go, depending on the responses you got from 1).
The issue is that they won't run in the same transaction, and that's something which was improved in 2016, as #Daryl said. There is also a new request in 2016 which might improve things even further, because you could merge the 2 batches into one: Upsert, therefore avoiding unnecessary roundtrips to the server.
Maybe this was inspired on Mongo Db's upsert concept which existed long time before? Who knows :)

If you just need to now how to perform an ExecuteMultipleRequest there are samples on the MSDN. Sample: Execute multiple requests.

How to create an EventBookmark when querying the event log

I have an EventLogReader object, and a query in the event log that looks like this:
string query = "*[System[(Level=2) and TimeCreated[#SystemTime>='%LastRun%']]]")
The code basically uses the reader to query for all the events that match the search query since the last time the reader was run.
I would rather use the EventBookmark for this purpose. That's what it is for, after all! But I am having trouble finding any working code.
My existing code run, in part, like this:
// This line replaces the %LastRun% code with the date
var myQuery = myEventLogQuery.Query.Replace("%LastRun%", myEventLogQuery.LastRun.ToString("o"));
var query = new EventLogQuery(myEventLogQuery.Log, myEventLogQuery.PathType, myQuery);
// Now set the LastRun date. I want to avoid this...
myEventLogQuery.LastRun = DateTime.UtcNow;
// ... by making this next line smarter.
var reader = new EventLogReader(query);
// var reader = new EventLogReader(query, ??? new EventBookmark());
EventRecord eventRecord;
while ((eventRecord = reader.ReadEvent()) != null)
{
EventRecords.Add(new EventRecordItem(eventRecord));
}
I should be able to use the EventBookmark property of the first (or last) EventRecord to restrict the next query, but I want to set the EventBookmark initially to be basically the highest record in the log.
When the application runs initially, I don't need it to tell me about all the event log entries from the past, I only care about ones that occur after the application starts.

OK, I went ahead and experimented a lot, and managed to work out a good system for this. Since this topic isn't really covered, I'm posting my answer here. I hope it's useful!
First challenge is creating the initial EventBookmark. You can't just instantiate one. You need to derive one from an existing EventRecord. To do this, I query for the last item in the log and base my EventBookmark on that.
using System.Diagnostics.Eventing.Reader;
public class MyEventLogQuery
{
public string Log { get; set; }
public PathType PathType { get; set; }
public string Query { get; set; }
public EventBookmark Bookmark { get; set; }
public MyEventLogQuery(string log = "Application", PathType pathType = PathType.LogName, string query = "*[System[(Level=2)]]")
{
Log = log;
PathType = pathType;
Query = query;
Bookmark = GetBookmark(log, pathType); // Query is not important here
}
// This method returns the bookmark of the most recent event
// log EventRecord or null if the log is empty
private static EventBookmark GetBookmark(string log, PathType pathType)
{
var elq = new EventLogQuery(log, pathType) {ReverseDirection = true};
var reader = new EventLogReader(elq);
var record = reader.ReadEvent();
if (record != null)
return record.Bookmark;
return null;
}
}
Next step is to use the bookmark (or lack of bookmark) when subsequently querying the Event Log.
using System.Diagnostics.Eventing.Reader;
public class EventLogTracker()
{
public List<MyEventLogQuery> Queries { get; set; }
// ... snipped some stuff
public int Run()
{
var count = 0;
foreach (var myQuery in Queries)
{
var query = new EventLogQuery(myQuery.Log, myQuery.PathType, myQuery.Query);
// This is the important bit. Must take account that the
// log may have been empty, so bookmark could be null
var reader = myQuery.Bookmark != null ? new EventLogReader(query, myQuery.Bookmark) : new EventLogReader(query);
EventRecord eventRecord;
while ((eventRecord = reader.ReadEvent()) != null)
{
// Do stuff
// ...
// Then update the bookmark
myQuery.Bookmark = eventRecord.Bookmark;
count++;
}
}
return count;
}
And voila, you have code that uses the EventBookmark to only give you events that have occurred since the application was started.

Entity Framework Code First and SQL Server 2012 Sequences

I was in the middle of implementing a database audit trail whereby CRUD operations performed through my controllers in my Web API project would serialize the old and new poco's and store their values for later retrieval (historical, rollback, etc...).
When I got it all working, I did not like how it made my controllers look during a POST because I ended up having to call SaveChanges() twice, once to get the ID for the inserted entity and then again to commit the audit record which needed to know that ID.
I set out to convert the project (still in its infancy) to use sequences instead of identity columns. This has the added bonus of further abstracting me from SQL Server, though that is not really an issue, but it also allows me to reduce the number of commits and lets me pull that logic out of the controller and stuff it into my service layer which abstracts my controllers from the repositories and lets me do work like this auditing in this "shim" layer.
Once the Sequence object was created and a stored procedure to expose it, I created the following class:
public class SequentialIdProvider : ISequentialIdProvider
{
private readonly IService<SequenceValue> _sequenceValueService;
public SequentialIdProvider(IService<SequenceValue> sequenceValueService)
{
_sequenceValueService = sequenceValueService;
}
public int GetNextId()
{
var value = _sequenceValueService.SelectQuery("GetSequenceIds #numberOfIds", new SqlParameter("numberOfIds", SqlDbType.Int) { Value = 1 }).ToList();
if (value.First() == null)
{
throw new Exception("Unable to retrieve the next id's from the sequence.");
}
return value.First().FirstValue;
}
public IList<int> GetNextIds(int numberOfIds)
{
var values = _sequenceValueService.SelectQuery("GetSequenceIds #numberOfIds", new SqlParameter("numberOfIds", SqlDbType.Int) { Value = numberOfIds }).ToList();
if (values.First() == null)
{
throw new Exception("Unable to retrieve the next id's from the sequence.");
}
var list = new List<int>();
for (var i = values.First().FirstValue; i <= values.First().LastValue; i++)
{
list.Add(i);
}
return list;
}
}
Which simply provides two ways to get IDs, a single and a range.
This all worked great during the first set of unit tests but as soon as I started testing it in a real world scenario, I quickly realized that a single call to GetNextId() would return the same value for the life of that context, until SaveChanges() is called, thus negating any real benefit.
I am not sure if there is a way around this short of creating a second context (not an option) or going old school ADO.NET and making direct SQL calls and use AutoMapper to get to the same net result. Neither of these are appeal to me so I am hoping someone else has an idea.

Don't know if this might help you, but this is how I did my audit log trail using code first.
The following is coded into a class inheriting from DbContext.
in my constructor I have the following
IObjectContextAdapter objectContextAdapter = (this as IObjectContextAdapter);
objectContextAdapter.ObjectContext.SavingChanges += SavingChanges;
This is my saving changes method wired up previously
void SavingChanges(object sender, EventArgs e) {
Debug.Assert(sender != null, "Sender can't be null");
Debug.Assert(sender is ObjectContext, "Sender not instance of ObjectContext");
ObjectContext context = (sender as ObjectContext);
IEnumerable<ObjectStateEntry> modifiedEntities = context.ObjectStateManager.GetObjectStateEntries(EntityState.Modified);
IEnumerable<ObjectStateEntry> addedEntities = context.ObjectStateManager.GetObjectStateEntries(EntityState.Added);
addedEntities.ToList().ForEach(a => {
//Assign ids to objects that don't have
if (a.Entity is IIdentity && (a.Entity as IIdentity).Id == Guid.Empty)
(a.Entity as IIdentity).Id = Guid.NewGuid();
this.Set<AuditLogEntry>().Add(AuditLogEntryFactory(a, _AddedEntry));
});
modifiedEntities.ToList().ForEach(m => {
this.Set<AuditLogEntry>().Add(AuditLogEntryFactory(m, _ModifiedEntry));
});
}
And these are the methods used previosly to build up the audit log details
private AuditLogEntry AuditLogEntryFactory(ObjectStateEntry entry, string entryType) {
AuditLogEntry auditLogEntry = new AuditLogEntry() {
EntryDate = DateTime.Now,
EntryType = entryType,
Id = Guid.NewGuid(),
NewValues = AuditLogEntryNewValues(entry),
Table = entry.EntitySet.Name,
UserId = _UserId
};
if (entryType == _ModifiedEntry) auditLogEntry.OriginalValues = AuditLogEntryOriginalValues(entry);
return auditLogEntry;
}
/// <summary>
/// Creates a string of all modified properties for an entity.
/// </summary>
private string AuditLogEntryOriginalValues(ObjectStateEntry entry) {
StringBuilder stringBuilder = new StringBuilder();
entry.GetModifiedProperties().ToList().ForEach(m => {
stringBuilder.Append(String.Format("{0} = {1},", m, entry.OriginalValues[m]));
});
return stringBuilder.ToString();
}
/// <summary>
/// Creates a string of all modified properties' new values for an entity.
/// </summary>
private string AuditLogEntryNewValues(ObjectStateEntry entry) {
StringBuilder stringBuilder = new StringBuilder();
for (int i = 0; i < entry.CurrentValues.FieldCount; i++) {
stringBuilder.Append(String.Format("{0} = {1},",
entry.CurrentValues.GetName(i), entry.CurrentValues.GetValue(i)));
}
return stringBuilder.ToString();
}
Hopefully this might point you into a direction that might help you solve your problem.

"The operation cannot be completed because the DbContext has been disposed"

I am new at Entity Framework Code first and I am building a small app to get used to it.When the site runs for the first time I access existing catalog values inside the database and display this in a drop down using razor.
public void GetCats()
{
using (context = new RecipeContext())
{
try
{
var query = (from r in context.Catalogues
select r).Distinct().ToList();
catalogues = query.Select(t => t.CatalogueName.ToString()).ToList();
catalogues.Sort();
}
catch (Exception exe)
{
labMessage = exe.Message;
}
}
}
Now when I try to add Catalogue values to the context I get the above error.
public void AddCatalogue(string catalogueName)
{
using(context = new RecipeContext())
{
try
{
catalogueName = catalogueName.ToLower();
var catalogue = new RecipeCatalogue { CatalogueName = catalogueName };
if (context.Catalogues.Where(t => t.CatalogueName == catalogueName).Count() > 0)
{
labMessage = "The value already exists";
CatalogueNameAdded = false;
return;
}
context.Catalogues.Add(catalogue);
context.SaveChanges();
catalogueNameAdded = true;
labMessage = "a new catalogue record was added";
}
catch (Exception exe)
{
catalogueNameAdded = false;
labMessage = exe.Message;
}
}
}
The values are being added to the database however but still get the above exception.
Advice perhaps as to why I get this error. This is my Controller method which calls the above method.
[HttpPost]
public JsonResult AddNewCatalogue(string catalogueName)
{
ViewModel model = new ViewModel();
model.AddCatalogue(catalogueName);
return Json(new { ViewModel = model });
}

Is context a field in your model?
I think you shouldn't assign to a field in a using statement. At the closing brace of the using context will be disposed. If you access that field in another place (without re-assigning) you are accessing a disposed object that might raise the exception you are getting.
Try changing your using statetments like this using (var context = new RecipeContext()).
(note var before context) and drop the field.

Your context is being disposed when the using block where you're performing your query is exited. That's the whole point of the using statement:
using(context = new RecipeContext()) {
// ...
}
// context has been disposed at this point
Instead of a using statement, give your class a field to hold a reference to it.
private RecipeContext _context;
public void GetCats() {
_context = new RecipeContext();
// ...
}
public void AddCatalogue(string catalogueName) {
// Use _context here
}
Just make sure that at some point, you call _context.Dispose(). Also, it's probably better to create the context in the constructor or someplace else that's only called once, prior to performing any operations with it.

Just my 2 cents:
The above answers are correct! If you're using some pattern like a repository, I sugest to implement it as a singleton! This way your objects will not be detached, and you're context will not be disposed!

MVVM Database Child_Update() not updateing

Ok here is the deal. using my Model to raise the Child_Update() Method. I know it is not good but im just learing right now. Now i given my Business class as Parameter to Change the already given rows.
It's doin everything fine without any error Messages and there is also no null variable but it's not changing anything in the database.
here i'm using the selectedIndex to choose the right items to update
public void ExecuteAngestellte(object obj)
{
try
{
_selectedIndex.Child_Update(new Farbe { FarbauswahlNr = SelectedIndex.FarbauswahlNr, Kurztext = SelectedIndex.Kurztext, Ressource = SelectedIndex.Ressource, Vari1 = SelectedIndex.Vari1, Vari2 = SelectedIndex.Vari2 });
}
catch (Exception e)
{
MessageBox.Show(e.ToString());
}
}
and here's the child update
public void Child_Update(Farbe data)
{
using (var ctx =Csla.Data.ObjectContextManager<TestDBEntities>.GetManager(EntitiesDatabase.Name))
{
var objectStateManager = ctx.ObjectContext.ObjectStateManager;
var _data = new Datenbank.Farbe();
_data.FarbauswahlNr = data.FarbauswahlNr;
_data.Kurztext = data.Kurztext;
_data.Ressource = data.Ressource;
_data.Var1 = data.Vari1;
_data.Vari2 = data.Vari2;
ctx.ObjectContext.SaveChanges();
}
}
thx for help

as far as I can tell, your _data is not part of the datacontext
I am not sure what the variables all mean so here is a simple update I use for a single record with entity framework
public void UpdateTable(int idRecord, YourContext Context)
{
MyRecord = Context.MyTable.Find(idRecord);
myRecord.Column = "New Value";
Context.SaveChanges();
}
so I think you need to do:
var MyData = ctx.ObjectContext.TableName.Find(id)
I am assuming you are using entity framework. Does this help?