We Keep Coding

Nopcommerce Update entity issue

Using NopCommerce 3.8, Visual Studio 2015 proff.
I have created a plugin that is responsible for making restful calls to my Web API that exposes a different DB to that of Nop.
The process is run via a nop Task, it successfully pulls the data back and i can step through and manipulate as i see fit, no issues so far.
Issue comes when i try to update a record on the product table, i perform the update... but nothing happens no change, no error.
I believe this is due to the Context having no idea about my newly instantiated product object, however I'm drawing a blank on what i need to do in relation to my particular example.
Similar questions usually reference a "model" object that is part of the parameter of the method call, "model" has the method ToEntity which seems to be the answer in similar question in stack.
However my example doesn't have the ToEntity class/method possibly because my parameter is actually a list of products. To Clarify here my code.
Method in RestClient.cs
public async Task<List<T>> GetAsync()
{
try
{
var httpClient = new HttpClient();
var json = await httpClient.GetStringAsync(ApiControllerURL);
var taskModels = JsonConvert.DeserializeObject<List<T>>(json);
return taskModels;
}
catch (Exception e)
{
return null;
}
}
Method in my Service Class
public async Task<List<MWProduct>> GetProductsAsync()
{
RestClient<MWProduct> restClient = new RestClient<MWProduct>(ApiConst.Products);
var productsList = await restClient.GetAsync();
InsertSyncProd(productsList.Select(x => x).ToList());
return productsList;
}
private void InsertSyncProd(List<MWProduct> inserted)
{
var model = inserted.Select(x =>
{
switch (x.AD_Action)
{
case "I":
//_productService.InsertProduct(row);
break;
case "U":
UpdateSyncProd(inserted);
.....
Then the method to bind and update
private void UpdateSyncProd(List<MWProduct> inserted)
{
var me = inserted.Select(x =>
{
var productEnt = _productRepos.Table.FirstOrDefault(ent => ent.Sku == x.Sku.ToString());
if(productEnt != null)
{
productEnt.Sku = x.Sku.ToString();
productEnt.ShortDescription = x.ShortDescription;
productEnt.FullDescription = x.FullDescription;
productEnt.Name = x.Name;
productEnt.Height = x.Pd_height != null ? Convert.ToDecimal(x.Pd_height) : 0;
productEnt.Width = x.Pd_width != null ? Convert.ToDecimal(x.Pd_width) : 0;
productEnt.Length = x.Pd_depth != null ? Convert.ToDecimal(x.Pd_depth) : 0;
productEnt.UpdatedOnUtc = DateTime.UtcNow;
}
//TODO: set to entity so context nows and can update
_productService.UpdateProduct(productEnt);
return productEnt;
});
}
So as you can see, I get the data and pass data through to certain method based on a result. From that list in the method I iterate over, and pull the the entity from the table, then update via the product service using that manipulated entity.
So what am I missing here, I'm sure its 1 step, and i think it may be either be because 1) The context still has no idea about the entity in question, or 2) Its Incorrect calls.
Summary
Update is not updating, possibly due to context having no knowledge OR my methodology is wrong. (probably both).
UPDATE:
I added some logger.inertlog all around my service, it runs through fine, all to the point of the call of update. But again I check the product and nothing has changed in the admin section.
plugin
I have provided the full source as i think maybe this has something to do with the rest of the code setup possibly?
UPDATE:
Added the following for testin on my execute method.
var myprod = _productRepos.GetById(4852);
myprod.ShortDescription = "db test";
productRepos.Update(myprod);
This successfully updates the product description. I moved my methods from my service into the task class but still no luck. The more i look at it the more im thinking that my async is killing off the db context somehow.

Turned of async and bound the getbyid to a new product, also removed the lambda for the switch and changed it to a foreach loop. Seems to finally update the results.
Cannot confirm if async is the culprit, currently the web api seems to be returning the same result even though the data has changed (some wierd caching by deafult in .net core? ) so im creating a new question for that.
UPDATE: It appears that the issue stems from poor debugging of async. Each instance I am trying to iterate over an await call, simply put im trying to iterate over a collection that technically may or may not be completed yet. And probably due to poor debugging, I was not aware.
So answer await your collection Then iterate after.

Executing part of code exactly 1 time inside Parallel.ForEach

I have to query in my company's CRM Solution(Oracle's Right Now) for our 600k users, and update them there if they exist or create them in case they don't. To know if the user already exists in Right Now, I consume a third party WS. And with 600k users this can be a real pain due to the time it takes each time to get a response(around 1 second). So I managed to change my code to use Parallel.ForEach, querying each record in just 0,35 seconds, and adding it to a List<User> of records to be created or to be updated (Right Now is kinda dumb so I need to separate them in 2 lists and call 2 distinct WS methods).
My code used to run perfectly before multithread, but took too long. The problem is that I can't make a batch too large or I get a timeout when I try to update or create via Web Service. So I'm sending them around 500 records at once, and when it runs the critical code part, it executes many times.
Parallel.ForEach(boDS.USERS.AsEnumerable(), new ParallelOptions { MaxDegreeOfParallelism = -1 }, row =>
{
...
user = null;
user = QueryUserById(row["USER_ID"].Trim());
if (user == null)
{
isUpdate = false;
gObject.ID = new ID();
}
else
{
isUpdate = true;
gObject.ID = user.ID;
}
... fill user attributes as generic fields ...
gObject.GenericFields = listGenericFields.ToArray();
if (isUpdate)
listUserUpdate.Add(gObject);
else
listUserCreate.Add(gObject);
if (i == batchSize - 1 || i == (boDS.USERS.Rows.Count - 1))
{
UpdateProcessingOptions upo = new UpdateProcessingOptions();
CreateProcessingOptions cpo = new CreateProcessingOptions();
upo.SuppressExternalEvents = false;
upo.SuppressRules = false;
cpo.SuppressExternalEvents = false;
cpo.SuppressRules = false;
RNObject[] results = null;
// <Critical_code>
if (listUserCreate.Count > 0)
{
results = _service.Create(_clientInfoHeader, listUserCreate.ToArray(), cpo);
}
if (listUserUpdate.Count > 0)
{
_service.Update(_clientInfoHeader, listUserUpdate.ToArray(), upo);
}
// </Critical_code>
listUserUpdate = new List<RNObject>();
listUserCreate = new List<RNObject>();
}
i++;
});
I thought about using lock or mutex, but it isn't gonna help me, since they will just wait to execute afterwards. I need some solution to execute only ONCE in only ONE thread that part of code. Is it possible? Can anyone share some light?
Thanks and kind regards,
Leandro

As you stated in the comments you're declaring the variables outside of the loop body. That's where your race conditions originate from.
Let's take variable listUserUpdate for example. It's accessed randomly by parallel executing threads. While one thread is still adding to it, e.g. in listUserUpdate.Add(gObject); another thread could already be resetting the lists in listUserUpdate = new List<RNObject>(); or enumerating it in listUserUpdate.ToArray().
You really need to refactor that code to
make each loop run as independent from each other as you can by moving variables inside the loop body and
access data in a synchronizing way using locks and/or concurrent collections

You can use the Double-checked locking pattern. This is usually used for singletons, but you're not making a singleton here so generic singletons like Lazy<T> do not apply.
It works like this:
Separate out your shared data into some sort of class:
class QuerySharedData {
// All the write-once-read-many fields that need to be shared between threads
public QuerySharedData() {
// Compute all the write-once-read-many fields. Or use a static Create method if that's handy.
}
}
In your outer class add the following:
object padlock;
volatile QuerySharedData data
In your thread's callback delegate, do this:
if (data == null)
{
lock (padlock)
{
if (data == null)
{
data = new QuerySharedData(); // this does all the work to initialize the shared fields
}
}
}
var localData = data
Then use the shared query data from localData By grouping the shared query data into a subordinate class you avoid the necessity of making its individual fields volatile.
More about volatile here: Part 4: Advanced Threading.
Update my assumption here is that all the classes and fields held by QuerySharedData are read-only once initialized. If this is not true, for instance if you initialize a list once but add to it in many threads, this pattern will not work for you. You will have to consider using things like Thread-Safe Collections.

Getting a list of all users via Valence

I am trying to get a list of all users in our instance of Desire2Learn using a looping structure through the bookmarks however for some reason it continuously loops and doesn't return. When I debug it it is showing massive amounts of users (far more than we have in the system as shown by the User Management Tool. A portion of my code is here:
public async Task<List<UserData>> GetAllUsers(int pages = 0)
{
//List<UserData> users = new List<UserData>();
HashSet<UserData> users = new HashSet<UserData>();
int pageCount = 0;
bool getMorePages = true;
var response = await Get<PagedResultSet<UserData>>("/d2l/api/lp/1.4/users/");
var qParams = new Dictionary<string, string>();
do
{
qParams["bookmark"] = response.PagingInfo.Bookmark;
//users = users.Concat(response.Items).ToList<UserData>();
users.UnionWith(response.Items);
response = await Get<PagedResultSet<UserData>>("/d2l/api/lp/1.4/users/", qParams);
if (pages != 0)
{
pageCount++;
if (pageCount >= pages)
{
getMorePages = false;
}
}
}
while (response.PagingInfo.HasMoreItems && getMorePages);
return users.ToList();
}
I originally was using the List container that is commented out but just switched to the HashSet to see if I could notice if duplicates where being added.
It's fairly simple, but for whatever reason it's not working. The Get<PagedResultSet<UserData>>() method simply wraps the HTTP request logic. We set the bookmark each time and send it on.
The User Management Tool indicates there are 39,695 users in the system. After running for just a couple of minutes and breaking on the UnionWith in the loop I'm showing that my set has 211,800 users.
What am I missing?

It appears that you’ve encountered a defect in this API. The next course of action is for you to have your institution’s Approved Support Contact open an Incident through the Desire2Learn Helpdesk. Please make mention in the Incident report that Sarah-Beth Bianchi is aware of the issue, and I will work with our Support team to direct this issue appropriately.

Foreach loop and error that not all code paths return a value: async-await issue?

I am not sure why the "not all code paths return a value" error appears (see commented code below). Is it because the foreach loop is considered as not one path but many, and it's possible that an iteration might not return a value and yet the loop would continue?
[BTW, my goal is to process 100K urls, creating many web requests so they are active simultaneously, and be able to capture the status of each url (alive, moved, timed out) while updating the UI as the set of urls is being processed with real-time information about the number of bad urls found so far and the number or urls processed so far. I've tried Parallel.ForEach approach with synchronous web requests but the UI became unresponsive.]
EDITED to include a test if Rows.Count == 0.
public async Task<UrlInfo> ProcessUrls(DataTable urls)
{
if (urls.Rows.Count == 0)
{
return new UrlInfo();
}
else
{
foreach (DataRow r in urls.Rows)
{
UrlInfo info = new UrlInfo()
{
Url = (string)r["url"],
status = UrlStatusCode.untested,
articleid = (int)r["articleid"]
};
return await Foo(info);
}
}
//return new UrlInfo(); // error unless this line is uncommented
}
public async Task<UrlInfo> Foo(UrlInfo info) {
<snip>
}

The compiler is complaining about the situation where urls.Rows does not contain any elements. In that case, the method will never encounter a return statement.
On a side note, I do not think that method means what you think it means. A method only returns once, so it would only process the first url.

Stuck on implementation of WebBrowser Control and working with AJAX responses

Here we go...
I am trying to create a bot to walk through various functions of a website I do not have control of. Initially, I thought it would be best to just connect to the database (MySql) that the site is tied to and make my associations there...The database is built so extensively that I can't make out where to point what to where how etc...It's beyond my (DBA)programmer casting... ;)
So, my next idea was, create a bot...simple enough right? First hurdle, not everything on the page carries an ID...bring on the loops...
Got it.
Now I'm stuck with working with my data and page response.
I'm trying to fill out part of a form and perform an AJAX search. The problem is, there is no DocumentCompleted for this. And honestly, that isn't where my problem lies. I've tried using Thread.Sleep, Timers, etc...no avail.
// The app reads categories from a csv file,
// then performs a search for the category
// Search results are only displayed if I break the foreach loop
foreach (var item in bar)
{
var document = wbBrowser.Document;
if (document != null)
{
var name = document.GetElementById("product_filter_name");
if (name != null)
{
name.SetAttribute("value", item.Key.ToString());
var buttons = document.GetElementsByTagName("button");
foreach (HtmlElement button in buttons)
{
var findSearch = button.InnerHtml.IndexOf("Search");
if (findSearch > -1)
{
button.InvokeMember("click");
}
}
}
// This where the problem starts...
// I want the AJAX to run, then perform Step two,
// but the WebBrowser doesn't return the search
// results until the end (break;)
// Step Two
var elems = document.GetElementsByTagName("tr");
foreach (HtmlElement elem in elems)
{
// find particular item in result table
}
break;
// Now the search results display!!!!
// I tried implementing a timer, Thread.Sleep,
// everything I could find via Google before
//starting Step Two, but it hasn't worked!
}
}

The actual browser control has a WebBrowser.OnDocumentCompleted event which you might need to hook into so that you can be alerted when the ajax call has returned back from the server.