Recently I came across kinda funny issue. Generally I am working on .NET web app hosted on IIS and using MS SQL server.
Having said that I wrote a simple piece of code to update specific record in db. The record should already exist and no further operations should occur on it (or actually even the entire table) for considerable time (no concurrent operation should appear to overwrite update).
var record = db.InputDataRecords.Where(i => i.Id == inputDataId).Single();
record.InputData = updatedInputData;
db.SaveChangesAsync();
So I wrote the above code, tested it on my local machine, everything worked perfectly fine so I just left it as it was.
Recently the code finally got to the test server (pretty slow sucker) and to my surprise I got info from the test crew that the records do not update accordingly. What is even worse no errors were logged although they should if something went wrong plus I am pretty sure the above code did not throw any errors as, for the sake of peace, I forced it to log message with some details after it is executed (I cannot debug on that machine unfortunately so I couldn't get any better feedback).
I struggled with this issue for couple of hours doing some small adjustments here and there, but nothing was really helping. Finally switching to synchronous SaveChanges() did the trick.
Did anyone of you come across similar issue? I am pretty curious why it could occur.
Related
I have just recently updated a web application on an IIS server, but after the update my users were receiving an incorrect input format error. This error is because the code is trying to convert a user's string input into a double which obviously doesn't work if the user passes in something like 55.5D.
Besides the bad coding practice I'm going to fix anyway, that's not the real issue. The issue is that the user was sending correct values (I watched and have input the exact same values myself) yet the program was still throwing this error. I reverted back to a previous version and the error disappeared. I haven't changed this section of code since the previous version. Anybody know what is going on here? I can't get the problem to repeat on development servers either without intentionally feeding the program bad input.
EDIT: I have tried clearing the user's and server's cache after the update, but that still received the same error, even after I put checks on the areas that were breaking (I missed some elsewhere in the code too). However, it worked great when only a few users were using it at a time, but it was breaking when many users were using it. Do ASP.NET controls have issues when many users are hitting the site?
Fully managed components in IIS do not require an application pool restart for changes to be applied. On the contrary, it will reload and JIT things again even if only web.config changes. If you are in doubt, you can always test this by including a test page with your application with a visible differentiator.
The most likely reason for your case is caching. Client-side is more likely culprit. Have you monitored http codes to verify that requests are really hitting your server and getting "live"-results (HTTP 200)?
Server-side in-process cache should be reset automatically but if you have persisted it somewhere out-of-process, they could be picked up after app upgrade..
And of course reverify that you really are testing exactly those dlls you think you are testing. If things don't make sense then verifying a wider set of assumptions can help.
I'm not optimistic about getting an answer to this one, since it's such a weird and seemingly machine-specific issue, but I'm posting in the hopes that someone else has figured this one out.
I'm developing an application in C# using Visual Studio and the MySQL.Data ADO.Net driver (version 6.9.8) for database access. Twice, I've run into a situation where very simple SQL queries that worked only moments before start timing out. From that point forward, they never succeed again -- but only on my machine. Every other query in the application (and there are hundreds) works fine, but the one query, even if I change it slightly, never works again. If I push the code and another developer runs it, the code works perfectly on his machine.
As an example of how simple these queries are, here's the one I'm most recently having issues with:
SELECT * FROM user_preferences WHERE (user_id=#user_id) LIMIT 1
The table I'm selecting from has only 1 row and 4 columns. It couldn't be simpler. While the query is waiting to time out, it does not appear in the process list for MySQL, so I have to assume that either it never made it to the server or the server returned instantly and the MySQL connector is just failing to return the response. I'd like to figure out which, but I doubt it would make a difference either way.
So to summarize, that query worked fine this morning. I changed nothing except some unrelated code, and it briefly started returning no data. I started debugging it, and now it just times out. Another developer runs the code, and it works perfectly.
I'm at a complete loss, here. Short of figuring this out, I fear I'm going to have to re-image my laptop, and I'd really like to avoid that if at all possible. Any help would be appreciated, though like I said, it's probably quite a shot in the dark.
Edit: The C# code in question:
using (var connection = new MySqlConnection(connectionSettings.GetConnectionString()))
{
await connection.OpenAsync(cancellationToken);
using (var reader = await MySqlHelper.ExecuteReaderAsync(connection, sql.Query.ToString(), cancellationToken, sql.Parameters.ToArray()))
{
while (await reader.ReadAsync(cancellationToken))
{
readerAction(reader);
}
}
}
Today I got an emergency call from the users on our ASP.NET production system. Some users (not all) were unable to enter certain data. The user posted the data, and the system then froze; the call never returned.
We tried to repro the problem on the QA system (which has a fresh restore of production data), and could not. I then ran from my dev environment and connected directly to the production DB, masquerading as one of the affected users. Again, no problem. Conclusion: there must be some kind of issue in the production environment, probably somewhere in the IIS process that's hosting the website.
So I fired up Visual Studio on the production server, and attached to the IIS process (Kids, don't do this at home!), set a breakpoint in the offending code, logged in as the user, and attempted to save the data. Hit the breakpoint and stepped line by line, until I hit a line of code like this:
try
{
...
using (var db = new MyDataContext())
{
...
var fooToUpdate = db.Foos.Single(f => f.ID == fooId); // <-- THIS LINE
...
}
}
catch (Exception ex)
{
// some error logging
}
After hitting "step" on that line, the thread simply vanished. Disappeared without a trace. I put a sniffer on the database, and no query was fired; needless to say there was no DB locking involved. No exception was thrown. The code entered Entity Framework and never left.
The way the data is is that every user has a different and unique fooId for every day, so no other user will have the same fooId. Most users were able to load their Foo, but a select handful of users fail consistently to load their personal Foo. I tried running the query to load the Foo in a SSMS window; no trouble at all. The only time it fails is in this particular IIS process, on the production server.
Now, I could just recycle the app pool or restart IIS, and that would probably paper over the problem. But something similar happened a week ago, and we couldn't trace it then, either. So we reset IIS then, hoping the problem would go away. And it did, for a week. And now it's back.
Does anyone have any ideas how it is possible for a thread to simply vaporize like this? Is Norman Bates hiding behind the EF door?
Given the fact that the thread did not magically vaporize, we could speculate some of the more likely options:
The debugger had a hard time following the production code compiled in Release mode. Just because debugging Release code works 90% of the time, don't fall under the illusion that it is dependable. Optimized code can very quickly throw the debugger off the track of actual execution. When this happens, it will look like the thread just vanished.
Assuming the thread does legitimately enter the call and not return (which seems to be supported by the original complaint of the application "freezing"), then the most likely scenario is a deadlock of some type. EntityFramework deadlocks are not common, but not unheard of either. The most common issues I'm aware of usually involve TransactionScope or CommitableTransaction. Are you using any transactions in the omitted code sections?
Turns out that the EF part was a red herring after all. I went and downloaded Telerik's JustDecompile and JustCode, in the hope of stepping into the EF code, but when I stepped in to that line, I found myself not in the Single() extension method, but inside one of my own method calls - that I thought I had executed on the previous line. Evidently the code was not perfectly in sync with the version in production.
LESSON 1: If you attach to a process, your execution point may not be where you think it is, if your code is not identical to the code that was
compiled into that process.
So anyway, now that I could step into the code without decompiling anything, the first thing I noticed was:
lock (_lockObj)
{
...
}
And when I tried to step into it, it froze there. Smoking gun.
So somewhere, some other thread is locking this object. Looked at other places where the lock is invoked, leading to a spaghetti of dependencies, along with another code-locked segment, with several DB calls and even a transaction boundary. It could be a code lock / db transaction deadlock, though a brief scan of the code in the DB transaction failed to pick up any contenders within the life of the transaction for blocking anything else. Plus, there's the evidence of the DB not showing any blocking or open transactions. Rather, it may just be the the fact of a few hundred queued up long-running processes, all inside code locks inside code locks, and in the end it all looks something like the West Side Highway at 17:05 on a Friday, with a jackknifed trailer truck lying across 3 lanes approaching the GW bridge.
LESSON 2: Code locks are dangerous, not only - but especially - when used in conjunction with DB transactions. Try to find ways to make your code thread safe without using code locks. And if you really must use code locks, make sure you get in and out as quickly as possible. Don't give your thread a magazine to read while it's occupying the only stall, so to speak.
I am trying to connect to a Domino Session which works fine when testing in isolation, however as soon as I put it in a thread I run into problems.
This works fine when I put in a Unit test, so I know I can connect to the server fine.
NotesSession session = new NotesSession();
string DominoPassword = Helpers.GetConfigSetting("DominoPassword");
session.Initialize(DominoPassword);
However when I put this in a thread (want to be able to connect to Domino several times) I
get the following error
"{Unable to evaluate expression because the code is optimized or a native frame is on top of the call stack.}"
A couple of years ago I definitely got farther than this with testing of multi-threaded Domino Interop in C#. I don't recall seeing that error. It might be due to a difference between our respective dev/test environments. It's long enough ago, though, that I don't recall what my environments were when I ran my tests.
But whether or not the Domino objects are thread-safe remains an open question. See here. As far as I know, IBM has never said that they are. (Also, last time I checked, IBM said that the classes are not supported on Win64 platforms even in Win32 applications, so that might be entering into your problem.
I have an ASP.NET application that does a large database read. It loads up a gridview inside an update panel. In VS2008, just running on my local machine, it runs fantastically. In production (identical code, just published and put on one of our network servers), it runs slow as dirt.
Debug is set to false, so this is not the cause of the slow down. I'm not an experienced web developer, so besides that, feel free to suggest the obvious.
I have been using Firebug to determine what's going on, and here is what that has turned up:
On production, there are around 500 requests. The timeline bar is very short. The size column varies from run to run, but is always the same for the duration of the run.
Locally, there are about 30 requests. The timeline bar takes up the entire space.
Can anyone shed some light on why this is happening and what I can do to fix it? Also, I can't find much of anything on the web about this, so any references are helpful too.
EDIT: I forgot to mention that I am using the exact same database for both local and production environments.
EDIT: __EVENTTARGET points to the timer that updates the progress.
EDIT: Timer appears to be working - I set it to a larger interval, which shows me that the local app is processing ~50000 records/second and that the production app (same database, same code!) is processing at best ~5000 records/sec.
Back to square one, mysteriously slow app. :(
My guess would be the size of the database you're using on your dev box vs. production. Is it possible there is something happening for each row of your gridview that only happens 30 times on your box becuase it has less data than production?
Systematic debugging - strip it down and add things back one at a time until you see the change.
Brandi,
Can you check the Event Viewer logs for the web server? Do you have anything failing in there?
Are you running the app in VS2008 off of IIS or the dev server?
Anytime I've experienced strange discrepancies between seemingly identical dev and prod code, I've learned to have a quick look at the machine.config file in production.
If you have access to it, you might compare it to your local copy, and specifically review these settings (although the linked post doesn't refer to your exact problem per se.)
I've come to accept that this may be a processor issue based on this other post:
ASP.NET Website Slow Performance on production server
Thanks to everyone who helped answer.