I am using asp.net MVC framework and using EntityFramework to connect to database. Sometimes while connecting to database I get timeout error. These errors are very rare.
System.Data.Entity.Infrastructure.DbUpdateException:
An error occurred while updating the entries. See the inner exception for details. ---> System.Data.UpdateException: An error occurred while updating the entries.
See the inner exception for details. ---> System.Data.SqlClient.SqlException:
Timeout expired. The timeout period elapsed prior to
completion of the operation or the server is not responding.
var myList = new List<MyConnection>();
using (var db = new MyDb())
{
var activeConns = db.Connections
.Where(c => c.Key == tKey)
.Where(c => c.IsActive)
.ToList();
if (activeConns.Count > 0)
{
//Some custom logic
myList.add(conn);
}
return myList;
}
I am not setting any connection timeout period. Is there any best practice when working with Entity Framework?
How can I generate this kind of defect artificially so I can test our software?
The way we've handled this sort of thing in our code base is to use mock objects - allowing us to swap out the real implementation (for example, database) with a fake implementation used only for testing.
From the client's perspective, the fake one behaves the same, but we have more control over the behaviour for testing things like errors (throw a timeout error on 5th request for example).
Mocking Entity Framework Context is another SO question that goes into a bit more detail on how to set this up, specifically for EntityFramework.
You will probably need to use Reflection to throw SQL Timeout Exception.
Check this article for help :
http://blog.developers.ba/post/2009/03/08/SqlException-class-doesnt-have-public-constructor.aspx
Related
I am binding a task in ews.
In Microsoft docs it says:
"Binds to an existing task and loads its first class properties."
Due to the List of First Class Properties IsReminderSet is a first class property in task.
Still that property returns an
"ServiceObjectPropertyException was unhandeld:
An unhandled exception of type 'Microsoft.Exchange.WebServices.Data.ServiceObjectPropertyException' occurred in Microsoft.Exchange.WebServices.dll"
ews.Task task = ews.Task.Bind(service, ID);
if (task.IsReminderSet)
{
//do something
}
As a start make sure you using the latest version of library from github https://github.com/OfficeDev/ews-managed-api. The other thing you can do is enable tracing https://blogs.msdn.microsoft.com/webdav_101/2015/05/03/ews-best-practices-tracing-and-logging/ this will allow you to see exactly what the server is actually sending back to. Its possible especially if mailboxes has been migrated that these properties can be null on the server side (due to the migration method used).
I am using MVC5 + Entity Framework and I am getting an strange exception.The exception seems to be due to a concurrency problem when multiple users are using the site at the same time. The exception says:
System.NotSupportedExceptionA second operation started on this context before a previous asynchronous operation completed. Use 'await' to ensure that any asynchronous operations have completed before calling another method on this context. Any instance members are not guaranteed to be thread safe.
Now based on the logs, the culprit is the following method:
public async Task<Customer> GetUserByUserIdAsync(string userId)
{
return await DbContext.Users.FirstOrDefaultAsync(u => u.Id == userId);
}
I can't see what is causing it. Looking at the error, which says use await...as you can see I have await behind it.
Anyone can help please?
Thank you.
Edit: My scenario is using Unit of Work and I have got several layers between controller and the depth of my data layer.
I'm trying to do a few integration tests involving an SQL merge procedure through Entity Framework. I have a C# unit test with this in testinitialize:
[TestInitialize()]
public void MyTestInitialize() {
if (Database.Exists("DefaultConnection"))
{
Database.Delete("DefaultConnection");
}
var people = new Person[]
{
new Person()
{
createDate=new DateTime(2005,01,01),
dob=new DateTime(1995,12,25).ToString("M/d/yyyy"),
firstname="Benjamin",
lastname="Jacobs"
},
new Person()
{
createDate=new DateTime(2018,05,13),
dob=new DateTime(2003,12,25).ToString("M/d/yyyy"),
firstname="Kenneth",
lastname="Cobb"
},
new Person()
{
createDate=new DateTime(2018,05,13),
dob=new DateTime(2003,12,25).ToString("M/d/yyyy"),
firstname="Ismael",
lastname="Glover"
}
};
using(MyContext db=new MyContext()){
db.People.AddRange(people);
db.SaveChanges();
}
}
And then I have a few test methods that test merging a few different kinds of data. I want to start with a clean test database like the above. When I run each test individually everything passes. The problem is that when I run all the tests, only the first one passes, and all the subsequent ones will throw an error.
This is the error message (lightly edited to remove identifiable info):
Initialization method MergePeopleTest.MyTestInitialize threw exception. System.Data.Entity.Core.EntityException: System.Data.Entity.Core.EntityException: The underlying provider failed on Open. ---> System.Data.SqlClient.SqlException: Cannot open database "MyDatabase" requested by the login. The login failed.
Login failed for user '[windows user]'..
Deleting and recreating the database is probably not the best approach (I'm open to hearing better ideas) but it seems like this should work. I've done it before and did not have this issue.
Edit: the error is thrown on the second test during TestInitialize on the line where it tries db.SaveChanges();. As best I can tell, the database was deleted but then it didn't automatically recreate the way code-first EF is supposed to.
Background
I have a central database my MVC EF web app interacts with following best practices. Here is the offending code:
// GET: HomePage
public ActionResult Index()
{
using (var db = new MyDbContext())
{
return View(new CustomViewModel()
{
ListOfStuff = db.TableOfStuff
.Where(x => x.Approved)
.OrderBy(x => x.Title)
.ToList()
});
}
}
I also modify the data in this database's table manually completely outside the web app.
I am not keeping an instance of the DbContext around any longer than is necessary to get the data I need. A new one is constructed per-request.
Problem
The problem I am having is if I delete a row or modify any data from this table manually outside the web app, the data being served by the above code does not reflect these changes.
The only way to get these manual edits of the data to be picked up by the above code is to either restart the web app, or use the web app to make a modification to the database that calls SaveChanges.
Log Results
After logging the query being executed and doing some manual tests there is nothing wrong with the query being generated that would make it return bad data.
However, in logging I saw a confusing line in the query completion times. The first query on app start-up:
-- Completed in 86 ms with result: CachingReader
Then any subsequent queries had the following completion time:
-- Completed in 0 ms with result: CachingReader
What is this CachingReader and how do I disable this?
Culprit
I discovered the error was introduced elsewhere in my web app as something that replaced the underlying DbProviderServices to provide caching, more specifically I am using MVCForum which uses EF Cache.
This forum's CachingConfiguration uses the default CachingPolicy which caches everything unless otherwise interacted with through the EF which was the exact behavior I was observing. More Info
Solution
I provided my own custom CachingPolicy that does not allow caching on entities where this behavior is undesirable.
public class CustomCachingPolicy : CachingPolicy
{
protected override bool CanBeCached(ReadOnlyCollection<EntitySetBase> affectedEntitySets, string sql, IEnumerable<KeyValuePair<string, object>> parameters)
{
foreach (var entitySet in affectedEntitySets)
{
var table = entitySet.Name.ToLower();
if (table.StartsWith("si_") ||
table.StartsWith("dft_") ||
table.StartsWith("tt_"))
return false;
}
return base.CanBeCached(affectedEntitySets, sql, parameters);
}
}
With this in place, the database logging now always shows:
-- Completed in 86 ms with result: SqlDataReader
Thanks everyone!
We're using ASP.NET Entity Framework Core for querying our MSSQL database in our Web API app. Sometimes when we have big traffic, querying to DB ends with this error:
Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
I wonder if our pattern of using DbContext and querying is correct or if I am missing some using/dispose pattern and error is caused by some memory leak (after some research I read then I should not use using because the lifetime is managed by the framework). I am following documentation...
My connectionString:
"myConnection": "Server=xxx;Database=xxx;user id=xxx;password=xxx;Max Pool Size=200;Timeout=200;"
My Startup.cs
public void ConfigureServices(IServiceCollection services)
{
.....
// scoped context
services.AddDbContext<MyDbContext>(
options => options.UseSqlServer(this.Configuration.GetConnectionString("myConnection")));
}
then in controllers I used dbcontext by dependency injection:
public class MyController : Controller
public MyController (MyDbContext context)
{
this.Context = context;
}
public ActionResult Get(int id)
{
// querying
return this.Context.tRealty.Where(x=>x.id == id).FirstOrDefault();
}
Should I use something like:
using (var context = this.Context)
{
return this.Context.tRealty.Where(x => x.id == id).FirstOrDefault();
}
But I think that this is bad pattern when I am using dependency injection of DbContext.
I think problem was caused by storing objects from database context queries to In memory cache. I had one big LINQ query to database context with some other subqueries inside. I called FirstOrDefault() on the end of main query but not inside subqueries. Controller was fine with it, it materialize queries by default.
return this.Context.tRealty.AsNoTracking().Where(
x => x.Id == id && x.RealtyProcess == RealtyProcess.Visible).Select(
s => new
{ .....
// subquery
videos = s.TVideo.Where(video => video.RealtyId == id && video.IsPublicOnYouTube).
Select(video => video.YouTubeId).ToList()), // missing ToList()
.....
}).FirstOrDefault();
And there was problem - subqueries were holding connection to database context when they where storing to In memory cache. When I implemented Redis distributed cache, it was first failing on some strange errors. It helps when I write ToList() or FirstOrDefault() to all my subqueries because distributed cache needs materialized objects.
Now I have all my queries materialized explicitly and I got no max pool size was reached error. So that one must be careful when stored objects from database context queries to In memory cache. It is need to materialize all queries to avoid to holding connection somewhere in memory.
You can set the lifetime of the DbContext in your startup.cs, see if this helps:
services.AddDbContext<MyDbContext>(options => options
.UseSqlServer(connection), ServiceLifetime.Scoped);
Also if your query is a simple read you can remove tracking by using .AsNoTracking().
Another way to improve your throughput is to prevent locks by using a transaction block with IsolationLevel.ReadUncommitted for simple reads.
You can also use the Snapshot isolation level - which is slightly more restrictive - if you do not want dirty reads.
TransactionOptions transactionOptions = new TransactionOptions() { IsolationLevel = IsolationLevel.ReadUncommitted};
using (TransactionScope transactionScope = new TransactionScope(TransactionScopeOption.Required, transactionOptions))
{
// insert magic here
}
Edit : As the author of the question mentioned, the above code is not (yet?) possible in EF Core.
A workaround can be found here using an explicit transaction:
using (var connection = new SqlConnection(connectionString))
{
connection.Open();
using (var transaction = connection.BeginTransaction())
{
// transaction.Commit();
// transaction.Rollback();
}
}
I have not tested this.
Edit 2: Another untested snippet where you can have executed commands to set isolation level:
using (var c1= new SqlConnection(connectionString))
{
c1.Open();
// set isolation level
Exec(c1, "SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;");
Exec(c1, "BEGIN TRANSACTION;");
// do your magic here
}
With Exec:
private static void Exec(SqlConnection c, string s)
{
using (var m = c.CreateCommand())
{
m.CommandText = s;
m.ExecuteNonQuery();
}
}
Edit 3: According to that thread, Transactions will be supported from .NET Core version 1.2 onwards.
#mukundabrt this is tracked by dotnet/corefx#2949. Note that
TransactionScope has already been ported to .NET Core but will only be
available in .NET Core 1.2.
I am adding an alternative answer, in case anyone lands here with a slightly different root cause, as was the case for my .NET Core MVC application.
In my scenario, the application was producing these "timeout expired... max pool size was reached" errors due to mixed use of async/await and Task.Result within the same controller.
I had done this in an attempt to reuse code by calling a certain asynchronous method in my constructor to set a property. Since constructors do not allow asynchronous calls, I was forced to use Task.Result. However, I was using async Task<IActionResult> methods to await database calls within the same controller. We engaged Microsoft Support, and an Engineer helped explain why this happens:
Looks like we are making a blocking call to an Async method inside
[...] constructor.
...
So, basically something is going wrong in the call to above
highlighted async method and because of which all the threads listed
above are blocked.
Looking at the threads which are doing same operation and blocked:
...
85.71% of threads blocked (174 threads)
We should avoid mixing async and blocking code. Mixed async and
blocking code can cause deadlocks, more-complex error handling and
unexpected blocking of context threads.
https://msdn.microsoft.com/en-us/magazine/jj991977.aspx
https://blogs.msdn.microsoft.com/jpsanders/2017/08/28/asp-net-do-not-use-task-result-in-main-context/
Action Plan
Please engage your application team to revisit the application code of above mentioned method to understand what is going
wrong.
Also, I would appreciate if you could update your application logic to
not mix async and blocking code. You could use await Task instead of
Task.Wait or Task.Result.
So in our case, I pulled the Task.Result out of the constructor and moved it into a private async method where we could await it. Then, since I only want it to run the task once per use of the controller, I store the result to that local property, and run the task from within that method only if the property value is null.
In my defense, I expected the compiler would at least throw a warning if mixing async and blocking code is so problematic. However, it seems obvious enough to me, in hindsight!
Hopefully, this helps someone...