I'm having an issue with a really simple password hash retrieval LINQ query. The problem is if the user logs out, then tries to log back it, it just uses the cached values of the query without querying the database again. The query in question is the following:
using (var db = new DataModel.DatabaseContext())
{
return (from emp in db.Employees where emp.Username == username select emp.Password).SingleOrDefault();
}
But when I break, it seems that EF IS executing a reader on a separate thread! Then why do I think it isn't really querying the database? Well the execution time is just too short. It messes up my async methods, it basically doesn't leave enough time for a MessageBox to be shown (works properly when I call the method for the first time). Maybe the database itself has some transient options set up?
EDIT: I thought I found out what the problem is but this is just unreal. It executes the query on a remote server faster than a ping request. <0.001s I'm stumped
It is because the first time you create a DbContext in your AppDomain (maybe first call to new YourDbContext() in your application) there is going a lot of initialization and configuration under the hood so it takes some time the first time, but after that (while application is running) the process speeds up so you can't feel it.
Related
In my website, I am using ASP.NET MVC 5 with EF6.
I experience slow performance with only 20K records for the first call.
For example, I need to get all rows from the Person table.
First invocation: 7500 ms (after that in the second row it takes only 1000ms)
List<Person> persons = await _context.Person.ToListAsync(); // Time : 7500ms
List<Person> persons2 = await _context.Person.ToListAsync(); // Time : 1000ms
What I tried:
Canceled the lazyload from edmx schema
Refresh the schema
The same query in SQL Server Management Studio takes 400 ms (and it's a really simple query without joins and conditions)
and it happens every time client goes to the person page
I would have posted this in a comment, but it's too long.
There are many things that can factor into that time difference, in order of less likely/impactful to more likely/impactful:
The first query, once in SQL Server (if that's the underlying engine) has to "Warm Up" SQL sometimes. I doubt that this is the actual problem since SQL Server probably hasn't enough time to go "Down" between your tries. Also, the execution plan shouldn't be too problematic for that query.
The first query has to open the communication channel. For example, if it has to route through VPNs, or simply open an SQL connection, it adds up a delay.
Migrations: Unless you manually force migrations, when you create the DbContext, EF6 doesn't run the migrations (and Seeding) at that moment. It waits for the first time it actually has to query, then builds the configurations and execute migrations.
If you want to investigate, put a breakpoint in the OnModelCreating method and see when it's called. You can also add another query before these two queries to an unrelated entity and you'll see that it's not because of caching (AFAIK, the Caching is only used when using DbSet<T>.Find(...))
I have a standard ASP.NET web page. It issues a query, using Ajax, to a SQL Server, and returns a table of results. The problem is, sometimes this table of results is very large and the query takes too long. (I don't have control over the SQL, this happens via a stored procedure.)
Is there a way to have a "Cancel Request" button on the page, so that when the user clicks the button the query on the SQL server is killed? If so, how would I do that? (I am new to ASP.NET/C#, but understand the architecture of web requests.) Thanks.
One approach:
Create the connection, and place it in a Dictionary, with a Guid.ToString() as key.
Run the query and return the key to your webpage, and save it somewhere.
If the query finishes the execution ok:
Find the connection, close it, and remove it from the dictionary.
If the user click on cancel query:
Send an ajax request to the web server with the key you saved.
Find the connection, close it, and remove it from the dictionary.
Make sure of locking the dictionary.
Make sure of catching exceptions.
I would think that would be extremely difficult to do, because you would need to know the specific spid for the exact request that issued the long running request. Something that will be coming from the same user as many more valid requests (if your site is set up like most).
I'm working on the same thing, but I'd like to offer some corrections to the above statements. The guy who said finding a SPID is difficult is incorrect. I've just implemented a system where when certain long running stored procedures run, they put (or update) a record into a table that I store that contains SPIDs, the User running the report, some report information, and the date started. Using ##SPID within the stored procedure gets me the SPID that I store in the table.
Also, the connection closing not ending the query is correct, you need a KILL statement.
I have asp.net page which allow users to run select queries on oracle data base. then result is shown on web page as a data table.
Some time users may enter queries that impact badly on DB due to long and deep query execution.
Is there any possibility to mention time out for each query run on DB . If query runs longer than specified time execution must be stop on database.
You can solve this problem on the application or DB side:
Application Side: SQLQueryTimeout
You can set an SQLQueryTimeout generally for all Statements on the Driver, or Individually with CommandTimeout for a certain command. If a query takes longer than that a TRAP is invoked and you get an Exception whcih you can Handle.
The Second possibility is to set a Timeout for an Oracle-user. If your Web-Application connects to the database via a shared user (as it should be) you can use ALTER USER to set a User Profile which will enforce a MAX CPU LIMIT, which is in effect a time-limit on all SQL Statments issued by that user.
The beauty of the second solution is that the Database enforces the rule, so you don't have to worry about your application code where you might miss a line...
We have a website which is using linq to entities, we found that it's very slow recently, after troubleshooting, I found whenever we use linq to entities to search data from database, it will consume very much CPU time, like toList() function. I know it might because we have lots of data in database which caused to slow response, but I just wonder if there are any other reasons which might cause this problem?
How should I do to optimize these kinds of problem? following is the possible reasons:
ToList() might load all object's foreign object(foreign key), how can I force it only load the object?
Is my connection pool too small?
Please let me know if there are any other possible reason, and point me the right direction to solve this issue.
In Linq - a query returns the results from a sequence of manipulations to sources when the query is enumerated.
IQueryable<Customer> myQuery = ...
foreach(Customer c in myQuery) //enumerating the query causes it to be executed
{
}
List<Customer> customers = myQuery.ToList();
// ToList will enumerate the query, and put the results in a list.
// enumerating the query causes it to be executed.
An executing query requires a few things (in no particular order)
A database connection is drawn from the pool.
The query is interpreted by the query provider (in this case, the provider is linq to entities and the interpretation is some form of sql)
The interpretted form is transmitted to the database, where it does what it does and returns data objects.
Some method must be generated to translate the incoming data objects into the desired query output.
The database connection is returned to the pool.
The desired query output may have state tracking done to it before it is returned to your code.
Additionally, the database has a few steps, here listed from the point of view of querying a sql server:
The query text is recieved and checked against the query plan cache for an existing plan.
If no plan exists, a new one is created and stuck into the plan cache by the query optimizer.
The query plan is executed - IO/locks/CPU/Memory - any of these may be bottlenecks
Query results are returned - network may be a bottleneck, particularly if the resultset is large.
So - to find out where the problem with your query is, you need to start measuring. I'll order these targets in the order I'd check them. This is not a complete list.
Get the translated sql text of the query. You can use sql server profiler for this. You can use the debugger. There are many ways to go about it. Make sure the query text returns what you require for your objects, no more no less. Make sure the tables queried match your expectations. Run the query a couple times.
Look at the result set. Is it reasonable or are we looking at 500 Gigs of results? Was a whole table queried, when the whole thing wasn't needed? Was a cartesian result generated unexpectedly?
Get the execution plan of the query (in sql studio, click the show estimated execution plan button). Does the query use the indexes you expect it to? Does the plan look wierd (possibly a bad plan came from the cache)? Does the query work on tables in the order you expect it to, and perform nested/merge/hash joins in the way you expect? Is there parallellization kicking in, when the query doesn't deserve it (this is a sign of bad indexes/TONS of IO)?
Measure the IO of the query. (in sql server, issue SET STATISTICS IO ON). Examine the logical IO per table. Which table stands out? Again, look for a wrong order of table access or an index that can support the query.
If you've made it this far, you've likely found and fixed the problem. I'll keep going though, in case you haven't.
Compare the execution time of the query to the execution time of the enumeration. If there's a large difference, it may be that the code which interprets the data objects is slow or that it generated slow. It could also be that the translation of the query took a while. These are tricky problems to solve (in LinqToSql we use compiled queries to sort them out).
Measure Memory and CPU for the machine the code is running on. If you are capped there, use a code profiler or memory profiler to identify and resolve the issue.
Look at the network stats on the machine, in particular you may want to use TCPView to see the TCP socket connections on the machine. Socket resources may be mis-used (such as opening and closing thousands in a minute).
Examine the database for locks held by other connections.
I guess that's enough. Hope I didn't forget any obvious things to check.
You might find the solution to your problem in Performance Considerations (Entity Framework) on MSDN. In particular
Return the correct amount of data
In some scenarios, specifying a query path using the Include method is
much faster because it requires fewer round trips to the database.
However, in other scenarios, additional round trips to the database to
load related objects may be faster because the simpler queries with
fewer joins result in less redundancy of data. Because of this, we
recommend that you test the performance of various ways to retrieve
related objects. For more information, see Loading Related Objects.
To avoid returning too much data in a single query, consider paging
the results of the query into more manageable groups. For more
information, see How to: Page Through Query Results.
I have a webpage that takes 10 minutes to run one query against a database, but the same query returns in less than a second when run from SQL Server Management Studio.
The webpage is just firing SQL at the database that is executing a stored procedure, which in turn is performing a pretty simple select over four tables. Again the code is basic ADO, setting the CommandText on an SqlCommand and then performing an ExecuteReader to get the data.
The webpage normally works quickly, but when it slows down the only way to get it speeded up is to defragment the indexes on the tables being queried (different ones different times), which doesn't seem to make sense when the same query executes so quickly manually.
I have had a look at this question but it doesn't apply as the webpage is literally just firing text at the database.
Does anyone have any good ideas why this is going slow one way and not the other?
Thanks
I would suspect parameter sniffing.
The cached execution plan used for your application's connection probably won't be usable by your SSMS connection due to different set options so it will generate a new different plan.
You can retrieve the cached plans for the stored procedure by using the query below. Then compare to see if they are different (e.g. is the slow one doing index seeks and bookmark lookups at a place where the other one does a scan?)
Use YourDatabase;
SELECT *
FROM sys.dm_exec_cached_plans
CROSS APPLY sys.dm_exec_sql_text(plan_handle)
CROSS APPLY sys.dm_exec_query_plan(plan_handle)
cross APPLY sys.dm_exec_plan_attributes(plan_handle) AS epa
where sys.dm_exec_sql_text.OBJECTID=object_id('YourProcName')
and attribute='set_options'
Is there any difference between the command text of the query in the app and the query you are executing manually? Since you said that reindexing helps performance (which also updates statistics), it sounds like it may be getting stuck on a bad execution plan.
You might want to run a sql trace and capture the showplanxml event to see what the execution plan looks like, and also capture sql statement complete (though this can slow the server down if a lot of statements are coming through the system so be careful) to be sure the statement sent to SQL server is the same one you are running manually.