Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have a UI which calls WebAPIs (WebAPI 2.0), Web API are basically LINQ queries (to MS SQL database) and some processing logic for the data. I want to do performance evaluation of the complete flow (click from UI to API, back to UI to display data) upon a huge DB with 30K - 60K records in it.
How can it be done? Let me know the methods/tools used for this.
currently I am tracking time-taken in chrome debug window, which shows the total time for each network call.
Wow. This is a subject in its own right but here's an approach:
The bits are independent so you break it down. You measure your LINQ queries without any of the logic or web api stuff getting in the way. If LINQ is against stored procedures then measure those first. Then you measure the cost of the logic, then you measure the cost of sending X rows of data using WebAPI. You should avoid including the cost of actually retrieving the rows from the database so you're checking just the connectivity. I'd also consider writing a browserless test client (i.e. GETS/POSTS or whatever) to eliminate the browser as a variable.
Now you've got a fairly good picture of where the time gets spent. You know if you've got DB issues, query issues, network issues or application server issues.
Assuming it all goes well, now add a bunch of instances to your test harness so you're testing concurrent access, load testing and the like. Often if you get something wrong you can't surface that with a single user so this is important.
Break it down into chunks and have a data set that you can consistently bring back to a known state.
As for tools, it really depends on what you use. VS comes with a bunch of useful things but there are tons of third party ones too. If you have a dedicated test team this might be part of their setup. SQL Server has a huge chunk of monitoring capability. Ask your DBAs. If you've got to find your own way, just keep in mind that you want to be able to do this by pressing a button, not by setting up a complex environment.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I am creating a Zillow-like IOS app. I am using Azure SQL (S-0 version) and C# for my backend. I store users data including the avatar of each user and the trips the person has been to (another table), where each user can have multiple trip pictures. I am using varbinary(max) to store those image types. So each user entity would have an average of 3 pictures.
my C# app retrieves all users with all their images and trip info and images using lazy loading in my linq-to-sql query.
The issue is naturally the time it takes to load all those date before user becomes able to consume it in the iphone clients (written in Swift). The end user will need all data to scroll through all those images. He at first load users, to see his images and read about him, and then scrolls up and would see trips the other user has gone to. It won't make sense for the user to see only avatars and then wait for all trips for example, every time he calls.
What strategy should I do to minimize the load time!?
The thoughts I have had so far:
Use Azure Blobs to store images and consume that right from my swift clients to see if it will be any faster. I ditched that option because the extra slow retrieval of files because I have to first access the image file from my C# code, and make binary out of it so I can transmit it to my iphone client app.
Instead of using linq-to-sql, use the good ole ADO.NET to minimize the extra layer linq-to-sql. I didn't see much difference not even a second extra.
optimize my linq query by adding as much as possible in the where clause and avoid using loops. Make them all one select statement. The query spat by the linq-to-sql seems right. It didn't change much because the issue is the extra huge binary write ups being retrieved.
I mentioned it already above, using silent calls where I load only the user-related image and then when user sees summary, behind the scene I start loading all those trips, but the user will quickly need to check the trips. He will be confronted of having to wait again to see those trips. And I would have to initiate new calls to DB, which is expensive. Very undesirable!
What other suggestions should I try?
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 6 years ago.
Improve this question
I have to execute a long running PL/SQL query and need to store the result set somewhere and serve to UI request. After some time need to refresh it automatically / manually.
there will be multiple result sets need to be handled in the same way. Each and every result set will have millions of records.
My project is using AngularJS with Web API.
I am using ADO.net with Oracle Client, not entity framework.
What I feel, .net MemoryCache is not suitable because the no.of resultset and size will be keep on growing.
I am planning to cache it in MongoDb. is there any other solution you suggest?
Thanks
Your question says "Each and every result set will have millions of records"
In that context,
Caching in server is not a good idea. Even distributed caching may result in performance issue.
Create dedicated physical tables in oracle to store your results
Load & refresh the result table whenever required
Fetch and return the result from results table
If you can't insert/delete records from your oracle database you may need to go with a dedicated application database.
Even if you return millions or records from web API, the angular application should process all your data. That might result in long running JavaScript and client side performance.
Redis is a good solution. Check out http://redis.io/
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I am trying to store a software "Calender"(complex logs that saved and browsed based on date).
I tried using both .ini and xml but when the application tried to read the entire file to find info for 1 specific day out of the 100 days (or so it seemed), it took almost 9 seconds to get 5 variables out of 500 variables. The size of the actual file might eventually be more than 40 variables per day.
Also, I would rather not make a file for each day, that will seems a little bit unprofessional and messy.
I am asking the question to know if there is an alternative to keep things fast and neat. The data includes different types of variables and different amounts of them. I know i am kinda overdoing it with logging thing but the program needs logs to do its work
If the data must be stored it has to be a file or a database (local or remote), I'd go for SQLite, it would end in a single file, but you could query the data with SELECT, JOIN, etc.
EDIT:
You can use SQLite3 from c# if you include this package:
https://www.nuget.org/packages/System.Data.SQLite/
You'll need to learn some SQL, but after that you'll just use something like:
select Message from Logs where Date > '2015-11-01' and Date < '2015-11-25';
which is easier, faster and clearer than messing with XML, and it will not load the whole file.
As mentioned above, SQLite will offer a great possibility. Since you (generally), and probably not a lot of people out here will be able to write a database management system that is as efficient as the ones out there.
https://www.sqlite.org
Whole point of using RDBMS because it's far more efficient that dealing with files.
SQL Lite is light weight and easier to deploy. But remember that,
SQLite only supports a single writer at a time (meaning the execution
of an individual transaction). SQLite locks the entire database when
it needs a lock (either read or write) and only one writer can hold a
write lock at a time. Due to its speed this actually isn't a problem
for low to moderate size applications, but if you have a higher volume
of writes (hundreds per second) then it could become a bottleneck.
Reference this question
If this is an enterprise level application requirement I would go for Azure Table storage based solution which is identical for this sort of scenario.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
Hi, I am relatively new to using databases in a web application and I am trying to develop a dynamic application for my users.
So, I was wondering how safe is it to have your application frequently (say every 2 seconds) execute an INSERT/UPDATE query from the same user.
I'm aware that INSERT/UPDATE queries are quite slow to execute (I've read its 12 INSERT per second) but that's not my question.My question is concerning the safety of my database.
The frequency of executing INSERTs or UPDATEs to your database is in no way related to the safety of of your database. But it might impact its performance, that's possible.
I think the concern of the OP is more about the robustness of the database's ability to handle load and not become corrupt, rather than issues of SQL injection. (Valuable comments to take note of though). The volumes suggested per user should no concern.
Database integrity for any DB should be checked with regular maintenance. Make sure you are doing the following and you should have no problems with performance and reliability.
Back up your DB. Full backup and transaction log backups.
Re-build and/or Re-Organize indexes
Delete old data backups and check free space often.
Check the maintenance logs for issues on your DB.
Then monitor performance for sufficient Memory, DISK I/O, and CPU.
wihout "with(nolock)" segment after the table name, the table being insert/update is always locked.
if so, how can it performance well
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I created a wpf browser application ,and I hosted it in a live server ,it is a Inventory Management application ,which uses database heavily for saving, updating and selecting data.On save and select operation it takes very long time to save and fetch data to and from database.In the meantime (when save or select data from database) the application is irresponsive .How to make the application fast when dealing with database and how to make the application responsive during that operation .I am totally messed up ,its very kind if anyone has the answer .
Is using stored procedures instead of raw sql bring performance or not?
Edit
My application interact with database by using raw sql statements.
Responsive Application
In order to make your application responsive you need to make every interaction with the database in a new thread or task. You could also use the new C# 5.0 features await and async.
Your current application is blocking because the main thread is handling database operations as well as user interface, so what you need to do is to handle these two separate functionalities in separate threads.
Fast Database Access
I don't know how are you dealing with the database right now, but you could definitely use some kind of ORM like Entity Framework or NHibernate.
For better performance, but much less readable and mantainable code, which is really important, you could use raw sql.
try this
to access DB faster in .NEt i am sure it not problem with c# but for querying and retireving DB u need to have better mechanism