I have a web site which uses one SQL database but the hosting company is very slow sometimes and I got database timeout, login and similar errors. Can I implement my code to use two databases simultaneously? I have stored procedures and the data is updated periodically.
EDIT:
Simply: When dbDefault is down and inaccessible I need to use dbSecondary so the web app keeps running. Ant these two databases must be always same.
EDIT:
Some errors:
A transport-level error has occurred when sending the request to the server. (provider: TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.)
A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)
Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Cannot open database "db" requested by the login. The login failed. Login failed for user 'root'.
Load balancing and/or fail-over clustering database servers typically involves a lot of work.
You will need to make sure ALL data is merge replicated between the two database servers. Hosting providers rarely provide this option unless you have a dedicated server.
Allowing for merge replication might involve redesigning parts of your database; which may not be feasible.
Unless you are willing to invest a lot of time and money, you are much better off just switching hosting providers to one that has better db support. Considering there are literally thousands upon thousands of such companies out there this is an easy fix.
UPDATE
Almost of all the errors you identified in your edit are generally attributable to failing to properly dispose of connections, commands, and readers. You might want to go through your code to make sure you are accessing the sql server correctly. Every connection, command, and reader should be wrapped in a using clause in order to make sure they are properly released back to the connection pool.
If you provide a data access code sample (new question please) we can help you rewrite it.
Not really.
Data consistency and integrity:
How do you decide what data or what call to make at what time?
What happens on write?
Firewalls, remote server etc:
If you use another hosting company, how will you connect?
Misconception:
Two databases on one server = absolutely no advantage.
The server is probably overloaded, a 2nd db will make it worse
A database timeout could be code related of course and it may not help you to have 2 databases with the same poor code or design...
Not a nice answer, but if your host is providing poor service then your options are limited
ist of all find the reasons of the Timeout if it is in your code than rectify the code by optimizing query etc.
i think what you need is a Failover Server , where you can switch if the one server is down.
Alternatively
you can maintain two connection string in web.config and can switch to other server if one is down.
in both the method , you need to devise an strategey to sync the servers.
If both your database are in synch (which is an obvious requirement for what you are trying to do), the best solution is to rely on a loadbalancer but if you can't, I guess you goal is to run the query against both database in the same time and returns the first result otherwise you will have to wait for the timeout to run the request against the second server.
SO what you need is asynchronous sql command right ?
Related
I'm running a load test on my system. At a certain level of load, I start getting SQL errors in my log:
System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Prprovidererror: 40 - Could not operrorconnection to SQL Server) ---> System.ComponentModel.Win32Exception (0x80004005): The network path was not found
By running performance monitor on the SQL server in question, I found the following:
CPU level rarely exceeds 50%. (On a previous iteration I saw that it was maxing out at 100%, so I increased the specs of the VM, which helped push the problem to a higher load level.)
Number of user connections got to a shade over 8,000. The Sql Server has the default setting of 32,767 max connections.
The connection string specifies a max pool size of 1000 connections to each database, and there are 100 databases on the server. The load test is randomly distributed between the 100 databases, so there should be a fairly even distribution, meaning about 80 connections per database. Nowhere near the 1k limit.
What other factors might cause Sql Server to stop being able to accept connections?
UPDATE: extra info: I'm using Entity Framework Core (EF7) for my DB connections, if that helps any.
"Network Path Not Found" does not seem like an error related to SQL Server's capacity. As a former "IT Guy," I suspect that a firewall is dropping your packets. If this is during a stress test, the firewall could be interpreting the numerous requests as a denial of service attack, and using some sort of predefined rule to drop connections for a specified time period.
What is your network environment? If you have a hardware firewall or router with IPS capabilities, I would check those logs to see if you find a smoking gun. You might have to create a special rule to allow unlimited traffic to your SQL Server.
It's a bit curious that you are getting that many connections to the database. You should be utilizing connection pooling; even under intense load, the connection pooling should greatly reduce the number of active connections being used.
Can you provide the code that's accessing the database? Are you calling the dispose() method or closing the connection?
Also, have you looked to see if data datacaching would ease the db load? A 2-5 second datacache can greatly reduce database calls.
You are running into the TCP listen() backlog limit for the SQL-Server's listening port. When this happens, Windows platforms (but not *nix platforms) will issue 'connection refused' for further incoming connections.
I'm not an SQL-Server guy but there is bound to be a parameter somewhere by which you can increase its listen backlog.
Alternatively you should look into better or more connection pooling at the client.
It turns out the problem wasn't on SQL at all. The problem was on our API server, where some of the APIs were spinning off hundreds of parallel threads, each making its own connection to the database. The load was simply too much for the API server, and it started returning "Access Denied" exceptions without even really attempting to connect to the database.
Solution: we throttled the number of threads being spun off, using the pattern shown in this answer.
I have a server and 'x' number of clients.
When the server is listening for an inbound connection, it will create a client handler instance (a class that manages the client communication) which will be spun off in a separate thread.
Depending on the command the client sends to the server, the server may need to access a SQL database to store information about that client.
The client handler instance will 'handle' this request. The only problem is, if multiple client handlers are wanting to access the SQL database to do the exact same thing then there is potential for read / write issues.
I was thinking about exposing a static method on the server, calling it from the client handle instances, then locking the function which accesses the SQL database (either read or write).
Is this a good approach or are there better approaches?
Thanks.
Well, you DO know that SQL has locks and a ton of internal mechanisms to serialize access? That this is part of the ACID conditions ingrained in SQL since the 1950's when SQL was created? That the locking mechanism in SQL is very very fine and basically you try to solve a problem that has been solved more than 60 years ago.... because it seems you need to read a book about the basics of SQL.
Under normal circumstances (standard connection string) resource access is serialized in SQL Server (TransactionIsolationLevel serialized), but that can be tuned. I really suggest learning some SQL fundamentals.
In my C# application I connect to a MySQL database and run 10,000 queries. If I keep a connection to my database, these queries take roughly 14 seconds. However, if I rely on the connection pooling my queries take around 15 seconds. (I have run this test multiple times.)
// Connection pooling.
using (var connection = CreateConnection())
{
connection.ConnectionString = ConnectionString;
connection.Open();
Most samples on the net make use of the 'connect and close' construction above. However, it seems connection pooling is slower than keeping the connection. So the question is...
Q: Why should I use connection pooling?
Its a big debatable topic and would find many blog out there would tell that why we use Pool.
It will not slow things down. There is a lot of time spend on Connecting to DB Server and Hand shake and establishing communication between client and DB server.
So in multi request paradigm where many request are entertained by the server, it would be hard to establish and put on wait each client. POOL helps us that it gives us pre prepared connection and we use it and discard it. POOL get that connection and re-establish it for the next request.
But in a single threaded environment it is the other way around. POOL would be a very heavy resource for a single threaded env.
Q: Why should I use connection pooling?
Usually so that you can use more than one connection at a time. This is clearly important for web applications - you wouldn't want one user query to have to wait for another user's query to finish.
If you're writing a thick client application which talks straight to the database and you know you'll only ever have one query executing at a time, it's less important - but it's still global state, and that tends to be something you should avoid. You're doing several independent things - why would you want to constrain them to use the same connection?
Connection pooling is great for scalability - if you have 100 threads/clients/end-users, each of which need to talk to the database, you don't want them all to have a dedicated connection open to the database (connections are expensive resources), but rather to share connections (via pooling).
The using mini-pattern is also great for ensuring the connection is closed in a timely fashion which will end any transactions on the connection and thus ensure any locks taken by the transactions are released. This can be a great help for performance, and for minimising the potential for deadlocks.
If all your application does is run the 10,000 queries and then close again without any user interaction then it's fine to use one single connection.
However it's generally not a good idea to keep a database connection open while your application is just sitting there waiting for user input. This is where connection pooling is appropriate.
Pseudo code ...
<open connection>
<fetch data>
<close connection>
<user interaction with data ...>
<open connection>
<save updated data>
<close connection>
Depending on the language / database used, the second connection will be generated from the connection pool.
My existing scenario is below:
I have an SQL server which resides in a main company office. The company has another branch which is 60 miles apart.
I have a WPF application installed in computers in the main office and the branch which connects to the SQL server in the main office for printing records etc.
I am specifying the connection parameters in the app.config file as below:
<add name="CompanyEntities" connectionString="metadata=res://*/LabModel.csdl|res://*/LabModel.ssdl|res://*/LabModel.msl;provider=System.Data.SqlClient;provider connection string="Data Source=publicIPaddress of remote SQL here;Initial Catalog=databasename;Persist Security Info=True;User ID=sa;Password=password;MultipleActiveResultSets=True"" providerName="System.Data.EntityClient" />
My problem is that the application in the branch hangs for ever at certain times. So my question is whether its the best practice to use WCF for connecting to the remote SQL server?
Which is the best way to go about it? Is there any links which best discribes this?
In your current scenario, WCF is useless to you. If you write a server side application, that manages the database connection, you can use WCF to send the data to your clients. However that requires your client side software adapted to the use of WCF also, but in that case your clients wouldn't access the database (you would change from a two-layer architecture to a three-layer architecture). It might solve your problems, and it might introduce some other problems.
The hanging problem you describe could be caused by many things. For example you can run out of database connections, get into a deadlock (altough that transaction would be terminated by the server normally), or just simply have a lock on data being edited, and the employee using it going for a break.
It is also possible, that the problem is not with the database connection, but something in the client side code. Since I have no details, I can not tell you anything more specific.
Picking on the wording you chose, I don't think you'll be "connecting to the remote SQL server" using WCF. However, you can certainly use WCF to host a service that provides your client (WPF app) with secure access to your data, though it will do so indirectly. This approach will be quite different from the approach you currently have: accessing the server directly from the WPF app.
Although I haven't used it myself, I believe one approach may be WCF Data Services. If you only need a few operations in your service (e.g. GetRecord) for printing you may also be able to just roll your own WCF service that just provides "Records" to your client app. Any introductory book or tutorial on WCF will probably get you on your way.
The above answers the questions you seemed to be asking. However, WCF doesn't solve connection issues for you: if you say "the application [...] hangs forever at certain times" you should investigate that no matter what. But that's a different question.
I am looking for a way to detect if a SQL Server fails to respond (timeout/exceptions etc) so that if our C#/.net program trying to access the server asks a server that is down for maintenance, it will jump and try another automatically.
I would like a solution where we do the SQL connection and THEN get the timeout. We could build a WCF service and ask that one, but that is a bit of an overkill in my opinion.
The sync between multiple server locations is not the issue here.
Our development platform at the moment is SQL2008express as its plenty at the moment, but later on we might switch to real SQL2008 servers (or whatever is latest when the time comes).
Clients will connect to "first known" in a "last known dynamic list" asking a "rootserver" or hardcoded configs for first lookup.
When clients loses connections, they will automatically have to try to reconnect to other nodes in the clusters and use whatever returns a reply first. The nodes will individualle connect and share data through other services which we also distribute in the cloud.
We know that mirroring and clustering might be available through large licenses, but our setup demands a more dynamically "linking" and we believe this approach suits our needs better.
So... to be specific:
we need to detect when a SQL-server goes offline, when its not available anymore. Eg. in the middle of a transaction or when we try to connect.
Is the best approach to do a "try-catch" exception handling or is there better tricks when looking over WAN's and using C#/.net =
EDIT
I've received a lot of good ideas to use failover servers, but I would like a more programatical approach, so whats the best way to query the server if its available?
Situation:
4 different SQL servers running on seperate WAN/IP's, each will maintain a list of "where the others are" (peer-to-peer). They will automatically be moving data from each other (much like a RAID-setup where data is spread out on multiple drives)
A client retries the list from an entry-point-server and asks the first available.
If the client asks a server that is "down for maintance" or the data has moved to one of the other servers, it must automatically ask the next in the list.
What we are looking for..
is the best way from within C#/.net to detect that the server is currently unavailble.
We could have a service we connect to and when we loose this, the server is off
We could make a "dbConnectionSqlServer3.open()" and wait for the time out.
We could invest in "real cluster servers", and pay a lot + bind ourselfs to 1 SQL-server type (The SQL platform might change in future, so this is not a real option)
So whats your vote here: 1 or 2?
or can you come up with a number 4 that beats the h**k out of ideas? :o)
I would probably take a different approach.
I'd designate a server (with a failover partner) to be the holder of the server list. It should monitor all of the other servers for availability.
When the client initially wants to connect, it should contact the list server and ask it what to use. From that point forward the client should stick with it, unless a failure is detected. At which point it should contact the list server to get a new one.
The list server would be able to implement any type of load balancing you wanted just by telling the clients which one to connect to. Further, deploying new servers would be easy as the only "list" would be maintained by this primary server.
This completely ignores any sort of server synchronization issues you might have.
One major benefit is that by having a central server doing the monitoring your clients won't have to fall through 3, 5, or 10 servers just to find one that's up. which would increase responsiveness.
UPDATE
#BerggreenDK: The best, and probably only assured way, to detect if a server has failed is to connect to it and run a simple query. All other mechanisms such as pinging, a simple check if the port is open, etc. might give a false positive. Even having a process running on that server itself may give a false reading if the server itself is up but SQL Server is down (e.g: the database is offline).
Ultimately it sounds like your client will have to connect to one of the gateway servers and get a short list of sql servers to attempt to connect to. On failure, it will have to rotate through that short list. If all are down it will have to go back to the gateway and ask for a new list to start the process over.
In the ConnectionString you can specify a failover partner like
Data Source=<MySQLServer>;Failover Partner=<MyAlternateSQLServer>;Initial Catalog=<MyDB>;Integrated Security=True
On http://www.connectionstrings.com there is a section talking of DataBase mirroring.