What's the best way to check if a table exists in a Sql database in a database independant way?
I came up with:
bool exists;
const string sqlStatement = #"SELECT COUNT(*) FROM my_table";
try
{
using (OdbcCommand cmd = new OdbcCommand(sqlStatement, myOdbcConnection))
{
cmd.ExecuteScalar();
exists = true;
}
}
catch
{
exists = false;
}
Is there a better way to do this? This method will not work when the connection to the database fails. I've found ways for Sybase, SQL server, Oracle but nothing that works for all databases.
bool exists;
try
{
// ANSI SQL way. Works in PostgreSQL, MSSQL, MySQL.
var cmd = new OdbcCommand(
"select case when exists((select * from information_schema.tables where table_name = '" + tableName + "')) then 1 else 0 end");
exists = (int)cmd.ExecuteScalar() == 1;
}
catch
{
try
{
// Other RDBMS. Graceful degradation
exists = true;
var cmdOthers = new OdbcCommand("select 1 from " + tableName + " where 1 = 0");
cmdOthers.ExecuteNonQuery();
}
catch
{
exists = false;
}
}
If you're trying for database independence you will have to assume a minimum standard. IIRC The ANSI INFORMATION_SCHEMA views are required for ODBC conformance, so you could query against them like:
select count (*)
from information_schema.tables
where table_name = 'foobar'
Given that you are using ODBC, you can also use various ODBC API calls to retrieve this metadata as well.
Bear in mind that portability equates to write-once test anywhere so you are still going to have to test the application on every platform you intend to support. This means that you are inherently limited to a finite number of possible database platforms as you only have so much resource for testing.
The upshot is that you need to find a lowest common denominator for your application (which is quite a lot harder than it looks for SQL) or build a platform-dependent section where the non-portable functions can be plugged in on a per-platform basis.
I don't think that there exists one generic way that works for all Databases, since this is something very specific that depends on how the DB is built.
But, why do you want to do this using a specific query ?
Can't you abstract the implementation away from what you want to do ?
I mean: why not create a generic interface, which has among others, a method called 'TableExists( string tablename )' for instance.
Then, for each DBMS that you want to support , you create a class which implements this interface, and in the TableExists method, you write specific logic for this DBMS.
The SQLServer implementation will then contain a query which queries sysobjects.
In your application, you can have a factory class which creates the correct implementation for a given context, and then you just call the TableExists method.
For instance:
IMyInterface foo = MyFactory.CreateMyInterface (SupportedDbms.SqlServer);
if( foo.TableExists ("mytable") )
...
I think this is how I should do it.
I fully support Frederik Gheysels answer. If you have to support multiple database systems, you should implement your code against an abstract interface with specific implementations per database system. There are many more examples of incompatible syntax than just checking for an existing table (e.g.: limiting the query to a certain number of rows).
But if you really have to perform the check using the exception handling from your example, you should use the following query that is more efficient than a COUNT(*) because the database has no actual selection work to do:
SELECT 1 FROM my_table WHERE 1=2
I would avoid executing the select count(x) from xxxxxx as the DBMS will actually go ahead and do it which may take some time for a large table.
Instead just prepare a select * from mysterytable query. The prepare will fail if mysterytable does not exist. There is no need to actually execute the prepared statement.
The following works well for me...
private bool TableExists(SqlConnection conn, string database, string name)
{
string strCmd = null;
SqlCommand sqlCmd = null;
try
{
strCmd = "select case when exists((select '['+SCHEMA_NAME(schema_id)+'].['+name+']' As name FROM [" + database + "].sys.tables WHERE name = '" + name + "')) then 1 else 0 end";
sqlCmd = new SqlCommand(strCmd, conn);
return (int)sqlCmd.ExecuteScalar() == 1;
}
catch { return false; }
}
In current project on my job I need to write 'data agent' which would support a lot of database types.
So I decided to do next: write a base class with the base (database independent) functionality using virtual methods and override in subclasses all database-specific moments
Very Simple
use YOUR_DATABASE --OPTIONAL
SELECT count(*) as Exist from INFORMATION_SCHEMA.TABLES where table_name = 'YOUR_TABLE_NAME'
If the answer is 1, There is a table.
If the answer is 0, There is no table.
If you want to avoid try-catch solutions, I'm suggesting this method, using sys.tables
private bool IsTableExisting(string table)
{
string command = $"select * from sys.tables";
using (SqlConnection con = new SqlConnection(Constr))
using (SqlCommand com = new SqlCommand(command, con))
{
SqlDataReader reader = com.ExecuteReader();
while (reader.Read())
{
if (reader.GetString(0).ToLower() == table.ToLower())
return true;
}
reader.Close();
}
return false;
}
Related
I have a slight issue, I have a ASP.NET Webforms application. I'm sending over a url?id=X were X is my database index or id.
I have a C# class file to run my SQL connection and query. Here is the code:
public DataTable ViewProduct(string id)
{
try
{
string cmdStr = "SELECT * Products WHERE Idx_ProductId = " + id;
DBOps dbops = new DBOps();
DataTable vpTbl = dbops.RetrieveTable(cmdStr, ConfigurationManager.ConnectionStrings["MyDatabase"].ConnectionString);
return vpTbl;
}
catch (Exception e)
{
return null;
}
}
So as you can see my problem lies within string cmdStr = "SQL Query" + variable;
I'm passing over my index or id through the URL then requesting it and turning it into a string then using ViewProduct(productId).
I don't know what syntax or how to add the id into my C# string sql query. I've tried:
string cmdStr = "SELECT * Products WHERE Idx_ProductId = #0" + id;
string cmdStr = "SELECT * Products WHERE Idx_ProductId = {0}" + id;
also what I have currently to no avail.
I was so sure this would be a duplicate of some canonical question about parameterized queries in C#, but apparently there isn't one (see this)!
You should parameterize your query - if you don't, you run the risk of a malicious piece of code injecting itself into your query. For example, if your current code could run against the database, it would be trivial to make that code do something like this:
// string id = "1 OR 1=1"
"SELECT * Products WHERE Idx_ProductId = 1 OR 1=1" // will return all product rows
// string id = "NULL; SELECT * FROM UserPasswords" - return contents of another table
// string id = "NULL; DROP TABLE Products" - uh oh
// etc....
ADO.NET provides very simple functionality to parameterize your queries, and your DBOps class most assuredly is not using it (you're passing in a built up command string). Instead you should do something like this:
public DataTable ViewProduct(string id)
{
try
{
string connStr = ConfigurationManager.ConnectionStrings["MyDatabase"].ConnectionString;
using (SqlConnection conn = new SqlConnection(connStr))
{
conn.Open();
using (SqlCommand cmd = conn.CreateCommand())
{
// #id is very important here!
// this should really be refactored - SELECT * is a bad idea
// someone might add or remove a column you expect, or change the order of columns at some point
cmd.CommandText = "SELECT * Products WHERE Idx_ProductId = #id";
// this will properly escape/prevent malicious versions of id
// use the correct type - if it's int, SqlDbType.Int, etc.
cmd.Parameters.Add("#id", SqlDbType.Varchar).Value = id;
using (SqlDataReader reader = cmd.ExecuteReader())
{
DataTable vpTbl = new DataTable();
vpTbl.Load(reader);
return vpTbl;
}
}
}
}
catch (Exception e)
{
// do some meaningful logging, possibly "throw;" exception - don't just return null!
// callers won't know why null got returned - because there are no rows? because the connection couldn't be made to the database? because of something else?
}
}
Now, if someone tries to pass "NULL; SELECT * FROM SensitiveData", it will be properly parameterized. ADO.NET/Sql Server will convert this to:
DECLARE #id VARCHAR(100) = 'NULL; SELECT * FROM SensitiveData';
SELECT * FROM PRoducts WHERE Idx_ProductId = #id;
which will return no results (unless you have a Idx_ProductId that actually is that string) instead of returning the results of the second SELECT.
Some additional reading:
https://security.stackexchange.com/questions/25684/how-can-i-explain-sql-injection-without-technical-jargon
Difference between Parameters.Add and Parameters.AddWithValue
SQL injection on INSERT
Avoiding SQL injection without parameters
How do I create a parameterized SQL query? Why Should I? (VB.NET)
How can I prevent SQL injection in PHP? (PHP specific, but many helpful points)
Is there a canonical question telling people why they should use SQL parameters?
What type Products.Idx_ProductId is?
Probably it is string, than you need to use quotes: "... = '" + id.Trim() + "'";
I am working on a project where the client has reported an SQL injection flaw in the code. Here is my code…
1 public int ExecuteNonQuery(string query, SqlParameter[] parameters)
2 {
3 using (SqlCommand command = CreateCommand(query, parameters))
4 {
5 int rowsAffected = command.ExecuteNonQuery();
6 return rowsAffected;
7 }
8 }
And the CreateCommand method goes as
private SqlCommand CreateCommand(string commandText, SqlParameter[] parameters)
{
SqlCommand retVal = this.connection.CreateCommand();
retVal.CommandText = commandText;
retVal.CommandTimeout = this.commandsTimeout;
retVal.Parameters.AddRange(parameters);
return retVal;
}
The flaw is reported at line number 3. I am unable to understand what kind of attack an happen here as this is a console application. But I have to fix the flaw and I don't know how to fix it.
Query is
#"delete from {0} where runId in
( select runId from {0}
inner join
( select sId as sId_last,
wfId as wfId_last,
max(runId) as runId_last from {0} where endTime is NULL
group by sId, wfId ) t1
on endTime is NULL and sId = sId_last and wfId = wfId_last
and (runId <> runId_last or startTime < #aDateTime)
)";
Help appreciated.
Thanks.
that code is injection-free... But note that the methods that call ExecuteNonQuery could build the query by composing strings.
An injection attack happens when you do something like:
string name = ...; // A name selected by the user.
string query = "SELECT * FROM MyTable WHERE Name = '" + name + "'";
so when you compose a query using pieces of text that are of external origin.
Note that a more subtle injection attack could be multi-level:
string name = // The result of a query to the db that retrieves some data
// sadly this data has been manipulated by the attacker
string query = "SELECT * FROM MyTable WHERE Name = '" + name + "'";
In general you don't need a user interface to cause an injection attack...
You could query something from a web site/from the db, and use the unsanitized result to query the db (as in the last example), causing an injection attack... Or even using the content of the configuration file could cause an injection attack: the priviledges needed to modify the configuration file could be different than the ones needed to do something on the DB, and a malicious user could have the priviledges to modify the configuration file but not have direct access to the DB. So he could use the program as a trojan horse against the DB.
about the query
The weak point of that query (that is a composition of strings) is in how the {0} is calculated. Is it a string chosen in a group of fixed strings? Something like:
string tableName;
if (foo)
tableName = "Foo";
else if (bar)
tableName = "Bar";
or is it something more user controlled?
If the table names are fixed in code, then there shouldn't be any injection attack possible. If the table names are "extracted" from some user input/some other table the user could have access, we return to the problem I showed before.
You've exposed a public method which can be accessed by any code that allows any SQL expression to be executed.
I would look at changing that method to being internal or private instead so that not just any code can call that method.
Line 3:
using (SqlCommand command = CreateCommand(query, parameters))
Both Query and parameters are available in this line.
SQL injection should not be prevented by trying to validate your input; instead, that input should be properly escaped before being passed to the database.
How to escape input totally depends on what technology you are using to interface with the database.
Use prepared statements and parameterized queries. These are SQL
statements that are sent to and parsed by the database server
separately from any parameters. This way it is impossible for an
attacker to inject malicious SQL.
Lesson on SQL injection for your reference.link2
Recent bug report states that a method being called is crashing the service causing it to restart. After troubleshooting, the cause was found to be an obnoxious Oracle SQL call with thousands of strings passed. There is a collection of strings being passed to a method from an external service which often is more than 10,000 records. The original code used a where clause on the passed collection using the LIKE keyword, which I think is really, really bad.
public IList<ContainerState> GetContainerStates(IList<string> containerNumbers)
{
string sql =
String.Format(#"Select CTNR_NO, CNTR_STATE FROM CONTAINERS WHERE CTRN_SEQ = 0 AND ({0})",
string.Join("OR", containerNumbers
.Select(item => string.Concat(" cntr_no LIKE '", item.SliceLeft(10), "%' ")))
);
return DataBase.SelectQuery(sql, MapRecordToContainerState, new { }).ToList();
}
Clarification of in house methods used which may be confusing:
DataBase.SelectQuery is an internal library method using generics which gets passed the sql string, a function to map the records to .NET objects, and the parameters being passed and returns an IEnumerable of Objects of type retuned by the Mapping function.
SliceLeft is an extension method from another internal helper library that just returns the first part of a string up to the number of characters specified by the parameter.
The reason that the LIKE statement was apparently used, is that the strings being passed and the strings in the database only are guaranteed to match the first 10 characters. Example ("XXXX000000-1" in the strings being passed should match a database record like "XXXX000000-8").
I believed that the IN clause using the SUBSTR would be more efficent than using multiple LIKE clauses and replaced the code with:
public IList<ContainerRecord> GetContainerStates(IList<string> containerNumbers)
{
string sql =
String.Format(#"Select CTNR_NO, CNTR_STATE FROM CONTAINERS WHERE CTRN_SEQ = 0 AND ({0})",
string.Format("SUBSTR(CNTR_NO, 1, 10) IN ({0}) ",
string.Join(",", containerNumbers.Select(item => string.Format("\'{0}\'", item.SliceLeft(10) ) ) )
)
);
return DataBase.SelectQuery(sql, MapRecordToContainerState, new { }).ToList();
}
This helped slightly, and there were fewer issues in my tests, but when there are huge amounts of records passed, there is still an exception thrown and core dumps occur, as the SQL is longer than the server can parse during these times. The DBA suggests saving all the strings being passed to a temporary table, and then joining against that temp table.
Given that advice, I changed the function to:
public IList<ContainerRecord> GetContainerStates(IList<string> containerNumbers)
{
string sql =
#"
CREATE TABLE T1(cntr_num VARCHAR2(10));
DECLARE GLOBAL TEMPORARY TABLE SESSION.T1 NOT LOGGED;
INSERT INTO SESSION.T1 VALUES (:containerNumbers);
SELECT
DISTINCT cntr_no,
'_IT' cntr_state
FROM
tb_master
WHERE
cntr_seq = 0
AND cntr_state IN ({0})
AND adjustment <> :adjustment
AND SUBSTR(CTNR_NO, 1, 10) IN (SELECT CNTR_NUM FROM SESSION.T1);
";
var parameters = new
{
#containerNumbers = containerNumbers.Select( item => item.SliceLeft(10)).ToList()
};
return DataBase.SelectQuery(sql, MapRecordToContainerState, parameters).ToList();
}
Now I'm getting a "ORA-00900: invalid SQL statement". This is really frustrating, how can I properly write a SQL Statement that will put this list of strings into a temporary table and then use it in a SELECT Statement to return the list I need?
There are couple possible places could cause this error, it seams that the "DECLARE GLOBAL TEMPORARY" is a JAVA API, I don't think .net has this function. Please try "Create global temporary table" instead. And, I don't know whether your internal API could handle multiple SQLs in one select sql. As far as I know, ODP.net Command class can only execute one sql per call. Moreover, "create table" is a DDL, it therefore has its own transaction. I can't see any reason we should put them in the same sql to execute. Following is a sample code for ODP.net,
using (OracleConnection conn = new OracleConnection(BD_CONN_STRING))
{
conn.Open();
using (OracleCommand cmd = new OracleCommand("create global temporary table t1(id number(9))", conn))
{
// actually this should execute once only
cmd.ExecuteNonQuery();
}
using (OracleCommand cmd = new OracleCommand("insert into t1 values (1)", conn)) {
cmd.ExecuteNonQuery();
}
// customer table is a permenant table
using (OracleCommand cmd = new OracleCommand("select c.id from customer c, t1 tmp1 where c.id=tmp1.id", conn)) {
cmd.ExecuteNonQuery();
}
}
I have a list Called ListTypes that holds 10 types of products. Below the store procedure loops and gets every record with the product that is looping and it stores it in the list ListIds. This is killing my sql box since I have over 200 users executing this constantly all day.
I know is not a good architecture to loop a sql statement, but this the only way I made it work. Any ideas how I can make this without looping? Maybe a Linq statement, I never used Linq with this magnitude. Thank you.
protected void GetIds(string Type, string Sub)
{
LinkedIds.Clear();
using (SqlConnection cs = new SqlConnection(connstr))
{
for (int x = 0; x < ListTypes.Count; x++)
{
cs.Open();
SqlCommand select = new SqlCommand("spUI_LinkedIds", cs);
select.CommandType = System.Data.CommandType.StoredProcedure;
select.Parameters.AddWithValue("#Type", Type);
select.Parameters.AddWithValue("#Sub", Sub);
select.Parameters.AddWithValue("#TransId", ListTypes[x]);
SqlDataReader dr = select.ExecuteReader();
while (dr.Read())
{
ListIds.Add(Convert.ToInt32(dr["LinkedId"]));
}
cs.Close();
}
}
}
Not a full answer, but this wouldn't fit in a comment. You can at least update your existing code to be more efficient like this:
protected List<int> GetIds(string Type, string Sub, IEnumerable<int> types)
{
var result = new List<int>();
using (SqlConnection cs = new SqlConnection(connstr))
using (SqlCommand select = new SqlCommand("spUI_LinkedIds", cs))
{
select.CommandType = System.Data.CommandType.StoredProcedure;
//Don't use AddWithValue! Be explicit about your DB types
// I had to guess here. Replace with the actual types from your database
select.Parameters.Add("#Type", SqlDBType.VarChar, 10).Value = Type;
select.Parameters.Add("#Sub", SqlDbType.VarChar, 10).Value = Sub;
var TransID = select.Parameters.Add("#TransId", SqlDbType.Int);
cs.Open();
foreach(int type in types)
{
TransID.Value = type;
SqlDataReader dr = select.ExecuteReader();
while (dr.Read())
{
result.Add((int)dr["LinkedId"]);
}
}
}
return result;
}
Note that this way you only open and close the connection once. Normally in ADO.Net it's better to use a new connection and re-open it for each query. The exception is in a tight loop like this. Also, the only thing that changes inside the loop this way is the one parameter value. Finally, it's better to design methods that don't rely on other class state. This method no longer needs to know about the ListTypes and ListIds class variables, which makes it possible to (among other things) do better unit testing on the method.
Again, this isn't a full answer; it's just an incremental improvement. What you really need to do is write another stored procedure that accepts a table valued parameter, and build on the query from your existing stored procedure to JOIN with the table valued parameter, so that all of this will fit into a single SQL statement. But until you share your stored procedure code, this is about as much help as I can give you.
Besides the improvements others wrote.
You could insert your ID's into a temp table and then make one
SELECT * from WhatEverTable WHERE transid in (select transid from #tempTable)
On a MSSQL this works really fast.
When you're not using a MSSQL it could be possible that one great SQL-Select with joins is faster than a SELECT IN. You have to test these cases by your own on your DBMS.
According to your comment:
The idea is lets say I have a table and I have to get all records from the table that has this 10 types of products. How can I get all of this products? But this number is dynamic.
So... why use a stored procedure at all? Why not query the table?
//If [Type] and [Sub] arguments are external inputs - as in, they come from a user request or something - they should be sanitized. (remove or escape '\' and apostrophe signs)
//create connection
string queryTmpl = "SELECT LinkedId FROM [yourTable] WHERE [TYPE] = '{0}' AND [SUB] = '{1}' AND [TRANSID] IN ({2})";
string query = string.Format(queryTmpl, Type, Sub, string.Join(", ", ListTypes);
SqlCommand select = new SqlCommand(query, cs);
//and so forth
To use Linq-to-SQL you would need to map the table to a class. This would make the query simpler to perform.
I want to read data from a table whose name is supplied by a user. So before actually starting to read data, I want to check if the database exists or not.
I have seen several pieces of code on the NET which claim to do this. However, they all seem to be work only for SQL server, or for mysql, or some other implementation. Is there not a generic way to do this?
(I am already seperately checking if I can connect to the supplied database, so I'm fairly certain that a connection can be opened to the database.)
You cannot do this in a cross-database way. Generally DDL (that is, the code for creating tables, indexes and so on) is completely different from database to database and so the logic for checking whether tables exist is also different.
I would say the simplest answer, though, would simply be something like:
SELECT * FROM <table> WHERE 1 = 0
If that query gives an error, then the table doesn't exist. If it works (though it'll return 0 rows) then the table exists.
Be very careful with what you let the user input, though. What's to stop him from from specifying "sysusers" as the table name (in SQL Server, that'll be the list of all database users)
You can use the DbConnection.GetSchema family of methods to retreive metadata about the database. It will return a DataTable with schema objects. The exact object types and restriction values may vary from vendor to vendor, but I'm sure you can set up your check for a specific table in a way that will work in most databases.
Here's an example of using GetSchema that will print the name and owner of every table that is owned by "schema name" and called "table name". This is tested against oracle.
static void Main(string[] args)
{
string providerName = #"System.Data.OracleClient";
string connectionString = #"...";
DbProviderFactory factory = DbProviderFactories.GetFactory(providerName);
using (DbConnection connection = factory.CreateConnection())
{
connection.ConnectionString = connectionString;
connection.Open();
DataTable schemaDataTable = connection.GetSchema("Tables", new string[] { "schema name", "table name" });
foreach (DataColumn column in schemaDataTable.Columns)
{
Console.Write(column.ColumnName + "\t");
}
Console.WriteLine();
foreach (DataRow row in schemaDataTable.Rows)
{
foreach (object value in row.ItemArray)
{
Console.Write(value.ToString() + "\t");
}
Console.WriteLine();
}
}
}
That's like asking "is there a generic way to get related data" in databases. The answer is of course no - the only "generic way" is to have a data layer that hides the implementation details of your particular data source and queries it appropriately.
If you are really supporting and accessing many different types of databases without a Stategy design pattern or similar approach I would be quite surprised.
That being said, the best approach is something like this bit of code:
bool exists;
try
{
// ANSI SQL way. Works in PostgreSQL, MSSQL, MySQL.
var cmd = new OdbcCommand(
"select case when exists((select * from information_schema.tables where table_name = '" + tableName + "')) then 1 else 0 end");
exists = (int)cmd.ExecuteScalar() == 1;
}
catch
{
try
{
// Other RDBMS. Graceful degradation
exists = true;
var cmdOthers = new OdbcCommand("select 1 from " + tableName + " where 1 = 0");
cmdOthers.ExecuteNonQuery();
}
catch
{
exists = false;
}
}
Source: Check if a SQL table exists
You can do something like this:
string strCheck = "SHOW TABLES LIKE \'tableName\'";
cmd = new MySqlCommand(strCheck, connection);
if (connection.State == ConnectionState.Closed)
{
connection.Open();
}
cmd.Prepare();
var reader = cmd.ExecuteReader();
if (reader.HasRows)
{
Console.WriteLine("Table Exist!");
}
else (reader.HasRows)
{
Console.WriteLine("Table Exist!");
}