I'm having trouble doing case insensitive string comparison using code first against an Oracle db. Code looks something like this;
String filter = "Ali";
var employee = dbContext.Employees.Where(x => x.Name.Contains(filter)).FirstOrDefault();
The code above acts to be case sensitive. So I converted both the Name and filter to Uppercase;
String filter = "Ali";
filter = filter.ToUpper();
var employee = dbContext.Employees.Where(x => x.Name.ToUpper().Contains(filter)).FirstOrDefault();
Everything seemed to work at first, but then I realized it's not working when the employee's name or the filter contains the character 'i'. The problem is how the letter i works in Turkish.
In most languages, 'i' stands for the lowercase, and 'I' stands for the uppercase version of the character. However in Turkish, 'i's uppercase is 'İ', and 'I's lowercase is 'ı'. Which is a problem as Oracle uppercases the letter 'i' in the db as 'I'.
We do not have access to the db's character encoding settings as its effects cannot be foreseen easily.
What I've come up with is this, and it is very ugly.
String filterInvariant = filter.ToUpper(CultureInfo.InvariantCulture);
String filterTurkish = filter.ToUpper(CultureInfo.CreateSpecificCulture("tr-TR"));
var employee = dbContext.Employees.Where(x => x.Name.ToUpper().Contains(filterInvariant) || x.Name.ToUpper().Contains(filterTurkish)).FirstOrDefault();
It seems to fix some of the issues, but feels like a brute force workaround rather than a solid solution. What are the best practices, or alternatives to this workaround, while using Code First C# against an Oracle database?
Thanks in advance
Ditch using all the UPPER functions. Simply let Oracle do your language aware case-insensitive matching. This is done by setting your DB connection from C# to have the appropriate language parameters. This setting is just for your DB session, not a global change for the whole DB. I'm no C# wizard, so you'd have to figure out where to make these session settings in your db connection/pool code.
ALTER SESSION SET nls_language=TURKISH;
ALTER SESSION SET nls_comp=LINGUISTIC;
ALTER SESSION SET nls_sort=BINARY_CI;
If C# proves too difficult to find where to change this, you could set this up as a user/schema logon trigger (below), which sets these automatically for you at db connect time (replace SOMEUSER with your actual db username). This only affects any NEW db sessions, so if you have connections pooled, you'll want to cycle the DB connection pool to refresh the connections.
CREATE OR REPLACE TRIGGER SOMEUSER.SET_NLS_CASE_INSENSITIVE_TRG AFTER
LOGON ON SOMEUSER.SCHEMA
BEGIN
EXECUTE IMMEDIATE 'ALTER SESSION SET nls_language=TURKISH';
EXECUTE IMMEDIATE 'ALTER SESSION SET nls_comp=LINGUISTIC';
EXECUTE IMMEDIATE 'ALTER SESSION SET nls_sort=BINARY_CI';
END;
/
Here's a little test I did in an Oracle DB:
CREATE TABLE mypeople (name VARCHAR2(10 CHAR));
INSERT INTO mypeople VALUES ('Alİ Hassan');
INSERT INTO mypeople VALUES ('AlI Hassan');
INSERT INTO mypeople VALUES ('Ali Hassan');
INSERT INTO mypeople VALUES ('Alı Hassan');
SELECT name FROM mypeople WHERE name LIKE 'Ali%';
NAME
----------
Ali Hassan
ALTER SESSION SET nls_language=TURKISH;
ALTER SESSION SET nls_comp=LINGUISTIC;
ALTER SESSION SET nls_sort=BINARY_CI;
SELECT name FROM mypeople WHERE name LIKE 'Ali%';
NAME
----------
Alİ Hassan
AlI Hassan
Ali Hassan
The implementation of String.Contains is different for different providers, for example Linq2Sql is always case insensitive. The search is case sensitive or not depends on server settings. For example SQL Server by default has SQL_Latin1_General_CP1_CI_AS Collation and that is NOT case sensitive. For Oracle you can change this behavior at the session level: Case insensitive searching in Oracle (Issue a raw SQL query using context.Database.ExecuteSqlCommand method at the beginning of the session)
The problem is in the database, not in .NET, for example this query:
FILES.Where(t => t.FILE_NAME.ToUpper() == "FILE.TXT") // Get rows from file-table
translates into this Oracle SQL with the oracle provider I have:
SELECT t0.BINARY_FILE, t0.FILE_NAME, t0.FILE_SIZE, t0.INFO, t0.UPLOAD_DATE
FROM FILES t0
WHERE (UPPER(t0.FILE_NAME) = :p0)
-- p0 = [FILE.TXT]
The contains with First() becomes this:
SELECT * FROM (SELECT t0.BINARY_FILE, t0.FILE_NAME, t0.FILE_SIZE, t0.INFO, t0.UPLOAD_DATE
FROM FILES t0
WHERE ((UPPER(t0.FILE_NAME) LIKE '%' || :p0 || '%')
OR (UPPER(t0.FILE_NAME) LIKE '%' || :p1 || '%')))
WHERE ROWNUM<=1
-- p0 = [FILE.TXT]
-- p1 = [FİLE.TXT]
So it depends on your database's culture settings, ie without knowing them I would say the "overlap" with your solution is the best way to solve it. Why can't you just check the database culture settings?
Related
Let's say a have a stored procedure SetCustomerName which has an input parameter Name, and I have a table customers with column Name.
So inside my stored procedure I want to set customer's name. If I write
UPDATE customers SET Name = Name;
this is incorrect and I see 2 other ways:
UPDATE customers SET Name = `Name`;
UPDATE customers SET customers.Name = Name;
First one works, but I didn't find in documentation that I can wrap parameters inside ` characters. Or did I miss it in the documentation (link is appreciated in this case).
What other ways are there and what is the standard way for such a case? Renaming input parameter is not good for me (because I have automatic object-relational mapping if you know what I mean).
UPDATE:
So, there is a link about backticks (http://dev.mysql.com/doc/refman/5.0/en/identifiers.html) but it's not explained deep enough how to use them (how to use them with parameters and column names).
And there is a very strange thing (at least for me): You can use backticks either way:
UPDATE customers SET Name = `Name`;
//or
UPDATE customers SET `Name` = Name;
//or even
UPDATE customers SET `Name` = `Name`;
and they all work absolutely the same way.
Don't you think this is strange? Is this strange behavior explained somewhere?
Simplest way to distinguished between your parameter and column (if both name is same) is to add table name in your column name.
UPDATE customers SET customers.Name = Name;
Even you can also add database prefix like
UPDATE yourdb.customers SET yourdb.customers.Name = Name;
By adding database name you can perform action on more than 1 database from single store procedure.
I think that your first example is actually backwards. If you're trying to set the "Name" column to the "Name" input parameter, I believe it should be:
UPDATE customers SET `Name` = Name;
And for the second example, you can set table aliases the same way that you do in all other statements:
UPDATE customers AS c SET c.Name = Name;
Not necessarily correct, but a fair way to better argument/parameter management, as well readability with easier understanding, especially while working with the SQL;
DROP PROCEDURE IF EXISTS spTerminalDataDailyStatistics; DELIMITER $$
CREATE PROCEDURE spTerminalDataDailyStatistics(
IN TimeFrom DATETIME,
IN DayCount INT(10),
IN CustomerID BIGINT(20)
) COMMENT 'Daily Terminal data statistics in a date range' BEGIN
# Validate argument
SET #TimeFrom := IF(TimeFrom IS NULL, DATE_FORMAT(NOW(), '%Y-%m-01 00:00:00'), TimeFrom);
SET #DayCount := IF(DayCount IS NULL, 5, DayCount);
SET #CustomerID := CustomerID;
# Determine parameter
SET #TimeTo = DATE_ADD(DATE_ADD(#TimeFrom, INTERVAL #DayCount DAY), INTERVAL -1 SECOND);
# Do the job
SELECT DATE_FORMAT(TD.TerminalDataTime, '%Y-%m-%d') AS DataPeriod,
COUNT(0) AS DataCount,
MIN(TD.TerminalDataTime) AS Earliest,
MAX(TD.TerminalDataTime) AS Latest
FROM pnl_terminaldata AS TD
WHERE TD.TerminalDataTime BETWEEN #TimeFrom AND #TimeTo
AND (#CustomerID IS NULL OR TD.CustomerID = #CustomerID)
GROUP BY DataPeriod
ORDER BY DataPeriod ASC;
END $$
DELIMITER ;
CALL spTerminalDataDailyStatistics('2021-12-01', 2, 1801);
Using backticks in MySQL query syntax is documented here:
http://dev.mysql.com/doc/refman/5.0/en/identifiers.html
So yes, your first example (using backticks) is correct.
Here is the link you are asking for:
http://dev.mysql.com/doc/refman/5.0/en/identifiers.html
The backticks are called "identifier quote" in MySql
I'm using the NPoco ORM (a branch of PetaPoco) but I've just noticed it's mapping the columns incorrectly in some cases.
I'm using a stored procedure and my POCO property names are identical to the column names produced by the stored procedure:
string sql = "EXEC API_GetVenueSummaryByID ##venueID = #venueID";
var venue = db.FirstOrDefault<VenueSummary>(sql, new { venueID = venueID });
The stored procedure is a simple SELECT statement with a couple of variables included (removing them doesn't help):
DECLARE #hasOffers bit
IF EXISTS(SELECT * FROM Offers WHERE dbo.Offers.EntryType='V' AND Offers.EntryID = #VenueID AND GETDATE() <= validToDate) SET #hasOffers = 1
SELECT
Venue.VenueID, VenueName, Town, County, Country, PackageCode,
MeetingRoomsNo, MaxMeetingCapacity, BedroomsNo,
MetaDescription AS ShortDescription,
'dummyresult.jpg' AS PrimaryImageFilename,
#hasOffers AS HasSpecialOffers,
CAST(TimeStamp AS BIGINT) AS RecordVersion
FROM dbo.Venue
WHERE Venue.VenueID = #VenueID
Is there a function in NPoco which causes it to guess the mappings (ignoring their names)? If so, how I can I disable this and force it to only match based on the column and property names?
Currently the only work around seems to be to use the column attribute <-- doesn't work either
At the moment, even someone auto-formatting stored procedure (or any change which results in a change of column order) is breaking the application.
Edit 2
I've noticed that if I restart the website application (eg by editing web.config or updating application code) then the column order fixes itself. So I can only assume the problem is related to NPoco internally caching the column indexes - and if the indexes change, the mappings will then be incorrect. I'm not sure if there's a mechanism to clear the cache that's perhaps not working?
This is a problem with how NPoco (and PetaPoco) caches the codegen that is used to map from a SQL statement to a POCO.
Usually this isn't a problem if you are changing the code when you are changing the SQL as the cache will be rebuild, however if you create your POCO first then start to change the SP after the first initial run the mappings will be incorrect.
This issues has now been fixed in 2.5.83-beta, and it will now look at the column names and their positions to determine the cache key.
Thanks for the help #NickG
I have multiple databases. Can I hit one of the databases based on an identifier which is dynamic?
e.g. I have three databases DB1,DB2,DB3.
I have a query select * from tblEmployees.(This table is present in all the three DBs). I have an identifier(or some variable ) whose value can be 1 or 2 or 3 and based on the value of this variable which I get dynamically when my service is hit, I would like to choose the DB from which the values should be obtained.
Can this be done? My DB is SQL Server and front end is asp.net.
My connection strings are stored in web.config file. Can I have multiple connection string which will have the same server with diff db names and select one of them based on the identifier.
1.In real world most of the time you have to store your connection strings in your web.config file .
so there you can let's keep three connection strings which will have the same server but different databases name, then you can select one of the connection-string for your app connection to the required database.
2.you can build that connection string on run-time if you need.
using these technique you will never have to write 2 or more queries just change the query string and your queries will work for all the databases.
You can do like this
if(val == 1)
{
select * from [DB1].[dbo].[tblEmployees]
}
else if(val == 2)
{
select * from [DB2].[dbo].[tblEmployees]
}
Try this one -
DECLARE #ID INT
SELECT #ID = 2
DECLARE #SQL NVARCHAR(500)
SELECT #SQL = 'SELECT * FROM DB' + CAST(#ID AS CHAR(1)) + '.dbo.tblEmployees'
PRINT #SQL
EXEC sys.sp_executesql #SQL
Output -
SELECT * FROM DB2.dbo.tblEmployees
T-SQL way:
declare #db int
if #db = 1
begin
use [db1]
select *
from tblEmployees
end
if #db = 2
begin
use [db2]
select *
from tblEmployees
end
-- and so on
IMO you're best bet would be to use a different connection-string to achieve multi-tenency against similar alike databases. Ideally, abstracted away behind some code so that most of your code doesn't need to know about it, but just does:
using(var conn = Somewhere.GetOpenConnection()) {
// ...
}
or worst-case:
using(var conn = Somewhere.GetOpenConnection(Environment.Published)) {
// ...
}
(here Environment is an enum to what the various databases represent)
where GetOpenConnection figures out which database is needed, and either looks up on constructs the correct connection string.
But to be specific:
you cannot parameterize the DB name in a query
using use between operations would be a really bad idea in terms of connection re-use
you can explicitly use three-part identifiers (i.e. DB1..SomeTable or DB1.dbo.SomeTable), but that does not scale naturally to lots of databases
Does anyone know how to store a single backslash into PostgreSQL Database?
I am using C# and Nqgsql to access PostgreSQL, and my case is I want to store "1\\0\\0\\0\\1\\0" into the database, and the expected string in DB field will be "1\0\0\0\1\0", that is I only need one backslash in the db field, thus when I get the data from db, it will still be "1\\0\\0\\0\\1\\0" in memory. But my problem is when the memory string is "1\\0\\0\\0\\1\\0", the string stored into db field is also "1\\0\\0\\0\\1\\0", then when I get the data from db, the memory string will be "1\\\\0\\\\0\\\\0\\\\1\\\\0".
The variables I used in c# code is set as the following format:
var a = "1\\0\\0\\0\\1\\0";
var b = #"1\0\0\0\1\0";
when store into db, it seems that the backslashes in both variables have been doubled. How to deal with this issue?
You should avoid this entirely by using parametrized queries. Consult an example in Npgsql: User's Manual in Using parameters in a query section.
But if you really want to construct a literal query then you can use E'' syntax, like this:
var sql = #"insert into table_name (column_name) values (E'1\\0\\0\\0\\1\\0')";
This syntax is independent of server or connection configuration like standard_conforming_strings. But it is Postgres specific.
If you want your code be portable between different database engines the you can issue set standard_conforming_strings=on just after connecting. Then this works:
var sql = #"insert into table_name (column_name) values ('1\0\0\0\1\0')";
This option is turned on by default since PostgreSQL 9.1 and available since 8.2.
I had this problem as well. I was able to solve it by going in the database's config and changing "standard_conforming_strings" to OFF.
public static bool TruncateTable(string dbAlias, string tableName)
{
string sqlStatement = string.Format("TRUNCATE TABLE {0}", tableName);
return ExecuteNonQuery(dbAlias, sqlStatement) > 0;
}
The most common recommendation to fight SQL injection is to use an SQL query parameter (several people on this thread have suggested it).
This is the wrong answer in this case. You can't use an SQL query parameter for a table name in a DDL statement.
SQL query parameters can be used only in place of a literal value in an SQL expression. This is standard in every implementation of SQL.
My recommendation for protecting against SQL injection when you have a table name is to validate the input string against a list of known table names.
You can get a list of valid table names from the INFORMATION_SCHEMA:
SELECT table_name
FROM INFORMATION_SCHEMA.Tables
WHERE table_type = 'BASE TABLE'
AND table_name = #tableName
Now you can pass your input variable to this query as an SQL parameter. If the query returns no rows, you know that the input is not valid to use as a table. If the query returns a row, it matched, so you have more assurance you can use it safely.
You could also validate the table name against a list of specific tables you define as okay for your app to truncate, as #John Buchanan suggests.
Even after validating that tableName exists as a table name in your RDBMS, I would also suggest delimiting the table name, just in case you use table names with spaces or special characters. In Microsoft SQL Server, the default identifier delimiters are square brackets:
string sqlStatement = string.Format("TRUNCATE TABLE [{0}]", tableName);
Now you're only at risk for SQL injection if tableName matches a real table, and you actually use square brackets in the names of your tables!
As far as I know, you can't use parameterized queries to perform DDL statements/ specify table names, at least not in Oracle or Sql Server. What I would do, if I had to have a crazy TruncateTable function, that had to be safe from sql injection would be to make a stored procedure that checks that the input is a table that is safe to truncate.
-- Sql Server specific!
CREATE TABLE TruncableTables (TableName varchar(50))
Insert into TruncableTables values ('MyTable')
go
CREATE PROCEDURE MyTrunc #tableName varchar(50)
AS
BEGIN
declare #IsValidTable int
declare #SqlString nvarchar(50)
select #IsValidTable = Count(*) from TruncableTables where TableName = #tableName
if #IsValidTable > 0
begin
select #SqlString = 'truncate table ' + #tableName
EXECUTE sp_executesql #SqlString
end
END
If you're allowing user-defined input to creep into this function via the tablename variable, I don't think SQL Injection is your only problem.
A better option would be to run this command via its own secure connection and give it no SELECT rights at all. All TRUNCATE needs to run is the ALTER TABLE permission. If you're on SQL 2005 upwards, you could also try using a stored procedure with EXECUTE AS inside.
CREATE OR REPLACE PROCEDURE truncate(ptbl_name IN VARCHAR2) IS
stmt VARCHAR2(100);
BEGIN
stmt := 'TRUNCATE TABLE '||DBMS_ASSERT.SIMPLE_SQL_NAME(ptbl_name);
dbms_output.put_line('<'||stmt||'>');
EXECUTE IMMEDIATE stmt;
END;
Use a stored procedure. Any decent db library (MS Enterprise Library is what I use) will handle escaping string parameters correctly.
Also, re:parameterized queries: I prefer to NOT have to redeploy my app to fix a db issue. Storing queries as literal strings in your source increases maintenance complexity.
Have a look at this link
Does this code prevent SQL injection?
Remove the unwanted from the tableName string.
I do not think you can use param query for a table name.
There are some other posts which will help with the SQL injection, so I'll upvote those, but another thing to consider is how you will be handling permissions for this. If you're granting users db+owner or db_ddladmin roles so that they can truncate tables then simply avoiding standard SQL injection attacks isn't sufficient. A hacker can send in other table names which might be valid, but which you wouldn't want truncated.
If you're giving ALTER TABLE permissions to the users on the specific tables that you will allow to be truncated then you're in a bit better shape, but it's still more than I like to allow in a normal environment.
Usually TRUNCATE TABLE isn't used in normal day-to-day application use. It's used for ETL scenarios or during database maintenance. The only situation where I might imagine it would be used in a front-facing application would be if you allowed users to load a table which is specific for that user for loading purposes, but even then I would probably use a different solution.
Of course, without knowing the specifics around why you're using it, I can't categorically say that you should redesign, but if I got a request for this as a DBA I'd be asking the developer a lot of questions.
Use parameterized queries.
In this concrete example you need protection from SQL injection only if table name comes from external source.
Why would you ever allow this to happen?
If you are allowing some external entity (end user, other system, what?)
to name a table to be dropped, why won't you just give them admin rights.
If you are creating and removing tables to provide some functionality for end user,
don't let them provide names for database objects directly.
Apart from SQL injection, you'll have problems with name clashes etc.
Instead generate real table names yourself (e.g DYNTABLE_00001, DYNTABLE_00002, ...) and keep a table that connects them to the names provided by user.
Some notes on generating dynamic SQL for DDL operations:
In most RDBMS-s you'll have to use dynamic SQL and insert table names as text.
Be extra careful.
Use quoted identifiers ([] in MS SQL Server, "" in all ANSI compliant RDBMS).
This will make avoiding errors caused by invalid names easier.
Do it in stored procedures and check if all referenced objects are valid.
Do not do anything irreversible. E.g. don't drop tables automatically.
You can flag them to be dropped and e-mail your DBA.
She'll drop them after the backup.
Avoid it if you can. If you can't, do what you can to minimize rights to other
(non-dynamic) tables that normal users will have.
You could use SQLParameter to pass in tableName value. As far as I know and tested, SQLParameter takes care of all parameter checking and thus disables possibility of injection.
If you can't use parameterized queries (and you should) ... a simple replace of all instances of ' with '' should work.
string sqlStatement = string.Format("TRUNCATE TABLE {0}", tableName.Replace("'", "''"));