I'm working with an asp.net website that uses a lot of inline SQL queries... and I'm wondering if it is best to create the inline queries on the fly:
int i = 500;
using (SqlConnection conn = new SqlConnection(connStr))
{
SqlCommand com = new SqlCommand(conn);
...
com.CommandText = "select from table where column < #parameter";
...
}
Or to have a class to hold all queries needed for the application. Something like this:
class SqlQueries
{
private string query1 =
"select * from tblEmployees where EmployeeName = #EmployeeName";
private string query2 =
"select * from tblVacation where EmployeeName = #EmployeeName";
public string Query(string s)
{
string str = string.Empty;
switch (s)
{
case "query1":
str = query1;
break;
case "query2":
str = query2;
break;
}
return str;
}
}
Thank you!
I've used a lot of ADO.NET queries in my day and I have always used the first method. The second method is an interesting idea, but it might be cumbersome to edit those queries if you are at another place in the code that uses it. It also makes it harder to see what a query is doing at a particular place in code. Example:
string sql = "Update User set age = #age where UserId = #UserId";
tells a developer what is happening, while:
string sql = SqlQueries.Query("updateAge");
Leaves questions about what table/columns are being updated. Also, with the first one, you know exactly what params need to be added.
If you are writing this query in several places that might change things
It's not terrible to put the literal directly in the method, as long as you always call that same method every time you want to run that query. However, if you are going to copy that string literal into multiple places in your code, then a constant is definitely preferred. However, rather than taking a string as the argument for the Query method in your second example, it should take an enumeration value.
However, if you are using the second method you described, I would ask you why you don't just start using stored procedures instead?
I would recommend using stored procedures as a much better solution to your problem than hard coded in-line queries. If you have to change the query at a later date, you don't have to rebuild your application, so bugs in your query can be fixed without needing to deploy the whole application. The 2nd option you have there is a maintenance nightmare waiting to happen. It all looks so nice when you have one or two queries in it, but that begins to look a bit more ugly when you have tens or hundreds in there. Your code looks like it's c#, so I would recommend checking out the Microsoft Enterprise Library,
http://msdn.microsoft.com/en-us/library/ff632023.aspx
You might need to download a different version depending on what version of the .NET framework you are developing with.
If you absolutely have to have "inline" sql as opposed to stored procedures (and I have done this for utility type applications that merely interact with a database, rather than own it), I would suggest putting your SQL into an embedded resource file. This will make your queries easier to maintain (although you will still need to re-compile your app to make changes).
I think it's OK to have the queries "inline" as long as they aren't repeated in several places. If that starts to happen, then you might want to start creating Query classes.
In both the cases you are ultimately building/fetching String which you will pass to CommandText. So there would be no such difference. Only thing you need to consider in your case is how you would maintain the code or how will other people understand your code.
If you're going to use inline SQL at least don't put it in the web page code because it will be painful when you make database changes to know what it affects. Putting all the queries in one classes might be a bit disorganized, but if you grouped them by functional classes (like manager classes for your business objects) it might be easier to deal with.
If your queries are longer than a line or two, you should consider putting them in their own .sql file. Set the build action on the file to embedded resource, and access it with a call to GetManifestResourceStream(). That way, you're elevating your sql to the status of a proper language, with syntax highlighting, validation and intellisense (when you connect VS to your DB). Needless to say, this hugely facilitates maintenance.
If all this seems like a hassle, grab my VS extension, QueryFirst. Create your .sql files with the provided template and they will automatically be wired up for compilation. But you won't care because you'll just access the queries via the generated classes.
Sql is the only computer language I can think of that we accept to see chopped up in string literals. It ought to be a scandal.
Related
I'm a bit newbie still and I have been assigned with the task of maintaining previosuly done code.
I have a web that simulates SQL Management Studio, limitating deleting options for example, so basic users don't screw our servers.
Well, we have a function that expects a query or queries, it works fine, but our server RAM gets blown up with complex queries, maybe it's not that much data, but its casting xml and all that stuff that I still don't even understand in SQL.
This is the actual function:
public DataSet ExecuteMultipleQueries(string queries)
{
var results = new DataSet();
using (var myConnection = new SqlConnection(_connectionString))
{
myConnection.Open();
var sqlCommand = myConnection.CreateCommand();
sqlCommand.Transaction = myConnection.BeginTransaction(IsolationLevel.ReadUncommitted);
sqlCommand.CommandTimeout = AppSettings.SqlTimeout;
sqlCommand.CommandText = queries.Trim();
var dataAdapter = new SqlDataAdapter { SelectCommand = sqlCommand };
dataAdapter.Fill(results);
return results;
}
}
I'm a bit lost, I've read many different answers but either I don't understand them properly or they don't solve my problems in any way.
I know I could use Linq-toSql- or Entity, I tried them but I really don't know how to use them with an "unknown" query, I could try to research more anyway so if you think they will help me approaching a solution, by any means, I will try to learn it.
So to the point:
The function seems to stop at dataAdapter.Fill(results) when debugging, at that point is where the server tries to answer the query and just consume all its RAM and blocks itself. How can I solve this? I thought maybe by making SQL return a certain amount of data, store it in a certain collection, then continue returning data, and keep going until there is no more data to return from SQL, but I really don't know how to detect if there is any data left to return from SQL.
Also I thought about reading and storing in two different threads, but I don't know how the data that is in one thread can be stored in other thread async (and even less if it solves the issue).
So, yes, I don't have anything clear at all, so any guidance or tip would be highly appreciated.
Thanks in advance and sorry for the long post.
You can use pagination to fetch only part of the data.
Your code will be like this:
dataAdapter.Fill(results, 0, pageSize);
pageSize can be at size you want (100 or 250 for example).
You can get more information in this msdn article.
In order to investigate, try the following:
Start SQL profiler (it is usually installed along with SSMS and can be started from Management Studio, Tools menu)
Make sure you fill up some filters (either NT username or at least the database you are profiling). This is to catch as specific (i.e. only your) queries as possible
Include starting events to see when your query starts (e.g. RPC:Starting).
Start your application
Start the profiler before issuing the query (fill the adapter)
Issue the query -> you should see the query start in the profiler
Stop the profiler not to catch other queries (it puts overhead on SQL Server)
Stop the application (no reason to mess with server until the analysis is done)
Take the query within SQL Management Studio. I expect a SELECT that returns a lot of data. Do not run as it is, but put a TOP to limit its results. E.g. SELECT TOP 1000 <some columns> from ....
If the TOPed select runs slowly, you are returning too much data.
This may be due to returning some large fields such as N/VARCHAR(MAX) or VARBINARY(MAX). One possible solution is to exclude these fields from the initial SELECT and lazy-load this data (as needed).
Check these steps and come back with your actual query, if needed.
Dilemma: I have a need to build an API into another application. In this API, we must allow the developers to run SQL statements on the SQL server... Unfortunately, we are very restricted (yes, the world is upside down here) as to what we can do to minimize SQL injections... We can't create SPs on the SQL server and we can't restrict what the user can enter for his/her query either. Now while I was told that because we are building an API for "other developers", we shouldn't have to worry about SQL injection, I don't tend to agree and I'd really like to prevent that if I can...
So I was thinking that what I could do, and this is where my question comes in, is parse the query to either:
Check for SQL Injection patterns and return an error if found; or
Remove any "assignment" sections, and replace then with parameters dynamically
Are these, given my situation, the only 2 options I have? And if so, how would you implement number 2 above so that this example statement:
SELECT * FROM Table WHERE Field1='test' AND Field2=1
Becomes:
SELECT * FROM Table WHERE Field1=#Field1 AND Field2=#Field2
Where the parameters have been extracted dynamically in C#? And if so, how would I be able to extract the data type for the params? Is that possible?
You can't solve it at the application side. You can restrict as much as you can, and parse all you want, but the SQL injection attacks are contiguously evolving and new vectors are being created that will bypass your parsing.
For running ad-hoc queries I strongly recommend relying on permissions, not on SQL parsing. Users should be able to inject themselves all they want, but the permissions should prevent any damage. You won't be able to prevent (intentional or accidental) DOS from running a bad query that brings the server to its knees, but for that there is resource governance and audit.
But I can't stress this enough: you won't parse yourself out of the problem. Multi-byte character exploits are +10 years old now, and I'm pretty sure I don't know the majority of filter/parse by-pass techniques out there.
If your assignment is just writing a wrapper around a database so that other developers can send in their own SQL and get results then SQL injections are the "normal use case". There is just no way of knowing if a request is malicious or not. If you are allowed to run "good" code, you'll always be able to run "evil" code.
you can extract parameters from sql syntax
SqlConnection sqlCon = new SqlConnection("...");
String sqlScript = "Somethings ...";
Regex r = new Regex(#"(?<Parameter>#\w*)", RegexOptions.Compiled);
string[] parameters = r.Matches(sqlScript).Cast<Match>().Select<Match, string>(x => x.Value.ToLower()).Distinct<string>().ToArray<string>();
SqlCommand sqlCom = new SqlCommand(sqlScript, sqlCon);
foreach (string sqlParam in parameters)
{
sqlCom.Parameters.AddWithValue(sqlParam, "PARAMETER VALUE");
}
Following this post, SimpleMemberShipProvider, Using multiple PropertyValues with custom Database tables, I successfully create a new user using the SimpleMembershipProvider and WebMatrix.WebData.WebSecurity.
WebSecurity.CreateUserAndAccount(model.UserName, model.Password,new { FirstName = model.FirstName,LastName = model.LastName,AppId = Guid.NewGuid(), MemberId = model.MemberId});
However, I have not been able to find any built-in methods for retrieving this information back from MembershipProvider or WebSecurity methods. I know I could query the database directly to get the information back and update, but then my code would become tightly coupled with this test implementation.
Anyone know how to get extended properties from a currently logged in user in the SimpleMembershipProvider? Thanks.
SBirthare is right, however, since he doesn't explain how to do this, allow me (this is assuming you use the C# server-side language option, which you tagged, so I believe you do. This also assumes that the table name in which the "FirstName" and "LastName" columns belong is, in fact, "UserProfile"):
Since WebMatrix comes with SQL Server-CE, we can just use a simple SQL query for this.
First set up your connection to the database:
var db = Database.Open("Users");
Next, compile the appropriate string to be used for the query (a simple string variable is used):
string selectQueryString = "SELECT FirstName, LastName FROM UserProfile";
The above string when passed to the Query() C# method (shown below), will however return a list (specifically IEnumerable<dynamic>) of all "FirstName"s and "LastName"s returned from the database. If you only wanted certain rows returned you can use the WHERE clause in the SQL string, so instead of the above string you could make this instead (this is just an example, you will have to tailor your own conditions to fit your situation, but this should give you an easy example):
string selectQueryString = "SELECT FirstName, LastName FROM UserProfile WHERE Email = #0 AND UserID = #1";
First let me address what the #0 and the #1 is, in case you don't already know. They are placeholders that will end up being filled with the conditions you want to test against (more on this further down) using this method, known as "paramaterized queries", this will be your smoking gun against SQL injection attacks. In short: ALWAYS USE THIS METHOD FOR CONDITIONS TESTED IN THE WHERE CLAUSE (I really can't stress this enough).
Now, I'm not sure how much you know about SQL querying, but they can be very powerful and useful and by using things like LIKE, ORDER BY, GROUP BY, JOIN, UNION, AND, OR, or even subqueries, as well as, many many other keywords and approaches, you can really make the SQL query return just about anything you desire, in the order you desire (a great beginner SQL query tutorial can be found here: http://www.sqlcourse.com/index.html).
Okay, moving on... Once you've written the selectQueryString the way that you want to, all you need left is to store it, like so:
int ID = 6;
string email = "testEmailName#gmail.com";
var queryResultsList = db.Query(selectQueryString, email, ID); //Here the first argument passed to the `db.Query()` method is the query string, but each additional argument passed (and you can actually pass an array of values here instead if you want) is used to fill the `#0` and `#1` placeholders you formed in your string earlier (order is ever important here!).
And display it, like so:
foreach (var row in queryResultsList)
{
//Do something for each row returned, here!
<div>FIRST NAME: #row.FirstName</div><br/>
<div>LAST NAME: #row.LastName</div>
{
Sorry if this is a lot to take in, but I thought I would at least show you all of the steps that you need to actually make this happen. Really there isn't that much to it, I just felt like giving you a bunch of helpful pointers a long the way :)
Anyway, just let me know if you have any questions or if I need to clarify anything.
EDIT:
Well, I just saw that you use MVC as opposed to Web-Pages. My answer is from the Web-Pages environment, but to be honest, I'm not sure if it would be any different or not because I have never worked with MVC. Either way, the SQL query itself should be the same and since we're both using C#, idk.
Either way, if I am told that this answer doesn't fit the scenario or is not useful to you, I will be glad to delete it.
As far as I know you will have to query the table using your current ORM framework.
SimpleMembershipProvider and WebSecurity allows you basic stuff e.g. WebSecurity.CurrentUserName, WebSecurity.UserExists, WebSecurity.GetUserId, etc.
The custom column you add into any of the tables created by SimpleMembershipProvider have to be fetched manually from the table.
SimpleMembershipProvider have no idea that you added FirstName and LastName into UserProfile (or other table) table.
I'm working a C# form application that ties into an access database. Part of this database is outside of my control, specifically a part that contains strings with ", ), and other such characters. Needless to say, this is mucking up some queries as I need to use that column to select other pieces of data. This is just a desktop form application and the issue lies in an exporter function, so there's no concern over SQL injection or other such things. How do I tell this thing to ignore quotes and such in a query when I'm using a variable that may contain them and match that to what is stored in the Access database?
Well, an example would be that I've extracted several columns from a single row. One of them might be something like:
large (3-1/16" dia)
You get the idea. The quotes are breaking the query. I'm currently using OleDb to dig into the database and didn't have an issue until now. I'd rather not gut what I've currently done if it can be helped, at least not until I'm ready for a proper refactor.
This is actually not as big problem as you may see it: just do NOT handle SQL queries by building them as plain strings. Use SqlCommand class and use query parameters. This way, the SQL engine will escape everything properly for you, because it will know what is the code to be read directly, and what is the parameter's value to be escaped.
You are trying to protect against a SQL Inject attack; see https://www.owasp.org/index.php/SQL_Injection.
The easiest way to prevent these attacks is to use query parameters; http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlparameter.aspx
var cmd = new SqlCommand("select * from someTable where id = #id");
cmd.Parameters.Add("#id", SqlDbType.Int).Value = theID;
At least for single quotes, adding another quote seems to work: '' becomes '.
Even though injection shouldn't be an issue, I would still look into using parameters. They are the simpler option at the end of the day as they avoid a number of unforeseen problems, injection being only one of them.
So as I read your question, you are building up a query as a string in C#, concatenating already queried column values, and the resulting string is either ceasing to be a string in C#, or it won't match stuff in the access db.
If the problem is in C#, I guess you'll need some sort of escaping function like
stringvar += escaped(columnvalue)
...
private static void escaped(string cv) as string {
//code to put \ in front of problem characters in cv
}
If the problem is in access, then
' escapes '
" escapes "
& you can put a column value containing " inside of '...' and it should work.
However my real thought is that, the SQL you're trying to run might be better restructured to use subqueries to get the matched value(s) and then you're simply comparing column name with column name.
If you post some more information re exactly what the query you're producing is, and some hint of the table structures, I'll try and help further - or someone else is bound to be able to give you something constructive (though you may need to adjust it per Jet SQL syntax)
I want to use ODP.NET to run various queries on an oracle database and I'd like to use parameters in the query. Here's a trivial example snippet (omitting all the obvious setup bits of the OracleConnection):
string query = "SELECT FIRSTNAME FROM EMPLOYEES WHERE LASTNAME=:pNAME";
OracleCommand command = new OracleCommand(query);
command.Parameters.Add(":pNAME", OracleDBType.Varchar2).Value = "O'Brien";
My question is, is there anyway to see the query that gets generated from this? I know this is a simple example and the output is probably very obvious, but I'm trying to see how it actually handles things like escaping characters such as the ' in O'Brien. And of course in the future if my queries get more complicated and I'm getting sql errors, I thought I might be able to use the generated query to debug.
Any help or pointers is greatly appreciated!
SQL parameters are passed as parameters directly to SQL server, so there is no way to see it from your application. You can try checking it from Oracle side.