This question already has answers here:
List of tables used in an SQL Query
(5 answers)
Closed 9 years ago.
I need extract from simple string that represent an sql query the tables that are used on the query without execute the query itself in C#.
Example:
string strQuery = "SELECT * FROM table1 LEFT JOIN (SELECT * FROM table2) tt WHERE tt.name IN (SELECT name FROM table3)";
ArrayList arrUsedTables = GetUsedTablesFromQuery(strQuery);
and after this line the object arrUsedTables would contain:
table1,table2,table3
Remember that the query may be much complicated!
Without going to the DB you can't know for certain the names of the tables used in the query.
What if your query uses a view or a stored procedure?
Without consulting the database, these are transparent to the consumer.
The only way to be certain is to query the list of the tables from the database and then to attempt to parse them from your inline sql.
You will have to add references and directives for the following assemblies:
using Microsoft.Data.Schema.ScriptDom;
using Microsoft.Data.Schema.ScriptDom.Sql;
using System.IO;
Then, you may create the GetUsedTablesFromQuery method:
private static ArrayList GetUsedTablesFromQuery(string strQuery)
{
var parser = new TSql100Parser(true);
IList<ParseError> errors = new List<ParseError>();
using (TextReader r = new StringReader(strQuery))
{
var result = parser.GetTokenStream(r, out errors);
var tables = result
.Select((i, index) => (i.TokenType == TSqlTokenType.From) ? result[index + 2].Text : null)
.Where(i => i != null)
.ToArray();
return new ArrayList(tables);
}
}
You can certainly use a SQL parser such as ANTLR, as described in this question and answer, in order to get a full parse of the SQL, and then extract the table names.
Another option is to execute some raw SQL to get the execution plan of the query (using the instructions here). The execution plan is in XML, and then you can use Linq to XML to query the plan for all Table attributes on any ColumnReference tag.
Related
I'm trying to retrieve all fields from two joined tables to any kind of a c# object.
So I'm trying to run this code:
var query = #$"EXEC('select *
from persons p join students s on p.id=s.id
where p.id = 21')";
var result = _context.Database.SqlQuery<?>(query).ToList();
But I don't get what should be instead of the question mark.
I've tried List<object> and Dictionary<string,string> but since I couldn't get exactly how this is being mapped, I don't understand to what it can be mapped.
There is a somewhat similar question here but its solution only addresses two columns, and it apparently doesn't support returning nulls.
You can try creating a stored procedure or a function in the SQL level.
Then, just select then generated table / result.
So, you already have an idea of what class it is.
I frequently use the dynamic type like this :
var lst = _context.Database.SqlQuery<dynamic>(query).ToList();
foreach (var item in lst)
{
var myVar = item.myfieldName;
}
It may be preferable to name each field in your query instead of using select *.
This question already has answers here:
Get SQL query from LINQ to SQL?
(4 answers)
Closed 7 years ago.
If I have a LINQ to SQL statement for example
var query = (from a in this.Context.Apples select a.Name).ToList();
When I want to see what SQL is being generated by LINQ, what I do is that I comment out the ToList() and put a breakpoint on the command after this LINQ statement and then I can hover it and read the SQL.
My question: Is that a correct way of getting the generated SQL?
You can also set the Log property of Your context to :
public class MyContext : DbContext{
MyContext(){
Database.Log = Console.WriteLine;
//or like this
//Database.Log = message => Trace.TraceInformation(message);
}
}
Yes, that's a correct way, but of course, there are others:
var context = new DataClasses1DataContext();
var sb = new StringWriter();
context.Log = sb;
var query = (from a in context.Persons select a.Name);
string s = query.ToString();
string command = context.GetCommand(query).CommandText;
//The log requires the query to actually hit the database
query.ToList();
string log = sb.ToString();
And also Linqpad:
You can also use SQL Server Profiler, add a trace, and see generated queries as they are being executed by the server.
Recent bug report states that a method being called is crashing the service causing it to restart. After troubleshooting, the cause was found to be an obnoxious Oracle SQL call with thousands of strings passed. There is a collection of strings being passed to a method from an external service which often is more than 10,000 records. The original code used a where clause on the passed collection using the LIKE keyword, which I think is really, really bad.
public IList<ContainerState> GetContainerStates(IList<string> containerNumbers)
{
string sql =
String.Format(#"Select CTNR_NO, CNTR_STATE FROM CONTAINERS WHERE CTRN_SEQ = 0 AND ({0})",
string.Join("OR", containerNumbers
.Select(item => string.Concat(" cntr_no LIKE '", item.SliceLeft(10), "%' ")))
);
return DataBase.SelectQuery(sql, MapRecordToContainerState, new { }).ToList();
}
Clarification of in house methods used which may be confusing:
DataBase.SelectQuery is an internal library method using generics which gets passed the sql string, a function to map the records to .NET objects, and the parameters being passed and returns an IEnumerable of Objects of type retuned by the Mapping function.
SliceLeft is an extension method from another internal helper library that just returns the first part of a string up to the number of characters specified by the parameter.
The reason that the LIKE statement was apparently used, is that the strings being passed and the strings in the database only are guaranteed to match the first 10 characters. Example ("XXXX000000-1" in the strings being passed should match a database record like "XXXX000000-8").
I believed that the IN clause using the SUBSTR would be more efficent than using multiple LIKE clauses and replaced the code with:
public IList<ContainerRecord> GetContainerStates(IList<string> containerNumbers)
{
string sql =
String.Format(#"Select CTNR_NO, CNTR_STATE FROM CONTAINERS WHERE CTRN_SEQ = 0 AND ({0})",
string.Format("SUBSTR(CNTR_NO, 1, 10) IN ({0}) ",
string.Join(",", containerNumbers.Select(item => string.Format("\'{0}\'", item.SliceLeft(10) ) ) )
)
);
return DataBase.SelectQuery(sql, MapRecordToContainerState, new { }).ToList();
}
This helped slightly, and there were fewer issues in my tests, but when there are huge amounts of records passed, there is still an exception thrown and core dumps occur, as the SQL is longer than the server can parse during these times. The DBA suggests saving all the strings being passed to a temporary table, and then joining against that temp table.
Given that advice, I changed the function to:
public IList<ContainerRecord> GetContainerStates(IList<string> containerNumbers)
{
string sql =
#"
CREATE TABLE T1(cntr_num VARCHAR2(10));
DECLARE GLOBAL TEMPORARY TABLE SESSION.T1 NOT LOGGED;
INSERT INTO SESSION.T1 VALUES (:containerNumbers);
SELECT
DISTINCT cntr_no,
'_IT' cntr_state
FROM
tb_master
WHERE
cntr_seq = 0
AND cntr_state IN ({0})
AND adjustment <> :adjustment
AND SUBSTR(CTNR_NO, 1, 10) IN (SELECT CNTR_NUM FROM SESSION.T1);
";
var parameters = new
{
#containerNumbers = containerNumbers.Select( item => item.SliceLeft(10)).ToList()
};
return DataBase.SelectQuery(sql, MapRecordToContainerState, parameters).ToList();
}
Now I'm getting a "ORA-00900: invalid SQL statement". This is really frustrating, how can I properly write a SQL Statement that will put this list of strings into a temporary table and then use it in a SELECT Statement to return the list I need?
There are couple possible places could cause this error, it seams that the "DECLARE GLOBAL TEMPORARY" is a JAVA API, I don't think .net has this function. Please try "Create global temporary table" instead. And, I don't know whether your internal API could handle multiple SQLs in one select sql. As far as I know, ODP.net Command class can only execute one sql per call. Moreover, "create table" is a DDL, it therefore has its own transaction. I can't see any reason we should put them in the same sql to execute. Following is a sample code for ODP.net,
using (OracleConnection conn = new OracleConnection(BD_CONN_STRING))
{
conn.Open();
using (OracleCommand cmd = new OracleCommand("create global temporary table t1(id number(9))", conn))
{
// actually this should execute once only
cmd.ExecuteNonQuery();
}
using (OracleCommand cmd = new OracleCommand("insert into t1 values (1)", conn)) {
cmd.ExecuteNonQuery();
}
// customer table is a permenant table
using (OracleCommand cmd = new OracleCommand("select c.id from customer c, t1 tmp1 where c.id=tmp1.id", conn)) {
cmd.ExecuteNonQuery();
}
}
This question already has answers here:
Parameterize an SQL IN clause
(41 answers)
Closed 9 years ago.
In my application written in C# , I am writing a SQL query. Following is the query
SELECT [Resource No_] where [Resource No_] In (#resources)
#resources is user input parameters having one or more that one strings.
My query is failing without showing an error
According to me, query is failing because in #resources parameter following is being passed
"'123,'124','125'"
(there are 2 inverted commas in the beginning and at the end and that is failing my query).
[Resource No_] is of type NVARCHAR in the database.
After Googling, I have found some help on this topic but all are applicable when [Resource No_] is of type Integer
While I don't agree with the selected answer (or many of the tricky answers) for the "duplicate question", here is an answer to it which shows an approach very similar with my following recommendation.
(I've voted to close this question as a duplicate, because there are such answers, even if buried.)
Only one SQL value can be bound to any given placeholder.
While there ways to send all the data as "one value", I'd recommend creating the placeholders dynamically: it's simple, clean, and will work reliably in most cases.
Consider this:
ICollection<string> resources = GetResources();
if (!resources.Any()) {
// "[Resource No_] IN ()" doesn't make sense
throw new Exception("Whoops, have to use different query!");
}
// If there is 1 resource, the result would be "#res0" ..
// If there were 3 resources, the result would be "#res0,#res1,#res2" .. etc
var resourceParams = string.Join(",",
resources.Select((r, i) => "#res" + i));
// This is NOT vulnerable to classic SQL Injection because resourceParams
// does NOT contain user data; only the parameter names.
// However, a large number of items in resources could result in degenerate
// or "too many parameter" queries so limit guards should be used.
var sql = string.Format("SELECT [Resource No_] where [Resource No_] In ({0})",
resourceParams);
var cmd = conn.CreateCommand();
cmd.CommandText = sql;
// Assign values to placeholders, using the same naming scheme.
// Parameters prevent SQL Injection (accidental or malicious).
int i = 0;
foreach (var r in resources) {
cmd.Parameters.AddWithValue("#res" + i, r);
i++;
}
Use a user defined table type to accept your parameter, then a JOIN clause in your select to limit the results set. See http://social.msdn.microsoft.com/Forums/en-US/2f466c93-43cd-436d-8a7c-e458feae71a0/how-to-use-user-defined-table-types
Do something like
resources.Aggregate(r1, r2 => r1 + "', '" + r2 + "'")
and pass the list in one string.
I'm doing an EF query with a list which contains many elements and I seem to be coming across the same issue listed here (where SQL Server throws an error because there are too many elements in the IN statement). I was wondering if there is another way to do it.
Would ExecuteStoreQuery or ExecuteStoreCommand work if the command set up a temporary table?
Thanks
Yep, the best way to select from a large list of keys is to use a temp table.
http://explainextended.com/2009/08/18/passing-parameters-in-mysql-in-list-vs-temporary-table/
If you're using MsSQL and C# then SqlBulkCopy will get your list of keys in the fastest.
public void bulkCopy(String tmpTableName, DataTable table)
{
using (SqlBulkCopy bulkCopy =
new SqlBulkCopy((SqlConnection)connection))
{
bulkCopy.DestinationTableName = tmpTableName;
bulkCopy.WriteToServer(table);
}
}
Then have a stored procedure to match to the temp table by key.
If that list data comes from a database query, do not call .ToList() on your source query. Instead pass the IQueryable to the query.
var statuses = context.Statuses.Where(o => o.IsActive).Select(o => o.Id);
var data = context.Orders.Where(o => statuses.Contains(o.StatusId));
Alternatively you can use something like this
var sessionId = Guid.NewGuid();
foreach (var s in statusList)
insert into SearchValues(sessionId, s); // pseudo code
var statuses = context.SearchValues.Where(o => o.SessionId == sessionId).Select(o => o.Id);
var data = context.Orders.Where(o => statuses.Contains(o.StatusId));
delete from SearchValues where SessionId == #sessionId // pseudo code
You might want to do the insert and delete using SQL queries (and not EF context operations) for optimal performance.