We Keep Coding

C# ADO.NET Loading SQL Script with Multiple Statements and Comments

I am building an application in which I will producing some reports based off the results of some SQL queries executed against a number of different databases and servers. Since I am unable to create stored procedures on each server, I have my SQL scripts saved locally, load them into my C# application and execute them against each server using ADO.NET. All of the SQL scripts are selects that return tables, however, some of them are more complicated than others and involve multiple selects into table variables that get joined on, like the super basic example below.
My question is, using ADO.NET, is it possible to assign a string of multiple SQL queries that ultimately only returns a single data table to a SqlCommand object - e.g. the two SELECT statements below comprising my complete script? Or would I have to create a transaction and execute each individual query separately as its own command?
-- First Select
SELECT *
INTO #temp
FROM Table1;
--Second Select
SELECT *
FROM Table1
JOIN #temp
ON Table1.Id = #temp.Id;
Additionally, some of my scripts have comments embedded in them like the rudimentary example above - would these need to be removed or are they effectively ignored within the string? This seems to be working with single queries, in other words the "--This is a comment" is effectively ignored.
private void button1_Click(object sender, EventArgs e)
{
string ConnectionString = "Server=server1;Database=test1;Trusted_Connection=True";
using (SqlConnection conn = new SqlConnection(ConnectionString))
{
SqlCommand cmd = new SqlCommand("--This is a comment \n SELECT TOP 10 * FROM dbo.Tablw1;");
DataTable dt = new DataTable();
SqlDataAdapter sqlAdapt = new SqlDataAdapter(cmd.CommandText.ToString(), conn);
sqlAdapt.Fill(dt);
MessageBox.Show(dt.Rows.Count.ToString());
}
}

Yes, that is absolutely fine. Comments are ignored. It should work fine. The only thing to watch is the scopin of temporary tables - if you are used to working with stored procedures, the scope is temporary (they are removed when the stored procedure ends); with direct commands: it isn't - they are connection-specific but survive between multiple operations. If that is a problem, take a look at "table variables".
Note: technically this is up to the backend provider; assuming you are using a standard database engine, you'll be OK. If you are using something exotic, then it might be a genuine question. For example, it might not work on "Bob's homemade OneNote ADO.NET provider".

Yes, you can positively do it.
You can play with different types of collections, or with string Builder for passing queries even you can put the string variable and assign the query to it.
While the loop is running put in temp table or CTE, its totally depends on you to choose the approach. and add the data to datatable.
So if you want the entire data to be inserted or Updated or deleted then you can go for transaction,it won't be any issue.

I don't use ado.net, I use Entity Framework but I think this is more a SQL question than an ADO.NET question; Forgive me if I'm wrong. Provided you are selecting from Table1 in both queries I think you should use this query instead.
select *
from Table1 tbl1
join Table1 tbl2
on tbl1.id = tbl2.id
Actually I really don't ever see a reason you would have to move things into temp tables with options like Common Table Expressions available to you.
look up CTEs if you don't already know about them
https://www.simple-talk.com/sql/t-sql-programming/sql-server-cte-basics/

using LINQ on SQL Server with a linked server to oracle db

I was wondering whether I can execute LINQ on an SQL server to speed up my query.
I will make a simple example. Currently I use this to fill my datatable:
using (var connection = new SqlConnection())
using (var da = new SqlDataAdapter())
using (da.SelectCommand = connection.CreateCommand())
{
da.SelectCommand.CommandText = newcmd;
da.SelectCommand.Connection.ConnectionString = connstring;
da.SelectCommand.CommandTimeout = 0;
DataTable ds = new DataTable(); //conn is opened by dataadapter
da.Fill(ds);
}
with this command:
newcmd = "select * from openquery("LinkedServer", 'select * FROM tbl_oracle p ')";
And then once I have the data in the DataTable I use LINQ to manipulate the data as I see fit. Howerever this means I have to transfer the entire table!
Since this returns a lot of data in the real query, the below (simple sum example) turns out to be much faster (mainly because of interface /transfer rates).
newcmd = "select * from openquery("LinkedServer", 'select p.timestep, SUM (p.position)
FROM tbl_oracle p GROUP BY p.timestep ')";
Obviously in reality the data manipulation is more complex. So my question:
Can I somehow use LINQ on oracle db or on Linked Server on SQL Server and execute it on the server so that data manipulation is done before data transfer to desktop? I would really like the power of LINQ without tranferring all the raw data.
UPDATE
I set up a view in sql server management studio on the linked oracle server as suggested in the answer below. I then ran a very simple query:
select * from view where ID=1
with an execution plan and this shows that the entire oracle table is scanned first (remote scan 100% cost) an query is not executed on oracle server. The same query executes in split seconds via openquery. This makes this approach unusable due to the size of the data involved. Any other suggestions would be appreciated.

You can create views on your tables of interest in the SQL Server, and use EF or LINQ to SQL on that tables. In this way, the query will be transfered to the Oracle Server.
EF or LINQ to SQL don't support the specification of the server part on the fully qulified name of a table. But, if you create a view like this:
create view MyView as SELECT * FROM LinkedServer.Database.Schema.Table
you can work on MyView as if it was a table in your local server, and the resulting SQL query will be executed directly on the linked Oracle server.

Sql Connection Catalog changes after statement

Issue:
After running an Sql command with SqlCommand() on a database that then inserts data into another database, all following statements error with ExceptionInvalid object name.
Question:
Why is this happening?
Additional Information:
I know how to fix it by adding The Temp database name before the table on the select portion but since it is being run in the context of that database that shouldn't be necessary and is not when I run the statements individually in SQL management studio
Program Logic:
Create and fill temp database (All tables ASI_...)
In context of temp database select data and then insert it into another database (#AcuDB)
Repeat Step 2 for X queries
Insertion code:
if (TempD.State == System.Data.ConnectionState.Closed) TempD.Open();
Command = new SqlCommand(temp, TempD);
Command.CommandTimeout = 0;
Command.ExecuteNonQuery();
Sample Sql being run that errors after previous similar statement:
insert into #AcuDB..Batch (CompanyID,BranchID,
Module,BatchNbr,CreditTotal,DebitTotal,ControlTotal,CuryCreditTotal,CuryDebitTotal,CuryControlTotal,CuryInfoID,LedgerID,BatchType,Status,AutoReverse,AutoReverseCopy,OrigModule,OrigBatchNbr,DateEntered,Released,Posted,LineCntr,CuryID,ScheduleID,NoteID,CreatedByID,CreatedByScreenID,CreatedDateTime,LastModifiedByID,LastModifiedByScreenID,LastModifiedDateTime,Hold,Description,Scheduled,Voided,FinPeriodID,TranPeriodID)
select 2,
1,Module,BatchNbr,CreditTotal,DebitTotal,ControlTotal,CuryCreditTotal,CuryDebitTotal,CuryControlTotal,i.CuryInfoID,
isnull((select a.LedgerID from #AcuDB..ledger a where a.LedgerCD =
b.LedgerID),0)
[LedgerID],BatchType,Status,AutoReverse,AutoReverseCopy,OrigModule,OrigBatchNbr,
DateEntered
[DateEntered],Released,Posted,LineCntr,b.CuryID,ScheduleID,NoteID,
'B5344897-037E-4D58-B5C3-1BDFD0F47BF9' [CreatedByID], '00000000'
[CreatedByScreenID], GETDATE() [CreatedDateTime],
'B5344897-037E-4D58-B5C3-1BDFD0F47BF9' [LastModifiedByID], '00000000'
[LastModifiedByScreenID], GETDATE()
[LastModifiedDateTime],Hold,Description,Scheduled,Voided,b.FinPeriodID,TranPeriodID
from Temp..ASI_GLBatch b inner join #AcuDB..CurrencyInfo i on
i.CuryEffDate = b.DateEntered cross join #AcuDB..glsetup g where
b.companyID = #CpnyCD and b.branchID = #BranchCD

Going across databases like this is always precarious due to the way SQL will try to imply contexts. In this case, unless #AcuDB contains the fully-qualified address that includes both the database and the schema, you're going to get errors because of the way you're switching contexts around. Get a reading on what #AcuDB contains and try to run the batch in a stored procedure. Set up a separate instance to sandbox the scenario if you have to. The C# end of this is going to continue to complicate things until you cut it out for a little bit and make sure your SQL is good. After you're sure it's okay, integrate it back into the C# code and work from there.

How to SELECT with optional columns?

I'm currently working on a c# application that grabs a bunch of data from a user specified access(.mdb) database and does a bunch of stuff with that data. A problem that I've recently come across is that some of the a database is missing a column that has existed in all of the others.
How can I do a select on a database, but gracefully fail (throw null in the data or something) when a column doesn't exist in the database?
Currently, my code looks something like this:
OleDbConnection aConnection = new
OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + FileName);
string sqlQuery = "SELECT [Table1].[Index], [Table1].[Optional Info],
[Table2].[Other Info], ....
FROM [Table1] INNER JOIN [Table2] ON [Table1].[Index]=[Table2].[Index]
ORDER BY [Table1].[Index]";
OleDbCommand aCommand = new OleDbCommand(sqlQuery, aConnection);
OleDbDataReader aReader = aCommand.ExecuteReader();
(proceed to read data in line by line, using fabulous magic numbers)
I think it's obvious that this is one of my first experiences with databases. I'm not overly concerned as long as it works, but it's stopped working for a database that does not contain the [Table1].[Optional Info] column. It's throwing an OleDbException: "No value given for one or more required parameters."
Any help would be appreciated.

I might be missing something but...
SELECT Table1.*, Table2.otherInfo
FROM ...
Should do the trick, and let the client process the result set, with an important caveat: there is no way to exclude a column from Table1 in the above.
(I am not aware of any method to "dynamically shape" -- with the viewpoint of the caller -- a SELECT except with a * in the column list as above.)
Happy coding.

The way to do that is to not use magic numbers, but to fetch the field names from the reader and use them - for example GetName etc.
Alternatively, use a mapper like "dapper" that will do this for you.

There is no way to do this in a single query: you cannot run a query that includes columns that don't exist in the source tables. When the server tries to compile the query, it will simply fail.
If you absolutely need to support different scemas, you will need different queries for each of them.
To make things even more awesome, there is no documented way to check if an Access table has a particular column on it via SQL. In SQL Server, you could query the system schema, like sys.objects or sys.columns. In Access, the MsysObjects table has the information you need but it's schema is liable to change on you without notice.
Probably the safest way to go about this is to do a single, up front check where you execute a command such as
SELECT * FROM Table1
then scan the resulting column names to see if your optional column exists; your C# code would then become:
string sqlQuery = string.Empty;
if (optionalColumnExists)
{
sqlQuery = "SELECT [Table1].[Index], [Table1].[Optional Info], -- etc."
}
else
{
sqlQuery = "SELECT [Table1].[Index], '' AS [Optional Info], -- etc."
}

There is a way to extract the table schema using OleDbDataReader.GetSchemaTable and that can be used
OleDbConnection aConnection = new
OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + FileName);
OleDbCommand aCommand = new OleDbCommand("Table1", aConnection);
aCommand.CommandType = CommandType.TableDirect;
aConnection.Open();
OleDbDataReader aReader = cmd.ExecuteReader(CommandBehavior.SchemaOnly);
DataTable schemaTable = aReader.GetSchemaTable();
aReader.Close();
aConnection.Close();
bool optionalInfoColumnExists = schemaTable.Columns.Contains("Optional Info");
Now later in the code
string sqlQuery = #"SELECT [Table1].[Index], {0}
[Table2].[Other Info], ....
FROM [Table1] INNER JOIN [Table2] ON [Table1].[Index]=[Table2].[Index]
ORDER BY [Table1].[Index]";
if (optionalInfoColumnExists)
{
sqlQuery = string.Format(sqlQuery, "[Table1].[Optional Info],");
}
else
{
sqlQuery = string.Format(sqlQuery, "");
}
and while reading use similar logic.
I don't know what kind of application this is but the optionalInfoColumnExists should be populated at the application or session start and reused throughout the life of the app i.e. don't execute the GetSchemaTable everytime a query is run on this table (assuming that the mdb won't change while the app is active).
Either way, it seems like that it is going to make the code to have "if else" just to take care of presense and absence of a column in a table.

Retrieving table schema information using C#

I want to retrieve the table schema information using C#.
I used the sp_help <table_name> query to retrieve it, which works well when I execute it in query window but fail to retrieve it from C#. I want to retrieve all table information which is necessary for mapping between two tables such as name, datatype, size, isprimary, key, etc.
I had write down following code
SqlCommand cmd = sourceCon.CreateCommand();
cmd.CommandText = string.Format("sp_help '{0}'", cmbSourceTable.SelectedItem.ToString()); //cmd.CommandType = CommandType.StoredProcedure;
sourceDataTable = new DataTable();
SqlDataReader dr = cmd.ExecuteReader();
sourceDataTable.Load(dr);
dr.Close();
It will returns only info about table that when it is created or not

Instead of using sp_help, you could try the following options:
1) use INFORMATION_SCHEMA.COLUMNS view:
SELECT *
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'YourTable'
2) query sys.columns
SELECT c.*, t.name
FROM sys.columns c
JOIN sys.types t ON c.user_type_id = t.user_type_id
WHERE c.object_id = OBJECT_ID('YourTable')
These are just 2 options that will return a single resultset. INFORMATION_SCHEMA.COLUMNS doesn't tell you (e.g.) if a column is an IDENTITY column, wherease the sys.columns route does. There's more information you can get from the catalog views, just depends what else you need. Here's the MSDN reference.

You might find it easier to play with the INFORMATION_SCHEMA views - you can build queries that will extract most of the data relating to the structure of and relationship between tables within a SQL Server database schema.
See here: http://msdn.microsoft.com/en-us/library/ms186778.aspx

I got a simple way to get schema of a table.
Just create a connection with any provider like SqlConnection object.
Create a command object for select query (select only top 1 record as you want only schema of table)
Execute that query which return a DataReader object.
Every DataReader object have a method called GetTableSchema() which returns DataTable object which have schema of a perticulat table.
This way you can easily get schema of any table with less effort.
Here I will provide a simple code for this
SqlConnection con=new SqlConnection("connString");
con.Open();
SqlCommand cmd= new SqlCommand("select top 0 * from Person.Address",con);
DataTable table = cmd.ExecuteReader().GetTableSchema();
Now this table object have schema information of Person.Address table.

I think sp_help returns multiple result sets so you'll need to use NextResult, there are more details here - http://support.microsoft.com/kb/311274 (the example is not sp_help specific but it should give you the general idea)

Also, you can use the GetSchema method from SqlConnection. http://msdn.microsoft.com/en-us/library/ms136367.aspx
like so:
var tableSchema = con.GetSchema(SqlClientMetaDataCollectionNames.Tables, new string[] { null, null, "[tableName]" });
The string array is a 'filter', here you can find the columns you can filter on: http://msdn.microsoft.com/en-us/library/ms254969.aspx
For other database servers and more info:
http://msdn.microsoft.com/en-us/library/kcax58fh.aspx

sp_help doesn't work in your code? maybe you need to include an execute statement: exec sp_help. or maybe the application is running under an account that doesn't have permission for sp_help.

Since .net 2.0 the SqlConnection class offers GetSchemaMethod you can use to retrieve the requested info.
http://msdn.microsoft.com/en-us/library/ms136364(v=vs.80).aspx

See this one ... it explains how
How To Retrieve Schema Information by Using GetOleDbSchemaTable and Visual C# .NET