For some integration tests I want to connect to the database and run a .sql file that has the schema needed for the tests to actually run, including GO statements. How can I execute the .sql file? (or is this totally the wrong way to go?)
I've found a post in the MSDN forum showing this code:
using System.Data.SqlClient;
using System.IO;
using Microsoft.SqlServer.Management.Common;
using Microsoft.SqlServer.Management.Smo;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
string sqlConnectionString = "Data Source=(local);Initial Catalog=AdventureWorks;Integrated Security=True";
FileInfo file = new FileInfo("C:\\myscript.sql");
string script = file.OpenText().ReadToEnd();
SqlConnection conn = new SqlConnection(sqlConnectionString);
Server server = new Server(new ServerConnection(conn));
server.ConnectionContext.ExecuteNonQuery(script);
}
}
}
but on the last line I'm getting this error:
System.Reflection.TargetInvocationException:
Exception has been thrown by the
target of an invocation. --->
System.TypeInitializationException:
The type initializer for ''
threw an exception. --->
.ModuleLoadException:
The C++ module failed to load during
appdomain initialization. --->
System.DllNotFoundException: Unable to
load DLL 'MSVCR80.dll': The specified
module could not be found. (Exception
from HRESULT: 0x8007007E).
I was told to go and download that DLL from somewhere, but that sounds very hacky. Is there a cleaner way to? Is there another way to do it? What am I doing wrong?
I'm doing this with Visual Studio 2008, SQL Server 2008, .Net 3.5SP1 and C# 3.0.
using Microsoft.SqlServer.Management.Common;
using Microsoft.SqlServer.Management.Smo;
You shouldn't need SMO to execute queries. Try using the SqlCommand object instead. Remove these using statements. Use this code to execute the query:
SqlConnection conn = new SqlConnection(sqlConnectionString);
SqlCommand cmd = new SqlCommand(script, conn);
cmd.ExecuteNonQuery();
Also, remove the project reference to SMO. Note: you will want to clean up resources properly.
Update:
The ADO.NET libraries do not support the 'GO' keyword. It looks like your options are:
Parse the script. Remove the 'GO' keywords and split the script into separate batches. Execute each batch as its own SqlCommand.
Send the script to SQLCMD in the shell (David Andres's answer).
Use SMO like the code from the blog post.
Actually, in this case, I think that SMO may be the best option, but you will need to track down why the dll wasn't found.
MSVCR80 is the Visual C++ 2005 runtime. You may need to install the runtime package. See http://www.microsoft.com/downloads/details.aspx?FamilyID=200b2fd9-ae1a-4a14-984d-389c36f85647&displaylang=en for more details.
In addition to resolving the DLL issue and Matt Brunell's answer (which I feel is more appropriate for what you're trying to do), you can use the SQLCMD command line tool (from the SQL Client tools installation) to execute these SQL scripts. Just be sure it's on your path so you don't struggle with path locations.
This would play out like so:
Actual command:
SQLCMD -S myServer -D myDatabase -U myUser -P myPassword -i myfile.sql
Parameters (case matters):
S: server
d: database
U: User name, only necessary if you don't want to use Windows authentication
P: Password, only necessary if you don't want to use Windows authentication
i: File to run
Code to execute SQL files:
var startInfo = new ProcessStartInfo();
startInfo.FileName = "SQLCMD.EXE";
startInfo.Arguments = String.Format("-S {0} -d {1}, -U {2} -P {3} -i {4}",
server,
database,
user,
password,
file);
Process.Start(startInfo);
See http://msdn.microsoft.com/en-us/library/ms162773.aspx for more information on the SQLCMD tool.
Having the same need to automatically run a generated database script from code, I set out to parse the SQL script to remove GO statements and split the script into separate commands (as suggested by #MattBrunell). Removing the GO statements was easy, but splitting the statements on "\r\n" did not work since that screwed up the multiline-statements. Testing a few different approaches, I was quite surprised to find out that the script doesn't have to be split into separate commands at all. I just removed all "GO" statements and sent the whole script (including comments) into SqlCommand:
using System.Data.SqlClient;
using(SqlConnection connection = new SqlConnection(connectionString))
using(SqlCommand command = connection.CreateCommand())
{
string script = File.ReadAllText("script.sql");
command.CommandText = script.Replace("GO", "");
connection.Open();
int affectedRows = command.ExecuteNonQuery();
}
This code has been tested with SQL Server 2008 R2 and the script generated by "Database -> Tasks -> Generate Scripts...". Below are some examples of the commands in the script:
USE [MyDatabase]
ALTER TABLE [MySchema].[MyTable] DROP CONSTRAINT [FK_MyTable_OtherTable]
DROP TABLE [MySchema].[MyTable]
SET ANSI_NULLS ON
SET QUOTED_IDENTIFIER ON
/****** Object: Table [MySchema].[MyTable] Script Date: 01/23/2013 13:39:29 ******/
CREATE TABLE [MySchema].[MyTable](
[Id] [int] IDENTITY(1,1) NOT NULL,
[Subject] [nvarchar](50) NOT NULL,
[Body] [nvarchar](max) NOT NULL,
CONSTRAINT [PK_MyTable] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
SET IDENTITY_INSERT [MySchema].[MyTable] ON
INSERT [MySchema].[MyTable] ([Id], [Subject], [Body]) VALUES (1, N'MySubject', N'Line 1
Line 2
Line 3
Multi-line strings are also OK.
')
SET IDENTITY_INSERT [MySchema].[MyTable] OFF
I guess there might be some maximum length for a single SqlCommand, above which the script have to be split. My script, which execute without problems, contains around 1800 statements and is 520 kB.
Have you tried running this with a very, very basic script in the .sql file? Maybe something that just inserts one row or creates an arbitrary table? Something that is very easy to verify? Essentially, this code is like hard coding the sql, except you're reading it from a file. If you can get it to work with a very simple file, then I would say that there is likely something wrong with the file structure itself. The post alluded to the fact that there are some stipulations regarding what can and cannot be in the file. If nothing else, it's a good place to start troubleshooting.
If you add following references in your project, then original code will work fine.
I use SQL 2008 Express.
Path:
C:\Program Files (x86)\Microsoft SQL Server\100\SDK\Assemblies\
Files:
microsoft.sqlserver.smo.dll, microsoft.sqlserver.connectioninfo.dll and Microsoft.SqlServer.Management.Sdk.Sfc.dll
You may be interested in this:
Link
It presents a general-purpose 'test fixture' to automatically execute sql-scripts. There is also sample code available, and there are no dependencies to any unusual assemblies whatsoever...
Related
I have a stored procedure in a myscript.sql file that looks like this:
CREATE PROCEDURE [dbo].[_GetUserID]
#EmailAddress NVARCHAR(254)
AS
DECLARE #UserID UNIQUEIDENTIFIER;
SELECT #UserID = [ID]
FROM [dbo].[User]
WHERE [EmailAddress] = #EmailAddress
PRINT #UserID
GO
I have some C# code that relies on Dapper to run this script. I can successfully run this script when I copy-and-paste it into Azure Data Studio. However, when I am trying to run this script from code, I get an error:
Incorrect syntax near 'GO'
My C# code looks like this:
try
{
var script = File.ReadAllText("<path to myScript.sql is here>");
using (var connection = new SqlConnection(dbConnectionString))
{
var command = connection.CreateCommand();
command.CommandText = script;
command.CommandType = CommandType.Text;
connection.Open();
command.ExecuteNonQuery();
}
Console.WriteLine("Success.");
}
catch (Exception ex)
{
Console.WriteLine($"Failed. Reason: '{ex.Message}')");
}
I don't understand why I can run myScript.sql from Azure Data Studio, however, it's not working from my C# code. I'm also creating tables using the same approach and it works fine. I'm not sure what I'm missing.
GO is not a valid T-SQL keyword - it's a separator that is used by SQL Server Management Studio and obviously also Azure Data Studio.
To fix this, just simply remove that GO line from your .sql script file and run it without this - should be just fine.
On a different note: having nothing but a PRINT statement in your stored procedure doesn't make a lot of sense - don't you want to actually SELECT #UserId to get that data sent back to the caller??
First off, Ill say that this issue is related to my previous post.
However, I'll move everything over here for reference.
The issue I am having is I am still getting the error:
Msg 6522, Level 16, State 1, Procedure PerfInsert, Line 0 [Batch Start Line 31]
A .NET Framework error occurred during execution of user-defined routine or aggregate "PerfInsert":
System.Security.SecurityException: Request failed.
System.Security.SecurityException:
at MiddleMan.MiddleMan.CreateCommand(SqlString tblString, SqlString featureName, SqlString connectionString, SqlString perfionConnectionString, SqlString logFile)
.
Even though I believe I have followed all the steps necessary to set this up correctly. I have even gone so far as to verify that SQL Server has permissions to the directory of the files.
Anyone know what else I can check to see what the missing piece is?
Or do I need to make this an "unsafe" assembly?
C# code:
using Microsoft.SqlServer.Server;
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.Diagnostics;
namespace MiddleMan
{
public static class MiddleMan
{
[SqlProcedure(Name = "PerfInsert")]
public static SqlInt32 CreateCommand(SqlString tblString, SqlString featureName, SqlString connectionString, SqlString perfionConnectionString, SqlString logFile)
{
Process compiler = new Process();
compiler.StartInfo.FileName = "C:\\SQL Server C# Functions\\PerfionLoader\\PerfionLoader\\bin\\Release\\PerfionLoader.exe";
compiler.StartInfo.Arguments = tblString.Value + " " + featureName.Value + " " + connectionString.Value + " " + perfionConnectionString.Value + " " + logFile.Value;
//compiler.StartInfo.UseShellExecute = false;
//compiler.StartInfo.RedirectStandardOutput = true;
compiler.Start();
return SqlInt32.Zero;
}
}
}
SQL code(s):
CREATE ASSEMBLY PerfInsert
AUTHORIZATION dbo
FROM '\\bk-int-1\c$\SQL Server C# Functions\MiddleMan\MiddleMan\bin\Release\MiddleMan.dll'
WITH PERMISSION_SET = SAFE
GO
CREATE ASYMMETRIC KEY [Brock.Retail_Brock.Retail_Brock]
AUTHORIZATION [dbo]
FROM EXECUTABLE FILE = '\\bk-int-1\c$\SQL Server C# Functions\MiddleMan\MiddleMan\bin\Release\MiddleMan.dll';
CREATE LOGIN BrokcRetail
FROM ASYMMETRIC KEY [Brock.Retail_Brock.Retail_Brock]
CREATE PROCEDURE PerfInsert
(
#tblString nvarchar(max)
, #featureName nvarchar(max)
, #connectionString nvarchar(max)
, #perfionConnectionString nvarchar(max)
, #logFiel nvarchar(max)
)
AS EXTERNAL NAME PerfInsert.[MiddleMan.MiddleMan].[CreateCommand]
GO
You are using multi-threading so yes, the Assembly 100% needs to have PERMISSION_SET = UNSAFE.
Also, since you already have the Asymmetric Key and associated Login set up (thank you for doing that and not using TRUSTWORTHY ON), you will need to do the following prior to setting the Assembly to UNSAFE:
USE [master];
GRANT UNSAFE ASSEMBLY TO [BrokcRetail];
and then:
USE [{db_containing_assembly_hopefully_not_master];
ALTER ASSEMBLY [PerfInsert] WITH PERMISSION_SET = UNSAFE;
or, if you create the Asymmetric Key-based Login and grant it the UNSAFE ASSEMBLY permission first, then you can simply use UNSAFE instead of SAFE in the CREATE ASSEMBLY statement.
Starting in SQL Server 2017, you will need to create the Asymmetric Key and associated Login before creating the Assembly. The Asymmetric Key and Login go into [master] while the Assembly can go into any DB (including [master], but usually best to not put custom code in there).
If you are already using SQL Server 2017 or newer, and if the code shown in the question is in the actual order in which you are executing it, then I would guess that you have already either set the database to TRUSTWORTHY ON or disabled "CLR strict security". Otherwise you should not have been able to create the Assembly at all without first having the signature-based login created and granted the UNSAFE ASSEMBLY permission. If I am correct about this, you can re-enable "CLR strict security" and/or turn TRUSTWORTHY OFF for that database.
Also, as I noted on your related question (the one linked to in this question), you should be using SqlString instead of SqlChars. SqlString.Value returns a .NET string while SqlChars.Value returns a char[]. Long ago people associated SqlChars with NVARCHAR(MAX), and SqlString with NVARCHAR(1-4000), but that was only due to Visual Studio / SSDT using those mappings as defaults when generating the DDL to publish the Database Project. But there never was any technical / string mapping between them. You can use either .NET type with either T-SQL datatype.
Also, please exercise caution (and lots of testing) when using multi-threading from within SQLCLR.
Please visit SQLCLR Info for more resources related to working with SQLCLR in general.
Related Posts:
System.Web in SQL Server CLR Function (on DBA.StackExchange)
CREATE PROCEDURE gets “Msg 6567, Level 16, State 2” for SQLCLR stored procedure
I am trying to setup my .NET 4.7.1 program that is connecting to a MySQL database 8.0 to use the minimum privileges to run.
The .NET program is using MySql.Data to make connection. The minimum right for a user to execute a stored procedure is typically only EXECUTE privilege. This works fine from MySQL workbench or command line.
Upon running the .NET program this does return the following exception:
System.Data.SqlTypes.SqlNullValueException: 'Data is Null. This method or property cannot be called on Null values.'
To make it easy, I have create a very small demo program to demonstrate the issue.
Setup of the database:
CREATE DATABASE Spike;
CREATE PROCEDURE TestAccess()
BEGIN
END;
CREATE USER Spike#localhost IDENTIFIED WITH mysql_native_password BY 'sample';
GRANT EXECUTE ON PROCEDURE `TestAccess` TO Spike#localhost;
Setup program code:
static void Main(string[] args)
{
using (MySqlConnection conn = new MySqlConnection("Server=localhost;Database=Spike;uid=Spike;pwd=sample"))
{
conn.Open();
Console.WriteLine("Connection open");
MySqlCommand cmd = new MySqlCommand();
cmd.Connection = conn;
cmd.CommandText = "TestAccess";
cmd.CommandType = System.Data.CommandType.StoredProcedure;
cmd.ExecuteNonQuery();
Console.WriteLine("Query executed");
}
Console.ReadKey();
}
The crash happens at the line cmd.ExecuteNonQuery();
The stack from the crash is interesting, since it seems to indicate that the information_schema is queried. When logging all statements I can see that the last statement before the exception is:
SELECT * FROM information_schema.routines WHERE 1=1 AND routine_schema LIKE 'Spike' AND routine_name LIKE 'TestAccess'
I cannot grant different rights on information_schema, but I could give more rights on the stored procedure to make more information visible in the routines table, this feels wrong however. Simple tests with granting CREATE and ALTER access also did not work.
Is there something else I can do, without granting too much privileges?
This appears to be a bug in Connector/NET, similar to bug 75301 but a little different. When it's trying to determine parameter metadata for the procedure, it first creates a MySqlSchemaCollection named Procedures with all metadata about the procedure. (This is the SELECT * FROM information_schema.routines WHERE 1=1 AND routine_schema LIKE 'Spike' AND routine_name LIKE 'TestAccess' query you see in your log.)
The Spike user account doesn't have permission to read the ROUTINE_DEFINITION column, so it is NULL. Connector/NET expects this field to be non-NULL and throws a SqlNullValueException exception trying to read it.
There are two workarounds:
1) The first, which you've discovered, is to set CheckParameters=False in your connection string. This will disable retrieval of stored procedure metadata (avoiding the crash), but may lead to harder-to-debug problems calling other stored procedures if you don't get the order and type of parameters exactly right. (Connector/NET can no longer map them for you using the metadata.)
2) Switch to a different ADO.NET MySQL library that doesn't have this bug: MySqlConnector on NuGet. It's highly compatible with Connector/NET, performs faster, and fixes a lot of known issues.
I found an answer with which I am quite pleased. It is changing the connection string by adding CheckParameters=false:
using (MySqlConnection conn = new MySqlConnection("Server=localhost;Database=Spike;uid=Spike;pwd=sample;CheckParameters=false"))
This disables parameter checking, and thereby information_schema queries.
I know most people will give responses like, right click database name, select task, generate scripts, etc. Most people give detail response on the internet include stackoverflow
I already know that long time ago.
My question is any suggestions to write scripts to export/import stored procedures. If I only have to do 5 or 10, it is no big issue. I just have to pick right ones. If there are lots like 50 or more, it is pain and bound to be errors. Sometimes, to make life easier, we want to export/import with suffix such as *test.sql.
Any suggestion using SQL scripts, C# or dos commands to do so?
Thanks
Since you tagged also C#. With .NET you can do it using Microsoft.SqlServer.Smo. There is a StoredProcedure class. This approach gives you really lots of control over the task you want to achieve. Example usage you can find here, also using PowerShell.
Here's a snippet I've put together that might help you(uses Microsoft.SqlServer.Smo, Microsoft.SqlServer.ConnectionInfo, Microsoft.SqlServer.Management.Sdk.Sfc DLLs):
var serverConnection = new ServerConnection(new SqlConnection(ConnectionString));
var server = new Server(serverConnection);
foreach (StoredProcedure sp in server.Databases["YourDB"].StoredProcedures)
{
if (sp.Name.Contains("yourSubstring"))
{
System.IO.File.WriteAllText(sp.TextHeader + Environment.NewLine + sp.TextBody);
}
}
Why have you tagged this question with C# when you ask for answers in SQL or DOS script?
This CMD script drops all stored procedures whose names begin with "usp":
FOR /F "tokens=* USEBACKQ" %%F IN (`sqlcmd -S %SERVER% -d %DATABASE% -Q "select name from sys.objects where type = 'P'" `) DO (echo %%F | findstr "^usp" >nul 2>&1 && sqlcmd -S %SERVER% -d %DATABASE% -Q "drop procedure %%F" )
It's all on one line and %SERVER% and %DATABASE% identify the server and database.
This script loads all stored procedures from the current directory whose names match "usp*.sql". You will need the setting QUOTED_IDENTIFIER to be ON if you invoke XML data type methods.
for %%f in (usp*.sql) do sqlcmd -S %SERVER% -d %DATABASE% -I -i %%f
I am trying to use a parametrized query to create an index on our Oracle server. I can create the index fine if I use string concatenation, so I believe its not an account or permissions issue. I get the error:
ORA-01036: illegal variable name/number
I can not find any error in the code, but I am sure I am missing something. I am using Oracle.DataAccess.dll version 4.112.3.0. The version the server reports is "Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production"
I have tried removing all the params, except one and get the same error. Any combination of 1 or more params causes the error. The error happens every time without fail. I have used watches to verify that the params are correctly getting set, and I have tried clearing the params before setting. No matter what I do so far, I still get the error. Here is the code:
using (OracleCommand mycom = new OracleCommand())
{
using (OracleConnection Connection = new OracleConnection(connectionString))
{
mycom.BindByName = true;
mycom.CommandText = "CREATE INDEX :indexname on :tablename (:colname)";
mycom.Parameters.Add("indexname", indexName);
mycom.Parameters.Add("tablename", tableName);
mycom.Parameters.Add("colname", colName);
mycom.Connection = Connection;
mycom.CommandType = CommandType.Text;
mycom.Connection.Open();
mycom.ExecuteNonQuery();
}
}
You cannot use bind variables in a DDL statement (like CREATE INDEX). Bind variables are only used in DML statements. You would need to build up the DDL statement in your C# code.
Is this a part of the installation process for your application? Otherwise, it would be exceedingly unusual to want to create an index from an application. Your application shouldn't be creating any database objects outside of the initial installation process.