SQL Server CLR TVF not multithread capable? - c#

Regarding this previous stackoverflow question:
DRY CLR table-valued functions
It seems it only runs in single thread mode. To test this I modified the code slightly to prepend the Name field with the current thread number. All of the returned results had the same thread number assigned. Is this action by design? Is there anyway to get it to multithread? Thanks.
private class ResultRow
// This class holds a row which we want to return.
{
public SqlInt32 CustId;
public SqlString Name;
public ResultRow(SqlInt32 custId_, SqlString name_)
{
int mythread = Thread.CurrentThread.ManagedThreadId;
CustId = custId_;
Name = "[" + mythread.ToString() + "] " + name_;
}
}
EDITED per Marc's question:
Here's the full piece of code. It returns 3470 records in 7 seconds.
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using System.Runtime.InteropServices;
using System.Text;
using System.Collections.Generic;
using System.Collections;
using System.Threading;
namespace CS_CLR_TVF
{
public partial class UserDefinedFunctions
{
// This class holds a row which we want to return.
private class ResultRow
{
public SqlString fldProductName;
public ResultRow(SqlString product_)
{
int mythread = Thread.CurrentThread.ManagedThreadId;
fldProductName = "[" + mythread.ToString() + "] " + product_;
}
}
[SqlFunction(DataAccess = DataAccessKind.Read, FillRowMethodName = "Test_FillRow", TableDefinition = "fldProductName nvarchar(1024)")]
public static IEnumerable xudf_cs_tvf(SqlString strSearchClue)
{
ArrayList results = new ArrayList();
using (SqlConnection connection = new SqlConnection("context connection=true"))
{
connection.Open();
string s1;
using (SqlCommand select = new SqlCommand("SELECT fldProductName FROM tblProducts", connection))
{
using (SqlDataReader reader = select.ExecuteReader())
{
while (reader.Read())
{
s1 = reader.GetSqlString(0).ToString();
// do a substring compare, if "match" grab the row
int idx = s1.IndexOf(strSearchClue.ToString());
if (idx > -1) results.Add(new ResultRow(reader.GetSqlString(0)));
}
}
}
}
return results;
}
// This function takes a row and tells SQL Server what variables we want to
// return from it and what types it contains.
public static void Test_FillRow(object resultsObj, out SqlString fldProductName)
{
ResultRow selectResults = (ResultRow)resultsObj;
fldProductName = selectResults.fldProductName;
}
}
}
Pretty straight forward internal select statement:
SELECT fldProductName FROM tblProducts
.
.
.
.
Here's a version implemented as a scalar UDF and it does do multithreading. It returns 3470 records in <1 second.
[Microsoft.SqlServer.Server.SqlFunction]
public static long xudf_csfake(SqlString strSearchClue, SqlString strStringtoSearch)
{
string s1 = strStringtoSearch.ToString();
// do a substring compare, if "match" grab the row
int idx = s1.IndexOf(strSearchClue.ToString());
if (idx > -1) return 1;
return 0;
}
Here is it's external select statement:
SELECT fldProductName FROM tblProducts WHERE (dbo.xudf_csfake('METAL' ,fldProductName) = 1)
So I seem to getting the opposite of what the article indicates.

Related

How to make a function that returns value from a list in C#

I have a list filled with objects. I have to make a function that when given a certain value of one of the class atributes, gives back another one. In my case, when you specify the "index", the function will return "standysp"(I'm sorry if I can't explain it to well). I need to make this function work but I don't even know when to start. Here is my Code:
using System;
using FirebirdSql.Data.FirebirdClient;
using System.Collections.Generic;
namespace dokselect
{
class PozycjaMagazynowa
{
public double standysp;
public string nazwagrupy;
public string index;
public string nazwadl;
}
class Program
{
public static void Main()
{
string conn = "SECRET";
FbConnection myConnection = new FbConnection(conn);
FbDataReader myReader = null;
string sql = "select STANMAG.standysp,GRUPAKART.nazwagrupy, KARTOTEKA.indeks, kartoteka.nazwadl FROM stanmag JOIN kartoteka using(ID_KARTOTEKA) JOIN wystgrkart using(ID_KARTOTEKA) JOIN grupakart using(ID_GRUPAKART) ORDER BY nazwagrupy;";
FbCommand myCommand = new FbCommand(sql, myConnection);
myConnection.Open();
myReader = myCommand.ExecuteReader();
List<PozycjaMagazynowa> lista1 = new List<PozycjaMagazynowa>();
double standysp;
string nazwagrupy;
string index;
string nazwadl;
while (myReader.Read())
{
standysp = Convert.ToDouble(myReader[0]);
nazwagrupy = myReader[1].ToString();
index = myReader[2].ToString();
nazwadl = myReader[3].ToString();
lista1.Add(new PozycjaMagazynowa { standysp = standysp, nazwagrupy = nazwagrupy, index = index, nazwadl = nazwadl });
}
myConnection.Close();
Console.WriteLine(lista1.Count);
//LISTA DONE
void wyswietl()
{
//????????
}
}
}
}
I'm not sure you're doing it the right way.
First the List you create is not accessible to others methods.
Secund, maybe the List is not appropriate and you may use a HashSet<T> or Dictionnary with your PozycjaMagazynowa class implementing IEqualityComparer
I used this method and it worked thank you for help:
double wyswietl()
{
string myIndex="WSIP_MAT_GIM";
var result = lista1.FirstOrDefault(lista1 => lista1.index == myIndex).standysp;
Console.WriteLine(myIndex + " Zostalo zwrocone przez funkcje");
Console.WriteLine(result + " - to stan dyspozycyjny pasujacy do podanego indexu");
return result;
}
wyswietl();

How To Pass a List Object From C# to an Oracle Stored Procedure?

I'm trying to send a List Object from my C# WebService method over to my stored procedure in Oracle.
Before posting here, I've tried all suggested duplicate links. Here's what I've accomplished so far:
Success: In C#, I can pass my List values from my HTML page over to my WebService method.
Success: In Oracle, I have created a Table, Object Type, Table Type, and Stored Procedure to accept the List values. I was able to test this using an Anonymous block and sample data.
Problem: I cannot get to pass my List values from my C# WebMethod over to my Oracle Stored Procedure.
I'm currently using the following setup:
Visual Studio 2017
.NET Framework 4.6.1
Oracle.ManagedDataAccess 18.6.0
Keep in mind that the version of Oracle.ManagedDataAccess 18.6.0 does NOT contain the OracleDbType.Array as suggested in the older examples.
public class Automobile
{
public string Make { get; set; }
public string Model { get; set; }
public string Year { get; set; }
public string Country { get; set; }
}
using Oracle.ManagedDataAccess.Client;
using Oracle.ManagedDataAccess.Types;
[WebMethod(EnableSession = true)]
[ScriptMethod(ResponseFormat = ResponseFormat.Json)]
public string InsertCars(List<Automobile> myCars, int userID)
{
DataSet dataSet = new DataSet();
using (OracleConnection sqlConnection = new OracleConnection(OracleDBConnection))
{
using (OracleCommand sqlCommand = new OracleCommand("sp_InsertCars", sqlConnection))
{
sqlConnection.Open();
sqlCommand.CommandType = CommandType.StoredProcedure;
sqlCommand.Parameters.Add(
new OracleParameter
{
CollectionType = OracleCollectionType.PLSQLAssociativeArray,
Direction = ParameterDirection.Input,
ParameterName = "p_CarList",
UdtTypeName = "tt_Automobile",
Size = myCars.Count,
Value = myCars.ToArray()
}
);
sqlCommand.Parameters.Add(
new OracleParameter
{
OracleDbType = OracleDbType.Int32,
Direction = ParameterDirection.Input,
ParameterName = "p_UserID",
Value = userID
}
);
sqlCommand.Parameters.Add(
new OracleParameter
{
OracleDbType = OracleDbType.RefCursor,
Direction = ParameterDirection.Output,
ParameterName = "o_Cursor"
}
);
using (OracleDataAdapter sqlAdapter = new OracleDataAdapter(sqlCommand))
{
sqlAdapter.SelectCommand = sqlCommand;
sqlAdapter.Fill(dataSet);
}
}
return JsonConvert.SerializeObject(dataSet);
}
}
CREATE TABLE tblCars
(
RecordID INT GENERATED BY DEFAULT AS IDENTITY NOMINVALUE NOMAXVALUE INCREMENT BY 1 START WITH 1 NOCACHE NOCYCLE NOORDER,
Make NVARCHAR2(100) NULL,
Model NVARCHAR2(100) NULL,
Year NVARCHAR2(4) NULL,
Country NVARCHAR2(100) NULL,
UserID INT NULL
);
CREATE OR REPLACE TYPE ot_Automobile AS OBJECT
(
Make varchar2(100),
Model varchar2(100),
Year varchar2(4),
Country varchar2(100)
);
CREATE OR REPLACE TYPE tt_Automobile AS TABLE OF ot_Automobile;
CREATE OR REPLACE PROCEDURE sp_InsertCars
(
p_CarList In tt_Automobile,
p_UserID In integer,
o_Cursor Out Sys_RefCursor
)
AS
BEGIN
DBMS_Output.Enable;
For RowItem In (Select * From Table(p_CarList))
Loop
Insert Into tblCars
(
Make,
Model,
Year,
Country,
UserID
)
Values(
RowItem.Make,
RowItem.Model,
RowItem.Year,
RowItem.Country,
p_UserID
);
End Loop;
-- Return our results after insert
Open o_Cursor For
Select Make, Model, Year, Country From tblCars Where UserID = p_UserID;
EXCEPTION
When Others Then
DBMS_Output.Put_Line('SQL Error: ' || SQLERRM);
END sp_InsertCars;
COMMIT
/
The result should allow me to pass my array Object from my WebService WebMethod over to my Oracle stored procedure and then loop through each item of the array to perform an Insert.
Here's an example of the data I'm trying to pass in.
this answer depends on commercial package, but if you're as desperate as i am, it's a lifesaver for very reasonable $300 (circa 2020-Q4)... scouts honor, i'm no shill
DevArt's Oracle provider makes elegant work of passing lists of objects to stored procs... it really works... .net core 3.1 compatible, tested on linux, no dependencies on native oracle client ... see my take on a working console app sample below based on the linked forum post
DevArt's "OracleType.GetObjectType()" API makes the UDT marshalling part of this incredibly trivial for us... way less to grok than the existing unmanaged ODP table type support samples i've seen out there for years
strategic consideration - if you already have a sizable code base on an oracle provider, consider just leaving all that code as-is and only take on regression testing this new dependency where the specialized table type support is actually needed
short and sweet sample for quick digestion
using System;
using System.Data;
using Devart.Data.Oracle;
namespace ConsoleApp1
{
class Program
{
private static int oraTable;
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
//good docs:
//direct connection: https://www.devart.com/dotconnect/oracle/docs/StoredProcedures-OracleCommand.html
//linux licensing: https://www.devart.com/dotconnect/oracle/docs/?LicensingStandard.html
using OracleConnection db = new OracleConnection("{insert yours}");
//devart trial licensing nutshell... on WINDOWS, download & run their installer...
//the only thing you really need from that install is the license key file dropped here:
// %programdata%\DevArt\License\Devart.Data.Oracle.key
// then just go nuget the "Devart.Data.Oracle" package reference
//on Windows, the trial license file gets automatically picked up by their runtime
//if you're on Linux, basically just read their good instructions out there
//i've just tested it on WSL so far and plan to try on Azure linux app svc within next day
//db.ConnectionString = "license key=trial:Devart.Data.Oracle.key;" + db.ConnectionString;
db.Direct = true; //nugget: crucial!! https://www.devart.com/dotconnect/oracle/docs/DirectMode.html
db.Open();
var cmd = db.CreateCommand("UserPermissions_u", CommandType.StoredProcedure);
cmd.DeriveParameters();
//tblParm.OracleDbType = OracleDbType.Table;
//passing "table" type proc parm example: https://forums.devart.com/viewtopic.php?t=22243
var obj = new OracleObject(OracleType.GetObjectType("UserPerm", db));
var tbl = new OracleTable(OracleType.GetObjectType("UserPerms", db));
obj["UserPermissionId"] = "sR1CKjKYSKvgU90GUgqq+w==";
obj["adv"] = 1;
tbl.Add(obj);
cmd.Parameters["IN_Email"].Value = "banderson#kingcounty.gov";
cmd.Parameters["IN_Permissions"].Value = tbl;
cmd.ExecuteNonQuery();
//"i can't believe it's not butter!" -me, just now =)
}
}
}
corresponding oracle db definitions:
create or replace type UserPerm as object ( UserPermissionId varchar2(24), std number(1), adv number(1) );
create or replace type UserPerms as table of UserPerm;
create or replace PROCEDURE UserPermissions_u (
IN_Email IN varchar2,
IN_Permissions IN UserPerms
) is
dummyvar number default 0;
begin
select count(*) into dummyvar from table(IN_Permissions);
end;
/
more elaborate shot at generically reflecting on an inbound object to hydrate the proc parms like the OP's request... take with caution, needs testing/bullet-proofing ... i'd love to get better alternatives if anybody cares to share
using System;
using System.Data;
using Devart.Data.Oracle;
using System.Linq;
using System.Collections.Generic;
using System.Text.RegularExpressions;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Collections;
namespace ConsoleApp1
{
public class User
{
public string Email { get; set; }
public List<UserPermissionEffective> Permissions { get; set; }
}
public class UserPermissionEffective
{
public string UserPermissionId { get; set; }
public string Email { get; set; }
public bool Std { get; set; }
public bool Adv { get; set; }
public string Mod { get; set; }
}
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
var dto = new User { Email = "testy#mctesterson.com", Permissions = new List<UserPermissionEffective> {
new UserPermissionEffective { UserPermissionId = "1", Std = false, Adv = true },
new UserPermissionEffective { UserPermissionId = "2", Std = true, Adv = false }
} };
if (dto == null) return;
//good docs:
//direct connection: https://www.devart.com/dotconnect/oracle/docs/StoredProcedures-OracleCommand.html
//linux licensing: https://www.devart.com/dotconnect/oracle/docs/?LicensingStandard.html
var dbstring = Environment.GetEnvironmentVariable("dbstring");
using OracleConnection db = new OracleConnection(dbstring);
db.ConnectionString = "license key=trial:Devart.Data.Oracle.key;" + db.ConnectionString;
db.Direct = true; //nugget: crucial!! https://www.devart.com/dotconnect/oracle/docs/DirectMode.html
db.Open();
var cmd = db.CreateCommand("UserPermissions_u", CommandType.StoredProcedure);
cmd.DeriveParameters();
//regex gets everything following the last underscore. e.g. INOUT_PARMNAME yields PARMNAME
var regex = new Regex(#"([^_\W]+)$", RegexOptions.Compiled);
//get the inboud model's root properties
var dtoProps = dto.GetType().GetProperties();
//loop over all parms assigning model properties values by name
//going by parms as the driver versus object properties to favor destination over source
//since we often ignore some superfluous inbound properties
foreach (OracleParameter parm in cmd.Parameters)
{
var cleanParmName = regex.Match(parm.ParameterName).Value.ToUpper();
var dtoPropInfo = dtoProps.FirstOrDefault(prop => prop.Name.ToUpper() == cleanParmName);
//if table type, then drill into the nested list
if (parm.OracleDbType == OracleDbType.Table)
{
//the type we're assigning from must be a list
//https://stackoverflow.com/questions/4115968/how-to-tell-whether-a-type-is-a-list-or-array-or-ienumerable-or/4115970#4115970
Assert.IsTrue(typeof(IEnumerable).IsAssignableFrom(dtoPropInfo.PropertyType));
var listProperty = (dtoPropInfo.GetValue(dto) as IEnumerable<Object>).ToArray();
//don't bother further logic if the list is empty
if (listProperty.Length == 0) return;
//get the oracle table & item Udt's to be instanced and hydrated from the inbound dto
var tableUdt = OracleType.GetObjectType(parm.ObjectTypeName, db);
var itemUdt = OracleType.GetObjectType(tableUdt.ItemObjectType.Name, db);
var dbList = new OracleTable(tableUdt);
//and the internal list item objects
var subPropInfos = dtoPropInfo.PropertyType.GenericTypeArguments[0].GetProperties().ToDictionary(i=>i.Name.ToUpper(), i=>i);
//for every item passed in...
foreach (var dtoSubItem in listProperty) {
//create db objects for every row of data we want to send
var dbObj = new OracleObject(itemUdt);
//and map the properties from the inbound dto sub items to oracle items by name
//using reflection to enumerate the properties by name
foreach (OracleAttribute field in itemUdt.Attributes)
{
var val = subPropInfos[field.Name.ToUpper()].GetValue(dtoSubItem);
//small tweak to map inbound booleans to 1's & 0's on the db since oracle doesn't support boolean!?!
var isDbBool = field.DbType == OracleDbType.Integer && field.Precision == 1;
dbObj[field] = isDbBool ? ((bool)val ? 1 : 0) : val;
}
//lastly add the db obj to the db table
dbList.Add(dbObj);
}
parm.Value = dbList;
}
else {
parm.Value = dtoPropInfo.GetValue(dto);
}
}
cmd.ExecuteNonQuery();
}
}
}
Please refer following link to setup ODAC
Setup Ref and use follwing link to get the ODAC
using Oracle.DataAccess.Client;
using Oracle.DataAccess.Types;
using System;
using System.Data;
namespace Strace_CustomTypes
{
class Program
{
static void Main(string[] args)
{
// Setup Ref - https://o7planning.org/en/10509/connecting-to-oracle-database-using-csharp-without-oracle-client
// ODAC 64bit ODAC122010Xcopy_x64.zip - https://www.oracle.com/technetwork/database/windows/downloads/index-090165.html
// .Net Framework 4
// 'Connection string' to connect directly to Oracle.
string connString = "Data Source=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=0.0.0.0)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=SIT)));Password=PASSWORD;User ID=USERID";
OracleConnection straceOracleDBConn = new OracleConnection(connString);
OracleCommand cmd = new OracleCommand("PKG_TEMP.TEST_ARRAY", straceOracleDBConn);
cmd.CommandType = CommandType.StoredProcedure;
try
{
straceOracleDBConn.Open();
CustomVarray pScanResult = new CustomVarray();
pScanResult.Array = new string[] { "hello", "world" };
OracleParameter param = new OracleParameter();
param.OracleDbType = OracleDbType.Array;
param.Direction = ParameterDirection.Input;
param.UdtTypeName = "USERID.VARCHAR2_ARRAY";
param.Value = pScanResult;
cmd.Parameters.Add(param);
cmd.ExecuteNonQuery();
}
catch (Exception ex)
{
Console.WriteLine($"Error: {ex.Message} {Environment.NewLine} {ex.StackTrace}");
}
finally
{
straceOracleDBConn.Close();
cmd.Dispose();
straceOracleDBConn.Dispose();
}
Console.WriteLine("Press any key to exit");
Console.ReadLine();
}
}
//Ref https://www.codeproject.com/Articles/33829/How-to-use-Oracle-11g-ODP-NET-UDT-in-an-Oracle-Sto
public class CustomVarray : IOracleCustomType, INullable
{
[OracleArrayMapping()]
public string[] Array;
private OracleUdtStatus[] m_statusArray;
public OracleUdtStatus[] StatusArray
{
get
{
return this.m_statusArray;
}
set
{
this.m_statusArray = value;
}
}
private bool m_bIsNull;
public bool IsNull
{
get
{
return m_bIsNull;
}
}
public static CustomVarray Null
{
get
{
CustomVarray obj = new CustomVarray();
obj.m_bIsNull = true;
return obj;
}
}
public void FromCustomObject(OracleConnection con, IntPtr pUdt)
{
OracleUdt.SetValue(con, pUdt, 0, Array, m_statusArray);
}
public void ToCustomObject(OracleConnection con, IntPtr pUdt)
{
object objectStatusArray = null;
Array = (string[])OracleUdt.GetValue(con, pUdt, 0, out objectStatusArray);
m_statusArray = (OracleUdtStatus[])objectStatusArray;
}
}
[OracleCustomTypeMapping("USERID.VARCHAR2_ARRAY")]
public class CustomVarrayFactory : IOracleArrayTypeFactory, IOracleCustomTypeFactory
{
public Array CreateArray(int numElems)
{
return new string[numElems];
}
public IOracleCustomType CreateObject()
{
return new CustomVarray();
}
public Array CreateStatusArray(int numElems)
{
return new OracleUdtStatus[numElems];
}
}
}

How to input into DataTable quickly? Or save data permanently into DataTable?

I am inputting a text file into a DataTable and then using SqlBulkCopy to copy to a Database. While BulkCopy is fast, inserting 50000+ lines into DataTable is not (around 5 mins). How do I make it efficient?
Can I insert data into the DataTable quickly?
If not, is there a way to save the inserted data permanently into the DataTable so I don't have to insert it every time I run the program?
for (; i < fares.Length; )
{
k = i;
Console.WriteLine("Inserting " + k + " out of " + (fares.Length));
for (; i <= (k + 3); i++)
{
if (i % 4 == 0)
{
for (int j = 0; j < fares.Length - 1; j++)
{
{
int space = fares[i].IndexOf(" ");
startStation = fares[i].Substring(0, space);
endStation = fares[i].Substring(space + 1, fares[i].Length - space - 1);
}
}
}
else if (i % 4 == 1)
{
valueFare = fares[i];
}
else if (i % 4 == 2)
{
standardFare = fares[i];
}
else if (i % 4 == 3)
{
time = int.Parse(fares[i]);
}
}
faresDT.Rows.Add(startStation, endStation, valueFare, standardFare, time);
If what you want is to optimize your load to the database, I suggest that you get rid of the DataTable completely. By making use of Marc Gravell's FastMember (and anyone who's using SqlBulkCopy should be using FastMember IMHO) you can get a DataReader directly from any IEnumerable.
I would use some variation of the below code whenever writing from a file directly to a database. The below code will stream the contents of the file directly to the SqlBulkCopy operation thru the clever use of yield returns and lazy load of IEnumerable.
using System;
using System.Collections.Generic;
using System.Data.SqlClient;
using System.IO;
using System.Text;
using FastMember;
namespace BulkCopyTest
{
public class Program
{
public static void Main(string[] args)
{
const string filePath = "SOME FILE THAT YOU WANT TO LOAD TO A DB";
WriteData(GetData<dynamic>(filePath));
}
private static void WriteData<T>(IEnumerable<T> data)
{
using (var bcp = new SqlBulkCopy(GetConnection(), SqlBulkCopyOptions.TableLock, null))
using (var reader = ObjectReader.Create(data))
{
SetColumnMappings<T>(bcp.ColumnMappings);
bcp.BulkCopyTimeout = 300;
bcp.BatchSize = 150000;
bcp.DestinationTableName = ""; //TODO: Set correct TableName
bcp.WriteToServer(reader);
}
}
private static void SetColumnMappings<T>(SqlBulkCopyColumnMappingCollection mappings)
{
//Setup your column mappings
}
private static IEnumerable<T> GetData<T>(string filePath)
{
using (var fileStream = File.OpenRead(filePath))
using (var reader = new StreamReader(fileStream, Encoding.UTF8))
{
string line;
while ((line = reader.ReadLine()) != null)
{
//TODO: Add actual parsing logic and whatever else is needed to create an instance of T
yield return Activator.CreateInstance<T>();
}
}
}
private static SqlConnection GetConnection()
{
return new SqlConnection(new SqlConnectionStringBuilder
{
//TODO: Set Connection information here
}.ConnectionString);
}
}
}
In this case I think you should take advantage of the BeginLoadData, LoadDataRow and EndLoadData methods provided in the DataTable class, you could use them like this:
try
{
faresDT.BeginLoadData();
// Your for loop...
{
// Logic defining the value of startStation, endStation, valueFare, standardFare and time removed for briefness.
faresDT.LoadDataRow(new object[] {startStation, endStation, valueFare, standardFare, time}, true);
}
}
finally
{
faresDT.EndLoadData();
}
What BeginLoadData() does is turning off some processing that happens every time you add a row, and only does it once when you are done loading data by calling EndLoadData().
You can find more details about these APIs here:
https://learn.microsoft.com/en-us/dotnet/api/system.data.datatable.loaddatarow?view=netframework-4.7.2

SQL, all column names are in brackets, C#

I am getting some SQL statement that user inputs on UI:
select * from [table] where [col1] is not null and col2 <> 0
I need to verify that all column names (col1 and col2) are in brackets (like col1). And show some popup message if any column name is not in brackets (in this case, col2).
Is there some way to do such SQL verification in C#?
Regarding parsing SQL, you can use the TSqlParser library that ships with Visual Studio. This parser accepts a visitor class that inherits from TSqlFragmentVisitor and overrides the visit method for ColumnReferenceExpression.
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using Microsoft.SqlServer.TransactSql.ScriptDom;
namespace ConsoleApplication8
{
public class QueryParser
{
public IEnumerable<string> Parse(string sqlSelect)
{
TSql100Parser parser = new TSql100Parser(false);
TextReader rd = new StringReader(sqlSelect);
IList<ParseError> errors;
var columns = new List<string>();
var fragments = parser.Parse(rd, out errors);
var columnVisitor = new SQLVisitor();
fragments.Accept(columnVisitor);
columns = new List<string>(columnVisitor.Columns);
return columns;
}
}
internal class SQLVisitor : TSqlFragmentVisitor
{
private List<string> columns = new List<string>();
private string GetNodeTokenText(TSqlFragment fragment)
{
StringBuilder tokenText = new StringBuilder();
for (int counter = fragment.FirstTokenIndex; counter <= fragment.LastTokenIndex; counter++)
{
tokenText.Append(fragment.ScriptTokenStream[counter].Text);
}
return tokenText.ToString();
}
public override void ExplicitVisit(ColumnReferenceExpression node)
{
columns.Add(GetNodeTokenText(node));
}
public IEnumerable<string> Columns {
get { return columns; }
}
}
public class Program
{
private static void Main(string[] args)
{
QueryParser queryParser = new QueryParser();
var columns = queryParser.Parse("SELECT A,[B],C,[D],E FROM T WHERE isnumeric(col3) = 1 Order by Id desc");
foreach (var column in columns)
{
Console.WriteLine(column);
}
}
}
}
However, I am not sure that parsing SQL that an user inputs in UI is a good idea. Even if you run the SQL statements in a sand boxed environment where the user only has read permissions (I do hope you are doing this), it is not very user friendly. As a user I would prefer to be able to select the columns I want using check boxes or by dragging and dropping them rather than having to write a SQL query.

Getting an int from a DataTable DataRow when strings normally appear

I have searched extensively for an answer here and elsewhere but have had little success.
I am using OLEDB to connect to a spreadsheet and extract data to port to a database, and all is going fairly smoothly except one minor detail.
There is a column called Skills where the entry I receive is usually a string, however the value can sometimes be an int like 334423, when I try to add this value to the new database, the entry is simply blank.
Usual approach to extracting:
foreach(DataRow dr in dt.Rows)
{
/* dealing with other columns */
sTemp.skill = dr[3].ToString();
/* more stuff */
}
But I get absolutely no response when the column in this row is an int, just a blank string. It doesn't even throw any exceptions, which would at least give me some direction.
I also tried casting it as an int:
sTemp.skill = (int)dr[3].ToString();
(which didn't work)
FYI: "sTemp" is an object with strings associated with it, skill being a string. The mysteriously quiet nature of this problem also makes it hard for me to know what I should be asking. Any OLEDB gurus out there, please help! Thanks
EDIT: After some more investigation, it seems that the problem was not with conversion/casting of strings to ints, but actually getting the data in the first place.
In the instance of a field having a number, it will totally ignore it (for reasons I cannot think of), so there is nothing to parse. Mysterious behaviour indeed.
Unless anyone knows of known problems getting ints from excel using an OleDbDataAdapter, then feel free to answer... Sorry to have wasted your time all, how embarrassing.
EDIT #2: Here is an updated code view:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Data.OleDb;
namespace SQEPSkillsImporter
{
public partial class MainWindow : Form
{
string skillsFileName = "";
List<SkillTemplate> TemplateList = new List<SkillTemplate>();
public MainWindow()
{
InitializeComponent();
}
private void skillsFilePathBtn_Click(object sender, EventArgs e)
{
this.skillsFileDialog.Filter = "Excel Worksheets|*.xls;*xlsx";
DialogResult result = skillsFileDialog.ShowDialog();
if (result == DialogResult.OK)
{
this.skillsFileName = skillsFileDialog.FileName;
skillsFilePathTxBx.Text = skillsFileName;
}
}
private void readSkillsBtn_Click(object sender, EventArgs e)
{
DataTable skillsTable = new DataTable("SkillsResults");
string connectionString = "Provider=Microsoft.Jet.OleDb.4.0;" +
" Data Source=" + this.skillsFileName +
";Extended Properties=Excel 8.0;";
using(OleDbConnection Connection = new OleDbConnection(connectionString))
{
Connection.Open();
using(OleDbCommand Command = new OleDbCommand())
{
Command.Connection = Connection;
/* The columns I want are in A through D */
Command.CommandText = "SELECT * FROM [Skills$A:D]";
using (OleDbDataAdapter Adapter = new OleDbDataAdapter())
{
Adapter.SelectCommand = Command;
Adapter.Fill(skillsTable);
}
}
Connection.Close();
Connection.Dispose();
}
SkillTemplate sTemp = new SkillTemplate();
foreach (DataRow dr in skillsTable.Rows)
{
/* most sGroup, group and heading columns are blank, so remain the same unless changed, meaning the name wont be null */
if(dr[0].ToString() != "")
sTemp.SuperGroup = dr[0].ToString();
if (dr[1].ToString() != "")
sTemp.Group = dr[1].ToString();
if (dr[2].ToString() != "")
sTemp.Heading = dr[2].ToString();
if (dr[3].ToString() != "")
sTemp.Skill = dr[3].ToString();
/* VERY rough test textbox to verify that my output is correct before sending to DB */
this.skillsDumpTxBx.Text += sTemp.SuperGroup + " /// " + sTemp.Group + " /// " + sTemp.Heading + " /// " + sTemp.Skill + System.Environment.NewLine;
}
}
}
public class SkillTemplate
{
public string SuperGroup { get; set; }
public string Group { get; set; }
public string Heading { get; set; }
public string Skill { get; set; }
}
}
sTemp.skill = Convert.ToInt32(dr[3].ToString());
or use this method to convert to any type:
/// <summary>
/// Safe converting to any type
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="value">Value to convert</param>
/// <param name="defaultValue">Default value, sets type of return value</param>
/// <returns>Converted value or defaul when error</returns>
public static T safeConvert<T>(object value, T defaultValue)
{
try
{
return value == null ? default(T) :
(T)Convert.ChangeType( value, Nullable.GetUnderlyingType(typeof(T)) ?? typeof(T) );
}
catch
{
return defaultValue;
}
}
usage:
sTemp.skill = safeConvert(dr[3],0);
// or
String s = safeConvert(obj, string.Empty);
If dr[3].ToString() is returning you blank then possibilities are it contains DBNull or blank string. Did you check your query to make sure your 4 th column (zero based index 3) is returning a valid value

Categories