I have two separate queries/methods and functions so that I can pass two strings in my stored procedures which basically look like this:
1234,12346,12690812,1259081 => UPCList parameter
1234,12346,12690812,1259081 => EANList parameter
Now the first method looks like following:
create procedure [dbo].[zsp_selectallupceans_list]
(
#UPCList nvarchar(4000),
#EANList nvarchar(4000),
#Type tinyint
)
as
select DISTINCT dd.UPC,dd.EAN,dd.EBAYID as ItemID
from ThirdPartyData as dd
where dd.UPC in (SELECT * FROM dbo.SplitString(#UPCList)) OR
dd.EAN in (SELECT * FROM dbo.SplitString(#EANList)) and dd.Type=#Type
Combined with splitstring function created in MSSQL:
create FUNCTION [dbo].[splitstring] ( #stringToSplit VARCHAR(MAX) )
RETURNS
#returnList TABLE ([Name] [nvarchar] (500))
AS
BEGIN
DECLARE #name NVARCHAR(255)
DECLARE #pos INT
WHILE CHARINDEX(',', #stringToSplit) > 0
BEGIN
SELECT #pos = CHARINDEX(',', #stringToSplit)
SELECT #name = SUBSTRING(#stringToSplit, 1, #pos-1)
INSERT INTO #returnList
SELECT #name
SELECT #stringToSplit = SUBSTRING(#stringToSplit, #pos+1, LEN(#stringToSplit)-#pos)
END
INSERT INTO #returnList
SELECT #stringToSplit
RETURN
END
And now the second method:
create procedure [dbo].[zsp_selectallupceans_listProduction]
(
#UPCList nvarchar(4000),
#EANList nvarchar(4000),
#Type tinyint
)
as
SELECT dd.UPC,dd.EAN,dd.EBAYID as ItemID
from ThirdPartyData as dd
WHERE EXISTS (SELECT 1 FROM dbo.SplitStringProduction(#UPCList,',') S1 WHERE dd.UPC=S1.val)
OR EXISTS (SELECT 1 FROM dbo.SplitStringProduction(#EANList,',') S2 WHERE dd.EAN=S2.val) and dd.Type=#Type
And now the second method splitstringProduction:
create FUNCTION [dbo].[SplitStringProduction]
(
#string nvarchar(max),
#delimiter nvarchar(5)
) RETURNS #t TABLE
(
val nvarchar(500)
)
AS
BEGIN
declare #xml xml
set #xml = N'<root><r>' + replace(#string,#delimiter,'</r><r>') + '</r></root>'
insert into #t(val)
select
r.value('.','varchar(500)') as item
from #xml.nodes('//root/r') as records(r)
RETURN
END
And now here comes the weird part... Benchmark results of both methods come up like this:
var findEans = ctx.zsp_selectallupceans_list(formedList.UPCList, formedList.EANList, 1).ToList();
//Execution time 4.8 seconds
var findEans = ctx.zsp_selectallupceans_listProduction(formedList.UPCList, formedList.EANList, 1).ToList();
//Execution time: 15 seconds
As far as my understanding goes, the second method should perform much faster than first... But this is not the case... Not only is the performance for the 2nd method worse, it is more than 3 times execution time than the first... And I cannot understand why ...
What is going on here guys? What would be the best method to reduce the execution time to <1 second?
Can someone help me out here please?
Related
I am trying to update a table using table type parameter. Currently I am using the below query. But I would like to perform update in batches having batch size as one of the parameters. Kindly help.
ALTER PROCEDURE UPDATEStatus
#Ids int ,
#numbers TypeofNumbers readonly,
#Status char(2),
#nname varchar(50),
AS
BEGIN
BEGIN TRY
update e
set
e.status = #Status,
e.user =#nname,
e.time = GETDATE()
from detailtable e
join #numbers en on en.ID =e.ID
where e.oddIDs = #Ids
I tried to do in a single update but I wanted to do in sets or batches one by one. say 100 records first and then next 100 records until all are done
You can use something like this to do your update in batches:
CREATE OR ALTER PROCEDURE UPDATEStatus
#Ids INT,
#numbers TypeOfNumbers READONLY,
#Status CHAR(2),
#nname VARCHAR(50)
AS
BEGIN
DECLARE #UpdatedRows INT;
DECLARE #Skip INT = 0;
DECLARE #BatchSize INT = 100;
WHILE ISNULL(#UpdatedRows, 1) > 0
BEGIN
WITH CTE
AS (SELECT *
FROM #numbers AS n
ORDER BY n.ID OFFSET #Skip * #BatchSize ROWS FETCH NEXT #BatchSize ROWS ONLY)
UPDATE e
SET
e.[Status] = #Status,
e.[User] = #nname,
e.[time] = GETDATE()
FROM CTE AS en
JOIN detailtable e ON en.ID = e.ID;
SET #UpdatedRows = ##ROWCOUNT;
SET #Skip = #Skip + 1;
END;
END;
GO
Next time please also provide scripts for the DDL and some testdata.
I have basic procedure which basically looks like following:
create procedure zsp_selectallupceans_list
(#UPCList nvarchar(4000),
#EANList nvarchar(4000))
as
select *
from data as dd
where dd.UPC in (--myUPC list) or dd.EAN in (--myEAN list)
This is the basic idea. Now I need to somehow split this string that I passed from my C# application and it would look like following for the UPC and EAN List:
where dd.UPC in ('123','456','567') or dd.EAN in('1234','5542','412')
The UPCList parameter that is passed from C# application looks like:
'123,456,567' and eanlist: '1234,5542,412'
I have found a method which looks like this:
CREATE FUNCTION dbo.splitstring
(#stringToSplit VARCHAR(MAX))
RETURNS
#returnList TABLE ([Name] [NVARCHAR](500))
AS
BEGIN
DECLARE #name NVARCHAR(255)
DECLARE #pos INT
WHILE CHARINDEX(',', #stringToSplit) > 0
BEGIN
SELECT #pos = CHARINDEX(',', #stringToSplit)
SELECT #name = SUBSTRING(#stringToSplit, 1, #pos-1)
INSERT INTO #returnList
SELECT #name
SELECT #stringToSplit = SUBSTRING(#stringToSplit, #pos+1, LEN(#stringToSplit)-#pos)
END
INSERT INTO #returnList
SELECT #stringToSplit
RETURN
END
And the usage of this function is like following:
SELECT * FROM dbo.splitstring('91,12,65,78,56,789')
where the output is these numbers where they are split and output as a result.
Now I just need to somehow combine all this so that I can form a proper where statement based on passed parameter UPCList and EANList
Can someone help me out with this?
Updating your stored proc as below should do the trick:
create procedure zsp_selectallupceans_list
(
#UPCList nvarchar(4000),
#EANList nvarchar(4000)
)
as
select *
from data as dd
where dd.UPC in (SELECT * FROM dbo.SplitString(#UPCList)) OR
dd.EAN in (SELECT * FROM dbo.SplitString(#EANList))
You pretty much have the answer:
Compile and save the splitstring function and then your where clause will look like the following:
where dd.UPC in (Select Name From splitstring(--myUpcList)) or dd.EAN in (Select Name from splitstring(--myEanList)
Here is an XML based function for string splitting, this method is much faster than the SUBSTRING method you already found. It is also recommended to use EXISTS instead of IN for performance improvement also, see here for more information on this.
CREATE FUNCTION [dbo].[SplitString]
(
#string nvarchar(max),
#delimiter nvarchar(5)
) RETURNS #t TABLE
(
val nvarchar(500)
)
AS
BEGIN
declare #xml xml
set #xml = N'<root><r>' + replace(#string,#delimiter,'</r><r>') + '</r></root>'
insert into #t(val)
select
r.value('.','varchar(500)') as item
from #xml.nodes('//root/r') as records(r)
RETURN
END
To use:
SELECT *
FROM data t
WHERE EXISTS (SELECT 1 FROM dbo.SplitString(#UPCList,',') S1 WHERE t.UPC=S1.val)
OR EXISTS (SELECT 1 FROM dbo.SplitString(#EANList,',') S2 WHERE t.EAN=S2.val)
I have a c# web page and i am using SQL for the database
i need to display the data from table into a gridview
SQL : I have a table as below
Exec_Date Plat Pass Fail
----------------------------------------------------------------
2017-02-19 12:32:43.570 MSSQL 10 12
2017-02-19 12:32:43.570 MSSQL 10 12
2017-02-18 12:32:43.570 Sybase 10 12
2017-02-18 12:32:43.570 Oracle 10 12
i would like to convert the table into a customised format for a presentation
Status 18/02 19/02
-------------------------
Pass 20 20
Fail 24 24
is the above possible to be done by Pivot ? if so can anyone give some idea on that pls?
I have tried the below so far and the #Names is not resolving the variable value in query
create table #TempTable
(Exec_Date varchar(max),
Pass int,
Ref_Status varchar(max)
)
Insert into #TempTable
select cast(Exec_Date as DATE) as Date,SUM(Pass) as Pass,'PASS' from F_Exec where cast(Exec_Date as DATE) >= '2017-02-17' and cast(Exec_Date as DATE) <= '2017-02-20' group by Exec_Date,Pass
DECLARE #Names varchar(max)
SELECT #Names = COALESCE(#Names + '],[', '[') + Exec_Date FROM #TempTable
select #Names=#Names+']'
Select * from #TempTable pivot
(SUM(Pass) for Exec_Date IN (#Names) )as PivotTable
The variable #Names is not considered as variable. If i manually replace that resultant value from the variable it works
Try this code:
declare #tbl table (Exec_Date datetime, Plat varchar(max), Pass int, Fail int)
insert into #tbl(Exec_Date , Plat , Pass , Fail ) values
('2017-02-19 12:32:43.570', 'MSSQL',10,12),
('2017-02-19 12:32:43.570', 'MSSQL',10,12),
('2017-02-18 12:32:43.570', 'Sybase',10,12),
('2017-02-18 12:32:43.570', 'Oracle',10,12)
select * from
(
select Date,Status,sum(Success) success from
(
select left(convert(varchar(20),Exec_Date,103),5) as [Date],Pass,Fail from #tbl
) as baseData
unpivot
(
Success for Status in (Pass,Fail)
) as tblUnPivot
group by date ,status
) as baseData
pivot
(
sum(success)
for Date in ([18/02],[19/02])
) as tblPivot
order by Status
I got the solution
declared nvarchar variable and concatenated the query & dynamic column names
DECLARE #sqlstring nvarchar(MAX)
SET #sqlstring =N'
Select * from #TempTable pivot (SUM(Pass) for Exec_Date IN (' +#Names+') )as PivotTable;';
EXEC sp_executesql #sqlstring;
I've had a search round the internet and cannot find anything that directly matches what I'm after....
I have a stored procedure that returns a fixed set of columns (let's say ColumnA, ColumnB and ColumnC) and then additionally returns an indeterminate number / signature of additional columns (maybe ColumnD, ColumnE and ColumnF, or ColumnF, ColumnP, ColumnT and ColumnW).
The dynamic part of the query is determined by an Id param to the proc.
Unfortunately I am not in control of the schema and it has been poorly designed to have a very large number of unrelated columns added to the end (and they keep adding them), rather than having an additional table(s) to store this effectively.
As a result, I am unable to return a determinate set of columns at compile time, so LINQ to SQL cannot determine the return type of the stored procedure.
Do I have any other options, or ways I can get around this? Here's an example of what I am doing below. Can I re-write this to work with LINQ to SQL? I am writing the application with C# and SQL Server 2008 if that helps.
Thanks.
CREATE PROCEDURE [Foo].[GetBar]
(
#Id INT,
#FooVar VARCHAR(50),
#BarVar DATE
)
AS
BEGIN
CREATE TABLE #TempTbl
(
[ColumnName] VARCHAR(50)
)
INSERT #TempTbl EXEC [Foo].GetColumnNames #Id
DECLARE #ColumnNameList NVARCHAR(max)
SET #ColumnNameList = ''
SELECT #ColumnNameList = COALESCE(#ColumnNameList + '],[', '') + ColumnName FROM #TempTbl
DROP TABLE #TempTbl
SELECT #ColumnNameList = #ColumnNameList + ']'
SELECT #ColumnNameList = SUBSTRING(#ColumnNameList, 3, LEN(#ColumnNameList) )
DECLARE #FixedColumns VARCHAR(200)
DECLARE #SelectQuery NVARCHAR(max)
DECLARE #WhereQuery VARCHAR (100)
DECLARE #FullQuery NVARCHAR(max)
DECLARE #ParamaterDef nvarchar (100)
DECLARE #Foo VARCHAR(50)
DECLARE #Bar DATE
SET #FixedColumns = N'[ColumnA], [ColumnB], [ColumnC], '
SET #SelectQuery = N'SELECT ' + #FixedColumns + #ColumnNameList + N' FROM [Foo].[FooBar] '
SET #WhereQuery = N'WHERE [Foo] = #Foo AND [Bar] = #Bar'
SET #FullQuery = #SelectQuery + #WhereQuery
SET #ParamaterDef = N'#Foo VARCHAR(50), #Bar DATE'
EXECUTE sp_executesql #FullQuery, #ParamaterDef, #Foo = #FooVar, #Bar = #BarVar
END
Can you not just use a SQL Data Adapter that fills a DataSet? You could then use LINQ to DataSet and perform all of your LINQ operations:
DataTable yourDataTable = ds.Tables["TableName"];
var query =
from yourDataTable in yourDataTable.AsEnumerable()
select new
{
NewField = yourDataTable.Field<int>("ColumnName"),
NewField2 = yourDataTable.Field<string>("ColumnName2"),
};
MSDN (LINQ to DataSet)
MSDN (Single table queries using LINQ to DataSet
I need to write a query in t-sql or linq that matches db records that contain the most of the user input words.
exmaple:
nvarchar field in db: "The quick brown fox jumps over the lazy dog"
User input: "brown cow"
The program would match that record because it has the word brown.
let me know if i need to provide more examples.
Assuming you're using T-SQL in a MS SQL Server environment, then you should use Full Text Search technology. It gives you the speed and keeps you away from reinventing the wheel.
We generally use a UDF to split a string into tabular data and can use like command for the same.
declare #searchStr nvarchar(100)
set #searchStr = 'brown fox'
selecT T.* from test T, dbo.fnc_SplitSTring(#searchStr,' ')
where T.name like '%' + token + '%'
CREATE FUNCTION [dbo].[fnc_SplitString]
(
#InString varchar(8000),
#Delim char(1)
)
RETURNS #Return table
(
Position int identity,
Token varchar(100) -- Maximum token size is 100 chars...
)
As
BEGIN
Declare #CR varchar(1),
#LF varchar(1)
Set #CR = char(10)
Set #LF = char(13)
--
If #InString is null return
--
Declare #Pos int
Declare #Pattern char(3)
Set #Pattern = '%' + #Delim + '%'
--
Declare #Token varchar(30)
SELECT #InString = #InString + #Delim -- add trailing delimiter
SELECT #Pos = PATINDEX(#Pattern, #InString)
WHILE (#Pos <> 0) BEGIN
SELECT #Token = ltrim(rtrim(SUBSTRING(#InString, 1, #Pos - 1)))
Select #Token = replace(#Token, #CR, '')
Select #Token = replace(#Token, #LF, '')
Insert #Return Values (#Token)
SELECT #InString = STUFF(#InString, 1, PATINDEX(#Pattern, #InString),'')
SELECT #Pos = PATINDEX(#Pattern, #InString)
END
--
return
--
END