I have an Oracle 19c database that I am trying to pull data from using a package procedure. It is working, but I am new to Oracle, previously very experienced in Microsoft SQL Server. The C# code I have below works and calls my stored procedure successfully. However, the stored procedure returns over one million rows. I do not want to have a DataSet filled with over a million rows because obviously this is very slow. I would like to return is a subset, like offset X rows and take N rows. Basically I want to do something like this:
SELECT * FROM STORED_PROCEDURE OFFSET 50 ROWS FETCH NEXT 50 ROWS ONLY
But I want to do it using my package procedure. Here is my C# code:
public async Task<List<DbModels.DocumentWipList>> GetWipDocumentsAsync(string sort = "limited_dodiss ASC")
{
using (var connection = new OracleConnection(_configuration.GetConnectionString("OracleDev")))
{
using (var command = connection.CreateCommand())
{
connection.Open();
command.CommandType = CommandType.StoredProcedure;
command.CommandText = "PKG_GET_COMPONENT_DETAIL.pr_get_wip_comp_list_sorted";
command.Parameters.Add("arg_sort", OracleDbType.Varchar2).Value = sort;
command.Parameters.Add("io_cursor", OracleDbType.RefCursor).Direction = ParameterDirection.Output;
using (var da = new OracleDataAdapter())
{
da.SelectCommand = command;
var dt = new DataTable();
await Task.Run(() => da.Fill(dt));
return MapDocumentWipList(dt);
}
}
}
}
It should be noted that I cannot modify the package procedure. I am hoping there is an easy way to do this, perhaps by somehow wrapping the package procedure as a subquery for a SELECT query.
In Oracle, a cursor is effectively a pointer to a memory address on the database server where the database stores the query the cursor executes and the current execution state of the cursor (a cursor never stores the result set waiting to be read, that is effectively generated on the fly as each row is read). You can read from a cursor once but you cannot change the cursor or rewind it.
I would like to return is a subset, like offset X rows and take N rows.
Don't use await Task.Run(() => da.Fill(dt));. Instead, read and ignore X rows for the cursor and then read and store N rows.
However, it would be better to change the procedure to allow pagination.
What about filtering? One of the columns that comes back is OWNER_NAME. What if I wanted to pull just the rows WHERE OWNER_NAME LIKE 'R%' or something like that?
It is impossible to modify a cursor, if you have to read it then you will need to read ALL the rows for the cursor and discard the rows that do not match your condition. So again, don't use await Task.Run(() => da.Fill(dt));, which would load all the rows into memory, instead read the rows one-by-one and only keep the ones you want in memory and forget the rest.
You can write a second procedure in PL/SQL to wrap around the cursor or you can do the processing in a third-party application like C# but you will need to read the cursor. All that changes between doing it in PL/SQL or C# is whether the processing occurs on the database server or on the third-party application server.
For example, if you have the table:
CREATE TABLE table_name (a, b, c) AS
SELECT LEVEL, CHR(64 + LEVEL), DATE '1970-01-01' + LEVEL - 1
FROM DUAL
CONNECT BY LEVEL <= 10;
And an existing procedure (that cannot be changed):
CREATE PROCEDURE get_cursor (
o_cursor OUT SYS_REFCURSOR
)
IS
BEGIN
OPEN o_cursor FOR
SELECT * FROM table_name;
END;
/
Then you can create the types:
CREATE TYPE table_name_type IS OBJECT(
a NUMBER,
b VARCHAR2(1),
c DATE
);
CREATE TYPE table_name_array IS TABLE OF table_name_type;
Which allows you to create a pipelined function:
CREATE FUNCTION wrap_cursor_fn (
i_cursor IN SYS_REFCURSOR
) RETURN table_name_array PIPELINED
IS
v_a table_name.a%TYPE;
v_b table_name.b%TYPE;
v_c table_name.c%TYPE;
BEGIN
LOOP
FETCH i_cursor INTO v_a, v_b, v_c;
EXIT WHEN i_cursor%NOTFOUND;
PIPE ROW (table_name_type(v_a, v_b, v_c));
END LOOP;
CLOSE i_cursor;
EXCEPTION
WHEN NO_DATA_NEEDED THEN
CLOSE i_cursor;
WHEN OTHERS THEN
CLOSE i_cursor;
RAISE;
END;
/
Which then allows you to use the returned pipelined collection in an SQL statement and read one cursor into another cursor and apply filters to it:
CREATE PROCEDURE wrap_cursor_proc (
i_cursor IN SYS_REFCURSOR,
o_cursor OUT SYS_REFCURSOR
)
IS
BEGIN
OPEN o_cursor FOR
SELECT *
FROM TABLE(wrap_cursor_fn(i_cursor))
WHERE MOD(a, 3) = 0;
END;
/
Which you can then read:
DECLARE
v_cur1 SYS_REFCURSOR;
v_cur2 SYS_REFCURSOR;
v_a table_name.a%TYPE;
v_b table_name.b%TYPE;
v_c table_name.c%TYPE;
BEGIN
get_cursor(v_cur1);
wrap_cursor_proc(v_cur1, v_cur2);
LOOP
FETCH v_cur2 INTO v_a, v_b, v_c;
EXIT WHEN v_cur2%NOTFOUND;
DBMS_OUTPUT.PUT_LINE( v_a || ', ' || v_b || ', ' || v_c );
END LOOP;
CLOSE v_cur2;
END;
/
and outputs:
3, C, 03-JAN-70
6, F, 06-JAN-70
9, I, 09-JAN-70
fiddle
It is a lot of work to read a cursor within the database and get it back into a format that can be used in an SQL query so that you can wrap it in another cursor just to apply a filter. It would be much simpler to just duplicate the original procedure and add the necessary filter condition to it.
Related
my Code is working, the function gives me the correct Select count (*) value but anyway, it throws an ORA-25191 Exception - Cannot reference overflow table of an index-organized table tips,
at retVal = Convert.ToInt32(cmd.ExecuteScalar());
Since I use the function very often, the exceptions slow down my program tremendously.
private int getSelectCountQueryOracle(string Sqlquery)
{
try
{
int retVal = 0;
using (DataTable dataCount = new DataTable())
{
using (OracleCommand cmd = new OracleCommand(Sqlquery))
{
cmd.CommandType = CommandType.Text;
cmd.Connection = oraCon;
using (OracleDataAdapter dataAdapter = new OracleDataAdapter())
{
retVal = Convert.ToInt32(cmd.ExecuteScalar());
}
}
}
return retVal;
}
catch (Exception ex)
{
exceptionProtocol("Count Function", ex.ToString());
return 1;
}
}
This function is called in a foreach loop
// function call in foreach loop which goes through tablenames
foreach (DataRow row in dataTbl.Rows)
{...
tableNameFromRow = row["TABLE_NAME"].ToString();
tableRows=getSelectCountQueryOracle("select count(*) as 'count' from " +tableNameFromRow);
tableColumns = getSelectCountQueryOracle("SELECT COUNT(*) as 'count' FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name='" + tableNameFromRow + "'");
...}
dataTbl.rows in this outer loop, in turn, comes from the query
SELECT * FROM USER_TABLES ORDER BY TABLE_NAME
If you're using a database-agnostic API like ADO.Net, you would almost always want to use the API's framework to fetch metadata rather than writing custom queries against each database's metadata tables. The various ADO.Net providers are much more likely to write data dictionary queries that handle all the various corner cases and are much more likely to be optimized than the queries you're likely to write. So rather than writing your own query to populate the dataTbl data table, you'd want to use the GetSchema method
DataTable dataTbl = connection.GetSchema("Tables");
If you want to keep your custom-coded data dictionary query for some reason, you'd need to filter out the IOT overflow tables since you can't query those directly.
select *
from user_tables
where iot_type IS NULL
or iot_type != 'IOT_OVERFLOW'
Be aware, however, that there are likely to be other tables that you don't want to try to get a count from. For example, the dropped column indicates whether a table has been dropped-- presumably, you don't want to count the number of rows in an object in the recycle bin. So you'd want a dropped = 'NO' predicate as well. And you can't do a count(*) on a nested table so you'd want to have a nested = 'NO' predicate as well if your schema happens to contain nested tables. There are probably other corner cases depending on the exact set of features your particular schema makes use of that the developers of the provider have added code for that you'd have to deal with.
So I'd start with
select *
from user_tables
where ( iot_type IS NULL
or iot_type != 'IOT_OVERFLOW')
and dropped = 'NO'
and nested = 'NO'
but know that you'll probably need/ want to add some additional filters depending on the specific features users make use of. I'd certainly much rather let the fine folks that develop the ADO.Net provider worry about all those corner cases than to deal with finding all of them myself.
Taking a step back, though, I'd question why you're regularly doing a count(*) on every table in a schema and why you need an exact answer. In most cases where you're doing counts, you're either doing a one-off where you don't much care how long it takes (i.e. a validation step after a migration) or approximate counts would be sufficient (i.e. getting a list of the biggest tables in the system in order to triage some effort or to track growth over time for projections) in which case you could just use the counts that are already stored in the data dictionary- user_tables.num_rows- from the last time that statistics were run.
This article helped me to solve my problem.
I've changed my query to this:
SELECT * FROM user_tables
WHERE iot_type IS NULL OR iot_type != 'IOT_OVERFLOW'
ORDER BY TABLE_NAME
I have a C# method which takes a SQL string statement and saves the data into xml format.
public XmlDocument GetDBRequestXml(String sql)
{
}
I have a stored procedure with output parameter as table. Is there any way to pass this stored procedure as an executable single SQL statement in the above C# method? Can somebody please help me on this!!!
create or replace PACKAGE BODY EMPLOYEE_DETAILS AS
PROCEDURE GET_EMPLOYEES(
EMP_DEPT_ID EMPLOYEES.DEPARTMENT_ID%TYPE,
EMP_SALARY employees.salary%TYPE,
TBL_EMPLOYEES OUT TABLE_EMPLOYEES)
IS
LC_SELECT SYS_REFCURSOR;
LR_DETAILS DETAILS;
TBL_EMPLOYEE EMPLOYEE_DETAILS.TABLE_EMPLOYEES := EMPLOYEE_DETAILS.TABLE_EMPLOYEES();
BEGIN
OPEN LC_SELECT FOR
SELECT EMPLOYEE_ID, FIRST_NAME, LAST_NAME
FROM EMPLOYEES
WHERE DEPARTMENT_ID = EMP_DEPT_ID AND
EMPLOYEES.SALARY > EMP_SALARY;
LOOP
FETCH LC_SELECT INTO LR_DETAILS;
EXIT WHEN LC_SELECT%NOTFOUND;
IF LR_DETAILS.EMPLOYEE_ID > 114 THEN
TBL_EMPLOYEE.extend();
TBL_EMPLOYEE(TBL_EMPLOYEE.count()) := LR_DETAILS;
END IF;
END LOOP;
CLOSE LC_SELECT;
TBL_EMPLOYEES := TBL_EMPLOYEE;
END GET_EMPLOYEES;
END EMPLOYEE_DETAILS;
I had a good run at this today. I don't know if you still need it but this is good to understand. You can output this from stored proc:
CREATE OR REPLACE PACKAGE Eidmadm.TestPkg AS
TYPE stringTbl IS TABLE OF varchar2(250) INDEX BY BINARY_INTEGER;
PROCEDURE TestProc (p_strings out stringTbl );
END;
The other table, like below, didn't work
TYPE numTbl IS TABLE OF varchar2(100);
The c# code for this is:
OracleParameter p2 = new OracleParameter(":p_strings", OracleDbType.Varchar2, ParameterDirection.Output);
p2.CollectionType = OracleCollectionType.PLSQLAssociativeArray;
p2.Size = 100; // allocate enough extra space to retrieve expected result
// assign amount of space for each member of returning array
p2.ArrayBindSize = Enumerable.Repeat(250, 100).ToArray();
cmd.Parameters.Add(p2);
cmd.ExecuteNonQuery();
// And this is how you retrieve values
OracleString[] oraStrings = (OracleString[])p2.Value;
string[] myP2Values = new string[oraStrings.Length];
for (int i = 0; i < oraNumbers.Length; i++)
myP2Values[i] = oraStrings[i].Value;
**But most important is this: **
When you fill your pl/sql table, it needs to start from something larger than `0`, and preferably from `1`. Because and also - if you have index with skipped numbers, i.e. `2,4,6,8`, all those spaces will be part of returning `oracle array` and there will be `oracle null` in them. You would need to check for `null` in your loop
if !oraStrings[i].IsNull {....}
else {....}
Enjoy!
I have a oracle stored procedure which updates a table with the following statement.
update boxes
set location = 'some value'
where boxid = passed value
I have a page where the user selects 100+ boxes and updates them with a new location value. Currently, I have to call the stored procedure 100+ times to update each box(by passing a boxid each time).
I want to know how I can pass a list of boxids from C# into the stored procedure so that I have to call the stored procedure just one time.
I am hoping to use a where in(boxids) kind of where clause in the update statement.
Please let know how can I achieve this. Thanks in advance!
Oracle allows you to pass arrays of values as parameters. Borrowing from this SO question and this one you can define an INT_ARRAY type like this:
create or replace type CHAR_ARRAY as table of INTEGER;
Then define your stored procedure as:
CREATE OR REPLACE PROCEDURE product_search(
...
myIds IN CHAR_ARRAY,
...)
AS
SELECT ...
...
WHERE SomeIdField IN (Select column_value FROM TABLE(myIds))
...
You can then pass the list of values by setting the OracleParameter.CollectionType property like this:
OracleParameter param = new OracleParameter();
param.OracleDbType = OracleDbType.Int32;
param.CollectionType = OracleCollectionType.PLSQLAssociativeArray;
I'd create a new procedure, designed to handle a list of values. An efficient approach would be to load the multiple values into a global temp table, using a bulk insert, and then have the procedure update using a join to the GTT.
A notional example would look like this:
OracleTransaction trans = conn.BeginTransaction(IsolationLevel.RepeatableRead);
OracleCommand cmd = new OracleCommand(insertSql, conn, trans);
cmd.Parameters.Add(new OracleParameter("BOX_ID", OracleDbType.Number));
cmd.Parameters[0].Value = listOfBoxIds; // int[] listOfBoxIds;
cmd.ExecuteArray();
OracleCommand cmd2 = new OracleCommand(storedProc, conn, trans);
cmd2.CommandType = CommandType.StoredProcedure;
cmd2.ExecuteNonQuery();
trans.Commit();
Your PL/SQL block may look like this one:
CREATE OR REPLACE PACKAGE YOUR_PACKAGE AS
TYPE TArrayOfNumber IS TABLE OF NUMBER INDEX BY BINARY_INTEGER;
PROCEDURE Update_Boxes(boxes IN TArrayOfNumber );
END YOUR_PACKAGE;
/
CREATE OR REPLACE PACKAGE BODY YOUR_PACKAGE AS
PROCEDURE Update_Boxes(boxes IN TArrayOfNumber) is
BEGIN
FORALL i IN INDICES OF boxes
update boxes
set location = boxes(i)
where boxid = ...;
END Update_Boxes;
END YOUR_PACKAGE;
The C# code you get already in answer from Panagiotis Kanavos
I understand your concern - the round trips will be taxing.
Unfortunately I don't have anything to test, but you can try
Oracle bulk updates using ODP.NET
or
-- edit1: go with Panagiotis Kanavos's answer if your provider supports it, else check below --
-- edit12 as highlighted by Wernfried, long is deprecated. Another thing consider is max length varchar2: it doesn't scale on a very big set. Use the one below as the last resort. --
changing your stored procedure to accept string
implement string_2_list in asktom.oracle.com.
create or replace type myTableType as table of varchar2 (255);
create or replace function in_list( p_string in varchar2 ) return myTableType
as
l_string long default p_string || ',';
l_data myTableType := myTableType();
n number;
begin
loop
exit when l_string is null;
n := instr( l_string, ',' );
l_data.extend;
l_data(l_data.count) :=
ltrim( rtrim( substr( l_string, 1, n-1 ) ) );
l_string := substr( l_string, n+1 );
end loop;
return l_data;
end;
Above is early variant and splice to varchar2, but if you read more (including other threads) at that site
you'll find more advanced variants (optimized, better exception handling)
Before marked as duplicate, I have read the following:
Oracle "IN clause" from parameter
Parameterize an SQL IN clause
problem using Oracle parameters in SELECT IN
Supposed I have this query on my DataSource in my .rdl report published in our report server:
SELECT ...
FROM ...
WHERE c.cluster_cd IN (:paramClusterCD)
Report Builder 2.0 automatically recognized a parameter as #paramClusterCD. On my wpf project, I have to create a parameter with multiple values like this:
var arrCluster = (lbCluster.SelectedItems.Cast<CLUSTER_MSTR>().ToList()).Select(x => x.CLUSTER_CD).ToArray();
string strCluster = string.Join(",", arrCluster); // result is "1,2,3"
Now whenever I run(pass the parameter in the report viewer), I have this error:
ORA-01722: invalid number
Workaround from the previous post won't work since this is a SSRS report.
It's not going to work this way, because Oracle won't recognize that you're actually trying to pass in a list of possible values.
What you want is a query like
select * from t where x in (1,2,3)
but what your code does is
select * from t where x = '1,2,3'
As x is numeric, Oracle tries to cast '1,2,3' into a number - and fails...
Please refer to this excellent thread at AskTom for correct solutions (and a sermon about the importance of bind variables).
Update: Tom's first answer already contains everything you need, but it used the now obsolete THE keyword instead of TABLE. So here are the steps that should work for you:
first create a type for a collection of numbers
create or replace type TableOfNumber as table of number;
then create a function that splits your string and returns your newly created collection
create or replace function in_list( p_string in varchar2 ) return TableOfNumber as
l_string long default p_string || ',';
l_data TableOfNumber := TableOfNumber();
n number;
begin
loop
exit when l_string is null;
n := instr( l_string, ',' );
l_data.extend;
l_data(l_data.count) := to_number( substr( l_string, 1, n-1 ) );
l_string := substr( l_string, n+1 );
end loop;
return l_data;
end;
Now you can use this function in a query:
SELECT ...
FROM ...
WHERE c.cluster_cd IN
(select * from TABLE (select cast(in_list(:paramClusterCD) as mytableType) from dual))
Kindly try the below if you can ensure that the parameters passed is a number and if c.cluster_cd is a number column
SELECT ...
FROM ...
WHERE to_char(c.cluster_cd) IN ((:paramClusterCD));
I have a stored procedure that contains like 10 different INSERTS, is it possible to return the COUNT of the rows affected on each INSERT to ASP.NET c# page so i can display Stored Procedure process for the client viewing that ASP.NET page?
You need to use following command in the start of your stored procedure:
SET NOCOUNT OFF
In this case SQL server will send text messages ("X rows affected" ) to client in real time after each INSERT/UPDATE. So all you need is to read these messages in your software.
Here is my answer how to do it in Delphi for BACKUP MS SQL command. Sorry I've not enough knowledge in C# but I guess you can do it in C# with SqlCommand class.
On the server side send the message to the client using RAISERROR function with severity 10 (severity higher than 10 causes exception that breaks procedure execution, i.e. transfers execution to the CATCH block, if there is one). In the following example I haven't added error number, so the default error number of 50000 will be used by RAISERROR function. Here is the example:
DECLARE #count INT = 0
DECLARE #infoMessage VARCHAR(1000) = ''
-- INSERT
SET #count = ##ROWCOUNT
SET #infoMessage = 'Number of rows affected ' + CAST(#count AS VARCHAR(10))
RAISERROR(#infoMessage, 10, 0) WITH NOWAIT
-- another INSERT
SET #count = ##ROWCOUNT
SET #infoMessage = 'Number of rows affected ' + CAST(#count AS VARCHAR(10))
RAISERROR(#infoMessage, 10, 0) WITH NOWAIT
On the client side, set the appropriate event handlers, here is an example:
using (SqlConnection conn = new SqlConnection(...))
{
conn.FireInfoMessageEventOnUserErrors = true;
conn.InfoMessage += new SqlInfoMessageEventHandler(conn_InfoMessage);
using (SqlCommand comm = new SqlCommand("dbo.sp1", conn)
{ CommandType = CommandType.StoredProcedure })
{
conn.Open();
comm.ExecuteNonQuery();
}
}
static void conn_InfoMessage(object sender, SqlInfoMessageEventArgs e)
{
// Process received message
}
After every Inserts, use ##ROWCOUNT, then get the value by query.
Returns the number of rows affected by the last statement. If the
number of rows is more than 2 billion, use ROWCOUNT_BIG.
Sample:
USE AdventureWorks2012;
GO
UPDATE HumanResources.Employee
SET JobTitle = N'Executive'
WHERE NationalIDNumber = 123456789
IF ##ROWCOUNT = 0
PRINT 'Warning: No rows were updated';
GO
Edit: How you can get ##rowcount with multiple query? Here's an example:
DECLARE #t int
DECLARE #t2 int
SELECT * from table1
SELECT #t=##ROWCOUNT
SELECT * from table2
SELECT #t2=##ROWCOUNT
SELECT #t,#t2'