In my application, i want to show the newly added RECORDS by an import operation in a gridview. Is there is any method in sql to retrive newly added rows.
I tried to do it in using code and tried to get the difference before and after the insertion and its working perfectly but makes the application very slow. So, i want to do it in database itself.
Im using Mysql, ASP.NET.
Eg:
table may have these records before the import operation
ID Name
1 A
2 B
3 C
and after import the table may be like this.
ID Name
1 A
2 B
3 C
4 D
5 E
6 F
I want result like
ID Name
4 D
5 E
6 F
You need to have AUTO_INCREMENT column defined on table or alternatively you can use TIMESTAMP field to retrieve newly added records, try this:
SELECT *
FROM table_name
ORDER BY id DESC
LIMIT 10;
For single row insert you can use LAST_INSERT_ID after you INSERT query:
SELECT LAST_INSERT_ID();
For multi-row insert you can follow these steps:
START TRANSACTION;
SELECT MAX(id) INTO #var_max_id FROM table_name;
INSERT INTO table_name VALUES(..),(..),...;
SELECT MAX(id) INTO #var_max_id_new FROM table_name;
COMMIT;
SELECT *
FROM table_name
WHERE id BETWEEN (#var_max_id + 1) AND #var_max_id_new;
i think this will be more simple:
SELECT MAX(id) INTO #Max_table_Id FROM table;
// Insert operation here//////
SELECT * FROM table WHERE id>#Max_table_Id;
In case you use auto incremental IDs for your records, you can use:
SELECT * FROM [table] ORDER BY [id column] DESC LIMIT [number of records]
Otherwise you should add a TIMESTAMP colum to your records for this purpose and select by this column.
Personally, if there is an option, I wouldn't use the record IDs for this, as it is not what they are for. Record IDs can change throughout the lifetime of an application and they don't necessarily represent the order in which the items were added. Especially in data import/export scenarios. I'd prefer to create special columns to store such information, e.g. "CreatedAt", "ModifiedAt".
Related
I have some code that
Looks at the max ID in Table A
Fetches the records from Table B with ID > Max ID from the Table A
Loops through each record that it needs to add (are these the records from Table A????)
Finds all records in Table C that is related to the ID from Table B and inserts those records into Table D.
For some reason the loop stops unexpectedly after fetching and inserting some records with no error. It stops at random stages. Sometimes it adds 20 records sometimes 100 and sometimes 1000+. Is it to do with my logic or is it a server setting maybe? The web application .Net Framework 4.5 and is hosted on a Windows 2019 Server.
Here is the logic I follow. I specifically don't add the code as it is quite elaborate and I thought I'd share the logic I follow rather to get some suggestions on why this might be happening.
GetNewRecords()
MaxIdA = Select max id in Table A
Select all records in Table B with id > MaxIdA
if reader has rows
While reader.Read
NextID = reader[0]
AddNewRecordToTableB()
Select all records in Table C where IdLink = NextID
if reader2 has rows
While reader2.Read
NewDetail = reader2[..]
AddNewRecordToTableD()
Connection2 Dispose
AddNewRecordToTableB()
Insert into TableB ….
AddNewRecordToTableD()
Insert into Tabled ….
I have some code that looks at the max ID in a table and then fetches the records from another table that has ID > Max ID from the first table. The code then loops through each record that it needs to add and finds all records in Table C that is related to the ID from Table B and adds those records all to Table D.
It sounds like this:
INSERT INTO d(column,list,here)
SELECT column,list,here --columns from b and c here
FROM
b
JOIN c ON b.id = c.b_id
WHERE
b.id > (SELECT MAX(id) FROM a)
column list lengths must be same in insert and select
I'm not sure if you need the join; if no data is needed from B (if C contains all the required data) you can skip B:
INSERT INTO d(column,list,here)
SELECT column,list,here --columns from c here
FROM
c
WHERE
c.b_id > (SELECT MAX(id) FROM a)
Problem: I need write stored procedure(s) that will return result set of a single page of rows and the number of total rows.
Solution A: I create two stored procedures, one that returns a results set of a single page and another that returns a scalar -- total rows. The Explain Plan says the first sproc has a cost of 9 and the second has a cost of 3.
SELECT *
FROM ( SELECT ROW_NUMBER() OVER ( ORDER BY D.ID DESC ) AS RowNum, ...
) AS PageResult
WHERE RowNum >= #from
AND RowNum < #to
ORDER BY RowNum
SELECT COUNT(*)
FROM ...
Solution B: I put everything in a single sproc, by adding the same TotalRows number to every row in the result set. This solution feel hackish, but has a cost of 9 and only one sproc, so I'm inclined to use this solution.
SELECT *
FROM ( SELECT ROW_NUMBER() OVER ( ORDER BY D.ID DESC ) RowNum, COUNT(*) OVER () TotalRows,
WHERE RowNum >= from
AND RowNum < to
ORDER BY RowNum;
Is there a best-practice for pagination in Oracle? Which of the aforementioned solutions is most used in practice? Is any of them considered just plain wrong? Note that my DB is and will stay relatively small (less than 10GB).
I'm using Oracle 11g and the latest ODP.NET with VS2010 SP1 and Entity Framework 4.4. I need the final solution to work within the EF 4.4. I'm sure there are probably better methods out there for pagination in general, but I need them working with EF.
If you're already using analytics (ROW_NUMBER() OVER ...) then adding another analytic function on the same partitioning will add a negligible cost to the query.
On the other hand, there are many other ways to do pagination, one of them using rownum:
SELECT *
FROM (SELECT A.*, rownum rn
FROM (SELECT *
FROM your_table
ORDER BY col) A
WHERE rownum <= :Y)
WHERE rn >= :X
This method will be superior if you have an appropriate index on the ordering column. In this case, it might be more efficient to use two queries (one for the total number of rows, one for the result).
Both methods are appropriate but in general if you want both the number of rows and a pagination set then using analytics is more efficient because you only query the rows once.
This may help:
SELECT * FROM
( SELECT deptno, ename, sal, ROW_NUMBER() OVER (ORDER BY ename) Row_Num FROM emp)
WHERE Row_Num BETWEEN 5 and 10;
In Oracle 12C you can use limit LIMIT and OFFSET for the pagination.
Example -
Suppose you have Table tab from which data needs to be fetched on the basis of DATE datatype column dt in descending order using pagination.
page_size:=5
select * from tab
order by dt desc
OFFSET nvl(page_no-1,1)*page_size ROWS FETCH NEXT page_size ROWS ONLY;
Explanation:
page_no=1
page_size=5
OFFSET 0 ROWS FETCH NEXT 5 ROWS ONLY - Fetch 1st 5 rows only
page_no=2
page_size=5
OFFSET 5 ROWS FETCH NEXT 5 ROWS ONLY - Fetch next 5 rows
and so on.
Refrence Pages -
https://dba-presents.com/index.php/databases/oracle/31-new-pagination-method-in-oracle-12c-offset-fetch
https://oracle-base.com/articles/12c/row-limiting-clause-for-top-n-queries-12cr1#paging
A clean way to organize your SQL code could be trough WITH statement.
The reduced version implements also total number of results and total pages count.
For example
WITH SELECTION AS (
SELECT FIELDA, FIELDB, FIELDC FROM TABLE),
NUMBERED AS (
SELECT
ROW_NUMBER() OVER (ORDER BY FIELDA) RN,
SELECTION.*
FROM SELECTION)
SELECT
(SELECT COUNT(*) FROM NUMBERED) TOTAL_ROWS,
NUMBERED.*
FROM NUMBERED
WHERE
RN BETWEEN ((:page_size*:page_number)-:page_size+1) AND (:page_size*:page_number)
This code gives you a paged resultset with two more fields:
TOTAL_ROWS with the total rows of your full SELECTION
RN the row number of the record
It requires 2 parameter: :page_size and :page_number to slice your SELECTION
Reduced Version
Selection implements already ROW_NUMBER() field
WITH SELECTION AS (
SELECT
ROW_NUMBER() OVER (ORDER BY FIELDA) RN,
FIELDA,
FIELDB,
FIELDC
FROM TABLE)
SELECT
:page_number PAGE_NUMBER,
CEIL((SELECT COUNT(*) FROM SELECTION ) / :page_size) TOTAL_PAGES,
:page_size PAGE_SIZE,
(SELECT COUNT(*) FROM SELECTION ) TOTAL_ROWS,
SELECTION.*
FROM SELECTION
WHERE
RN BETWEEN ((:page_size*:page_number)-:page_size+1) AND (:page_size*:page_number)
Try this:
select * from ( select * from "table" order by "column" desc ) where ROWNUM > 0 and ROWNUM <= 5;
I also faced a similar issue. I tried all the above solutions and none gave me a better performance. I have a table with millions of records and I need to display them on screen in pages of 20. I have done the below to solve the issue.
Add a new column ROW_NUMBER in the table.
Make the column as primary key or add a unique index on it.
Use the population program (in my case, Informatica), to populate the column with rownum.
Fetch Records from the table using between statement. (SELECT * FROM TABLE WHERE ROW_NUMBER BETWEEN LOWER_RANGE AND UPPER_RANGE).
This method is effective if we need to do an unconditional pagination fetch on a huge table.
Sorry, this one works with sorting:
SELECT * FROM (SELECT ROWNUM rnum,a.* FROM (SELECT * FROM "tabla" order by "column" asc) a) WHERE rnum BETWEEN "firstrange" AND "lastrange";
I have the following query which works in SQL Server :-
query = "select * from (select *,ROW_NUMBER() OVER (ORDER BY LogDate)
AS ROW_NUM from table1) x where ROW_NUM>0";
But I want this in Access OLEDB, which doesn't support ROW_NUMBER() function.
What is another way to get ROW NUMBER in OLEDB Provider?
Access does not support rownum. You could add a column called ID which would use AutoIncrement to simulate a row number. But if a record is removed from the table the ID will not change, meaning a ID will be missing from your records.
This anwser also explores some possibilities:
How to use the same function like Oracle Rownum in MS ACCESS
I dont know if i choose the right title for my question.
I have that table like this in my database.
NAME class_num
-------------------------------------------------- --------
name1 2
name2 4
name3 1
And what I want to do is:
Update the class_num to be in order like the name. The result from what I want is that:
NAME class_num
-------------------------------------------------- ---------
name1 1
name2 2
name3 3
Is any way to do that with a QUERY?
Or I need to do from C# code?
Sorry my english, please help. Thanks.
In SQL Server, you can use an updatable CTE (or subquery):
with toupdate as (
select t.*, row_number() over (order by name) as new_class_num
from table t
)
update toupdate
set class_num = new_class_num;
Assuming:
that your ID field is not an IDENTITY field, and can be updated
That your Name column holds unique values (there are no duplicate names)
You can do something like this (example in T-SQL for SQL Server):
-- Select all Names in alphabetical order, and give them a row number
INSERT INTO #NewIdTable
SELECT ROW_NUMBER() OVER(ORDER BY name ASC) as Row, Name
FROM MyTable
-- Update the ID's using the RowNumber as a new ID.
UPDATE MyTable
SET Id = Row
FROM #NewIdTable INNER JOIN MyTable ON #NewIdTable.Name = MyTable.Name
Edit: Looks like you clarified in the comments that your class_num column is a key. Therefore you will have to adjust this method. Try updating "Name" by joining on the row-number instead, if your ID's are sequential. Alternately, you could drop the key constraint temporarily, update the ID's, and create the key again. Another method might be to simply create a new table with IDENTITY INSERT ON and copy the names over to it in alphabetical order.
Firstly You can delete your table that query
"TRUNCATE TABLE table_name"
And later, you can create your table again with that query
"CREATE TABLE table_name
(
column_name1 data_type(size),
column_name2 data_type(size),
column_name3 data_type(size),
....
);"
I am inserting records through a query similar to this one:
insert into tbl_xyz select field1 from tbl_abc
Now I would like to retreive the newly generated IDENTITY Values of the inserted records. How do I do this with minimum amount of locking and maximum reliability?
You can get this information using the OUTPUT clause.
You can output your information to a temp target table or view.
Here's an example:
DECLARE #InsertedIDs TABLE (ID bigint)
INSERT into DestTable (col1, col2, col3, col4)
OUTPUT INSERTED.ID INTO #InsertedIDs
SELECT col1, col2, col3, col4 FROM SourceTable
You can then query the table InsertedIDs for your inserted IDs.
##IDENTITY will return you the last inserted IDENTITY value, so you have two possible problems
Beware of triggers executed when inserting into table_xyz as this may change the value of ##IDENTITY.
Does tbl_abc have more than one row. If so then ##IDENTITY will only return the identity value of the last row
Issue 1 can be resolved by using SCOPE__IDENTITY() instead of ##IDENTITY
Issue 2 is harder to resolve. Does field1 in tbl_abc define a unique record within tbl_xyz, if so you could reselect the data from table_xyz with the identity column. There are other solutions using CURSORS but these will be slow.
SELECT ##IDENTITY
This is how I've done it before. Not sure if this will meet the latter half of your post though.
EDIT
Found this link too, but not sure if it is the same...
How to insert multiple records and get the identity value?
As far as I know, you can't really do this with straight SQL in the same script. But you could create an INSERT trigger. Now, I hate triggers, but it's one way of doing it.
Depending on what you are trying to do, you might want to insert the rows into a temp table or table variable first, and deal with the result set that way. Hopefully, there is a unique column that you can link to.
You could also lock the table, get the max key, insert your rows, and then get your max key again and do a range.
Trigger:
--Use the Inserted table. This conaints all of the inserted rows.
SELECT * FROM Inserted
Temp Table:
insert field1, unique_col into #temp from tbl_abc
insert into tbl_xyz (field1, unique_col) select field1, unique_col from tbl_abc
--This could be an update, or a cursor, or whatever you want to do
SELECT * FROM tbl_xyz WHERE EXISTS (SELECT top 1 unique_col FROM #temp WHERE unique_col = tbl_xyz.unique_col)
Key Range:
Declare #minkey as int, #maxkey as int
BEGIN TRANS --You have to lock the table for this to work
--key is the name of your identity column
SELECT #minkey = MAX(key) FROM tbl_xyz
insert into tbl_xyz select field1 from tbl_abc
SELECT #maxkey = MAX(key) FROM tbl_xyz
COMMIT Trans
SELECT * FROM tbl_xyz WHERE key BETWEEN #minkey and #maxkey