I've been asked by my boss to replicate an MS Access feature that we're going to lose shortly after migrating our product to .NET.
The feature is the ability to view and update any data in the database, particularly Tables or Views, in a tabular grid.
I can do it for pure tables that have a identity column because the SqlDataAdapter can auto-generate the relevant CRUD methods on the fly, to fill / update via DataTables.
However, views are somewhat more tricky. SQL Server Management Studio does allow it. If you click 'Edit top xx rows' on a View, it allows you to edit the data in some columns in what looks to be a standard .NET DataGridView - though it feels a bit magical.
So, a few questions:
How does SSMS infer which primary key to use, even if the key is not in the view?
How does SSMS determine which column inside a view can or can not be edited / inserted / deleted etc.?
What would be my best option to replicate this inside a .NET application?
Is it possible to connect a DataGridView to an old style oledb / obdc connection that has a constant direct connection to the database?
Any guidance as normal will be highly appreciated.
Marlon
SQL Server views can be updated just as if they were a single table, as long as they conform to certain conditions.
From the documentation:
Updatable Views
You can modify the data of an underlying base table through a view, as
long as the following conditions are true:
Any modifications, including UPDATE, INSERT, and DELETE statements,
must reference columns from only one base table.
The columns being modified in the view must directly reference the
underlying data in the table columns. The columns cannot be derived in
any other way, such as through the following:
An aggregate function: AVG, COUNT, SUM, MIN, MAX, GROUPING, STDEV,
STDEVP, VAR, and VARP.
A computation. The column cannot be computed from an expression that
uses other columns. Columns that are formed by using the set operators
UNION, UNION ALL, CROSSJOIN, EXCEPT, and INTERSECT amount to a
computation and are also not updatable.
The columns being modified are not affected by GROUP BY, HAVING, or
DISTINCT clauses.
TOP is not used anywhere in the select_statement of the view together
with the WITH CHECK OPTION clause.
The previous restrictions apply to any subqueries in the FROM clause
of the view, just as they apply to the view itself. Generally, the
Database Engine must be able to unambiguously trace modifications from
the view definition to one base table. For more information, see
Modify Data Through a View.
I don't believe SSMS is doing anything special - editing the contents of a view offers exactly the same functionality as editing the contents of a table. If the user attempts to make a change that does not conform to the above conditions, SSMS will likely display an error.
How does SSMS infer which primary key to use, even if the key is not in the view?
It doesn't. SQL Server does since only one underlying table can be edited at a time.
How does SSMS determine which column inside a view can or can not be edited / inserted / deleted etc.?
Again, it's SQL Server that determines this, not SSMS.
What would be my best option to replicate this inside a .NET application?
As long as all your views conform to the above conditions, simply do the same as you're doing for tables, but be ready to handle the errors from users doing something they can't (this implies some user training will be required, just as it would be if they were using SSMS directly).
Related
I have a webservice which tries to connect to a database of a desktop accounting application.
It have tables with same name but with different schema names such as:
[DatabaseName].[202001].[CustomerCredit]
[DatabaseName].[202002].[CustomerCredit]
.
.
.
[DatabaseName].[202014].[CustomerCredit]
[DatabaseName].[202015].[CustomerCredit]
[DatabaseName].[202016].[CustomerCredit]
...
..
[DatabaseName].[2020xx].[CustomerCredit]
Schema name is in format [Year+IncrementalNumber] such as [202014], [202015],[202016] and etc.
Whenever I want to query customer credit information in database, I should fetch information from schema with biggest number such as [DatabaseName].[202016].[CustomerCredit] if 202016 is latest schema in my db.
Note:
Creation of new schema in accounting application database have no rules and is completely decided by user of accounting application and every instance of application installed on different place may have different number of schemas.
So when I'm developing my webservice I have no idea to connect to which schema prior to development. In run-time I can find correct schema to query from its tables but I don't know how to manage to fetch table information with correct schema name in query.
I ususally creat a linq-to-sql dbml class and use its definitions to read information from db but I don't know how to manage schema change in this way?
DBML designer manage Scehma names like this:
[global::System.Data.Linq.Mapping.TableAttribute(Name="[202001].CustomerCredit")]
However since my app can retrieve schema name in run time, I don't know how to fix table declaration in my special case.
It is so easy to handle in ADO.NET but I don't know its equivalent in Linq2SQL:
select count(*) from [" + Variables.FinancialYearSchemaName + "].CustomerCredit where SFC_Status = 100;
Ultimately, no: most ORMs do not expect the schema change to vary at runtime, so most - including EF and LINQ-to-SQL do not support this scenario. One possible option would be to have different connection strings, each with different user accounts, that each has a different default schema configured at the database - and intialize your DB-context with a connection-string or connection that matches the required account. Then if EF asks the RDBMS for [CustomerCredit], it will look first in that account's schema ([202014].[CustomerCredit]). You should probably avoid having a [202014].[CustomerCredit] in that scenario, to prevent confusion. This is, however, a pretty hacky and ugly solution. But... it should work.
Alternatively, you would have to take more control over the data access, essentially writing your own SQL (presumably with a token replacement for the schema, which has problems of its own).
That schema is essentially a manual partitioning of the CustomerCredit table. The best solution would one that makes partitioning transparent to all users. The code shouldn't know how the data is partitioned.
Database Solutions
The benefit of database solutions is that they are transparent or almost transparent to users and require minimal maintenance
Table Partitioning
The clean solution would be to use table partitioning, making the different partitions transparent to all users. Table partitioning used to be an Enterprise-only feature but it became available in all editions since SQL Server 2016 SP1, even Express. This means it's free in all versions still in mainstream support.
The table is partitioned based on a function (eg a date based function) and stored in different files. Whenever possible, the query optimizer can check the partition boundaries and the query conditions and use only the file that contains the relevant data. Eg in a date-partitioned table, queries that contain a date filter can search only the relevant partitions.
Partitioned views
Another option, available since 2000 at least, is to use partitionend views, essentially a UNION ALL view that combines all table partitions, eg :
SELECT <select_list1>
FROM [202001].[CustomerCredit]
UNION ALL
SELECT <select_list2>
FROM [202002].[CustomerCredit]
UNION ALL
...
SELECT <select_listn>
FROM Tn;
EF can map entities to views instead of tables. If the criteria for updatable views are met, the partitioned view itself will be updatable and any modifications will be made to the correct table.
The query optimizer can take advantage of CHECK constraints on the tables to search only one table at a time, similar to how partitioned tables work.
Code solutions
This requires raw SQL queries, and a way to identify the correct table/schema each time a change is made. It requires modifications to the application each time the table partitioning changes, whether those are code modifications, or changes in a configuration file.
In all cases, one query can only read from one table at a time
Keep ADO.NET
One possibility is to keep using ADO.NET, replacing the table/schema name in a query template. The code will have to map to objects if needed, the same way it already did.
EF Raw SQL
Another, is to use EF's raw SQL features, eg EF Core's FromSqlRaw to query from a specific table , the same way ADO.NET would. The benefit is that EF will map the query results to objects. In EF Core, the raw query can be combined with LINQ operators :
var query=$"select * from [DatabaseName].[{schemaName}].[CustomerCredit]"
var credits = context.CustomerCredits
.FromSqlRaw(query)
.Where(...)
.ToList();
Dapper
Another option is to use Dapper or another micro-ORM with an ad-hoc query, similar to ADO.NET, and map the results to objects:
var query=$"select * from [DatabaseName].[{schemaName}].[CustomerCredit] where customerID=#ID";
var credits=connection.Query<CustomerCredit>(query,new {ID=someID});
I'm setting up a data warehouse (in SQL Server) together with our engineers we got almost everything up and running. Our main application also uses SQL Server as backend, and aims to be code first while using the entity framework. In most tables we added a column like updatedAt to allow for incremental loading to our data warehouse, but there is a many-to-many association table created by the entity framework which we cannot modify. The table consists of two GUID columns with a composite key, so they are not iterable like an incrementing integer or dates. We are now basically figuring out the options on how to enable incremental load on this table, but there is little information to be found.
After searching for a while I mostly came across posts which explained how it's not possible to manually add columns (such as updatedAt) to the association table, such as here Create code first, many to many, with additional fields in association table. Suggestions are to split out the table into two one-to-many tables. We would like to prevent this if possible.
Another potential option would be to turn on change data capture on the server, but that would potentially defeat the purpose of code first in the application.
Another thought was to add a column in the database itself, not in code, with a default value of the current datetime. But that might also be impossible / non compatible with the entity framework, as well as defeating the code first principle.
Are we missing anything? Are there other solutions for this? The ideal solution would be a code first solution, or a solution in the ETL process without affecting the base application, without changing too much. Any suggestions are appreciated.
I'm currently trying to read data from a table out of SQL server. The table has 10 columns and when i'm reading the base table by itself, everything works out just fine.
The trouble is, there are X number of extra property tables that may or may not go with my base table. Some databases only have the base table with 10 columns, while others have property tables containing more columns that must be joined into the base table in order to properly display the needed data. Is there a way via EF6 to load that data into a queryable source in a decoupled way?
Basically due to the fact that the extra tables are constantly in flux, I cannot rely on generating models for them and using the mapping EF provides. I do have a model for the base table as its 10 columns never change. I also have a mechanism to read the relational information in order to get the names of the property tables and columns that my program needs to display with the base table when they are available.
Any insight is greatly appreciated.
Good old-fashioned ADO.NET does a good job giving you run-time access to arbitrary query results. You can examine the column data types returned in a DataReader or after loaded into a DataTable, and access columns by name or ordinal position.
And you can interop beteween EF (for your design-time models) and ADO.NET tables that vary at runtime.
What is the standard way of copying data from one oracle database to another.
1) Read data from source table and copy to temp table on destination using configuration( i.e. there are more than 1 table and each table has separate temp table)
2) Right now there is no clob data, but in future clob data might be used.
3) Read everything to memory(if large data read in chunks)
Should not use Oracle links
Should not use files
Code should be only using C# but not any database procedures.
One way that I've used to do this is to use a DataReader on the source database and just perform inserts on the target database (using Bind Parameters for sure).
Note that the DataReader is excellent at not using much memory as it moves through a table (I believe that by default it uses a Fast Forward, Read Only cursor). This means that only a small amount of data is held in memory at a given time.
Here are the things to watch out for:
Relationships
If you're working with data that has relationships, you're going to need to deal with that. There are two ways that I've seen to deal with this:
Temporarily drop the relationships in the target database before doing the copy, then recreate them after.
Copy the data in the correct order for the relationships to work correctly (this is usually pretty difficult / inefficient)
Auto Generated Id Values
These columns are usually handled by disabling the auto increment functionality for the given table and allowing identity insert (I'm using some SQL Server terms, I can't remember how it works on Oracle).
Transactions
If you're moving a lot of data, transactions will be expensive.
Repeatability / Deleting Target Data
Unless you're way more awesome than the rest of us, you'll probably have to run this thing more than once (at least during development). That means you might want a way to delete the target data.
Platform Specific Methods
In SQL Server, there are ways to perform bulk inserts that are blazingly fast (by giving up little things like referential integrity checking). There might be a similar feature within the Oracle toolset.
Table / Column Metadata
I haven't had to do this in Oracle yet, but it looks like you can get metadata on tables and columns using the views mentioned here.
I have 3 tables:
Item_Detail -----------------ID, Name
ItemPurchased_Detail ---QtyPurchased , RateOfPurchase , DiscountReceived
ItemSold_Detail -----------QtySold , RateofSale , DiscountGiven
Also, i have a MAIN TABLE, ITEM_FULL_DETAIL, which contains all the columns from above 3 tables.
i have a winform application, with a single form which contains all the textboxes to insert data in the ITEM_FULL_DETAIL table. the user would input all the data, and click the SUBMIT button
i want to insert data first in the MAIN TABLE, and then it should distribute data individually to all the 3 tables. for this what shall i use? like triggers, porcedures, or views or joins?
Also, am using the ITEM_FULL_DETAIL table, because i want to protect my actual tables from any loss of data such as in case of power outage.
Shall I use a temporary table in place of ITEM_FULL_DETAIL table or is it fine using the current one only?
Is there any other way also?
You can use database triggers or insert all records on the application level.
You should probably re-think your design: duplicating the same data in different tables is usually a bad idea. In this case, you could replace ITEM_FULL_DETAIL with a view, then maintain the data in the underlying tables. That way you only have one copy of the data, so you don't need to worry about inconsistencies between tables.
If you did that then you could either insert new data into the 3 underlying tables in the correct order (probably the best idea) or use an INSTEAD OF trigger on the ITEM_FULL_DETAIL view (more complicated). The INSERTs can be done using an ORM, ADO.NET, a stored procedure or whatever suits your application design best.
If you do have a good reason for duplicating your data, then it would be helpful if you could share it, because someone may have a better suggestion for that scenario.
Also, am using the ITEM_FULL_DETAIL table, because i want to protect my actual tables from any loss of data such as in case of power outage.
..What? How do you suppose you are protecting your tables? What are you trying to prevent? There is absolutely no need to have the ITEM_FULL_DETAIL table if what you are worried about is data integrity. You're probably creating a situation in which data integrity can be compromised by using this intermediate table.
Are you aware of transactions? Use them. If two out of three tables are written to, then the power on the client goes off and can't complete the 3rd write, the transaction will fail and the partial data will be rolled back.
Unless I'm totally missing the point here..