I'm still learning MVC and have gone through several online tutorials. But I'm missing something sort of vital to my application, so this is a general question not necessarily requiring code examples to answer. If you can just steer me in the right direction in conceptual terms...
My application is completely read-only to the database, I don't need or want to write back. I need to pull data from multiple tables in one database, which are the exact same schema, into what I think would be a single model that I can then filter, then display the results. To complicate things somewhat, the table names need to be variables, these tables are built upstream on the fly using the date as part of the table name.
The tables are television automation schedules, different table for each day, but each contain a number of fields for scheduled time, house ID, title, etc. I need to get several days into one model (I think), and then I'm going to query a different database that will tell me for each row in the table whether the House ID exists on a video server or not. I want to then display the list of rows that do not exist in the video server.
I have an example in VB but feel like I should tackle this in C# as it seems to be more universally supported.
I don't think I can use VS tools to create a model from the database table since the table name is different every day.
So is the proper plan of attack to load the multiple table data into one model?
Maybe I don't even need a model in the true sense of the word, there's no binding required to be able to write the data back to the db. I just essentially need to load the table data into an array, doesn't need to continue to be bound to the db, that I can then analyze and figure out which of these items don't exist in the server.
Thanks!
Related
I am working on a project where I need to create a database to track the status of units throughout the production process. My current blockade involves getting the users to interact with a DataGridView that is supplied from a Microsoft Access Query instead of a Microsoft Access Table.
What I want to do is create a query in Microsoft Access and have it link to the DataGridView so end users can interact with a query instead of the actual tables, while populating all parent tables.
I am not sure if what I am attempting to do is possible or advised. This is the first time I have built a database in the professional world and want to make sure I am doing things properly. I have also never built a C# application for business use and have very limited experience with the language itself.
I have tried creating the Query in Access and linking it to the application in the same way you would add a table from a data source. That would allow me to view the data in the query...but it would display as a read-only and not allow for any data to be altered (the query builder in the TableAdapter Query Configuration Wizard indicated it was a read-only) . I have tried adding all related table adapters to the TableAdapterManager and it still didn't help.
I apologize if this question sounds disjointed as I am trying to overcome one obstacle at a time and do not want to overload one question with multiple issues. I can supply my ERD if it will make things easier and I have it normalized to at least 2NF.
Background
My backend has a database in SQL server 2012 which has around 20 tables (maybe will increase in time) and each table will have approx 100 - 1000 rows initially might increase in future.
Now one of my colleague developed an web application which uses this database and let clients do CRUD and usual business logic.
Problem
My task is to create a reporting page for this web application, what I will be doing is to give client ability to export all of the data for all of there deep nested objects from SQL from all tables or only couple with all columns or only few... in excel, pdf and other formats in future. I might also need to query 3rd party in my business logic for gathering further information (out of context for now).
What can I do to achieve above ?
What I know
I can't think of any efficient and extendable solution, as it will involve 100s of columns and 20s of tables. All I can think of adding 100s of views for what I might require but it doesn't sound particle either.
Should I look into BI or SQL reporting or should this be done in code using ORM like EF ? or is there any open source code already out there for such Generic operations I am totally confused.
Please note I am asking what to use not how to use. Hope I didn't offended anyone.
If you aren't concerned with the client having access to all your database object names, you could write up something yourself without too much effort. If you are creating a page you could query the system views to get a list of all table and column names to populate some sort of filtering (dropdowns, listbox, etc).
you can get a list of all the tables:
select object_id, name from sys.tables
you could get a list of all columns per table:
select object_id, name from sys.columns
object_id is the common key between the views.
Then you could write some dynamic SQL based on the export requirements if you plan to export through SQL.
Introduction:
I'm refactoring (pretty much rewriting) a legacy application in my current internship. The part that this question will be concerned about is the database it uses and the way they retrieve data from it.
The database structure is:
There's a table that has the main records. Let's say each record is a measurement. It has some info about the measured material and different measurement information.
There's a table view they use that has the same information columns, plus some extra columns that contains data calculated from the given measurements. And it also filters some of the data from the table.
So let's say we have the main table with columns:
Measurement ID
Measurement A
Measurement B
The view has something like this:
Measurement ID
Measurement A
Measurement B
Some extra data (for example Measurement A * Measurement B)
The guy that is leading the development only knows some SQL, so he likes adding new columns that is calculated by some columns in the main table for experimenting. And this is definitely a need at the moment.
Requirements are:
Different types of databases should be supported (like SQL Server, Oracle, and probably some others).
The frontend should be able to show the view, which means even though some main columns will always stay the same, there may be some new columns including newly calculated values.
My question is:
What kind of system should I use to accommodate the needs of this application? I wanted to use Entity Framework, but the fact that the view may have new columns in the future is I think a problem. As far as I understand, I should map my classes to the database before compiling.
The other thing that I'm considering is maybe using Entity Framework to get data from the main table and do the calculations and the filtering that is currently done in the table view directly in the frontend, and skip the view altogether. Which sounds fine, though I don't know if they will allow me to do that.
What would you do in my case? Please take into account that I have virtually no experience with databases and ORMs.
You are correct in that using Entity Framework will be a problem if the underlying DB schema is always changing. It will require you to update the EF model on your end every time to grab those new columns.
Ideally, all of your database access is hidden behind the interface to your DAL, so that your application doesn't need to know about which ORM is being used -- if any -- or which database it's connecting to.
I hate to say it, but given your requirements, an ORM might not make sense. You might want to go with something more generic without any strong-typing. You could just simply always return a DataTable to your application layer, and it could loop through the columns and values to display whatever is returned. If there are fields you know will never change, you could create a manual mapping for those fields only into your application object(s).
You may have a look to NoSQL system that are a lot more flexible on the schema. Or have a look to document database like RavenDB. All these systems allow the schema to change dynamically. You need to check the Pro's and Con's to see if it can fill you requirements.
(This answer is a bit out of subject as it's about replacing the SQL server and not really creating a DAL, but other answers cover the subject well and I would like to propose another way that may help.)
If your schema is unstable, then using Entity Framework as a beginner is going to be a headache. The assumption is that you can just refresh the design canvas periodically to let the tool handle database table changes. You can try that for a time to see when it becomes too much of a pain, but without any prior experience using ORMs or Entity Framework it may not be worth the effort.
I would probably use something like Rob Conery's Massive ORM (https://github.com/robconery/massive). It gives you more flexibility with the underlying database schema and is a very small library. I remember it being ~300 lines of code and very easy to use. It uses C# dynamics so you'll have to be using >= C# 4.0 and be comfortable with that one concept but IMO it's worth it for the low-overhead. A full-fledged ORM like Entity Framework or NHibernate is going to cost a lot of learning cycles.
You could, of course, just stick to ADO.NET DataTables. They're a bit ugly and verbose, but they'll do the job.
You can use Entity Framework - Database First if the DB is changing. Of course, you will have to regenerate your classes when you want to be able to access new columns, when the DB schema changes.
If you need to accomodate different database servers, then you should take a look into implementing a repository pattern and abstract all your data access that way.
Your comment
it involves write operations to the main table but the main table never changes
confirms what I was hoping for. It means you can use Entity Framework as the core of you application and a different route to display data.
Suppose that for display (of the view) you use a classic DataTable (because all common grids support them, contrary to displaying dynamic objects). I don't know how create/update/delete will be done, but saving changes will at some point involve mapping a DataRow to a MainEntity object. You can write one method for that like
MainEntity DataRowToEntity(DataRow row)
{
var entity = new MainEntity();
entity.PropertyA = row["PropertyA"];
....
}
The MainEntity can be attached to a context, its status changed to Modified, and saved.
We have a requirement on our project for custom fields. We have some standard fields on the table and each customer wants to be able to add their own custom fields. At the moment I am not interested in how this will work in the UI, but I want to know what the options are for the back end storage and retrieval of the data. The last time I did something like this was about 10 years ago in VB6 so I would be interested to know what the options are for this problem in today's .Net world.
The project is using SQL server for the backend, linq-to-sql for the ORM and a C# asp.net front end.
What are my options for this?
Thanks
There are four main options here:
actually change the schema (DDL) at runtime - however, pretty much no ORM will like that, and generally has security problems as your "app" account shouldn't normally be redefining the database; it does, however, avoid the "inner platform" effect inherent in the next two
use a key-value store as rows, i.e. a Customer table might have a CustomerValues table with pairs like "dfeeNumber"=12345 (one row per custom key/value pair) - but a pain to work with (instead of a "get", this is a "get" and a "list" per entity)
use a single hunk of data (xml, json, etc) in a CustomFields single cell - again, not ideal to work with, but it easier to store atomically with the main record (downside: forces you to load all the custom fields to read a single one)
use a document database (no schema at all) - but then: no ORM
I've used all 4 at different points. All 4 can work. YMMV.
I have a similar situation on the project I'm working on now.
Forget about linq-to-sql when you are having a flexible database schema. There is no way to update the linq-to-sql models on the fly when the DB schema changes.
Solutions:
Keep an extra table with the table name the values belong to , column name , value etc
Totally dynamically change your table schema each time they add a field.
Use a NOSQL solution like mongoDB or the Azure Table Storage. A NOSQL solution doesn't require a schema and can be changed on the fly.
This is a handy link 2 read:
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:10678084117056
You're referring to an EAV model (entity-attribute-value).
Here's an article: http://hanssens.org/post/Generic-Entity-Attribute-Value-Model-e28093-A-POCO-Implementation.aspx
I'm wondering if the following DB schema would have repercussions later. Let's say I'm writing a place entity. I'm not certain what properties of place will be stored in the DB. I'm thinking of making two tables: one to hold the required (or common) info, and one to hold additional info.
Table 1 - Place
PK PlaceId
Name
Lat
Lng
etc... (all the common fields)
Table 2 - PlaceData
PK DataId
PK FieldName
PK FK PlaceId
FieldData
Usage Scenario
I want certain visitors to have the capability of entering custom fields about a place. For example, a restaurant is a place that may have the following fields: HasParking, HasDriveThru, RequiresReservation, etc... but a car dealer is also a place, and those fields wouldn't make sense for a car dealer.
I want to support any type of place, from a single table (well, 2nd table has custom fields), because I don't know the number of types of places that will eventually be added to my site.
Overall goal
On my asp.net MVC (C#/Razor) site, where I display a place, it will show the attributes, as a unordered list populated by: SELECT * FROM PlaceData WHERE PlaceId = #0.
This way, I wouldn't need to show empty field names on the view (or do a string.IsNullOrWhitespace() check for each and every field. Which I would be forced to do if every attribute was a column on the table.
I'm assuming this scenario is quite common, but are there better ways to do it? Particularly from a performance perspective? What are the major drawbacks of this schema?
Your idea is referred to as an Entity-Attribute-Value table and is generally bad news in a RDBMS. RDBMSes are geared toward highly structured data.
The overall options are:
Model the db further in an RDBMS, which is most likely if someone is holding back specs from you.
Stick with the RDBMS, using XML columns for the data whose structure is variable. This makes the most sense if a relatively small portion of your data storage schema is semi- or un-structured. Speaking from a MS SQL Server perspective, this data can be indexed and you can perform checks that your data complies with an XML schema definition.
Move to a non-relational DB such as MongoDB, Cassandra, CouchDB, etc. This is what a lot of social sites and I suspect blog sites run with. Also, it is within reason to use a combination of RDBMS and non-relational stores if that's what your needs call for.
EAV gets to be a mess because you're creating a database within a database and lose all of the benefits a RDBMS can provide (foreign keys, data type enforcement, etc.) and the SQL code needed to reconstruct your objects goes from lasagna to fettuccine to spaghetti in the blink of an eye.
Given the information that's been added to the question, it would seem a good fit to create a PlaceDetails column of type XML in the Place table. You could also split that column into another table with a 1:1 relationship if performance requirements dictate it.
The upside to doing it that way is that you can retrieve the data using very simple SQL code, even using the xml data type's methods for searching the data. But that approach also allows you to do the more complex presentation-oriented data parsing in C#, which is better suited to that purpose than T-SQL is.
If you want your application to be able to create its own custom fields, this is a fine model. The Mantis Bugtracker uses this as well to allow Admins to add custom fields to their tickets.
If in any case, it's going to be the programmer that is going to create the field, I must agree with pst that this is more a premature optimization.
At any given time you can add new columns to the database (always watching for the third normalization rule) so you should go with what you want and only create a second table if needed or if such columns breaks any of the normal forms.