We Keep Coding

Run linq-to-sql queries on database with changing structure

Is it possible to run Linq-to-SQL queries when underling database structure is changing from time to time (I mean database updates that happens due to business requirements and since database is shared among several apps It may be happens without announcements to me)?
Is there any way that I can connect to new database structure in Linq-to-SQL without updating the .dbml file in my source code?
If I want to run raw queries knowing that my database structure changes during time, can I use any of Linq-to-SQL benefits somehow?

Provided the structure you have in your classes match to your tables (at least covering all the fields you need) you can do that. ie: Northwind customers table have more than 4 fields in reality. Provided below 4 are still in that table this would work:
void Main()
{
DataContext db = new DataContext(#"server=.\SQLexpress;trusted_connection=yes;database=Northwind");
Table<Customer> Customers = db.GetTable<Customer>();
var data = Customers.Where(c => c.Country == "USA");
foreach (var customer in data)
{
Console.WriteLine($"{customer.CustomerID}, {customer.CompanyName}");
}
}
[Table(Name = "Customers")]
public class Customer
{
[Column]
public string CustomerID { get; set; }
[Column]
public string CompanyName { get; set; }
[Column]
public string ContactName { get; set; }
[Column]
public string Country { get; set; }
}
For raw SQL, again you could use a type covering fields in select list or dynamic.
Note: For inserts, for this to work, fields that are not in your model should either accept null or have default values.

EF & Web API 2 Multiple Round trips to the database to populate object model

It seems this problem would have been encountered before me but I'm not finding much help online probably because I don't really know what to search for.
My problem in short is that I have a db table. That table has 5 keys to other tables.
I then have a model that represents this table in EF. Of course this object that represents the db table has List<T> properties that are representations of foreign keys in the db. That doesn't seem to be the problem as much as the EF model that has this table representation but also List<T> properties to other models.
The problem I am experiencing is that a call to a stored procedure to populate the main modelforces additional calls to the db to populate the related List<T> models.
I am looking to improve performance namely by eliminating the multiple calls.
My only thought to this point is to modify the stored procedure to return multiple recordsets and match each List<T> property to its corresponding recordset.
My sterilized structure is something like this.
DB:
sql_Id Int PK
sql_Status Int FK
sql_Reason Int FK
sql_GuestId Int
sql_Name varchar
sql_Created DateTime
sql_Original Int FK
EF:
public class OrderHeader : ClassBase
{
public OrderHeader()
{
TaskCodeAssignments = new List<OrderHeaderTaskCodeAssignment>();
StatusReasonCode = new OrderHeaderStatusReasonCode();
StatusCode = new OrderHeaderStatusCode();
Links = new OrderHeaderLinks();
}
public int OrderHeaderID { get; set; }
public short OrderHeaderStatusCodeID { get; set; }
public short? OrderHeaderStatusReasonCodeID { get; set; }
public short? OriginatingApplicationId { get; set; }
public string CustomerFirstName { get; set; }
public string CustomerLastName { get; set; }
public OrderHeaderStatusCode StatusCode { get; set; }
public OrderHeaderStatusReasonCode StatusReasonCode { get; set; }
public CustomerStatusCode CustomerStatusCode { get; set; }
public ICollection<OrderHeaderTaskCodeAssignment> TaskCodeAssignments { get; set; }
}
public class OrderHeaderStatusCode
{
public OrderHeaderStatusCode()
{
OrderHeaderStatusReasonCodes = new List<OrderHeaderStatusReasonCode>();
}
public ICollection<OrderHeaderStatusReasonCode> OrderHeaderStatusReasonCodes { get; set; }
public virtual ICollection<OrderHeader> OrderHeader { get; set; }
}
The other custom types like OrderHeaderStatusReasonCode are pretty similar in design so I'm leaving out for brevity.
C# Web API
public async Task<IHttpActionResult>GetOrdersHistory([FromUri]GetOrderRequestParameters orderParams)
{
....removed for brevity....
var query = await TheOrderRepository.GetOrderHistory(getOrder);
}
Order Repository:
public async Task<IQueryable<OrderHeader>> GetOrderHistory(GetOrderParameters orderParams)
{
// this is the call to stored procedure that I would modify to return multiple recordsets
var storedProcedure = StoredProcedure.Name.MyStoredProc.ToString();
var ordersHistory = await dbctx.Database.SqlQuery<OrderHeader>(...), storedProcParam).ToListAsync();
// now I jump off to fill in the other properties and their data has to come from the db
await GetOrdersData(ordersHistory, orderParams.Include);
}
private async Task GetOrdersData(List<OrderHeader> ordersHistory)
{
if (ordersHistory != null)
{
await LoadOrderStatusCodeForList(ordersHistory);
await LoadOrderStatusReasonCodeForList(ordersHistory);
await LoadCustomerStatusCodeForList(ordersHistory);
await LoadOrderHeaderTaskCodeAssignmentsForList(ordersHistory);
await LoadOrderHeaderTaskCodeForList(ordersHistory);
}
}
Again most of these awaits are similar so I'm just going to give an example of one...
private async Task LoadOrderStatusCodeForList()
{
....snipped for brevity...
await LoadOrderStatusCode(order.OrderHeaderStatusCodeID));
}
private async Task<OrderHeaderStatusCode> LoadOrderStatusCode(short orderHeaderStatusCodeId)
{
....snipped brevity....
var storedProcedure = StoredProcedure.Name.MySprocStatusCode.ToString();
return await _dbctx.Database.SqlQuery<OrderHeaderStatusCode>(...), ...).FirstOrDefaultAsync();
}
EDIT:
The crux is this. OrderHeader has properties with a custom type and basically those custom types have a List<T> that has to be populated. My current design is such that I repeatedly hit the db to populate those custom types List properties.
Is there a way to make one trip to the db to get all my information. As mentioned earlier the only way I can think of is to modify the stored procedure to return multiple record sets and then match them up.
BTW the architecture may be the flaw...in which case educate me on how to properly populate a complex object like this.
TIA

The root problem is that stored procedures aren't composable. In SQL you can't join a stored procedure call with anything (a database table or another stored procedure). So EF can't do that either.
If you want to get data with loaded collections from the database, normally you'd have to use Includes. EF will translate that into the appropriate joins and figure out how to load the entities and their collections from one big result set. But, as said, joins are no option here.
There is a way to load multiple result sets from one stored procedure. IMO it's pretty messy and very procedural. I would keep loading the data separately as you do now, if you want to keep using stored procedures. Others may suggest that you could load the additional data by lazy loading. Unfortunately that's not as straightforward as it should be with SqlQuery.
Another option of course is to start using regular DbSets (with Includes), but I can't judge if that's possible for you.

Entity Framework - Non Key Relationships

Problem
I have a situation whereby I need to use Entity Framework 6, Code First, with a legacy database structure which cannot be changed. The database has a very generic table which stores text based data alongside some non key data which can be used to relate the record back to another table.
To illustrate:
Assume the Notes table has a model as follows:
[Table("Notes")]
public class Notes
{
[Key]
public int RecordId { get; set; }
[Required]
public string RelatedTableName { get; set; }
[Required]
public int RelatedTableRecordId { get; set; }
[Required]
public string NotesText { get; set; }
}
I then have another model which could look like so:
[Table("Drivers")]
public class Drivers
{
[Key]
public int RecordId { get; set; }
[Required]
public string DriverName { get; set; }
public ICollection<Notes> DriverNotes { get; private set; }
}
There is no foreign key which links the tables. The Drivers table is linked to the Notes table by way of the RelatedTableName and RelatedTableRecordId fields.
I do not have a problem reading data from the database and hydrating the models using entity framework.
The problem I have is that I want to be able to save a new Driver and its newly created Notes in one transaction and have the RelatedTableRecordId field set to the primary key of the Driver.
If a foreign key existed entity framework would know to back fill the property but in this case it doesn't know about the relationship.
Key Points
Database Structure must not change.
Must use Entity Framework 6 Code First
Must be able to use an Execution Strategy.
Require a relationship between non key fields.
Need to be able to persist all data in a single transaction.
What I've Tried
I had a similar issue with Audit type data and solved it by doing something similar to the following (note that this is very pseudo here):
public override int SaveChanges()
{
int changes = 0;
//Disable the current execution strategy as the default ones do not support user instantiated transactions.
this.ContextConfiguration.SuspendExecutionStrategy();
try
{
//Wrap a whole transaction inside an execution strategy so that auditing can be combined with regular saving of changes.
this.ExecutionStrategy.Execute(
() =>
{
using (var transaction = this.Database.BeginTransaction())
{
//Reset the change count so that it doesn't increase each time the transaction fails.
changes = 0;
//Remove any audit records created by previous failed transactions.
this.AuditTableChanges.Local.Clear();
//Evaluate the change tracker to identify entities which will potentially require an audit trail.
var insertedEntities = this.ChangeTracker.Entries().Where(entryEntity => entryEntity.State == EntityState.Added).ToList();
//Save all changes to get identities.
changes = base.SaveChanges();
//Create the audit trail for inserted entities. This step must occur after the initial call to SaveChanges() so that the identities are set.
foreach (DbEntityEntry entryEntity in insertedEntities)
{
//For each inserted record, get the audit record entries and add them
foreach (AuditTableChange auditTableChange in GetAuditRecords(entryEntity, AuditTableChangeType.Insert).Result)
this.AuditTableChanges.Add(auditTableChange);
}
//Save the audit trail for inserted entities.
changes += base.SaveChanges();
//Commit all changes to the database
transaction.Commit();
}
});
}
finally
{
//Re-enable the execution strategy so that other calls can benefit from the retry policy.
this.ContextConfiguration.UnSuspendExecutionStrategy();
}
return changes;
}
This worked fine for the Audit data as the implementation was hidden away in the framework. I do not want my development team to have to do all of the above each time they persist records.
In its simplistic form this is as much as I'd want people to be doing:
public void CreateDriver()
{
using (MyContext context = new MyContext())
{
Drivers driver = new Drivers();
driver.DriverName = "Joe Bloggs";
Notes driverNote = new Notes();
driverNote.RelatedTableName = "Drivers";
driverNote.NotesText = "Some very long text";
driver.DriverNotes.Add(driverNote);
context.Drivers.Add(driver);
context.SaveChanges();
}
}
In a way I want a foreign key which exists in code but not in the database so that entity framework knows to fill in the RelatedTableRecordId field. I've read some articles on hacking the EDMX but this project is purely Code First only.
There are older questions on stack overflow which are similar but relate to older versions of entity framework and don't help much or have as much detail as the above.
I'm hoping that someone may have experienced a similar problem and has an answer which may involve perhaps some custom mapping/metadata or some overrides to entity framework logic.
Any help would be appreciated.
Thanks,
Greg

Entity Framework inheritance

SQL Layer:
I have a table
Entity Framwork Layer:
I have the following rule: all Offers, which have State is null, are Outstanding offers, State is true are Accepted offers, State is false are Declined offers. Also, part of fields used only for Outstanding, part - only for Accepted etc... I use Database first approach, so, I updated EF model from DB and renamed Offer entity to OfferBase and created 3 child classes:
it works fine for add/select entities/records. Right now I want to "move" offer from outstanding to accept offer, so, I need to set Status=true (from Status is null) for appropriate record. But how to do it by Entity Framework? If I try to select outstanding offer as Accept offer I get an null reference (and clearly why)
// record with ID=1 exists, but State is null, so, EF can not find this record and offer will be null after the following string
var offer = (from i in _db.OfferBases.OfType<EFModels.OfferAccepted>() where i.ID == 1 select i).FirstOrDefault();
if I try to select as OfferBase entity I get the following error:
Unable to cast object of type
'System.Data.Entity.DynamicProxies.OfferOutstanding_9DD3E4A5D716F158C6875FA0EDF5D0E52150A406416D4D641148F9AFE2B5A16A'
to type 'VTS.EFModels.OfferAccepted'.
var offerB = (from i in _db.OfferBases where i.ID == 1 select i).FirstOrDefault();
var offer = (EFModels.OfferAccepted)offerB;
ADDED NOTES ABOUT ARCHITECTURE:
I have 3 types of Offer entity. There are: AcceptOffer, DeclineOffer and OutstandingOffer.
AcceptOffer:
UserID
ContactID
Notes
FirstContactDate
LastContactDate
[... and 5-10 the unique fields...]
DeclineOffer:
UserID
ContactID
Notes
[... and 5-10 the unique fields...]
OutstandingOffer:
UserID
ContactID
FirstContactDate
LastContactDate
[... and 5-10 the unique fields...]
How to do it correctly? Of course, I can select a record, remove from DB and add new with appropriate state value, but how to do it normally?

You can't change the type of an object once it's created. Your object model seems wrong.
Either you delete the outstanding offer and create an accepted offer from it (looks like what you are currently doing) but you may lose relations as you created a new object with a new identity (you can also copy them before removing the old object). Or you want to keep the same object and change its state.
If you want to keep the same identity then preffer composition over inheritance.
Your code could look like this :
public class Offer
{
public int Id { get; set; }
public virtual OfferState State { get; set }
}
public class OfferState
{
public int OfferId { get; set; }
public string Notes { get; set; }
}
public class AcceptedOfferState : OfferState
{
public DateTimeOffset AcceptDate { get; set; }
}
public class DeclinedOfferState : OfferState
{
public DateTimeOffset DeclinedDate { get; set; }
}
If you still want to change the type of the object and keep its identity then you may use stored procedures ; as stated by Noam Ben-Ami (PM owner for EF) : Changing the type of an entity.

Rather than trying to add these custom classes to your entity framework model, just create them as normal c# classes and then use a projection to convert from the entity framework generated class to your own class e.g.
var accepteOffers= from i in _db.Offers
where i.ID == 1 && i.Status == true
select new OfferAccepted { AcceptDate = i.AcceptDate, StartTime = i.StartTime /* map all releaveant fields here */};

C# ServiceStack OrmLite GetDialectProvider.ToAddForeignKeyStatement

Just recently tried OrmLite for MySql in a C# console project:
I'm loving it, since it's easy to implement with MySql, in contrast to Entity Framework. However I'm trying to enable automatic updates of the database schema from my model. I extended the code given in the answers in:
With OrmLite, is there a way to automatically update table schema when my POCO is modified?
And came up with a solution that can add and delete columns in your database schema, depending on the model. Works pretty good, and utilizes a lot of the features OrmLite.ModelDefinition gives.
See the solution here:
https://github.com/contradel/Extending-NServiceKit.OrmLite
However I can't get it to automatically create foreign keys with GetDialectProvider.ToAddForeignKeyStatement(). And I can't find any documentation. Does anyone know how it works?
I would like to be able to do this:
public class Order
{
//Try to add or comment out properties here, SQL will be generated automatic
[AutoIncrement]
public int Id { get; set; } //pk
public DateTime? OrderDate { get; set; }
[References(typeof(Customer))] //Creates Foreign Key
public int CustomerId { get; set; }
}
And then:
foreach (FieldDefinition field in missingOnDb)
{
//if (field != ForeignKey)
var addSql = string.Format(db.GetDialectProvider().ToAddColumnStatement(typeof(T), field));
//else
//addSql = db.GetDialectProvider().ToAddForeignKeyStatement(??,??,??,??,??);
Console.WriteLine(addSql);
db.ExecuteSql(addSql);
}