Say I have the below entities. (Heavily Simplified for brevity, but the key properties are included)
public class Crime
{
[Key]
public int CrimeId {get;set;}
public virtual ICollection<Victim> Victims {get;set;}
public virtual CrimeDetail Detail {get;set}
}
public class Victim
{
[Key]
public int VictimId {get;set;}
public string VictimCategory {get;set;}
}
public class CrimeDetail
{
[Key]
public int id {get;set;}
public string DetailText {get;set;}
}
I have 600,000+ of these records to insert into SQL Server 2008 Express R2, which takes quite some time using Entity Framework 4.4.
Ideally I'd like to use SQLBulkCopy to batch insert these records, but since that doesn't support complex types (At least not out of the box), then I'm trying to find some other potential solutions.
I'm open to changing my model if necessary.
EDIT: would the AsDataReader Extension method from MSDN help in this instance?
When having the same issue we end up having code-first generated database with EF and strongly typed generated datasets to be used for SQLBulkCopy.
(We never really coded those classes, they were generated using xsd util from xsd definition of 1-10gb xml file. I'm trying to recall right now when we havent generated typed datasets from the same xsd, but that seems irrelevant to your issue.)
Depending on how you are getting those 600k+ records you either can change the code to use generated strongly-typed datasets or use some object-to-object mapper to map your EF POCO objects to datasets as properties going to be named the same.
Here is a link on generating strongly typed datasets.
Here is an example how to use SqlBulkInsert.
Related
I have a class like this:
public class GeneralClass
{
public int Id {get; set;}
public string Name {get; set;}
}
which is a pretty generic class. I also have a derived class like this:
public class DerivedClass: GeneralClass
{
public int SpecificProperty {get; set;}
public string AnotherSpecificProperty {get; set;}
public bool BooleanSpecificProperty {get;set;}
}
The problem is, I can have lots of derived classes (more than 10) in my app, that´s why inheritance in a database IS NOT AN OPTION.
The solution I came up with is to have GeneralClass as a table, with an XML column containing the specific properties.
Something like this:
CREATE TABLE
General(Id int primary key,
Name nvarchar(50),
SpecificProperties xml);
where the specific properties contain the properties of the derived class.
The question is: How can I SAVE and QUERY this xml column using Entity Framework, and deserialize the xml into the properties?
"...I can have lots of derived classes (more than 10) in my app, that´s why inheritance in a database IS NOT AN OPTION" - why not!?
There are two main inheritance patterns - table per hierarchy (TPH) and table per type (TPT). While the first one might seem quite a waste of SQL resources, the second one perhaps will match your requirements. There is one more pattern - table per concrete type (TPC), which may be considered a variation of TPT. You do not have abstract classes in your hierarchy, or at least the code snippet you've posted does not, so it is quite natural to use
TPT and query the way you're very familiar with, say LINQ to entities. More information about EF and inheritance patterns here and here.
Still, if you do insist on using XML you may declare the field as XML type (this is a column data type, which SQL server supports). You can query it like described here, here, and here.
I got a sqlite table in xamarain (native android / pcl):
[Table("Customer")]
public class Customer
{
[PrimaryKey, AutoIncrement]
public int Id { get; set; }
public Address Address{ get; set; }
}
"Address" represents a second table.
1) Is it possible to automatically create the "Address" Table when I call
connection.CreateTable<CustomerDto>();
because it is it's dependency?
2) Is it possible to use a LINQ expression which automatically maps the correct "Address" to this "Customer?
In my .NET Standard library I'm using:
"sqlite-net": "1.0.8"
"sqlite-net-pcl": "1.3.1"
My approach was to create "initial state models" of all the tables, marked as abstract (so there is no risk that somebody could instantiate them), defining only the fields necessary in the database and the primary keys (GUID in my case), used only to create tables at the beginning. Following modification to the data structures always with ALTER instructions.
In another namespace a duplication of all the models, this time with getters/setters and other utilities, and I used these as "real models".
For representing linked models I used a field as Id and another one as model (refreshed when necessary):
public int IdAddress { get; set; }
[Ignore]
public Address Address { get; set; }
I don't think sqlite-net can do what you are asking because it's a very lightweight orm, and even if it could I prefer don't automate too much because of my past experiences with Hibernate.
https://github.com/praeclarum/sqlite-net
https://components.xamarin.com/view/sqlite-net
It sounds like you should look at using Entity Framework because that will allow you to use LINQ with SQLite. The standard library on the web (not Entity framework) is very light and doesn't have much functionality for the ORM like functionality you are looking for.
If you're looking for a more lightweight library, you can use this, but it will not allow you to write LINQ expressions without writing your own ORM:
https://github.com/MelbourneDeveloper/SQLite.Net.Standard
I used EF6 Database First tools to generate C# classes for 2 tables from my database, then (as advised in the blog post that helped me through the steps to do that) copied the resulting .cs files into a new project. I made a few edits to the classes to support sensible names in my C# code. Here's a snippet of one of the classes with "LongTableName" replacing a strangely long name used in the database.
namespace RidesData
{
[Table("LongTableName")]
public partial class PhoneData
{
[Key]
[Column("LongTableNameID")]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int ID { get; set; }
[Column("LongTableNameAccountID")]
public int AccountID { get; set; }
// more fields
}
}
I am not in control of the table names, nor the fact that the many of the column names have the table name as prefixes. But the Code First ideas in EF6 should, I thought, let me use reasonable class and field names despite that. (The Database First code generator did a good job of adding code to OnModelCreating to specify that none of the columns corresponding to C# string data used Unicode.)
My model (generated by the EF6 tools and that inherits from DbContext) includes (after some renaming by me)
public virtual DbSet<PhoneData> PhoneRecs { get; set; }
and I thought all would be fine when I created an instance of PhoneData, populated it, and did
Model.PhoneRecs.Add(phoneData);
but the first thing that happened when I ran the code -- well before any call to SaveChanges() -- was that EF generated CREATE TABLE statements for the two tables; the table corresponding to the snippet above was named PhoneDatas (not using the specified table name) and the column names were the same as the field names in the class (not what was specified in the Column(...) attributes).
Of course the table I had specified did not need to be created. EF just had to grok that I wanted to use the table and column names I had specified via attributes.
I did not expect this failure of explicit Code First attributes. Does anyone have a clue why this isn't doing what I want, or how to fix it? (Do I have to do something to specify the table & column names in OnModelCreating as well as -- or instead of -- the attributes?)
Note that the project that I copied these classes into had never "seen" the database before. There are no vestiges of any "models" left over from tooling having looked at the database. Also, I hope it does not matter that I've tried to keep things on .Net 4.0 (avoiding going to 4.5 in this code).
Any assistance would be appreciated.
I'm not a big fan of DataAnotations either. Use EntityTypeConfiguration. It gives you the naming flexibility I think you are looking for.
Example.
public class PhoneData
{
public int ID {get;set;}
public string SomeProperty {get;set;}
}
public class PhoneDataMap : EntityTypeConfiguration<PhoneData>
{
public PhoneDataMap()
{
ToTable("WhatEverYou_Want_to_call_this");
HasKey(m => m.Id);
Property(m => m.SomeProperty).HasColumnName("whatever").IsRequired();
//etc.
}
}
Then in your on ModelCreating you add
modelBuilder.Configuration.Add(new PhoneDataMap());
On a side note, if you are having trouble with pluralization of your table names you can add this to OnModelCreating as well
modelBuilder.Conventions.Remove<PluralizingTableNameConvention>();
I got a code first EF and I want to use native sql for the more complex select statements.
When I try to execute:
using (VaultsDbContext db = new VaultsDbContext())
{
var contracts = db.Contracts.SqlQuery("select * from Contracts").ToList<Contract>();
}
I got:
Cannot create a value for property 'MetaProps' of type
'DskVault.Models.DbModels.MetaProps'. Only properties of primitive or
enumeration types are supported.
MetaProps is a class that holds deleteflag, creator etc. and it's a property of all my classes. It's not mapped to a different table, every table has deleteflag, createor, etc.
public class Contract
{
public long Id { get; set; }
...
public MetaProps MetaProps { get; set; }
}
Is there a way to map from the native sql to the class if the class contains a complex type or does EF not support that? Also what if the complex type is entity mapped to another table(join)?
Edit:
Version: Entity Framework 6
I know from experience not all the fields in your table have to be contained in your model. This is a good thing when it comes to installing updates into production.
Have you tried reverse engineering your tables on a SEPARATE temporary project using the Entity Framework Power tools? This is a Nuget package that I have found to be extremely useful in code first programming. Reverse engineering will overwrite existing files, so make sure not to do this on your live code.
I just learned about a genius type that would simplify a lot of my work but it looks like my preferred ORM does not recognize it.
Is there a workaround to let ServiceStack OrmLite recognize HierarchyId in SQL Server? Any suggestions about which files to modify and any hints how to proceed?
EDIT :
Here is a better illustration of the problem. I have the following class:
public class MyClass
{
public int Id { get; set; }
public SqlHierarchyId HierarchyId { get; set; }
}
SqlHierarchyId is a custom SQL Server data type. OrmLite will generate the following class for it:
Funny enough, I can use the [StringLength(255)] attribute on the property and it will get the varchar(255) type instead:
I manually changed the table here and added the column data type to showcase the difference. Please note the data type of the third column:
Having a varchar representation is perfectly fine with other DBMS as it can be converted within C#, but with SQL Server it is preferable to have it match the corresponding data type. This will make the creation of views easier (due to the built-in functions of the hierarchyid data type).
I know the type is not supported by EF4 (not sure about 5). I also browsed the OrmLiteDialectProviderBase.cs file on GitHub and I can see a list of supported ADO.NET data types.
My simple question is: Is this a strong limitation by ADO.NET or this can be seen sometime in OrmLite? I am willing to help extending this part if any suggestions are made.
ADO.NET has support for the hierarchyid type, an example can be found here and shows ADO.NET can read values from Sql Server as a hierarchyid directly but you need to pass parameters to the server as a string.
Adding support for the hierarchyid type methods to a ORM framework would break the abstraction between the ORM API and the RDMS. I would assume this is the reason such functionality has not been added to Entity Framework.
You could work around the issue by keeping a string representation of the hierarchy in your database and having the hierarchyid version as a computed property in both your database and your C# class, you would need to exclude the computed C# property from the ORM mapping.
For example your table column would be declared as:
[SqlHierarchyId] AS ([hierarchyid]::Parse([HierarchyId])) PERSISTED
and your class as:
public class MyClass {
public string HierarchyId {
get;
set;
}
[Ignore]
public SqlHierarchyId SqlHierarchyId {
get {
return SqlHierarchyId.Parse(new SqlString(HierarchyId));
}
set {
HierarchyId = value.ToString();
}
}
}
This will persisted updates from the .Net layer and allow you to use the hierarchyid methods to construct queries in SQL Server and work with materialised objects in the .Net layer.
You would have to construct queries against the string representation in you ORM layer but this could still leverage some of the hierarchyid helper methods, for example:
public IEnumerable<MyClass> GetDescendants(MyClass node) {
string currentLocation = node.HierarchyId;
string followingSibling
= node.SqlHierarchyId.GetAncestor(1)
.GetDescendant(node.SqlHierarchyId, SqlHierarchyId.Null)
.ToString();
return db.Select<MyClass>(n => n.HierarchyId > currentLocation
&& n.HierarchyId < followingSibling);
}
Aplogies if I have got the ORMLite syntax wrong.