I'm looking for a database solution in which some parts of my data may become unstructured.
More specifically, I'm using Entity Framework 6, but really I suppose this is more of a SQL rooted question.
Suppose I have a collection of generic geometric objects (GeoObj) that can be represented by an arbitrary amount of values (collection of int).
This is one solution, but I'm really not sure about the correctness/efficiency:
GeoObj
{
public int ID { get; set; }
public virtual ICollection<GeoValue> Values { get; set; }
}
GeoValue
{
public int ID { get; set; }
public int Value { get; set; }
}
Now the problem I see is that since I expect a lot of GeoObjs, and each GeoObj has a good amount of GeoValues, the GeoValues Table with get HUGE.
Will this slow down performance significantly?
Is there a better solution?
Thanks!
You're correct in your suspicion that the potential performance issue is ultimately rooted in your SQL database server. Are you using MS SQL Server?
The answer will be found in testing the performance of your specific version of SQL Server running on the hardware and configuration of the machine hosting the database.
As far as I know, Entity Framework doesn't have facilities that support control of table partitioning and things like that to work around the bottleneck of a HUGE table. That is in the scope of the database.
Check out https://msdn.microsoft.com/en-us/library/ms190787.aspx for some info on that.
Related
I have a question regarding the setup of foreign keys in entity framework 6. Our project stores data from a few other services (to have faster access to the data) and provides the users with charts and statistics depending on the stored data. For the storage of the data we´ve setup a cronjob which runs daily at about 3 AM.
Here are 2 example database models:
public class Project {
public string Id { get; set; }
public string Title { get; set; }
}
public class Issue {
public string Id { get; set; }
public string Title { get; set; }
[ForeignKey("Project")]
public string ProjectId { get; set; }
[ForeignKey("ProjectId")]
public Project Project { get; set; }
}
The problem now is for some issues we don´t save the project it depends on but we have to save the ProjectId (because at a later point it might be possible that the project exists in our database). So when I try to save this issues it tells me that I can´t save them because the project does not exist.
Is there any way I can tell entity framework that it doesn´t matter if the project exists or not? Currently I´ve just removed the ForeignKeys but this makes it very slow when I try to get the full list of issues with their projects.
Or is there any other way to read out all issues with their projects if there are no foreign keys? Currently I´m using a foreach loop to go threw each issue and then I search for the project but with more than 10.000 issues this get´s very slow.
The navigation property you've defined is requiring the data in the Project table in order to save an Issue. This isn't Entity Framework, this is a SQL Server foreign key constraint issue. Entity Framework is doing preliminary validation to not waste a connection that will ultimately fail. While you can turn off enforcing the foreign key constraint, there is not a good way to turn this validation off in Entity Framework
Keep in mind, having a foreign key does not mean it will help with your query's performance. It's simply a way to enforce referential integrity. I suspect that your real problem is the way you've written your query. Without seeing your query and metrics around "slow", it be hard to point you in the right direction.
I got a sqlite table in xamarain (native android / pcl):
[Table("Customer")]
public class Customer
{
[PrimaryKey, AutoIncrement]
public int Id { get; set; }
public Address Address{ get; set; }
}
"Address" represents a second table.
1) Is it possible to automatically create the "Address" Table when I call
connection.CreateTable<CustomerDto>();
because it is it's dependency?
2) Is it possible to use a LINQ expression which automatically maps the correct "Address" to this "Customer?
In my .NET Standard library I'm using:
"sqlite-net": "1.0.8"
"sqlite-net-pcl": "1.3.1"
My approach was to create "initial state models" of all the tables, marked as abstract (so there is no risk that somebody could instantiate them), defining only the fields necessary in the database and the primary keys (GUID in my case), used only to create tables at the beginning. Following modification to the data structures always with ALTER instructions.
In another namespace a duplication of all the models, this time with getters/setters and other utilities, and I used these as "real models".
For representing linked models I used a field as Id and another one as model (refreshed when necessary):
public int IdAddress { get; set; }
[Ignore]
public Address Address { get; set; }
I don't think sqlite-net can do what you are asking because it's a very lightweight orm, and even if it could I prefer don't automate too much because of my past experiences with Hibernate.
https://github.com/praeclarum/sqlite-net
https://components.xamarin.com/view/sqlite-net
It sounds like you should look at using Entity Framework because that will allow you to use LINQ with SQLite. The standard library on the web (not Entity framework) is very light and doesn't have much functionality for the ORM like functionality you are looking for.
If you're looking for a more lightweight library, you can use this, but it will not allow you to write LINQ expressions without writing your own ORM:
https://github.com/MelbourneDeveloper/SQLite.Net.Standard
We're currently trying SQLite Extentions (PCL) as an ORM.
We're wondering if the mapping is supposed to build a SELECT with INNER JOINs on children if they are correctly configured in the entity?
public class Project
{
[PrimaryKey]
public long Id { get; set; }
[ForeignKey(typeof(EnterpriseClient))]
public long EnterpriseClientId { get; set; }
[ManyToOne]
public EnterpriseClient EnterpriseClient { get; set; }
[OneToMany(CascadeOperations = CascadeOperation.All)]
public List<WorkOrderHead> WorkOrderHeads { get; set; }
}
If we get all the Projects with GetAllWithChildren:
var x = _db.GetAllWithChildren<Project>(p => true);
Our result is multiple select for each child (EnterpriseClient) and we were hoping that it would en in one select and a join to collect all the data at once.
Is our configuration wrong or it's supposed to be that way?
Right now SQLite-Net Extensions performs a SELECT for each property to be fetched and also suffers from the N+1 issue in read operations (it is already solved for write operations). It's implemented as a very thin layer over SQLite.Net providing you some convenience methods for accessing entity relationships. Currently it works the way you described as an intended behavior. Accessing registers by primary key or an indexed property it's very fast, and performance is not an issue for small databases like the used in most mobile projects.
SQLite-Net Extensions is an evolving project, so feature requests (and pull requests, of course) are always welcome. However, INNER JOINs would break the SQLite.Net mapping, so a single SELECT returning all the required information would require re-implementing the SQLite.Net mapping mechanism.
It is theoretically possible to workaround the N+1 issue performing a single SELECT for each property, so recursive TO-MANY operations would see a performance improvement. I've created an issue to keep track of this request.
Happy coding!
I'm using Stack Exchange .Net Redis provider to store and retrieve values. I would like to know how can I search for certain records inside Redis (like any database, search needs to be executed in Redis instance not in .Net application)
Example:
public class Employee
{
public string FirstName { get; set; }
public string LastName { get; set; }
public int Age { get; set; }
public int Salary {get;set;}
}
If I have 100,000 records of employees stored as .Net "List<Employee> lstEmployee = new List<Employee>();" in Redis cache server and would like to fetch only the record whose age >50 and salary > 5000, how should I code for it?
Disclosure: I'm just getting started with Redis using this example.
First, a "cache server" is not intended to be used as a queryable store. If we assume instead that you mean simply a nosql backend, then ... well, frankly, that doesn't sound like the sort of query I would try and do via redis. The point of redis is that you build whatever indexes you need yourself. If you want ordered range queries (the age / salary), then a sorted set and ZRANGEBYSCORE is probably a viable option; however, intersecting these two queries is more difficult. You could try asking the same question ib the redisdb google-group, but just as a general redis question - not specific to any client library such as SE.Redis. If the operations exist ib redis, then you can use the client library to invoke them.
I'm wondering, however, whether "elastic" might be a better option for what you describe.
I am currently developing a web service which provides basic CRUD operations on business objects. The service will be used by legacy applications which currently use direct database access.
I decided to use ServiceStack instead of WCF due to ServiceStacks great architecture.
However know I am trying to decide wether to use OrmLite, nHibernate or Entity Framework to access the existing legacy database.
Requirements for the ORM are as follows
Support for joins
Support for stored procedures
I already tried OrmLite (as it's fast and already included with ServiceStack). The only way I managed to join two tables was by using SQL (not an option). Is there any better way?
// #stackoverflow: This is my POCO DTO
public class Country
{
public long Id { get; set; }
public string Alpha2 { get; set; }
public string Alpha3 { get; set; }
public string ShortText { get; set; }
public string LongText { get; set; }
}
public class CountryRepository : ICountryRepository
{
// #stackoverflow: This is the query to join countries with translated names stored in another table
private const string CountriesSql =
#"SELECT C.Id, C.Alpha2, C.Alpha3, L.ShortText, L.LongText FROM COUNTRY AS C INNER JOIN LOCALIZATION AS L ON C.LocId = L.Id WHERE (L.Lang_Id = {0})";
private const string CountrySql = CountriesSql + " AND C.Id={2}";
private IDbConnection db;
public IDbConnectionFactory DbFactory { get; set; }
private IDbConnection Db
{
get { return db ?? (db = DbFactory.Open()); }
}
public List<Country> GetAll()
{
return Db.Select<Country>(CountriesSql, 0);
}
public Country GetById(long id)
{
return Db.SingleOrDefault<Country>(CountrySql, 0, id);
}
}
The example above shows one of the simple business objects. Most others require Insert, Update, Delete, multiple Joins, and Read with many filters.
If all you need are joins (lazy-loading or eager loading) and stored procedure support and want to get setup quickly then Entity Framework and nHibernate are great options. Here is a cool link about EntityFramework and the repository and unit of work pattern. http://blogs.msdn.com/b/adonet/archive/2009/06/16/using-repository-and-unit-of-work-patterns-with-entity-framework-4-0.aspx
If you are very concerned with performance and want more control over how your classes will look (ie POCOs) and behave then you can try something more lightweight like ORMLite or Dapper. These two are just thin wrappers with less features but they will give you the best performance and most flexibility -- even if that means writing some SQL every once in a while.
You can also use hybrid approaches. Don't be afraid to mix and match. This will be easiest when using POCOs.
I think the important thing is to code for your current database and current needs. However, to do so using proper interfaces so if the time came to switch to a different database or storage mechanism then you simply have to create a new data provider and plug it in.
Ormlite supports primitive Join functions using expressions. The new JoinSqlBuilder class can help with this. For SPs, I have added a new T4 file to generate corresponding c# functions. Currently the SP generation code supports Sql Server alone; if you are using any other db, you can easily add support for it.
You might consider LLBLGen Pro -- it's got great support for database first design and also has designer tools that speed up getting started if you use nHibernate or EF. But it is $$.
http://llblgen.com
As a follow up to this Matt Cowan has created an AWESOME template generator for building this sort of thing with LLBLGen. Check out the blog post here:
http://www.mattjcowan.com/funcoding/2013/03/10/rest-api-with-llblgen-and-servicestack/
and demo here:
http://northwind.mattjcowan.com/
The demo is entirely autogenerated!
Also check this comparison from an OO perspective between NHibernate 3.x and Entity Framework 5/6
http://www.dennisdoomen.net/2013/03/entity-framework-56-vs-nhibernate-3.html