I have data stored in a MySQL database and I'm accessing it through entity framework 6. What I want to do is to extract the data and store it in a server-independent manner, so that I can use this data in the future for bootstraping of a larger database (where the data from MySQL is only a small portion of). The target database will not be MySQL.
The MySQL data was there before, so I used Database First. For the new database I use Code First. The MySQL data also needs some clean-ups, what is done in code. So the workflow is:
Read data from MySQL with Entity Framework database first
Optimize/enhance/correct it in code
Store in another database using code first
So my first thought was using SQLite, but sadly, Code First is not supported.
Using some other format (i.e. XML/DataSets/CSV) to store the data would require me to reimplement the import code.
Right now I didn't find another embedded database with Entity Framework support. As Entity Framework is meant for relational databases, I would say that Redis (or other NoSQL dbs) won't do the trick.
I also thought if "converting" H2 using IKVM.NET and then using H2 as SQLite replacement would be an option -- but there's the risk that this might fail due to some internal dependency not supported by IKVM.NET.
So I would be glad to get answers to the following questions:
Would the H2 way work? Is there even a stable/working ADO.NET provider?
I wouldn't care much about the local storage format (could also be JSON etc.) -- is there anything not-SQL, but providing Enitity Framework support?
Did I miss something? Is there a way I just didn't see yet?
Here are some related questions I stumbled upon which deal with the possibility of using Entity Framework to use files:
Entity Framework with text files (no database!)
A list of Entity Framework providers for various databases
Entity Framework with XML Files
Microsoft Entity Framework using a flat file as a data source
There is no complete solution, but there are many ready-made parts:
This project wraps the compiled Jar of H2 after IKVM.Net
with classes that implement the ADO.Net interface
to allow for easy use in .Net projects:
https://code.google.com/p/h2sharp/
EF provider for SharpHSQL (which doesn't work, because, SharpHSQL is too old, it's port of Hypersonic 1.4):
https://github.com/ArsenShnurkov/SharpHSQL/tree/master/src/SharpHsql.Linq
here is the description of build process:
https://groups.google.com/forum/#!topic/h2-database/QAvFqbyd4_0
https://code.google.com/p/h2sharp/wiki/BuildingH2Sharp
The authors of above code did a great job of writing such huge amount, so i think it's nothing wrong with writing some more to finish your task.
Related
So I am using Neo4J and thought about how to update existing databases with schema changes e.g. a Node has new properties, edges changed, values changed, etc.
So far I was only using SQL Server and the Entity Framework includes a migration tool that can automatically write migrations when the schema changes.
How is this done in Neo4J?
E.g. a customer has v. 1.0 from our software and neo4j database. We make changes for v.1.1 and now we need to update the db from all our customers.
I am using Neo4J in C#. I have so far only found something similar for Ruby: https://neo4jrb.readthedocs.io/en/8.2.x/Migrations.html
You can use: https://www.liquigraph.org/.
Where you can create the cypher queries you need to migrate database data.
Since neo4j is basically "schema-less" (in the relational DB sense), there is no tool for migrating to a new neo4j "schema".
You will have to write code to update the DB.
I do actively maintain Neo4j-Migrations:
https://github.com/michael-simons/neo4j-migrations
It's basically an automated script runner, that stores schema versions inside the same database as a subgraph. It is also able to store them separately in a Neo4j server supporting multi databases.
As you mention that you are using C#: This tool is Java based, but I do also provide native binaries for all 3 major OSses availabe on GitHub. You might can integrate them into your workflow.
I'm a very beginner in .NET and now I'm developing a little project (web API) using NancyFX framework. In my project, I need to use SQL database for some very basic tasks like storing registered users' details or getting some user information. I'd like to know what is the most popular, convenient and modern way of using SQL in .NET for beginners? I mean, should I use LINQ or just pure SQLClient functionality or are there any good libraries for working with SQL on .NET? I've tried to implement LINQ to SQL pattern but ended up with huge chunks of unused auto generated code and even bigger mess in my head...
For a framework to communicate with you're database I would recommend using Entity framework, its very convenient and easy and has the Code first approach which you should read about.
More over i suggest you follow the repository pattern,
https://msdn.microsoft.com/en-us/library/ff649690.aspx
This basically means - each object you save in the db, will have a repository which will contain all the object of its kind and that will be you're entry point to reading/inserting/updatibg/and deleting rows from the db, while abstracting away all details of implementation - in our case I recommend entity framework as I mentioned before.
Good luck
I'm opening a discussion here on a subject I couldn't find any answer good enough to be called a final answer: MySQL and .NET.
While I know there is a lot of ways to make this connection, I'm trying to find a list of pros and cons of each approach.
I've been using ADO.NET with the MySQL NETconnector since the beggining of my project, and everything was ok when the database was new and didn't have many records. But now I'm facing a situation where the number of records grows exponentially, and I found other way of querying against the database, which is the ODBC connector. Using the ADO.NET + NETConnector solution I had my O/RM and didn't have to write my queries, while ODBC makes my code look awful now (since I didn't switch completely to ODBC, I have Linq queries and plain SQL queries inside my code).
Is there any solution (free or not) where I can have both an O/RM without the need of writing SQL queries myself and the speed of ODBC?
What you should be doing is using the MySQL ADO.NET Connector and storing your queries in the database in the form of stored procedures. Version 6.0 of the MySQL connector also supports The Entity Framework. If you are interested in using the Entity Framework, check out this link which describes how to set that up.
NHibernate
Update to Comments
NHibernate Proxy Generators
It is a byte code generator for your object model that allows NHibernate to perform lazy loading and other operations. The link provided explains the benefits.
Castle and LinFu are two different implementations of those Proxy Generators.
While NHibernate does not have coincide documentation all the information on how to use it, is on the internet. This could be a barrier to usability for some people though. I understand more about NHibernate because of my past experience with Hibernate.
In this program I'm writing, it would need frequent database communication, and at the moment I'm using just XML files. Is there really a benefit from using MySQL or SQL in general over XML. Just note that I'm using C# so MySQL is not very fun to deal with in it (from what little experience I have).
In terms of maintaining data stored in XML files vs. a relational database (Mysql, in your case), the database is far more robust than simple XML files. But this is simply an exercise in determining the needs of your application.
MySql, like many other RDBMSs, will provide much more than just a place to park your data. The biggest advantage to using a modern db such as MySql is ACID support. This means you get all-or-nothing transactions, ensuring consistency through your data.
You also get referential integrity to ensure that related records stay intact and don't leave you with abandoned references to other data records. We could go on and on to discuss the value of locking or the power of stored procedures.
But really, you should consider the needs of your application. If you do significant gymnastics to keep your data in order or you care about shared access and file locks while trying to read and write data, you need to punt on your XML file basis. No need trying to find ways around these issues when a basic mysql database will solve those issues.
If there's truly relational data...you'll almost always benefit from using a RDBMS. Retrieving data will be faster with the backing of a query engine rather than tying together XML nodes. You'll also get referential integrity when inserting data into the structure.
There is an ADO.NET provider for MySQL, so you shouldn't have any more difficulty dealing with a MySQL database than MS SQL Server.
You could even download DbLinq and give their LINQ to MySQL functionality a shot. Could make things even easier (or you could use Entity Framework with the MySQL ADO.NET provider).
The size of XML documents can be a large factor. In XML you either produce large and complicated text files with a huge amount of additional data or your data is split up accross several files. Managing these files can be a headache. Using a SQL database will allow you waste less disk space.
SQL is faster than using XML.
Any SQL database will give you access to a whole set of permissions and role capabilities that may be difficult to enforce using XML.
If you have relational data, a database would work. As an alternative to MySQL, if you aren't looking for a centralized solution, you can use SQLite. SQLite runs in-process (meaning the program running it is it's own "database server") and requires no installation other than distributing the DLL file containing it.
Robert Simpson has written System.Data.SQLite, a SQLite Data Provider for the .Net framework. It's free and open source (like SQLite) and works and feels as native as System.Data.SqlClient does. It supports standard ADO.Net conventions, Linq, and the Entity Framework.
I've used System.Data.SQLite for projects at work for applications that need to run fast and cache data locally for comparison between multiple runs (data processing and job scheduling). Firefox is a good example of an application using SQLite, Firefox 3 uses SQLite for it's Cookies, the Downloads history, Form autocomplete, and most importantly your web browsing history.
Again SQLite is meant for direct application use and lacks features like user authentication and schema permissions. It has issues if multiple programs try to write to the same database (those can be worked around but nothing like what a real RDBMS can do). It's biggest advantage is it doesn't need to be installed and set up to work like MySQL does. In the C# case all you have to do is reference System.Data.SQLite and copy the .dll file along with your program and it'll work.
In EJB 3.0 you can write your domain objecs, then convert them into entitities (adding #Entity attribute, etc.) and underlying JPA mechanism (let's say Hibernate) can automaticly generate db schema. What's better it will also update db when you update you domain model in Java code.
I'm looking for equivalent of that functionality on .NET platform. Is is possible with ADO.NET Entity Framework?
DataObjects.Net also automatically generates and upgrades database schema according to domain model. But the most interesting thing is how to upgrade stored data if model and database schema are changed. Is it possible to do it on entity level, rather then using low-level SQL?
Mindscape LightSpeed supports this - full schema round tripping with model first or database first development. It is a commercial product but there is a free version for small databases.
As mentioned, Entity Framework will add some of these features in their next release but that is some time away.
Details of the LightSpeed designer with the model first support
Active Record is the way forward! You mark up your objects with attributes and from there you can generate the database schema or the database itself. There is also a tool called Active Writer which allows you to draw the models and it writes the codes with the correct attributes for you.
It is essentially a wrapper for NHibernate but it makes things a bit easier as you do the mapping on the objects rather than in XML documents.
We have used this on several projects and found it to be a fast way of implementing complex systems.
Yes - in the future :-) The current Entity Framework doesn't support the "domain-first" approach - but the next version (EF v4) will. This will ship with .NET 4.0 / Visual Studio 2010 - but don't ask me, when! I don't know (neither does Microsoft).
Marc
NHibernate is a .NET port of Hibernate, and I think it includes tools for generating database schema for your entities.
There are third party frameworks that do this in .NET today.