I am writing a software which stores all the information of a users interaction in a global session object/class. I would like to store this values collected in a persistent storage. However i cannot use heavy databases such as sql server or mysql in the target pc as i need to keep the installer minimum in size.
I also need to retrieve values from the storage by passing simple Linq queries,etc.
My question is what is the next best thing to databases which can be manipulated by C# code?
Probably either SQLite or SQL Server Compact Edition - these are both fairly full-featured database systems than run entirely in-process and are frequently used these sorts of thing (for example Firefox uses SQLite to store bookmarks).
The next rung down the ladder of complexity would probably be either XML (using LINQ to XML), or just serialisable objects (using LINQ to objects) - you would of course incur performance penalties over a "proper" compact database like SQLite if you started storing a lot of data, however you would probably need store more than you think before it became noticable, and for small data sets the simplicity would even make this faster than SQLite (for example you could restrict your application to storing the last 100 or so actions).
SQL Server CE and SQLite are popular for the scenario you are describing. XML is as well
You could connect to Access MDB files. You don't need an SQL server for this, and it uses the same syntax.
Just need to use OleDb.
Example: DataEasy: Connect to MS Access (.mdb) Files Easily using C#
Related
I am current building (in C#) a fairly basic point-of-sale program for a local community in Uganda to use in tracking business at their sunflower seed press. I was thinking that I would need some sort of database (like a SQL database), but I've never set up a database before, so I'm wondering what the best way to do this is. Maybe a database isn't the best way. The program will not have internet access, so everything will have to be done locally on the machine.
I think your first step should be designing out what data you need to store. Build an Entity Relationship Model and decide what your domain model is going to be. There are many different Database Engines out there that you can use that have different features, installation requirements, etc. A database engine can be installed locally, or on a remote machine to connect to. If you're writing a C# app, you'll probably want to use the System.Data namespace. You can use plain ADO .NET, or use something like Linq To Enttiies to help create proxy classes for your data tables.
You can access a SQL database using the same API for queries / record extraction regardless of the DB Engine uses. In some caess, you may need to use a seperate library that provides an implementation (or a better one), as in the case of an Oracle Database and the Oracle Data Access Components. Right out of the gate, .NET works very well with Microsoft SQL Server, but other options would work.
The details of what database engine are not as important as defining a good set of data tables to represent your data.
Yes. If it has lots of data you have to consider using database. Whether you have internet or not, as long as you have local network, you can easily do database.
Set up a database server ( maybe sql)
Do your database and install it on the database server
Do your application and connect to your database through connection string.
You are on the right track to use a database to store data. It is pretty easy to accomplish. Your computer does not need to be connected to the internet.
SQL Server Express Edition is free with a limit of 10 gigs of data. This will probably be much, much more space than you will need.
From C#, use ADO.NET. It is very simple if you know some SQL. Code samples here.
I'm currently trying to develop a program which stores lots of data, similar to an address book, I suppose, with the intent that new data will be periodically added to the program over time.
I know that I could set up a SQL server, and have the program interface with that, but if I want to share my program with other people, I can't guarantee that they'd have access to the server, or that they can set up a server of their own to hold the data.
I also know that I could simply hard-code all of the data into instantiated objects, but that is inelegant, and promises to be incredibly irritating to alter or maintain.
Is there someway I could design the program in such a way that it maintains a database-like structure, yet has no reliance on external programs (such as a SQL server)?
"SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine."
http://www.sqlite.org/
Sure. You can use XML. XML can be also used as a datasource for ASP.NET components, just like a database.
You could use a flat file database like SQLite which is linked in on compile and could be distributed with your code.
You have several options:
Use flat-file database like SQLite (ADO.NET provider)
Save your data into a file in some format like XML or CSV, or use binary serialization (or something more elaborate if you don't want to have all the data in memory)
Have a public SQL server accessible from the internet
Of course these is always SQL Server express from MS for this purpose.
no you do not neet a server. the most flexibel solution would be to use NHibernate with FluentNHibernate.
There are different drivers for different databases like SQlite, MsAccess and also for Server like MsSql, Oracle, ....
Here's our situation.
We're receiving a dump of relational data in Access 2007 format. There are quite a few tables involved. We're writing a console app in c# to run various queries against this data. We only need read-only access - we're not updating the Access database.
I haven't used Access in a project since pre-Linq days, and I'm hoping we don't have to go back to coding strings of sql against an ADO.Net connection just because the database is Access. I gather Linq 2 Sql is out of the question, but might Entity Framework be usable?
How would you approach this problem?
EDIT: The console app will be dropped by a business analyst into a folder containing the Access database, and when run will generate a text file created by querying the data. So unfortunately it's not an option to transfer the data to Sql beforehand!
If you must keep the data into Access, you can pull it into a Dataset via ADO.NET, and then use the LINQ extension methods that work against a Dataset to work against the data.
It's not nearly as nice as working with SQL Server, but it does work.
I would fully import the data into Sql Server, after which I would gleefully destroy all copies of the original Access file. Then you could get as Linq-y as you like.
Linq2Sql no, LinqXml yes! How hard would it be to dump the access file to xml?
I'm not sure I understand the question, but have you considered a SQL Server linked server to the Access file? You could then use SQL Server to do the manipulation. I don't know what the limitations/pitfalls of that are, but it's a solution often suggested when there's no good direct way to manipulate the Jet/ACE database.
My project requires a solution to store billions of rows of data with minimal relational data.
The raw data is currently in a text file and looks something like this
Id(), Type(int), Data(Binary data between 1-10MB)
The Id column in the raw text file can be ignored when importing, and replace with either a new int, bigint or uniqueidentifier, which ever has better performance.
Any suggestions on what I should use and how I should design the database?
Also the front end will be written in C# with EF4 (or something else, im open to all suggestions).
I think you might be interrested in a serverless database. Like SQLite or SQL Server Compact.
You do not have to install a server, but you can query your data using SQL, LINQ etc.
Windows Azure Storage Services is the closest your gonna get if your looking for a NoSQL product by Microsoft
It's a cloud thing and Microsoft doesn't have a separate product that you yourself host.
Windows Azure Storage Services is however, built on top of MS SQL Server, just not exposed through the normal TDS protocol. That way, they never allow access to the database without NoSQL in mind. That doesn't stop you from treating your typical SQL Server database as if it was NoSQL, and if you did, you should be able to scale really well. The idea of NoSQL is just that you don't do stuff that doesn't scale horizontally.
http://en.wikipedia.org/wiki/NoSQL
NoSQL is not equviliant of any RDBMS. so "What's the NoSQL equivalent of MS SQL Server" makes no sense. it should either be NoSQL vs MS SQL or no mention of NoSQL at all.
There's a provider giving you some feeling of having a nosql document store over sql-server. http://www.sisodb.com
What is the problem you are attempting to solve with the system?
What type of data are you analysing?
Assuming this is an analysis system rather than a transactional processing system there are tools for analysing large data sets that might have the functionality you need without requiring you to write too much code. For example the Visualisation Toolkit from Kitware www.vtk.org or MIDAS http://www.kitware.com/products/midas.html
MIDAS integrates multimedia server technology with Kitware’s open-source data analysis and visualization clients. The server follows open standards for data storage, access and harvesting. MIDAS has been optimized for storing massive collections of scientific data and related metadata and reports. MIDAS is available under a non-restrictive (BSD) open-source license.
Alternatively IBM have OpenDX http://www.research.ibm.com/dx/
I suggest you conider using SQL Server with the Filesteam feature for the binary data.
http://technet.microsoft.com/en-us/library/bb933993.aspx
Your question has nothing much to do with NoSQL. Don't go thinking that filestream is the SQL Server "equivalent" of NoSQL!
I think the closest approach you will get from MS SQL - is XML column type. Since xml is by definition semi-structured data. So you can make field of xml type and add there your document with binary data encoded in hex or base64 format (if data storage space is not an issue to you).
In this program I'm writing, it would need frequent database communication, and at the moment I'm using just XML files. Is there really a benefit from using MySQL or SQL in general over XML. Just note that I'm using C# so MySQL is not very fun to deal with in it (from what little experience I have).
In terms of maintaining data stored in XML files vs. a relational database (Mysql, in your case), the database is far more robust than simple XML files. But this is simply an exercise in determining the needs of your application.
MySql, like many other RDBMSs, will provide much more than just a place to park your data. The biggest advantage to using a modern db such as MySql is ACID support. This means you get all-or-nothing transactions, ensuring consistency through your data.
You also get referential integrity to ensure that related records stay intact and don't leave you with abandoned references to other data records. We could go on and on to discuss the value of locking or the power of stored procedures.
But really, you should consider the needs of your application. If you do significant gymnastics to keep your data in order or you care about shared access and file locks while trying to read and write data, you need to punt on your XML file basis. No need trying to find ways around these issues when a basic mysql database will solve those issues.
If there's truly relational data...you'll almost always benefit from using a RDBMS. Retrieving data will be faster with the backing of a query engine rather than tying together XML nodes. You'll also get referential integrity when inserting data into the structure.
There is an ADO.NET provider for MySQL, so you shouldn't have any more difficulty dealing with a MySQL database than MS SQL Server.
You could even download DbLinq and give their LINQ to MySQL functionality a shot. Could make things even easier (or you could use Entity Framework with the MySQL ADO.NET provider).
The size of XML documents can be a large factor. In XML you either produce large and complicated text files with a huge amount of additional data or your data is split up accross several files. Managing these files can be a headache. Using a SQL database will allow you waste less disk space.
SQL is faster than using XML.
Any SQL database will give you access to a whole set of permissions and role capabilities that may be difficult to enforce using XML.
If you have relational data, a database would work. As an alternative to MySQL, if you aren't looking for a centralized solution, you can use SQLite. SQLite runs in-process (meaning the program running it is it's own "database server") and requires no installation other than distributing the DLL file containing it.
Robert Simpson has written System.Data.SQLite, a SQLite Data Provider for the .Net framework. It's free and open source (like SQLite) and works and feels as native as System.Data.SqlClient does. It supports standard ADO.Net conventions, Linq, and the Entity Framework.
I've used System.Data.SQLite for projects at work for applications that need to run fast and cache data locally for comparison between multiple runs (data processing and job scheduling). Firefox is a good example of an application using SQLite, Firefox 3 uses SQLite for it's Cookies, the Downloads history, Form autocomplete, and most importantly your web browsing history.
Again SQLite is meant for direct application use and lacks features like user authentication and schema permissions. It has issues if multiple programs try to write to the same database (those can be worked around but nothing like what a real RDBMS can do). It's biggest advantage is it doesn't need to be installed and set up to work like MySQL does. In the C# case all you have to do is reference System.Data.SQLite and copy the .dll file along with your program and it'll work.