C# Program - Implementing Local Storage to complement SQL Server Storage - c#

My team has a tool that we use for storing and analyzing our main product's test data. The test data is stored on an SQL Server by a loader program. We then have a client program that is used to view and analyze the data. The client program queries the database and displays it to the user.
We want to implement a way for the client program to store this information locally without having a server instance or additional software installed. We may need to use the tool on customer computers where can't easily install software (no admin). Thus installing SQL Express locally won't work for our needs. We just want to run the EXE on a PC.
Meanwhile we want to minimize the amount of re-work to enable this capability. Currently the tool queries the DB utilizing a connection string and stores the data into a DataTable object. I doubt it can be as easy as setting up a local object and changing the connection string, but that would be nice. Also, I don't mind changing how the querying works if it allows interfacing with both options. What I'm trying to avoid is having to maintain two completely separate methods for interfacing with the data.

If you can't install the database, then embed it.
Just use the old reliable SQLite; export the data you need to a testdatabase.db or a dbgenerator.sql.
SQLServer dialect and SQLite are for the most part compatible.
And if you're using Entity Framework then just plug in Microsoft.EntityFrameworkCore.Sqlite.

Related

Does windows have some inbuilt database engine?

So far, in my applications, I stored all my data via serialization, so I never really needed anything else. But now I am working with more complicated data and I need more than a simple file to store them. Is there some inbuilt storage engine in Windows I can use, which will allow me to pull and edit data with SQL queries? (Since I doubt the user will be willing to install and configure standalone MySQL server just for my application and I don't really want to use 3rd party solutions)
And if there is, how can I access such database engine?
Yes, Windows does have an embedded database engine, which has been there since Windows 2000. It is called Esent.
There is a project called Managed Esent to expose the features of Esent to managed code. There is a NuGet package for it.
I do not know whether it fits your exact need of updates via SQL queries but it does boast a number of features:
ACID transactions with savepoints, lazy commits, and robust crash recovery.
Snapshot isolation.
Highly concurrent database access.
Flexible meta-data (tens of thousands of columns, tables, and indexes are possible).
Indexing support for integer, floating point, ASCII, Unicode, and binary columns.
Update
I have since had some experience with Esent in C# via the nuget package. It does NOT have SQL query capabilities. There is support for indices and basic cursors over them but nothing like the power of SQL. My suggestion if you need a light-weight SQL is definitely SQLite.
SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine.
http://www.sqlite.org
SQLite will work without requiring your App User to install SQLExpress or any other database engine, making it a good option for desktop applications.
Use SQLite as standalone database and integrated part of your applications.
Main benefits:
data is stored inside a single file, easy to pull/edit
ships as part of your application as dll, no separate
installation/configuration needed
i found some info about the database on the Microsoft website database info
there is a embedded database engine. Esent
If it is small set of user specific data, storing at windows registry is a good option.
No, there is no inbuilt database available.
But there are good alternatives, like the mqsql express server or a SQLITE database, which is basically a single database file which is accessed through a library in your project. So if you use SQLITE, you do not need to instal anything.

.net windows application store data offline and store to db when there is network

I am developing a windows application for agricultural purpose. This application will be used by multiple users to maintain the data. The main issue is there won't be network connectivity on the work location. But however by end of the day they can go and synchronize if there are any option.
I just want to know how can we import and store all the data locally and update the data to database when there is network.
The options that i thought is to have SQL on every machine that runs this application. Store the data to local database when there is no network.
Having a separate button to export the local data to the centralized database when there is network.
Looks like this is complicated. Is there any better and easier option.
I prefer using c#, Visual studio.
Thanks.
You can use SQLite for storing data locally. It's fast, lightweight, and public domain.
You can use whatever the database of choice for the centralized server.
Well, this a quite broad question, as it has many options and scenarios. The questions you should ask yourself are:
Does user handle new information only or any information from any other user from the previous syncing?
Do you have to handle update conflicts?
Do you handle text information only or you have complex types and binary files?
As for the solution, the easiest way, from my point of view, would be using SQL Lite on portable devices, is a lightweight SQL client that will allow you to handle information easily. On the server you can use whatever you want, SQL Server, MySQL or any other SQL flavor you may like. Just make sure there is a connector for your portable device OS.
If you keep thinking of using SQL server on the portable device, it's a battery hogger!!!, you might want to check Microsoft Sync framework, as it provides almost all possible scenarios for handling data syncing, manage conflicts, etc.
Thanks for the answers. Please find the below solution that we implemented.
1) Installed SQL express on all the local machines
2) Used Microsoft Sync framework to sync the data. The sync is configured on demand.
Issues faced:
1) We were using geometry datatype on few tables and this was not supported by sync framework.
2) Any change in the database schema will not reflect on the client machine. We will have to delete all the system generated procedures used to track the table change and regenerate it. I am sure there will be a much better way to do this.
Cheers,
Jebli

How can I store a lot of data locally for a program

I am current building (in C#) a fairly basic point-of-sale program for a local community in Uganda to use in tracking business at their sunflower seed press. I was thinking that I would need some sort of database (like a SQL database), but I've never set up a database before, so I'm wondering what the best way to do this is. Maybe a database isn't the best way. The program will not have internet access, so everything will have to be done locally on the machine.
I think your first step should be designing out what data you need to store. Build an Entity Relationship Model and decide what your domain model is going to be. There are many different Database Engines out there that you can use that have different features, installation requirements, etc. A database engine can be installed locally, or on a remote machine to connect to. If you're writing a C# app, you'll probably want to use the System.Data namespace. You can use plain ADO .NET, or use something like Linq To Enttiies to help create proxy classes for your data tables.
You can access a SQL database using the same API for queries / record extraction regardless of the DB Engine uses. In some caess, you may need to use a seperate library that provides an implementation (or a better one), as in the case of an Oracle Database and the Oracle Data Access Components. Right out of the gate, .NET works very well with Microsoft SQL Server, but other options would work.
The details of what database engine are not as important as defining a good set of data tables to represent your data.
Yes. If it has lots of data you have to consider using database. Whether you have internet or not, as long as you have local network, you can easily do database.
Set up a database server ( maybe sql)
Do your database and install it on the database server
Do your application and connect to your database through connection string.
You are on the right track to use a database to store data. It is pretty easy to accomplish. Your computer does not need to be connected to the internet.
SQL Server Express Edition is free with a limit of 10 gigs of data. This will probably be much, much more space than you will need.
From C#, use ADO.NET. It is very simple if you know some SQL. Code samples here.

A GUI which creates and uses a database and installes easily

I want to create a GUI with C++ (QT4). The GUI should work on Windows and should be able to
create a database
use the database created by it (I should use an existing DBMS, in order not to worry for queries)
database should be specific to the GUI, other software should not be able to use that database (the database may be for example encoded)
the gui with its ability of working with database should be easily installed on the other computers, that is I don't won't to ask user to change some options on his computer manually
So my questions are:
What kind of database can help me to do this, what I should learn connected with database to be able to perform this task?
Should I encode the database by my GUI, or databases have such command to save them on disk already encoded?
Thanks!
You could try looking into SQLite. The library can be used with C++. It will not need an external DBMS. SQLite is embedded into your application, and you can access you database through it. Also, the database files it produces can be encoded, so it will be accessible to your application only.
first, you should decide what are the scenario your system going to be applied.
then only proceed to source for database provider (MySQL, Postgres, etc).
you can't really jump to UI implementation straight away because all of the database mentioned above can do what you need.

Any ORMs that work with MS-Access (for prototyping)?

I'm in the early stages of a project, and it's not clear yet whether we'll need a "real" database (i.e. SQL Server et al). So I've been doing some prototyping using MS-Access, which is working fine so far. (developing in C#/VS2008/.Net 3.5/MS-Access 2000).
However, the object-relational impedance mismatch is already becoming annoying, and will only get worse as the project evolves.
I have not been able to find an ORM that will work with MS-Access. Any suggestions?
Edit - Follow Up
We ended up using Fluent NHibernate, mainly because it Automaps our object model to a relational database, which has been a huge win for us. Most of the FNH code samples we found used SQLite, and this worked so well that we intend to use it for our production database. (The app is a desktop scientific data collection and analysis package).
MSAccess files can be set up as an ODBC source on Windows machines. Almost any ORM will allow you to use ODBC. Here is a quick tutorial on how to set that up, it's outlined for Win2k but the process is the same for XP+. You also need to have MDAC installed on your box.
NHibernate seems to have native support of MSAccess as well, see here. I've never used it though. It also has an ODBC driver.. Many others support ODBC as well.
And again, as others are saying.. MSAccess does not scale... period. Installing a real database server is fairly easy, so I'd recommend SQL Server Express as others have, or even MySQL or Postgre, whatever is easier to set up.
If this is an application that you intend to deploy to clients, with each client having their own unique database, I would recommend another solution entirely, SQLite. SQLite gives you database power on an app by app basis. If you have a central database server, one of the previously mentioned solutions would be best.
There's only one scenario when choosing the Access Database Engine is a good choice: when building a self-contained Access application using Access Forms (though choosing to use Access in the first place is a questionable choice ;)
The database engine that VS2008 plays nicest with is SQL Server and you will have no problem finding an ORM that plays nice with SQL Server.
Can't give you an answer to your question, but instead of Access you might want to consider one of the following options:
SQL Server Express: is free and compatible with the full SQL Server
SQL Server Compact: also free, does not require any deployment/installation, does not support all features (e.g. no stored procedures).
At this stage, if you are unsure whether you need a "real" database or not, I'd skip MS Access and go straight to sql server express. It's free and still allows you to do everything you need to.
Plus, if you later decide you need to scale up, then you can without any pain.
I recommend you to use something like Microsoft SQL Server or PostgreSQL for prototyping. If you don't want to learn specific SQL syntax and install special tools for designing database schema, you can use ORM that automatically generates database schema from your persistent classes declaration. Anyway this approach is very effective for prototyping.
LLBLGen works with Access
Access is just a bad, bad idea. I believe MS only includes Access in Office to keep legacy users happy.
Even if you find an ORM that will work with an Access database, with few exceptions you're locking yourself into a niche tool that likely will not work out-of-the box with a real database engine. If you decide to switch to a real database engine later on, you'll not only have to deal with migrating the database, but switching to a different ORM.
See this comparison between SQL Server Express and SQL Server Compact. The comparison document also mentions some problems with other data stores, including Access.
If you are REALLY concerned about being able to install SQL Server Express, consider SQL Server Compact:
it can be linked into your redistributable app. No need to install a service (which may require admin rights during install of your application); everything is taken care of when you install your app. This makes the most sense if you need the data to reside on the user's machine instead of a server, and is most analogous to using Access.
It's less powerful than Express (doesn't support views, triggers, stored procedures, which I consider a requirement)
Can be scaled up to Express or other SQL Server versions very easily
Suitable for small-footprint installs like tablets, mobile devices, etc.
Always keep scalability in mind when designing any application. You don't want to wind up having to write a PHP->C++ compiler if/when your app becomes successful just because you picked the wrong tool up front.
While we're at it:
The big issue with Access (or, in this case, the Jet engine, which is the part you'd really be using when integrating an Access database with a .NET app) is that there is no "server" that handles datase requests. The engine, hosted in your app, must read and write directly to a file on disk that contains the database. Whenever this happens, the file must be locked to prevent concurrent writes. Dirty reads become more common as the number of users grows, as does the potential for database corruption.
Imagine having every customer at a large restaurant trying to simultaneously enter the kitchen to write down their orders or retrieve their food. Chaos would result. There'd be a lot of broken dishes, the kitchen would be a mess, you'd be lucky to get what you ordered in any sort of edible condition. With one customer, this probably works fine. With 5, eh, maybe. With 20,50,1000? Not so much.
So, the restaurant industry introduced waiters and managers that buffer IO to the kitchen. The database server application does something roughly analogous to this by restricting access to the files on disk. Everyone gets what they want, faster and in a much more reliable way, and the data store is protected.

Categories