I have a WPF application that stores a large amount of information in XML files and as the user uses the application they add more information to the XML files. It's basically using the XML files as a database. Since over the life of the program the XML files have gotten quite large, and I've been think about putting the data on a website, I've been looking into how to move all the information into an SQL database.
I've used SQL databases with web applications (PHP, Ruby, and ASP.NET) but never with a Desktop application. Ideally I'd like to be able to keep all the information in one database file and distribute it along with the application without requiring the user to connect to a remote database (so they don't need an internet connection - though eventually it would be nice if could compare the local file's version with one online somewhere and update if necessary) and without making them install a local database server on their computer. Is this possible?
I'd also like to use LINQ with any new database solution so switching to a database doesn't force to many changes (I read the XML with LINQ).
I'm sure this question has been asked and that there are already some good tutorials on the subject but I just can't find them.
SQLite is a good embedded database that you can ship along with your application. I have not done much more than some prototyping with it, so I personally cannot say with 100% certainty that it will meet your needs. But from what I have read, and what little I have written against it, it seems appropriate for the job.
SQLite Homepage
ADO.NET Provider
If you know how your objects are all going to fit together, you could serialize them/deserialize them to store them on disk as a set of ProtoBuf objects (depending on their size, of course). I've found that it's a pretty simple, elegant solution to storing a set of interconnected classes. Each class that should be savable, all your data, can be serialized using this method, and then restored as necessary.
Here's the .NET link to it.
This is a previous question I asked on SO, and got several good responses.
Related
I wrote a reminder program that runs automatically on startup. I want to know if there is a way, other than SQL-Server, to store event, date and time data. I do not want to use SQL-Server for this work, because I think SQL-Server is very big for this simple task. I think that I can use a file to store data in it. What do you think about this?
Some common ways to store information:
As a file. You have many options where you can store the file. For instance, user directory, and program directory. Further explanation here and here. I prefer using a serializer (xml or json).
As a registry entry. You store your information as key-value pairs.
In a light-weight database:
RavenDB: its document-oriented, and stores data in json format
SQLite: relational; I recommend this SQLite Admin for managing purpose
Registry entries are more safe regarding user actions. On the other hand, files can be easily deleted.
You always have the option, to encrypt your information.
As a side note, you can also use PostSharp to declare variables to be stored in your registry. The code becomes something like this:
[RegistryBacking]
private bool _boolean;
I can provide code later if you need it... when I'm home again.
For the part where to persist
From this document (Managing User Data Deployment Guide, download):
Windows uses the Local and LocalLow folders for application data
that does not roam with the user. Usually this data is either machine
specific or too large to roam.
Windows uses the Roaming folder for application specific data, such
as custom dictionaries, which are machine independent and should roam
with the user profile.
So, I suggest using AppData\Roaming and persisting to a file since I consider a 'reminder app' to be user specific. And domain users for example would consider that valuable (syncing to server).
Local and LocalLow (the latter is used for low integrity mode, for applications with reduced privileges) would be more appropriate for some machine/installation specific data which can be calculated on-the-fly.
Registry seems great for some low amount of keys, but doesn't seem to be the best option for such use.
There is another option - IsolatedStorage, which should be used when mentioned options are not applicable, like when using ClickOnce deployments.
For the part how to persist your data to a file ... well, pick your favorite. You could use SQLite database which comes really lightweigt if you want more control and power or just use XML serialization to a file if you consider using SQLite an overkill. Or any of other viable options.
XML. .NET has classes that makes handling xml files easy. If you're saving structured data then XML might be your best bet.
I have for very similar reasons tried some easy to deploy databases and yet use the knowledge i have.
VistaDB 3.x and 4 are my first choice because they are very much SQL Server compaible and allows me to switch to sql server anytime i like. This supports EF too!!!
Next is db4o by Versant which is very very handy. I use it mostly for quick prototyping but i have deployed to several small solutions and perfect for your kind of application.
I hope that helps!
I want to be able to maintain a count and a last accessed date across application loads for a web service polling application. I'm not too sure what the best way to do this is. I dont like the idea of storing that data in a database as I would have to create one specifically for the purpose. What other options do I have and are there any particularly nice ways of keeping application state between subsequent runs of the app?
Persisting data eh? I suggest a database or file.
File solutions you can just XML serialize to a file and load it again when the app starts.
If the data is shared or might ever grow, then a database is probably the best solution. You can find one that fits your need among the many free projects if you wish:
couchdb
mysql
postgres
mangodb
membase
sqlite
etc
You could roll your own solution that doesn't involve a database, but most likely there is one that fits your needs and learning it would be useful beyond just the project at hand.
Don't be afraid to make a 'configuration' style table for your website, that simply has only a few rows and let's you store runtime information as needed.
Perfectly fine.
I'm trying to prepare to build a database driven .net application and I have hit a roadblock early on due to my lack of knowledge on this topic. Searching around didn't yield anything so here I am asking for help.
I'm receiving weekly data in xml format that will be added to a database and then reports generated using that data. I have a limited license on the xml files so only I can download them and I need to get the results to my end users as well. As far as I can see, I have 2 options:
Feed the data from the xml files into a web hosted database and then have each user connect to the database.
Upload the xml data to a server, have each user download it and keep a local copy of their own database. I'm thinking this will invalidate my license to the original data.
Things / questions of note:
The database holds weekly sports historical data for about the last 10 years.
I need to limit access to the database to only subscribed users.
I'll need to decide how the database will be built.
I need to decide what kind of hosting I'll need.
As you can see, quite an ambitious project for someone new to this. I haven't asked any specific questions so far:
What kind of hosting solutions shall I look for?
Should I use SQL? (Complete newbie on this subject)
Should I use clickonce and then host the application?
Do you have any book or tutorial recommendations that would cover a project like this?
Do I need a script to feed the xml into the database if I go that route? Will that script reside on the server and do it automatically even if I'm not there to instigate it?
I hope the general topic isn't too vague. I tried to actually ask specific questions on it and I'm aware I don't have any code to show as it's just in the early stages of thinking.
The question is a bit vague since you are early on in the decision-making process. However, I do believe that I can offer some help in directing your thinking as you proceed. I think in the situation you are describing, one key thing you should consider is to host your data via JSON/WCF/REST. If you look into these technologies, you will see that there are different ways you can offer your data based upon your developing requirements. For example, how are you going to do authentication? Are you going to allow third-party clients?
What you really don't want to do is allow direct database access, even for authenticated users. Instead, put something in front of it. If you are working in the .NET space, look into all of the different things WCF offers and pick one based upon what fits best. Once you pick that, then you will know what you need for hosting and deployment. Even if you are going to provide the clients as well as the server, this is still a good way to protect your data and provide a way to expand your offering in the future.
If this question seems common to you, I apologise, I did a quick search around this site and a few google searches and could not find a satisfying answer.
My question is this;
I have only been a software developer for 3-4 years now. This may seem like a time long enough to answer this question myself however in all my time, I have never had to develop software where the main body of data-storage is not required to be in an on-line database. This time however, my latest development requires only for its data to be stored only to disk.
The actual data itself is light-weight. In-code the main asset will be a class with only a few, string based properties on it which must be persisted. My initial thoughts are on simple serialisation. On application close new assets are simply serialised and stored on disk as a file. I also though maybe for backup purposes (or if it is somehow a better option to a serialised class) an XML file would be appropriate.
I cannot think of any distinct disadvantages of either of these approaches, it is this fact which causes me to ask this question publicly. In my experience, there is rarely a solution to a problem which does not have it's downsides.
Serialization (binary or XML) is appropriate for a small amount of data. The problem with this approach is when you get large amounts of data (that you may need to query).
If you are on a windows platform and in need of a proper database, you can use the embedded database engine that comes with windows - ESENT. It is the backing store of Exchange and RavenDB.
Here are the .NET wrapper libraries for it.
ManagedEsent provides managed access to ESENT, the embeddable database engine native to Windows. ManagedEsent uses the esent.dll that is part of Microsoft Windows so there are no extra unmanaged binaries to download and install.
The most lightweight solution, is of course to use XML and serialization. The main advantage of that is that it is very easy, requiring little code, and is easily editable using a text editor. The other advantage of this is being able to have multiple files, and they will be easy to transfer from PC to PC.
Here is a nice tutorial on XML serialization.
However, if your application is going to be reading, writing, and changing the data a lot, and there is only one source of data, it would be better to use a light-weight database. Many people like SQLite, while I personally prefer Firebird.
See this question for using SQLite with C#, and see here for information for using Firebird with .net.
Another embedded database option is Sql Server Compact Edition. The latest version of this is v4 and it seems to be much improved over previous versions.
It's functionally equivalent to using an XML file, or an access database, or even a plain old text file, in that you don't need to have a Sql Server service running or install anything special on the machine that your application runs on.
I've been using Sqlite in a project and it works very well and it's easy to use too, one thing to keep it mind when using Sqlite though is that it's designed to be used in a single user environment, so if you use it as the database for the backend of a website for instance you're likely to find that it'll struggle under the slightest of load..
Check out this link for the C# wrapper:
http://sqlite.phxsoftware.com/
I also use NHibernate and NHibernate.Linq to interact with the data, you can get a build of both which are compatible here: http://www.dennisdoomen.net/2009/07/nhibernate-210-ga-with-linq-and-fluent.html
NHibernate.Linq allows you to use those nice Linq query syntax on your Sqlite db:
var onePiece = from s in session.Linq() where s.Name == "One Piece" select s;
I am just beginning to write an application. Part of what it needs to do is to run queries on a database of nutritional information. What I have is the USDA's SR21 Datasets in the form of flat delimited ASCII files.
What I need is advice. I am looking for the best way to import this data into the app and have it easily and quickly queryable at run time. I'll be using it for all the standard things. Populating controls dynamically, Datagrids, calculations, etc. I will also need to do user specific persistent data storage as well. This will not be a commercial app, so hopefully that opens up the possibilities. I am fine with .Net Framework 3.5 so Linq is a possibility when accessing the data (just don't know if it would be the best solution or not). So, what are some suggestions for persistent storage in this scenario? What sort of gotchas should I be watching for? Links to examples are always appreciated of course.
It looks pretty small, so I'd work out an appropriate object model, load the whole lot into memory, and then use LINQ to Objects.
I'm not quite sure what you're asking about in terms of "persistent storage" - aren't you just reading the data? Don't you already have that in the text files? I'm not sure why you'd want to introduce anything else.
I would import the flat files into SQL Server and access via standard ADO.NET functionality. Not only is DB access always better (more robust and powerful) than file I/O as far as data querying and manipulation goes, but you can also take advantage of SQL Server's caching capabilities, especially since this nutritional data won't be changing too often.
If you need to download updated flat files periodically, then look into developing a service that polls for these files and imports into SQL Server automatically.
EDIT: I refer to SQL Server, but feel free to use any DBMS.
My temptation would be to import the data into SQL Server (Express if you aren't looking to deploy the app) as it's a familiar source for me. Alternatively you can probably create an ODBC data source using the text file handler to get you a database-like connection.
I agree that you would benefit from a database, especially for rapid querying, and even more so if you are saving user changes to the data. In order to load the flat file data into a SQL Server (including Express), you can use SSIS.
Use Linq or text data to list method
1.create a list.
2.Read the text file line by line (or all lines).
3.process the line - get required data and attach to the list.
4.process the list for any further use.
the persistence storage will be files and List is volatile.