I'm making a small game using XNA, but it is cumbersome to effectively type-in the stats to all the entities in the game.
I was thinking that it would be much simpler to save the required information in a separate file with a table-format and use that instead.
I have looked into reading Excel tables with C# but it seems to be overly complex.
Is there any other table-format file types that let me easily edit the contents of the file and also read the contents using C# without too much hassle??
Basically, is there any other simple alternative to Excel? I just need the simplest table files to save some text in.
CSV is probably the simplest format to store table data, Excel can save data in it. There is no built in classes to read data from CSV as far as I know.
You may also consider XML or JSON to store data if you want some more structured data. Both have built in classes to serialize objects to/from.
If you are comfortable using Excel try exporting to a .CSV (Comma Seperate Value) file. The literal string will look like below.
row1col1,row1col2\nrow2col1,row2col2\nrow3col1,row3col2
The format is incredibly simple. Each row is on a separate line (separated by "\n") and the columns within a line are separated by commas. Very easy to parse just iterate though the lines and split on the commas.
while ((row = tr.ReadLine()) != null)
{
row.split(",")[0] //first column
row.split(",")[1] //second column
row.split(",")[2] //ect...
}
This may be overkill, but SQLlite might be worth looking into if you want expand-ability and maintainability. It is an easy setup and learning SQL will be useful in many applications.
This is a good tutorial to get you started:
http://www.dreamincode.net/forums/topic/157830-using-sqlite-with-c%23/
I understand if this isn't exactly what you were looking for, but I wanted to give you a broader range of options. If you are going for absolute simplicity go with CSV or XML like Alexei said.
Edit: If necessary there is a C# SQLlite version for managed environments(XBOX,WP7) http://forums.create.msdn.com/forums/p/47127/282261.aspx
Related
I use csv files as database in seperate processes. I only store all data or read all data in my datagrid in singular relationship. Every field in every txt file is one and only number starting from zero.
//While reaching countries, i read allcountries.txt,
//while reaching cities, i read allcities.txt
//while reaching places i read allplaces.txt.
but one country has many cities and one city has many places. Yet, i don't use any relationship. I want to use and i know there is some needs for this. How can i reach data for reading and writing by adding all text files one extra column?
And is it possible to reach data without sql queries?
Text files don't have any mechanism for SELECTs or JOINs. You'll be at a pretty steep disadvantage.
However, LINQ gives you the ability to search through object collections in SQL-like ways, and you can certainly create entities that have relationships. The application (whatever you're building) will have to load everything from the text files at application start. Since your code will be doing the searching, it has to have the entities loaded (from text files) and in-memory.
That may or may not be a pleasant or effective experience, but if you're set on replacing SQL with text files, it's not impossible.
CSV files are good for mass input and mass output. They are not good for point manipulations or maintaining relationships. Consider using a database. SQLite might be something useful in your application.
Based on your comments, it would make more sense to use XML instead of CSV. This meets your requirements for being human and machine readable, and XML has nice built in libraries for searching, manipulating, serializing etc.
You can use SQL queries in CSV files: How to use SQL against a CSV file, I have done it for reading but never for writing so I don't know if this will work for you.
I have lot of Forex Tick Data to be saved. My question is what is the best way?
Here is am example: I collect only 1 month data from the EURUSD pair. It is originally in CSV file which is 136MB large and has 2465671 rows. I use a library written by : http://www.codeproject.com/Articles/9258/A-Fast-CSV-Reader and it took around 30 seconds to read all the ticks and saved it in 2465671 objects. first of all, whether it is fast enough?
Secondly, is there any way better than CSV? For example, the binary file which might be faster and whether you have any recommendation about any database which is best? I tried the db4o but it is not very impressive. I think here are some overhead to save data as properties of object and when we have to save 2465671 objects in Yap file of db4o.
I've thought about this before, and if I was collecting this data, I would break up the process:
collect data from the feed, form a line (I'd use fixed width), and append it to a text file.
I would create a new text file every minute and name it something like rawdata.yymmddhhmm.txt
Then I would have another process working in the background reading these files and pushing then into a database via a parameterized insert query.
I would probably use text over a binary file because I know that would append without any problems, but I'd also look into opening a binary file for append as well. This might actually be a bit better.
Also, you want to open the file in append mode since that's the fastest way to write to a file. This will obviously need to be super fast.
Perhaps look at this product:
http://kx.com/kdb+.php
it seems to made for that purpose.
One way to save data space (and hopefully time) is to save numbers as numbers and not as text, which is what CSV does.
You can perhaps make an object out of each row, and the make the reading and writing each object a serialization problem, which there is good support for in C#.
Kx's kdb database would be a great of-the-shelf package if you had a few million to spare. However you could easily write your own column-orientated database to store and analyse high-frequency data for optimal performance.
I save terabytes as compressed binary files (GZIP) that I dynamically uncompress using C#/.NETs built-in gzip compression/decompression readers.
HDF5 is widely used for big data, including by some financial firms. Unlike KDB it's free to use, and there are plenty of libraries to go on top of it, such as the .NET wrapper
This SO question might help you get started.
HDF5 homepage
Similar to What is the best format to store test data in a file? I want make some tests with data in my class.
This class must receive a list of data with date and value. So basically, I just need to store values and when I read I can say the initial date and just add a constant to get next correspondent date - value. Something like: 1234.56;6543.21;1111.22
I'm using csv right now, but I'm wondering if some xml, sqlite or pure IEnumerable in C# will be a better option. Some experience about this?
With the requirements you have communicated, the answer is whatever is easiest. If you are talking numeric data so you don't have to worry about quoted strings and the like, CSV seems quite reasonable.
Choose the simplest thing that works and is maintainable. In order of preference
If you your data is relatively simple (as depicted in the question), you could do it in-memory within the test-fixture class. Use factory methods / types to get the expected Data.
If you want structured data that is easy to read/edit, use CSV (everyone knows how to use Excel) or YAML (another text-based format) or a custom text file.
On the other hand, if you need structured schema-valid data, then you could choose something like XML or canned test-databases.
I'm new to C# and WPF. I will need to write a program to collect and display RS232 data and save it as a CSV file. Do I need database or XML? Is there any relevant tutorial anyone could recommend?
Around the storage..
Do you want to sort / filter / search the data?
Then I would store it in a SQL Database and export it later on to CSV (using a simple StringWriter).
If you don't want to access the data, just collect it and put it into CSV I would simply write it to CSV as the data comes in.
For reading CSV I would recommend using a LINQ to CSV implementation.
E.g. http://www.codeproject.com/KB/linq/LINQtoCSV.aspx
or http://www.thinqlinq.com/Post.aspx/Title/LINQ-to-CSV-using-DynamicObject
Don't know much about RS232, but I gather this is binary data coming off a serial port?
I would question the whole idea of outputing it as a CSV file - text based formats are not ideal for this.
XML is a better choice - you could use Base64 to sort out the encoding issues, and you could structure your document sensibly to meet your purposes.
Another idea, depending on what you're trying to do, might be to use a file based Database. I don't mean MS Access either (Spit!) - look up SQLite.
If someone is really forcing you down a CSV route for binary data, then I suggest you and they need to have a talk about whether this really is, when it comes down to it, really such a great idea.
Hi experts am trying to parse an excel file. its structure is very complex. The possible way i know are.
Use Office introp libraries
Use OLEDB provider and read excel file in a dataset.
But the issue is of its complexity like some columns,cells or rows blank etc.
What are the best possible ways to do this ?
thanks in advance.
I can recommend the ExcelDataReader (licensed under LGPL I think). It loads both .xls and .xlsx files, and lets you get the spreadsheet as a DataSet, with each worksheet being an individual DataTable. As far as I know from the scenarios I have used it in, it honours blank rows, empty cells, etc. Try it and see if you think it will handle your "very complex" structure. [I do notice one negative review on the site - but the rest are pretty positive. I've experienced an issue reading .xlsx if a worksheet is renamed]
I've also used the OLEDB approach in the past, but be warned that this has real problems in the way it tries to infer datatypes in the first few rows. If the datatype changes for a column, then this may well infer it wrongly. To make matters worse, when it does get it wrong, it will often return null as the value, making it difficult (or impossible) to tell a true null value from a datatype that changed after the first six or seven rows.
Personally i prefer to either use the OLEDB way, which is a bit clunky at best at times, or you can use a third party library that has put in the time/effort/energy to get access to the data.
SyncFusion has a pretty nice library for this.
I have my users first save the Excel spreadsheet as a CSV file. Then they upload the CSV file to my app. That makes it much simpler to parse.
I've used OLEDB myself to read uploaded Excel files, and its presents no real problems (except for nulls in fields, instead of blanks, which can be checked with IsDBNull). Also, third party open source tools like NPOI and Excel2007ReadWrite (http://www.codeproject.com/KB/office/OpenXML.aspx) can be useful.
I have thoroughly evaluated both of these third party tools, and both are pretty stable and easy to integrate. I would recommend NPOI for Excel 2003 files, and Excel2007ReadWrite for Excel 2007 files.
It sounds like you have a good understanding of the task at hand. You'll have to write business logic to untangle the complexities of the spreadsheet format and extract the data you're looking for.
It seems to me that VTSO/Interop is the best platform strategy for 2 reasons:
Access to the spreadsheet data will be a small part of the effort needed for your solution. So if using OLEDB saves a little time in data access, it will probably be irrelevant in terms of the overall project scope.
You may need to examine the contents of individual cells closely and take context information like formatting into account. With interop, you get full visibility of cell contents, context, and other sheet level context information like named ranges and lists. It is a risk to assume you won't need this type of information while decoding the spreadsheet.