I need guidance, someone to point me in the right direction. As the tittle says, I need to save information to a file: Date, string, integer and an array of integers. And I also need to be able to access that information later, when an user wants to review it.
Optional: File is plain text and I can directly check it and it is understandable.
Bonus points if chosen method can be "easily" converted to working with a database in the future instead of individual files.
I'm pretty new to C# and what I've found so far is that I should turn the array into a string with separators.
So, what'd you guys suggest?
// JSON.Net
string json = JsonConvert.SerializeObject(objOrArray);
File.WriteAllText(path, json);
// (note: can also use File.Create etc if don't need the string in memory)
or...
using(var file = File.Create(path)) { // protobuf-net
Serializer.Serialize(file, objOrArray);
}
The first is readable; the second will be smaller. Both will cope fine with "Date, string, integer and an array of integers", or an array of such objects. Protobuf-net would require adding some attributes to help it, but really simple.
As for working with a database as columns... the array of integers is the glitch there, because most databases don't support "array of integers" as a column type. I'd say "separation of concerns" - have a separate model for DB persistence. If you are using the database purely to store documents, then: pretty much every DB will support CLOB and BLOB data, so either is usable. Many databases now have inbuilt JSON support (helper methods, etc), which might make JSON as a CLOB more tempting.
I would probably serialize this to json and save it somewhere. Json.Net is a very popular way.
The advantage of this is also creating a class that can be later used to work with an Object-Relational Mapper.
var userInfo = new UserInfoModel();
// write the data (overwrites)
using (var stream = new StreamWriter(#"path/to/your/file.json", append: false))
{
stream.Write(JsonConvert.SerializeObject(userInfo));
}
//read the data
using (var stream = new StreamReader(#"path/to/your/file.json"))
{
userInfo = JsonConvert.DeserializeObject<UserInfoModel>(stream.ReadToEnd());
}
public class UserInfoModel
{
public DateTime Date { get; set; }
// etc.
}
for the Plaintext File you're right.
Use 1 Line for each Entry:
Date
string
Integer
Array of Integer
If you read the File in your code you can easily seperate them by reading line to line.
Make a string with a specific Seperator out of the Array:
[1,2,3] -> "1,2,3"
When you read the line you can Split the String by "," and gets a Array of Strings. Parse each Entry to int into an Array of Int with the same length.
How to read and write the File get a look at Easiest way to read from and write to files
If you really wants the switch to a database at a point, try a JSON Format for your File. It is easy to handle and there are some good Plugins to work with.
Mfg
Henne
The way I got started with C# is via the game Space Engineers from the Steam Platform, the Mods need to save a file Locally (%AppData%\Local\Temp\SpaceEngineers\ or %AppData%\Roaming\SpaceEngineers\Storage\) for various settings, and their logging is similar to what #H. Sandberg mentioned (line by line, perhaps a separator to parse with later), the upside to this is that it's easy to retrieve, easy to append, easy to overwrite, and I'm pretty sure it's even possible to retrieve File Size, which when combined with File Deletion and File Creation can prevent runaway File Sizes as this allows you to set an Upper Limit to check against, allowing you to run it on a Server with minimal impact (probably best to include a minimum Date filter {make sure X is at least Y days old before deleting it for being over Z Bytes} to prevent Debugging Data Loss {"Why was it over that limit?"})
As far as the actual Code behind the idea, I'm approximately at the same Skill Level as the OP, which is to say; Rookie, but I would advise looking at the Coding in the Space Engineers Mods for some Samples (plus it's not half bad for a Beta Game), as they are almost all written in C#. Also, the Programmable Blocks compile in C# as well, so you'll be able to use that to both assist in learning C# and reinforce and utilize what you already know (although certain C# commands aren't allowed for security reasons, utilizing the Mod API you'll have more flexibility to do things such as Creating/Maintaining Log Files, Retrieving/Modifying Object Properties, etc.), You are even capable of printing Text to various in Game Text Monitors.
I apologise if my Syntax needs some work, and I'm sorry I am not currently capable of just whipping up some Code to solve your issue, but I do know
using System;
Console.WriteLine("Hello World");
so at least it's not a total loss, but my example Code likely won't compile, since it's likely missing things like: an Output Location, perhaps an API reference or two, and probably a few other settings. Like I said, I'm New, but that is a valid C# Command, I know I got that part correct.
Edit: here's a better attempt:
using System;
class Test
{
static void Main()
{
string a = "Hello Hal, ";
string b = "Please open the Airlock Doors.";
string c = "I'm sorry Dave, "
string d = "I'm afraid I can't do that."
Console.WriteLine(a + b);
Console.WriteLine(c + d);
Console.Read();
}
}
This:
"Hello Hal, Please open the Airlock Doors."
"I'm sorry Dave, I'm afraid I can't do that."
Should be the result. (the "Quotation Marks" shouldn't appear in the readout {the last Code Block}, that's simply to improve readability)
Related
is there a way to convert C# code to any block representation and back?
Something like this:
int foo(int a){return a+1;}
to
{function:{name:"foo", return:"int", args:[{type:"int", name:"a"}], operations:[{type:"return", operations:[{type:"add", args:[{type:"vairable", value:"a"},{type:"const", value: 1}]}]}]}}
Does not have to be JSON, but I need it to be split to smallest parts.
UPDATE:
Lets say I generate a function that fills a structure based on a data from database:
public Person GetPerson(int id)
{
try { // <-- entire block added by user
using (var query = db.GetPerson(id))
{
return new Person(){
/*0*/name = query['name'], // /*#*/ is my mark of generated line
/*1*/age = query['age']
};
}
}
...
}
Assume that a user changed the line:
/*1*/age = query['age']
to /*1*/age = 10 - query['age'] for some reason.
Now the database column age is changed to years.
The new line should be /*1*/years = 10 - query['years']. The problem is that I need to keep the 10 - entered by the user.
If I had this code is JSON (or any graph) I could find the part that needs to be changed and only affect the nodes I genereated before keeping the excess.
This exmaple is trivial, but it can get complicated very quickly. Especially with quotes and brackets. This is the only approach I can see to work right now. Just hoped that tools for it already exists.
Is there a way to convert C# code to JSON
Sure. You can either make your json contain a string with your c# code and use runtime code generation to execute it, or encode dlls as base64 data and put it into your json. If you want to dig deeper you could probably extract the CIL code and make some custom JSON encoding of it. There are also expression trees, but I only think they allow encoding of expressions, not arbitrary code.
But in any case it is probably not a good idea to let the user customize code at that level. If you want to allow customization you should probably go for an actual plugin architecture. Or if you just want some customization of some simple mathematical expression, just store it as a string and write a simple parser to validate & evaluate it.
I have a noteapp, two pages:
MainPage.xaml — the creation of notes;
NoteList.xaml — a list of notes.
Notes are saved by means of IsolatedStorage, and appear in NoteList.xaml (listbox), but notes with the same name is not stored, how to fix it?
I need to be able to add notes with the same name (but with different content).
Thanks!
Are you using the note name as the file name? If so... don't do that. Save each file with a unique name. There are myriad ways of doing this. You could use a GUID or a timestamp, or you could append a timestamp to the end of the file name. If you were so inclined you could store all of the notes in a single formatted file-- perhaps XML.
What you need is a way to uniquely identify each note without using:
a. The note's name
b. The note's contents
While using a timestamp might make sense for your application right now (since a user probably cannot create two disparate notes simultaneously), using a timestamp to identify each note could lead to problems down the line if you wanted to implement say... a server side component to your application. What happens if in version 23 of your application (which obviously sells millions in the first months), you decide to allow users to collaborate on notes, and a Note is shared between two instances of your app where they happened to be created at the EXACT same time? You'd have problems.
A reasonable solution to finding a unique identifier for each Note in your application is through the use of the Guid.NewGuid method. You should do this when the user decides to "save" the note (or if your app saves the note the moment it's created, or at some set interval to allow for instant "drafts".
Now that we've sufficiently determined a method of uniquely identifying each Note that your application will allow a user to create, we need to think about how that data should be stored.
A great way to do this is through the use of XmlSerializer, or better yet using the third party library Json.Net. But for the sake of simplicity, I recommend doing something a bit easier.
A simpler method (using good ole' plain text) would be the following:
1: {Note.Name}
2: {Guid.ToString()}
3: {Note.Contents}
4: {Some delimiter}
When you are reading the file from IsolatedStorage, you would read through the file line by line, considering each "chunk" of lines between the start of the file and each {Some delimiter} and the end of the file to be the data for one "Note".
Keep in mind there are some restrictions with this format. Mainly, you have to keep the user from having the last part of their note's contents be equal to the {Some delimiter} (which you are free to arbitrarily define btw). To this end, it may be helpful to use a string of characters the user is not likely to enter, such as "##&&ENDOFNOTE&&##" Regardless of how unlikely it is the user will type that in, you need to check to make sure before you save to IsolatedStorage that the end of the Note does not contain this string, because it will break your file format.
If you want a simple solution that works, use the above method. If you want a good solution that's scalable, use JSON or XML and figure out a file format that makes sense to you. I highly encourage you to look into JSON, it's value reaches so much further than this isolated scenario.
I've had a need to write notes to IsolatedStorage. What I did was to them them to a file.IsolatedStorageFile I write date on which the note was written and then note. From the list box i store them to two arrays. Then before exiting the app, write them to a file.
try
{
using (IsolatedStorageFile storagefile = IsolatedStorageFile.GetUserStoreForApplication())
{
if (storagefile.FileExists("NotesFile"))
{
using (IsolatedStorageFileStream fileStream = storagefile.OpenFile("NotesFile", FileMode.Open, FileAccess.ReadWrite))
{
StreamWriter writer = new StreamWriter(fileStream);
for (int i = 0; i < m_noteCount; i++)
{
//writer.Write(m_arrNoteDate[i].ToShortDateString());
writer.Write(m_arrNoteDate[i].ToString("d", CultureInfo.InvariantCulture));
writer.Write(" ");
writer.Write(m_arrNoteString[i]);
writer.WriteLine("~`");
}
writer.Close();
}
}
I'm looking for some suggestions on better approaches to handling a scenario with reading a file in C#; the specific scenario is something that most people wouldn't be familiar with unless you are involved in health care, so I'm going to give a quick explanation first.
I work for a health plan, and we receive claims from doctors in several ways (EDI, paper, etc.). The paper form for standard medical claims is the "HCFA" or "CMS 1500" form. Some of our contracted doctors use software that allows their claims to be generated and saved in a HCFA "layout", but in a text file (so, you could think of it like being the paper form, but without the background/boxes/etc). I've attached an image of a dummy claim file that shows what this would look like.
The claim information is currently extracted from the text files and converted to XML. The whole process works ok, but I'd like to make it better and easier to maintain. There is one major challenge that applies to the scenario: each doctor's office may submit these text files to us in slightly different layouts. Meaning, Doctor A might have the patient's name on line 10, starting at character 3, while Doctor B might send a file where the name starts on line 11 at character 4, and so on. Yes, what we should be doing is enforcing a standard layout that must be adhered to by any doctors that wish to submit in this manner. However, management said that we (the developers) had to handle the different possibilities ourselves and that we may not ask them to do anything special, as they want to maintain good relationships.
Currently, there is a "mapping table" set up with one row for each different doctor's office. The table has columns for each field (e.g. patient name, Member ID number, date of birth etc). Each of these gets a value based on the first file that we received from the doctor (we manually set up the map). So, the column PATIENT_NAME might be defined in the mapping table as "10,3,25" meaning that the name starts on line 10, at character 3, and can be up to 25 characters long. This has been a painful process, both in terms of (a) creating the map for each doctor - it is tedious, and (b) maintainability, as they sometimes suddenly change their layout and then we have to remap the whole thing for that doctor.
The file is read in, line by line, and each line added to a
List<string>
Once this is done, we do the following, where we get the map data and read through the list of file lines and get the field values (recall that each mapped field is a value like "10,3,25" (without the quotes)):
ClaimMap M = ClaimMap.GetMapForDoctor(17);
List<HCFA_Claim> ClaimSet = new List<HCFA_Claim>();
foreach (List<string> cl in Claims) //Claims is List<List<string>>, where we have a List<string> for each claim in the text file (it can have more than one, and the file is split up into separate claims earlier in the process)
{
HCFA_Claim c = new HCFA_Claim();
c.Patient = new Patient();
c.Patient.FullName = cl[Int32.Parse(M.Name.Split(',')[0]) - 1].Substring(Int32.Parse(M.Name.Split(',')[1]) - 1, Int32.Parse(M.Name.Split(',')[2])).Trim();
//...and so on...
ClaimSet.Add(c);
}
Sorry this is so long...but I felt that some background/explanation was necessary. Are there any better/more creative ways of doing something like this?
Given the lack of standardization, I think your current solution although not ideal may be the best you can do. Given this situation, I would at least isolate concerns e.g. file read, file parsing, file conversion to standard xml, mapping table access etc. to simple components employing obvious patterns e.g. DI, strategies, factories, repositories etc. where needed to decouple the system from the underlying dependency on the mapping table and current parsing algorithms.
You need to work on the DRY (Don't Repeat Yourself) principle by separating concerns.
For example, the code you posted appears to have an explicit knowledge of:
how to parse the claim map, and
how to use the claim map to parse a list of claims.
So there are at least two responsibilities directly relegated to this one method. I'd recommend changing your ClaimMap class to be more representative of what it's actually supposed to represent:
public class ClaimMap
{
public ClaimMapField Name{get;set;}
...
}
public class ClaimMapField
{
public int StartingLine{get;set;}
// I would have the parser subtract one when creating this, to make it 0-based.
public int StartingCharacter{get;set;}
public int MaxLength{get;set;}
}
Note that the ClaimMapField represents in code what you spent considerable time explaining in English. This reduces the need for lengthy documentation. Now all the M.Name.Split calls can actually be consolidated into a single method that knows how to create ClaimMapFields out of the original text file. If you ever need to change the way your ClaimMaps are represented in the text file, you only have to change one point in code.
Now your code could look more like this:
c.Patient.FullName = cl[map.Name.StartingLine].Substring(map.Name.StartingCharacter, map.Name.MaxLength).Trim();
c.Patient.Address = cl[map.Address.StartingLine].Substring(map.Address.StartingCharacter, map.Address.MaxLength).Trim();
...
But wait, there's more! Any time you see repetition in your code, that's a code smell. Why not extract out a method here:
public string ParseMapField(ClaimMapField field, List<string> claim)
{
return claim[field.StartingLine].Substring(field.StartingCharacter, field.MaxLength).Trim();
}
Now your code can look more like this:
HCFA_Claim c = new HCFA_Claim
{
Patient = new Patient
{
FullName = ParseMapField(map.Name, cl),
Address = ParseMapField(map.Address, cl),
}
};
By breaking the code up into smaller logical pieces, you can see how each piece becomes very easy to understand and validate visually. You greatly reduce the risk of copy/paste errors, and when there is a bug or a new requirement, you typically only have to change one place in code instead of every line.
If you are only getting unstructured text, you have to parse it. If the text content changes you have to fix your parser. There's no way around this. You could probably find a 3rd party application to do some kind of visual parsing where you highlight the string of text you want and it does all the substring'ing for you but still unstructured text == parsing == fragile. A visual parser would at least make it easier to see mistakes/changed layouts and fix them.
As for parsing it yourself, I'm not sure about the line-by-line approach. What if something you're looking for spans multiple lines? You could bring the whole thing in a single string and use IndexOf to substring that with different indices for each piece of data you're looking for.
You could always use RegEx instead of Substring if you know how to do that.
While the basic approach your taking seems appropriate for your situation, there are definitely ways you could clean up the code to make it easier to read and maintain. By separating out the functionality that you're doing all within your main loop, you could change this:
c.Patient.FullName = cl[Int32.Parse(M.Name.Split(',')[0]) - 1].Substring(Int32.Parse(M.Name.Split(',')[1]) - 1, Int32.Parse(M.Name.Split(',')[2])).Trim();
to something like this:
var parser = new FormParser(cl, M);
c.PatientFullName = FormParser.GetName();
c.PatientAddress = FormParser.GetAddress();
// etc
So, in your new class, FormParser, you pass the List that represents your form and the claim map for the provider into the constructor. You then have a getter for each property on the form. Inside that getter, you perform your parsing/substring logic like you're doing now. Like I said, you're not really changing the method by which your doing it, but it certainly would be easier to read and maintain and might reduce your overall stress level.
for part of a small assignment I have, i've been asked to create an array to store names and addresses taken from input that the user gives and to be able to later delete a name and address from the array.
Any help or links to helping me understand how to achieve this would be highly appreaciated, thanks.
EDIT - The array is to be set up like an address book, and when printed to the screen it displays like so: "Bloggs, Joe"
It must be surname then forename. I know how to acquire and store the information the user will give, being their names and addresses, but I am stuck on how to add this into an array. The array doesn't have to be infinite, as I am supposed to allocate the array whatever size I wish.
At the start of the program it will be part of, the user will be given a menu, and they can choose to add a record, delete a record or print the book to the screen. So i am meant to be using methods where suitable.
Well, to start with, an array is the wrong data structure to use here.
Arrays are always a fixed size - whereas you want to be able to add elements and later remove them. Assuming you're using C# 2 or higher, you should probably use a List<T>.
Now, the next thing is to work out what T should be. It sounds like you want to store details of people - so you should create a Person class (or perhaps Contact) to encapsulate the name and address... that way you can have a List<Person>.
The next task is probably to work out how to ask the user for input and convert that input into an instance of Person.
Basically, break the task up into small bits - and then feel free to ask questions about any specific bits which you find hard.
I seem to remember this exact same assignment from my CS classes.
The prof wanted us to use linked lists. As John Skeet points out above, .NET has List<T>, which is basically a linked list (with the added feature of being able to be reference each item by index like an array)
You can use a Serializer for the saving part.
Check out the BinaryFormatter class and XmlSerializer.
XmlSerializer is preferred because the file is human-readable and efficiency is usually less important considering the type and purpose of your app.
Using XmlSerializer is as simple as:
var filename = "c:\....\addressbook.xml";
if (File.Exists(filename))
File.Delete(filename);
using (var sw = new StreamWriter(filename))
{
var xs = new XmlSerializer(typeof(List<Person>));
xs.Serialize(sw, myAddressBook);
}
I've been trying to deal with some delimited text files that have non standard delimiters (not comma/quote or tab delimited). The delimiters are random ASCII characters that don't show up often between the delimiters. After searching around, I've seem to have only found no solutions in .NET will suit my needs and the custom libraries that people have written for this seem to have some flaws when it comes to gigantic input (4GB file with some field values having very easily several million characters).
While this seems to be a bit extreme, it is actually a standard in the Electronic Document Discovery (EDD) industry for some review software to have field values that contain the full contents of a document. For reference, I've previously done this in python using the csv module with no problems.
Here's an example input:
Field delimiter =
quote character = þ
þFieldName1þþFieldName2þþFieldName3þþFieldName4þ
þValue1þþValue2þþValue3þþSomeVery,Very,Very,Large value(5MB or so)þ
...etc...
Edit:
So I went ahead and created a delimited file parser from scratch. I'm kind of weary using this solution as it may be prone to bugs. It also doesn't feel "elegant" or correct to have to write my own parser for a task like this. I also have a feeling that I probably didn't have to write a parser from scratch for this anyway.
Use the File Helpers API. It's .NET and open source. It's extremely high performance using compiled IL code to set fields on strongly typed objects, and supports streaming.
It supports all sorts of file types and custom delimiters; I've used it to read files larger than 4GB.
If for some reason that doesn't do it for you, try just reading line by line with a string.split:
public IEnumerable<string[]> CreateEnumerable(StreamReader input)
{
string line;
while ((line = input.ReadLine()) != null)
{
yield return line.Split('þ');
}
}
That'll give you simple string arrays representing the lines in a streamy fashion that you can even Linq into ;) Remember however that the IEnumerable is lazy loaded, so don't close or alter the StreamReader until you've iterated (or caused a full load operation like ToList/ToArray or such - given your filesize however, I assume you won't do that!).
Here's a good sample use of it:
using (StreamReader sr = new StreamReader("c:\\test.file"))
{
var qry = from l in CreateEnumerable(sr).Skip(1)
where l[3].Contains("something")
select new { Field1 = l[0], Field2 = l[1] };
foreach (var item in qry)
{
Console.WriteLine(item.Field1 + " , " + item.Field2);
}
}
Console.ReadLine();
This will skip the header line, then print out the first two field from the file where the 4th field contains the string "something". It will do this without loading the entire file into memory.
Windows and high performance I/O means, use IO Completion ports. You may have todo some extra plumbing to get it working in your case.
This is with the understanding that you want to use C#/.NET, and according to Joe Duffy
18) Don’t use Windows Asynchronous Procedure Calls (APCs) in managed
code.
I had to learn that one the hard way ;), but ruling out APC use, IOCP is the only sane option. It also supports many other types of I/O, frequently used in socket servers.
As far as parsing the actual text, check out Eric White's blog for some streamlined stream use.
I would be inclined to use a combination of Memory Mapped Files (msdn point to a .NET wrapper here) and a simple incremental parse, yielding back to an IEnumerable list of your record / text line (or whatever)
You mention that some fields are very very big, if you try to read them in their entirety to memory you may be getting yourself into trouble. I would read through the file in 8K (or small chunks), parse the current buffer, keep track of state.
What are you trying to do with this data that you are parsing? Are you searching for something? Are you transforming it?
I don't see a problem with you writing a custom parser. The requirements seem sufficiently different to anything already provided by the BCL, so go right ahead.
"Elegance" is obviously a subjective thing. In my opinion, if your parser's API looks and works like a standard BCL "reader"-type API, then that is quite "elegant".
As for the large data sizes, make your parser work by reading one byte at a time and use a simple state machine to work out what to do. Leave the streaming and buffering to the underlying FileStream class. You should be OK with performance and memory consumption.
Example of how you might use such a parser class:
using(var reader = new EddReader(new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.Read, 8192)) {
// Read a small field
string smallField = reader.ReadFieldAsText();
// Read a large field
Stream largeField = reader.ReadFieldAsStream();
}
While this doesn't help address the large input issue, a possible solution to the parsing issue might include a custom parser that users the strategy pattern to supply a delimiter.