I need a bit of advice/opinions on storing data.
Im looking at essentially storing user input data and then reference it on a frequent basis. I would like to avoid data base storage as this is all going to be referenced locally. Ive seen that you can do XML serialised objects etc and wanted to know what you think about data storage.
If i give you an idea of what im looking at doing in a step by step it may give a better idea of what im trying to achieve:
User loads the program
Program reads the storage file
Different controls such as GridView and text boxes displays specific parts of the document such as Names, Dates etc.
User can add to the file by entering data into different fields across the whole sheet.
User is free to save the file when required, can create new files when required and can open sheets at any point
All information needs to scale. i.e. user can add up to 50 dates, 30 names etc and each of these cannot interfere with another data field. All of this data for each specific type will be able to be referenced by the ui controls
Its a bit of a big task and i was looking at doing it all using XML. Im not sure if this is the best way to do it and wanted your opinions. Mainly due to the way the data can dynamically adjust all the time through user input.
Any input on this would be appreciated. I can provide images of what im trying to achieve if necessary.
I personally use Newtonsoft JSON for saving my user data. You can build all your classes as normal and then fairly easily integrate the JSON serialiser/deserialiser without too much hassle. There will be some exceptions that may require a custom serialiser, but it's not too difficult to make those, especially if you only need to interpret one small part and then can just pass the rest of the file off to the default serialiser.
This is a very quick example how you could store some key-value preferences for a number of users in a file.
using Newtonsoft.Json;
[JsonObject]
public class UserData {
[JsonProperty] //Add a JSON property "Username" which is a string
public string Username { get; set; }
//IEnumerable types are converted to/from arrays automatically by Newtonsoft
[JsonProperty("Options")] //Set the name in JSON file to "Options"
public Dictionary<string, string> Preferences { get; set; }
[JsonIgnore] //Excludes this property from the JSON output
public bool SaveRequired { get; set; } //Set true when a change is made, set false when saved
}
There are very similar libraries like this that do the exact same thing for XML, but I've not had much luck figuring them out and I'm usually on a very tight time scale when I just need something that I know works.
If you can find a decent XML library and understand how to use it correctly, I'd recommend XML over JSON due to its strictly structured nature, and you can include schema versions and aid with integration into other systems by providing a well written schema.
Related
I have an MVC web app where users upload a text file and I parse it out.
The requirement just changed and they will be uploading multiple files of the same kind now. I parse a single file by sending a file-path to the method below, ReadParts which opens a stream and calls the method parseReplicateBlock to retrieve desired fields. For multiple files I could read all the files into one big stream but I am afraid it could exceed the buffer limit etc.
So I am thinking to parse file by file and populate results into an object. My requirement then, is to sort the records based on a date field.
I just need some help in how to write this method ReadLogFile in a better way, espceially for sorting based on initialtionDate and initiationTime. I want to find the minimum record based on initiationDate and Time and then do some other logic.
The problem is if I sort the list member within the object, I would loose positiong of the other records.
You appear to be storing each field of the record in a separate collection within LogFile. This seems a very strange way to store your data.
If you sort one of these collections, then of course it will bear no relationship to the other fields any longer since they are unrelated. There are huge areas for bugs too if you are relying on all the collections tallying up (eg if a field is missing from one of the parsed records)
Instead you should be have a class that represents a SINGLE record, and then Logfile has a SINGLE collection of these records. eg:
public class ReplicateBlock
{
public string ReplicateId { get; set; }
public string AssayNumber { get; set; }
public DateTime InitiationDate { get; set; }
//etc
}
public class LogFile
{
public List<ReplicateBlock> ReplicateBlocks = new List<ReplicateBlock>();
}
I have to say that your code is very difficult to follow. The fact that all your functions are static makes me think that you're not particularly familiar with object oriented programming. I would suggest getting a good book on the subject.
I'm going to do a project task as the last part of a programming course in C#, and I have choosen to do a media library for storing information of DVD movie collections. The thing I need some advice is about how to save this information since the task should be solved by using simple text files and not databases which had been better.
So, which is the best way to save this information to be able to add, edit and allow search for titles and actors and also sort after genre to mention some of the idéas I have for this program.
Should I just save the information by just add the title, year, playtime, director, actor1, actor2, actor3, genre, grade, comment for each movie lika a row at the end of the file? Can I search in the file or should I, in some way, first read it all into an array, and then do the serach and perhaps edit and save the complete file again?
Or is it better to use XML like this:
<movie_collections>
<movie>
<title=Tron Legacy"></title>
<year=2010></year>
<playtime=120></playtime>
etc.
</movie>
</movie_collections>
If I use XML, can I search for a title and just load or edit that part? Are there better alternatives than these?
You may store the data in XML file. An XML file can store data similar to a database table. You can add multiple records, hierarchical data etc... You may easily query the data using LINQ to XML.
If you dont want to use LINQ to XML, You can use so typical XMLDocument to handle the XML data.
Maybe you're approaching this at too low a level. The file is realistically just for persistance rather than implement your own DB as a file.
Just implement the object that you're after and serialize it out imo.
public class MovieCollections
{
private ICollection<Movie> Movies {get; set;}
// etc...
}
public class Movie
{
public string Title {get; private set;}
public int PlayTime {get; private set;}
public DateTime ReleaseDate {get; private set;}
}
Serialize instance of MovieCollections.
Keep MovieCollections perhaps as a Singleton instance since you only want one collection.
By the way this seems to be very general question & a quite common homework question too!!
I think you should google for this, you will get better projects ideas.
Something similar on this lines :
1.Simple Movie Database in C# using Microsoft Access
2. Create a Movie Database Application
3. imdbapi
& finally
4. SO's similar post
And as far as comparison between a database & XML is concern, I did recommend you a database because several pros over XML as far such type of project is considered.
As #M_Affifi suggests, think of all your interaction with your data (e.g. sort, add, find, delete, update, etc) through your objects which reside in memory (e.g. Instance of MovieCollections class and instances of Movie class), the XML file is only used to store the state of your objects, so you have that data available next time you run your application.
Once you're done manipulating your data just serialize it to XML. Serialization is just the process to convert your objects into a format that you can store or transfer, in this case will be conversion to XML. Take a look at this reference, Your serialization needs are very basic, so you'll need just few lines of code.
System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(YourObject.GetType());
x.Serialize(Console.Out, YourObject)
Finally, I would suggest thinking of your application logic. When program starts you may want to check if your Movies.xml exists, if so, then 'Deserialize' it, which means loading it from XML to your object collection. If XML file doesn't exist then just start manipulating your objects and when you're done give the option to Save(Serialize). I can help with the serialization/deserialization process if needed.
Hope this is helpful
;
I have a challenging question. My requirement is like i have a xml file which has values like
Name 0,20
age 21,4
This is like field names values will be there in the following position. I have 100 records like this in my xml file. I want to dynamically read this xml file which is the layout. and to compare two output files generated by two different companies. Output1 and Output2... where the values for Name at the position and age at the position will be there. I want to read the values and manually show the user that there is a differences in the two files at these areas.
Is there any dynamic viewer control available in .net, C# windows , wpf app to display these differences in two files?
this is not a control but a full file difference tool written in c#. Source code is available as well(CharityWare).
http://www.menees.com/
You can see the source code to see how the file difference is implemented.
One of several possible solutions that come to my mind is:
Use XPathNavigator to parse those 3, in practice, files and compare their content.
For visualization use WPF RichTextBox, were you can load complete formatted text and format/color part of it, which you can use for highlighting the differnces found.
WPF RichTextBox Sample1
WPF RichTextBox Sample2
You need to consider that the WPF TextBox is, basically, easy to use but also addicted to consume a lot of memory, so keep an eye on the memory consumption of your app.
There are, naturally other editors, that you can use, like Scintilla, but I think the story in this case become more complicated.
EDIT:
There could be another non programming solution, is just using WinMerge and run that program (which is free) with required parameters. So it will care about showing/highlighting the differences found between 2 different files.
Choice is up to you.
The existing DataGrid class in WPF will meet this need nicely if you design a class like...
public class Difference
{
public string PropertyName { get; set; }
public string File1Value { get; set; }
public string File2Value { get; set; }
}
This class would hold the differences. You would create an instance of this class each time you found a difference you wanted the user to see, and you would add each instance to a collection...
public ObservableCollection<Difference> Differences = new ObservableCollection<Difference>();
And this collection would be bound to the ItemsSource property of the DataGrid.
Using this approach, you would not need to create a new control or use a 3rd party control. If you wanted to tart up the column headers, you can use DataGridColumns and Templates to that end.
I am tacking a large refactor of a project, and I had asked this question to confirm/understand the direction I should go in and I think I got the answer that I wanted, which is not to throw away years worth of code. So, now begins the challenge of refactoring the code. I've been reading Martine Fowler and Martin Feathers' books, and they have a lot of insight, but I am looking for advice on the ultimate goal of where I want the application to be.
So to reiterate the application a little bit, its a dynamic forms system, with lots of validation logic and data logic between the fields. The main record that gets inserted is the set of form fields that is on the page. Another part of it is 'Actions' that you can do for a person. These 'Actions' can differ client by client, and there are hundreds of 'Actions'. There is also talk that we can somehow make an engine that can eventually take on other similar areas, where a 'person' can be something else (such as student, or employee). So I want to build something very de-coupled. We have one codebase, but different DBs for different clients. The set of form fields on the page are dynamic, but the DB is not - it is translated into the specific DB table via stored procs. So, the generic set of fields are sent to the stored proc and the stored proc then decides what to do with the fields (figure out which table it needs to go to). These tables in fact are pretty static, meaning that they are not really dynamic, and there is a certain structure to it.
What I'm struggling specifically is how to setup a good way to do the dynamic form control page. It seems majority of the logic will be in code on the UI/aspx.cs page, because its loading controls onto the webpage. Is there some way I can do this, so it is done in a streamlined fashion, so the aspx.cs page isn't 5000 lines long? I have a 'FORM' object, and one of the properties is its' 'FIELDS'. So this object is loaded up in the business layer and the Data layer, but now on the fron end, it has to loop through the FIELDS and output the controls onto the page. Also, someway to be able to control the placement would be useful, too - not sure how do get that into this model....
Also, from another point of view - how can I 'really' get this into an object-oriented-structure? Because technically, they can create forms of anything. And those form fields can represent any object. So, for example, today they can create a set of form fields, that represent a 'person' - tomorrow they can create a set of form fields that represent a 'furniture'. How can I possibly translate this to to a person or a furniture object (or should I even be trying to?). And I don't really have controls over the form fields, because they can create whatever....
Any thought process would be really helpful - thanks!
How can I possibly translate this to to a person or a furniture object
(or should I even be trying to?)
If I understand you correctly, you probably shouldn't try to convert these fields to specific objects since the nature of your application is so dynamic. If the stored procedures are capable of figuring out which combination of fields belongs to which tables, then great.
If you can change the DB schema, I would suggest coming up with something much more dynamic. Rather than have a single table for each type of dynamic object, I would create the following schema:
Object {
ID
Name
... (clientID, etc.) ...
}
Property {
ID
ObjectID
Name
DBType (int, string, object-id, etc.)
FormType ( textbox, checkbox, etc.)
[FormValidationRegex] <== optional, could be used by field controls
Value
}
If you can't change the database schema, you can still apply the following to the old system using the stored procedures and fixed tables:
Then when you read in a specific object from the database, you can loop through each of the properties and get the form type and simple add the appropriate generic form type to the page:
foreach(Property p in Object.Properties)
{
switch(p.FormType)
{
case FormType.CheckBox:
PageForm.AddField(new CheckboxFormField(p.Name, p.Value));
break;
case FormType.Email:
PageForm.AddField(new EmailFormField(p.Name, p.Value));
break;
case FormType.etc:
...
break;
}
}
Of course, I threw in a PageForm object, as well as CheckboxFormField and EmailFormField objects. The PageForm object could simply be a placeholder, and the CheckboxFormField and EmailFormField could be UserControls or ServerControls.
I would not recommend trying to control placement. Just list off each field one by one vertically. This is becoming more and more popular anyway, even with static forms who's layout can be controlled completely. Most signup forms, for example, follow this convention.
I hope that helps. If I understood your question wrong, or if you'd like further explanations, let me know.
Not sure I understand the question. But there's two toolboxes suitable for writing generic code. It's generics, and it's reflection - typically in combination.
I don't think I really understand what you're trying to do, but a method using relfection to identify all the properties of an object might look like this:
using System.Reflection;
(...)
public void VisitProperties(object subject)
{
Type subjectType = subject.GetType();
foreach (PropertyInfo info in subjectType.GetProperties()
{
object value = info.GetValue(subject, null);
Console.WriteLine("The name of the property is " + info.Name);
Console.WriteLine("The value is " + value.ToString());
}
}
You can also check out an entry on my blog where I discuss using attributes on objects in conjunction with reflection. It's actually discussing how this can be utilized to write generic UI. Not exactly what you want, but at least the same principles could be used.
http://codepatrol.wordpress.com/2011/08/19/129/
This means that you could create your own custom attributes, or use those that already exists within the .NET framework already, to describe your types. Attributes to specify rules for validation, field label, even field placement could be used.
public class Person
{
[FieldLabel("First name")]
[ValidationRules(Rules.NotEmpty | Rules.OnlyCharacters)]
[FormColumn(1)]
[FormRow(1)]
public string FirstName{get;set;}
[FieldLabel("Last name")]
[ValidationRules(Rules.NotEmpty | Rules.OnlyCharacters)]
[FormColumn(2)]
[FormRow(1)]
public string LastName{get;set;}
}
Then you'd use the method described in my blog to identify these attributes and take the apropriate action - e.g. placing them in the proper row, giving the correct label, and so forth. I won't propose how to solve these things, but at least reflection is a great and simple tool to get descriptive information about an unknown type.
I found xml invaluable for this same situation. You can build an object graph in your code to represent the form easily enough. This object graph can again be loaded/saved from a db easily.
You can turn your object graph into xml & use xslt to generate the html for display. You now also have the benefit of customising this transform for differnetn clients/versions/etc. I also store the xml in the database for performance & to give me a publish function.
You need some specific code to deal with the incoming data, as you're going to be accessing the raw request post. You need to validate the incoming data against what you think you was shown. That stops people spoofing/meddling with your forms.
I hope that all makes sense.
SOME CONTEXT
one of my projects requires carrying around some of "metadata" (yes I hate using that word).
What the metadata specifically consists of is not important, only that it's more complex than a simple "table" or "list" - you could think of it as a mini-database of information
Currently I have this metadata stored in an XML file and have an XSD that defines the schema.
I want to package this metadata with my project, currently that means keeping the XML file as a resource
However, I have been looking for a more strongly-typed alternative. I am considering moving it from an XML file to C# code - so instead of using XML apis to traverse my metadata, relying on .NET code via reflection on types
Besides the strong(er) typing, some useful characteristics I see from using an assembly are for this: (1) I can refactor the "schema" to some extent with tools like Resharper, (2) The metadata can come with code, (3) don't have to rely on any external DB component.
THE QUESTIONS
If you have tried something like this, I am curious about what you learned.
Was your experience positive?
What did you learn?
What problems in this approach did you uncover?
What are some considerations I should take into account?
Would you do this again?
NOTES
Am not asking for how to use Reflection - no help is needed there
Am fundamentally asking about your experiences and design considerations
UPDATE: INFORMATION ABOUT THE METADATA
Because people are asking I'll try describing the metadata a bit more. I'm trying to abstract a bit - so this will seem a bit artificial.
There are three entities in the model:
A set of "groups" - each group has a unique name and several properites (usually int values that represent ID numbers of some kind)
Each "group" contains 1 or more "widgets" (never more than 50) - each item has properties like name (therea are multiple names), IDs, and various boolean properties.
Each widget contains a one or more "scenarios". Each "scenario" is documentation- a URL to a description of how to use the widget.
Typically I need to run these kinds of "queries"
Get the names of all the widgets
Get the names of all groups that contain at least one widget where BoolProp1=true
Get given the ID of a widget, which group contains that widget
How I was thinking about modelling the entities in the assembly
There are 3 classes: Group, Widget, Documentation
There are 25 Groups so I will have 25 Group classes - so "FooGroup" will derive from Group, same pattern follows for widgets and documentation
Each class will have attributes to account for names, ids, etc.
I have used and extended Metadata for a large part of my projects, many of them related to describing components, relationships among them, mappings, etc.
(Major categories of using attributes extensively include O/R Mappers, Dependency Injection framework, and Serialization description - specially XML Serialization)
Well, I'm going to ask you to describe a little bit more about the nature of the data you want to embed as resource. Using attributes are naturally good for the type of data that describes your types and type elements, but each usage of attributes is a simple and short one. Attributes (I think) should be very cohesive and somehow independent from each other.
One of the solutions that I want to point you at, is the "XML Serialization" approach. You can keep your current XMLs, and put them into your assemblies as Embedded Resource (which is what you've probably done already) and read the whole XML at once into a strongly-typed hierarchy of objects.
XML Serialization is very very simple to use, much simpler than the typical XML API or even LINQ2XML, in my opinion. It uses Attributes to map class properties to XML elements and XML attributes. Once you've loaded the XML into the objects, you have everything you want in the memory as "typed" data.
Based on what I understand from your description, I think you have a lot of data to be placed on a single class. This means a large and (in my opinion) ugly attribute code above the class. (Unless you can distribute your data among members making each of them small and independent, which is nice.)
I have many positive experiences using XML Serialization for large amount of data. You can arrange data as you want, you get type safety, you get IntelliSence (if you give your XSD to visual studio), and you also get half of the Refactoring. ReSharper (or any other refactoring tool that I know of) don't recognize XML Serialization, so when you refactor your typed classes, it doesn't change the XML itself, but changes all the usage of the data.
If you give me more details on what your data is, I might be able to add something to my answer.
For XML Serialization samples, just Google "XML Serialization" or look it up in MSDN.
UPDATE
I strongly recommend NOT using classes for representing instances of your data. Or even using a class to encapsulate data is against its logical definition.
I guess your best bet would be XML Serialization, provided that you already have your data in XML. You get all the benefits you want, with less code. And you can perform any query on the XML Serializable objects using LINQ2Objects.
A part of your code can look like the following:
[XmlRoot]
public class MyMetadata
{
[XmlElement]
public Group[] Groups { get; set; }
}
public class Group
{
[XmlAttribute]
public string Name { get; set; }
[XmlAttribute]
public int SomeNumber { get; set; }
[XmlElement]
public Widget[] Widgets { get; set; }
}
public class Widget
{
...
}
You should call new XmlSerializer(typeof(MyMetadata)) to create a serializer, and call its Deserialize method giving it the stream of your XML, and you get a filled instance of MyMetadata class.
It's not clear from your description but it sounds like you have assembly-level metadata that you want to be able to access (as opposed to type-level). You could have a single class in each assembly that implements a common interface, then use reflection to hunt down that class and instantiate it. Then you can hard-code the metadata within.
The problems of course are the benefits that you lose from the XML -- namely that you can't modify the metadata without a new build. But if you're going this direction you probably have already taken that into account.