I'm thinking of a mid / large scale project, which will need to store different data types and present them to different clients.
What I'm struggling now is, how to build a data and service layer that can capable of storing different types of objects and query them when needed.
As an example, think of a client - server application in which, clients can only read each individual server's broadcasts, and now think of a scenario where a flower shop and restaurant broadcasts their data to a person on the street with a smart phone.
class SoCalledServer
{
public AccessibleClientData Broadcast(ClientData broadcastMessage)
{
Broadcast(broadcastMessage)
}
}
class RestaurantClient : AbstractClient
{
public SomeGenericDataType menu;
public RestaurantClient()
{
menu = new SomeGenericDataType<List>();
menu.Add("Sushi");
menu.Add("Fried potatoes");
}
public override void BeginBroadcast()
{
server.Broadcast(menu);
}
}
class FlowerShopClient : AbstractClient
{
public SomeGenericDataType flowersOnSale;
public FlowerShopClient()
{
flowersOnSale = new SomeGenericDataType<List>();
flowersOnSale.Add("Daisy");
flowersOnSale.Add("Rose");
}
public void BeginBroadcast()
{
server.Broadcast(flowersOnSale);
}
}
In this example, I have two different types of data (one is a restaurant's menu, and the other is flower shop's flowers) which can have different members of its own (eg. menu has prices and ingredients, flower shop's data has flower names and a description, quantity and / or price etc...) and this "client" samples can be extended.
How should I model such type of application? What kind of database schema I should use to store unidentified and various types of data? How my server and client application should communicate with each other? And the most important how should client get the broadcasted data type (from the generic type)?
How will this service manipulate this data? Will it only save it? Will it do some computations with this data? Your description is too generic.
Assuming you only want to write/persist/read data, you can simply save strings and let client do the parsing themselves. You can query based on id. Key/value and document databases work like this.
For anything more, you should think what the responsibility of the service should be and design the internal structure accordingly.
Another idea is to de/serialize them as XML or Json. Some hints:
// get stuff here
String json = GetJsonString(expression));
List<T> result;
using (var ms = new MemoryStream(Encoding.Unicode.GetBytes(json)))
{
var serializer = new DataContractJsonSerializer(typeof(List<T>));
result = (List<T>)serializer.ReadObject(ms);
}
Or XML:
http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx
You can convert the objects to XML/Json for transmission. Then, for storing, deserialize them as anonymous objects (no specified class) on the side where the class is unknown.
With this, you have always the possiblity to store all of the data even when the classes are unknown. Everywhere, everytime.
Related
I have created a class to store data from API calls I am making. It returns JSON with some meta information, and then an array of data depending on the call being made. The meta information will always have the same fields, so I have created a "Root" class for this, but the data will be different depending on the call being made, so I have created different classes for each type of data, e.g. user data, company data, etc. As shown below, I currently have the "data" property set to a list of objects, but I am trying to figure out the best way to incorporate the different types of data that can be returned, since it will vary based on the call being made.
Right now I have the data saved as a list of objects, but I would like this to change depending on what data I am receiving. Like, if I am retrieving users, I would like for it to be a list of users.
What is the ideal way to accommodate for this? The only way I can think to do it now is to create a different "Root" class for every type of data I am expecting to receive, but that doesn't feel like it should be the most concise way to do it. I was looking into making this a factory design pattern but I wasn't sure that it fit this scenario.
Just use a generic base class:
public abstract class ApiCallResult<T>
{
// With your properties
// public int Limit { get; set; }
// [...]
//
public IEnumerable<T> Data { get; set; }
}
Then define a result per api call.
public class UserApiCallResult : ApiCallResult<User>
{
}
Created a small working example here:
dotnet fiddle
I've got a scenario where I need to backup thousands of rather small (1-3KB) files (in Azure blob storage, but that's not the point) which I have as a list of these models (source code of which I don't own - it's Azure SDK):
class Model {
public string Name {get;set;}
public Stream Data {get;}
//bunch of other things I'd like to ignore
}
My best attempt (from performance point of view) is now merging them all in a single file using JsonTextWriter from Json.NET, but obviously I don't need text format here and it introduces a lot of overheads. I'm wondering if there is any (binary?) serializer available which won't require me to decorate existing model with some specific attributes and will also have a nice API including something like
var writer = new MagicWriterThatImLookingFor();
foreach(var model in models){
writer.WriteString(model.Name);
writer.WriteByteArray(model.Data.ToArray());
}
with corresponding deserialize?
I am building webservices for many different clients to connect to a database of automotive parts. This parts have a wide variety of properties. Different clients will need different subsets of properties to 'do their thing.'
All clients will need at least an ID, a part number, and a name. Some might need prices, some might need URL's to images, etc. etc. The next client might be written years from now and require yet a different subset of properties. I'd rather not send more than they need.
I have been building separate 'PartDTO's' with subsets of properties for each of these requirements, and serving them up as separate webservice methods to return the same list of parts but with different properties for each one. Rather than build this up for each client and come up with logical names for the DTO's and methods, I'd like a way for the client to specify what they want. I'm returning JSON, so I was thinking about the client passing me a JSON object listing the properties they want in the result-set:
ret = { ImageUrl: true, RetailPrice: true, ... }
First off, does this make sense?
Second, What I'd rather not lose here is the nice syntax to return an IEnumerable < DTO > and let the JSON tools serialize it. I could certainly build up a 'JSON' string and return that, but that seems pretty kludgey.
Suggestions? C# 'dynamic'?
This is a very good candidate for the Entity-Attribute-Value model. Basically you have a table of ID, Name, Value and you allow each customer/facet to store whatever they want... Then when they query you return their name-value pairs and let them use them as they please.
PROS: super flexible. Good for situations where a strong schema adds tons of complexity vs value. Single endpoint for multiple clients.
CONS: Generally disliked pattern, very hard to select from efficiently and also hard to index. However, if all you do is store and return collections of name-value, it should be fine.
I ended up going the dictionary-route. I defined a base class:
public abstract DictionaryAsDTO<T> : IReadOnlyDictionary<string, object>
{
protected DictionaryAsDTO(T t, string listOfProperties)
{
// Populate an internal dictionary with subset of t's props based on string
}
}
Then a DTO for Part like so:
public PartDTO : DictionaryAsDTO<Part>
{
public PartDTO(Part p, string listOfProperties) : base(p, listOfProperties) {}
// Override method to populate base's dictionary with Part properties based on
// listOfProperties
}
Then I wrote a JSON.NET converter for DictionaryAsDTO which emits JSON-y object-properties instead of key-value-pairs.
The web service builds an IEnumerable based on queries that return IEnumerable and serializes them.
Viola!
I have an MVC web app where users upload a text file and I parse it out.
The requirement just changed and they will be uploading multiple files of the same kind now. I parse a single file by sending a file-path to the method below, ReadParts which opens a stream and calls the method parseReplicateBlock to retrieve desired fields. For multiple files I could read all the files into one big stream but I am afraid it could exceed the buffer limit etc.
So I am thinking to parse file by file and populate results into an object. My requirement then, is to sort the records based on a date field.
I just need some help in how to write this method ReadLogFile in a better way, espceially for sorting based on initialtionDate and initiationTime. I want to find the minimum record based on initiationDate and Time and then do some other logic.
The problem is if I sort the list member within the object, I would loose positiong of the other records.
You appear to be storing each field of the record in a separate collection within LogFile. This seems a very strange way to store your data.
If you sort one of these collections, then of course it will bear no relationship to the other fields any longer since they are unrelated. There are huge areas for bugs too if you are relying on all the collections tallying up (eg if a field is missing from one of the parsed records)
Instead you should be have a class that represents a SINGLE record, and then Logfile has a SINGLE collection of these records. eg:
public class ReplicateBlock
{
public string ReplicateId { get; set; }
public string AssayNumber { get; set; }
public DateTime InitiationDate { get; set; }
//etc
}
public class LogFile
{
public List<ReplicateBlock> ReplicateBlocks = new List<ReplicateBlock>();
}
I have to say that your code is very difficult to follow. The fact that all your functions are static makes me think that you're not particularly familiar with object oriented programming. I would suggest getting a good book on the subject.
I'm currently developing an application based on no-sql (using raven db). The core aspect of this application is a tree-like data structure with many nodes, subnodes and so on.
Currently, each node or subnode is represented by a c# object. A parent-child relationship is made with a collection of subnodes on the parent node, a forward-only relationship.
The whole thing is handled by ad hoc forms, in an Mvc application, with proper GETs and POSTs for each data type. The whole graph is stored as JSON on Raven DB.
Now the goal is to modify the UI part using knockoutjs. Since KO works with json data structures as well, I was wondering if there is a way to make the ravendb json structure "knockout compatible", meaning I can directly use it without having to make a KO specific structure (to implement observables, etc) and then create a mapping between the two.
A sample of the object graph:
public class NodeA
{
public string Name {get;set;}
public List<SubNode> Childs {get;set;}
}
public class SubNode
{
public string Name {get;set;}
public bool SomeBool {get;set;}
}
public class NodeB
{
public string Name {get;set;}
public int SomeInt {get;set;}
}
public class GraphToStore
{
public List<NodeA> NodeAList {get;set;}
public List<NodeB> NodeBList {get;set;}
}
The read/write part would still be handled server side, with ajax calls after stuff gets updated on the UI. Validation would be server-side and returned to the client via ajax calls too. My problem is as I said making the ravendb json work with knockoutjs, otherwise I have to reconstruct the whole thing and map it, and the graph is huge (50+ classes).
Take a look at Knockout-mapping-plugin. It will "automagically" generate the knockout compatible viewmodel with one call.
You would do something like
var viewModel.myRavenDbData = ko.mapping.fromJSON(json data variable);
var unwrappedData = viewModel.myRavenDbData(); // need only if viewModel.myRavenDbData is an observable
After you have this working, breakpoint after the call to mapping and explore the data structure. Generally, it will look like your data structure with ko.observables for the actual values. All the nodes needed for navigation will be normal javascript objects.
Yes you can use the Knockouts Mapping capabilities and create ViewModels directly from the model objects. But, I have two points:
1) I think that the fact that the objects are stored in RavenDB does not metter. Your MVC applications retrieves the objects from RavenDB - so they are deserialized from JSON and than they are served to your JS page via REST interface, so they are serialized again into JSON. So you are not working directly with the RavenDB's JSON structure, it is a standard CLR object serialized to JSON.
If you want to work directly with Raven, you have to plug your application directly to raven's interface - and that is not a good idea (but of course in the metter of performance it should work great).
2) I don't think that it is good idea to use your model objects as ViewModels, only by using the knockout mapping plugin.
Soon you will need to add some logic to the view model. Either for computing values to be showed in the view, or adding some action logic (save/edit...etc).
For the first case, you can define your viewmodels on the server side and use the mapping plugin.
For the later, you will have to write the view models in javascript anyway. I would recomend start writing the viewmodels in javascript directly.
It works the best for me.