I have following entities: Game, Team, Player, Contract, with the following relationships: One Team, has many Players, which have many Contracts. Also Team has many Games. API already handles standard CRUD operations for each entity alone, but I'm planning to create one request to insert/update whole tree. Here's how:
It would be insert/update Game and in the json body (beside the game info) I would attach two corresponding teams, along with their players and players contracts. All in one json. In the request I will check with the database, and potentially insert/update sub-entities (for example if the team doesn't exists I will insert it, otherwise update it).
My questions:
Is it a good practice to do bulk request on multiple different entities?
What should be the method type? POST or PUT?
What should be return status? (I thought since it's a transaction, it's either all good or fail, thus 201 would be appropriate. I'm only not sure, because there could be some updates along the way, so I wanted to ask)
Thanks!
PS. Optional to read: Why? It's huge improvement from the client perspective, because there are a lot of foreign keys (specially if the tree gets bigger, with statistics etc), and API can do it all in one transaction. For example: When adding new game with new team, with new players and new contracts, to be able to insert game, I need FK of the team, contracts need FKs of the player and the team. So doing it one by one, I would need to send POST then wait for returned FK, so that I can send more request. If something fails along the way, there is an issue reverting all the changes.
There is no problem doing that, actually is pretty normal.
1 - If it is part of your business rules, go for it, there is nothing saying that it is wrong, actually is a good practice to do everything you need inside a single request (queue things, save a bunch of entities, whatever), this avoids business rules leaking to the front-end and additional work like perform fallbacks if anything goes wrong at the previous request.
2 - Depends, if the main action is to add something, then POST, else PUT fits nicelly
3 - If you are creating a resource 201 is the one to go, if you are updating something, 204-No Content (the server processed the request but there is nothing to return to you)
Related
I have an application with several Web API controllers and I now I have a requirement which is to be able to filter GET results by the object properties. I've been looking at using OData but I'm not sure if it's a good fit for a couple reasons:
The Web API controller does not have direct access to the DataContext, instead it gets data from our database through our "domain" layer so it has no visibility into our Entity Framework models.
Tying into the first item, the Web API deals with lightweight DTO model objects which are produced in the domain layer. This is effectively what hides the EF models. The issue here is I want these queries to be executed in our database but by the time the Web API method gets a collection from the domain layer all of the objects in the collection have been mapped to these DTO objects, so I don't see how the OData filter could possibly do it's job when the objects are once-removed from EF in this way.
This item may be the most important one: We don't really want to allow arbitrary querying against our Web API/Database. We just sort of want to leverage this OData library to avoid writing our own filters, and filter parsers/builders for every type of object that could be returned by one of our Web API endpoints.
Am I on the wrong track based on #3? If not, would we be able to use this OData library without significant refactoring to how our Web API and our EF interact?
I haven't had experience with OData, but from what I can see it's designed to be fed a Context and manages the interaction and returning of those models. I am definitely not a fan of returning Entities in any form to a client.
It's an ugly situation to be in, but when faced with this, my first course of action is to push back to the clients to justify their searching needs. The default request is almost always "Well, it would be nice to be able to search against everything." My answer to that is that I don't want to know what you want, I want to know what you need because I don't want to give you a loaded gun to shoot your own foot off with and then have you blame me because the system came grinding to a halt. Searching is a huge performance killer if it's too open-ended. It's hard to test for accuracy/relevance, and efficiently index for 100% of possible search cases when users only need 25% of those scenarios. If the client cannot tell you what searching they will need, and just want everything because they might need it, then they don't need it yet.
Personally I stick to specific search DTOs and translate those into the linq expressions.
If I was faced with a hard requirement to implement something like that, I would:
Try to push for these searches/reports to be done off a reporting replica that is synchronized with the live database. (To minimize the bleeding when some idiot managers fire up some wacky non-indexed search criteria so that it doesn't tie up the production DB where people are trying to do work.)
Create a new bounded DbContext specific for searching with separate entity definitions that only expose the minimum # of properties to represent search criteria and IDs.
Hook this bounded context into the API and OData. It will return "search results". When a user selects a search result, use the ID(s) against the API to load the applicable domain, or initiate an action, etc.
no. 1. is optional, a nice to have provided they can live with searches not "seeing" updated criteria until replicated. (I.e. a few seconds to minutes depending on replication strategy/size) Normally these searches are used for reporting-type queries so I'd push to keep these separate from the normal day-to-day searching options that users use. (I.e. an advanced search option or the like.)
I want to save edited values from a WPF mobile app, via a Web API, as the user tabs out of each field. So on the LostFocus event.
When using EF then the whole entity graph is posted (put) to the Web API each time a field is updated. Even if I just make a DTO for the basic fields on the form, I would still be posting unnecessary data each time.
I was thinking of forgetting about EF in the Web API and simply posting the entity ID, field name and new value. Then in the controller, create my own SQL update statement and use good old ADO.Net to update the database.
This sounds like going back to the noughties or even the nineties, but is there any reason why I should not do that?
I have read this post which makes me lean towards my proposed solution.
Thanks for any comments or advice
Sounds like you are trying to move away from having a RESTful Web API and towards something a little more RPC-ish. Which is fine, as long as you are happy that the extra hassle of implementing this is worth it in terms of bandwith saved.
In terms of tech level, you're not regressing by doing what you proposed; I use EF every day but I still often need to use plain old ADO.NET every now and then and there is a reason why it's still well supported in the CLR. So there is no reason not to, as long as you are comfortable with writing SQL, etc.
However, I'd advise against your current proposal for a couple of reasons
Bandwidth isn't necessarily all that precious
Even for mobile devices, sending 20 or 30 fields back at a time probably isn't a lot of data. Of course, only you can know for your specific scenario if that's too much but considering the wide-spread availability of 3 & 4G networks, I wouldn't see this as a concern unless those fields contain huge amounts of data - of course, it's your use case so you know best :)
Concurrency
Unless the form is actually a representation of several discrete objects which can be updated independently, then by sending back individual changes every time you update a field, you run the risk of ending up with invalid state on the device.
Consider for example if User A and User B are both looking at the same object on their devices. This object has 3 fields A, B, C thus:
A-"FOO"
B-"42"
C-"12345"
Now suppose User A changes field "A" to "BAR" and tabs out of the field, and then User B changes field "C" to "67890" and tabs.
Your back-end now has this state for the object:
A - "BAR"
B - "42"
C - "67890"
However, User A and User B now both have an incorrect state for the Object!
It gets worse if you also have a facility to re-send the entire object from either client because if User A re-sends the entire form (for whatever reason) User B's changes will be lost without any warning!
Typically this is why the RESTful mechanism of exchanging full state works so well; you send the entire object back to the server, and get to decide based on that full state, if it should override the latest version, or return an error, or return some state that prompts the user to manually merge changes, etc.
In other words, it allows you to handle conflicts meaningfully. Entity Framework for example will give you concurrency checking for free just by including a specially typed column; you can handle a Concurreny exception to decide what to do.
Now, if it's the case that the form is comprised of several distinct entities that can be independently updated, you have more of a task-based scenario so you can model your solution accordingly - by all means send a single Model to the client representing all the properties of all of the individual entities on the form, but have separate POST back models, and a handler for each.
For example, if the form shows Customer Master data and their corresponding Address record, you can send the client a single model to populate the form, but only send the Customer Master model when a Customer Master field changes, and only the Address model when an address field changes, etc. This way you can have your cake and eat it because you have a smaller POST payload and you can manage concurrency.
I have a form with few tabs, and in each tab an grid control. When user select a row to be deleted i want to remove it from the grid, and if the object exist in the database remove it too, but not permanent - only if and when user clicks save on form.
For now, if object doesn't exist in db i remove it from the list, and if objects exist in db i delete it from db and remove it from the list. But, if user clicks Cancel button he expects row/s not to be deleted from database.
I have two possible solutions on my mind: 1) - remove object from list, and if objects exist in db add it to the list of objects to be deleted 2) - implement another list, getter will return only objects with state != ToBeDeleted (performance?)
Note: i'm not using ORM tool, working with my own ado.net based data access framework.
I think the case you are descibing just asks pretty much for a Transaction.
ADO.Net handles them easily, provided you are using a reasonable database engine (so: no SqlServerCE for example:))
See for example the TransactionScope class. You construct such object before interacting with the database, and the changes will be "commited" if and only if you call Complete(). If you just leave it alone or if you Dispose() it, the transaction will be cancelled and all changes on the DB will be "rolledback", so, reverted.
So, in your case, you may open the transaction in the Form's ctor or onLoaded(), and Complete() at "save", and Dispose() at any other window closing.
While this is the normal way of handling such things for small systems, especially single-user ones, but be careful: if your system has to handle many concurent useres, you may be not able to use it in this way. The Transaction blocks rows and tables until it is completed or cancelled, and the therefore "other users" may see large delays..
So, how many users do you have to support and how often they will try to edit the same things?
-- edit: (10 users)
With that many users, you will want to avoid long-running transactions. Opening transaction at form-load will be unacceptable, and will lock many users away until that one current user closes the window. But, using transactions at Save() that push all the changes in one batch are OK.
Of course, if you can eliminate transactions at all - that's great! But, it is very hard thing to do if you also need to preserve data integrity.. To eliminate the need of transactions, almost always you have to redesign both the data structure on the DB side, and the way you obtain and work with the data. If you want to redesign both, then I'd really recommend to first try redesigning it to use some existing data-access framework, as even the basic .Net ADO has really nice features for online editing of databases held at SqlClient-compliant databases..
So, assuming you don't want to rewrite/rethink most of your code, you just need to buffer the data and also, delay all of the actual operations on the database.
You may want to do it in a "simple" form: when you display your form, instead of binding your Form directly to the database-driven datasources - download all required data to some BindingList<>s, DataTables, etc - whatever container you like. And bind your form to them instead. Probably you have something like that already set up. But, the important thing is that all those datacontainers must be offline or at least readonly+delayloaded.
Next, you've got to intercept all operations that the user performs on the UI. Surely you have it done already, as I'm assuming the application works:) As your Forms are bound to that offline cached items, your application should perform the operation on that cached data, and don't touch the database at all. But there's more: along with performing them on cached data, you should record what happens to which table.
Then, when finally the user stops playing around and presses CANCEL :) - you just trash everything and close the form. database not changed.
On Save - you open a fresh transaction, then iterate over the list of changes and effectively replay your recorder changes on the database, then commit transaction.
Please note two things though: the database could have changed during the time the users cached the data and the time he pressed Save. You have to detect this and abort, or resolve conflicts. You should do that inside that transaction, either during or before executing the recorded changes. You may detect it by simply comparing the online data with offline cached data (the unchanged original values, not those modified by user), or you may use some other mechanisms like OptimisticLocking and just compare the version tags on the rows.
If you don't like record-replay, you may implement a "DIFF"ing utility that takes the modified offline data and compares it in a generic way with the current-online tables. This is somewhat harder, but has a bonus: with such utility, you can initially doubly-cache the data: one copy for offline reference (just stored and never touched by the user) and one copy for offline editing (all those bound to the Forms). Now, upon Save you open transaction and diff the reference data against the online database. If there are any difference - you've just detected a collision. Solve/merge/abort/etc. If no differences, then you diff the modified data against online-data, and apply all differences found to the database and commit transaction.
Either of those methods has its pros and cons: aside from difficulty of implementation, there's memory issues of caching, latency issues if you dare to copy too large tables, etc.
But - once solved, it would work pretty nice.
And as you finish, you can go and boast that you have just implemented a smaller sis' of the DataSet+DataTable. I'm not joking, and I'm not laughing at you. I'm just trying to show you why everyone is telling you to rewise your DAO layer and try understanding and using the hard work that was already done for you by the platform designers/developers :)
Anyways, I've said you can avoid the clashes and transactions at all if you rethink your data structure.. For example: why do you DELETE the rows at all? I know there's a nifty DELETE statement in the SQL, but, well, do you really need to delete that row? Can't you just add some 'bool isDeleted' column and when user deletes the row from the Grid - just make set that rowcell to True and make the application filter-out any isDeleted=true rows and not show them? and not include them in views and aggregations? Bonus: sys/db admins now have a magic tool: undelete..
Let's take it further: do you need to UPDATE the rows? Maybe you can just APPEND some information that from (this-date) that row should have a new price? of course, the structure must be greatly altered: entities doesn't have properties, but have logs of timestamped property changes (or either the rows must have version numbers and be duplicated..), queries must be done against only the newest versiosn data, etc. Pros: database is now append-only. Transactions, if needed at all, are hyper-short. Cons: SELECT queries are complicated and may be slow, especially when joining many tables..
Pro/Con: and your db actually starts looking very meta- instead of data-base...
Con: and this is really hard task to "upgrade" existing application to such db structure. Writing a new app from scratch and importing data from odl system may be few times faster.
Now, to summarise:
I do not recommend any of the ways described.
First, I recommend you to take some ORM framework like NHibernate, EntityFramework, XPO from DevExpress, or whetever else. Any of them will save you lots of time. Those three I list here even have OptimisticLocking collision detection built-in. Why use SQL-self-written framework when such tools exist?
If not, then next I recommed to use existing tools found in the framework. you use SqlClient, whydontya use DataSet and DataTables? They are provided along with SqlClient and they have many useful mechanisms just built-in, which otherwise you will spend weeks on implementing and testing all by yourself. Learn to use DataSets and its collision detection, and its merging algorithms, and use them. You will loose a bit of time on experimenting and learning, but you will save huge amounts of time on not-reinventing the wheel.
If you really want to do it manually, start with data-caching and record-replay. It is easy to comprehend, it is quite easy to introduce anywhere where you currently use plain SQL queries, and will quickly introduce you to all kinds of cache-syncing and version-checking problems, and you will soon learn in details why all those strange mechanisms in the above-mentioned frameworks were implemented, how they work and what pros/cons they have.
and about the doubly-cached diffing approach.. it will be more tempting to write that record-repay, but please: use it only if you know very well how to detect/solve/merge collisions. Have at least one record-replay approach implemented before you try it..
..and of course yo umay use long-lasting transactions. Dumb-easy to introduce, and they "just irritate" the users.. Well, or even make the system unusable when >90% of the users constantly collide and hit the locks, heh.. No, that was a joke. Don't use long-lasting transactions. They are ok for 1-4 users, or for very sparse databases..
This is a general architecture question, hopefully to folks out there already using EF in final applications.
We have a typical N-Tier application:
WPF Client
WCF Services
EF STE DTO's
EF Data Layer
The application loads all known business types during load time (at the same time as the user logs in) then loads a very large "Work Batch" on demand, this batch is around 4-8Mg and is composed of over 1.000 business objects. When we finish loading this "Batch" we then link everything with the previously loaded business types, etc...
In the end we have around 2K-5K business objects in memory all correctly reference so we can use and abuse LINQ on the client side, we also do some complex math on all these objects on the client side, so we really need the large graph.
The issue comes when we want to save changes to the Database. With such a large object graph, we hardly want to send over everything again through the Network.
Our current aproach, which I dislike, given the complexity of the T4 templates so far, is to detach and attach everything on update. We basically want to update a given object, detach it from the rest of the graph, send it over the network, updated it on the WCF side, and then reattach it again on the client side. The main problem is when you want to update linked objects, let's say you add something that has a reference for something that is also added, then another reference to something modified, etc. This forces a lot of client code to make sure we don't break anything.
All this is done with generated code, so we are talking about 200-800 lines of T4 code per template.
What I'm looking at right now is a way to customize serialization and deserialization of the STE's, so that I can control what is sent over the network or not, and be able to update batches instead of just a single STE. Checking references, see if those references are Unchanged or not; if not don't serialize, if yes serialize and update everything just by attaching it to the context on the WCF side.
After some studying I found 2 solutions to this method.
One is by writing a custom DataContractSerializer.
The second one is by changing the STE template created by EF and playing around with the KnownTypeAttribute, instead of generating it for each reference type, have it reference a method that inspects the object and only marks for serialization references that are not unchanged.
Has anyone ever come across this
issue before?
What solutions did you use?
What problems did you encounter down
the line?
How easy was it to maintain the
templates created?
I don't know whole application design but if you generally load the work batch to the service and then send it to the client to play with it, it looks like service layer is somehow unnecessary and you can directly load data from database (and you will get much better performance). Depending on complexity of computation you can also do some computation directly in the database and you will again get much better performance.
Your approach to save only part of the graph is abuse to STE concept. STE works in manner - you load the graph, modify the graph and save the same graph. If you want to have a big dataset for reading and save only small chunks it is probably better to load data set for reading and once you decide to update a chunk, load only the chunk again, modify it and send it back.
Interfering the internal STEs behavior is imho the best way to lost some changes in some corner / unexpected scenarios.
Btw. this somehow looks like a scenario for syncing local database with a global one - I have never done that but it is quite common in smart-clients.
Sorry about the title - hopefully the question will make clear what I want to know, then maybe someone can suggest a better title and I'll edit it!
We have a domain model. Part of this model is a collection of "Assets" that the user currently has. The user can then create "Actions" that are possible future changes to the state of these "Assets". At present, these actions have an "Apply" method and a reference to their associated "Asset". This "Apply" method makes a modification to the "Asset" and returns it.
At various points in the code, we need to pull back a list of assets with any future dated actions applied. However, we often need to do this within the scope of an NHibernate transaction and therefore when the transaction is committed the changes to the "Asset" will be saved as well - but we don't want them to be.
We've been through various ways of doing this
Cloning a version of the "Asset" (so that it is disconnected from Nhibernate) and then applying the "Action" and returning this cloned copy.
Actually using Nhibernate to disconnect the object before returning it
Obviously these each have various (massive!) downsides.
Any ideas? Let me know if this question requires further explanation and what on earth I should change the title to!
It's been a while since I had any NHibernate fun, but could you retrieve the Assets using a second NHibernate session? Changes made to the Asset will then not be saved when the transaction on the first session commits.
You could manage this with NHibernate using ISession.Evict(obj) or similar techniques, but honestly it sounds like you're missing a domain concept. I would model this as:
var asset = FetchAsset();
var proposedAsset = asset.ApplyActionsToClone();
The proposedAsset would be a clone of the original asset with the actions applied to it. This cloned object would be disconnected from NHibernate and therefore not persisted when the Unit of Work commits. If applying the actions is expensive, you could even do the following:
asset.ApplyProposedChanges(proposedAsset);
I have been working around a similar problem where performance was also an issue, thus it was not possible to re-load the aggregate using a secondary (perhaps stateless) session. And because the entities that needed to be changed "temporarily" where very complex, I could not easily clone them.
What I ended up with was "manually" rolling back the changes to what would be the assets in your case. It turned out to work well. We stored each action applied to the entity as a list of events (in memory that is). After use the events could be re-read and each change could be rolled back by a counter-action.
If it's only a small variety of actions that can be applied, I would say it's easily manageable to create a counter-action for each, or else it might be possible to create a more generic mechanism.
We had only four actions, so we went for the manual edition.
Sounds like you want to use a Unit of Work wrapper, so you can commit or revert changes as needed.