We Keep Coding

Microservices design part in WebApi

Hi I am trying to create a project skeleton uses CQRS pattern and some external services. Below are the structure of the solution.
WebApi
Query Handlers
Command Handlers
Repository
ApiGateways ( here is the interfaces and implementation of microservice calls)
We want to keep controller as thin. So we are using query handlers and command handlers to handle respective operations.
However, we use a external microservices to get the data we are calling them from Query handlers.
All the http clinet construction and calls will be abstracted in them.The response will be converted to a view model and pass it back to Query handler.
We name it as a ApiGateways. But it is not composing from multiple services.
How do we call this part in our solution? Proxy or something? Any good example for thin controllers and microservice architecture

We name it as API Gateways. But it is not composed from multiple
services. How do we call this part in our solution? Proxy or
something? Any good example for thin controllers and microservice
architecture
Assumption:
From the image you attached, I see Command Handler and Query Handler are calling "external/micro-services". I guess by this "external/micro-services" you mean that you are calling another micro-service from your current micro-service Handler(Command and Query). These "external/micro-services" are part of your architecture and deployed on the same cluster and not some external system that just exposes a public API?
If this is correct I will try to answer based on this assumption.
API Gateway would probably be misleading in this case as the concept of API Gateway is something different then what you are trying to do here.
API Gateway per definition:
Quote from here:
An API Gateway is a server that is the single entry point into the
system. It is similar to the Facade pattern from object-oriented
design. The API Gateway encapsulates the internal system architecture
and provides an API that is tailored to each client. It might have
other responsibilities such as authentication, monitoring, load
balancing, caching, request shaping and management, and static
response handling.
What you actually are trying to do is to call from your Command or Query Handler from one of your micro-service A another micro-service B. This is an internal micro-service communication that should not be done through API Gateway as that would be the approach for outside calls. For example, with "outside calls" I mean frontend application API or public API calls that are trying to call your micro-services. In that case, you would use API Gateways.
A better name for this component would be something like "CrossMicroServiceGateway" or "InterMicroServiceGateway"; if you want to have it as the full CQRS way you could have it like a direct call to other Command or Query and then you could use something like "QueryGate" or "CommandGate" or similar.
Other suggestions:
WebApi
Query Handlers
Command Handlers
Repository
API Gateways ( here is the interfaces and implementation of
microservice calls)
This sounds reasonable except for the point of API Gateway which I described above. Of course, it is hard for me to tell based on the limited information that I have about your project. To give you a more precise suggestion here I would need to know whether you use DDD or not? How do you use CQRS and other information?
However, we use an external microservices to get the data we are
calling them from Query handlers. All the HTTP client construction and
calls will be abstracted in them. The response will be converted to a
view model and pass it back to Query handler.
You could extract all this code/logic that handles the cross micro-service communication over HTTP or other protocols, handling general responses and similar to some core library and include it into each of your micro-service as a package. In this way, you will reuse the solution for all your micro-service. You can extend that and add all core domain-agnostic things (like data access or repository base classes, wrappers around HTTP, unit-test infrastructure setup, and similar) to that or other shared libraries. This way your micro-services will only focus on the part of the Domain it is supposed to do.

I think CQRS is the right choice to keep the reading and writing operations decoupled.
The integration with third party systems (if it's the case), need some attention.
Do not call these services directly from your handlers, this could lead to various performance and/or maintainability issues.
You have to keep these integrations very well separated, because them are out of your domain. They may be subject to inefficiencies, changes or a number of problems out of your control.
One solution that I could recommend is a "Middleware" service.
In your application context this can be constituted by another service (always REST for example) that will have the task of 'talk' (only him) with external systems, acting as a single point of integration between your domain and the external environment. This can be realized from scratch or using a commercial/opens-source solution like (just as example) this.
This lead to many benefits, same of these:
A middleware is a unique mockable point during integration test of your application.
You can change the middleware implementation in the future without touch your handlers.
Of course, changing 3pty providers won't affect your domain services.
Middleware is the unique point dedicated to manage 3pty service interruptions.
Your services remain agnostic compared to the outside world.
Focus on these questions can be useful to design your integration middleware service:
Which types of 3pty data do they provide? Are they on time? This might help you figure out whether to introduce a cache system into your integration service.
Can 3pty be subject to frequent interruptions? Then you must ensure that your system must tolerate any disruption of external services. In other words, you must ensure a certain resilience of your services. There are many techniques to do that.
Do you really need to interrogate these 3pty services all the time? Maybe a more or less sophisticated cache system could speed up your services a lot.
Finally, it is also very important to understand if the need to have a microservices-oriented system is a real and immediate need.
Due to the fact these architectures are more expensive and complex then the classic ones, it might be reasonable to think about starting by building a monolith system and then moving towards a more segmented solution later.
Thinking (organizing) your system as many "bounded context" does not prevent you from creating a good monolith system and at the same time, it prepares you for a possible switch to microservices-oriented one.
As a summary advice, start by keeping things as separate as possible. Define a language to speak about your business model. These lead to you potentially change a lot when needs will come without to much effort during the inevitable evolution of your software. "Hexagonal" architecture is a good starting point to do that for both choises (Microservices vs Monolith).
Recently, Netflix posted a nice article about this architecture with a lot of ideas for a fresh start.

I will give my answer from DDD and the clean architecture perspective. Ideally, you application should have following layers.
Api (ideally very thin layer of Controllers).The controller will create queries and commands and push them on a common channel. (refer MediatR)
Application This will be your orchestration layer. It will contain definitions of queries and command and their handlers. For queries, you will directly interact form your infrastructure layer. For commands, you will interact with domain and then save them through repositories in infrastructure.
Domain Depends upon your business logic and complexity, this layer will contain all your business models.
Infrastructure It will contain mostly two types of objects Providers and Repositories. Providers should be used with queries and will return DAO. Repositories should be used where ever domain is involved, ideally commands in CQRS. Repositories should always receive and return only domain objects.
So after setting the base context about different layers on clean architecture, the answer to your original question is --> I would create third party interactions in the provider layer. For example, you need to connect with a user microservice, I will create a UserProvider in the provider folder in the infrastructure layer and consume it through a interface.

Dependency Injection vs Layered Architecture

I've been reading a lot about dependency injection and the service locator (anti-?) pattern - a lot of it on StackOverflow (thanks guys :). I have a question about how this pattern works when it's within a n-layer architecture.
I've seen a lot of blog posts where they describe injecting a IDataAccess component into the business objects. E.g.
public class Address
{
IDataAccess _dataAccess;
public Address(IDataAccess dataAccess)
{
this._dataAccess = dataAccess;
}
}
However, I was under the impression that in an n-layer architecture, the UI layer should not need to have any knowledge of the data access layer... or even know that there /is/ a data access layer! If DI requires exposing the IDataAccess interface in the constructors of the BusinessObjects, this then exposes to the UI the fact that the Business Layer uses a data access layer under the hood - something the UI doesn't need to know or care about surely?
So, my fundamental question is: Does DI require that I expose all my lower layer interfaces to all upper layers and is this a good or a bad thing?
Thanks
Edit: To clarify (after a few comments), I know my business object should be ignorant of the which specific implementation of which IDataAccess it uses (hence the Dependency being injected in the constructor) but I thought that the layers above the BO should not know that the Business Object even requires a dependency on a DAL.

This is really a fairly complex topic, and there are many ways of doing an n-tier architecture. No one way is "the right way", and how you do it depends on your needs as much as it does your personal preferences.
Dependency Injection is about managing dependencies. If your object should be unaware of any dependency, then you would not write your objet in the way you mentioned. You would instead have some other service or method that would populate the data in an agnostic way. Data doesn't mean "Database" either. So IDataAccess could mean it comes from a database, or it comes from a network socket or it comes from a file on disk. The whole point here is that Address does not choose what dependencies it creates. This is done through configuration at the composition root.
Things need data, otherwise your app is probably useless. Making your Address object load itself, however, may not be the best way to go about things. A better approach may be with a factory class or service method.

I think the answer is rather simple. Your bottom layers (interface, bll, dal, entities) are just a bunch of libraries. It is up to the client to decide which libraries to be used and it will increase client's flexibility. Moreover they are libraries, so any application-related configurations (connection strings, data caching, etc) lies on the client. Those configuration itself, sometimes also need to be injected and included into Composition Root.
However, if you want to has an uniform logic and not client's flexibility, you can choose web/app services as an additional layer.
1st Layer Entities
2nd Layer Interface
3rd Layer BLL & DAL
4th Layer Web/App Services
5th Layer UI
This way, your composition root exists in one layer (4th). And add your UI just need to add service reference to 4th layer (or 1st if needed). However, this implies the same Mark Seeman's article again, layering is worth the mapping. I assume that you can change the app/web service to Composition Root.
Moreover, this (app/web service) design has pros/cons. Pros:
Your app is encapsulated
Your app is being bridged by app/web services. It is guranteed that your UI don't know the DataAccess, thus fulfill your requirements.
Your app is secured
Simply said, having UI need to access app service is a huge gain in security aspect.
Access Portability
Now your app can be accessed everywhere. It can be connected by 3rd party app (other web) without has relying on dlls.
Cons:
Overhead cost during service call
Authentication, network connection, etc, will cause overhead during webservice call. I'm inexperienced for the performance impact but it should be enough for high traffic app.
Inflexibility of client
Client now need to access BLL/Services by using services instead of normal objects.
More Service for Different Type of Client
Now you need to provide more service than needed. Such as WebRequestRetriever, MobileRequestRetriever instead of accessing to a mere IRequestRetriever and let the composition root wire up the rest.
Apologize if this answer boarden the topic (just realized after finished).

IMHO:
It depends on who does the injection !-
It seems you need to/expect to have an MVC or MVP architecture to be in place, where a controller or Presenter does the job of translating the UI calls to business objects ,back and forth -
Creating concrete implementations of IDataAccess, Sending it to Address class.
So that the UI is totally unaware of who is providing the data it needs, and it provides you the expected scalability.
Thanks
Tarriq

wcf decision: one service multiple contracts or many services

I am using .NET 4 to create a small client server application for a customer. Should I create one giant service that implements many contracts (IInvoice, IPurchase, ISalesOrder, etc) or should I create many services running one contract each on many ports? My questions specifically is interested in the pros/cons of either choice. Also, what is the common way of answering this question?
My true dilemma is that I have no experience making this decision, and I have little enough experience with wcf that I need help understanding the technical implications of such a decision.

Don't create one large service that implements n-number of service contracts. These types of services are easy to create, but will eventually become a maintenance headache and will not scale well. Plus, you'll get all sorts of code merging conflicts if there's a development group competing for check-ins/check-outs.
Don't create too many services either. Avoid the trap of making your services too fine-grained. Try to create services based on a functionality. The methods exposed by these services shouldn't be fine-grained either. You're better off having fewer methods that do more. Avoid creating similar functions like GetUserByID(int ID), GetUserByName(string Name) by creating a GetUser(userObject user). You'll have less code, easier maintenance and better discoverability.
Finally, you're probably only going to need one port no matter what you do.
UPDATE 12/2018
Funny how things have changed since I wrote this. Now with the micro-services pattern, I'm creating a lot of services with chatty APIs :)

You would typically create different services for each main entity like IInvoice, IPurchase, ISalesOrder.
Another option is to seperate queries from commands. You could have a command service for each main entity and implement business operations accepting only the data they need in order to perform the operation (avoid CRUD-like operations); and one query service that returns the data in the format required by the client. This means that the command part uses the underlying domain model/business layer; while the query service directly operates on the database (bypassing the business, which is not needed for querying). This simplifies your querying a lot and makes it more flexible (return only what the client needs).

In real time applications you have one service contract for each entity like Invoice, Purchase and SalesOrder will have separate ServiceContract
However for each service contract there will be heterogeneous clients like Invoice will be called by backoffice through windows application using netNamedPipeBinding or netTcpBinding and same time client application needs to call the service using basicHttpBinding or wsHttpBindings. Basically you need to create multiple endpoints for each service.

Its seems that you are mixing between DataContract(s) and ServiceContract(s).
You can have one ServiceContract and many DataContract(s) and that would perfectly suit your needs.

The truth is that splitting up WCF services - or any services is a balancing act. The principle is that you want to to keep downward pressure on complexity while still considering performance.
The more services you create, the more configuration you will have to write. Also, you will increase the number of proxy classes you need to create and maintain on the client side.
Putting too many ServiceContracts on one service will increase the time it takes to generate and use a proxy. But, if you only end up with one or two Operations on a contract, you will have added complexity to the system with very little to gain. This is not a scientific prescription, but a good rule of thumb could be say about 10-20 OperationContracts per ServiceContract.
Class coupling is of course a consideration, but are you really dealing with separate concerns? It depends on what your system does, but most systems deal with only a few areas of concern, so splitting things up may not actually decrease class coupling that much anyway.
Another thing to remember, and this is ultra important is to always make your methods as generic as possible. WCF deals in DataContracts for a reason. DataContracts mean that you can send any object to and from the server so long as the DataContracts are known.
So, for example, you might have 3 OperationContracts:
[OperationContract]
Person GetPerson(string id);
[OperationContract]
Dog GetDog(string id);
[OperationContract]
Cat GetCat(string id);
But, so long as these are all known types, you could merge these in to one operation like:
[OperationContract]
IDatabaseRecord GetDatabaseRecord(string recordTypeName, string id);
Ultimately, this is the most important thing to consider when designing service contracts. This applies for REST if you are using a DataContract serialization like serialization method.
Lastly, go back over your ServiceContracts every few months and delete operations that are not getting used by the clients. This is another big one!

You should take the decision based the load expected, extensibility needed and future perspective. As you wrote " small client server application for a customer" it is not giving clear idea of intended use of the development in hand. Mr. Big's answer must be considered too.
You are most welcome to put forward further question backed with specific data or particulars about the situation in hand. Thanks.

SOA Architecture understanding

I have been asked to work on a project, based on SOA, using WCF. I have dabbled with WCF (Creating and consuming), but never with SOA. Am I right in saying that a single service would have the usual service layer, business layer and data access layer (if one's needed). The service layer would then expose methods.
Can Service A reference Service B, and service B reference service A?
And then a UI can access these services, via references - and that's essentially SOA? I am battling to find up to date, recent tutorials (Youtube), and the 'guides' I see online seem extremely complicated.

This Wikipedia entry is pretty clear I think?
Lets try a simple example. Say we have Library application that lets you check books in and out.
If you look at the "traditional" non-SOA way to approach n-tier systems then you have a service called MyService that has methods called something like CheckOutBook. This would go away and internally have a Book class and a Person class and would perform say Book.IsAvailable = False and Person.NumberOfBooks.
That is fine, but say you now have another application that wants to work with People. You can't just use the above service because the logic is tightly coupled with what you are doing, i.e. Library transactions. Instead you would have to copy / paste code into a new service "BookShop".
With SOA you would have a Book service and a Person service. The Person service would have an action such as Person.AssociateWithBook that both Library and BookShop could use without having to alter as it is simple enough to do the minimum. It is then down to the application to call the right service(s) to do the job required. This means that it is reusable without needing to modify the various services.
This is very simplistic but hopefully shows the architectural differences and get you going?

I'd skip question about SOA, since each one can call SOA whatever he understand SOA (Service Oriented Architecture) is. I mean, each architecture, using services, can be called SOA...
From technical aspect, I'd build it in next way:
IMO, Services by themselves should have as less logic as they can (like facade pattern), all there logic should be moved down to Business logic.
Service A using ServiceA.BusinessLogic, calls service B (proxy for service B is available for ServiceA.BL).
Same for Service B, calling service A.
This will give you bi-directional communication, without issues of Duplex (broken callbacks, ...).
UI should access the services as well - using UI.BusinessLogic ( I usually prefer think about service communication as sort of Communication Data Access Layer).

Linq To SQL, WebServices, Websites - Planning it all

Several "parts" (a WinForms app for exmaple) of my project use a DAL that I coded based on L2SQL.
I'd like to throw in several WebApps into the mix, but the issue is that the DAL "offers" much more data than the WebApps need. Way more.
Would it be OK if I wrapped the data that the websites need within a web-service, and instead of the website connecting directly to the DAL it would go through the web-service which in turn would access the DAL?
I feel like that would add a lot of overhead, but on the other hand, I definitely don't like the feeling of knowing that the WebApps have the "capabilities" of accessing much more data than they actually need.
Any input would be greatly appreciated.
Thank you very much for the help.

You can either create web services, or add a repository layer that presents only the data that your applications require. A repository has the additional benefit of being a decoupling layer, making it easier to unit test your application (by providing a mock repository).
If you plan on eventually creating different frontends (say, a web UI and a WPF or Silverlight UI), then web services make a lot of sense, since they provide a common data foundation to build on, and can be accessed across tiers.

If your data access layer were pulling all data as IQueryable, then you would be able to query your DAL and drill down your db calls with more precision.
See the very brief blog entry I wrote on Repository and Service layers using Linq to SQL. My article is built around MVC but the concept of Repository and Service layers would work just fine with WebForms, WinForms, Web Services, etc.
Again, the key here is to have your Repository or your Dal return an object AsQueryable whereby you wait until the last possible moment to actually commit to requesting data.
Your structure would look something like this
Domain Layer
Repository Layer (IQueryable)
Service layer for Web App
Website
Service layer for Desktop App
Desktop App
Service layer for Web Services
Web Service
Inside your Service layer is where you customize the specific calls based on the application your developing for. This allows for greater security and configuration on a per-app basis while maintaining a complete repository that doesn't need to be modified until you swap out your ORM (if you ever decide you need to swap out your ORM)

There is nothing inherently wrong with having more than you need in this case. The entire .NET 4 Client Profile contains over 50MB of assemblies, classes, etc. I might use 5% of it in my entire career. That doesn't mean I don't appreciate having all of it available in case I need it.
If you plan to provide the DAL to developers that should not have access to portions of the data, write a wrapper or derive a new DAL. I would avoid the services route unless you're confident you can accommodate for the overhead.

Sounds like you are on the right track. If many applications are going to use the this data you gain a few advantages by having services with DTOs.
If the domain model changes, just the mapping to the DTO needs to change. You can isolate the consuming application from these changes.
Less data over the wire
You can isolate you applications from the implementation of the DAL.
You can expose different services (maybe different DTOs) for different applications if it is necessary to restrict what parts of the object model should be exposed.