We Keep Coding

ASP.NET Web-api core: Handling client connections problems and find conflicts

Background
I have a web api server (asp.net core v2.1) that serve some basic operation, like managing entities on the server. This is the interface:
[HttpPost]
[Route("create")]
public async Task<ActionResult<NewEntityResponse>> Create(CreateEntityModel model)
{
// 1) Validate the request.
// 2) Create a new row on the database
// 3) Return the new entity in response.
}
The user running this REST method in this way:
POST https://example.com/create
Content-Type: application/json
{
"firstName": "Michael",
"lastName": "Jorden"
}
And getting response like this:
Status 200
{
"id": "123456" // The newly created entity id
}
The Problem
When sending thousands of requests like this, at some point it will fail because of network connections. When connection fails, it can leads us into two different situations:
The network call was ended on the way to the server - In this case, the server don't know about this request. Therefore, the entity wasn't created. The user just have to send the same message again.
The network call was sent from the server to back to the client but never rich the destination - In this case the request was fulfill completely, but the client don't aware for this. The expected solution is to send the same request again. In this case, it will create the same entity twice - and this is the problem.
The Requested Solution
I want to create an generic solution for web-api that "remmeber" which commands it already done. if he got same request twice, it's return HTTP status code Conflict.
Where I got so far
I thought to add the client an option to add a unique id to the request, in this way:
POST https://example.com/create?call-id=XXX
Add to my server a new filter that check if the key XXX is already fulfill. If yes, return Conflict. Otherwise - continue.
Add another server filter that checks the response of the method and marking it as "completed" for further checks.
The problem with this solution on concurrency calls. If my method takes 5 seconds to be returned and the client sent the same message again after 1 second - it will create two entities with same data.
The Questions:
Do you think that this is good approach to solve this problem?
Do you familiar with ready to use solutions that doing this?
How to solve my "concurrency" problem?
Any other tips will be great!
thanks.

Isnt the easiest solution to make the REST action idempotent?
I mean by that: the call should check if the resource already exists and either create a new resource if it doesnt or return the existing if it does?

OK, I just figure it up how to make it right. So, I implemented it by myself and share it with you.
In order to sync all requests between different servers, I used Redis as cache service. If you have only one server, you can use Dictionary<string, string> instead.
This filter do:
Before processing the request - add a new empty value key to Redis.
After the server processed the request - store the server response in Redis. This data will be used when the user will ask again for same request.
public class ConflictsFilter : ActionFilterAttribute
{
const string CONFLICT_KEY_NAME = "conflict-checker";
static readonly TimeSpan EXPIRE_AFTER = TimeSpan.FromMinutes(30);
private static bool ShouldCheck(ActionDescriptor actionDescription, IQueryCollection queries)
{
return queries.ContainsKey(CONFLICT_KEY_NAME);
}
private string BuildKey(string uid, string requestId)
{
return $"{uid}_{requestId}";
}
public override void OnActionExecuting(ActionExecutingContext context)
{
if (ShouldCheck(context.ActionDescriptor, context.HttpContext.Request.Query))
{
using (var client = RedisConnectionPool.ConnectionPool.GetClient())
{
string key = BuildKey(context.HttpContext.User.GetId(), context.HttpContext.Request.Query[CONFLICT_KEY_NAME]);
string existing = client.Get<string>(key);
if (existing != null)
{
var conflict = new ContentResult();
conflict.Content = existing;
conflict.ContentType = "application/json";
conflict.StatusCode = 409;
context.Result = conflict;
return;
}
else
{
client.Set(key, string.Empty, EXPIRE_AFTER);
}
}
}
base.OnActionExecuting(context);
}
public override void OnResultExecuted(ResultExecutedContext context)
{
base.OnResultExecuted(context);
if (ShouldCheck(context.ActionDescriptor, context.HttpContext.Request.Query) && context.HttpContext.Response.StatusCode == 200)
{
string key = BuildKey(context.HttpContext.User.GetId(), context.HttpContext.Request.Query[CONFLICT_KEY_NAME]);
using (var client = RedisConnectionPool.ConnectionPool.GetClient())
{
var responseBody = string.Empty;
if (context.Result is ObjectResult)
{
ObjectResult result = context.Result as ObjectResult;
responseBody = JsonConvert.SerializeObject(result.Value);
}
if (responseBody != string.Empty)
client.Set(key, responseBody, EXPIRE_AFTER);
}
}
}
}
The code is executed only if the query ?conflict-checker=XXX is exists.
This code is provide you under MIT license.
Enjoy the ride :)

C# WCF Service Get Status Code

I have a very basic WCF service which has a method named SaveSchoolName(string SchoolName) which basically returns as boolean value of True if the operation is good.
.
I added a service reference to my client application and is consuming the service as follows:
MyService.WebServicesClient svc = new MyService.WebServicesClient();
bool dataSaved = false;
dataSaved = svc.SaveSchoolName("School Name");
if(dataSaved){
// do something.
}
else{
// log not saved.
}
I want to know how do I determine the Http Status Code (200 - OK) for the WCF Service call. I have tried to search but none seems to provide any detailed info on how I would be able to get the response headers from invoking the method.

You need to create a client message inspector for this.
Check the below code out ... to make it work just add the inspector to your client. BTW, obviously this only works for HTTP :)
public class HttpStatusCodeMessageInspector : IClientMessageInspector
{
public void AfterReceiveReply(ref Message reply, object correlationState)
{
if (reply.Properties.ContainsKey(HttpResponseMessageProperty.Name))
{
var httpResponseProperty = (HttpResponseMessageProperty)reply.Properties[HttpResponseMessageProperty.Name];
Console.WriteLine($"Response status is {(int)httpResponseProperty.StatusCode}");
}
}
public object BeforeSendRequest(ref Message request, IClientChannel channel)
{
return null;
}
}

To extend #Jorge's answer, I started by implementing the IClientMessageInspector and IEndpointBehavior from the example here. I replaced the contents of IClientMessageInspector.AfterReceiveReply with the code provided by Jorge:
public void AfterReceiveReply(ref Message reply, object correlationState)
{
if (reply.Properties.ContainsKey(HttpResponseMessageProperty.Name))
{
var httpResponseProperty = (HttpResponseMessageProperty)reply.Properties[HttpResponseMessageProperty.Name];
Console.WriteLine($"Response status is {(int)httpResponseProperty.StatusCode}");
}
}
Now, you can choose to continue inheriting BehaviorExtensionElement and add the behavior to your client configuration file. If you take this approach you'll be done.
However, since my client configuration was created programmatically, I needed a few additional steps here.
In the docs you can see that an Endpoint Behavior can be added by using the ServiceEndpoint.Behaviors.Add method. But how do we get access to the ServiceEndpoint object?
This can be done by first creating your service client object, and then using its Endpoint property like so:
''' VB.NET
'' Creating the binding obj
Dim objMyBinding As New System.ServiceModel.WSHttpBinding()
objMyBinding.Name = "YOUR_BINDING_NAME"
'' Other details of binding added here ''
'' Creating the endpoint address obj
Dim objMyEndpoint As System.ServiceModel.EndpointAddress
objMyEndpoint = New System.ServiceModel.EndpointAddress(New Uri(ENDPOINT_ADDR_HERE))
'' Creating the Service obj
Dim objWebServices As CalculatorService = New CalculatorService(objMyBinding, objMyEndpoint)
'' Finally adding the behavior to the Service Endpoint (CustomEndpointBehavior is implementation of IEndpointBehavior)
objWebServices.Endpoint.EndpointBehaviors.Add(New CustomEndpointBehavior)
/// C#.NET
// Creating the binding obj
System.ServiceModel.WSHttpBinding objMyBinding = New System.ServiceModel.WSHttpBinding();
objMyBinding.Name = "YOUR_BINDING_NAME";
// Other details of binding added here //
// Creating the endpoint address obj
System.ServiceModel.EndpointAddress objMyEndpoint;
objMyEndpoint = New System.ServiceModel.EndpointAddress(New Uri(ENDPOINT_ADDR_HERE));
// Creating the Service obj
CalculatorService objWebServices = New CalculatorService(objMyBinding, objMyEndpoint);
// Finally adding the behavior to the Service Endpoint (CustomEndpointBehavior is implementation of IEndpointBehavior)
objWebServices.Endpoint.EndpointBehaviors.Add(New CustomEndpointBehavior());

How to return 302 redirect from Web service environment

I am in a restful service environment and we are using ServiceStack as our service Framework. At this moment, I need to do a redirect directly from the service and as soon as I try to do it from my Get method it fails because I think my Get function looks somethinglike this:
public override object OnGet(ShareDTO request)
{
HttpContext.Current.Response.Redirect(#"http://www.google.com");
return new object();
}
May be because first it is trying to redirect and then it is trying to return a value but this is just wild guess. I am not sure if it is even possible in service environment because all the time whenever I have used Response.Redirect, it was always always a .NET aspx page and never tried in service environment.
Any thoughts?

The easiest way is to just return a custom HttpResult, e.g:
public object Any(Request request)
{
return HttpResult.Redirect("http://www.google.com");
}
Which is equivalent to the more explicit:
return new HttpResult
{
StatusCode = HttpStatusCode.Found,
Headers =
{
{ HttpHeaders.Location, "http://www.google.com" },
}
};
Alternatively you can write directly to the Response, either in a Request Filter or within a Service, e.g:
public object Any(Request request)
{
base.Response.StatusCode = 302;
base.Response.AddHeader(HttpHeaders.Location, "");
base.Response.EndRequest();
return null;
}

ServiceStack RESTful WebService and passing data in message body

I am evaluating ServiceStack at the moment. I am in need to create bunch of RESTful webservices. I have the initial code running, and I am quite happy with that. What I was struggling a bit was how to create a service that could consume POST (or PUT) HTTP request that has data in its body.
I've found this thread on ServiceStack forum (http://groups.google.com/group/servicestack/browse_thread/thread/693145f0c3033795) and folliwng it I've been guided to have a look at the following thread on SO (Json Format data from console application to service stack) but it was not really helpful - it described how to create a request, and not how to create a service that can consume such a HTTP request.
When I tried to pass additional data (in the HTTP message body) my servuce returned following error (HTTP 400):
<TaskResponse xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="">
<ResponseStatus>
<ErrorCode>SerializationException</ErrorCode>
<Message>Could not deserialize 'application/xml' request using ServiceStackMVC.Task'
Error: System.Runtime.Serialization.SerializationException: Error in line 1 position 8.Expecting element 'Task' from namespace 'http://schemas.datacontract.org/2004/07/ServiceStackMVC'..
Encountered 'Element' with name 'Input', namespace ''.
at System.Runtime.Serialization.DataContractSerializer.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
at System.Runtime.Serialization.XmlObjectSerializer.ReadObject(XmlDictionaryReader reader)
at System.Runtime.Serialization.XmlObjectSerializer.ReadObject(Stream stream)
at ServiceStack.Text.XmlSerializer.DeserializeFromStream(Type type, Stream stream) in C:\src\ServiceStack.Text\src\ServiceStack.Text\XmlSerializer.cs:line 76
at ServiceStack.WebHost.Endpoints.Support.EndpointHandlerBase.CreateContentTypeRequest(IHttpRequest httpReq, Type requestType, String contentType) in C:\src\ServiceStack\src\ServiceStack\WebHost.Endpoints\Support\EndpointHandlerBase.cs:line 107</Message>
<StackTrace> at ServiceStack.WebHost.Endpoints.Support.EndpointHandlerBase.CreateContentTypeRequest(IHttpRequest httpReq, Type requestType, String contentType) in C:\src\ServiceStack\src\ServiceStack\WebHost.Endpoints\Support\EndpointHandlerBase.cs:line 115
at ServiceStack.WebHost.Endpoints.RestHandler.GetRequest(IHttpRequest httpReq, IRestPath restPath) in C:\src\ServiceStack\src\ServiceStack\WebHost.Endpoints\RestHandler.cs:line 98
at ServiceStack.WebHost.Endpoints.RestHandler.ProcessRequest(IHttpRequest httpReq, IHttpResponse httpRes, String operationName) in C:\src\ServiceStack\src\ServiceStack\WebHost.Endpoints\RestHandler.cs:line 60</StackTrace>
</ResponseStatus>
</TaskResponse>
This led me to https://github.com/ServiceStack/ServiceStack/wiki/Serialization-deserialization I thought that I will give IRequiresRequestStream a go. At the moment my code is as follows:
public class Task : IRequiresRequestStream
{
public string TaskName { get; set; }
public string bodyData { get; set; }
public override bool Equals(object obj)
{
Task task = obj as Task;
if (task == null)
return false;
return TaskName.Equals(task.TaskName);
}
public override int GetHashCode()
{
return TaskName.GetHashCode();
}
public System.IO.Stream RequestStream
{
get
{
return new MemoryStream(System.Text.Encoding.UTF8.GetBytes(bodyData));
}
set
{
if (value.Length == 0)
{
bodyData = string.Empty;
}
else
{
byte[] buffer = new byte[value.Length];
int bytesRead = value.Read(buffer, 0, (int)value.Length);
bodyData = System.Text.Encoding.UTF8.GetString(buffer);
}
}
}
}
and service itself:
public class TaskService : RestServiceBase<Task>
{
public List<Task> tasks { get; set; }
public override object OnGet(Task request)
{
if (string.IsNullOrEmpty(request.TaskName))
{
if (tasks == null || tasks.Count == 0)
return "<tasks/>";
StringBuilder sb = new StringBuilder();
sb.AppendLine("<tasks>");
foreach (Task t in tasks)
{
sb.AppendFormat(" <task id={0}><![CDATA[{2}]]><task/>{1}", t.TaskName, System.Environment.NewLine, t.bodyData);
}
sb.AppendLine("</tasks>");
return sb.ToString();
}
else
{
if (tasks.Contains(request))
{
var task = tasks.Where(t => t.TaskName == request.TaskName).SingleOrDefault();
return String.Format("<task id={0}><![CDATA[{2}]]><task/>{1}", task.TaskName, System.Environment.NewLine, task.bodyData);
}
else
return "<task/>";
}
}
public override object OnPost(Task request)
{
if (tasks.Contains( request ))
{
throw new HttpError(System.Net.HttpStatusCode.NotModified, "additional information");
}
tasks.Add(new Task() { TaskName = request.TaskName, bodyData = request.bodyData });
return null;
}
}
My routes:
Routes.Add<Task>("/tasks/{TaskName}").Add<Task>("/tasks");
It works but... as I couldn't find any similar example I would like to ask if this is the correct way of creating a service that is capable of processing POST requests that have additional informaion included in their message body. Am I doing anything wrong? Is there anything that I've missed?
It was also mentioned on the SO thread link to which I have provided, that using DTO is the preferred way to pass data to ServiceStack based service. Assuming that client needs to send a lot of data, how could we achieve that? I don't want to pass data as JSON object in the URI. Am I making any false assumption here?
Client is not going to be written in C# / .Net. Completely different technology is going to be used. This was one of the reasony why RESTful webservices.
I know returning xml as string may not be the best idea. At the moment it is just a sample code. This will be changed later on.
The most important part is if the solution provided for me is the proper way to create a webservice that can consume HTTP request that has xml data attached in its body. What I've shared with you works I am just not 100% sure that this is the best way to achieve my goal.
Edited on Thursday 8th of March, 2012:
After reading the answer and the comments I've changed my code a little bit. I was pretty sure that if I wanted to use serialization I had to use namespaces (when passing data in the HTTP message body to the service).
I've used http://localhost:53967/api/servicestack.task/xml/metadata?op=Task to get more information about the service I've created.
REST Paths:
All Verbs /tasks/{TaskName}
All Verbs /tasks
HTTP + XML:
POST /xml/asynconeway/Task HTTP/1.1
Host: localhost
Content-Type: application/xml
Content-Length: length
<Task xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/ServiceStackMVC">
<AuxData>String</AuxData>
<TaskName>String</TaskName>
</Task>
What I wanted to check was if it was possible to "mix" REST URI and pass the rest of the data as an xml.
Using Fiddler, I've created following POST request:
POST http://localhost:53967/api/tasks/22
Request headers:
User-Agent: Fiddler
Host: localhost:53967
Content-Type: application/xml
Content-Length: 165
Request Body:
<Task xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/ServiceStackMVC">
<AuxData>something</AuxData>
</Task>
My DTO right now is as follows:
public class Task
{
public string TaskName { get; set; }
public string AuxData { get; set; }
public override bool Equals(object obj)
{
Task task = obj as Task;
if (task == null)
return false;
return TaskName.Equals(task.TaskName);
}
public override int GetHashCode()
{
return TaskName.GetHashCode();
}
}
And my service code is:
public class TaskService : RestServiceBase<Task>
{
public List<Task> tasks { get; set; }
public override object OnGet(Task request)
{
return tasks;
}
public override object OnPost(Task request)
{
if (tasks.Contains( request ))
{
throw new HttpError(System.Net.HttpStatusCode.NotModified, "additional information");
}
tasks.Add(new Task() { TaskName = request.TaskName });
return null;
}
}
So is this the proper way of passing XML data to the service? I think I am quite happy with xml namespaces included - that makes it even easier to develop services.

Nope, returning an xml string it's not the recommended approach since any string returned gets written directly to the response stream so the service will only work with XML services and not all the other endpoints.
The ServiceStack Way
Is to keep your DTOs that you define your web services with in their own largely dependency-free assembly (I generally will only have a reference the impl and dep-free ServiceStack.Interfaces.dll). You can then re-use these DTOs with ServiceStack's generic Service Clients to get a succinct, typed, end-to-end API without any code-gen.
Different built-in Service Clients
Your C#/.NET clients only need to use the Service Clients contained in the
ServiceStack.Common NuGet package which just contains the ServiceStack.Text.dll, ServiceStack.Interfaces.dll and ServiceStack.Common.dll for full .NET and Silverlight 4/5 client builds.
ServiceStack.Common contains the following Service Clients:
JsonServiceClient - The lightweight, ubiquitous, self-describing resilient format
JsvServiceClient - Faster more compact than JSON ideal for .NET to .NET services
XmlServiceClient - For folks who like using XML (Slower than JSON/JSV)
Soap11ServiceClient / Soap12ServiceClient - If your company mandates the use of SOAP.
If you Install the ProtoBuf Format plugin you also have the option to use the ProtoBufServiceClient which is the fastest binary serializer for .NET.
Easy to swap, easy to test
The C# Service Clients share the same IServiceClient and IRestClient interfaces making it easy to swap out if you want to take advantage of a superior format. Here's an example taking advantage of this where the same Unit Test is also used as a JSON, XML, JSV and SOAP integration test.
By default out-of-the-box, ServiceStack makes all your services available via pre-defined routes in the following convention:
/api/[xml|json|html|jsv|csv]/[syncreply|asynconeway]/[servicename]
This is what the Service Clients use when you use the Send<TResponse> and SendAsync<TResponse> API methods which allows you to call your web services without having to define any custom routes, e.g:
var client = new JsonServiceClient();
var response = client.Send<TaskResponse>(new Task());
If you want you can use the Get, Post, Put, Delete API methods that allows you to specify a url so you can call web services using your custom routes, e.g:
Async API example
FilesResponse response;
client.GetAsync<FilesResponse>("files/", r => response = r, FailOnAsyncError);
Sync API example
var response = client.Get<FilesResponse>("files/README.txt");
Here are some Sync and Async API examples from the RestFiles example project.
XML and SOAP issues
Generally XML and SOAP are more strict and brittle compared to the other formats, to minimize interop issues and reduce payload bloat you should set a global XML Namespace for all your DTOs by adding an Assembly attribute in your DTO Assembly.cs file, e.g:
[assembly: ContractNamespace("http://schemas.servicestack.net/types",
ClrNamespace = "MyServiceModel.DtoTypes")]
If you want to use a different ContractNamespace than the above you will also need to also set it in the EndpointHostConfig.WsdlServiceNamespace if you wish to make use of the SOAP endpoints.
Here are some more versioning tips when developing SOAP/XML web services:
https://groups.google.com/d/msg/servicestack/04GQLsQ6YB4/ywonWgD2WeAJ
SOAP vs REST
Since SOAP routes all requests through the HTTP POST verb, if you wish to make each service available via SOAP as well you will need to create a new class per service and define custom REST-ful routes to each service as described here.
Due to the brittleness, bloated payload size and slower perf of SOAP/XML, it is recommended to use either the JSON, JSV or ProtoBuf formats/endpoints.
Request Model Binders
Another alternative to using IRequiresRequestStream is to use Request Model Binders you can define in your AppHost, e.g:
base.RequestBinders[typeof(Task)] = httpReq => ... requestDto;
C# Client Recommendation
The recommendation is to use ServiceStack's built-in service clients for C# clients although if you wish to use your own HttpClient, than still using the XmlServiceClient will come in handy as you can use Fiddler to see the exact wire-format ServiceStack expects.

Operation Handler WCF Web Api malfunctionging?

I'm building a web api using WCF web api preview 6, currently I'm stuck with a little problem. I would like to have an operation handler to inject an IPrincipal to the operation in order to determine which user is making the request. I already have that Operation Handler and is already configured. But I noticed that when I decorate the operation with the WebInvoke attribute and simultaneously the operation receives an IPrincipal and other domain object, the system throws an exception telling me:
The HttpOperationHandlerFactory is unable to determine the input parameter that should be associated with the request message content for service operation 'NameOfTheOperation'. If the operation does not expect content in the request message use the HTTP GET method with the operation. Otherwise, ensure that one input parameter either has it's IsContentParameter property set to 'True' or is a type that is assignable to one of the following: HttpContent, ObjectContent1, HttpRequestMessage or HttpRequestMessage1.
I do not know what is happening here. To give you some background I'll post some of my code to let you know how am I doing things.
The operation:
[WebInvoke(UriTemplate = "", Method = "POST")]
[Authorization(Roles = "")]
public HttpResponseMessage<dto.Diagnostic> RegisterDiagnostic(dto.Diagnostic diagnostic, IPrincipal principal)
{
......
}
WCF web api knows when to inject the IPrincipal because I decorate the operation with a custom Authorization attribute.
The configuration in the Global file:
var config = new WebApiConfiguration() {EnableTestClient = true};
config.RegisterOAuthHanlder(); //this is an extension method
routes.SetDefaultHttpConfiguration(config);
routes.MapServiceRoute<MeasurementResource>("Measurement");
routes.MapServiceRoute<DiagnosticResource>("Diagnostic");
Then the RegisterOAuthHandler method adds an operation handler to the operation if it's been decorated with the custom authorization attibute. this is how it looks:
public static WebApiConfiguration RegisterOAuthHanlder(this WebApiConfiguration conf)
{
conf.AddRequestHandlers((coll, ep, desc) =>
{
var authorizeAttribute = desc.Attributes.OfType<AuthorizationAttribute>().FirstOrDefault();
if (authorizeAttribute != null)
{
coll.Add(new OAuthOperationHandler(authorizeAttribute));
}
});
return conf;
}
public static WebApiConfiguration AddRequestHandlers(
this WebApiConfiguration conf,
Action<Collection<HttpOperationHandler>, ServiceEndpoint, HttpOperationDescription> requestHandlerDelegate)
{
var old = conf.RequestHandlers;
conf.RequestHandlers = old == null ? requestHandlerDelegate : (coll, ep, desc) =>
{
old(coll, ep, desc);
};
return conf;
}
Can somebody help me with this? Thank you in advanced!

Try wrapping your Diagnostic param in ObjectContent i.e. ObjectContent<Diagnostic>. Then you will use the ReadAs() method to pull out the object.
It should work.