We Keep Coding

Repeated deserializing and serializing when calling API - is there a better way?

I am developing a website along with an API to serve it data and have noticed that my current process involves repeated serialization and deserialization of the same data when an API call is made. I was wondering if there is a better way to go about things. Both the website and API are written in C# using ASP.Net Core 2.0.
My current process is as follows:
1) End user loads website page
2) AJAX call is made from client side JavaScript, calling a C# function in the website.
3) The C# website function calls the API (using a Swagger generated client).
4) The API serializes data and returns it to website as JSON.
5) The Swagger client in the web site deserializes the data back to a POCO class.
6) The Website C# function serializes the data back to JSON to return it to AJAX function.
7) Something is done with the JSON data in the client - inevitably after first parsing the JSON.
Example Code:
AJAX call:
var xhr = new XMLHttpRequest();
xhr.open('get', "/GetData", true);
xhr.onload = function () {
var data = JSON.parse(xhr.responseText);
doSomething(data);
}.bind(this);
xhr.send();
Website method:
[Route("/GetData")]
public async Task<IActionResult> GetData()
{
var data = await ApiClient.ApiDataGetAsync();
return Json(data);
}
API Method:
[HttpGet]
public Dictionary<int, string> GetData()
{
return _logic.GetData();
}
As the above shows the data is serialized to JSON by the API before being deserialized to a POCO by the Swagger client, then serialized back to JSON in the website to be processed client side.
I realize I could simplify this by calling the API directly from the AJAX rather than going through the website but I would rather avoid this for a number of reasons:
1) It would tightly couple the API to the front end of the website, at the moment I have the option of implementing an interface layer in the site to simplify things if the API needs to be replaced in the future.
2) It would reduce my control over who can access the API - at the moment it is locked down by IP address.
Is there a way I can improve this process so the data doesn't have to be serialized twice whilst retaining the Swagger client?

What problem are you actually trying to solve? The time taken to serialize/deserialize the json is tiny compared to the time taken for networked i/o.
At this point I would say you are trying to optimise something without knowing whether it will improve application performance or reduce cost, which is generally considered a waste of time.

Well, you can forgo the client and simply make your action a proxy using HttpClient to fetch from the API.
The client is intended to actually give you C# objects to work with, so it serializes/deserializes the request/response to make it easier for you to work with the API. That's sort of the whole point. What you're saying is that you don't actually want this, so simply drop the client.
Instead, you can make the request directly to your API from the action, using HttpClient, and then simply return the the response from the HttpClient call. Since the API returns JSON and you need JSON, no further processing needs to be done.

Write live stream to response in WCF

I couldn't find an exact example of what I'm looking for.
I have a REST/WCF endpoint that takes in a Stream and returns a Stream. I am able to do "live" streaming of the request, by that I mean that the incoming stream gets read by the service as it is written to by the client (using a shared buffer stream in the service).
I would like to achieve the same thing on the outgoing stream but all the examples I found write all the data to a local stream then return it back to the client.
How can I access the underlying stream of the response and start writing to it, allowing the client to start reading from it without exiting my service method?
Is it even possible to achieve what I need with WCF and HTTP streaming?

Sending large data from WCF Server to Delphi Client

I need to create a WCF Service that will have a download file function. This WCF will be consumed by a Delphi application.
The problem: The files that will be downloaded are very large and may cause memory problems on Delphi side. So far, I have this code:
[OperationContract]
byte[] DownloadFile(string filePath);
But this code will cause the client app to hold all data in memory which can be an issue.
I have read that WCF is capable of streaming data as you can read at: How to: Enable Streaming
But I have a question regarding this piece of code cut from MSDN:
[OperationContract]
Stream GetStream(string data);
On the client side I want to pass a TFileStream to the function. By using TFileStream every byte read will go directly to the disk. But the function RETURNS a stream and what I want will not be possible since the stream will not a parameter to the function.
How can I download a file from a WCF service directly to the disk?

I have found that relying on "built-in" streaming capability in WCF when working with other (non-.NET) clients is a big source for strange problems...
Basically we solve this kind of scenario by defining:
[OperationContract]
string DownloadFile(string filePath);
The method generates a HTTP(S) url and returns it...
This way any http-capable client can work with the data in a robust fashion...
BEWARE that this makes the server a bit more complicated since you now need to have some mechanism to generate (and serve HTTP GET on) URLs (security, "globally" unique, only usable for a limited time etc.).
BUT the big advantage is that any client out there (mobile or some strange embedded device or whatever you might encounter) will be able to implement this scheme as long as it has http-support available (Delphi has some very good http-client options).

First of all, I'm not sure whether you can consume a streaming WCF service at all in Delphi 2010. If you can, then it works as follows:
The WCF service must be a streamed service, which means that you need to set the transferMode of the binding to Streamed or StreamedResponse. If you want to pass in a string as parameter, it must be StreamedResponse, otherwise, the parameter must be a stream as well.
Having a streamed service also means that there can be no method that does not return a stream or void. It is, for example, not possible to have the following two methods in the same service when it is a streamed service.
Stream GetStream(string s);
int GetInteger(string s);
Also it is not possible to have:
Stream GetStream(string s);
in a service which is configured to be Streamed, as the parameter would have to be a stream, too.
It is not possible to call the method with a stream which will be "filled", even if you make the method take a Stream parameter - not the real instance of Stream is passed back and forth at that point, but the content is actually copied back and forth.
In Delphi you'd get a stream as a result of the method call. You can then copy the contents of that stream into a TFileStream as you'd do if the source was another stream in Delphi. Code for that can be googled. Basically Adriano has posted something that should work. Basically: Read from the source stream, write to the destination stream until everything was read and written, or you could try something like that:
stream1 := wcfServiceClient.GetTheStream();
try
stream2:= TFileStream.Create('to.txt', fmCreate);
try
stream2.CopyFrom(stream1, stream1.Size);
finally
stream2.Free;
end;
finally
stream1.Free;
end;
Again: This works only under the assumption that you can access a WCF streamed service from Delphi as you'd access it from C# or VB.NET.

SoapHttpClientProtocol: get response as a stream instead of a string?

I'm using a webservice which spits out very large amounts of data in one piece. The response string can be something like 8MB. While not an issue on a desktop PC, an embedded device goes nuts dealing with an 8MB string object.
I wonder if there is a way to get the response as a stream? Currently I'm using the method like below. I tried using a POST request instead, but SOAP is just more convenient (the response is XML and with the POST I have to convert the plain text reply back to valid XML) and I'd like to stick with it. Is it possible to use a different kind of "Invoke" which won't return strings but streams? Any ideas?
[System.Web.Services.Protocols.SoapDocumentMethodAttribute("MyAPI/MyMethod", RequestNamespace="MyAPI", ResponseNamespace="MyAPI", ParameterStyle=System.Web.Services.Protocols.SoapParameterStyle.Wrapped, Use=System.Web.Services.Description.SoapBindingUse.Literal)]
public string MyMethod(string sID)
{
object[] results = this.Invoke("MyMethod", new object[] { sID });
return ((string)(results[0]));
}

If you use the old ASMX web service client infrastructure, then you're stuck with its limitations. One limitation is that there's no simple way to get the response except as deserialized data.
If it were necessary, then you could use a partial class to override the GetWebResponse method to return your own custom WebResponse. This latter would in turn override the GetResponseStream method to call the base version, consume the stream, then to return a stream containing an "empty" web request (otherwise .NET will choke on a stream with no contents).
You might also try something similar by overriding the GetReaderForMessage method. This is passed a SoapClientMessage instance which has a Stream property that you might be able to use. Again, you'll have to set the stream to something that the web service infrastructure can consume.
The better way to do this is with a WCF client. WCF has much more powerful and easy to use extensibility mechanisms.
In fact, you might not even need to extend a WCF client. You might simply be able to configure it to not have this buffering problem at all.

Any web service call is going to return SOAP, isn't it? I don't think a stream could be serialized into a soap packet to be returned from your service. And even if it could, wouldn't the serialized stream be at least as big as the string itself?

I believe the answer is no, there is no concept of a stream for SOAP.
Probably the simplest answer is to have your method:
Parse your response into segments your mobile device can handle
Cache your response in a application variable as a dictionary of these segments
return an arraylist of GUIDs.
You can then have your client request each of these segments separately via their GUIDs, then reassemble the original response when and handle it all the web services return.

ASMX can't do much about this. WCF's BasicHttpBinding can return a Stream to the caller.
http://msdn.microsoft.com/en-us/library/ms733742.aspx

Does WCF automatically URL encode/decode streams?

I'm programming a service for a program that uses HTTP post/get requests, so I handle all incoming requests with a hook method that takes a System.IO.Stream and returns a System.IO.Stream.
When I parse the incoming request (contained in an HTML form) by converting to a string and then using System.Web.HttpUtility.ParseQueryString(string), it seems to automatically URL-decode the data. When I return a file path (a Windows UNC, not going to explain why I do that), I initially URL-encoded the string before converting to a stream and returning it using a return-statement, the client seems to get a doubly-coded string.
So, just to be sure, does WCF automatically URL encode/decode streams for me as part of using System.ServiceModel.WebHttpBinding?

Apparently, it does:
"For RESTful services, WCF provides a binding named System.ServiceModel.WebHttpBinding.
This binding includes pieces that know how to read and write information using the HTTP and HTTPS transports, as well as encode messages suitable for use with HTTP."
from here.