We Keep Coding

How can I return XML from BeforeSendRequest and AfterReceiveReply to the calling method in a thread-safe way?

We have a console application using the Azure WebJob SDK. The WebJob relies on a WCF service using SOAP, which it accesses through a DLL we wrote that wraps the auto-generated WCF types in something a bit more friendly.
For logging purposes, we want to save the request and response XML bodies for requests that we make. These XML bodies would be saved in our database. But, because the WCF code lives in a low-level DLL, it has no concept of our database and can't save to it.
The DLL uses Microsoft's DI extensions to register types, and the WebJob calls into it like this:
class WebJobClass
{
IWCFWrapperClient _wcfWrapperClient;
public WebJobClass(IWCFWrapperClient wcfWrapperClient)
{
_wcfWrapperClient = wcfWrapperClient;
}
public async Task DoThing()
{
var callResult = await _wcfWrapperClient.CallWCFService();
}
}
IWCFWrapperClient looks like this:
class WCFWrapperClient : IWCFWrapperClient
{
IWCF _wcf; // auto-generated by VS, stored in Reference.cs
public async Task<object> CallWCFService()
{
return await _wcf.Call(); // another auto-generated method
}
}
I've implemented an IClientMessageInspector, and it works fine to get me the XML request/response, but I don't have a way to pass it back up to WCFWrapperClient.CallWCFService so that it can be returned to WebJobClass.DoThing(), who could then save it to the database.
The problem is multithreading. WebJobs, IIRC, will run multiple requests in parallel, calling into the DLL from multiple threads. This means we can't, say, share a static property LastRequestXmlBody since multiple threads could overwrite it. We also can't, say, give each call a Guid or something since there's no way to pass anything from IWCFWrapperClient.CallWCFService into the auto-generated IWCF.Call except what was auto-generated.
So, how can I return XML to WebJobClass.DoThing in a thread-safe way?

I was able to find a solution that uses ConcurrentDictionary<TKey, TValue>, but it's a bit ugly.
First, I amended the auto-generated classes in Reference.cs with a new property Guid InternalCorrelationId. Since the auto-generated classes are partial, this can be done in separate files that aren't changed when the client is regenerated.
public partial class AutoGeneratedWCFType
{
private Guid InternalCorrelationIdField;
[System.Runtime.Serialization.DataMember()]
public Guid InternalCorrelationId
{
get { return InternalCorrelationIdField; }
set { InternalCorrelationIdField = value; }
}
}
Next, I made all my request DTO types derive from a type named RequestBase, and all my response DTO types derive from a typed named ResponseBase, so I could handle them generically:
public abstract class RequestBase
{
public Guid InternalCorrelationId { get; set; }
}
public abstract class ResponseBase
{
public string RequestXml { get; set; }
public string ResponseXml { get; set; }
}
I then added a type RequestCorrelator that simply holds on to a ConcurrentDictionary<Guid, XmlRequestResponse>:
public sealed class RequestCorrelator : IRequestCorrelator
{
public ConcurrentDictionary<Guid, XmlRequestResponse> PendingCalls { get; }
public RequestCorrelator() => PendingCalls = new ConcurrentDictionary<Guid, XmlRequestResponse>();
}
public sealed class XmlRequestResponse
{
public string RequestXml { get; set; }
public string ResponseXml { get; set; }
}
RequestCorrelator is its own type for DI purposes - you may just be able to use a ConcurrentDictionary<TKey, TValue> directly.
Finally, we have the code that actually grabs the XML, a type implementing IClientMessageInspector:
public sealed class ClientMessageProvider : IClientMessageInspector
{
private readonly IRequestCorrelator _requestCorrelator;
public ClientMessageProvider(IRequestCorrelator requestCorrelator) =>
_requestCorrelator = requestCorrelator;
public object BeforeSendRequest(ref Message request, IClientChannel channel)
{
var requestXml = request.ToString();
var internalCorrelationId = GetInternalCorrelationId(requestXml);
if (internalCorrelationId != null)
{
if (_requestCorrelator.PendingCalls.TryGetValue(internalCorrelationId.Value,
out var requestResponse))
{
requestResponse.RequestXml = requestXml;
}
request = RemoveInternalCorrelationId(request);
}
return internalCorrelationId;
}
public void AfterReceiveReply(ref Message reply, object correlationState)
{
// WCF can internally correlate a request between BeforeSendRequest and
// AfterReceiveReply. We reuse the same correlation ID we added to the
// XML as our correlation state.
var responseXml = reply.ToString();
var internalCorrelationId = (correlationState is Guid guid)
? guid
: default;
if (_requestCorrelator.PendingCalls.TryGetValue(internalCorrelationId,
out var requestResponse))
{
requestResponse.ResponseXml = responseXml;
}
}
private static Guid? GetInternalCorrelationId(string requestXml)
{
var document = XDocument.Parse(requestXml);
var internalCorrelationIdElement = /* You'll have to write this yourself;
every WCF XML request is different. */
return internalCorrelationIdElement != null
? Guid.Parse(internalCorrelationIdElement.Value)
: null;
}
private static Message RemoveInternalCorrelationId(Message oldMessage)
{
// https://stackoverflow.com/a/35639900/2709212
var buffer = oldMessage.CreateBufferedCopy(2 * 1024 * 1024);
var tempMessage = buffer.CreateMessage();
var dictionaryReader = tempMessage.GetReaderAtBodyContents();
var document = new XmlDocument();
document.Load(dictionaryReader);
dictionaryReader.Close();
var internalCorrelationIdNode = /* You'll also have to write this yourself. */
var parent = internalCorrelationIdNode.ParentNode;
parent.RemoveChild(internalCorrelationIdNode);
var memoryStream = new MemoryStream();
var xmlWriter = XmlWriter.Create(memoryStream);
document.Save(xmlWriter);
xmlWriter.Flush();
xmlWriter.Close();
memoryStream.Position = 0;
var xmlReader = XmlReader.Create(memoryStream);
var newMessage = Message.CreateMessage(oldMessage.Version, null, xmlReader);
newMessage.Headers.CopyHeadersFrom(oldMessage);
newMessage.Properties.CopyProperties(oldMessage.Properties);
return newMessage;
}
}
In short, this type:
Finds the correlation ID in the XML request.
Finds the XmlRequestResponse with the same correlation ID and adds the request to it.
Removes the correlation ID element so that the service doesn't get elements they didn't expect.
After receiving a reply, uses correlationState to find the XmlRequestResponse and write the response XML to it.
Now all we have to do is change IWCFWrapperClient:
private async Task<TDtoResult> ExecuteCallWithLogging<TDtoRequest,
TWcfRequest,
TWcfResponse,
TDtoResult>(TDtoRequest request,
Func<TDtoRequest, TWcfRequest> dtoToWcfConverter,
Func<TWcfRequest, Task<TWcfResponse>> wcfCall,
Func<TWcfResponse, TDtoResult> wcfToDtoConverter)
where TDtoRequest : CorrelationBase
where TDtoResult : WcfBase
{
request.InternalCorrelationId = Guid.NewGuid();
var xmlRequestResponse = new XmlRequestResponse();
_requestCorrelator.PendingCalls.GetOrAdd(request.InternalCorrelationId,
xmlRequestResponse);
var response = await contractingCall(dtoToWcfConverter(request));
_requestCorrelator.PendingCalls.TryRemove(request.InternalCorrelationId, out _);
return wcfToDtoConverter(response).WithRequestResponse(xmlRequestResponse);
}
public async Task<DoThingResponseDto> DoThing(DoThingRequestDto request) =>
await ExecuteCallWithLogging(request,
r => r.ToWcfModel(),
async d => await _wcf.Call(d),
d => d.ToDtoModel());
WithRequestResponse is implemented as follows:
public static T WithRequestResponse<T>(this T item, XmlRequestResponse requestResponse)
where T : ResponseBase
{
item.RequestXml = requestResponse?.RequestXml;
item.ResponseXml = requestResponse?.ResponseXml;
return item;
}
And there we go. WCF calls that return their XML in the response object rather than just something you can print to console or log to a file.

Implementing a generic API caller

I am trying to implement a generic caller that uses OpenWeatherMap's different weather API's, but I got stuck in regards to how I would put in the right identifier for the link.
.../weather?q=... returns JSON data for the current weather;
.../forecast?q=... returns JSON data for a five day forecast.
I am looking for the textbook way to maybe retrieve the API type of each class through accessing GetAPIType(), cast that to an int and put it in the index, so that I would be able to use identifiers[index]. Or perhaps there is an easier way to do it.
Checking for the typeof(T) also crossed my mind, and I would assign the index depending on the if(typeof(T).Equals(typeof(...))) construct, but that seems very messy and if OpenWeatherMap had 100 API's in theory, I would need 100 different if constructs. With this in mind, wouldn't creating those checks beat the purpose of Client being generic?
A third solution I thought of would be passing APIType type as a parameter for the Client constructor,
e.g. var client = new Client<CurrentWeatherDTO>(APIType.CurrentWeather, location, apiKey),
but given the fact that Client is generic and I already provide a type when I instantiate it, it would seem awfully redundant.
Client.cs
using System.IO;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Newtonsoft.Json;
using System.Reflection;
namespace Rainy.OpenWeatherMapAPI
{
public class Client<T>
{
private readonly string location;
private readonly string apiKey;
private readonly string requestUri;
private readonly string[] identifiers = { "weather", "forecast" };
private readonly int index;
public Client(string location, string apiKey)
{
// Get the type of API used in order to get the right identifier for the link.
// ??? Maybe use Reflection, somehow.
this.location = location;
this.apiKey = apiKey;
requestUri = $"api.openweathermap.org/data/2.5/{}?q={location}&appid={apiKey}";
}
public async Task<T> GetWeather(CancellationToken cancellationToken)
{
using (var client = new HttpClient())
using (var request = new HttpRequestMessage(HttpMethod.Get, requestUri))
using (var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken))
{
var stream = await response.Content.ReadAsStreamAsync();
if (response.IsSuccessStatusCode)
return DeserializeJsonFromStream<T>(stream);
var content = await StreamToStringAsync(stream);
throw new APIException
{
StatusCode = (int)response.StatusCode,
Content = content
};
}
}
private U DeserializeJsonFromStream<U>(Stream stream)
{
if (stream == null || stream.CanRead == false)
return default(U);
using (var sr = new StreamReader(stream))
using (var jtr = new JsonTextReader(sr))
{
var js = new JsonSerializer();
var searchResult = js.Deserialize<U>(jtr);
return searchResult;
}
}
private async Task<string> StreamToStringAsync(Stream stream)
{
string content = null;
if (stream != null)
using (var sr = new StreamReader(stream))
content = await sr.ReadToEndAsync();
return content;
}
}
}
APIType.cs
namespace Rainy.OpenWeatherMapAPI
{
public enum APIType
{
CurrentWeather = 0,
FiveDayForecast = 1
}
}
IWeather.cs
namespace Rainy.OpenWeatherMapAPI
{
public interface IWeather
{
APIType GetAPIType();
}
}
CurrentWeatherDTO.cs
namespace Rainy.OpenWeatherMapAPI.CurrentWeatherData
{
class CurrentWeatherDTO : IWeather
{
public APIType GetAPIType()
{
return APIType.CurrentWeather;
}
}
}
FiveDayForecastDTO.cs
namespace Rainy.OpenWeatherMapAPI.WeatherForecastData
{
class FiveDayForecastDTO : IWeather
{
public APIType GetAPIType()
{
return APIType.FiveDayForecast;
}
}
}

I would not use an enum to drive the index of an array.
I would directly return the string in a static way.
This solution can also work with the index of the array if you want.
Here is the code and the dotnetfiddle:
using System;
public class Program
{
public static void Main()
{
var client1 = new Client<CurrentWeatherDTO>(null);
Console.WriteLine("Client CurrentWeather type: " + client1.Type);
var client2 = new Client<FiveDayForecastDTO>(null);
Console.WriteLine("Client FiveDay type: " + client2.Type);
}
public class Client<T> where T : IWeather, new()
{
public string Type { get; set; }
public Client(string apiKey)
{
var dto = (IWeather)new T();
this.Type = dto.GetAPIType();
}
}
public static class APIType
{
public static string CurrentWeather = "weather";
public static string FiveDayForecast = "forecast";
}
public interface IWeather
{
string GetAPIType();
}
class CurrentWeatherDTO : IWeather
{
public string GetAPIType()
{
return APIType.CurrentWeather;
}
}
class FiveDayForecastDTO : IWeather
{
public string GetAPIType()
{
return APIType.FiveDayForecast;
}
}
}

I would probably use a solution like this, but maybe a bit more error handling.
There's a couple of references for how to use HttpClient.
I don't really understand the part in the requestUri with {}, maybe that's part of your problem, I changed it to {???} in my sample code.
class Client
{
// Problems using HttpClient and look into using IHttpClientFactory...
// http://byterot.blogspot.com/2016/07/singleton-httpclient-dns.html
// https://www.hanselman.com/blog/HttpClientFactoryForTypedHttpClientInstancesInASPNETCore21.aspx
static HttpClient _httpClient = new HttpClient();
readonly string WeatherUri = $"api.openweathermap.org/data/2.5/{???}?q={0}&appid={1}";
public async Task<T> GetWeather<T>(string location, CancellationToken cancellationToken)
{
var apiKey = ApiKeyAttribute.GetApiKey<T>();
if (apiKey == null) throw new Exception("ApiKeyAttirbute missing");
var requestUri = string.Format(WeatherUri, location, apiKey);
return await GetItem<T>(requestUri, cancellationToken);
}
public async Task<T> GetItem<T>(string requestUri, CancellationToken cancellationToken)
{
var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, requestUri);
var response = await _httpClient.SendAsync(httpRequestMessage, cancellationToken);
if (!response.IsSuccessStatusCode) throw new Exception("Error requesting data");
if (response.Content == null) return default(T);
var content = await response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<T>(content);
}
}
[ApiKeyAttribute("weather")]
class CurrentWeatherDTO { /* add appropriat properties */ }
[ApiKeyAttribute("forecast")]
class FiveDayForecastDTO { /* add appropriat properties */ }
[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = true)]
class ApiKeyAttribute : Attribute
{
public string Name { get; private set; }
public ApiKeyAttribute(string name)
{
Name = name;
}
public static string GetApiKey<T>()
{
var attribute = typeof(T).GetCustomAttribute<ApiKeyAttribute>();
return attribute?.Name;
}
}

inject - InSessionScope Extension problems with AppFabric Cache

Ninject doesn’t provide a InSessionScope Binding for Websites, so we have created our own extension:
public static IBindingNamedWithOrOnSyntax<T> InSessionScope<T>(this IBindingInSyntax<T> parent)
{
return parent.InScope(SessionScopeCallback);
}
private const string _sessionKey = "Ninject Session Scope Sync Root";
private static object SessionScopeCallback(IContext context)
{
if (HttpContext.Current.Session[_sessionKey] == null)
{
HttpContext.Current.Session[_sessionKey] = new object();
}
return HttpContext.Current.Session[_sessionKey];
}
This extension is working fine until we are using the standard local SessionStore.
But we changed the SessionStore and we now use the „AppFabricCacheSessionStoreProvider“ and this store is no longer on the local machine its on the server.
And the problem is that Ninject tries to resolve the reference of an object which was serialized and deserialized and comes from the server and not from the local memory and so ninject can’t find the reference. The result is, that ninjects allways creates a new Object and the SessionScope does not work any more.
Edit 1:
We are using this functionality
https://msdn.microsoft.com/en-us/library/hh361711%28v=azure.10%29.aspx
and here I can use the standard "HttpContext.Current.Session" Object and the list content is stored on the server and not on the local machine.

So architecturally you have a problem in that you need to store the settings for AppFabric somewhere, and this is an issue with your static method. But assume you create a public static class like so:
public static class AppCache
{
public static DataCache Cache { get; private set; }
static AppCache()
{
List<DataCacheServerEndpoint> servers = new List<DataCacheServerEndpoint>(1);
servers.Add(new DataCacheServerEndpoint("ServerName", 22233)); //22233 is the default port
DataCacheFactoryConfiguration configuration = new DataCacheFactoryConfiguration
{
Servers = servers,
LocalCacheProperties = new DataCacheLocalCacheProperties(),
SecurityProperties = new DataCacheSecurity(),
RequestTimeout = new TimeSpan(0, 0, 300),
MaxConnectionsToServer = 10,
ChannelOpenTimeout = new TimeSpan(0, 0, 300),
TransportProperties = new DataCacheTransportProperties() { MaxBufferSize = int.MaxValue, MaxBufferPoolSize = long.MaxValue }
};
DataCacheClientLogManager.ChangeLogLevel(System.Diagnostics.TraceLevel.Off);
var _factory = new DataCacheFactory(configuration);
Cache = _factory.GetCache("MyCache");
}
}
then you can change extension like so:
public static IBindingNamedWithOrOnSyntax<T> InSessionScope<T>(this IBindingInSyntax<T> parent)
{
return parent.InScope(SessionScopeCallback);
}
private const string _sessionKey = "Ninject Session Scope Sync Root";
private static object SessionScopeCallback(IContext context)
{
var cachedItem = AppCache.Cache.Get("MyItem"); // IMPORTANT: For concurrency reason, get the whole item down to method scope.
if (cachedItem == null)
{
cachedItem = new object();
AppCache.Cache.Put("MyItem", cachedItem);
}
return cachedItem;
}

I've found a "Solution" that works so far it's not perfect because I am avoiding the AppFabric Store with an Localstore for the Object Reference.
public static IBindingNamedWithOrOnSyntax<T> InSessionScope<T>(this IBindingInSyntax<T> parent)
{
return parent.InScope(SessionScopeCallback);
}
public static Dictionary<string, object> LocalSessionStore = new Dictionary<string, object>();
private const string _sessionKey = "Ninject Session Scope Sync Root";
private static object SessionScopeCallback(IContext context)
{
var obj = new object();
var key = (string)HttpContext.Current.Session[_sessionKey];
if (string.IsNullOrEmpty(key))
{
var guid = Guid.NewGuid().ToString();
HttpContext.Current.Session[_sessionKey] = guid;
LocalSessionStore.Add(guid, obj);
}
else if(!LocalSessionStore.ContainsKey(key))
{
LocalSessionStore.Add(key, obj);
return LocalSessionStore[key];
}
else if (LocalSessionStore.ContainsKey(key))
{
return LocalSessionStore[key];
}
return HttpContext.Current.Session[_sessionKey];
}
}

C# error(using interface methods): An object reference is required for the non-static field, method, or property

I'm having trouble using a third party API that has outdated documentation, so I'm trying to figure out why this piece of ##$! isn't working. And by ##$! i mean "code", of course :)
So as far as i know WAPISoap is a public interface that I have obtained by adding a web reference in visual studio.
I also know the Describe() method accepts two parameters, a string and an object of type credential and it returns a string. Any help would be greatly appreciated :)
Here's what i got so far:
using WAPIClient;
using System;
using Project1.WsWWDAPI;
namespace WAPIClient
{
class ResellerAPI
{
public void CallDescribe()
{
String sReturnXml;
Credential m_Crededential = new Project1.WsWWDAPI.Credential();
m_Crededential.Account = "account";
m_Crededential.Password = "password";
String sCLTRID = System.Guid.NewGuid().ToString();
sReturnXml = WAPISoap.Describe(sCLTRID, m_Crededential);
Console.WriteLine(sReturnXml);
}
static void Main(string[] args)
{
ResellerAPI reseller = new ResellerAPI();
reseller.CallDescribe();
}
}
}

The Describe method is not static, which means you need to call it on an instance of the WAPI class:
WsWWDAPI.WAPI m_WAPIObj = null;
WsWWDAPI.Credential m_Crededential = null;
public void Init()
{
m_WAPIObj = new WsWWDAPI.WAPI();
m_Crededential = new WsWWDAPI.Credential();
m_Crededential.Account = "account";
m_Crededential.Password = "password";
}
public void CallDescribe()
{
String sReturnXml;
String sCLTRID = System.Guid.NewGuid().ToString();
sReturnXml = m_WAPIObj.Describe(sCLTRID, m_Crededential);
Console.WriteLine( sReturnXml );
}
static void Main(string[] args)
{
ResellerAPI reseller = new ResellerAPI();
reseller.Init();
reseller.CallDescribe();
}
See: http://products.secureserver.net/guides/wsapiquickstart.pdf

The error is because you use non-static method in static context - you should have instance of the WAPISoap in order to call member function which is not static

It sounds like you need to create an instance of WAPISoap and then call Describe on that instance.

Getting a parameterless method to act like a Func<ReturnT>

I'm trying to make a part of my code more fluent.
I have a string extension that makes an HTTP request out of the string and returns the response as a string. So I can do something like...
string _html = "http://www.stackoverflow.com".Request();
I'm trying to write an extension that will keep trying the request until it succeeds. My signature looks something like...
public static T KeepTrying<T>(this Func<T> KeepTryingThis) {
// Code to ignore exceptions and keep trying goes here
// Returns the result of KeepTryingThis if it succeeds
}
I intend to call it something like...
string _html = "http://www.stackoverflow.com".Request.KeepTrying();
Alas, that doesn't seem to work =). I tried making it into a lambda first but that doesn't seem to work either.
string _html = (() => "http://www.stackoverflow.com".Request()).KeepTrying();
Is there a way to do what I'm trying to do while keeping the syntax fairly fluent?
Suggestions much appreciated.
Thanks.

You can't use a method group for extension methods, or lambda expressions. I blogged about this a while ago.
I suspect you could cast to Func<string>:
string _html = ((Func<string>)"http://www.stackoverflow.com".Request)
.KeepTrying();
but that's pretty nasty.
One alternative would be to change Request() to return a Func, and use:
string _html = "http://www.stackoverflow.com".Request().KeepTrying();
Or if you wanted to keep the Request method itself simple, just add a RequestFunc method:
public static Func<string> RequestFunc(this string url)
{
return () => url.Request();
}
and then call:
string _html = "http://www.stackoverflow.com".RequestFunc().KeepTrying();

Why not turn this on its head?
static T KeepTrying<T>(Func<T> func) {
T val = default(T);
while (true) {
try {
val = func();
break;
} catch { }
}
return val;
}
var html = KeepTrying(() => "http://www.stackoverflow.com".Request());

What about enhancing the Request?
string _html = "http://www.stackoverflow.com".Request(RequestOptions.KeepTrying);
string _html = "http://www.stackoverflow.com".Request(RequestOptions.Once);
RequestOptions is a enum. You could also have more options, timeout arguments, number of retries etc.
OR
public static string RepeatingRequest(this string url) {
string response = null;
while ( response != null /* how ever */ ) {
response = url.Request();
}
return response;
}
string _html = "http://www.stackoverflow.com".RepeatingRequest();

AFAIK you can write an extension method that extends a Func<T> delegate, but the compiler doesn't know what do you mean:
string _html = "http://www.stackoverflow.com".Request.KeepTrying(); // won't work
But if you explicitly cast the delegate will work:
string _html = ((Func<string>)"http://www.stackoverflow.com".Request).KeepTrying(); // works
The question here it whether the code readability is really improved in this case by an extension method.

I wouldn't write an extension method for string. Use a more specific type, like the Uri.
The full code:
public static class Extensions
{
public static UriRequest Request(this Uri uri)
{
return new UriRequest(uri);
}
public static UriRequest KeepTrying(this UriRequest uriRequest)
{
uriRequest.KeepTrying = true;
return uriRequest;
}
}
public class UriRequest
{
public Uri Uri { get; set; }
public bool KeepTrying { get; set; }
public UriRequest(Uri uri)
{
this.Uri = uri;
}
public string ToHtml()
{
var client = new System.Net.WebClient();
do
{
try
{
using (var reader = new StreamReader(client.OpenRead(this.Uri)))
{
return reader.ReadToEnd();
}
}
catch (WebException ex)
{
// log ex
}
}
while (KeepTrying);
return null;
}
public static implicit operator string(UriRequest uriRequest)
{
return uriRequest.ToHtml();
}
}
Calling it:
string html = new Uri("http://www.stackoverflow.com").Request().KeepTrying();