We Keep Coding

How can I return XML from BeforeSendRequest and AfterReceiveReply to the calling method in a thread-safe way?

We have a console application using the Azure WebJob SDK. The WebJob relies on a WCF service using SOAP, which it accesses through a DLL we wrote that wraps the auto-generated WCF types in something a bit more friendly.
For logging purposes, we want to save the request and response XML bodies for requests that we make. These XML bodies would be saved in our database. But, because the WCF code lives in a low-level DLL, it has no concept of our database and can't save to it.
The DLL uses Microsoft's DI extensions to register types, and the WebJob calls into it like this:
class WebJobClass
{
IWCFWrapperClient _wcfWrapperClient;
public WebJobClass(IWCFWrapperClient wcfWrapperClient)
{
_wcfWrapperClient = wcfWrapperClient;
}
public async Task DoThing()
{
var callResult = await _wcfWrapperClient.CallWCFService();
}
}
IWCFWrapperClient looks like this:
class WCFWrapperClient : IWCFWrapperClient
{
IWCF _wcf; // auto-generated by VS, stored in Reference.cs
public async Task<object> CallWCFService()
{
return await _wcf.Call(); // another auto-generated method
}
}
I've implemented an IClientMessageInspector, and it works fine to get me the XML request/response, but I don't have a way to pass it back up to WCFWrapperClient.CallWCFService so that it can be returned to WebJobClass.DoThing(), who could then save it to the database.
The problem is multithreading. WebJobs, IIRC, will run multiple requests in parallel, calling into the DLL from multiple threads. This means we can't, say, share a static property LastRequestXmlBody since multiple threads could overwrite it. We also can't, say, give each call a Guid or something since there's no way to pass anything from IWCFWrapperClient.CallWCFService into the auto-generated IWCF.Call except what was auto-generated.
So, how can I return XML to WebJobClass.DoThing in a thread-safe way?

I was able to find a solution that uses ConcurrentDictionary<TKey, TValue>, but it's a bit ugly.
First, I amended the auto-generated classes in Reference.cs with a new property Guid InternalCorrelationId. Since the auto-generated classes are partial, this can be done in separate files that aren't changed when the client is regenerated.
public partial class AutoGeneratedWCFType
{
private Guid InternalCorrelationIdField;
[System.Runtime.Serialization.DataMember()]
public Guid InternalCorrelationId
{
get { return InternalCorrelationIdField; }
set { InternalCorrelationIdField = value; }
}
}
Next, I made all my request DTO types derive from a type named RequestBase, and all my response DTO types derive from a typed named ResponseBase, so I could handle them generically:
public abstract class RequestBase
{
public Guid InternalCorrelationId { get; set; }
}
public abstract class ResponseBase
{
public string RequestXml { get; set; }
public string ResponseXml { get; set; }
}
I then added a type RequestCorrelator that simply holds on to a ConcurrentDictionary<Guid, XmlRequestResponse>:
public sealed class RequestCorrelator : IRequestCorrelator
{
public ConcurrentDictionary<Guid, XmlRequestResponse> PendingCalls { get; }
public RequestCorrelator() => PendingCalls = new ConcurrentDictionary<Guid, XmlRequestResponse>();
}
public sealed class XmlRequestResponse
{
public string RequestXml { get; set; }
public string ResponseXml { get; set; }
}
RequestCorrelator is its own type for DI purposes - you may just be able to use a ConcurrentDictionary<TKey, TValue> directly.
Finally, we have the code that actually grabs the XML, a type implementing IClientMessageInspector:
public sealed class ClientMessageProvider : IClientMessageInspector
{
private readonly IRequestCorrelator _requestCorrelator;
public ClientMessageProvider(IRequestCorrelator requestCorrelator) =>
_requestCorrelator = requestCorrelator;
public object BeforeSendRequest(ref Message request, IClientChannel channel)
{
var requestXml = request.ToString();
var internalCorrelationId = GetInternalCorrelationId(requestXml);
if (internalCorrelationId != null)
{
if (_requestCorrelator.PendingCalls.TryGetValue(internalCorrelationId.Value,
out var requestResponse))
{
requestResponse.RequestXml = requestXml;
}
request = RemoveInternalCorrelationId(request);
}
return internalCorrelationId;
}
public void AfterReceiveReply(ref Message reply, object correlationState)
{
// WCF can internally correlate a request between BeforeSendRequest and
// AfterReceiveReply. We reuse the same correlation ID we added to the
// XML as our correlation state.
var responseXml = reply.ToString();
var internalCorrelationId = (correlationState is Guid guid)
? guid
: default;
if (_requestCorrelator.PendingCalls.TryGetValue(internalCorrelationId,
out var requestResponse))
{
requestResponse.ResponseXml = responseXml;
}
}
private static Guid? GetInternalCorrelationId(string requestXml)
{
var document = XDocument.Parse(requestXml);
var internalCorrelationIdElement = /* You'll have to write this yourself;
every WCF XML request is different. */
return internalCorrelationIdElement != null
? Guid.Parse(internalCorrelationIdElement.Value)
: null;
}
private static Message RemoveInternalCorrelationId(Message oldMessage)
{
// https://stackoverflow.com/a/35639900/2709212
var buffer = oldMessage.CreateBufferedCopy(2 * 1024 * 1024);
var tempMessage = buffer.CreateMessage();
var dictionaryReader = tempMessage.GetReaderAtBodyContents();
var document = new XmlDocument();
document.Load(dictionaryReader);
dictionaryReader.Close();
var internalCorrelationIdNode = /* You'll also have to write this yourself. */
var parent = internalCorrelationIdNode.ParentNode;
parent.RemoveChild(internalCorrelationIdNode);
var memoryStream = new MemoryStream();
var xmlWriter = XmlWriter.Create(memoryStream);
document.Save(xmlWriter);
xmlWriter.Flush();
xmlWriter.Close();
memoryStream.Position = 0;
var xmlReader = XmlReader.Create(memoryStream);
var newMessage = Message.CreateMessage(oldMessage.Version, null, xmlReader);
newMessage.Headers.CopyHeadersFrom(oldMessage);
newMessage.Properties.CopyProperties(oldMessage.Properties);
return newMessage;
}
}
In short, this type:
Finds the correlation ID in the XML request.
Finds the XmlRequestResponse with the same correlation ID and adds the request to it.
Removes the correlation ID element so that the service doesn't get elements they didn't expect.
After receiving a reply, uses correlationState to find the XmlRequestResponse and write the response XML to it.
Now all we have to do is change IWCFWrapperClient:
private async Task<TDtoResult> ExecuteCallWithLogging<TDtoRequest,
TWcfRequest,
TWcfResponse,
TDtoResult>(TDtoRequest request,
Func<TDtoRequest, TWcfRequest> dtoToWcfConverter,
Func<TWcfRequest, Task<TWcfResponse>> wcfCall,
Func<TWcfResponse, TDtoResult> wcfToDtoConverter)
where TDtoRequest : CorrelationBase
where TDtoResult : WcfBase
{
request.InternalCorrelationId = Guid.NewGuid();
var xmlRequestResponse = new XmlRequestResponse();
_requestCorrelator.PendingCalls.GetOrAdd(request.InternalCorrelationId,
xmlRequestResponse);
var response = await contractingCall(dtoToWcfConverter(request));
_requestCorrelator.PendingCalls.TryRemove(request.InternalCorrelationId, out _);
return wcfToDtoConverter(response).WithRequestResponse(xmlRequestResponse);
}
public async Task<DoThingResponseDto> DoThing(DoThingRequestDto request) =>
await ExecuteCallWithLogging(request,
r => r.ToWcfModel(),
async d => await _wcf.Call(d),
d => d.ToDtoModel());
WithRequestResponse is implemented as follows:
public static T WithRequestResponse<T>(this T item, XmlRequestResponse requestResponse)
where T : ResponseBase
{
item.RequestXml = requestResponse?.RequestXml;
item.ResponseXml = requestResponse?.ResponseXml;
return item;
}
And there we go. WCF calls that return their XML in the response object rather than just something you can print to console or log to a file.

Obtain Assembly attributes from unloaded executable

I am attempting to retrieve the typical AssemblyInfo attributes from an executable file, but not from the currently executing assembly. I wish to 'look into' a program file (.exe) elsewhere on the drive that I have written in C#.NET and check the AssemblyProduct string.
This is fairly easy and straightforward when you're looking for this information from the currently executing assembly. However, apparently not so much when you attempt to pull it from an unloaded assembly.
When I use the following code, it returns "Microsoft® .NET Framework" instead of the Product name that I put in my AssemblyInfo.cs file.
Note: I use the System.Reflection.AssemblyName object to pull the version info e.g:AssemblyName.GetAssemblyName(pathToAssembly) and this works correctly, but I'm unable to pull my assembly's attributes using that class or by any means I've tried thus far. Is there some other special class, or what am I missing or doing incorrectly here?
public static string GetAppProdIDFromPath(string pathToForeignAssembly)
{
var atts = GetForeignAssemblyAttributes(pathToForeignAssembly);
var id = string.Empty;
foreach (var att in atts)
{
if (att.GetType() == typeof(AssemblyProductAttribute))
{
id = ((AssemblyProductAttribute)att).Product;
}
}
return id;
}
private static object[] GetForeignAssemblyAttributes(string pathToAssembly)
{
if(File.Exists(pathToAssembly))
{
try
{
var assm = System.Reflection.Assembly.LoadFrom(pathToAssembly);
return assm.GetType().Assembly.GetCustomAttributes(false);
}
catch(Exception ex)
{
// logger etc
}
}
else
{
throw...
}
return null;
}

As Duncanp mentioned, there is a bug in my code. Posting it for clarity and for anyone down the road who looks for the same solution:
public static string GetAppProdIDFromPath(string pathToForeignAssembly)
{
var atts = GetForeignAssemblyAttributes(pathToForeignAssembly);
var id = string.Empty;
foreach (var att in atts)
{
if (att.GetType() == typeof(AssemblyProductAttribute))
{
id = ((AssemblyProductAttribute)att).Product;
}
}
return id;
}
private static object[] GetForeignAssemblyAttributes(string pathToAssembly)
{
if(File.Exists(pathToAssembly))
{
try
{
var assm = System.Reflection.Assembly.LoadFrom(pathToAssembly);
return assm.GetCustomAttributes(false); // fixed line
}
catch(Exception ex)
{
// logger etc
}
}
else
{
throw...
}
return null;
}

Unit testing xUnit with JSON files data

I want to unit test my technical analysis indicators. I have historical candle data downloaded in JSON format.
What is the correct way to load the JSON data into xUnit and re-use it on multiple methods? All I want is to be able to use the historical data (List<OHLCV>) from all these methods in IndicatorTests.cs.
Is it important to move Data() into a fixture? Since my indicators will be tested in a same file IndicatorTests.cs?
How do I find the JSON files because they are inside the project and not in /bin/Debug. AppDomain.CurrentDomain.BaseDirectory is kinda looking for them in /bin/Debug, which is wrong.
public class IndicatorTests
{
public static IEnumerable<object[]> Data()
{
var filePath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, #"Data\TRX_USDT-5m.json");
if (!File.Exists(filePath))
throw new FileNotFoundException($"The historical data file '{filePath}' was not found.");
var data = File.ReadAllText(filePath);
var deserializedData = JsonConvert.DeserializeObject<List<List<decimal>>>(data);
var candles = deserializedData.Select(e => new OHLCV()
{
Timestamp = DateTimeOffset.FromUnixTimeMilliseconds((long)e[0]).DateTime,
Open = e[1],
High = e[2],
Low = e[3],
Close = e[4],
Volume = e[5]
});
foreach (var candle in candles ?? Enumerable.Empty<OHLCV>())
{
yield return new[] { candle };
}
}
[Theory]
[MemberData(nameof(Data))]
public void Test1(List<OHLCV> candle)
{
Console.WriteLine(candle);
}
[Theory]
[MemberData(nameof(Data))]
public void Test2(List<OHLCV> candle)
{
Console.WriteLine(candle);
}
[Theory]
[MemberData(nameof(Data))]
public void Test3(List<OHLCV> candle)
{
Console.WriteLine(candle);
}
}

Using mocking with .NET Reflection [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I am trying to attempt mocking on some reflection (code below). I have been advised to use NSubstitue but I am struggling on how to implement this and to get it started.
At the moment my test stubs are simply like the one below, however on the build server these obviously fail as the DLLs are not present.
[TestMethod]
public void CanGetStudentXml()
{
var student = new ReadStudent();
var results = student.GetStudentXml();
Assert.AreNotEqual(string.Empty, results);
}
Can anyone give me any pointers on how I should go about doing this? Do I need to create mock assemblies? If so, based on the one below, how would I achieve that?
Also is Nsubsitute the best for the job, or would moq be better suited? Which would be the best mocking framework to use?
Sample code:
namespace MokPoc
{
using System.Reflection;
using System;
using System.Linq;
class Program
{
static void Main(string[] args)
{
var students = new ReadStudent();
var results = students.GetStudentXml();
var contacts = students.GetTelephoneXml();
}
}
public enum ReflectedAssembyType
{
SimsProcessesTpPersonStudent,
SimsProcessesTpPersonContact
}
internal class ReflectedAssemblyFactory
{
public static ReflectedAssemblyBase GetReflectedAssembly(ReflectedAssembyType reflectedAssembyType)
{
ReflectedAssemblyBase value = null;
switch (reflectedAssembyType)
{
case ReflectedAssembyType.SimsProcessesTpPersonStudent:
value = new SimsProcessesTpPersonStudent("ThirdPartyProcesses.dll");
break;
case ReflectedAssembyType.SimsProcessesTpPersonContact:
value = new SimsProcessesTpPersonContact("PersonContacts.dll");
break;
}
return value;
}
}
internal abstract class ReflectedAssemblyBase
{
private string path = string.Empty;
private string type = string.Empty;
public string Path
{
get { return this.path; }
set { this.path = value; }
}
public string Type
{
get { return this.type; }
set { this.type = value; }
}
public object InvokeFunction(string name, object[] args)
{
var assemblyToLoad = Assembly.LoadFrom(this.path);
var typeToLoad = assemblyToLoad.GetType(this.type);
var methodToInvoke = typeToLoad.GetMethod(name, args.Select(o => o.GetType()).ToArray());
object obj = Activator.CreateInstance(typeToLoad);
return methodToInvoke.Invoke(obj, args);
}
}
internal sealed class SimsProcessesTpPersonStudent : ReflectedAssemblyBase
{
public SimsProcessesTpPersonStudent(string assembly)
{
this.Path = System.IO.Path.Combine(#"C:\Program Files\Zoosk", assembly);
this.Type = "SIMS.Processes.TPPersonStudent";
}
}
public class ReadStudent
{
public string GetStudentXml()
{
var contacts = ReflectedAssemblyFactory.GetReflectedAssembly(ReflectedAssembyType.SimsProcessesTpPersonStudent);
return (string)contacts.InvokeFunction("GetXmlStudents", new object[] { DateTime.Today });
}
public string GetTelephoneXml()
{
var contacts = ReflectedAssemblyFactory.GetReflectedAssembly(ReflectedAssembyType.SimsProcessesTpPersonContact);
return (string)contacts.InvokeFunction("GetXmlTelephone", new object[] { DateTime.Today });
}
}
}

I have refactored you code to understand what you are trying to test, it seems like you had a lot of classes to do something that seems could be the responsibility of one class, the heart of what you are trying to do is in GetStudentAttributes, I would create a test.dll with a class and public method that returns some strings and then run an actual method to test, in that case you are not using a stub or mock but it is a valid test to ensure your code works. You should also test GetTelephoneXml and GetStudentXML but the only thing you are really testing there is that GetStudentAttributes is inkoved with the appropriate parameters, so when GetStudentXML is called you invokeGetStudentAttributes with "ThirdpartyProcesses.dll" and "GetXmlStudents".
Depending on the framework you use the solution to testing will be different, with Rhynomocks you will have to make the methods virtual to allow the proxy to inherit and invoke your methods, but you can certainly test that the method was called and that the parameters are what you expect, I haven't used nSubstitute, so not sure how to do it there but if the framework is decent you should be able to test those calls and the parameters.
One of the first things that you should do when using driven development is to start by writing the tests first, making sure it fails, making it pass and refactor, usually when you try and retrofit tests to existing code it could get really hard, there are some good resources out there about unit testing, this is a great book about it http://www.amazon.com/Test-Driven-Development-By-Example/dp/0321146530, but in my experience when something is hard to test it usually tells you that your code is too complex or something can be improved, once the code is simplified or fixed testing is usually not a problem.
Good luck and hope this helped a bit!
using System;
using System.IO;
using System.Linq;
using System.Reflection;
namespace MokPoc
{
internal class Program
{
private static void Main(string[] args)
{
var students = new ReadStudentsService();
string results = students.GetStudentXml();
string contacts = students.GetTelephoneXml();
}
}
public class ReadStudentsService
{
private const string ProgramFilesZooskDirectory = #"C:\Program Files\Zoosk";
private const string SimsProcessesTppersonstudent = "SIMS.Processes.TPPersonStudent";
public string GetStudentXml()
{
return GetStudentAttributes("ThirdPartyProcesses.dll", "GetXmlStudents");
}
public string GetTelephoneXml()
{
return GetStudentAttributes("ThirdPartyContacts.dll", "GetXmlTelephone");
}
public string GetStudentAttributes(string dllToUse, string methodToExecute)
{
var fullpath = Path.Combine(ProgramFilesZooskDirectory, dllToUse);
var args = new object[] {DateTime.Today};
var assemblyToLoad = Assembly.LoadFrom(fullpath);
var typeToLoad = assemblyToLoad.GetType(SimsProcessesTppersonstudent);
var methodToInvoke = typeToLoad.GetMethod(methodToExecute, args.Select(o => o.GetType()).ToArray());
var obj = Activator.CreateInstance(typeToLoad);
return (string) methodToInvoke.Invoke(obj, args);
}
}
}

Remove Extraneous Semicolons in C# Using Roslyn - (replace w empty trivia)

I've figured out how to open a solution and then iterate through the Projects and then Documents. I'm stuck with how to look for C# Classes, Enums, Structs, and Interfaces that may have an extraneous semicolon at the end of the declaration (C++ style). I'd like to remove those and save the .cs files back to disk. There are approximately 25 solutions written at my current company that I would run this against. Note: The reason we are doing this is to move forward with a better set of coding standards. (And I'd like to learn how to use Roslyn to do these 'simple' adjustments)
Example (UPDATED):
class Program
{
static void Main(string[] args)
{
string solutionFile = #"S:\source\dotnet\SimpleApp\SimpleApp.sln";
IWorkspace workspace = Workspace.LoadSolution(solutionFile);
var proj = workspace.CurrentSolution.Projects.First();
var doc = proj.Documents.First();
var root = (CompilationUnitSyntax)doc.GetSyntaxRoot();
var classes = root.DescendantNodes().OfType<ClassDeclarationSyntax>();
foreach (var decl in classes)
{
ProcessClass(decl);
}
Console.ReadKey();
}
private static SyntaxNode ProcessClass(ClassDeclarationSyntax node)
{
ClassDeclarationSyntax newNode;
if (node.HasTrailingTrivia)
{
foreach (var t in node.GetTrailingTrivia())
{
var es = new SyntaxTrivia();
es.Kind = SyntaxKind.EmptyStatement;
// kind is readonly - what is the right way to create
// the right SyntaxTrivia?
if (t.Kind == SyntaxKind.EndOfLineTrivia)
{
node.ReplaceTrivia(t, es);
}
}
return // unsure how to do transform and return it
}
}
Example Code I Want to Transform
using System;
public class Person
{
public string FirstName { get; set; }
public string LastName { get; set; }
};
// note: the semicolon at the end of the Person class

Here is a little program that removes the optional semicolon after all class-, struct-, interface and enum-declarations within a solution. The program loops through documents within the solution, and uses a SyntaxWriter for rewriting the syntaxtree. If any changes were made, the original code-files are overwritten with the new syntax.
using System;
using System.IO;
using System.Linq;
using Roslyn.Compilers.CSharp;
using Roslyn.Services;
namespace TrailingSemicolon
{
class Program
{
static void Main(string[] args)
{
string solutionfile = #"c:\temp\mysolution.sln";
var workspace = Workspace.LoadSolution(solutionfile);
var solution = workspace.CurrentSolution;
var rewriter = new TrailingSemicolonRewriter();
foreach (var project in solution.Projects)
{
foreach (var document in project.Documents)
{
SyntaxTree tree = (SyntaxTree)document.GetSyntaxTree();
var newSource = rewriter.Visit(tree.GetRoot());
if (newSource != tree.GetRoot())
{
File.WriteAllText(tree.FilePath, newSource.GetText().ToString());
}
}
}
}
class TrailingSemicolonRewriter : SyntaxRewriter
{
public override SyntaxNode VisitClassDeclaration(ClassDeclarationSyntax node)
{
return RemoveSemicolon(node, node.SemicolonToken, t => node.WithSemicolonToken(t));
}
public override SyntaxNode VisitInterfaceDeclaration(InterfaceDeclarationSyntax node)
{
return RemoveSemicolon(node, node.SemicolonToken, t => node.WithSemicolonToken(t));
}
public override SyntaxNode VisitStructDeclaration(StructDeclarationSyntax node)
{
return RemoveSemicolon(node, node.SemicolonToken, t => node.WithSemicolonToken(t));
}
public override SyntaxNode VisitEnumDeclaration(EnumDeclarationSyntax node)
{
return RemoveSemicolon(node, node.SemicolonToken, t => node.WithSemicolonToken(t));
}
private SyntaxNode RemoveSemicolon(SyntaxNode node,
SyntaxToken semicolonToken,
Func<SyntaxToken, SyntaxNode> withSemicolonToken)
{
if (semicolonToken.Kind != SyntaxKind.None)
{
var leadingTrivia = semicolonToken.LeadingTrivia;
var trailingTrivia = semicolonToken.TrailingTrivia;
SyntaxToken newToken = Syntax.Token(
leadingTrivia,
SyntaxKind.None,
trailingTrivia);
bool addNewline = semicolonToken.HasTrailingTrivia
&& trailingTrivia.Count() == 1
&& trailingTrivia.First().Kind == SyntaxKind.EndOfLineTrivia;
var newNode = withSemicolonToken(newToken);
if (addNewline)
return newNode.WithTrailingTrivia(Syntax.Whitespace(Environment.NewLine));
else
return newNode;
}
return node;
}
}
}
}
Hopefully it is something along the lines of what you were looking for.

This information would have to be stored in the ClassDeclaration node - as, according to the C# specification, the semi-colon is an optional token in the end of its productions:
class-declaration:
attributesopt class-modifiersopt partialopt class identifier type-parameter-listopt
class-baseopt type-parameter-constraints-clausesopt class-body ;opt
UPDATE
According to Roslyn's documentation, you cannot actually change Syntax Trees - as they are immutable structures. That's probably the reason why kind is readonly. You may, however, create a new tree, using With* methods, defined for each changeable tree property, and using ReplaceNode. There is a good example on Roslyn documentation:
var root = (CompilationUnitSyntax)tree.GetRoot();
var oldUsing = root.Usings[1];
var newUsing = oldUsing.WithName(name); //changes the name property of a Using statement
root = root.ReplaceNode(oldUsing, newUsing);
For converting your new tree into code again (aka pretty printing), you could use the GetText() method from the compilation unit node (in our example, the root variable).
You can also extend a SyntaxRewriter class for performing code transformations. There is an extensive example for doing so in the official Roslyn website; take a look at this particular walkthrough. The following commands write the transformed tree back to the original file:
SyntaxNode newSource = rewriter.Visit(sourceTree.GetRoot());
if (newSource != sourceTree.GetRoot())
{
File.WriteAllText(sourceTree.FilePath, newSource.GetFullText());
}
where rewriter is an instance of a SyntaxRewriter.