We Keep Coding

EasyNetQ - How to retry failed messages & persist RetryCount in message body/header?

I am using EasyNetQ and need to retry failed messages on the original queue. The problem is: even though I successfully increment the TriedCount variable (in the body of every msg), when EasyNetQ publishes the message to the default error queue after an exception, the updated TriedCount is not in the msg! Presumably because it just dumps the original message to the error queue without the consumer's changes.
The updated TriedCount works for in-process republishes, but not when republished through EasyNetQ Hosepipe or EasyNetQ Management Client. The text files Hosepipe generates do not have the TriedCount updated.
public interface IMsgHandler<T> where T: class, IMessageType
{
Task InvokeMsgCallbackFunc(T msg);
Func<T, Task> MsgCallbackFunc { get; set; }
bool IsTryValid(T msg, string refSubscriptionId); // Calls callback only
// if Retry is valid
}
public interface IMessageType
{
int MsgTypeId { get; }
Dictionary<string, TryInfo> MsgTryInfo {get; set;}
}
public class TryInfo
{
public int TriedCount { get; set; }
/*Other information regarding msg attempt*/
}
public bool SubscribeAsync<T>(Func<T, Task> eventHandler, string subscriptionId)
{
IMsgHandler<T> currMsgHandler = new MsgHandler<T>(eventHandler, subscriptionId);
// Using the msgHandler allows to add a mediator between EasyNetQ and the actual callback function
// The mediator can transmit the retried msg or choose to ignore it
return _defaultBus.SubscribeAsync<T>(subscriptionId, currMsgHandler.InvokeMsgCallbackFunc).Queue != null;
}
I have also tried republishing myself through the Management API (rough code):
var client = new ManagementClient("http://localhost", "guest", "guest");
var vhost = client.GetVhostAsync("/").Result;
var errQueue = client.GetQueueAsync("EasyNetQ_Default_Error_Queue",
vhost).Result;
var crit = new GetMessagesCriteria(long.MaxValue,
Ackmodes.ack_requeue_true);
var errMsgs = client.GetMessagesFromQueueAsync(errQueue,
crit).Result;
foreach (var errMsg in errMsgs)
{
var pubRes = client.PublishAsync(client.GetExchangeAsync(errMsg.Exchange, vhost).Result,
new PublishInfo(errMsg.RoutingKey, errMsg.Payload)).Result;
}
This works but only publishes to the error queue again, not on the original queue. Also, I don't know how to add/update the retry information in the body of the message at this stage.
I have explored this library to add headers to the message but I don't see if the count in the body is not being updated, how/why would the count in the header be updated.
Is there any way to persist the TriedCount without resorting to the Advanced bus (in which case I might use the RabbitMQ .Net client itself)?

Just in case it helps someone else, I eventually implemented my own IErrorMessageSerializer (as opposed to implementing the whole IConsumerErrorStrategy, which seemed like an overkill). The reason I am adding the retry info in the body (instead of the header) is that EasyNetQ doesn't handle complex types in the header (not out-of-the-box anyway). So, using a dictionary gives more control for different consumers. I register the custom serializer at the time of creating the bus like so:
_defaultBus = RabbitHutch.CreateBus(currentConnString, serviceRegister => serviceRegister.Register<IErrorMessageSerializer>(serviceProvider => new RetryEnabledErrorMessageSerializer<IMessageType>(givenSubscriptionId)));
And just implemented the Serialize method like so:
public class RetryEnabledErrorMessageSerializer<T> : IErrorMessageSerializer where T : class, IMessageType
{
public string Serialize(byte[] messageBody)
{
string stringifiedMsgBody = Encoding.UTF8.GetString(messageBody);
var objectifiedMsgBody = JObject.Parse(stringifiedMsgBody);
// Add/update RetryInformation into objectifiedMsgBody here
// I have a dictionary that saves <key:consumerId, val: TryInfoObj>
return JsonConvert.SerializeObject(objectifiedMsgBody);
}
}
The actual retrying is done by a simple console app/windows service periodically via the EasyNetQ Management API:
var client = new ManagementClient(AppConfig.BaseAddress, AppConfig.RabbitUsername, AppConfig.RabbitPassword);
var vhost = client.GetVhostAsync("/").Result;
var aliveRes = client.IsAliveAsync(vhost).Result;
var errQueue = client.GetQueueAsync(Constants.EasyNetQErrorQueueName, vhost).Result;
var crit = new GetMessagesCriteria(long.MaxValue, Ackmodes.ack_requeue_false);
var errMsgs = client.GetMessagesFromQueueAsync(errQueue, crit).Result;
foreach (var errMsg in errMsgs)
{
var innerMsg = JsonConvert.DeserializeObject<Error>(errMsg.Payload);
var pubInfo = new PublishInfo(innerMsg.RoutingKey, innerMsg.Message);
pubInfo.Properties.Add("type", innerMsg.BasicProperties.Type);
pubInfo.Properties.Add("correlation_id", innerMsg.BasicProperties.CorrelationId);
pubInfo.Properties.Add("delivery_mode", innerMsg.BasicProperties.DeliveryMode);
var pubRes = client.PublishAsync(client.GetExchangeAsync(innerMsg.Exchange, vhost).Result,
pubInfo).Result;
}
Whether retry is enabled or not is known by my consumer itself, giving it more control so it can choose to handle the retried msg or just ignore it. Once ignored, the msg will obviously not be tried again; that's how EasyNetQ works.

CheckID does not have associated TTL

I'm trying to utilize Consul .NET API to register and fire health checks via TTL. First I'm registering my service with following code:
var address = node.Address;
var id = ServiceId(address);
var registration = new AgentServiceRegistration
{
ID = id,
Name = node.ClusterName,
Address = node.Address.Host,
Port = node.Address.Port.Value,
Check = new AgentServiceCheck
{
TTL = settings.AliveInterval, // 10sec
DeregisterCriticalServiceAfter = settings.AliveTimeout, // 60sec
}
};
// first, try to deregister service, if it has been registered previously
await consul.Agent.ServiceDeregister(registration.ID);
await consul.Agent.ServiceRegister(registration);
Right afterwards, I'm trying to fire a TTL via:
await consul.Agent.PassTTL("service:" + ServiceId(addr), string.Empty);
However, what I end up with is an exception thrown during PassTTL: Consul.ConsulRequestException: Unexpected response, status code InternalServerError: CheckID "service:{service-id}" does not have associated TTL
And the related log from consul agent itself:
[ERR] http: Request PUT /v1/agent/check/pass/service:{service-id}, error: CheckID "service:{service-id}" does not have associated TTL from=127.0.0.1:25419
I'd like to know what I'm doing wrong here.
I'm using consul agent -dev (version: 1.0.1) and Nuget package Consul (version: 0.7.2.3).

Turns out AgentServiceRegistration.Check property is pretty useless. I've achieved the expected result with CheckRegister method.
Here's the code
var registration = new AgentServiceRegistration
{
ID = "serviceId",
Name = node.ClusterName,
Address = node.Address.Host,
Port = node.Address.Port.Value
};
// first, try to deregister service, if it has been registered previously
await consul.Agent.ServiceDeregister(registration.ID);
await consul.Agent.ServiceRegister(registration);
await consul.Agent.CheckRegister(new AgentCheckRegistration()
{
ID = "checkId",
Name = "Check Name",
Status = HealthStatus.Passing,
TTL = settings.AliveInterval,
ServiceID = "serviceId",
DeregisterCriticalServiceAfter = settings.AliveTimeout, // 60sec
})
Now you can pass TTL via
await consul.Agent.PassTTL("checkId", string.Empty);
Just be sure to deregister your check afterwards

It looks like my example was missing a crucial detail here: a ServiceId(address) method was constructing a service ID in form of protocol://service#host:port/ which resulted in Consul complaining about lack of TTL. Changing it to service#host:port seems to fix the error.
I guess in this case a consul error message was very misleading.

Check if id format is:
"service:{service id}:{number}"
In your case, you must pass:
"service:" + ServiceId(addr) + ":1"
as your check id.

WF 4.5 Compile CSharpValue<T> error. What is the correct method?

I've been uniting testing two simple examples of compiling CSharpValue activities. One works and the other doesn't I can't figure out why. If someone could point out the issue and optionally a change to correct it if possible.
Details:
The first unit test works SequenceActivityCompile() the second CodeActivityCompile fails with a NotSupportedException (Expression Activity type CSharpValue requires compilation in order to run. Please ensure that the workflow has been compiled.)
I heard somewhere this can be related to ForImplementation but CodeActivityCompile has the same error whether its value is true or false.
This example is a basic adaption of the Microsoft example at: https://msdn.microsoft.com/en-us/library/jj591618(v=vs.110).aspx
This example blog post discussing compiling C# expressions in WF 4+ at length. If anyone reaching this question needs a basic introduction to the topic:
http://blogs.msdn.com/b/tilovell/archive/2012/05/25/wf4-5-using-csharpvalue-lt-t-gt-and-csharpreference-lt-t-gt-in-net-4-5-compiling-expressions-and-changes-in-visual-studio-generated-xaml.aspx
Related Code:
[TestMethod]
public void SequenceActivityCompile()
{
Activity sequence = new Sequence
{
Activities = { new CSharpValue<string>("\"Hello World \"") }
};
CompileExpressions(sequence);
var result = WorkflowInvoker.Invoke(sequence);
}
[TestMethod]
public void CodeActivityCompile()
{
var code = new CSharpValue<String>("\"Hello World\"");
CompileExpressions(code);
var result = WorkflowInvoker.Invoke(code);
}
void CompileExpressions(Activity activity)
{
// activityName is the Namespace.Type of the activity that contains the
// C# expressions.
string activityName = activity.GetType().ToString();
// Split activityName into Namespace and Type.Append _CompiledExpressionRoot to the type name
// to represent the new type that represents the compiled expressions.
// Take everything after the last . for the type name.
//string activityType = activityName.Split('.').Last() + "_CompiledExpressionRoot";
string activityType = "TestType";
// Take everything before the last . for the namespace.
//string activityNamespace = string.Join(".", activityName.Split('.').Reverse().Skip(1).Reverse());
string activityNamespace = "TestSpace";
// Create a TextExpressionCompilerSettings.
TextExpressionCompilerSettings settings = new TextExpressionCompilerSettings
{
Activity = activity,
Language = "C#",
ActivityName = activityType,
ActivityNamespace = activityNamespace,
RootNamespace = null,
GenerateAsPartialClass = false,
AlwaysGenerateSource = true,
ForImplementation = false
};
// Compile the C# expression.
TextExpressionCompilerResults results =
new TextExpressionCompiler(settings).Compile();
// Any compilation errors are contained in the CompilerMessages.
if (results.HasErrors)
{
throw new Exception("Compilation failed.");
}
// Create an instance of the new compiled expression type.
ICompiledExpressionRoot compiledExpressionRoot =
Activator.CreateInstance(results.ResultType,
new object[] { activity }) as ICompiledExpressionRoot;
// Attach it to the activity.
System.Activities.Expressions.CompiledExpressionInvoker.SetCompiledExpressionRoot(
activity, compiledExpressionRoot);
}

HttpClient ReadAsAsync does not deserialize

I am not sure if the issue I am having is related to the way I'm using Task or if I am an not using ReadAsAsync correctly. I am following the pattern I found here:
http://blogs.msdn.com/b/henrikn/archive/2012/02/11/httpclient-is-here.aspx
Background:
Object I am deserializing is a POCO. Properties have no attributes. It is just a few value type properties and a couple collection properties. REST service appears to work ok also. When I look at the JSON returned by the service it appears to be OK.
Using Web API 2.1 5.1.2
Problem:
.. is calling HttpResponseMessage.Content.ReadAsAsync(). Sometimes it works (returns an object) and sometimes it doesn't (throws "Thread was being aborted" or returns null). It appears the content property can be read once only and subsequent reads throw errors. See comments in code below.
Related questions:
HttpContent.ReadAsAsync Deserialization issue
Question appears to be similar to mine. Answer indicates a bug but this is over two years old.
Code:
[TestMethod]
public void AddSiteTest()
{
// Use POST to create a resource i.e. insert. Use PUT to update.
Site site = new Site {SiteName = "Test", Active = true, URI="www.test.com" };
Site newSite = null;
client.PostAsJsonAsync<Site>(baseURI + "/Sites/AddSite?securityKey="+ SecurityKey, site).ContinueWith(x =>
{
HttpResponseMessage response = x.Result;
response.EnsureSuccessStatusCode();
if (response.IsSuccessStatusCode)
{
try
{
string str = Task.Run(() => response.Content.ReadAsStringAsync()).Result; // yep its json and it is a proprety serialized object
// Method 1 (preferred... ):
//Site siteA = Task.Run(() => response.Content.ReadAsAsync<Site>()).Result; // usuallly throws if content has been read
// Method 2:
Site siteB = response.Content.ReadAsAsync<Site>().Result; // usully returns a valid result (when I dont put a breakpoint on it). Does not deadlock.
// Method 3:
response.Content.ReadAsAsync<Site>().ContinueWith(d =>
{
Site siteC = d.Result; // returns null
});
}
catch (Exception ex)
{
string y = ex.Message;
}
}
});
}

try to use await:
string str = await response.Content.ReadAsStringAsync();
And you have to add async before void in your method.

Modify instance variable used by multiple Tasks

I need some help with handling Tasks. I have an XML String which is deserialized into a class. The class itself contains a property, e.g. rssProvider which is set to an unique value like MSN or YAHOO. However there can be multiple values delimited with an , in this field.
I am using this deserialized class instance in multiple Tasks. However the function which gets called in this task can only work with one rssProvider value, so I have a split on that string and a foreach loop which creates the task.
In the task itself the function I call needs the full object but with one rssProvider. However when I change the value of that property to the value in the foreach the other tasks will fail as they will get the same single value from the task which runs first.
Any ideas how I should restructure the logic? Thanks!
My code:
List<Task<ResponseOBJ>> tasks = new List<Task<ResponseOBJ>>();
// get List of rssProviders
string[] providers = request.requestBody.rssProvider.Split(',');
//go through each provider
foreach (string provider in providers)
{
Task<ResponseOBJ> task = Task.Factory.StartNew<ResponseOBJ>(() =>
{
request.requestBody.rssProvider = provider;
doStuff(request);
}
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());

I would create a copy constructor in your Request object which copies the content of the original Request and creates a fresh one:
public class Request
{
public Request(Request oldRequest)
{
// initalize new request from the old
}
}
And then change my code to create a new request per task:
List<Task<ResponseOBJ>> tasks = new List<Task<ResponseOBJ>>();
// get List of rssProviders
string[] providers = request.requestBody.rssProvider.Split(',');
//go through each provider
foreach (string provider in providers)
{
Task<ResponseOBJ> task = Task.Factory.StartNew<ResponseOBJ>(() =>
{
request.requestBody.rssProvider = provider;
var newRequest = new Request(request);
doStuff(newRequest);
}
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());

One option might be to change doStuff(request) to doStuff(request, provider) and remove the line request.requestBody.rssProvider = provider;, and then change your doStuff accordingly.
foreach (string provider in providers)
{
Task<ResponseOBJ> task = Task.Factory.StartNew<ResponseOBJ>(() =>
{
doStuff(request, provider);
}
tasks.Add(task);
}
Another option (as also mentioned in the above comments) is to create a new request object for each provider.