We Keep Coding

How to use AvroSerializer without a schema registry

I am trying to write a unit test that verifies that adding a new property to an Avro schema is backwards compatible.
First I took the Avro generated .cs model and saved it as MyModelOld.cs and renamed the class inside to MyModelOld.
Then I re-ran Avro gen against the avsc file with the new property.
What I'm trying to do is this:
var schemaRegistry = -> something that doesn't require a running docker image <-;
var deserializerOld = new AvroDeserializer<MyModelOld>(schemaRegistry);
var serializerNew = new AvroSerializer<MyModel>(schemaRegistry);
var myModel = new MyModel() {...};
var myModelBytes = await serializerNew.SerializeAsync(myModel, new());
var myModelOld = await deserializerOld.DeserializeAsync(myModelBytes, false, new());
// Check properties...
Then I was going to go the opposite direction and check that the new property uses the specified default value.
The problem I'm having is what to use for the schema registry. I don't want to have a docker image running for these tests because I don't think it shouldn't be necessary.
I've tried a mock of ISchemaRegistry, but it appears to need a fully functional class in order for serialize/deserialize to work.
I could probably walk through the logic for CachedSchemaRegistryClient and try to munge it to work, but before I do so I'd like to find out if someone knows of an ISchemaRegistry implementaion that would work for my use case.
Has anyone tried to write tests to validate backwards compatibility of Avro schema updates?
If so, how did you go about doing so?
Thanks.

I ended up doing it this way:
private ISchemaRegistryClient NewTestRegistry(string topic)
{
// Code to mock SchemaRegistry taken from:
// https://github.com/confluentinc/confluent-kafka-dotnet/blob/master/test/Confluent.SchemaRegistry.Serdes.UnitTests/SerializeDeserialize.cs
Dictionary<string, int> store = new Dictionary<string, int>();
var schemaRegistryMock = new Mock<ISchemaRegistryClient>();
#pragma warning disable CS0618 // Type or member is obsolete
schemaRegistryMock.Setup(x => x.ConstructValueSubjectName(topic, It.IsAny<string>()))
.Returns($"{topic}-value");
schemaRegistryMock.Setup(x => x.RegisterSchemaAsync($"{topic}-value", It.IsAny<string>(), It.IsAny<bool>()))
.ReturnsAsync((string topic, string schema, bool normalize) =>
store.TryGetValue(schema, out int id) ? id : store[schema] = store.Count + 1
);
#pragma warning restore CS0618 // Type or member is obsolete
schemaRegistryMock.Setup(x => x.GetSchemaAsync(It.IsAny<int>(), It.IsAny<string>()))
.ReturnsAsync((int id, string format) =>
new Schema(store.Where(x => x.Value == id).First().Key, null, SchemaType.Avro)
);
return schemaRegistryMock.Object;
}
[TestMethod]
public async Task BackwardsCompatible()
{
var topic = "MyCoolTopic";
var schemaRegistry = NewTestRegistry(topic);
var context = new SerializationContext(MessageComponentType.Value, topic);
var deserializerOld = new AvroDeserializer<MyModelOld>(schemaRegistry);
var serializerNew = new AvroSerializer<MyModel>(schemaRegistry);
var myModel = new MyModel() { /* Set properties */};
var myModelBytes = await serializerNew.SerializeAsync(myModel, context);
var myModelOld = await deserializerOld.DeserializeAsync(myModelBytes, false, context);
// Check properties...
}
[TestMethod]
public async Task ForwardsCompatible()
{
// Similar to the above test.
}

If you want to test schemas, you don't need Kafka-related serializers; just use raw Avro C# library.
Alternatively, look at the existing tests
var config = new SchemaRegistryConfig { Url = "irrelevanthost:8081" };
var src = new CachedSchemaRegistryClient(config);
Assert...(src... );

Enrich callback not being called in OpenTelemetry.net, SqlClient

This is on .NET 6.
So I've added an enrich callback with the purpose of adding each parameter as a tag:
appBuilder.Services.AddOpenTelemetryTracing((builder) =>
builder
.AddAspNetCoreInstrumentation()
.SetSampler(new AlwaysOnSampler())
.AddSqlClientInstrumentation(options =>
{
options.SetDbStatementForText = true;
options.SetDbStatementForStoredProcedure = true;
options.RecordException = true;
options.EnableConnectionLevelAttributes = true;
options.Enrich = (activity, eventName, rawObject) =>
{
if (eventName.Equals("OnCustom"))
{
activity.SetTag("ParametersAdded", "true");
if (rawObject is SqlCommand cmd)
{
foreach (SqlParameter parameter in cmd.Parameters)
{
activity.SetTag(parameter.ParameterName, parameter.Value.ToString());
}
}
}
};
})
.AddZipkinExporter(options =>
{
options.Endpoint = new Uri(appBuilder.Configuration["TraceExporterUrl"]);
}));
I'm getting the export in Zipkin, but it's not added the tags, and it doesn't seem to be hitting that Enrich callback at all - the ParametersAdded tag isn't being hit either. Can't figure out why this isn't working - Have I fundamentally misunderstood something?

The Enrich is not called for activities that don't have the IsAllDataRequested property set to true.

How to use hangfire in .net core with mongodb?

I want to use Hangfire for background jobs in registration form process,But I am unable to find Startup.cs file code for Hangfire.mongo.

Just an update to this thread,
As of v0.7.11, MongoMigrationOptions have been updated and no longer contains Strategy. As per the release notes, you now have to use MigrationStrategy instead of Strategy. Also, the values you use for these are different as well. See example below.
var migrationOptions = new MongoMigrationOptions
{
MigrationStrategy = new MigrateMongoMigrationStrategy(),
BackupStrategy = new CollectionMongoBackupStrategy()
};

In Startup class
In ConfigureServices method
Add
//you will use some way to get your connection string
var mongoConnection = Configuration.GetConnectionString("MongoDBAtlasJaken");
var migrationOptions = new MongoMigrationOptions
{
Strategy = MongoMigrationStrategy.Drop,
BackupStrategy = MongoBackupStrategy.Collections
};
services.AddHangfire(config =>
{
config.SetDataCompatibilityLevel(CompatibilityLevel.Version_170);
config.UseSimpleAssemblyNameTypeSerializer();
config.UseRecommendedSerializerSettings();
config.UseMongoStorage(mongoConnection, "Hangfire",new MongoStorageOptions { MigrationOptions = migrationOptions });
});
services.AddHangfireServer();
In Configure method you can add if you want
app.UseHangfireDashboard();

#Graeme's anwser works on the older version of Hangfire. For the new version,
The Migration and Backup strategy are changed to class from enum.
var migrationOptions = new MongoMigrationOptions
{
MigrationStrategy = new DropMongoMigrationStrategy(),
BackupStrategy = new CollectionMongoBackupStrategy()
};
The UseMongoStorage method does not accept the collection name anymore.
services.AddHangfire(config =>
{
config.SetDataCompatibilityLevel(CompatibilityLevel.Version_170);
config.UseSimpleAssemblyNameTypeSerializer();
config.UseRecommendedSerializerSettings();
config.UseMongoStorage(mongoConnection, new MongoStorageOptions { MigrationOptions = migrationOptions, CheckConnection = false });
});
services.AddHangfireServer();

Moq Callback not working with 3 parameter method

I'm trying to understand why the following unit test does not execute the callback. If I modify the code so that the UpdateWorklowInstanceState method only contains 2 parameters (Guid and IList), it works. However, something about having 3 parameters interferes.
What I mean by interferes is that the Callback doesn't appear to get executed. There's no error message. I expect to see the "Error Occurred" message but instead receive an "Element Updated" message which means the Callback did not populate the resultMessages with the NotificationMessage.
public void BusinessObjectReturnsErrorNotification_ReturnErrorMessage()
{
var workflowInstanceGuid = Guid.NewGuid();
var workflowElementModel = new WorkflowElementModel
{
ElementName = "ValidName",
WorkflowInstanceId = workflowInstanceGuid.ToString()
};
var workflowElementInstance = new WorkflowElementInstance
{
ElementName = workflowElementModel.ElementName,
FullDescription = "Full Description",
SummaryDescription = "Summary Description",
RefTime = DateTime.Now,
ElementType = "ElementType"
};
var mockWebApiBusinessObject = new Mock<IWebApiBusinessObject>();
mockWebApiBusinessObject.Setup(m => m.UpdateWorkflowInstanceState(workflowInstanceGuid, workflowElementInstance, It.IsAny<List<NotificationMessage>>()))
.Callback<Guid, WorkflowElementInstance, IList<NotificationMessage>>(
(workflowInstanceId, elementDetails, resultMessages) =>
{
resultMessages.Add(new NotificationMessage("An Error Occured!", MessageSeverity.Error));
});
var controller = new WorkflowElementController(mockWebApiBusinessObject.Object);
var result = controller.UpdateWorkflowElement(workflowElementModel);
Assert.AreEqual("An Error Occured!", result.Content.ReadAsStringAsync().Result);
}
Method under test:
[HttpPost]
[ActionName("UpdateWorkflowElement")]
public HttpResponseMessage UpdateWorkflowElement(WorkflowElementModel workflowElementModel)
{
if (!ModelState.IsValid || workflowElementModel == null)
{
return new HttpResponseMessage(HttpStatusCode.BadRequest);
}
var response = new HttpResponseMessage(HttpStatusCode.OK);
string responseMessage;
if (workflowElementModel.RefTime == DateTime.MinValue)
{
workflowElementModel.RefTime = DateTime.UtcNow;
}
var resultMessages = new List<NotificationMessage>();
var instanceId = new Guid();
if (string.IsNullOrWhiteSpace(workflowElementModel.WorkflowInstanceId) ||
string.IsNullOrWhiteSpace(workflowElementModel.ElementName))
{
responseMessage = "WorkflowInstanceId or ElementName are null or empty";
}
else if (!Guid.TryParse(workflowElementModel.WorkflowInstanceId, out instanceId))
{
responseMessage = "WorkflowInstanceId is not a valid Guid";
}
else
{
var element = new WorkflowElementInstance
{
ElementName = workflowElementModel.ElementName,
RefTime = workflowElementModel.RefTime,
SummaryDescription = workflowElementModel.SummaryDescription ?? "",
FullDescription = workflowElementModel.FullDescription ?? ""
};
_webApiBusinessObject.UpdateWorkflowInstanceState(instanceId, element, resultMessages);
responseMessage = "Element Updated";
}
if (NotificationMessage.HasErrors(resultMessages))
{
responseMessage = resultMessages.Find(m => m.Status == MessageSeverity.Error).Message;
}
response.Content = new StringContent(responseMessage);
return response;
}

It does not work in you case for 3 parameters because you are mixing the expression parameter types.
It.IsAny<List<NotificationMessage>>()
in the setup, as apposed to the
IList<NotificationMessage>
in the callback parameters.
That means the setup expression parameters does not match the callback expression parameters so the call back is not going to be called.
Stick with one type so either go with the List<NotificationMessage> for both
You are also creating new instances of the parameters in the method under test, which would be different instance to the ones used in the setup. That is why the call back is not working. To prove it. Use It.IsAny<>() for all the parameters and it should work
mockWebApiBusinessObject
.Setup(m => m.UpdateWorkflowInstanceState(It.IsAny<Guid>(), It.IsAny<WorkflowElementInstance>(), It.IsAny<List<NotificationMessage>>()))
.Callback<Guid, WorkflowElementInstance, List<NotificationMessage>>(
(workflowInstanceId, elementDetails, resultMessages) =>
{
resultMessages.Add(new NotificationMessage("An Error Occured!", MessageSeverity.Error));
});
Or the more generic interface
mockWebApiBusinessObject
.Setup(m => m.UpdateWorkflowInstanceState(It.IsAny<Guid>(), It.IsAny<WorkflowElementInstance>(), It.IsAny<IList<NotificationMessage>>()))
.Callback<Guid, WorkflowElementInstance, IList<NotificationMessage>>(
(workflowInstanceId, elementDetails, resultMessages) =>
{
resultMessages.Add(new NotificationMessage("An Error Occured!", MessageSeverity.Error));
});
You should also take some time and review Moq Quickstart to get a better understanding of how to use the mocking framework.

Please consider updating at minor places in your unit test.
Add before mocking IWebApiBusinessObject object:
List<NotificationMessage> messages = new List<NotificationMessage>();
Additionally, update Callback :
var mock = new Mock<IWebApiBusinessObject>();
mock.
Setup(m => m.UpdateWorkflowInstanceState(It.IsNotNull<Guid>(), It.IsNotNull<WorkflowElementInstance>(),It.IsAny<List<NotificationMessage>>() )).
Callback(() =>
{
messages.Add(new NotificationMessage("error msg", MessageSeverity.Severe));
messages.Add(new NotificationMessage("Ignore Message", MessageSeverity.Normal)); // this is optional... u can remove it if u want.
});
And need to update the source code method UpdateWorkflowElement(WorkflowElementModel model) to
UpdateWorkflowElement(WorkflowElementModel model, List<NotificationMessage> messages);
Consider changes in unit test code calling UpdateWorkflowElement to
var result = controller.UpdateWorkflowElement(workflowElementModel, messages);
If I have understood your UpdateWorkflowInstanceState() method correctly,
then you are using IWebApiBusinessObject to call UpdateWorkflowInstanceState( , , ) method.
When UpdateWorkflowInstanceState( , , ) executes during unit testing, it fires the Callback in your unit test and adds messages in list of NotificationMessage.

ReactiveUI: Why do I have to specify the scheduler explicitly in "...Throttle..."when using the TestScheduler

I'm new to ReactiveUI. I have the following simple setup: a path to a csv can be specified and the containing datapoints will be displayed to the user (using oxyplot).
Now I'm trying to test the following subscription:
public GraphViewModel(IScreen hostScreen)
{
HostScreen = hostScreen;
setupGraphFormatting();
// Data Loading if path is valid
this.WhenAnyValue(viewModel => viewModel.PathToDataCsv)
.ObserveOn(RxApp.MainThreadScheduler)
.Throttle(TimeSpan.FromMilliseconds(500), RxApp.TaskpoolScheduler)
.Select(csvPath => csvPath?.Trim('"'))
.Where(csvPath => !string.IsNullOrEmpty(csvPath) && File.Exists(csvPath))
.Subscribe(csvPath =>
{
csvPath = csvPath?.Trim('"');
updatePlotModel(csvPath);
}, exception => {});
/* additional Code*/
}
And that's the corresponding UnitTest:
[Test]
public void If_PathToDataCsv_has_a_valid_value()
{
new TestScheduler().With(scheduler =>
{
string pathToValidCsvFile = "data.log";
var viewModel = new GraphViewModel(null);
scheduler.AdvanceByMs(1000);
viewModel.PathToDataCsv = pathToValidCsvFile;
scheduler.AdvanceByMs(1000);
viewModel.PlotModel.Series.Count.Should().Be(6);
});
}
My first implementation of WhenAnyValue didn't set any of the Schedulers specifically ( in Throttle and lacking any ObserverOn ):
public GraphViewModel(IScreen hostScreen)
{
HostScreen = hostScreen;
setupGraphFormatting();
// Data Loading if path is valid
this.WhenAnyValue(viewModel => viewModel.PathToDataCsv)
.Throttle(TimeSpan.FromMilliseconds(500))
.Select(csvPath => csvPath?.Trim('"'))
.Where(csvPath => !string.IsNullOrEmpty(csvPath) && File.Exists(csvPath))
.Subscribe(csvPath =>
{
csvPath = csvPath?.Trim('"');
updatePlotModel(csvPath);
}, exception => {});
/* additional Code*/
}
But then my Unittest failed. My assumption was that TestScheduler was being used for Throttle behind the scenes and I didn't have to do anything. Am I doing something wrong or is this the right way: If I want to use TestScheduler/TimeTravel™ I have to specify the schedulers the way I did?
Edit in response to Glenn Watsons answer:
Ok, now it's clear: The methods in question (Throttle, ObserverOn) of course do not use ReactiveUI's Schedulers, because these are methods from the Reactive Extensions Framework. So they can't be replaced implicitly by ReactiveUI in case of a UnitTest except I tell the methods to use the RxApp Schedulers...

RxApp provides the ThreadPoolScheduler when you are in release mode, and the testing scheduler when you are in unit test mode.
By default the reactive extensions (separate to ReactiveUI) will use their own default schedulers which are unaware of unit tests.