We Keep Coding

Kafka Avro Value deserialization issue

I send data using the AvroSpecific or AvroGeneric method and it works fine. The problem occurs when I try to consume the data sent by the above methods.
I use version libraries:
Confluent.Kafka v1.4.0
Confluent.SchemaRegistry v1.4.0
Confluent.SchemaRegistry.Serdes v1.3.0
Here is my Consumer code:
var schemaRegistryConfig = new SchemaRegistryConfig
{
Url = schemaRegistryUrl
};
using (var schemaRegistry = new CachedSchemaRegistryClient(schemaRegistryConfig))
using (var consumer =
new ConsumerBuilder<Ignore, GenericRecord>(consumerConfig)
.SetValueDeserializer(new AvroDeserializer<GenericRecord>(schemaRegistry).AsSyncOverAsync())
.Build())
{
consumer.Subscribe(topicName);
try
{
while (true)
{
try
{
var consumeResult = consumer.Consume(token);
//do something
}
catch (ConsumeException ex)
{
logger.LogError($"Error occured: {ex.Error.Reason}", ex);
}
}
}
catch (OperationCanceledException)
{
consumer.Close();
}
}

How can computation on an IotEdge module be triggered from within a .net core app?

I need to trigger some computation on an IotEdge module from an Administration-Backend Application.
On https://learn.microsoft.com/en-us/azure/iot-edge/module-development it says
Currently, a module cannot receive cloud-to-device messages
So it seems that calling direct methods seems to be the way to go. How can I implement a direct method and trigger it from within a .NET Core App?

In Main or Init Method of your IotEdge module you have to create a ModuleClient and connect it to a MethodHandler:
AmqpTransportSettings amqpSetting = new AmqpTransportSettings(TransportType.Amqp_Tcp_Only);
ITransportSettings[] settings = { amqpSetting };
ModuleClient ioTHubModuleClient = await ModuleClient.CreateFromEnvironmentAsync(settings);
await ioTHubModuleClient.OpenAsync();
await ioTHubModuleClient.SetMethodHandlerAsync("MyDirectMethodName", MyDirectMethodHandler, null);
Then you have to add the DirectMethodHandler to your IotEge module:
static async Task<MethodResponse> MyDirectMethodHandler(MethodRequest methodRequest, object userContext)
{
Console.WriteLine($"My direct method has been called!");
var payload = methodRequest.DataAsJson;
Console.WriteLine($"Payload: {payload}");
try
{
// perform your computation using the payload
}
catch (Exception e)
{
Console.WriteLine($"Computation failed! Error: {e.Message}");
return new MethodResponse(Encoding.UTF8.GetBytes("{\"errormessage\": \"" + e.Message + "\"}"), 500);
}
Console.WriteLine($"Computation successfull.");
return new MethodResponse(Encoding.UTF8.GetBytes("{\"status\": \"ok\"}"), 200);
}
From within your .Net core Application you can then trigger the direct method like this:
var iotHubConnectionString = "MyIotHubConnectionString";
var deviceId = "MyDeviceId";
var moduleId = "MyModuleId";
var methodName = "MyDirectMethodName";
var payload = "MyJsonPayloadString";
var cloudToDeviceMethod = new CloudToDeviceMethod(methodName, TimeSpan.FromSeconds(10));
cloudToDeviceMethod.SetPayloadJson(payload);
ServiceClient serviceClient = ServiceClient.CreateFromConnectionString(iotHubConnectionString);
try
{
var methodResult = await serviceClient.InvokeDeviceMethodAsync(deviceId, moduleId, cloudToDeviceMethod);
if(methodResult.Status == 200)
{
// Handle Success
}
else if (methodResult.Status == 500)
{
// Handle Failure
}
}
catch (Exception e)
{
// Device does not exist or is offline
Console.WriteLine(e.Message);
}

Microsoft.Azure.NotificationHub - Async call in sync method

I currently have to provide a sync as async method in my API: Please find the code below. The only problem is that I don’t have a
sync method in the backend. I use Azure.NotificationHub client. That client has only *Async methods. Is my way reasonable?
public PushHubNotificationResult SendPushMessage(string userId, string message)
{
PushHubNotificationResult result = new PushHubNotificationResult();
try
{
result = SendPushMessageAsync(userId, message).GetAwaiter().GetResult();
} catch (Exception ex)
{
result.Status = PushHubNotificationResultType.Error;
result.Error = ex.Message;
result.Exception = ex;
}
return result;
}
public async Task<PushHubNotificationResult> SendPushMessageAsync(string userId, string message)
{
PushHubNotificationResult result = new PushHubNotificationResult();
// EnableTestSend see: https://azure.microsoft.com/en-us/documentation/articles/notification-hubs-push-notification-fixer/#self-diagnose-tips
// Create a new Notification Hub client.
Microsoft.Azure.NotificationHubs.NotificationHubClient hub =
Microsoft.Azure.NotificationHubs.NotificationHubClient.CreateClientFromConnectionString(NotificationHub, NotificationHubName);
// Sending the message so that all template registrations that contain "messageParam"
// will receive the notifications. This includes APNS, GCM, WNS, and MPNS template registrations.
Dictionary<string, string> templateParams = new Dictionary<string, string>();
templateParams["messageParam"] = message;
string userTag = "_UserId:" + userId; // That line sets the IMEI or SerialNo (WLAN only device) == userId to which the push message is sent
try
{
// Send the push notification and log the results.
NotificationOutcome outcome = await hub.SendTemplateNotificationAsync(templateParams, userTag);
result.Status = PushHubNotificationResultType.Success;
foreach (RegistrationResult hubResult in outcome.Results)
{
result.PushNotificationHub = hubResult.ApplicationPlatform;
result.RegistrationId = hubResult.RegistrationId;
result.Outcome = hubResult.Outcome;
}
}
catch (System.Exception ex)
{
result.Status = PushHubNotificationResultType.Error;
result.Error = ex.Message;
result.Exception = ex;
}
return result;
}
thanks for any advice,
Eric

If you want to use sync-over-async, it's very important that you use ConfigureAwait(false) in your async code, otherwise you are very likely to get a deadlock.
NotificationOutcome outcome =
await hub.SendTemplateNotificationAsync(templateParams, userTag).ConfigureAwait(false);
The async method already converts exceptions to PushHubNotificationResultType.Error, why does the sync version do it too?

C# Get IP of Remote Computer on Separate DNS Server

I am working on a utility for managing multiple computers in a specified domain (not necessarily the domain the computer the application is running on is a member of) within a specified directory root. The problem I am running into is once I have the collection of computer names from the external AD OU, I am unable to manage based on computer name because of DNS. Is it possible to perform DNS lookup on an external DNS server provided the IP of DNS server and credentials for that domain?
Here is the code that I have for the initialization process (works within the same domain). Really appreciate any input!
private Task InitializeComputers()
{
try
{
Computers.Clear();
object cLock = new object();
PrincipalContext context = new PrincipalContext(ContextType.Domain, CurrentConfiguration.LDAPAddress,
CurrentConfiguration.DirectoryRoot, ContextOptions.Negotiate,
CurrentConfiguration.AdminUser, CurrentConfiguration.AdminPassword);
ComputerPrincipal computer = new ComputerPrincipal(context);
computer.Name = "*";
PrincipalSearcher searcher = new PrincipalSearcher(computer);
PrincipalSearchResult<Principal> result = searcher.FindAll();
Parallel.ForEach(result, (r) =>
{
ComputerPrincipal principal = r as ComputerPrincipal;
DirectoryObject cObject = new DirectoryObject(CurrentConfiguration)
{
Name = principal.Name
};
lock (cLock)
{
Computers.Add(cObject);
}
});
}
... // Catch stuff here
}
private async Task InitializeConnections()
{
Task[] tasks = Computers.Select(x => x.CheckConnectionAsync()).ToArray();
await Task.WhenAll(tasks);
}
// This is where I need to be able to get the IP Address. Thoughts???
public Task CheckConnectionAsync()
{
return Task.Run(() =>
{
try
{
Ping PingCheck = new Ping();
PingReply Reply = PingCheck.Send(Name); // <--- need IP Address instead
if (Reply.Status == IPStatus.Success)
{
IsAvailable = true;
IPHostEntry host = Dns.GetHostEntry(Name); // Does not work for machines on different domain.
IPAddress = host.AddressList.FirstOrDefault().ToString();
}
else
{
IsAvailable = false;
IsWMIActive = false;
}
}
... // catch stuff here ...
});
}

To follow up, the solution that I found is derived from Heijden's DNS Resolver. I wrote a simple DnsManager class with a single static GetIPAddress method for extracting the IP out of an A Record.
public static string GetIPAddress(string name)
{
Resolver resolver = new Resolver();
resolver.DnsServer = ((App)App.Current).CurrentConfiguration.DNSAddress;
resolver.Recursion = true;
resolver.Retries = 3;
resolver.TimeOut = 1;
resolver.TranportType = TransportType.Udp;
Response response = resolver.Query(name, QType.A);
if (response.header.ANCOUNT > 0)
{
return ((AnswerRR)response.Answer[0]).RECORD.ToString();
}
return null;
}
Then, the updated CheckConnectionAsync method is now written like so:
public Task CheckConnectionAsync()
{
return Task.Run(() =>
{
try
{
IPAddress = DnsManager.GetIPAddress(Name + "." + CurrentConfig.DomainName);
... // check availability by IP rather than name...
}
// catch stuff here...
});
}

As soon as you plan to query dedicated server as DNS source, you have to step aside from default libs. I tried this (if remember correctly) once investigating dedicated DNS requests:
http://www.codeproject.com/Articles/12072/C-NET-DNS-query-component

How do I implement a C# Thrift service and consume it with a Silverlight client?

I'm current looking at Thrift to use as a RPC framework for our apps (mostly written in C# and Silverlight). I've come as far as implementing a service and consuming it from a C# console app (using a socket as transport).
For the C# server side code my code looked like: (basically copying the tutorials included with the source code)
MyServiceHandler handler = new MyServiceHandler();
MyService.Processor processor = new MyService.Processor(handler);
TServerTransport serverTransport = new TServerSocket(9090);
TServer server = new TSimpleServer(processor, serverTransport);
server.Serve();
For the client side code it looked like:
TTransport transport = new TSocket("localhost", 9090);
TProtocol protocol = new TBinaryProtocol(transport);
MyService.Client client = new MyService.Client(protocol);
transport.Open();
client.SomeServiceCall();
However, we will be consuming the service from a Silverlight client, and unfortunately there is no support for sockets in Silverlight for Thrift. I assume I'm forced to use HTTP communication between the client and service, using Thrift's C# THttpClient and THttpHandler classes? I could not find any examples of how to do this out there, can anyone point me in the right direction? Some example server and client side code would be appreciated.

It seems that this issue was already addressed by this guy. According to this JIRA, the fix is available in Thrift 0.9. You can either try this snapshot (note that, as it's not a final release, it might not be stable) or you can apply this patch to the 0.8 release.

I believe by now you would have understood, there is no direct way of communicating from Silverlight to the Cassandra database either using Thrift or any other clients.
I have one simple option related to this. Write a Silverlight enabled web service and consume it from the client.
For example, on the server side you can have a web service which does insert/update/read etc., like this. I just managed to pull out some code which we use for our project. Hope this helps.
using Apache.Cassandra;
using Thrift.Protocol;
using Thrift.Transport;
namespace CassandraWebLibrary
{
public class MyDb
{
String _host;
int _port;
String _keyspace;
bool _isConnected;
TTransport _transport = null;
Apache.Cassandra.Cassandra.Client _client = null;
String columnFamily = "ColumnFamilyName";
public VazhikaattiDB(String host, int port, String keyspace)
{
_host = host;
_port = port;
_keyspace = keyspace;
_isConnected = false;
}
public bool Connect()
{
try
{
_transport = new TFramedTransport(new TSocket(_host, _port));
TProtocol protocol = new TBinaryProtocol(_transport);
_client = new Apache.Cassandra.Cassandra.Client(protocol);
_transport.Open();
_client.set_keyspace(_keyspace);
_isConnected = true;
}
catch (Exception ex)
{
log.Error(ex.ToString());
}
return _isConnected;
}
public bool Close()
{
if (_transport.IsOpen)
_transport.Close();
_isConnected = false;
return true;
}
public bool InsertData(Send your data as parameters here)
{
try
{
List<Column> list = new List<Column>();
string strKey = keyvalue;
#region Inserting into Coulmn family
List<Byte> valbytes = new List<byte>(BitConverter.GetBytes(value)); //You might have to pad this with more bytes to make it length of 8 bytes
Column doublecolumn1 = new Column()
{
Name = Encoding.UTF8.GetBytes("column1"),
Timestamp = timestampvalue,
Value = valbytes.ToArray()
};
list.Add(doublecolumn1);
Column stringcolumn2 = new Column()
{
Name = Encoding.UTF8.GetBytes("column2"),
Timestamp = timestampvalue,
Value = Encoding.UTF8.GetBytes("StringValue")
};
list.Add(stringcolumn2);
Column timecolumn3 = new Column()
{
Name = Encoding.UTF8.GetBytes("column3"),
Timestamp = timestampvalue,
Value = BitConverter.GetBytes(DateTime.Now.Ticks)
};
list.Add(timecolumn3);
#endregion
ColumnParent columnParent = new ColumnParent();
columnParent.Column_family = columnFamily;
Byte[] key = Encoding.UTF8.GetBytes(strKey);
foreach (Column column in list)
{
try
{
_client.insert(key, columnParent, column, ConsistencyLevel.QUORUM);
}
catch (Exception e)
{
log.Error(e.ToString());
}
}
return true;
}
catch (Exception ex)
{
log.Error(ex.ToString());
return false;
}
}
public List<YourReturnObject> GetData(parameters)
{
try
{
ColumnParent columnParent = new ColumnParent();
columnParent.Column_family = columnFamily;
DateTime curdate = startdate;
IndexExpression indExprsecondkey = new IndexExpression();
indExprsecondkey.Column_name = Encoding.UTF8.GetBytes("column");
indExprsecondkey.Op = IndexOperator.EQ;
List<Byte> valbytes = PadLeftBytes((int)yourid, 8);
indExprsecondkey.Value = valbytes.ToArray();
indExprList.Add(indExprsecondkey);
IndexClause indClause = new IndexClause()
{
Expressions = indExprList,
Count = 1000,
Start_key = Encoding.UTF8.GetBytes("")
};
SlicePredicate slice = new SlicePredicate()
{
Slice_range = new SliceRange()
{
//Start and Finish cannot be null
Start = new byte[0],
Finish = new byte[0],
Count = 1000,
Reversed = false
}
};
List<KeySlice> keyslices = _client.get_indexed_slices(columnParent, indClause, slice, ConsistencyLevel.ONE);
foreach (KeySlice ks in keyslices)
{
String stringcolumnvalue = Encoding.UTF8.GetString(cl.Column.Value);
double doublevalue= (Double)BitConverter.ToDouble(cl.Column.Value);
long timeticks = BitConverter.ToInt64(cl.Column.Value, 0);
DateTime dtcolumntime = new DateTime(timeticks);
}
}
catch (Exception ex)
{
log.Error(ex.ToString());
}
return yourdatalist;
}
}
}
Now the above class can be used by your webservice, which in turn will be used by Silverlight. Btw, you'll have to take care of other silverlight issues like size of data to be downloaded from server/webservice etc.,
FYI, our client service of Cassandra runs on port 9160..