We Keep Coding

c# Synchronise asynchronous messages processing - TPL Dataflow pipelines

What am I doing:
I have a microservice, part of an order processing system, which constantly consumes order messages from RabbitMQ and I need to temporary keep them in my microservice DB, until they are handled (accepted/declined). The order messages are forwarded to a TPL Dataflow pipeline, which has two branches.
The first one being for processing 'created' orders.
When an order is created I perform some operations, such as statistics for the order, validations, persisting in to DB and notifying the user through SignalR. (I have 5 blocks for this branch, and some include I/O network calls).
The second branch (2 blocks) is for accepted/declined orders. When I receive an order with status accepted/declined I need to remove it from my DB and also notify the user through SignalR.
The message for 'created' and 'accepted/declined' order differs only in its status property.
The forwarding to each of the two branches happens through TPL Dataflow's link predicate, based on the order status.
My problem:
Sometimes a message for declined/accpeted order arrives 50-150 ms after the message for the same order being created. Usually, 50-150 ms are a lot of time in computing, but the first dataflow branch is dependent on external calls to other services, which might cause a delay in the processing.
I want to make sure that I have fully processed the message with status 'created' and only after that to process the message for the same order being 'accepted/declined'.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using System.Threading.Tasks.Dataflow;
namespace ConsoleApp12
{
public static class Program
{
static void Main(string[] args)
{
var linkOptions = new DataflowLinkOptions { PropagateCompletion = true };
var executionOptions = new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 1, // potentially would be increased
BoundedCapacity = 50
};
var deserialize = new TransformBlock<OrderStatus, Order>(o =>
{
return new Order { Status = o };
}, executionOptions);
#region Creted order
var b11 = new TransformBlock<Order, Order>(async o =>
{
await Task.Delay(15); // do something
return o;
}, executionOptions);
var b12 = new TransformBlock<Order, Order>(async o =>
{
await Task.Delay(15); // do something
return o;
}, executionOptions);
var b13 = new TransformBlock<Order, Order>(async o =>
{
await Task.Delay(15);
Console.WriteLine("Saved In DB");
return o;
}, executionOptions);
var b14 = new ActionBlock<Order>(async o =>
{
await Task.Delay(5);
Console.WriteLine("SignalR order created");
}, executionOptions);
#endregion
#region Accepted/Declined
var b21 = new ActionBlock<Order>(async o =>
{
await Task.Delay(5);
Console.WriteLine("Deleted from DB");
}, executionOptions);
var b22 = new ActionBlock<Order>(async o =>
{
await Task.Delay(5);
Console.WriteLine("SignalR order deleted");
}, executionOptions);
#endregion
var deleteFromDbAndSignalRInParallelJob = new List<ITargetBlock<Order>> { b21, b22 }.CreateGuaranteedBroadcastBlock();
deserialize.LinkTo(b11, linkOptions, x => x.Status == OrderStatus.Created);
b11.LinkTo(b12, linkOptions);
b12.LinkTo(b13, linkOptions);
b13.LinkTo(b14, linkOptions);
deserialize.LinkTo(deleteFromDbAndSignalRInParallelJob, linkOptions);
deserialize.Post(OrderStatus.Created);
Thread.Sleep(30); // delay between messaged
deserialize.Post(OrderStatus.Declined);
Console.ReadKey();
}
}
class Order
{
public OrderStatus Status { get; init; }
}
enum OrderStatus
{
Created = 1,
Declined = 2
}
public static class DataflowExtensions
{
public static ITargetBlock<T> CreateGuaranteedBroadcastBlock<T>(this IEnumerable<ITargetBlock<T>> targets)
{
var targetsList = targets.ToList();
return new ActionBlock<T>(async item =>
{
var tasks = targetsList.Select(t => t.SendAsync(item));
await Task.WhenAll(tasks);
},
new ExecutionDataflowBlockOptions { BoundedCapacity = 100 });
}
}
}
Here is a sample with simplified models and logic simulating that the 'created' order branch is taking more time to complete.
The output is:
SignalR order deleted
Deleted from DB
Saved In DB
SignalR order created

Using "Queue" in C# for sending SMS

I am trying to send SMS to clients from transactional data. It works fine but sometimes the staging table gets tanked due to huge data load. I was thinking if there is any way to queue data in a First-In-First-Out (FIFO) order. I am still learning C#. Please help me to implement the following code in a smart way.
I am sharing what I did.
private static void Main(string[] args)
{
var datetime = DateTime.Now;
var hr = datetime.Hour;
var mm = datetime.Minute + 1;
var ss = double.Parse(ConfigurationManager.AppSettings["SMSTIME"]);
// For Interval in Seconds
// This Scheduler will start at current time + 1 minute and call after every 3 Seconds
// IntervalInSeconds(start_hour, start_minute, seconds)
SchedulerTasks.IntervalInSeconds(hr, mm, ss,
() =>
{
CreateNewSms.Instance.PostNewSmsText();
});
Console.ReadLine();
}
public async void PostNewSmsText()
{
var smsDt = await Task.Run(() => DataAccessLayer.Instance.InsertSmsDataToLocalDb());
if (smsDt.Rows.Count != 0)
{
var smsData = GetSmsData(smsDt);
var xmlRespDoc = new XmlDocument();
await Task.Run(() =>
{
foreach (var sms in smsData)
{
var smsText = GetSmsText(sms.Channel, sms.AccountNumber, sms.Amount, sms.DrCr, sms.CurrentBalance, sms.BranchName,
sms.CurrencyName, sms.TrnDescription, sms.UbsTrnDateTime);
var smsSubmitTime = DateTime.Now.ToString(CultureInfo.InvariantCulture);
var response = ExecuteSmsService.Instance.PostSmsText(sms.MobileNo, smsText, sms.TrnRefNo);
var smsDeliveryTime = DateTime.Now.ToString(CultureInfo.InvariantCulture);
var operatorInfo = ConfigurationManager.AppSettings["Operator"];
xmlRespDoc.LoadXml(response);
var smsCsmsId = xmlRespDoc.SelectSingleNode("//SMSINFO/CSMSID")?.InnerText;
var smsRefNo = xmlRespDoc.SelectSingleNode("//SMSINFO/REFERENCEID")?.InnerText;
var smsDeliveryStatus = xmlRespDoc.SelectSingleNode("//SMSINFO/MSISDNSTATUS")?.InnerText;
if (smsRefNo != null) //Save response data if sms delivery successful
{
DataAccessLayer.Instance.InsertSmsResponseDetails(
sms.EntrySerialNo,
smsCsmsId,
smsRefNo,
smsText,
smsSubmitTime,
smsDeliveryTime,
true,
"Success",
operatorInfo);
DataAccessLayer.Instance.UpdateSmsStatus(sms.EntrySerialNo);
DataAccessLayer.Instance.DeleteSmsData(sms.EntrySerialNo, true);
Console.WriteLine("SMS successfully sent to: " + sms.MobileNo);
}
else
{
//If any sms delivery failed here we set a counter to retry again in next date
DataAccessLayer.Instance.SetSmsCounter(sms.EntrySerialNo);
//Save response data if sms delivery unsuccessful with failed status
DataAccessLayer.Instance.InsertSmsResponseDetails(
sms.EntrySerialNo,
smsCsmsId,
null,
smsText,
smsSubmitTime,
smsDeliveryTime,
false,
smsDeliveryStatus,
operatorInfo);
DataAccessLayer.Instance.DeleteSmsData(sms.EntrySerialNo, false);
Console.WriteLine("Invalid mobile no: " + sms.MobileNo);
}
}
});
}
else
{
Console.WriteLine("Waiting for transactions.....");
}
}
I was searching on Queue Implementations and found this but confused how to use it here in my code.
public class SendSmsQueue
{
private ConcurrentQueue<object> _jobs = new ConcurrentQueue<object>();
public SendSmsQueue()
{
var thread = new Thread(new ThreadStart(OnStart));
thread.IsBackground = true;
thread.Start();
}
public void Enqueue(object job)
{
_jobs.Enqueue(job);
}
private void OnStart()
{
while (true)
{
if (_jobs.TryDequeue(out object result))
{
Console.WriteLine(result);
}
}
}
}

ZeroMQ Broadcast with Proxy losing messages

I am doing some performance tests on ZeroMQ in order to compare it with others like RabbitMQ and ActiveMQ.
In my broadcast tests and to avoid "The Dynamic Discovery Problem" as referred by ZeroMQ documentation I have used a proxy. In my scenario, I am using 50 concurrent publishers each one sending 500 messages with 1ms delay between sends. Each message is then read by 50 subscribers. And as I said I am losing messages, each of the subscribers should receive a total of 25000 messages and they are each receiving between 5000 and 10000 messages only.
I am using Windows and C# .Net client clrzmq4 (4.1.0.31).
I have already tried some solutions that I found on other posts:
I have set linger to TimeSpan.MaxValue
I have set ReceiveHighWatermark to 0 (as it is presented as infinite, but I have tried also Int32.MaxValue)
I have set checked for slow start receivers, I made receivers start some seconds before publishers
I had to make sure that no garbage collection is made to the socket instances (linger should do it but to make sure)
I have a similar scenario (with similar logic) using NetMQ and it works fine. The other scenario does not use security though and this one does (and that's also the reason why I use clrzmq in this one because I need client authentication with certificates that is not yet possible on NetMQ).
EDIT:
public class MCVEPublisher
{
public void publish(int numberOfMessages)
{
string topic = "TopicA";
ZContext ZContext = ZContext.Create();
ZSocket publisher = new ZSocket(ZContext, ZSocketType.PUB);
//Security
// Create or load certificates
ZCert serverCert = Main.GetOrCreateCert("publisher");
var actor = new ZActor(ZContext, ZAuth.Action, null);
actor.Start();
// send CURVE settings to ZAuth
actor.Frontend.Send(new ZFrame("VERBOSE"));
actor.Frontend.Send(new ZMessage(new List<ZFrame>()
{ new ZFrame("ALLOW"), new ZFrame("127.0.0.1") }));
actor.Frontend.Send(new ZMessage(new List<ZFrame>()
{ new ZFrame("CURVE"), new ZFrame(".curve") }));
publisher.CurvePublicKey = serverCert.PublicKey;
publisher.CurveSecretKey = serverCert.SecretKey;
publisher.CurveServer = true;
publisher.Linger = TimeSpan.MaxValue;
publisher.ReceiveHighWatermark = Int32.MaxValue;
publisher.Connect("tcp://127.0.0.1:5678");
Thread.Sleep(3500);
for (int i = 0; i < numberOfMessages; i++)
{
Thread.Sleep(1);
var update = $"{topic} {"message"}";
using (var updateFrame = new ZFrame(update))
{
publisher.Send(updateFrame);
}
}
//just to make sure it does not end instantly
Thread.Sleep(60000);
//just to make sure publisher is not garbage collected
ulong Affinity = publisher.Affinity;
}
}
public class MCVESubscriber
{
private ZSocket subscriber;
private List<string> prints = new List<string>();
public void read()
{
string topic = "TopicA";
var context = new ZContext();
subscriber = new ZSocket(context, ZSocketType.SUB);
//Security
ZCert serverCert = Main.GetOrCreateCert("xpub");
ZCert clientCert = Main.GetOrCreateCert("subscriber");
subscriber.CurvePublicKey = clientCert.PublicKey;
subscriber.CurveSecretKey = clientCert.SecretKey;
subscriber.CurveServer = true;
subscriber.CurveServerKey = serverCert.PublicKey;
subscriber.Linger = TimeSpan.MaxValue;
subscriber.ReceiveHighWatermark = Int32.MaxValue;
// Connect
subscriber.Connect("tcp://127.0.0.1:1234");
subscriber.Subscribe(topic);
while (true)
{
using (var replyFrame = subscriber.ReceiveFrame())
{
string messageReceived = replyFrame.ReadString();
messageReceived = Convert.ToString(messageReceived.Split(' ')[1]);
prints.Add(messageReceived);
}
}
}
public void PrintMessages()
{
Console.WriteLine("printing " + prints.Count);
}
}
public class Main
{
static void Main(string[] args)
{
broadcast(500, 50, 50, 30000);
}
public static void broadcast(int numberOfMessages, int numberOfPublishers, int numberOfSubscribers, int timeOfRun)
{
new Thread(() =>
{
using (var context = new ZContext())
using (var xsubSocket = new ZSocket(context, ZSocketType.XSUB))
using (var xpubSocket = new ZSocket(context, ZSocketType.XPUB))
{
//Security
ZCert serverCert = GetOrCreateCert("publisher");
ZCert clientCert = GetOrCreateCert("xsub");
xsubSocket.CurvePublicKey = clientCert.PublicKey;
xsubSocket.CurveSecretKey = clientCert.SecretKey;
xsubSocket.CurveServer = true;
xsubSocket.CurveServerKey = serverCert.PublicKey;
xsubSocket.Linger = TimeSpan.MaxValue;
xsubSocket.ReceiveHighWatermark = Int32.MaxValue;
xsubSocket.Bind("tcp://*:5678");
//Security
serverCert = GetOrCreateCert("xpub");
var actor = new ZActor(ZAuth.Action0, null);
actor.Start();
// send CURVE settings to ZAuth
actor.Frontend.Send(new ZFrame("VERBOSE"));
actor.Frontend.Send(new ZMessage(new List<ZFrame>()
{ new ZFrame("ALLOW"), new ZFrame("127.0.0.1") }));
actor.Frontend.Send(new ZMessage(new List<ZFrame>()
{ new ZFrame("CURVE"), new ZFrame(".curve") }));
xpubSocket.CurvePublicKey = serverCert.PublicKey;
xpubSocket.CurveSecretKey = serverCert.SecretKey;
xpubSocket.CurveServer = true;
xpubSocket.Linger = TimeSpan.MaxValue;
xpubSocket.ReceiveHighWatermark = Int32.MaxValue;
xpubSocket.Bind("tcp://*:1234");
using (var subscription = ZFrame.Create(1))
{
subscription.Write(new byte[] { 0x1 }, 0, 1);
xpubSocket.Send(subscription);
}
Console.WriteLine("Intermediary started, and waiting for messages");
// proxy messages between frontend / backend
ZContext.Proxy(xsubSocket, xpubSocket);
Console.WriteLine("end of proxy");
//just to make sure it does not end instantly
Thread.Sleep(60000);
//just to make sure xpubSocket and xsubSocket are not garbage collected
ulong Affinity = xpubSocket.Affinity;
int ReceiveHighWatermark = xsubSocket.ReceiveHighWatermark;
}
}).Start();
Thread.Sleep(5000); //to make sure proxy started
List<MCVESubscriber> Subscribers = new List<MCVESubscriber>();
for (int i = 0; i < numberOfSubscribers; i++)
{
MCVESubscriber ZeroMqSubscriber = new MCVESubscriber();
new Thread(() =>
{
ZeroMqSubscriber.read();
}).Start();
Subscribers.Add(ZeroMqSubscriber);
}
Thread.Sleep(10000);//to make sure all subscribers started
for (int i = 0; i < numberOfPublishers; i++)
{
MCVEPublisher ZeroMqPublisherBroadcast = new MCVEPublisher();
new Thread(() =>
{
ZeroMqPublisherBroadcast.publish(numberOfMessages);
}).Start();
}
Thread.Sleep(timeOfRun);
foreach (MCVESubscriber Subscriber in Subscribers)
{
Subscriber.PrintMessages();
}
}
public static ZCert GetOrCreateCert(string name, string curvpath = ".curve")
{
ZCert cert;
string keyfile = Path.Combine(curvpath, name + ".pub");
if (!File.Exists(keyfile))
{
cert = new ZCert();
Directory.CreateDirectory(curvpath);
cert.SetMeta("name", name);
cert.Save(keyfile);
}
else
{
cert = ZCert.Load(keyfile);
}
return cert;
}
}
This code also produces the expected number of messages when security is disabled, but when turned on it doesn't.
Does someone know another thing to check? Or has it happened to anyone before?
Thanks

M2MQTT client disconnecting without an exception or error message

I'm trying to create an API that consumes various topics.
For this, I'm trying to multi-thread things, so that the whole thing can be scalable into multiple APIs, later on, but that's very besides the point.
I'm using ASP.net Core 4.0, if that's got anything to do with it. Entity Framework as well.
My problem is based on my connection to my Mosquitto server being broken without throwing an exception or anything of the like, after a minute or so. It doesn't matter how big the messages are, or how many are exchanged. I have no idea of how I can create a callback or anything of the kind to know what's going on with my connection. Can anyone help?
I'll link the code I use to establish a connection and subscribe to a connection below. Using the Subscribe method or doing it manually also changes nothing. I'm at a loss, here.
Thanks in advance!
Main.cs:
Task.Factory.StartNew(() => DataflowController.ResumeQueuesAsync());
BuildWebHost(args).Run();
DataflowController.cs:
public static Boolean Subscribe(String topic)
{
Console.WriteLine("Hello from " + topic);
MqttClient mqttClient = new MqttClient(brokerAddress);
byte code = mqttClient.Connect(Guid.NewGuid().ToString());
// Register to message received
mqttClient.MqttMsgPublishReceived += client_recievedMessageAsync;
string clientId = Guid.NewGuid().ToString();
mqttClient.Connect(clientId);
// Subscribe to topic
mqttClient.Subscribe(new String[] { topic }, new byte[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
System.Console.ReadLine();
return true;
}
public static async Task ResumeQueuesAsync()
{
var mongoClient = new MongoClient(connectionString);
var db = mongoClient.GetDatabase(databaseName);
var topics = db.GetCollection<BsonDocument>(topicCollection);
var filter = new BsonDocument();
List<BsonDocument> result = topics.Find(filter).ToList();
var resultSize = result.Count;
Task[] subscriptions = new Task[resultSize];
MqttClient mqttClient = new MqttClient(brokerAddress);
byte code = mqttClient.Connect(Guid.NewGuid().ToString());
// Register to message received
mqttClient.MqttMsgPublishReceived += client_recievedMessageAsync;
string clientId = Guid.NewGuid().ToString();
mqttClient.Connect(clientId);
int counter = 0;
foreach(var doc in result)
{
subscriptions[counter] = new Task(() =>
{
Console.WriteLine("Hello from " + doc["topic"].ToString());
// Subscribe to topic
mqttClient.Subscribe(new String[] { doc["topic"].ToString() }, new byte[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
System.Console.ReadLine();
});
counter++;
}
foreach(Task task in subscriptions)
{
task.Start();
}
}
static async void client_recievedMessageAsync(object sender, MqttMsgPublishEventArgs e)
{
// Handle message received
var message = System.Text.Encoding.Default.GetString(e.Message);
var topic = e.Topic;
var id = topic.Split("/")[2];
BsonDocument doc = new BsonDocument {
{"Plug ID", id },
{"Consumption", message }
};
await Save(doc, "smartPDM_consumption");
System.Console.WriteLine("Message received from " + topic + " : " + message);
}

This line was the issue:
byte code = mqttClient.Connect(Guid.NewGuid().ToString());
Deleted it, and it just worked.

Bing Speech to Text API - Communicate via websocket in c#

I'm trying to get the Bing Speech API to work in C# via WebSockets. I've looked through the implementation in Javascript here and have been following the protocol instructions here, but I've come up against a complete brick wall. I can't use the existing C# service because I'm running in a Linux container, so I need to use an implementation on .net Core. Annoyingly, the existing service is closed-source!
I can connect to the web socket successfully, but I can't ever get the server to respond to my connection. I'm expecting to receive a turn.start text message from the server, but I get booted off the server as soon as I've sent a few bytes of an audio file. I know the audio file is in the right format because I've got it directly from the C# service sample here.
I feel like I’ve exhausted the options here. The only thing I can think of now is that I’m not sending the audio chunks correctly. Currently, I’m just sending the audio file in consecutive 4096 bytes. I know the first audio message contains the RIFF header which is only 36 bytes, and then I'm just sending this along with the next (4096-36) bytes.
Here is my code in full. You should just be able to run it as a .net core or .net framework console application, and will need an audio file and an API key.
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Net.WebSockets;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleApp3
{
class Program
{
static void Main(string[] args)
{
Task.Run(async () =>
{
var bingService = new BingSpeechToTextService();
var audioFilePath = #"FILEPATH GOES HERE";
var authenticationKey = #"BING AUTHENTICATION KEY GOES HERE";
await bingService.RegisterJob(audioFilePath, authenticationKey);
}).Wait();
}
}
public class BingSpeechToTextService
{
/* #region Private Static Methods */
private static async Task Receiving(ClientWebSocket client)
{
var buffer = new byte[128];
while (true)
{
var result = await client.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
var res = Encoding.UTF8.GetString(buffer, 0, result.Count);
if (result.MessageType == WebSocketMessageType.Text)
{
Console.WriteLine(Encoding.UTF8.GetString(buffer, 0, result.Count));
}
else if (result.MessageType == WebSocketMessageType.Close)
{
Console.WriteLine($"Closing ... reason {client.CloseStatusDescription}");
var description = client.CloseStatusDescription;
//await client.CloseOutputAsync(WebSocketCloseStatus.NormalClosure, "", CancellationToken.None);
break;
}
else
{
Console.WriteLine("Other result");
}
}
}
/* #endregion Private Static Methods */
/* #region Public Static Methods */
public static UInt16 ReverseBytes(UInt16 value)
{
return (UInt16)((value & 0xFFU) << 8 | (value & 0xFF00U) >> 8);
}
/* #endregion Public Static Methods */
/* #region Interface: 'Unscrypt.Bing.SpeechToText.Client.Api.IBingSpeechToTextJobService' Methods */
public async Task<int?> RegisterJob(string audioFilePath, string authenticationKeyStr)
{
var authenticationKey = new BingSocketAuthentication(authenticationKeyStr);
var token = authenticationKey.GetAccessToken();
/* #region Connect web socket */
var cws = new ClientWebSocket();
var connectionId = Guid.NewGuid().ToString("N");
var lang = "en-US";
cws.Options.SetRequestHeader("X-ConnectionId", connectionId);
cws.Options.SetRequestHeader("Authorization", "Bearer " + token);
Console.WriteLine("Connecting to web socket.");
var url = $"wss://speech.platform.bing.com/speech/recognition/interactive/cognitiveservices/v1?format=simple&language={lang}";
await cws.ConnectAsync(new Uri(url), new CancellationToken());
Console.WriteLine("Connected.");
/* #endregion*/
/* #region Receiving */
var receiving = Receiving(cws);
/* #endregion*/
/* #region Sending */
var sending = Task.Run(async () =>
{
/* #region Send speech.config */
dynamic speechConfig =
new
{
context = new
{
system = new
{
version = "1.0.00000"
},
os = new
{
platform = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36",
name = "Browser",
version = ""
},
device = new
{
manufacturer = "SpeechSample",
model = "SpeechSample",
version = "1.0.00000"
}
}
};
var requestId = Guid.NewGuid().ToString("N");
var speechConfigJson = JsonConvert.SerializeObject(speechConfig, Formatting.None);
StringBuilder outputBuilder = new StringBuilder();
outputBuilder.Append("path:speech.config\r\n"); //Should this be \r\n
outputBuilder.Append($"x-timestamp:{DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ss.fffK")}\r\n");
outputBuilder.Append($"content-type:application/json\r\n");
outputBuilder.Append("\r\n\r\n");
outputBuilder.Append(speechConfigJson);
var strh = outputBuilder.ToString();
var encoded = Encoding.UTF8.GetBytes(outputBuilder.ToString());
var buffer = new ArraySegment<byte>(encoded, 0, encoded.Length);
if (cws.State != WebSocketState.Open) return;
Console.WriteLine("Sending speech.config");
await cws.SendAsync(buffer, WebSocketMessageType.Text, true, new CancellationToken());
Console.WriteLine("Sent.");
/* #endregion*/
/* #region Send audio parts. */
var fileInfo = new FileInfo(audioFilePath);
var streamReader = fileInfo.OpenRead();
for (int cursor = 0; cursor < fileInfo.Length; cursor++)
{
outputBuilder.Clear();
outputBuilder.Append("path:audio\r\n");
outputBuilder.Append($"x-requestid:{requestId}\r\n");
outputBuilder.Append($"x-timestamp:{DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ss.fffK")}\r\n");
outputBuilder.Append($"content-type:audio/x-wav");
var headerBytes = Encoding.ASCII.GetBytes(outputBuilder.ToString());
var headerbuffer = new ArraySegment<byte>(headerBytes, 0, headerBytes.Length);
var str = "0x" + (headerBytes.Length).ToString("X");
var headerHeadBytes = BitConverter.GetBytes((UInt16)headerBytes.Length);
var isBigEndian = !BitConverter.IsLittleEndian;
var headerHead = !isBigEndian ? new byte[] { headerHeadBytes[1], headerHeadBytes[0] } : new byte[] { headerHeadBytes[0], headerHeadBytes[1] };
//Audio should be pcm 16kHz, 16bps mono
var byteLen = 8192 - headerBytes.Length - 2;
var fbuff = new byte[byteLen];
streamReader.Read(fbuff, 0, byteLen);
var arr = headerHead.Concat(headerBytes).Concat(fbuff).ToArray();
var arrSeg = new ArraySegment<byte>(arr, 0, arr.Length);
Console.WriteLine($"Sending data from {cursor}");
if (cws.State != WebSocketState.Open) return;
cursor += byteLen;
var end = cursor >= fileInfo.Length;
await cws.SendAsync(arrSeg, WebSocketMessageType.Binary, true, new CancellationToken());
Console.WriteLine("Data sent");
var dt = Encoding.ASCII.GetString(arr);
}
await cws.SendAsync(new ArraySegment<byte>(), WebSocketMessageType.Binary, true, new CancellationToken());
streamReader.Dispose();
/* #endregion*/
{
var startWait = DateTime.UtcNow;
while ((DateTime.UtcNow - startWait).TotalSeconds < 30)
{
await Task.Delay(1);
}
if (cws.State != WebSocketState.Open) return;
}
});
/* #endregion*/
/* #region Wait for tasks to complete */
await Task.WhenAll(sending, receiving);
if (sending.IsFaulted)
{
var err = sending.Exception;
throw err;
}
if (receiving.IsFaulted)
{
var err = receiving.Exception;
throw err;
}
/* #endregion*/
return null;
}
/* #endregion Interface: 'Unscrypt.Bing.SpeechToText.Client.Api.IBingSpeechToTextJobService' Methods */
public class BingSocketAuthentication
{
public static readonly string FetchTokenUri = "https://api.cognitive.microsoft.com/sts/v1.0";
private string subscriptionKey;
private string token;
private Timer accessTokenRenewer;
//Access token expires every 10 minutes. Renew it every 9 minutes.
private const int RefreshTokenDuration = 9;
public BingSocketAuthentication(string subscriptionKey)
{
this.subscriptionKey = subscriptionKey;
this.token = FetchToken(FetchTokenUri, subscriptionKey).Result;
// renew the token on set duration.
accessTokenRenewer = new Timer(new TimerCallback(OnTokenExpiredCallback),
this,
TimeSpan.FromMinutes(RefreshTokenDuration),
TimeSpan.FromMilliseconds(-1));
}
public string GetAccessToken()
{
return this.token;
}
private void RenewAccessToken()
{
this.token = FetchToken(FetchTokenUri, this.subscriptionKey).Result;
Console.WriteLine("Renewed token.");
}
private void OnTokenExpiredCallback(object stateInfo)
{
try
{
RenewAccessToken();
}
catch (Exception ex)
{
Console.WriteLine(string.Format("Failed renewing access token. Details: {0}", ex.Message));
}
finally
{
try
{
accessTokenRenewer.Change(TimeSpan.FromMinutes(RefreshTokenDuration), TimeSpan.FromMilliseconds(-1));
}
catch (Exception ex)
{
Console.WriteLine(string.Format("Failed to reschedule the timer to renew access token. Details: {0}", ex.Message));
}
}
}
private async Task<string> FetchToken(string fetchUri, string subscriptionKey)
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);
UriBuilder uriBuilder = new UriBuilder(fetchUri);
uriBuilder.Path += "/issueToken";
var result = await client.PostAsync(uriBuilder.Uri.AbsoluteUri, null);
Console.WriteLine("Token Uri: {0}", uriBuilder.Uri.AbsoluteUri);
return await result.Content.ReadAsStringAsync();
}
}
}
}
}

I knew it was going to be simple.
After a frustrating few hours of coding, I've found the problem. I've been forgetting to send a request id along with the speech.config call.