We Keep Coding

Using httplistener handle more concurrent request

Goal: To be able to handle thousands of request per second
Current Result: while the test code was at request 15k the http server had only processed about 3-400 request
My first iteration of this code I had set HTTP_HANDLER_THREADS to 2 where as after about 200 proccessed the server is overloaded and crashes.
I then up this number to 5 with similiar results.
I then up the number to 5000 and I got to around 800. This seems to me I am doing something very wrong because there is no way my system is running 5000 threads and based on running top -H -p <pid> I could see the thread pool and it did not open 5000 threads.
Very confused and would like help on how to adjust this to handle thousands of request
TestCase
class Program
{
private static readonly HttpClient client = new HttpClient();
private static int send_amount = 200000;
private static int sent_request = 0;
static void Main(string[] args)
{
Parallel.For(0, send_amount, i =>
{
var values = new Dictionary<string, string>{
{ "request", i.ToString() }
};
var content = new FormUrlEncodedContent(values);
var response = client.PostAsync("http://192.168.102.165:1990", content);
sent_request += 1;
Console.WriteLine($"Sent request {sent_request}/{send_amount}");
});
}
}
Constants
internal class Constants
{
public static bool IsDebugMode = false;
public const string PRODUCTION_IP = "192.168.102.165";
public const string DEVELOPMENT_IP = "192.168.102.165";
public const Int32 HTTP_PORT = 1990;
public const Int32 HTTPS_PORT = 1990;
public const Int32 HTTP_HANDLER_THREADS = 5000;
public static string[] SERVER_BINDS
{
get
{
var server_binds = new string[]
{
$"http://{(IsDebugMode ? DEVELOPMENT_IP : PRODUCTION_IP)}:{HTTP_PORT}/"
};
return server_binds;
}
}
}
Http server class
internal class HTTPServer
{
private readonly ProcessDataDelegate handler;
private readonly HttpListener listener;
public HTTPServer(HttpListener listener, string[] prefixes, ProcessDataDelegate handler)
{
this.listener = listener;
this.handler = handler;
for (var i = 0; i < prefixes.Length; i++)
{
listener.Prefixes.Add(prefixes[i]);
}
}
public void Start()
{
if (listener.IsListening)
{
return;
}
listener.Start();
for (var i = 0; i < Constants.HTTP_HANDLER_THREADS; i++)
{
listener.GetContextAsync().ContinueWith(ProcessRequestHandler);
}
}
public void Stop()
{
if (listener.IsListening)
{
listener.Stop();
}
}
private void ProcessRequestHandler(Task<HttpListenerContext> result)
{
var context = result.Result;
if (!listener.IsListening) return;
//Start a new listener which will replace this
listener.GetContextAsync().ContinueWith(ProcessRequestHandler);
//Read request
var request = new StreamReader(context.Request.InputStream).ReadToEnd();
//Prepare response
var response_bytes = handler.Invoke(request);
context.Response.ContentLength64 = response_bytes.Length;
var output = context.Response.OutputStream;
output.WriteAsync(response_bytes, 0, response_bytes.Length);
output.Close();
}
}
Program
class Program
{
private static Thread _console_thread;
private static int request_amount = 0;
static void Main(string[] args)
{
InitConsoleThread();
InitHttpServer();
}
private static void InitHttpServer()
{
var http_listener = new HttpListener();
var http_server = new HTTPServer.HTTPServer(http_listener, Constants.SERVER_BINDS, ProcessResponse);
http_server.Start();
}
private static byte[] ProcessResponse(string response)
{
request_amount += 1;
Console.WriteLine(request_amount);
//Sleep was added here to simulate the code doing something
Thread.Sleep(2000);
return new byte[0];
}
private static void InitConsoleThread()
{
_console_thread = new Thread(ConsoleLoop)
{
Name = "ConsoleThread"
};
_console_thread.Start();
}
private static void ConsoleLoop()
{
SpinWait.SpinUntil(() => false);
}
}

How to wait for methods to finish then do new action?

I'm setting up my architechture to use Cef.Offscreen. In order to make it easy to work with I have divided some parts. But I run into a problem that controller loading finshes and serves a view before everything has been able to load.
Here's my structure --> Controller
public ActionResult InitBrowser()
{
ICefSharpRenderer renderer = RendererSingelton.GetInstance();
//Try to render something in default appdomain
renderer.LoginToTradingView(null, null);
ViewBag.SiteTitle = BrowserActions.RunScriptInNamedBrowser("loginbrowser", #"(function() {return document.title;} )();");
ViewBag.ImagesixtyfourUrl = BrowserActions.TakeScreenshot("loginbrowser");
//this is returned to fast, we have to wait for all
return View();
}
I have this class to get do some basic actions and initialize if needed.
public class CefSharpRenderer : MarshalByRefObject, ICefSharpRenderer
{
private ChromiumWebBrowser _browser;
private TaskCompletionSource<JavascriptResponse> _taskCompletionSource;
private string _name;
public void LoginToTradingView(string url, string browserName)
{
CheckIfCefIsInitialized();
BrowserFactory.GetBrowserInstance(#"https://se.tradingview.com/", "loginbrowser");
}
public void CreateBrowserAndGoToUrl(string url, string browserName)
{
CheckIfCefIsInitialized();
BrowserFactory.GetBrowserInstance(url, "browserName");
}
public void CheckIfCefIsInitialized()
{
if (!Cef.IsInitialized)
{
var settings = new CefSettings();
var assemblyPath = Path.GetDirectoryName(new Uri(GetType().Assembly.CodeBase).LocalPath);
settings.BrowserSubprocessPath = Path.Combine(assemblyPath, "CefSharp.BrowserSubprocess.exe");
settings.ResourcesDirPath = assemblyPath;
settings.LocalesDirPath = Path.Combine(assemblyPath, "locales");
var osVersion = Environment.OSVersion;
//Disable GPU for Windows 7
if (osVersion.Version.Major == 6 && osVersion.Version.Minor == 1)
{
// Disable GPU in WPF and Offscreen examples until #1634 has been resolved
settings.CefCommandLineArgs.Add("disable-gpu", "1");
}
//Perform dependency check to make sure all relevant resources are in our output directory.
Cef.Initialize(settings, performDependencyCheck: false, cefApp: null);
}
}
}
I get my browserinstance here and connected the events to be fired.
public static class BrowserFactory
{
public static ChromiumWebBrowser GetBrowserInstance(string _url, string browsername)
{
if (!BrowserContainer.CheckIfBrowserExists(browsername))
{
ChromiumWebBrowser _browser = new ChromiumWebBrowser(_url);
_browser.LoadingStateChanged += BrowserEvents.OnLoadingStateChanged;
BrowserContainer.AddDataHolder(browsername, new DataBrowserHolder { BrowserName = browsername, ChromiumWebBrow = _browser });
return _browser;
}
return null;
}
}
Browserevent loads correct page.
public static class BrowserEvents
{
public static void OnLoadingStateChanged(object sender, LoadingStateChangedEventArgs args)
{
if (args.IsLoading == false)
{
ChromiumWebBrowser cwb = (ChromiumWebBrowser)sender;
if (cwb.Address == "https://se.tradingview.com/")
{
BrowserActions.LogInToTradingView("xxxxx", "yyyyyyy", "loginbrowser");
}
}
}
}
Last my browseractions, spare med for the thread sleeps it's just under construction and it works atm.
public static class BrowserActions
{
public static void LogInToTradingView(string twusername, string twpassword, string browserName)
{
ChromiumWebBrowser _dataholder = BrowserContainer.GetDataHolderByName(browserName).ChromiumWebBrow;
IFrame ifww = _dataholder.GetMainFrame();
// var lull = #"(function() { var serielength = TradingView.bottomWidgetBar._widgets.backtesting._reportWidgetsSet.reportWidget._data.filledOrders.length; return serielength; })();";
// JavascriptResponse _js = Task.Run(async () => { return await _browser.GetMainFrame().EvaluateScriptAsync(lull); }).Result;
ifww.ExecuteJavaScriptAsync(#"(function() { window.document.getElementsByClassName('tv-header__link tv-header__link--signin js-header__signin')[0].click();})();");
// var loginusernamescript =
var loginpasswordscript = #"(function() { window.document.getElementsByClassName('tv-control-material-input tv-signin-dialog__input tv-control-material-input__control')[1].value= " + twpassword + "; })();";
var clkloginbtn = #"(function() { document.getElementsByClassName('tv-button tv-button--no-border-radius tv-button--size_large tv-button--primary_ghost tv-button--loader')[0].click();})();";
Thread.Sleep(300);
ifww.ExecuteJavaScriptAsync(#"(function() { window.document.getElementsByClassName('tv-control-material-input tv-signin-dialog__input tv-control-material-input__control')[0].click();})();");
Thread.Sleep(50);
ifww.ExecuteJavaScriptAsync(#"(function() { window.document.getElementsByClassName('tv-control-material-input tv-signin-dialog__input tv-control-material-input__control')[0].value = '" + twusername + "';})();");
Thread.Sleep(50);
ifww.ExecuteJavaScriptAsync(#"(function() { window.document.getElementsByClassName('tv-control-material-input tv-signin-dialog__input tv-control-material-input__control')[1].click();})();");
Thread.Sleep(50);
ifww.ExecuteJavaScriptAsync(#"(function() { window.document.getElementsByClassName('tv-control-material-input tv-signin-dialog__input tv-control-material-input__control')[1].value = '" + twpassword + "';})();");
Thread.Sleep(50);
ifww.ExecuteJavaScriptAsync(#"(function() { document.getElementsByClassName('tv-button tv-button--no-border-radius tv-button--size_large tv-button--primary_ghost tv-button--loader')[0].click();})();");
}
public static string TakeScreenshot(string browserName)
{
try
{
Bitmap img = Task.Run(async () => { return await BrowserContainer.GetDataHolderByName(browserName).ChromiumWebBrow.ScreenshotAsync(); }).Result;
// object mgss = img.Clone();
string baseen = ExtraFunctions.ToBase64String(img, ImageFormat.Png);
return baseen;
}
catch (Exception e)
{
var x = e.InnerException;
return null;
}
}
public static string RunScriptInNamedBrowser(string browserName, string script)
{
try
{
string str = Task.Run(async () => { return await BrowserContainer.GetDataHolderByName(browserName).ChromiumWebBrow.GetMainFrame().EvaluateScriptAsync(script); }).Result.ToString();
// object mgss = img.Clone();
return str;
}
catch (Exception e)
{
var x = e.InnerException;
return null;
}
}
}
How can I get my browser actions to report back to my controller so that I can wait for them to finish?

For a Task asynchronous operation to report back, it's possible to use Progress<T>. How that's done is detailed in Enabling Progress and Cancellation in Async APIs. The key is:
var progressIndicator = new Progress<int>(ReportProgress);
This creates a Progress<T> object that can indicate how far a task is complete, and also call a custom method (ReportProgress) at set intervals. You can create a custom class if necessary instead of using int.
So your browser actions can report back to the controller with the progress reporting method until everything is complete.

how to create last file with renaming messages where message batch size is not greater than equal to 240

In below application,
Producer method adding messages to a blocking collection.
In Consumer method, I'm consuming blocking collection and adding messages to a list and when size >= 240, writing that list to json file.
At some point I don't have any new messages in blocking collection, but in Consumer, I have a list of messages which is not >=240 in size, then in this case , the app is not able to write to a new JSON file (rest of the data).
How can I let the Consumer know that no new messages coming up, write whatever left with you in a new file?
Is this possible? let say Consumer will wait for 1 minute and if there is no new messages, then write whatever left in an new file?
Here is the code (here I'm adding 11 messages. Till 9 messages the batch size is 240 and it's generates a file, but message no 10 & 11 not able to write in new file),
class Program
{
private static List<Batch> batchList = new List<Batch>();
private static BlockingCollection<Message> messages = new BlockingCollection<Message>();
private static int maxbatchsize = 240;
private static int currentsize = 0;
private static void Producer()
{
int ctr = 1;
while (ctr <= 11)
{
messages.Add(new Message { Id = ctr, Name = $"Name-{ctr}" });
Thread.Sleep(1000);
ctr++;
}
}
private static void Consumer()
{
foreach (var message in messages.GetConsumingEnumerable())
{
var msg = JsonConvert.SerializeObject(message);
Console.WriteLine(msg);
if (currentsize + msg.Length >= maxbatchsize)
{
WriteToFile(batchList);
}
batchList.Add(new Batch { Message = message });
currentsize += msg.Length;
}
}
private static void WriteToFile(List<Batch> batchList)
{
using (StreamWriter outFile = System.IO.File.CreateText(Path.Combine(#"C:\TEMP", $"{DateTime.Now.ToString("yyyyMMddHHmmssfff")}.json")))
{
outFile.Write(JsonConvert.SerializeObject(batchList));
}
batchList.Clear();
currentsize = 0;
}
static void Main(string[] args)
{
var producer = Task.Factory.StartNew(() => Producer());
var consumer = Task.Factory.StartNew(() => Consumer());
Console.Read();
}
}
}
Supporting classes,
public class Message
{
public int Id { get; set; }
public string Name { get; set; }
}
public class Batch
{
public Message Message { get; set; }
}

Update:
class Program
{
private static readonly List<Batch> BatchList = new List<Batch>();
private static readonly BlockingCollection<Message> Messages = new BlockingCollection<Message>();
private const int Maxbatchsize = 240;
private static int _currentsize;
private static void Producer()
{
int ctr = 1;
while (ctr <= 11)
{
Messages.Add(new Message { Id = ctr, Name = $"Name-{ctr}" });
Thread.Sleep(1000);
ctr++;
}
Messages.CompleteAdding();
}
private static void Consumer()
{
foreach (var message in Messages.GetConsumingEnumerable())
{
if (_currentsize >= Maxbatchsize)
{
var listToWrite = new Batch[BatchList.Count];
BatchList.CopyTo(listToWrite);
BatchList.Clear();
_currentsize = 0;
WriteToFile(listToWrite.ToList());
}
else
{
Thread.Sleep(1000);
if (Messages.IsAddingCompleted)
{
var remainSize = Messages.Select(JsonConvert.SerializeObject).Sum(x => x.Length);
if (remainSize == 0)
{
var lastMsg = JsonConvert.SerializeObject(message);
BatchList.Add(new Batch { Message = message });
_currentsize += lastMsg.Length;
Console.WriteLine(lastMsg);
var additionListToWrite = new Batch[BatchList.Count];
BatchList.CopyTo(additionListToWrite);
BatchList.Clear();
_currentsize = 0;
WriteToFile(additionListToWrite.ToList());
break;
}
}
}
var msg = JsonConvert.SerializeObject(message);
BatchList.Add(new Batch { Message = message });
_currentsize += msg.Length;
Console.WriteLine(msg);
}
}
private static void WriteToFile(List<Batch> listToWrite)
{
using (StreamWriter outFile = System.IO.File.CreateText(Path.Combine(#"C:\TEMP", $"{DateTime.Now.ToString("yyyyMMddHHmmssfff")}.json")))
{
outFile.Write(JsonConvert.SerializeObject(listToWrite));
}
}
static void Main(string[] args)
{
var producer = Task.Factory.StartNew(() => Producer());
var consumer = Task.Factory.StartNew(() => Consumer());
Console.Read();
}
}

Azure Redis Cache - pool of ConnectionMultiplexer objects

We are using C1 Azure Redis Cache in our application. Recently we are experiencing lots of time-outs on GET operations.
According to this article, one of possible solutions is to implement pool of ConnectionMultiplexer objects.
Another possible solution is to use a pool of ConnectionMultiplexer
objects in your client, and choose the “least loaded”
ConnectionMultiplexer when sending a new request. This should prevent
a single timeout from causing other requests to also timeout.
How would implementation of a pool of ConnectionMultiplexer objects using C# look like?
Edit:
Related question that I asked recently.

You can also accomplish this in a easier way by using StackExchange.Redis.Extensions
Sample code:
using StackExchange.Redis;
using StackExchange.Redis.Extensions.Core.Abstractions;
using StackExchange.Redis.Extensions.Core.Configuration;
using System;
using System.Collections.Concurrent;
using System.Linq;
namespace Pool.Redis
{
/// <summary>
/// Provides redis pool
/// </summary>
public class RedisConnectionPool : IRedisCacheConnectionPoolManager
{
private static ConcurrentBag<Lazy<ConnectionMultiplexer>> connections;
private readonly RedisConfiguration redisConfiguration;
public RedisConnectionPool(RedisConfiguration redisConfiguration)
{
this.redisConfiguration = redisConfiguration;
Initialize();
}
public IConnectionMultiplexer GetConnection()
{
Lazy<ConnectionMultiplexer> response;
var loadedLazys = connections.Where(lazy => lazy.IsValueCreated);
if (loadedLazys.Count() == connections.Count)
{
response = connections.OrderBy(x => x.Value.GetCounters().TotalOutstanding).First();
}
else
{
response = connections.First(lazy => !lazy.IsValueCreated);
}
return response.Value;
}
private void Initialize()
{
connections = new ConcurrentBag<Lazy<ConnectionMultiplexer>>();
for (int i = 0; i < redisConfiguration.PoolSize; i++)
{
connections.Add(new Lazy<ConnectionMultiplexer>(() => ConnectionMultiplexer.Connect(redisConfiguration.ConfigurationOptions)));
}
}
public void Dispose()
{
var activeConnections = connections.Where(lazy => lazy.IsValueCreated).ToList();
activeConnections.ForEach(connection => connection.Value.Dispose());
Initialize();
}
}
}
Where RedisConfiguration is something like this:
return new RedisConfiguration()
{
AbortOnConnectFail = true,
Hosts = new RedisHost[] {
new RedisHost()
{
Host = ConfigurationManager.AppSettings["RedisCacheAddress"].ToString(),
Port = 6380
},
},
ConnectTimeout = Convert.ToInt32(ConfigurationManager.AppSettings["RedisTimeout"].ToString()),
Database = 0,
Ssl = true,
Password = ConfigurationManager.AppSettings["RedisCachePassword"].ToString(),
ServerEnumerationStrategy = new ServerEnumerationStrategy()
{
Mode = ServerEnumerationStrategy.ModeOptions.All,
TargetRole = ServerEnumerationStrategy.TargetRoleOptions.Any,
UnreachableServerAction = ServerEnumerationStrategy.UnreachableServerActionOptions.Throw
},
PoolSize = 50
};

If you're using StackExchange.Redis, according to this github issue, you can use the TotalOutstanding property on the connection multiplexer object.
Here is a implementation I came up with, that is working correctly:
public static int POOL_SIZE = 100;
private static readonly Object lockPookRoundRobin = new Object();
private static Lazy<Context>[] lazyConnection = null;
//Static initializer to be executed once on the first call
private static void InitConnectionPool()
{
lock (lockPookRoundRobin)
{
if (lazyConnection == null) {
lazyConnection = new Lazy<Context>[POOL_SIZE];
}
for (int i = 0; i < POOL_SIZE; i++){
if (lazyConnection[i] == null)
lazyConnection[i] = new Lazy<Context>(() => new Context("YOUR_CONNECTION_STRING", new CachingFramework.Redis.Serializers.JsonSerializer()));
}
}
}
private static Context GetLeastLoadedConnection()
{
//choose the least loaded connection from the pool
/*
var minValue = lazyConnection.Min((lazyCtx) => lazyCtx.Value.GetConnectionMultiplexer().GetCounters().TotalOutstanding);
var lazyContext = lazyConnection.Where((lazyCtx) => lazyCtx.Value.GetConnectionMultiplexer().GetCounters().TotalOutstanding == minValue).First();
*/
// UPDATE following #Luke Foust comment below
Lazy<Connection> lazyContext;
var loadedLazys = lazyConnection.Where((lazy) => lazy.IsValueCreated);
if(loadedLazys.Count()==lazyConnection.Count()){
var minValue = loadedLazys.Min((lazy) => lazy.Value.TotalOutstanding);
lazyContext = loadedLazys.Where((lazy) => lazy.Value.TotalOutstanding == minValue).First();
}else{
lazyContext = lazyConnection[loadedLazys.Count()];
}
return lazyContext.Value;
}
private static Context Connection
{
get
{
lock (lockPookRoundRobin)
{
return GetLeastLoadedConnection();
}
}
}
public RedisCacheService()
{
InitConnectionPool();
}

How to pass a data from new thread?

I have a class:
class ShowComboBoxUpdater
{
private ComboBox _showComboBox;
private String _searchString;
private RequestState _endState;
public event EventHandler ResultUpdated;
public string[] getShowList()
{
if (_endState.serverQueryResult != null)
return _endState.serverQueryResult;
return new string[] { "" };
}
public ShowComboBoxUpdater(ComboBox combo, Image refreshImage)
{
_showComboBox = combo;
_refreshImage = refreshImage;
_endState = new RequestState();
}
public void RequestUpdatingComboSource()
{
_searchString = _showComboBox.Text;
Thread t = new Thread(new ThreadStart(MakeServerConnectionThread));
t.IsBackground = true;
t.Start();
}
private void MakeServerConnectionThread()
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://services.tvrage.com/myfeeds/search.php?show=" + _searchString);
_endState.request = request;
IAsyncResult result = request.BeginGetResponse(new AsyncCallback(RequestingThread), _endState);
ThreadPool.RegisterWaitForSingleObject(result.AsyncWaitHandle, new WaitOrTimerCallback(ScanTimeoutCallback), _endState, (30 * 1000), true);
}
private void RequestingThread(IAsyncResult result)
{
RequestState state = (RequestState)result.AsyncState;
WebRequest request = (WebRequest)state.request;
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(result);
Stream bodyStream = response.GetResponseStream();
StreamReader r = new StreamReader(bodyStream);
string xmlResponse = r.ReadToEnd().Trim();
using (StringReader XMLStream = new StringReader(xmlResponse))
{
XPathNavigator feed = new XPathDocument(XMLStream).CreateNavigator();
XPathNodeIterator nodesNavigator = (XPathNodeIterator)feed.Evaluate("descendant::show/name/text()");
int titlesCount = nodesNavigator.Count;
string[] titles = new string[titlesCount];
foreach (XPathNavigator n in nodesNavigator)
{
titles[--titlesCount] = n.Value;
}
state.serverQueryResult = titles;
if (this.ResultUpdated != null) this.ResultUpdated(this, new EventArgs());
}
}
private static void ScanTimeoutCallback(object state, bool timedOut)
{
if (timedOut)
{
RequestState reqState = (RequestState)state;
if (reqState != null)
reqState.request.Abort();
}
}
}
In my main thread I create ShowComboBoxUpdater and connect event ResultUpdate to other event. Then I am calling RequestUpdatingComboSource() method. I have my event activated but, how can I get the resulting serverQueryResult ? I know it's there but everything that I try results in exception that what I want to get is "owned by other thread".

How to pass value ?
public class MyArgs : EventArgs
{
//Declare any specific type here
public string ResultToPass { get; private set; }
public MyArgs()
{
}
}
if (this.ResultUpdated != null) this.ResultUpdated(this, new MyArgs(){ResultToPass="Your actual result"} );
How to update result ?
Capture SynchronizationContext of the UI ( main thread) in order to instruct whenever you want the value to be updated back in the UI. Send/Post method on the captured SynchronizationContext reference in order to push the message into UI thread.
public partial class MainWindow : Window
{
SynchronizationContext UISyncContext;
public MainWindow()
{
InitializeComponent();
}
public StartProcessing()
{
//Let say this method is been called from UI thread. i.e on a button click
//capture the current synchronization context
UISyncContext=TaskScheduler.FromCurrentSynchronizationContext;
}
public UpdateResultInUI()
{
//Let's say this is is the method which user triggers at
//some point in time ( with the assumption that we have Myresult in hand)
if(UISyncContext!=null)
UISyncContext.Send(new SendOrPostCallback(delegate{ PutItInUI }),null);
//Use Send method - to send your request synchronously
//Use Post method- to send your request asynchronously
}
void PutItInUI()
{
//this method help you to put your result in UI/controls
}