interprocess C# python real time - c#

I'm working on a project where I'll have one application in C# and another one in Python.
The C# application will continuously analyse stream of data and raise a flag each time something interesting is detected. So each time there will be an event, my Python application will have to read it and continues with it own process, while other flags will continue being sent.
As you understand the C# app won't wait for the Python one to finish its computation before sending another flag.
So I was wondering if it was possible to create a sub/pub (C# being the Publisher, and Python the Subscriber), if yes how can I do it, and do you think it's a good idea?
I'm pretty new in this field, so could you tell me if there are other possibilities?
Thx for your help.

Redis pub/sub is awesome... or ZeroMQ.

Simplest way is PIPE communication. another simple way that not suggested is SOCKET programming. Pipes and Named pipes are good solution to communicate between different processes (over different programming languages). SOCKET programming is like this but may need more Access Level and may be less security.
other type of IPCs seems be unusable.
see for more info:
C# - Pipes
python - Pipes

You may use our aktos-dcs and aktos-dcs-cs libraries. I have successfully used these libraries in production in order to make an RFID Reader (from Impinj) communicate with (actually, integrated in) our telemetry system. The RFID reader has a C# API and we are heavily using Python in our telemetry system.
The simplest test case is a pinger-ponger application, and here is how it looks like with these libraries:
pinger.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using aktos_dcs_cs;
namespace pinger
{
class Pinger : Actor
{
public void handle_PingMessage(Dictionary<string, object> msg)
{
Console.WriteLine("Pinger handled PingMessage: {0} ", msg["text"]);
string msg_ser = #"
{""PongMessage"":
{""text"": ""this is proper message from csharp implementation""}
}
";
System.Threading.Thread.Sleep(2000);
send(msg_ser);
}
}
class Program
{
static void Main(string[] args)
{
Pinger x = new Pinger();
Actor.wait_all();
}
}
}
ponger.py:
from aktos_dcs import *
class Pinger(Actor):
def handle_PingMessage(self, msg_raw):
msg = get_msg_body(msg_raw)
print "Pinger got ping message: ", msg['text'], (time.time() - msg_raw['timestamp'])
sleep(2)
self.send({'PongMessage': {'text': "Hello ponger, this is pinger 1!"}})
if __name__ == "__main__":
ProxyActor()
pinger = Pinger()
pinger.send({'PongMessage': {'text': "Hello ponger, this is STARTUP MESSAGE!"}})
wait_all()

Related

call controller by timer

I have to send query to remote program and recieve data. Then put them in my DB.
This is possible to call controller's action every 60 seconds for example?
Thanks.
PS. I think it's must be done on server side. Not JS solution.
UPDATE:
First, I have MS SQL Server DB.
Second, There is remote program that listen specific TCP port and waiting a query. I want to send a query every 60 seconds and parse response, then put parsed data in my MS SQL Server DB.
Using Ajax you could create a timer to send data to the controller every x seconds .
A spellchecker plugin in my web application does this , to do spell checking as you type .
I think you would be better off using a standalone service (windows, wcf, msmq, etc) that runs in the background and "sends the query" and saves to your DB.
Web Applications are not designed to be utilized as time-based "always alive" mechanisms. "Timing" needs state, and HTTP is a stateless protocol.
Don't try to shoehorn functionality into something that isn't designed to handle it.
Then again i could be completely misunderstanding your question. Quite possible.
Confusing statements:
This is possible to call controller's action
If it's external, how do you know it's a controller? Is this an external API?
There is remote program that listen specific TCP port and waiting a query
That doesn't sound like a web application/controller.
You could use cron on *nix base system. Or your program could trigger events every hours
0 * * * * lynx url-of-your-program-address.com/action/to/call
Your question is confusing. But since we are only here to guess at what you are on about here's a solution that might come close.
(P.S. I haven't compiled or tested this...because why should I care)
class ConsoleApplication
{
public static void Main()
{
Timer myTimer = new Timer();
myTimer.Elapsed += new ElapsedEventHandler( DoAction );
myTimer.Interval = 1000;
myTimer.Start();
while ( Console.Read() != 'q' )
{
; // do nothing...
}
}
public static void DoAction( object source, ElapsedEventArgs e )
{
Console.WriteLine("Made request at {0}", DateTime.Now);
using (WebClient client = new WebClient())
{
using (Stream stream = client.OpenRead("http://whereever"))
using (StreamReader reader = new StreamReader(stream))
{
Console.WriteLine(reader.ReadToEnd());
}
}
}
}
Depending on how flexible your solution should be I would play around with Windows Service solution with Timer in it either with Quartz.Net.
You can find more details using the link below http://quartznet.sourceforge.net/

Sockets on Windows - did I miss something in my program?

Thanks for reading and answering in advance!
I wrote a simple C# program that connects via sockets with a third-party tool. Whenever I send a string longer than 1024 characters, the third-party software throws an error. Now I am trying to find out if this is a problem of my socket code or one of the other software (EnergyPlus).
It is only a few lines of code, and if anyone has suggestions, they would be highly appreciated!
using System.IO;
using System.Net;
using System.Net.Sockets;
...
private int port = 1410;
private TcpListener listener;
private Stream s;
private StreamReader sr;
private StreamWriter sw;
private Socket soc;
...
Here it really starts:
listener = new TcpListener(port);
listener.Start();
soc = listener.AcceptSocket();
// now, the other program connects
soc.SetSocketOption(SocketOptionLevel.Socket,
SocketOptionName.ReceiveTimeout, 10000);
s = new NetworkStream(soc);
sr = new StreamReader(s);
sw = new StreamWriter(s);
sw.AutoFlush = true; // enable automatic flushing
sw.WriteLine("more or less than 1024 characters");
...
This is the code I use. Anything I forgot? Anything I should take care of?
I am glad about any suggestions.
The error I get from E+ is the following:
ExternalInterface: Socket communication received error value " 1" at time = 0.00 hours.
ExternalInterface: Flag from server " 0".
Yu need to look at the specification defined by EnergyPlus; any socket communication needs rules. There are two obvious options here:
you aren't following the rules (maybe it is limited length, or maybe you need to write special marker bytes for this scenario)
their server code doesn't implement the specification correctly (biggest causes are: buffer issues, or: assuming a logical frame arrives in a single network packet)
Actually, I find it interesting that it is doing anything yet, as there is no obvious "frame" there; TCP is a stream, so you ususally need frames to divide logical messages. This usually means one of:
a length prefix, with or without other header data
a cr/lf/crlf terminator (or other terminator), usually for text-based protocols
closing the socket (the ultimate terminator)
You do none of those, so in any server I write, that would be an incomplete message until something else happens. It sounds text-based; I'd try adding a cr/lf/crlf

Are there any high level, simple IPC libraries

Are there any high level, simple IPC libraries for C#? It seems most posts point to using WCF. But this looks far more complicated than I need. I just want to send asynchronous messages between two C# apps that may or may not be on the same system.
I am hoping something exists that is as simple to use as the following which comes down to just a Send and an OnRead command. (Link is the IPC library I hope exists).
private void StartServer()
{
using (var link = new Link("InstanceName"))
{
link.OnRead += delegate(Link client, object data)
{
client.Send("Echoing " + data);
};
link.Connect();
Console.WriteLine("Press any key to stop the server.");
Console.ReadKey();
}
}
private void StartClient()
{
using (var link = new Link("serverName", "InstanceName"))
{
link.OnRead += delegate(Link server, object data) { Console.WriteLine(data); };
link.Connect();
link.Send("Hello There!");
Console.WriteLine("Press any key to stop the client.");
Console.ReadKey();
}
}
Self-hosted WCF would come close.
You can take a look at our MsgConnect which offers exactly what you need and for many platforms including .NET. Design of MsgConnect is inspired by Windows API - PostMessage/SendMessage/GetMessage functions, yet MsgConnect has Socket transport which lets you send messages across network. Samples are available in the installation package.

Is a non-blocking, single-threaded, asynchronous web server (like Node.js) possible in .NET?

I was looking at this question, looking for a way to create a single-threaded, event-based nonblocking asynchronous web server in .NET.
This answer looked promising at first, by claiming that the body of the code runs in a single thread.
However, I tested this in C#:
using System;
using System.IO;
using System.Threading;
class Program
{
static void Main()
{
Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
var sc = new SynchronizationContext();
SynchronizationContext.SetSynchronizationContext(sc);
{
var path = Environment.ExpandEnvironmentVariables(
#"%SystemRoot%\Notepad.exe");
var fs = new FileStream(path, FileMode.Open,
FileAccess.Read, FileShare.ReadWrite, 1024 * 4, true);
var bytes = new byte[1024];
fs.BeginRead(bytes, 0, bytes.Length, ar =>
{
sc.Post(dummy =>
{
var res = fs.EndRead(ar);
// Are we in the same thread?
Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
}, null);
}, null);
}
Thread.Sleep(100);
}
}
And the result was:
1
5
So it seems like, contrary to the answer, the thread initiating the read and the thread ending the read are not the same.
So now my question is, how do you to achieve a single-threaded, event-based nonblocking asynchronous web server in .NET?
The whole SetSynchronizationContext is a red herring, this is just a mechanism for marshalling, the work still happens in the IO Thread Pool.
What you are asking for is a way to queue and harvest Asynchronous Procedure Calls for all your IO work from the main thread. Many higher level frameworks wrap this kind functionality, the most famous one being libevent.
There is a great recap on the various options here: Whats the difference between epoll, poll, threadpool?.
.NET already takes care of scaling for you by have a special "IO Thread Pool" that handles IO access when you call the BeginXYZ methods. This IO Thread Pool must have at least 1 thread per processor on the box. see: ThreadPool.SetMaxThreads.
If single threaded app is a critical requirement (for some crazy reason) you could, of course, interop all of this stuff in using DllImport (see an example here)
However it would be a very complex and risky task:
Why don't we support APCs as a completion mechanism? APCs are really not a good general-purpose completion mechanism for user code. Managing the reentrancy introduced by APCs is nearly impossible; any time you block on a lock, for example, some arbitrary I/O completion might take over your thread. It might try to acquire locks of its own, which may introduce lock ordering problems and thus deadlock. Preventing this requires meticulous design, and the ability to make sure that someone else's code will never run during your alertable wait, and vice-versa. This greatly limits the usefulness of APCs.
So, to recap. If you want a single threaded managed process that does all its work using APC and completion ports, you are going to have to hand code it. Building it would be risky and tricky.
If you simply want high scale networking, you can keep using BeginXYZ and family and rest assured that it will perform well, since it uses APC. You pay a minor price marshalling stuff between threads and the .NET particular implementation.
From: http://msdn.microsoft.com/en-us/magazine/cc300760.aspx
The next step in scaling up the server is to use asynchronous I/O. Asynchronous I/O alleviates the need to create and manage threads. This leads to much simpler code and also is a more efficient I/O model. Asynchronous I/O utilizes callbacks to handle incoming data and connections, which means there are no lists to set up and scan and there is no need to create new worker threads to deal with the pending I/O.
An interesting, side fact, is that single threaded is not the fastest way to do async sockets on Windows using completion ports see: http://doc.sch130.nsc.ru/www.sysinternals.com/ntw2k/info/comport.shtml
The goal of a server is to incur as few context switches as possible by having its threads avoid unnecessary blocking, while at the same time maximizing parallelism by using multiple threads. The ideal is for there to be a thread actively servicing a client request on every processor and for those threads not to block if there are additional requests waiting when they complete a request. For this to work correctly however, there must be a way for the application to activate another thread when one processing a client request blocks on I/O (like when it reads from a file as part of the processing).
What you need is a "message loop" which takes the next task on a queue and executes it. Additionally, every task needs to be coded so that it completes as much work as possible without blocking, and then enqueues additional tasks to pick up a task that needs time later. There is nothing magical about this: never using a blocking call and never spawn additional threads.
For example, when processing an HTTP GET, the server can read as much data as is currently available on the socket. If this is not enough data to handle the request, then enqueue a new task to read from the socket again in the future. In the case of a FileStream, you want to set the ReadTimeout on the instance to a low value and be prepared to read fewer bytes than the entire file.
C# 5 actually makes this pattern much more trivial. Many people think that the async functionality implies multithreading, but that is not the case. Using async, you can essentially get the task queue I mentioned earlier without ever explicility managing it.
Yes, it's called Manos de mono
Seriously, the entire idea behind manos is a single threaded asynchronous event driven web server.
High performance and scalable. Modeled after tornadoweb, the technology that powers friend feed, Manos is capable of thousands of simultaneous connections, ideal for applications that create persistent connections with the server.
The project appears to be low on maintenance and probably wouldn't be production ready but it makes a good case study as a demonstration that this is possible.
Here's a great article series explaining what IO Completion Ports are and how they can be accessed via C# (i.e. you need to PInvoke into Win32 API calls from the Kernel32.dll).
Note: The libuv the cross platform IO framework behind node.js uses IOCP on Windows and libev on unix operating systems.
http://www.theukwebdesigncompany.com/articles/iocp-thread-pooling.php
i am wondering nobody mentioned kayak it's basicly C#s answer to Pythons twisted, JavaScripts node.js or Rubys eventmachine
I've been fiddling with my own simple implementation of such an architecture and I've put it up on github. I'm doing it more as a learning thing. But it's been a lot of fun and I think I'll flush it out more.
It's very alpha, so it's liable to change, but the code looks a little like this:
//Start the event loop.
EventLoop.Start(() => {
//Create a Hello World server on port 1337.
Server.Create((req, res) => {
res.Write("<h1>Hello World</h1>");
}).Listen("http://*:1337");
});
More information about it can be found here.
I developed a server based on HttpListener and an event loop, supporting MVC, WebApi and routing. For what i have seen the performances are far better than standard IIS+MVC, for the MVCMusicStore i moved from 100 requests per seconds and 100% CPU to 350 with 30% CPU.
If anybody would give it a try i am struggling for feedbacks!
Actually is present a template to create websites based on this structure.
Note that I DON'T USE ASYNC/AWAIT until absolutely necessary. The only tasks i use there are the ones for the I/O bound operations like writing on the socket or reading files.
PS any suggestion or correction is welcome!
Documentation
MvcMusicStore sample port on Node.Cs
Packages on Nuget
you can this framework SignalR
and this Blog about it
Some kind of the support from operating system is essential here. For example, Mono uses epoll on Linux with asynchronous I/O, so it should scale really well (still thread pool). If you are looking and performance and scalability, definitely try it.
On the other hand, the example of C# (with native libs) webserver which is based around idea you have mentioned can be Manos de Mono. Project has not been active lately; however, idea and code is generally available. Read this (especially the "A closer look at Manos" part).
Edit:
If you just want to have callback fired on your main thread, you can do a little abuse of existing synchronization contexts like the WPF dispatcher. Your code, translated to this approach:
using System;
using System.IO;
using System.Threading;
using System.Windows;
namespace Node
{
class Program
{
public static void Main()
{
var app = new Application();
app.Startup += ServerStart;
app.Run();
}
private static void ServerStart(object sender, StartupEventArgs e)
{
var dispatcher = ((Application) sender).Dispatcher;
Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
var path = Environment.ExpandEnvironmentVariables(
#"%SystemRoot%\Notepad.exe");
var fs = new FileStream(path, FileMode.Open,
FileAccess.Read, FileShare.ReadWrite, 1024 * 4, true);
var bytes = new byte[1024];
fs.BeginRead(bytes, 0, bytes.Length, ar =>
{
dispatcher.BeginInvoke(new Action(() =>
{
var res = fs.EndRead(ar);
// Are we in the same thread?
Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
}));
}, null);
}
}
}
prints what you wish. Plus you can set priorities with dispatcher. But agree, this is ugly, hacky and I do not know why I would do it that way for another reason than answer your demo request ;)
First about SynchronizationContext. It's just like Sam wrote. Base class won't give You single-thread functionality. You probably got that idea from WindowsFormsSynchronizationContext which provides functionality to execute code on UI thread.
You can read more here
I've written a piece of code that works with ThreadPool parameters. (Again something Sam already pointed out).
This code registers 3 asynchronous actions to be executed on free thread. They run in parallel until one of them changes ThreadPool parameters. Then each action is executed on the same thread.
It only proves that you can force .net app to use one thread.
Real implementation of web server that would receive and process calls on only one thread is something entirely different :).
Here's the code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.IO;
namespace SingleThreadTest
{
class Program
{
class TestState
{
internal string ID { get; set; }
internal int Count { get; set; }
internal int ChangeCount { get; set; }
}
static ManualResetEvent s_event = new ManualResetEvent(false);
static void Main(string[] args)
{
Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
int nWorkerThreads;
int nCompletionPortThreads;
ThreadPool.GetMaxThreads(out nWorkerThreads, out nCompletionPortThreads);
Console.WriteLine(String.Format("Max Workers: {0} Ports: {1}",nWorkerThreads,nCompletionPortThreads));
ThreadPool.GetMinThreads(out nWorkerThreads, out nCompletionPortThreads);
Console.WriteLine(String.Format("Min Workers: {0} Ports: {1}",nWorkerThreads,nCompletionPortThreads));
ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), new TestState() { ID = "A ", Count = 10, ChangeCount = 0 });
ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), new TestState() { ID = " B ", Count = 10, ChangeCount = 5 });
ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), new TestState() { ID = " C", Count = 10, ChangeCount = 0 });
s_event.WaitOne();
Console.WriteLine("Press enter...");
Console.In.ReadLine();
}
static void LetsRunLikeCrazy(object o)
{
if (s_event.WaitOne(0))
{
return;
}
TestState oState = o as TestState;
if (oState != null)
{
// Are we in the same thread?
Console.WriteLine(String.Format("Hello. Start id: {0} in thread: {1}",oState.ID, Thread.CurrentThread.ManagedThreadId));
Thread.Sleep(1000);
oState.Count -= 1;
if (oState.ChangeCount == oState.Count)
{
int nWorkerThreads = 1;
int nCompletionPortThreads = 1;
ThreadPool.SetMinThreads(nWorkerThreads, nCompletionPortThreads);
ThreadPool.SetMaxThreads(nWorkerThreads, nCompletionPortThreads);
ThreadPool.GetMaxThreads(out nWorkerThreads, out nCompletionPortThreads);
Console.WriteLine(String.Format("New Max Workers: {0} Ports: {1}", nWorkerThreads, nCompletionPortThreads));
ThreadPool.GetMinThreads(out nWorkerThreads, out nCompletionPortThreads);
Console.WriteLine(String.Format("New Min Workers: {0} Ports: {1}", nWorkerThreads, nCompletionPortThreads));
}
if (oState.Count > 0)
{
Console.WriteLine(String.Format("Hello. End id: {0} in thread: {1}", oState.ID, Thread.CurrentThread.ManagedThreadId));
ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), oState);
}
else
{
Console.WriteLine(String.Format("Hello. End id: {0} in thread: {1}", oState.ID, Thread.CurrentThread.ManagedThreadId));
s_event.Set();
}
}
else
{
Console.WriteLine("Error !!!");
s_event.Set();
}
}
}
}
LibuvSharp is a wrapper for libuv, which is used in the node.js project for async IO. BUt it only contains only low level TCP/UDP/Pipe/Timer functionality. And it will stay like that, writing a webserver on top of it is an entire different story. It doesn't even support dns resolving, since this is just a protocol on top of udp.
I believe it's possible, here is an open-source example written in VB.NET and C#:
https://github.com/perrybutler/dotnetsockets/
It uses Event-based Asynchronous Pattern (EAP), IAsyncResult Pattern and thread pool (IOCP). It will serialize/marshal the messages (messages can be any native object such as a class instance) into binary packets, transfer the packets over TCP, and then deserialize/unmarshal the packets at the receiving end so you get your native object to work with. This part is somewhat like Protobuf or RPC.
It was originally developed as a "netcode" for real-time multiplayer gaming, but it can serve many purposes. Unfortunately I never got around to using it. Maybe someone else will.
The source code has a lot of comments so it should be easy to follow. Enjoy!
Here is one more implementation of the event-loop web server called SingleSand. It executes all custom logic inside single-threaded event loop but the web server is hosted in asp.net.
Answering the question, it is generally not possible to run a pure single threaded app because of .NET multi-threaded nature. There are some activities that run in separate threads and developer cannot change their behavior.

in c#.net how to send message to remote computer throught internet?

c#.net framework 4.0 client profile,Windows application..
i am developing a game which needs to send its current movements of the game through internet to remote computer where the same application(game) is installed.In Same way current movements of the game of remote computer should be send back...
How this could be possible ?
All the answers so far are using a TCP based approach. If you need high performance and low latency then you might find it better to use UDP instead.
TCP brings a lot of overhead with it to guarantee that packets will be resent if they are lost (and various other bits of functionality). UDP on the other hand leaves it up to you to deal with packets not arriving. If you have a game where losing the odd update isn't important you can achieve far better bandwidth use, latency and scalability by using UDP instead of TCP.
UDP still leaves you with all the issues of firewalls, security etc though.
If you need to have it work without worrying about firewalls being an issue then you want to choose a solution that uses HTTP over port 80.
To do that you need to implement a client-server behavior through TCP/IP
There are very different ways to do this
This code I've written could give you a start (it's an option, but not the only one, I leave it off to you to choose the method that suits you best)
using System.Runtime.Remoting;
using System.Runtime.Remoting.Channels;
using System.Runtime.Remoting.Channels.Tcp;
static class ServerProgram
{
[STAThread]
static void Main()
{
ATSServer();
}
static void ATSServer()
{
TcpChannel tcpChannel = new TcpChannel(7000);
ChannelServices.RegisterChannel(tcpChannel);
Type commonInterfaceType = Type.GetType("ATSRemoteControl");
RemotingConfiguration.RegisterWellKnownServiceType(commonInterfaceType,
"RemoteATSServer", WellKnownObjectMode.SingleCall);
}
}
public interface ATSRemoteControlInterface
{
string yourRemoteMethod(string parameter);
}
public class ATSRemoteControl : MarshalByRefObject, ATSRemoteControlInterface
{
public string yourRemoteMethod(string GamerMovementParameter)
{
string returnStatus = "GAME MOVEMENT LAUNCHED";
Console.WriteLine("Enquiry for {0}", GamerMovementParameter);
Console.WriteLine("Sending back status: {0}", returnStatus);
return returnStatus;
}
}
class ATSLauncherClient
{
static ATSRemoteControlInterface remoteObject;
public static void RegisterServerConnection()
{
TcpChannel tcpChannel = new TcpChannel();
ChannelServices.RegisterChannel(tcpChannel);
Type requiredType = typeof(ATSRemoteControlInterface);
//HERE YOU ADJUST THE REMOTE TCP/IP ADDRESS
//IMPLEMENT RETRIEVAL PROGRAMATICALLY RATHER THAN HARDCODING
remoteObject = (ATSRemoteControlInterface)Activator.GetObject(requiredType,
"tcp://localhost:7000/RemoteATSServer");
string s = "";
s = remoteObject.yourRemoteMethod("GamerMovement");
}
public static void Launch(String GamerMovementParameter)
{
remoteObject.yourRemoteMethod(GamerMovementParameter);
}
}
Hope this Helps.
You should look into some middleware teknologies like WCF, Web service
this is object oriented and easy to develop when you first get the hang of it
You have a lot to consider for this.
You will need to think about security, firewall issues etc.
If that is all put to one side, then you can set up a tcp socket server / client approach.
A quick google will yield plenty of examples.
Check out the Microsoft example http://msdn.microsoft.com/en-us/library/system.net.sockets.socket.aspx
What have you tried?
You can use the System.Net and System.Net.Sockets namespaces to send TCP packets.

Categories