I have a situation where I am running an interactive C# console-program from node/express. The program runs in an infinite loop, accepts a string from the command-line, and echoes it back.
The following code works for the first time I call http://localhost:3000?command=hello
Next time around, Node crashes by reporting Can't set headers after they are sent.
If the move the const script = spawn('/Users/amarshanand/shadowClient/myscript.sh'); in the sendToShell(), it works, but since I have to start a new shell and the script, it takes a lot longer.
How can I make it work like start once and accept command for each request.
const express = require('express')
const app = express()
const { spawn } = require('child_process');
const script = spawn('/Users/amarshanand/shadowClient/myscript.sh');
const sendToShell = (command, done) => {
script.stdout.on('data', (stdout) => {
console.log(`stdout: ${stdout}`);
done(stdout);
});
script.stderr.on('data', (stderr) => {
console.log(`error: ${stderr}`);
});
script.stdin.write(`${command}\n`);
}
app.get('/', (req, res) => {
sendToShell(req.query.command, result => res.send(`${result}`));
})
app.get('/getstate', (req, res) => {
res.send('state');
})
app.post('/setstate:state', (req, res) => res.send('posted state'));
app.listen(3000, () => console.log('Example app listening on port 3000!'))
That particular error occurs when you try to send more than one response to an incoming request. When I examine your code, I see that this particular piece of code:
script.stdout.on('data', (stdout) => {
console.log(`stdout: ${stdout}`);
done(stdout);
});
Can receive the data event more than once and when it does, it will call done(stdout) more than once which will cause the caller to call res.send() more than once.
With streams, you have no idea how many times the data event will be called. It could be called only once or it could be called many times with lots of small pieces of data.
In addition, you only have one script that all your requests use. So, each time you call sendToShell(), you add yet another script.stdout.on('data', ...) event handler so they will pile up and you will have duplicates causing you to call done() more than once for each data event. If you're going to stick with this structure, then you need a way to know when all the data has been sent for the last command and then you need to remove that event handler so they don't pile up.
FYI, this code also has concurrency issues because multiple requests could come into your server that cause you to run a command and you'd have no idea which response belongs with which command. If you're going to keep just one shell open, they you probably need to queue commands to the shell so you don't send the next command or set up its event handlers to read the response until the previous command is done. That way you won't be reading the response from the wrong command.
Related
I have a windows service that uses a websocket (from http://sta.github.io/websocket-sharp/) to conenct to Slack and monitor messages.
My connection code looks something like this:
ws = new WebSocket(connection.Url);
ws.OnMessage += async (sender, e) =>
{
var msg = JsonConvert.DeserializeObject<MessageFromSlack>(e.Data);
if (msg.Type == "message" && msg.Text != null && msg.User != UserId)
{
if (userMatcher.IsMatch(msg.Text))
{
await ProcessDirectMessage(msg);
}
await ProcessMessage(msg);
}
if (msg.Type == "channel_joined")
{
await ChannelJoined(msg.ChannelModel.Id);
}
};
ws.OnClose += (sender, e) =>
{
var reason = e.Reason;
var code = e.Code;
System.Diagnostics.Debug.WriteLine($"{code}:{reason}");
};
ws.Connect();
Basically it waits for a message and then if it's directed # my bot, it'll call ProcessDirectMessage and if not it'll call ProcessMessage. The details of those functions are, I think, unimportant (they do some matching looking for key phrases and respond by sending a message back).
This all works fine. For a while. But after some period of time (usually more than a day), it just stops responding altogether. My OnMessage handler never gets hit. I thought that maybe what is happening is the websocket is getting closed on the server side, so I added the OnClose handler, but that never seems to get hit either.
Does anybody have an idea what might be happening here? Is there a way to keep the connection alive, or else reconnect it when it dies?
By the nature of TCP connection - the only reliable way to detect its gone is to write something to it. If you are just reading (waiting for data to arrive) - you can do that for a very long time while the other side is long time dead. That happens if that other side did not close connection gracefully (which involves an exchange of some TCP packets).
Web socket protocol defines special Ping frame, and corresponding Pong frame, which you should use to avoid situation described in the question. From time to time you should send Ping frame and wait (for a certain timeout) for server to respond with Pong frame. If it did not respond in given timeout - assume connection is dead and reconnect.
As far as I know - library you use does not automatically send ping requests on your behalf. However, it allows you to do that via Ping method.
You need to configure timeout with
ws.WaitTime = TimeSpan.FromSeconds(5);
And then, from time to time (for example - when you did not receive any new messages in last X seconds), do:
bool isAlive = ws.Ping();
There is also boolean property which does the same:
bool isAlive = ws.IsAlive;
This is a blocking call (both of the above). It will return true if server replied with Pong during ws.WaitTime interval, and false otherwise. Then you can act accordingly.
I have the following Actor where I am trying to restart and resend the failing message back to the actor :
public class BuildActor : ReceivePersistentActor
{
public override string PersistenceId => "asdad3333";
private readonly IActorRef _nextActorRef;
public BuildActor(IActorRef nextActorRef)
{
_nextActorRef = nextActorRef;
Command<Workload>(x => Build(x));
RecoverAny(workload =>
{
Console.WriteLine("Recovering");
});
}
public void Build(Workload Workload)
{
var context = Context;
var self = Self;
Persist(Workload, async x =>
{
//after this line executes
//application goes into break mode
//does not execute PreStart or Recover
var workload = await BuildTask(Workload);
_nextActorRef.Tell(workload);
context.Stop(self);
});
}
private Task<Workload> BuildTask(Workload Workload)
{
//works as expected if method made synchronous
return Task.Run(() =>
{
//simulate exception
if (Workload.ShowException)
{
throw new Exception();
}
return Workload;
});
}
protected override void PreRestart(Exception reason, object message)
{
if (message is Workload workload)
{
Console.WriteLine("Prestart");
workload.ShowException = false;
Self.Tell(message);
}
}
}
Inside the success handler of Persist I am trying to simulate an exception being thrown but on exception the application goes in to break mode and PreRestart hook is not invoked. But if I make BuildTask method synchronous by removing Task.Run then on exception both PreRestart and Recover<T> methods are invoked.
I would really appreciated if someone can point to me what should be the recommended pattern for this and where I am going wrong.
Most probably, Akka.Persistence is not the good solution for your problem here.
Akka.Persistence uses eventsourcing principles for storing actor's state. Few key points important in this context:
What you're sending to actor, is a command. It describes a job, you want to be done. Executing that command may result in doing some actual processing and eventually may lead to persist actor's linear state change history in form of the events.
In Akka.NET Persist method is used only to store events - they describe the fact, that something has happened: because of that, they cannot be denied and they cannot fail (a thing that you're doing in your Persist callback).
When an actor restarts at any point in time, it will always try to recreate its own state by replaying all events Persisted up to the last known point in time. For this reason it's important that Recover method should only focus on replaying actor's state (it can be called multiple times over the same event) and never result in side effects (example of side effect is sending an email). Any exception thrown there will mean, that actor state is irrecoverably corrupted and that actor will be killed.
If you want to resend the message to your actor, you could:
Put a reliable message queue (i.e. RabbitMQ or Azure Service Bus) or log (Kafka or Event Hub) in front of your actor processing pipeline. This is actually the most reasonable scenario in many cases.
Use at-least-once delivery semantics from Akka.Persistence - but IMHO only if for some reason you cannot use 1st solution.
The most simplistic and unreliable option (since messages are residing only in memory and never persisted) is dead letter queue. Every unhandled message is send there. You can subscribe to it and filter the incoming data to detect which messages should be send again to their recipients.
EDIT: I've edited a few lines of code, when running in the IDE it fails without an error or anything.
I'm new to Reactive Extensions and have a problem that I am trying to sort out. I'm using RX to queue events on a machine, then every so often send that data to a server. My problem appears to be that when the application is shutting down, anything that is an async call of any sort seems to just cancel and not run, thus the last batch of events never gets sent.
I have a Subject, where Event is my data class. I know now that a Subject might not be the best class to use, but here we are.
My code looks mostly like the following, added a few comments for clarity:
IObservable<IList<Event>> eventsObserver = Instance.EventBuffer.ToList<Event>();
var eventsEnumerable = eventsObserver.ToEnumerable();
List<Event> events = new List<Event>();
try
{
events = (List<Event>)eventsEnumerable.First(); // THIS LINE FAILS SILENTLY, EVEN IN DEBUGGER...
}
catch(Exception ex)
{
System.Diagnostics.Debug.WriteLine("Error: " + ex.Message);
}
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(someURI);
HttpResponseMessage response = client.PostAsync(somePage, new StringContent(SerializeToJSON(events))).Result;
response.EnsureSuccessStatusCode();
}
If I don't make the call to the web server synchronous (with the '.Result'), it fails there. I've tried a lot of ways to get the data from the IObservable into something I can send, but either the code fails (usually with a bad cast of some sort), or the events are not yet put into the data structure that I want to send. I know that RX is by nature async, and I'm asking to deal with it in a synchronous way, I would figure that there would be a solution. Thanks in advance!
Supposing that you control the Observable source, you could call Observable.OnComplete() like Enigmativity has pointed out. Otherwise, you could try to keep a copy of every value received before buffering it:
Observable.Do(x => localCopy = x).Buffer(..)
This local copy would be accessible to you at shutdown.
In any case, please note that .First() is marked obsolete in the latest Rx versions, possibly to avoid the problem you are experiencing.
I wrote a class, that should provide basic interaction with NodeJS server. This class also includes TcpClient instance and works asynchronously (methods like BeginRead, BeginWrite, ...) so many functions takes callbacks (anonymous function using lambda expr.) as an argument. Problem starts in this scenario (I try to illustrate on a peace of code similar to original).
this._nodeJS.Send("DATA_TO_SEND", (Response) => {
Console.WriteLine(Response);
// Section1
this._nodeJS.Send("ANOTHER_DATA", (Response1) => {
Console.WriteLine(Response1);
});
});
Problem occurs in Section1. When it's commented out, program behaves as expected. Response is received a written to Console. When it's not commented out, it starts to behave very strange. Write to Console in Section1 doesn't work (in many cases) and, what is important, code below Console.WriteLine in this section is not executed, the "thread" just stops right before command. I tried to debug it, step by step, variables in Section1 are correctly set, and when I get to the next command, debugging just stops. Is there anything wrong with this approach? Is there some things I need to be aware of?
For better imagination here is my example. I connect to the NodeJS server, when it's done, I send a request and wait for response containing data, that are processed and again sent to the NodeJS server in next request (Section1). NodeJS give me final response and program works ... Thanks for any advice!
Finally, I figured out, what's goin' on. I know, I din't show my original code and instead used some kind of pseudo-code, but if someone is interested to know, problem, probably, was causing wrong conversion of received data. I have buffer of size 1024 and StringBuilder variable where the data was appended. So when I received the data, converted it this way: Encoding.UTF8.GetString(state.Buffer) - this is wrong, instead, I had to use Encoding.UTF8.GetString(state.Buffer, 0, bytesRead). I don't know how, but this part of code was causing troubles and stopping callback to execute.
Without knowing your implementation details, I would suspect that the second _nodeJS.Send(... call is being blocked by the response from the first Send call.
At a guess, your code does something like this (made up pseudocode!):
void Send(String data, Action<Response> callback)
{
_socket.BeginSend(Magic(data), r => {
callback.Invoke(r.ResponseData);
_socket.EndSend(r.Result); //you cannot call _socket.Send() again until you have done this!
});
}
Due to order of operations, it should be doing this:
void Send(String data, Action<Response> callback)
{
_socket.BeginSend(Magic(data), r => {
_socket.EndSend(r.Result);
callback.Invoke(r.ResponseData);
});
}
I am attempting to improve the stability of the web dashboard I have created. I have noticed that the main cause of getting into a bad state is when the user moves too quickly and requests get cut short.
For instance, I have the following javascript:
//When a resize occurs a lot of handling needs to happen to convert the user's action into
//a visibly-pleasing result. In addition, the server has to be spoken with in order to save the controls.
function OnClientResizing(pane, eventArgs) {
eventArgs.set_cancel(true);
var parameters = new Array();
parameters.push("Resize");
parameters.push(pane.get_id());
parameters.push(eventArgs.get_delta());
__doPostBack(pane.get_splitter()._uniqueID, parameters);
}
This function passes the hard work back to the server, so that it can calculate the appropriate ways to resize the controls on the page during resizes. This takes X seconds. If the user then resizes the page again before X seconds has elapsed -- I enter into a bad state. Either the old request gets cut off prematurely, or the new one runs at the same time. Either way, controls become mishapen on the page.
As such, I would like to queue future resizes, or play around with canceling current requests. I read that the best way to do something like this is to simply set a flag outside the scope of this function. I can do that, but I am not sure how to detect the end of a doPostBack. Am I supposed to change the javascript variable from the server-side somehow in PageRequestManager - EndRequest?
Cheers
First off, don't let your server participate in UI resize algorithms. Do that entirely client side. You can send resulting data to the server at any time, but don't make a real-time UI positioning depend upon a server response. That should be handled client-side with CSS or javascript logic.
Second off, if your code can't handle two ajax calls in flight at the same time, then your options are as follows:
Fix your code so it can handle sequential ajax responses in flight at the same time.
Cancel/ignore the first ajax response the moment you send a second one so that you ignore the response from the first and wait for the response from the second.
Prevent a second ajax request until the first one completes. I wouldn't suggest queueing them because that's just going to lead to an even worse user experience.
The details of how to do 1, 2 or 3 depend upon how your code works which you have not yet shared.
The easiest is option 3). That can be done with just a global flag. Just define a global variable, set it to true when you start an ajax call and clear it when the ajax call completes (in a completion function):
var ajaxInFlight = false; // global declaration
function OnClientResizing(pane, eventArgs) {
if (ajaxInFlight) return; // do nothing if ajax call already in flight
ajaxInFlight = true;
eventArgs.set_cancel(true);
var parameters = new Array();
parameters.push("Resize");
parameters.push(pane.get_id());
parameters.push(eventArgs.get_delta());
__doPostBack(pane.get_splitter()._uniqueID, parameters);
}
function postBackCompletionHandler(id, parms) {
ajaxInFlight = false; // clear global flag, ajax call done
// ... rest of your function here
}
You will also have to make sure that error conditions are handled so that the global flag is reset if the ajax call fails for any reason.