I am doing an embedded project in C# and I have written a socket based web server. Everything works well, except I can't for the life of me get the request body. Content-Length says that there is 12 characters, but the socket.Recieve method only gets the headers.
while (true)
{
using (Socket clientSocket = listeningSocket.Accept())
{
IPEndPoint clientIP = clientSocket.RemoteEndPoint as IPEndPoint;
Debug.Print("Received request from " + clientIP.ToString());
var x = clientSocket.RemoteEndPoint;
int availableBytes = clientSocket.Available;
Debug.Print(DateTime.Now.ToString() + " " + availableBytes.ToString() + " request bytes available");
int bytesReceived = (availableBytes > maxRequestSize ? maxRequestSize : availableBytes);
if (bytesReceived > 0)
{
byte[] buffer = new byte[bytesReceived]; // Buffer probably should be larger than this.
int readByteCount = clientSocket.Receive(buffer, bytesReceived, SocketFlags.None);
using (Request r = new Request(clientSocket, Encoding.UTF8.GetChars(buffer)))
{
Debug.Print(DateTime.Now.ToString() + " " + r.URL);
if (requestReceived != null) requestReceived(r);
}
}
}
Thread.Sleep(10);
}
availableBytes = 499
bytesReceived = 499
readByteCount = 487 (12 characters short)
What am I missing here? The body is multipart form data if that makes any difference.
int bytesReceived = (availableBytes > maxRequestSize ? maxRequestSize : availableBytes);
If maxRequestSize is 487, then you will get the results you describe.
Also remember Content-Length is not bytes - it's octets: What's the "Content-Length" field in HTTP header? (OK I'm being pedantic - and octet is 8 bits ;))
I wonder how reliable that Available property is. It looks like it's only useful for non-blocking sockets, and then only as a boolean flag saying that something is available.
Why not just read the stream in a loop parsing messages as you go?
Related
Im currently creating a project that will run in a browser & has an c# server connected.
The Server uses an TcpListener to accept connections & receive messages, but I want the server to be able to respond to the client. This has given me a few issues.
Here is the code for my client:
private ClientWebSocket socket;
internal async Task InitAsync(string host, int port, GamePacketParser parser)
{
Logger.Info("Setting up the socket connection...");
socket = new ClientWebSocket();
await socket.ConnectAsync(new Uri($"ws://{host}:{port}/"), CancellationToken.None);
Logger.Info("Successfully established the connection.");
this.parser = parser;
buffer = new byte[GameSocketManagerStatics.BUFFER_SIZE];
Task.Run(recieve);
}
private async Task recieve()
{
Logger.Debug("Starting Reciever.....");
var result = await socket.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);
var packet = new byte[result.Count];
Array.Copy(buffer, packet, result.Count);
///parser.handlePacketData(packet);
Logger.Debug($"Recieved: {Encoding.UTF8.GetString(packet)}");
///Task.Run(recieve); //Start receiving again
}
public async Task SendData(byte[] data)
{
Logger.Debug("Triggerd send");
string packet = BitConverter.ToString(data);
await socket.SendAsync(new ArraySegment<byte>(data), WebSocketMessageType.Text, true, CancellationToken.None);
Logger.Info($"Sended Data: {packet}");
}
The code above simply connects to the server over a web socket. Sending packets works fine. The second the server sends data back, the client won't send any data anymore to the server. Like its stuck.
static void Main(string[] args)
{
string ip = "127.0.0.1";
int port = 30000;
var server = new TcpListener(IPAddress.Parse(ip), port);
server.Start();
Console.WriteLine("Server has started on {0}:{1}, Waiting for a connection...", ip, port);
TcpClient client = server.AcceptTcpClient();
Console.WriteLine("A client connected.");
NetworkStream stream = client.GetStream();
if (Regex.IsMatch(s, "^GET", RegexOptions.IgnoreCase))
{
Console.WriteLine("=====Handshaking from client=====\n{0}", s);
// 1. Obtain the value of the "Sec-WebSocket-Key" request header without any leading or trailing whitespace
// 2. Concatenate it with "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" (a special GUID specified by RFC 6455)
// 3. Compute SHA-1 and Base64 hash of the new value
// 4. Write the hash back as the value of "Sec-WebSocket-Accept" response header in an HTTP response
string swk = Regex.Match(s, "Sec-WebSocket-Key: (.*)").Groups[1].Value.Trim();
string swka = swk + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
byte[] swkaSha1 = System.Security.Cryptography.SHA1.Create().ComputeHash(Encoding.UTF8.GetBytes(swka));
string swkaSha1Base64 = Convert.ToBase64String(swkaSha1);
// HTTP/1.1 defines the sequence CR LF as the end-of-line marker
byte[] response = Encoding.UTF8.GetBytes(
"HTTP/1.1 101 Switching Protocols\r\n" +
"Connection: Upgrade\r\n" +
"Upgrade: websocket\r\n" +
"Sec-WebSocket-Accept: " + swkaSha1Base64 + "\r\n\r\n");
stream.Write(response, 0, response.Length);
}
byte[] message = Encoding.UTF8.GetBytes("Connection is established");
stream.Write(message, 0, message.Length);
}
The problem is probably because it is not encoded for WebSockets, but I tried a lot of online solutions for encoding (For example: How can I send and receive WebSocket messages on the server side?) But even with those encoders, it did not seem to solve the problem.
Thanks for your help in advance. Im still new to WebSockets, so spoonfeeding is allowed.
aepot your answer is a good one, but i really wanted my server on the TCP level, I would have needed to change to much code if I wanted to use it on my official server that uses sockets.
I have been doing some more digging into WebSockets, after some searching I figured it out, I basically needed to send a header before sending the data. I did not know how to create that header, but I found some code online that did. (I have been searching for about 12 hours :?)
The solution:
protected int GetHeader(bool finalFrame, bool contFrame)
{
int header = finalFrame ? 1 : 0;//fin: 0 = more frames, 1 = final frame
header = (header << 1) + 0;//rsv1
header = (header << 1) + 0;//rsv2
header = (header << 1) + 0;//rsv3
header = (header << 4) + (contFrame ? 0 : 1);//opcode : 0 = continuation frame, 1 = text
header = (header << 1) + 0;//mask: server -> client = no mask
return header;
}
protected byte[] IntToByteArray(ushort value)
{
var ary = BitConverter.GetBytes(value);
if (BitConverter.IsLittleEndian)
{
Array.Reverse(ary);
}
return ary;
}
public static IEnumerable<string> SplitInGroups(this string original, int size)
{
var p = 0;
var l = original.Length;
while (l - p > size)
{
yield return original.Substring(p, size);
p += size;
}
yield return original.Substring(p);
}
public static void SendMessage(string packet) {
Queue<string> que = new Queue<string>(packet.SplitInGroups(125)); //Make it so the message is never longer then 125 (Split the message into parts & store them in a queue)
int len = que.Count;
while (que.Count > 0)
{
var header = GetHeader(
que.Count > 1 ? false : true,
que.Count == len ? false : true
); //Get the header for a part of the queue
byte[] list = Encoding.UTF8.GetBytes(qui.Dequeue()); //Get part of the message out of the queue
header = (header << 7) + list.Length; //Add the length of the part we are going to send
//Send the header & message to client
stream.write(IntToByteArray((ushort)header));
stream.write(list);
}
}
I have not coded this myself, but sadly I cant find the link where I got it from to credit the person who did.
I've written my own HTTP server to embed in a desktop application and it works (spins up a server on localhost), except that I'm getting a strange request on the socket that is causing the connection to be closed so that no new requests are being handled. The request is occuring about 15 seconds after I open the web page. The packet contains a long series of \0\0\0\0\0\0\0\0\0\0\0\0...... when I examine it. I have no idea what's causing this or how to handle it. For those first 15 seconds, I'm able to do everything I need to do, but once this happens, no new requests can be made and the server won't respond to any new requests. It also doesn't happen every time I launch the application but I can't pin down why.
while (true)
{
//Accept a new connection
Socket mySocket = _listener.AcceptSocket();
if (mySocket.Connected)
{
Byte[] bReceive = new Byte[1024];
int i = mySocket.Receive(bReceive, bReceive.Length, 0);
string sBuffer = Encoding.ASCII.GetString(bReceive);
if (sBuffer.Substring(0, 3) != "GET")
{
Console.WriteLine(sBuffer);
mySocket.Close();
return;
}
........handle valid requests
}
}
Try something like this:
while (true)
{
//Accept a new connection
Socket mySocket = _listener.AcceptSocket();
if (mySocket.Connected)
{
Byte[] bReceive = new Byte[1024];
int i = mySocket.Receive(bReceive, bReceive.Length, 0);
if(i > 0) // added check to make sure data is received
{
string sBuffer = Encoding.ASCII.GetString(bReceive, 0, i); // added index and count
if (sBuffer.Substring(0, 3) != "GET")
{
Console.WriteLine(sBuffer);
mySocket.Close();
return;
}
........handle valid requests
}
}
}
Ultimately - you will need to do something if i == 1024 - because if it does, there is more data than you are reading in mySocket.Receive.
As a side note - I might change
if (sBuffer.Substring(0, 3) != "GET")
to
if (sBuffer.StartsWith("GET"))
It's slightly easier to read - and has the added benefit of not requiring a change to the substring length if (for some odd reason) GET changed to something else.
Edit - this will allow for multiple calls to mySocket.Receive when more data than 1024 bytes is encountered:
while (true)
{
//Accept a new connection
Socket mySocket = _listener.AcceptSocket();
if (mySocket.Connected)
{
StringBuilder sb = new StringBuilder();
Byte[] bReceive = new Byte[1024];
int i;
while ((i = mySocket.Receive(bReceive, bReceive.Length, 0) > 0))
{
sb.Append(Encoding.ASCII.GetString(bReceive, 0, i)); // added index and count
}
string sBuffer = sb.ToString();
if (sBuffer.Substring(0, 3) != "GET")
{
Console.WriteLine(sBuffer);
mySocket.Close();
return;
}
}
}
Good afternoon. I apologize for my English as itself from Ukraine and speak badly)) I have the following problem, my program makes requests on different urls and then parse some info from answers. Number of urls are more than several millions. To quickly process I use a lot of threads, sometimes about 500-700 threads. On some machines program is running well, but there are those on which errors occur. Errors like: System.Net.Sockets.SocketException (0x80004005): The remote host forcibly broke the existing connection.
My code:
void _thread()
{
while(true)
{
string request =
"POST http://" + hostf + "/ HTTP/1.1\r\n" +
"Host: " + host +
"\r\nConnection: Close\r\n" +
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n" +
"Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3\r\n" +
"Content-Length: " + ByteArr.Length +
"\r\nContent-Type: application/x-www-form-urlencoded; charset=UTF-8\r\n\r\n" +
parametres;
Byte[] bytesSent = Encoding.GetEncoding("UTF-8").GetBytes(request);
Byte[] bytesReceived = new Byte[256];
Socket s = null;
IPHostEntry hostEntry = null;
hostEntry = Dns.GetHostEntry(host);
foreach (IPAddress address in hostEntry.AddressList)
{
IPEndPoint ipe = new IPEndPoint(address, 80);
Socket tempSocket =new Socket(ipe.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
tempSocket.Connect(ipe);
if (tempSocket.Connected)
{
s = tempSocket;
break;
}
else
{
continue;
}
}
if (s == null)
continue;
s.Send(bytesSent, bytesSent.Length, 0);
int bytes = 0;
string page = "";
do
{
bytes = s.Receive(bytesReceived, bytesReceived.Length, 0);
page = page + Encoding.GetEncoding("UTF-8").GetString(bytesReceived, 0, bytes);
}
while (bytes > 0);
s.Shutdown(SocketShutdown.Both);
s.Close();
//here some work whith page content
}
}
As you see, each thread creates socket, sends request, then get answer and closes socket and so on with each iteration. Each thread opens its own socket and works with different urls, but on some machines when numbers of threads are more than some number, the errors begin and all sockets don't work normally. Could someone help me with some advise, why does it happen? Some machines have some kind of limits on connection or what? Thank you all in advance.
Dont do the shutdown(both); followed by a close. Remove the s.shutdown() and leave the s.Close() then try. I think I remeber that shutdown both makes the socket descriptor available for use so in the next close you can be closing some other socket instead the one you owned.
EDIT: some code modification:
I'd increment a little bit the receive buffer
bytes[] bytesReceived = new bytes[1024];
Also when sending the request string tell the reciver you are done:
s.Send(bytesSent, bytesSent.Length, 0);
// Tell the receiver we are done sending data
s.Shutdown(SocketShutdown.Send);
You also have to check for error when reading the socket and use a StringBuilder instead of a String (it's somehow faster than String appending text):
StringBuilder page = new StringBuilder();
do
{
bytes = s.Receive(bytesReceived, bytesReceived.Length, 0);
if (bytes == -1)
{
// Error in socket, quit
s.Close();
return;
}
else if (bytes > 0)
page.Append(Encoding.GetEncoding("UTF-8").GetString(bytesReceived, 0, bytes));
}
while (bytes > 0);
At the end, just close the socket:
// s.ShutDown(Socketshutdown.Both);
s.close();
You may try with this modifications and see if it is solved.
I developed a simple TCP client on Windows Phone, as shown here on MSDN
This is working just as expected.
Now, I want to send very large Base64 strings though this client (for transferring images).
But, when I try to send Base64 strings from this client, I only receive a portion of the string at the server, due to which I'm not able to generate the entire image at the server.
The server side code for receiving strings is: (Edited)
IPAddress ipAd = IPAddress.Any;
Console.Write("Port No. (leave blank for port 8001): ");
string port;
port = Console.ReadLine();
if (port == "")
port = "8001";
/* Initializes the Listener */
TcpListener myList = new TcpListener(ipAd, int.Parse(port));
/* Start Listeneting at the specified port */
myList.Start();
Console.WriteLine("\nThe server is running at port " + port);
Console.WriteLine("The local End point is :" +
myList.LocalEndpoint);
Console.WriteLine("\nWaiting for a connection.....");
Socket s = myList.AcceptSocket();
Console.WriteLine("\nConnection accepted from " + s.RemoteEndPoint);
byte[] b = new byte[5 * 1024 * 1024]; // BIG SIZE for byte array, is this correct?
String message = String.Empty;
int k = s.Receive(b);
Console.WriteLine("\nRecieved...");
for (int i = 0; i < k; i++)
{
message += Convert.ToChar(b[i]);
Console.Write(Convert.ToChar(b[i]));
}
System.IO.File.WriteAllText(#"Message.txt", message); // write it to a file
ASCIIEncoding asen = new ASCIIEncoding();
s.Send(asen.GetBytes("The string was recieved by the server."));
Console.WriteLine("\n\nSent Acknowledgement");
/* clean up */
s.Close();
myList.Stop();
I'm really stuck here.
Please help me.
I think the problem is with the client and not the server.
Please assist me.
The class I've used in the client can be found at the MSDN article referred to above.
PS: I've already tried to increase the values of TIMEOUT_MILLISECONDS and MAX_BUFFER_SIZE in the class. But it did not help.
Update:
Here's some client side code (look here on MSDN for reference):
// Make sure we can perform this action with valid data
if (ValidateRemoteHost() && ValidateInput())
{
// Instantiate the SocketClient
SocketClient client = new SocketClient();
// Attempt to connect to the echo server
Log(String.Format("Connecting to server '{0}' over port {1} (echo) ...", txtRemoteHost.Text, ECHO_PORT), true);
string result = client.Connect(txtRemoteHost.Text, ECHO_PORT);
Log(result, false);
byte[] bytearray = null;
// Attempt to send our message to be echoed to the echo server
Log(String.Format("Sending '{0}' to server ...", txtInput.Text), true);
if (checkBox1.IsChecked == true) // this checkbox is for image selection
{
// This is the part where we send images
using (MemoryStream ms = new MemoryStream())
{
WriteableBitmap wbitmp = new WriteableBitmap((BitmapImage)image1.Source);
wbitmp.SaveJpeg(ms, (int)wbitmp.PixelWidth, (int)wbitmp.PixelHeight, 0, 10);
bytearray = ms.ToArray();
string str = Convert.ToBase64String(bytearray);
result = client.Send(str);
System.Diagnostics.Debug.WriteLine("\n\nMessge sent:\n\n" + str + "\n\n");
}
}
else
{
result = client.Send(txtInput.Text);
}
Log(result, false);
// Receive a response from the server
Log("Requesting Receive ...", true);
result = client.Receive();
Log(result, false);
// Close the socket connection explicitly
client.Close();
}
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0)
{
Fs.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
Edit: now that section of code has been reduced down to only
int k = s.Receive(b);
is bad: it assumes all the data sends perfectly in one go, which isn't how networking works.
Two options:
At the start of the string, include how long it should.
At the end of the string, have an end symbol (a null perhaps) that won't be anywhere else in the message
Then that while loop should keep going until either the entire length is found or the end symbol is found.
(Also, sending Base64 is a bad idea when you can avoid it. Why not send as a stream of raw bytes?)
[ED: This portion is no longer relevant](Also, why is the server choosing where to save the file [and delaying everything until the server makes a pick] - the client should indicate where to save it at the start, and then the server merely sanity checks it. [Unless you have a very good reason not do it that way])
EDIT: a quick simple implementation of what I was saying:
This server code
int k = s.Receive(b);
Console.WriteLine("\nRecieved...");
for (int i = 0; i < k; i++)
{
message += Convert.ToChar(b[i]);
Console.Write(Convert.ToChar(b[i]));
}
change to
while (true)
{
int k = s.Receive(b);
Console.WriteLine("\nRecieved...");
for (int i = 0; i < k; i++)
{
char bc = Convert.ToChar(b[i]); // This is a very wrong way of doing this but it works I guess meh.
if (bc == ' ')
{ // You've struck the end! Get out of this infinite loop!
goto endmyloop;
}
message += bc;
Console.Write(bc);
}
}
endmyloop:
This piece of client code
result = client.Send(str);
change to
result = client.Send(str + " ");
-- Base64 can never have a space in it, so that will be used to mark the end.
Be warned that if the client errors (and doesn't send a space at the end for some weird reason), this code will be trapped in the while loop forever (zero-CPU-usage infinite-wait)
I am working on C# MailClient and which is follow IMAP Protocol, but I am getting wrong response that means the response will give me some repetitive resul.
Like say first time I am sending command like this.
byte[] commandBytes = System.Text.Encoding.ASCII.GetBytes(("$ UID FETCH " + index + " (BODY[HEADER.FIELDS (SUBJECT FROM DATE)])\r\n"));
and second time I am sending like this.
byte[] commandBytes = System.Text.Encoding.ASCII.GetBytes(("$ UID FETCH " + index + " (BODYSTRUCTURE)" + "\r\n"));
so I am getting again first command result in twice or more than 2 times;
and some times it's continue giving me first result.
my Response() method is like this.
private string Response()
{
string response = string.Empty;
byte[] data = new byte[_imapClient.ReceiveBufferSize];
int ret = _imapNs.Read(data, 0, data.Length);
response = Encoding.ASCII.GetString(data,0,ret);
return response;
}
_imapClient is a object of an
private TcpClient _imapClient;
and I am taking value of _imapClient is like this.
public string GetMessageBodyStructure(int index)
{
byte[] commandBytes = System.Text.Encoding.ASCII.GetBytes(("$ UID FETCH " + index + " (BODYSTRUCTURE)" + "\r\n"));
_imapNs.Write(commandBytes, 0, commandBytes.Length);
_imapNs.Flush();
return Response();
}
Where I am wrong correct me thanks..
Do not use char array:
byte[] commandBytes = System.Text.Encoding.ASCII.GetBytes("$ UID FETCH " + index + " (BODYSTRUCTURE)" + "\r\n");
TCP is stream based and not message based.
This means that nothing guarantees that the entire message is received with the same Read. A Read may receive a half message, a complete message or two messages. You need to handle that accordingly.
3 Use return Encoding.ASCII.GetString(data, 0, ret);
4 There are several open source IMAP libraries out there. Why not use one of those since you are new to socket programming?