I am trying to create an asynchronous TPL file server using sockets and NetworkStream. When testing it, my browser small HTML file (1.9 KB) sends just fine, and sometimes even Javascript or CSS files that it links send to, but it won't download much more from the HTML page, including flash, images, etc. I receive no errors, including no connection errors. I can download a 96K image but that's about the limit. I set Connection: Keep-Alive in all response headers.
Does anyone know why my output streaming seems to be stalling?
async Task<> WriteToStream(NetworkStream _networkStream, string filePath, int startingPoint = 0)
{
using (FileStream sourceStream = new FileStream(filePath,
FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: 4096, useAsync: true))
{
byte[] buffer = new byte[4096];
int numRead;
while ((numRead = await sourceStream.ReadAsync(buffer, 0, buffer.Length)) != 0)
{
_networkStream.Write(buffer, 0, numRead);
}
}
}
I also tried replacing this:
_networkStream.Write(buffer, 0, numRead);
with this:
await _networkStream.WriteAsync(buffer, 0, numRead);
and I still have the same problem.
I'm using sockets because I can't use HttpListener or TcpListener classes since I need to access incoming UDP and TCP requests.
I can call WriteToStream() with this simplified method:
private async void SendFileExample()
{
//This method is only for demonstration, so parameters are hardcoded.
// Get info and assemble header
string file = #"C:\www\webpage.html";
byte[] data = null;
string responseCode = "200 OK";
string contentType = "text/html";
long dataLength = 1901;
string serverName = "my Stack Overflow server is overflowing with...";
string header = string.Format("HTTP/1.1 {0}\r\n"
+ "Server: {1}\r\n"
+ "Content-Length: {2}\r\n"
+ "Content-Type: {3}\r\n"
+ "Connection: Keep-Alive\r\n"
+ "\r\n",
responseCode, serverName, dataLength, contentType);
var headerBytes = Encoding.ASCII.GetBytes(header);
//send header
await _networkStream.WriteAsync(headerBytes, 0, headerBytes.Length);
//send payload
await WriteToStream(_networkStream, file, 0);
//flush networkstream
await _networkStream.FlushAsync();
}
EDIT:
Here what calls the listen loop:
_listenTask = Task.Factory.StartNew(() => ListenLoop());
Here is the loop that spools the requests, spawning a client each time:
private async void ListenLoop()
{
for (; ; )
{
// Wait for connection
var socket = await _tcpListener.AcceptSocketAsync();
if (socket == null)
break;
// Got new connection, create a client handler for it
var client = new Client(socket,dbInfo,frmClient);
// Create a task to handle new connection
Task.Factory.StartNew(client.Do);
}
}
Connections are handled by this method:
public async void Do()
{
byte[] buffer = new byte[4096];
for (; ; )
{
// Read a chunk of data
int bytesRead = await _networkStream.ReadAsync(buffer, 0, buffer.Length);
// If Read returns with no data then the connection is closed.
if (bytesRead == 0)
return;
// Write to buffer and process request
_memoryStream.Seek(0, SeekOrigin.End);
_memoryStream.Write(buffer, 0, bytesRead);
bool done = ProcessHeader();
if (done)
break;
}
}
ProcessHeader() mostly just gets meta data like MIME types then passes the stream to the WriteToStream() method at the top of this post.
Related
I have a little complication i encounter.
I may not be expert in TCP Connections but i hope someone here would help me.
This is my Client Code:
void Connect(String server, String message)
{
try
{
Int32 port = 8968;
TcpClient client = new TcpClient(server, port);
Byte[] data = File.ReadAllBytes(curSelectedFile);
NetworkStream stream = client.GetStream();
Byte[] fileData = File.ReadAllBytes(curSelectedFile);
Byte[] msgData = Encoding.ASCII.GetBytes("SendFile");
Byte[] sendData = new byte[fileData.Length + msgData.Length];
// Copy data to send package.
msgData.CopyTo(sendData, 0);
fileData.CopyTo(sendData, 4);
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
Console.WriteLine("Sent: {0}", message);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
Console.WriteLine("Received: {0}", responseData);
// Close everything.
stream.Close();
client.Close();
}
catch (ArgumentNullException e)
{
Console.WriteLine("ArgumentNullException: {0}", e);
}
catch (SocketException e)
{
Console.WriteLine("SocketException: {0}", e);
}
Console.WriteLine("\n Press Enter to continue...");
Console.Read();
}
This is my server one:
// Listen loop.
while(true)
{
using (TcpClient tcpClient = myListener.AcceptTcpClient())
{
Console.WriteLine("[Server] Acceptam client.");
using (NetworkStream networkStream = tcpClient.GetStream())
{
// Buffer for reading data
Byte[] bytes = new Byte[1024];
var data = new List<byte>();
int length;
while ((length = networkStream.Read(bytes, 0, bytes.Length)) != 0)
{
var copy = new byte[length];
Array.Copy(bytes, 0, copy, 0, length);
data.AddRange(copy);
}
// Incercam sa vedem ce doreste clientul.
string msg = Encoding.ASCII.GetString(data[0], 0, length);
if(msg.StartsWith("SendFile"))
{
using (Stream stream = new FileStream(#"C:\test.mp3", FileMode.Create, FileAccess.ReadWrite))
{
BinaryFormatter binaryFormatter = new BinaryFormatter();
networkStream.Position = 4;
binaryFormatter.Serialize(networkStream, data.ToArray());
}
}
}
}
}
What i'm trying to do here:
- I want the client to send a Message.. like "SaveFile" & after this string to be the filedata.
- The server should read the client message, and to process stuff according to the Client sentstring, before doing something with the file.
I believe that i don't know how to do it.
May i have an example on how to send/receive and read certain strings from the beggining of the file? How i can put them in the byte array and how to read it... It's quite overwhelming..
PS: The current Server Code is reading the data and CAN write as i coded it, without losing any packages. But also he's writing the aditional packets i sent before i converted the bytes of the file.
networkStream.Position = 4; isn't legal, because NetworkStream is not seekable.
I would discourage you from mixing text and binary data, just because of the complication it makes in the application protocol. But if you really want to do that, you should use BinaryWriter and BinaryReader, because it can write strings to a stream which can then be read later without consuming the bytes after the string.
Then you can do something like this...
In the client:
BinaryWriter writer = new BinaryWriter(networkStream);
writer.Write("SendFile");
writer.Write(fileData.Length);
writer.Write(fileData);
In the server:
BinaryReader reader = new BinaryReader(networkStream);
switch (reader.ReadString())
{
case "SendFile":
{
int length = reader.ReadInt32();
byte[] fileData = reader.ReadBytes(length);
// ... then do whatever with fileData, like write to a file
break;
}
}
The BinaryWriter/Reader implementation of the length-counted string is non-standard, so if you wanted to interact with any other non-.NET code using this technique, it would be more complicated because you have to replicate/reimplement the non-standard length-counting logic yourself.
IMHO a better approach is to encode commands as fixed-length data, e.g. an 8-, 16-, or 32-bit value, which is just some integer that specifies the command. Then you can list your commands in the code as an enum type, casting to/from the network stream for the I/O. This would be more portable, easier to implement on non-.NET platforms.
In my client/server application my client wiil communicate with the server for 2 functions: the client will either request data from the server or it will send data so the server will save it. I'm using one socket for both methods, and the method to be used is defined by the first byte sent. If the first byte is "1" it is requesting data. If it is "2", it will send data (data bytes are sent after the "2" byte). It works perfectly for sending data. But when I'm requesting data it works, as long as I don't read the socket stream in the client. It's like if I make the client read data after sending data, the server will have no data to read, and it just crashes when trying to read the data.
Here is my server code:
private const int BufferSize = 1024;
NetworkStream netstream = null;
byte[] RecData = new byte[BufferSize];
int RecBytes;
try {
netstream = clientSocket.GetStream();
int totalrecbytes = 0;
using (MemoryStream ms = new MemoryStream()) {
//When I get here, there is no data to read
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0) {
ms.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
byte[] bytes = ms.ToArray();
byte b = bytes[0];
switch (b) {
case 1:
//Here I gather data and put it in "stream" variable
byte[] SendingBuffer = null;
int NoOfPackets = Convert.ToInt32(Math.Ceiling(Convert.ToDouble(stream.Length) / Convert.ToDouble(BufferSize)));
int TotalLength = (int)stream.Length, CurrentPacketLength, counter = 0;
for (int i = 0; i < NoOfPackets; i++) {
if (TotalLength > BufferSize) {
CurrentPacketLength = BufferSize;
TotalLength = TotalLength - CurrentPacketLength;
}
else
CurrentPacketLength = TotalLength;
SendingBuffer = new byte[CurrentPacketLength];
stream.Read(SendingBuffer, 0, CurrentPacketLength);
netstream.Write(SendingBuffer, 0, (int)SendingBuffer.Length);
}
netstream.Flush();
}
catch (Exception e) {
Console.WriteLine("EXCEPTION:\n" + e.ToString());
}
break;
case 2:
//Code to read data
break;
}
}
netstream.Close()
clientSocket.Close();
And here is my client code:
using (System.Net.Sockets.TcpClient clientSocket = new System.Net.Sockets.TcpClient()) {
string returnData = "";
IAsyncResult ar = clientSocket.BeginConnect("127.0.0.1", 8080, null, null);
System.Threading.WaitHandle wh = ar.AsyncWaitHandle;
try {
if (!ar.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(5), false)) {
clientSocket.Close();
Console.WriteLine("Timeout");
return;
}
System.Net.Sockets.NetworkStream serverStream = clientSocket.GetStream();
byte b = 1;
byte[] outStream = { b };
serverStream.Write(outStream, 0, outStream.Length);
serverStream.Flush();
//If I comment following lines, the server can read sent data, but server can't otherwise
byte[] RecData = new byte[1024];
int RecBytes;
int totalrecbytes = 0;
MemoryStream MS = new MemoryStream();
while ((RecBytes = serverStream.Read(RecData, 0, RecData.Length)) > 0) {
MS.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
serverStream.Close();
clientSocket.Close();
clientSocket.EndConnect(ar);
}
catch (Exception ex) {
Console.WriteLine("Exceção: " + ex.ToString());
}
finally {
wh.Close();
}
}
So, how can I send data to server and read the response? (I tried even putting the thread to sleep after sending data, with no luck.)
Thanks in advance.
EDIT:
With some debug messages I discovered that the server do read the "1" byte that was sent, but somehow it gets stuck inside the while loop, like, the server just stops there, no more loops and it does not leave the while loop. I saw that after writing "loop" in console inside the while loop, and writing read bytes also in console. It wrote "loop" once, and the read byte.
This code worries me:
//When I get here, there is no data to read
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0) {
ms.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
You are reading until the client closes the connection (or shuts down sending, which you don't do). But the client only closes when the server has replied. The server reply will never come. It is a deadlock.
Solution: Read a single byte to determine the requests command (b).
Unrelated to the question, your "packetised" sending (NoOfPackets, ...) does not seem to serve any purpose. Just use Stream.Copy to write. TCP does not have packets.
An even better solution would be to abandon your custom TCP protocol and use an HTTP library. All these concerns just go away. There are various smaller problems with your code that are very typical to see in TCP code.
Good morning, I have an desktop app that uploads files to a WCF service and then WCF Service uploads to Amazon S3.
This is my WCF method that receives the file and uploads to S3.
public void UploadFile(RemoteFileInfo request)
{
config = new AmazonS3Config();
config.CommunicationProtocol = Protocol.HTTP;
accessKeyID = "XXXXXXX";
secretAccessKeyID = "YYYYYYYY";
client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKeyID, secretAccessKeyID, config);
int chunkSize = 2048;
byte[] buffer = new byte[chunkSize];
using (System.IO.MemoryStream writeStream = new System.IO.MemoryStream())
{
do
{
// read bytes from input stream
int bytesRead = request.FileByteStream.Read(buffer, 0, chunkSize);
if (bytesRead == 0) break;
// simulates slow connection
System.Threading.Thread.Sleep(3);
// write bytes to output stream
writeStream.Write(buffer, 0, bytesRead);
} while (true);
// report end
Console.WriteLine("Done!");
// start the uploading to S3
PutObjectRequest fileRequest = new PutObjectRequest();
fileRequest.WithInputStream(writeStream);
fileRequest.Key = "testfile.pdf";
fileRequest.WithBucketName("tempbucket");
fileRequest.CannedACL = S3CannedACL.Private;
fileRequest.StorageClass = S3StorageClass.Standard;
client.PutObject(fileRequest);
writeStream.Close();
}
}
On my client I get the progress in real time when upload the file to the WCF Service but when I get the 100% complete it doesnt mean that the file has already uploaded to S3, so I would like to know if its possible to being uploading the file to S3 while Im writing the stream (inside of the using
(System.IO.MemoryStream writeStream = new System.IO.MemoryStream())
{
Is this possible? Any guideline on how to do it?
Appreciate in advance.
You can use InputStream property of PutObjectRequest
public void UploadFile(RemoteFileInfo request)
{
config = new AmazonS3Config();
config.CommunicationProtocol = Protocol.HTTP;
accessKeyID = "XXXXXXX";
secretAccessKeyID = "YYYYYYYY";
client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKeyID,secretAccessKeyID,config);
int chunkSize = 2048;
byte[] buffer = new byte[chunkSize];
PutObjectRequest fileRequest = new PutObjectRequest();
fileRequest.Key = "testfile.pdf";
fileRequest.WithBucketName("tempbucket");
fileRequest.CannedACL = S3CannedACL.Private;
fileRequest.StorageClass = S3StorageClass.Standard;
using (fileRequest.InputStream = new System.IO.MemoryStream())
{
do
{
// read bytes from input stream
int bytesRead = request.FileByteStream.Read(buffer, 0, chunkSize);
if (bytesRead == 0) break;
// simulates slow connection
System.Threading.Thread.Sleep(3);
// write bytes to output stream
fileRequest.InputStream.Write(buffer, 0, bytesRead);
} while (true);
// report end
Console.WriteLine("Done!");
client.PutObject(fileRequest);
}
}
I would recommend uploading the file to the WCF as chunks instead of a stream. I did so and it works just fine. also you need to return a message of the actual bytes written to the amazon later on you can increase the progress bar based on that. I know it will cause you to write a while loop in the client application but it will help you to show the progress with 100% accuracy for large files. Your WCF function should take the parameter like these
[DataContract]
class RemoteFileInfo
{
[DataMember]
Byte[] myChunk;
[DataMember]
long myOffset;
// other stuff you think you need to be sent each time.
}
I'm creating a socket server that needs to continuously listen for incoming messages from the connected clients. Those messages will be sent in a byte[] array. I had the server working great with a StreamReader but StreamReader only works with textual represenations of the data being sent...not byte[] arrays.
Here's what I had:
StreamReader reader = new StreamReader(Client.GetStream());
string line = "";
while (true)
{
line = reader.ReadLine();
if (!string.IsNullOrEmpty(line))
{
parentForm.ApplyText(line + "\r\n");
SocketServer.SendBroadcast(line);
}
}
I need to now convert that into a raw stream somehow that will convert the stream contents into a byte[] array but I can't seem to get a handle on it.
I tried this:
while (true)
{
var bytes = default(byte[]);
using (var memstream = new MemoryStream())
{
var buffer = new byte[512];
var bytesRead = default(int);
while ((bytesRead = reader.BaseStream.Read(buffer, 0, buffer.Length)) > 0)
memstream.Write(buffer, 0, bytesRead);
bytes = memstream.ToArray();
}
//parentForm.ApplyText(bytes.Length + "\r\n");
}
but as you might guess, the while(true) loop doesn't quite work how I need it to. Can anyone help me with some code adjustment to make this work as I need it to. It needs to continuously listen for incoming messages, then when a message is received, it needs to do something with that message (the byte[] array) then go back to listening again.
TIA
I guess "listening continuously" is not task of reader its a task of listener. I ran into same problem when i was writing server using TcpListener. I am not sure what you want to do but i am posting solution for your "listening continuous" and reading into byte[] problem. I guess this code might help you:
TcpListener t = new TcpListener(IPAddress.Loopback, _port);
t.Start();
Console.WriteLine("Server is started and waiting for client\n\n");
byte[] buff = new byte[255];
NetworkStream stream;
TcpClient client;
while(true)
{
client = t.AcceptTcpClient();
if (!client.Connected)
return;
stream = client.GetStream();
while ((stream.Read(buff, 0, buff.Length)) != 0)
{
break;
}
if (0 != buff.Length)
break;
}
There's no need to convert anything. GetStream() returns a NetworkStream. See the sample Microsoft includes in the NetworkStream.Read Method. All you have to do is replace the myCompleteMessage.AppendFormat("{0}", Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead)); line with an appropriate storage mechanism.
I have implemented my Server application regarding this post here: http://www.codeguru.com/csharp/csharp/cs_network/sockets/article.php/c8781#Client1
Sum up: I am using async Sockets ala BeginAccept(..), BeginReceive(..).
My Server is capable of handling mutliple clients and everything works fine until a client performas two or more synchronous send operation without waiting some time. The client does not get any error and so is not notified, that the server does not get the second message! If the client waits approx. 100ms after the first send operation, everything works fine.
I thought that when i use TCP i can ensure that the server receives the message. (Except there is an exception thrown)!
Could you provide me a solution to fix this.
Here are the WaitForData(..) & OnDataReceive(..) Methods that i implemented in the server
public void WaitForData(MyClient client)
{
try
{
if (pfnCallBack == null)
{
pfnCallBack = new AsyncCallback(OnDataReceived);
}
iarResult = client.Socket.BeginReceive(client.DataBuffer,
0, client.DataBuffer.Length,
SocketFlags.None,
pfnCallBack,
client);
}
catch (SocketException se)
{
MessageBox.Show("SocketException#WaitForData" + se.Message);
}
}
public void OnDataReceived(IAsyncResult asyn)
{
try
{
MyClient user= (MyClient)asyn.AsyncState;
int iRx = user.Socket.EndReceive(asyn);
byte[] receivedData = user.DataBuffer;
MemoryStream memStream = new MemoryStream();
BinaryFormatter binForm = new BinaryFormatter();
memStream.Write(receivedData, 0, receivedData.Length);
memStream.Seek(0, SeekOrigin.Begin);
MyMessage msg = (MyMessage)binForm.Deserialize(memStream);
switch (msg.Command)
{
case (MyMessage.MyCommand.ConnId):
this.connId = (int) msg.MyObject;
tsslConnStatus.Text += " | ID: " + connId.ToString();
break;
case (MyMessage.MyCommand.Text):
MessageBox.Show(msg.MyObject.ToString());
break;
}
WaitForData(server);
}
catch (ObjectDisposedException ode)
{
MessageBox.Show("ObjectDisposedException#OnReceiveData" + ode.Message);
}
catch (SocketException se)
{
MessageBox.Show("SocketException#OnReceiveData" + se.Message);
}
}
The CLIENT calls a synchronous SEND METHOD TWICE or MORE! server INSTANCEOF MyClient
if (server.Socket.Connected)
{
BinaryFormatter bf = new BinaryFormatter();
MemoryStream ms = new MemoryStream();
bf.Serialize(ms, message);
MyMessage = new MyMessage(something);
server.Socket.Send(ms.ToArray());
}
so, i think this code snippets must be enough for you to get the idea i was trying to use!
If you need further details or code snippets, just tell me i will post it!
Thanx!
TCP is stream based and not message based. One Read can contain any of the following alternatives:
A teeny weeny part of message
A half message
Excactly one message
One and a half message
Two messages
Thus you need to use some kind of method to see if a complete message have arrived. The most common methods are:
Add a footer (for instance an empty line) which indicates end of message
Add a fixed length header containing the length of the message
Update
Simple example having just length as header.
Server side:
var buffer = binaryFormmater.Serialize(myobj);
var length = buffer.Length;
networkStream.Send(length);
networkStream.Send(buffer, 0, buffer.Length);
Client side:
var header = new buffer[4];
// TODO: You need to make sure that 4 bytes have been read.
networkStream.Read(header, 0, 4);
var length = BitConverter.ToInt32(buffer);
var readbuffer= new byte[65535];
var bytesLeft = length;
var messageStream = new MemoryStream();
while (bytesLeft > 0)
{
var read = networkStream.Read(readbuffer, 0, bytesLeft);
messageStream.Write(readbuffer, 0, read);
bytesLeft -= read,
}
messageStream.Seek(0, SeekOrigin.Begin);
MyMessage msg = (MyMessage)binForm.Deserialize(messageStream);