This is how I send a file using a NetworkStream.
private void go()
{
byte[] send = File.ReadAllBytes("example.txt");
ns.Write(send, 0, send.Length);
}
ns is a NetworkStream of course.
Now I would like to know how I could receive and read an incoming NetworkStream?
I know that I need to specify a buffer to read from like this,
ns.Read(buffer,0,buffer.length).
but which buffer should be there?
TCP is a stream based protocol, which means that there is no notation of application messages like in UDP. Thus you cannot really detect by TCP itself where an application message ends.
Therefore you need to introduce some kind of detection. Typically you add a suffix (new line, semicolon or whatever) or a length header.
In this case it's easier to add a length header since the chosen suffix could be found in the file data.
So sending the file would look like this:
private void SendFile(string fileName, NetworkStream ns)
{
var bytesToSend = File.ReadAllBytes(fileName);
var header = BitConverter.GetBytes(bytesToSend.Length);
ns.Write(header, 0, header.Length);
ns.Write(bytesToSend, 0, bytesToSend.Length);
}
On the receiver side it's important that you check the return value from Read as contents can come in chunks:
public byte[] ReadFile(NetworkStream ns)
{
var header = new byte[4];
var bytesLeft = 4;
var offset = 0;
// have to repeat as messages can come in chunks
while (bytesLeft > 0)
{
var bytesRead = ns.Read(header, offset, bytesLeft);
offset += bytesRead;
bytesLeft -= bytesRead;
}
bytesLeft = BitConverter.ToInt32(header, 0);
offset = 0;
var fileContents = new byte[bytesLeft];
// have to repeat as messages can come in chunks
while (bytesLeft > 0)
{
var bytesRead = ns.Read(fileContents, offset, bytesLeft);
offset += bytesRead;
bytesLeft -= bytesRead;
}
return fileContents;
}
Related
I am working on socket C#. I've implemented a client server application using socket, but the problem is that the client doesn't receive all data sent by the server.
Here is the client application code. What should I do so that it would receive all data sent by the server?
strRecieved = "";
Socket soc = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
IPEndPoint endPoint = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 9001);
soc.Connect(endPoint);
byte[] msgBuffer = Encoding.Default.GetBytes(path);
soc.Send(msgBuffer, 0, msgBuffer.Length, 0);
byte[] buffer = new byte[2000];
int rec = soc.Receive(buffer);
strRecieved = String.Format(Encoding.Default.GetString(buffer));
First of all. If you're implementing some kind of streaming feature ( tcp/udp/file ) you should consider using some kind of protocol.
What is a protocol? It's just a scheme to use when streaming data. Example:
[4Bytes - length][lengthBytes - message][1Byte - termination indicator]
Knowing the protocol you can read all of the incoming bytes simply as such :
byte[] buffer = new byte[4];
stream.ReadBytes(buffer, 0, 4); // cast that to int and read the rest
int packetLen = BitConverter.ToInt32(buffer, 0);
buffer = new byte[packetLen];
stream.ReadBytes(buffer, 0, buffer.Length); // all bytes that was sent
Remember that you have to subtract thease 4 bytes in the length before sending the message.
EDIT:
Simple example on how to send and receive data using shared protocol.
// sender.cs
string _stringToSend = "some fancy string";
byte[] encodedString = Encoding.UTF8.GetBytes(_stringToSend);
List<byte> buffer = new List<byte>();
buffer.AddRange(BitConverter.GetBytes(encodedString.Length));
buffer.AddRange(encodedString);
netStream.WriteBytes(buffer.ToArray(), 0, buffer.Count);
// netStream sent message in protocol [#LEN - 4Bytes][#MSG - #LENBytes]
// simply speaking something like: 5ABCDE
// receiver.cs
byte[] buffer = new byte[sizeof(int)];
netStream.ReadBytes(buffer, 0, buffer.Length);
// receiver got the length of the message eg. 5
int dataLen = BitConverter.ToInt32(buffer, 0);
buffer = new byte[dataLen];
// now we can read an actual message because we know it's length
netStream.ReadBytes(buffer, 0, buffer.Length);
string receivedString = Encoding.UTF8.GetString(buffer);
// received string is equal to "some fancy string"
Making it simpler
This technique forces you to use desired protocol which in this example will be :
First 4 bytes sizeof(int) are indicating the length of the incoming packet
Every byte further is your packet until the end.
So right now you should make ProtocolHelper object:
public static class ProtocolHelper
{
public byte[] PackIntoProtocol(string message)
{
List<byte> result = new List<byte>();
byte[] messageBuffer = Encoding.UTF8.GetBytes(message);
result.AddRange(BitConverter.GetBytes(messageBuffer.Length), 0); // this is the first part of the protocol ( length of the message )
result.AddRange(messageBuffer); // this is actual message
return result.ToArray();
}
public string UnpackProtocol(byte[] buffer)
{
return Encoding.UTF8.GetString(buffer, 0, buffer.Length);
}
}
Now ( depending on method you've chosen to read from network ) you have to send and receive your message.
// sender.cs
string meMessage = "network message 1";
byte[] buffer = ProtocolHelper.PackIntoProtocol(meMessage);
socket.Send(buffer, 0, buffer.Length, 0);
// receiver.cs
string message = string.Empty;
byte[] buffer = new byte[sizeof(int)]; // or simply new byte[4];
int received = socket.Receive(buffer);
if(received == sizeof(int))
{
int packetLen = BitConverter.ToInt32(buffer);// size of our message
buffer = new byte[packetLen];
received = socket.Receive(buffer);
if( packetLen == received ) // we have full buffer
{
message = PacketHelper.UnpackProtocol(buffer);
}
}
Console.WriteLine(message); // output: "network message 1"
You're limiting the size of received messages by 2KB as you're using new byte[2000].
I think you could either:
Size up you buffer to meet you message's size needs; and/or
Split you message into more than one socket messages.
Given that 4-8K is a good size for buffering socket messages and assuming RAM size is not a issue I would start with that, say, new byte[8000].
Also, you can send socket messages splitted in chunks. Maybe this is a good idea for the case. For example, if you have msg as the message (or object) you want to send:
private static async Task SendAnswer(Message msg, WebSocket socket)
{
var answer = System.Text.Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(msg).ToCharArray());
var bufferSize = 8000;
var endOfMessage = false;
for (var offset = 0; offset < answer.Length; offset += bufferSize)
{
if (offset + bufferSize >= answer.Length)
{
bufferSize = answer.Length - offset;
endOfMessage = true;
}
await socket.SendAsync(new ArraySegment<byte>(answer, offset, bufferSize),
WebSocketMessageType.Text, endOfMessage, CancellationToken.None);
}
}
And when receiving, you can also split the reception in chunks, so you can control you buffer (and therefore you memory consumption). After hanlding the whole message, you should wait for another message from the client to do more stuff. Source
private async Task ReceiveMessage(WebSocket webSocket)
{
var buffer = new byte[8000];
var result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
while (!result.CloseStatus.HasValue)
{
string msg = Encoding.UTF8.GetString(new ArraySegment<byte>(buffer, 0, result.Count).ToArray());
while (!result.EndOfMessage)
{
result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
msg += Encoding.UTF8.GetString(new ArraySegment<byte>(buffer, 0, result.Count).ToArray());
}
//At this point, `msg` has the whole message you sent, you can do whatever you want with it.
// [...]
//After you handle the message, wait for another message from the client
result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
}
await webSocket.CloseAsync(result.CloseStatus.Value, result.CloseStatusDescription, CancellationToken.None);
}
I'm sending data to Lector device.
Normally I received data from device when I sending on hercules.
Hercules is returning "sRA eExtIn1 0 0 0".
The below code has waiting line stream.Read() function.
How can I getting data from device?
string responseData = null;
using (TcpClient client = new TcpClient("10.1.13.102", 2111))
{
using (NetworkStream stream = client.GetStream())
{
byte[] sentData = System.Text.Encoding.ASCII.GetBytes("<STX>sRN eExtIn1<ETX>");
stream.Write(sentData, 0, sentData.Length);
byte[] buffer = new byte[32];
int bytes;
if (client.Connected)
{
while ((bytes = stream.Read(buffer, 0, buffer.Length)) != 0)
{
for (int i = 0; i < bytes; i++)
{
responseData += (char)buffer[i];
}
}
}
}
}
The mistake you're making, and the other answer is also making, is assuming that stream.Read won't return until it has read 32 bytes. That is incorrect.
https://msdn.microsoft.com/en-us/library/system.net.sockets.networkstream.read(v=vs.110).aspx
This method reads data into the buffer parameter and returns the
number of bytes successfully read. If no data is available for
reading, the Read method returns 0. The Read operation reads as much
data as is available, up to the number of bytes specified by the size
parameter.
It will return when there is no data available to read or 32 bytes have been read, whichever comes first. So if, for example, the client is slow or the network very busy, the response may not have arrived yet when you call stream.Read. Consequently, there will be nothing to read so it will return 0 and you will exit, failing to read the data. In fact, you may have to call stream.Read any number of times to get the full 32 bytes if the network is very saturated and data is arriving a few bytes at a time (not likely with such a small packet, but you have to code it that way).
So your code needs to look like this (note the additional while loop):
using (TcpClient client = new TcpClient("10.1.13.102", 2111))
{
using (NetworkStream stream = client.GetStream())
{
byte[] sentData = System.Text.Encoding.ASCII.GetBytes("<STX>sRN eExtIn1<ETX>");
stream.Write(sentData, 0, sentData.Length);
byte[] buffer = new byte[32];
int bytes;
if (client.Connected)
{
int bytesRead = 0;
while (bytesRead < buffer.Length)
{
while ((bytes = stream.Read(buffer, 0, buffer.Length)) != 0)
{
for (int i = 0; i < bytes; i++)
{
responseData += (char)buffer[i];
}
bytesRead += bytes;
}
}
}
}
}
Thanks everybody.
I found solution of my question.
and tags should be describe as bytes. Like below.
byte[] byt = System.Text.Encoding.ASCII.GetBytes("sRN DItype");
stream.Write(STX, 0 , 1);
stream.Write(byt, 0, byt.Length);
stream.Write(ETX, 0, 1);
stream.Read(buffer, 0, buffer.Length);
In C/C++ read() on regular non-blocking network sockets will return immediately with the amount of data currently available in the buffer, up to the amount specified (so if we ask for 16 bytes and there are only 8 available at the moment, those 8 we'll get and it's up to us to call read() again and fetch all data).
In C# there's NetworkStream, which has built-in timeouts - does this mean that NetworkStream.Read() waits until either the timeout is reached or the amount of data requested is read, or will it give us any amount of data currently available in the buffer larger than 0 up to the amount requested (as the standard sockets do) even if there's time left?
It will read available data up to the number of bytes specified in the parameters, as described on MSDN, unless the stream is closed via timeout or other exception.
The Read operation reads as much data as is available, up to the number of bytes specified by the size parameter. If the remote host shuts down the connection, and all available data has been received, the Read method completes immediately and return zero bytes.
I solved it like this:
byte[] TotalData = new byte[0];
byte[] TempData = new byte[0];
using (TcpClient TCPClient = new TcpClient())
{
try
{
TCPClient.Connect(somehost, someport);
}
catch (Exception eee)
{
// Report the connection failed in some way if necessary
}
if (TCPClient.Connected)
{
using (NetworkStream clientStream = TCPClient.GetStream())
{
// You can reduce the size of the array if you know
// the data received is going to be small,
// don't forget to change it a little down too
byte[] TCPBuffer = new byte[524288];
int bytesRead = 0;
int loop = 0;
// Wait for data to begin coming in for up to 20 secs
while (!clientStream.DataAvailable && loop< 2000)
{
loop++;
Thread.Sleep(10);
}
// Keep reading until nothing comes for over 1 sec
while (clientStream.DataAvailable)
{
bytesRead = 0;
try
{
bytesRead = clientStream.Read(TCPBuffer, 0, 524288);
Array.Resize(ref TempData, bytesRead);
Array.Copy(TCPBuffer, TempData, bytesRead);
// Add data to TotalData
TotalData = JoinArrays(TotalData, TempData);
}
catch
{
break;
}
if (bytesRead == 0)
break;
Thread.Sleep(1000);
}
}
}
}
The JoinArrays() method:
byte[] JoinArrays(byte[] arrayA, byte[] arrayB)
{
byte[] outputBytes = new byte[arrayA.Length + arrayB.Length];
Buffer.BlockCopy(arrayA, 0, outputBytes, 0, arrayA.Length);
Buffer.BlockCopy(arrayB, 0, outputBytes, arrayA.Length, arrayB.Length);
return outputBytes;
}
The safe method is to use MemoryStream wich will make sure to wait and read all the stream to the memory , then u can use it as you like
public void SaveUserTemplate(Stream stream)
{
MemoryStream memoryStream = new MemoryStream();
stream.CopyTo(memoryStream);
memoryStream.Position = 0;
byte[] templatePathLength = new byte[4];
memoryStream.Read(templatePathLength, 0, templatePathLength.Length);
int nBytesTemplatePathLength = BitConverter.ToInt32(templatePathLength,0);
....
CopyTo function finally calls to this function:
github.com/microsoft/referencesource
private void InternalCopyTo(Stream destination, int bufferSize)
{
Contract.Requires(destination != null);
Contract.Requires(CanRead);
Contract.Requires(destination.CanWrite);
Contract.Requires(bufferSize > 0);
byte[] buffer = new byte[bufferSize];
int read;
while ((read = Read(buffer, 0, buffer.Length)) != 0)
destination.Write(buffer, 0, read);
}
I am writing what is essentially an image backup server to store images. It is a one way service that will not return anything beyond a basic success or failure message to the client.
The issue that I am experienceing is that when I send a byte array through the network stream, it is being cut-off before the end of the stream at random locations. I do not have this issue when I run the server on my development machine and connect locally, but rather it only occurs when the server is deployed on a remote server.
When I send very small arrays ( < 512 bytes) the server recieves the entire stream successfully, but on streams larger than 2000 bytes I experience issues. The code for the client is as follows:
try
{
TcpClient Voice = new System.Net.Sockets.TcpClient();
//Obviously I use the remote IP when it is deployed - but have altered it for privacy.
IPEndPoint BackupServer = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 57000);
Voice.Connect(BackupServer);
NetworkStream DataStream = Voice.GetStream();
byte[] buffer = new ASCIIEncoding().GetBytes(ImageData.GetXml());
DataStream.Write(buffer, 0, buffer.Length);
DataStream.Flush();
}
catch
{
}
try
{
buffer = new byte[4096];
int read = DataStream.Read(buffer, 0, buffer.Length);
MessageBox.Show(new ASCIIEncoding().GetString(buffer) + " : " + read.ToString());
}
catch
{
}
The client code executes without any errors or problems regardless of the size of data I send.
And the code for the server side is as follows:
private void BackMeUp(object voice)
{
TcpClient Voice = (TcpClient)voice;
Voice.ReceiveTimeout = 30000;
NetworkStream DataStream = Voice.GetStream();
try
{
bool ShouldLoop = true;
//int loops = 0;
int loops = -1;
byte[] input = new byte[2048];
byte[] buffer = new byte[0];
//while (ShouldLoop)
while(loops != 0)
{
loops = DataStream.Read(input, 0, 2048);
for (int x = 0; x < loops; x++)
{
Array.Resize(ref buffer, buffer.Length + 1);
buffer[buffer.Length - 1] = input[x];
}
//if (loops < 2048)
//{
//ShouldLoop = false;
//break;
//}
}
while (true)
{
StringReader Reader = new StringReader(new ASCIIEncoding().GetString(buffer, 0, buffer.Length));
DataSet DS = new DataSet();
DS.ReadXml(Reader);
if (DS.Tables.Count > 0)
{
if (DS.Tables["Images"].Rows.Count > 0)
{
foreach (DataRow row in DS.Tables["Images"].Rows)
{
//
}
}
}
string response = "Got it!";
DataStream.Write(new ASCIIEncoding().GetBytes(response), 0, response.Length);
DataStream.Flush();
Voice.Close();
break;
}
}
catch (Exception Ex)
{
File.WriteAllText("Elog.txt", Ex.Message + " " + (Ex.InnerException != null ? Ex.InnerException.ToString() : " no Inner"));
Voice.Close();
}
}
The server recieves the data fine, and closes the stream when it reaches the end, however the data is cut-off and I get an error when I try to rebuild the dataset.
I have the impression this has to do with the time it takes to send the stream, and I have played around with the Close and Flush commands but I feel like I'm just shooting in the dark. Any help would be appreciated.
Concise version of question: What factors are involved with a TcpListener that could cause a) the truncation of the stream. or b) premature closing of the stream prior to all bytes being read. When the listener in question is on a remote host rather than a local server.
The Read method doesn't have to return the number of bytes that you requested, or the entire stream at once. Especially if the stream is slow, it will be returned in small chunks.
Call the Read method repeatedly, and handle the data for each block that you get. The Read method returns zero when the stream is read to the end:
buffer = new byte[4096];
do {
int read = DataStream.Read(buffer, 0, buffer.Length);
if (read != 0) {
// handle the first "read" bytes of the buffer (index 0 to read-1)
}
} while (read != 0);
If you know that your buffer is enough for any stream, you can fill up the buffer and handle it afterwards:
buffer = new byte[4096];
int offset = 0;
do {
int read = DataStream.Read(buffer, offset, buffer.Length - offset);
offset += read;
} while (read != 0);
// handle the first "offset" bytes of the buffer (index 0 to offset-1)
server side
stream.BeginWrite(clientData, 0, clientData.Length,
new AsyncCallback(CompleteWrite), stream);
client side
int tot = s.Read(clientData, 0, clientData.Length);
I have used TCPClient,TCPlistener classes
clientData is a byte array.Size of ClientData is 2682 in server side.I have used NetworkStream class to write data
but in client side received data contains only 1642 bytes.I have used stream class to read data in client side
What's wrong?
The Read method is permitted to return fewer bytes than you requested. You need to call Read repeatedly until you have received the number of bytes you want.
Use this method to read properly from a stream:
public static void ReadWholeArray (Stream stream, byte[] data)
{
int offset=0;
int remaining = data.Length;
while (remaining > 0)
{
int read = stream.Read(data, offset, remaining);
if (read <= 0)
throw new EndOfStreamException
(String.Format("End of stream reached with {0} bytes left to read", remaining));
remaining -= read;
offset += read;
}
}
You may want to Write the length of the file into the stream first (say as an int)
eg,
server side:
server.Write(clientData.Length)
server.Write(clientData);
client side:
byte[] size = new byte[4];
ReadWholeArray(stream, size);
int fileSize = BitConverter.ToInt32(size, 0);
byte[] fileBytes = new byte[fileSize];
ReadWholeArray(stream, fileBytes);
see http://www.yoda.arachsys.com/csharp/readbinary.html for more info on reading from streams.