I send the serialized data over LAN network but sometimes the information is lost! Process is the following:
Sender:
string mydata is being serialized
string mydata is converted to byte[] bytes_of_mydata
int size_of_mydata is the length of byte[] bytes_of_mydata
int size_of_mydata itself is turned into byte[] bytes_size_of_mydata
byte[] bytes_of_mydata and byte[] bytes_size_of_mydata are sent
Receiver:
i first receive byte[] bytes_size_of_mydata
retrieve length int size_of_mydata of the second message from byte[] bytes_size_of_mydata
i then receive byte[] bytes_of_mydata, knowing exact length!
i then convert byte[] bytes_of_mydata to string mydata
deserialize string mydata
This approach usually works in most situations, but sometimes my data is not transmitted fully, so the string can't be deserialized.
I've debugged the received byte[] on the "receiver" and here is what happens:
I get the size of second message:
int size_of_second_message = BitConverter.ToInt32(dataByteSize, 0); // 55185
I start to receive the second message to byte array:
Byte[] dataByte = new Byte[55185];
But starting from position 5840 I start to receive 0 (nulls), so the part "5840 - 55185" are all "0":
byte[5836] = 53;
byte[5837] = 57;
byte[5838] = 54;
byte[5839] = 49;
byte[5840] = 0; // information ends to flow
byte[5841] = 0;
byte[5842] = 0;
byte[5843] = 0;
//....
byte[55185] = 0;
The example from the above is taken from an actual debugger!
So what's the problem? It's like the connection is being lost during transmission!! Why is it happening and how do I counter this problem? It doesn't happen on "every-time" basis.
And here comes the code
Send:
//text_message - my original message
//Nw - network stream
MemoryStream Fs = new MemoryStream(ASCIIEncoding.Default.GetBytes(text_message));
Byte[] buffer = Fs.ToArray(); // total 55185 bytes (as in example)
Byte[] bufferSize = BitConverter.GetBytes(Fs.Length); // 32 bytes represent size
bufferSize = GetNewByteSize(bufferSize);
Nw.Write(bufferSize, 0, bufferSize.Length); // send size
Nw.Flush();
Nw.Write(buffer, 0, buffer.Length); // send message
Nw.Flush();
Receive:
//get first(SIZE) bytes:
int ReadSize = 0; int maxSize = 32; // 32 - constant!
Byte[] dataByteSize = new Byte[maxSize];
int origsize;
using (var strm = new MemoryStream())
{
ReadSize = Nw.Read(dataByteSize, 0, maxSize);
strm.Write(dataByteSize, 0, ReadSize);
strm.Seek(0, SeekOrigin.Begin);
origsize = BitConverter.ToInt32(dataByteSize, 0); // origsize = 55185
}
Nw.Flush();
//get next(MESSAGE) bytes:
string message = ""; int thisRead = 0;
int max = Convert.ToInt32(origsize); // origsize = 55185
Byte[] dataByte = new Byte[max];
using (var strm = new MemoryStream())
{
thisRead = Nw.Read(dataByte, 0, max);
strm.Write(dataByte, 0, thisRead);
strm.Seek(0, SeekOrigin.Begin);
using (StreamReader reader = new StreamReader(strm))
{
message = reader.ReadToEnd();
}
}
Nw.Flush();
// message - the message that is being transmitted partly (sometimes)!
I didn't want to post the code but you guys usually ask "show us what you've done", so here it is!
Edit
Temporary fix is to switch to StreamWriter, reader.
Receive + send (server):
NetworkStream Nw = new NetworkStream(handlerSocket.Client);
string toreceive = "";
StreamReader reader = new StreamReader(Nw);
toreceive = reader.ReadLine();
string text_message = "to send back";
StreamWriter writer = new StreamWriter(Nw);
writer.WriteLine(text_message);
writer.Flush();
Nw.Close();
Send + receive (client):
NetworkStream Nw = new NetworkStream(handlerSocket.Client);
StreamWriter writer = new StreamWriter(Nw);
writer.WriteLine("to send");
writer.Flush();
string toreceive = new StreamReader(Nw).ReadLine();
writer.Close();
Nw.Close();
I'm looking for a solution regarding original problem, but so far everything is working due to temporary fix.
TCP is a stream based protocol which lets you read as much data has yet been received. Just because you send data in one step, there is no guarantee that the data will be received in the same block. You have to loop on the receiving side until you have received all the data you anticipate.
By the way I think that your protocol with an explicit length field at the start is really good, it makes up for simple client code (as simple as it gets at least).
Some time back I asked a question on built in functionality to wait until X bytes of data is available: .NET blocking socket read until X bytes are available?. The answer is unfortunately no.
Related
My goal is to send a file over a TCP connection using NetworkStream.
I first send the length of the data I'm going to send, and then I use a filestream and a binary writter to send the data byte by byte.
While debugging the process, I found out that some '0' bytes are being put at the beggining of the file on the receiving end.
For example, the base file's content azertyuiop is received as azerty (4 spaces replacing uiop), causing files like images to be corrupted.
The code I've got so far :
(Where br is a BinaryReader and bw is a BinaryWriter)
Sender:
using (var readStream = new FileStream(fileLocation, FileMode.Open))
{
// Send the data length first
bw.Write(new FileInfo(fileLocation).Length);
bw.Flush();
var buffer = new byte[1];
while (readStream.Read(buffer, 0, 1) > 0)
{
bw.Write(buffer[0]);
bw.Flush();
}
}
Receiver:
// Get data length
var dataLength = br.ReadInt32();
using (var fs = new FileStream(newFileLocation, FileMode.Create))
{
var buffer = new byte[1];
for(int i = 0; i < dataLength; i++)
{
br.Read(buffer, 0, 1);
fs.Write(buffer, 0, 1);
}
}
What am I missing or doing wrong ?
The problem could be the following:
bw.Write(new FileInfo(fileLocation).Length);
...
var dataLength = br.ReadInt32();
The Length property is actually of type long (8 bytes). But you are reading the value as Int32 (4 bytes), leaving the other 4 bytes in the stream.
fileinfo.length is a long not an int32
I wrote a tcp listener to receive a sequence of images from single client , this is the code of server :
new Thread(() =>
{
while (true)
{
displayingFrame.Start(); // another thread to display images
Socket socket = listener.AcceptSocket();
TcpClient client = new TcpClient();
client.Client = socket;
Debug.WriteLine("Connection accepted.");
var childSocketThread = new Thread(() =>
{
NetworkStream ns = client.GetStream();
BinaryReader br = new BinaryReader(ns);
while (true)
{
using (MemoryStream ms = new MemoryStream())
{
int length = br.ReadInt32();
Debug.WriteLine("length : " + length);
byte[] buf = new byte[1024];
int totalReaded = 0;
int readed = 0;
while (totalReaded < length)
{
readed = br.Read(buf, 0, buf.Length);
ms.Write(buf, 0, readed);
totalReaded += readed;
}
byte[] frame = ms.ToArray();
this.frames.Enqueue(frame);
Debug.WriteLine("frame enqueued with length " + frame.Length);
}
}
});
childSocketThread.Start();
}
}).Start();
it receive frames very well but suddenly br.ReadInt32(); returns a very big length so br.Read(buf, 0, buf.Length); takes a very long time writing to memory stream and it writes a wrong data inside frame .
this is the client :
TcpClient client = new TcpClient();
client.Connect(new IPEndPoint(IPAddress.Loopback, 20000));
NetworkStream ns = client.GetStream();
BinaryWriter bw = new BinaryWriter(ns);
while ( true )
{
byte[] frame = Screenshot();
bw.Write(frame.Length);
Console.WriteLine("a frame length has flushed : " + frame.Length);
bw.Write(frame);
Console.WriteLine("a frame itself has flushed");
}
Console.ReadKey();
and here the debug info :
If you check the hex value you're getting - 1196314761 - you'll get 0x474E5089 and finally convert to ASCII you will get GNP\x89 which gives us the known magic value \x89PNG that is the marker of PNG file. You're actually reading the content of your screenshot as the length.
Make sure that you're code for reading the data does not read too much from the previous frame. I think you code for reading the data does not include the fact that you might get content of 2 frames in one .Read but then later you just don't care if you have too much data. You only check if it's not less than length.
I am trying to write a simple client/server application in C#. The following is an example server reply sent to my client:
reply {20}<entry name="test"/>
where {20} indicates number of chars that full reply contains.
In the code I wrote below how can I use this number to loop and read ALL chars?
TcpClient tcpClient = new TcpClient(host, port);
NetworkStream networkStream = tcpClient.GetStream();
...
// Server Reply
if (networkStream.CanRead)
{
// Buffer to store the response bytes.
byte[] readBuffer = new byte[tcpClient.ReceiveBufferSize];
// String that will contain full server reply
StringBuilder fullServerReply = new StringBuilder();
int numberOfBytesRead = 0;
do
{
numberOfBytesRead = networkStream.Read(readBuffer, 0, readBuffer.Length);
fullServerReply.AppendFormat("{0}", Encoding.UTF8.GetString(readBuffer, 0, tcpClient.ReceiveBufferSize));
} while (networkStream.DataAvailable);
}
You're not using numberOfBytesRead. It is fascinating to me that every 2nd TCP question has this same issue as its answer.
Apart from that, you cannot split UTF-8 encoded string at arbitrary boundaries. Encoding.UTF8.GetString will return garbage. Use StreamReader.
The code is just horribly wrong. #usr already pinpointed two big mistakes.
Here is corrected code:
// Server Reply
if (networkStream.CanRead) {
// Buffer to store the response bytes.
byte[] readBuffer = new byte[tcpClient.ReceiveBufferSize];
string fullServerReply = null;
using (var writer = new MemoryStream()) {
while (networkStream.DataAvailable) {
int numberOfBytesRead = networkStream.Read(readBuffer, 0, readBuffer.Length);
if (numberOfBytesRead <= 0) {
break;
}
writer.Write(readBuffer, 0, numberOfBytesRead);
}
fullServerReply = Encoding.UTF8.GetString(writer.ToArray());
}
}
I wanted to create a project for streaming multiple images through net. I just wanted to start with small, functional code but already encountered a funky problem for me. Image is received, but it contains graphical bugs like its only a part of it.
Sadly cant show images cause of low rputation, here is link.
http://img543.imageshack.us/img543/1508/buggedy.jpg
Of course, million dollars for answer. ^^
TcpListener listener;
TcpClient client;
TcpClient datatoclient;
NetworkStream stream;
Host part:
listener = new TcpListener(5000);
listener.Start();
datatoclient = listener.AcceptTcpClient();
NetworkStream nowystream = datatoclient.GetStream();
MemoryStream ms = new MemoryStream();
byte[] image = File.ReadAllBytes("default.jpg");
switch (trackBar1.Value)
{
case 0:
image = File.ReadAllBytes("mirrion.jpg");
break;
case 1:
image = File.ReadAllBytes("tenis.jpg");
break;
case 2:
image = File.ReadAllBytes("marisasold.jpg");
break;
}
// get the image size in bytes
int numberOfBytes = image.Length;
// put the size into an array
byte[] numberOfBytesArray = BitConverter.GetBytes(numberOfBytes);
// send the image size
nowystream.Write(numberOfBytesArray, 0, numberOfBytesArray.Length);
// send the image
nowystream.Write(image, 0, numberOfBytes);
Client part:
client = new TcpClient("127.0.0.1", 5000);
stream = client.GetStream();
byte[] data = new byte[4];
// read the size
stream.Read(data, 0, data.Length);
int size = BitConverter.ToInt32(data, 0);
label1.Text = size.ToString();
// prepare buffer
data = new byte[size];
// load image
stream.Read(data, 0, data.Length);
// save image to file for test
File.WriteAllBytes("received.jpg", data);
MemoryStream MS = new MemoryStream(data);
pictureBox1.Image = Image.FromStream(MS);
stream.Read doesn't guarantee that it will read data.Length bytes. Instead it returns number of bytes read. So you should check its return value and continue reading till you get all the bytes.
See http://msdn.microsoft.com/en-us/library/system.io.stream.read(v=vs.90).aspx (Section Return Value)
Thr read method can be something like this
void Read(Stream stream, byte[] buffer,int offset,int len)
{
int read = 0;
while (read < len)
{
read += stream.Read(buffer, offset + read, len-read);
}
}
I just spent quite a few hours reading up on TCP servers and my desired protocol I was trying to implement, and finally got everything working great. I noticed the code looks like absolute bollocks (is the the correct usage? Im not a brit) and would like some feedback on optimizing it, mostly for reuse and readability.
The packet formats are always int, int, int, string, string.
try
{
BinaryReader reader = new BinaryReader(clientStream);
int packetsize = reader.ReadInt32();
int requestid = reader.ReadInt32();
int serverdata = reader.ReadInt32();
Console.WriteLine("Packet Size: {0} RequestID: {1} ServerData: {2}", packetsize, requestid, serverdata);
List<byte> str = new List<byte>();
byte nextByte = reader.ReadByte();
while (nextByte != 0)
{
str.Add(nextByte);
nextByte = reader.ReadByte();
}
// Password Sent to be Authenticated
string string1 = Encoding.UTF8.GetString(str.ToArray());
str.Clear();
nextByte = reader.ReadByte();
while (nextByte != 0)
{
str.Add(nextByte);
nextByte = reader.ReadByte();
}
// NULL string
string string2 = Encoding.UTF8.GetString(str.ToArray());
Console.WriteLine("String1: {0} String2: {1}", string1, string2);
// Reply to Authentication Request
MemoryStream stream = new MemoryStream();
BinaryWriter writer = new BinaryWriter(stream);
writer.Write((int)(1)); // Packet Size
writer.Write((int)(requestid)); // Mirror RequestID if Authenticated, -1 if Failed
byte[] buffer = stream.ToArray();
clientStream.Write(buffer, 0, buffer.Length);
clientStream.Flush();
}
I am going to be dealing with other packet types as well that are formatted the same (int/int/int/str/str), but different values. I could probably create a packet class, but this is a bit outside my scope of knowledge for how to apply it to this scenario. If it makes any difference, this is the Protocol I am implementing.
http://developer.valvesoftware.com/wiki/Source_RCON_Protocol
Thoughts:
you arent really using the reader except for a few ints; otherwise, all you need is ReadByte you can do that from the Stream, and save some indirection/confusion
read the ints manually to avoid Endianness issues
reading byte by byte can be expensive; if possible, try to fill a buffer (or rather: read the right amount of data) by looping over Read rather that ReadByte
if multiple messages are coming down the same pipe, reading to EOF will probably fail (either corrupt the data or block forever); you usually need either a terminator sequence or a length-prefix. I prefer the latter, as it let's you use Read instead of ReadByte
I assume that is packetSize in your example; it is critical to use this: to separate the messages, to verify you have an entire message, and to deny over-sized data
consider whether async (BeginRead) is suitable - sometimes yes, sometimes no; and note that this makes disposal trickier as you can't use "using" with async
when using MemoryStream, using .GetBuffer() in combination with .Length has less overhead than using .ToArray()
The first thing that jumps out to me is to always use the using statement with any object that implements IDisposable. This will ensure that your objects are properly disposed of even in the event of an exception.
private void FillList(BinaryReader reader, List list)
{
while (reader.PeekChar() != -1)
{
list.Add(reader.ReadByte());
}
}
...
try
{
int packetsize, requestid, serverdata;
string string1, string2;
List<byte> str = new List<byte>();
using (BinaryReader reader = new BinaryReader(clientStream))
{
packetsize = reader.ReadInt32();
requestid = reader.ReadInt32();
serverdata = reader.ReadInt32();
Console.WriteLine("Packet Size: {0} RequestID: {1} ServerData: {2}", packetsize, requestid, serverdata);
FillList(reader, str);
// Password Sent to be Authenticated
string1 = Encoding.UTF8.GetString(str.ToArray());
str.Clear();
FillList(reader, str);
}
// NULL string
string2 = Encoding.UTF8.GetString(str.ToArray());
Console.WriteLine("String1: {0} String2: {1}", string1, string2);
// Reply to Authentication Request
using (MemoryStream stream = new MemoryStream())
using (BinaryWriter writer = new BinaryWriter(stream))
{
writer.Write((int)(1)); // Packet Size
writer.Write((int)(requestid)); // Mirror RequestID if Authenticated, -1 if Failed
byte[] buffer = stream.ToArray();
clientStream.Write(buffer, 0, buffer.Length);
clientStream.Flush();
}
}