We Keep Coding

Sending large file consuming a lot of memory in c#

I am sending very large file over TCP/IP network which is working as expected.
Just one concern that how to send it efficiently so that memory consumption should be optimized.
Below is the working code
Stream fileStream = File.OpenRead(tbFilename.Text);
byte[] fileBuffer = new byte[fileStream.Length];
fileStream.Read(fileBuffer, 0, (int)fileStream.Length);
// Open a TCP/IP Connection and send the data
TcpClient clientSocket = new TcpClient(tbServer.Text,8080);
NetworkStream networkStream = clientSocket.GetStream();
networkStream.Write(fileBuffer,0,fileBuffer.GetLength(0));
networkStream.Close();

Firstly, Stream.Read will probably not read the whole file just a chunk, but you omit the return value, which is the number of read bytes.
Secondly, you should use a smaller buffer (eg. 4K) and use that to send the file.
const int BUFSIZE = 4096;
long transferred = 0L;
long length = fileStream.Length;
using (BinaryReader br = new BinaryReader(fileStream))
{
while (transferred < length)
{
int chunkSize = Math.Min(length - transferred, BUFSIZE);
byte[] buffer = br.ReadBytes(chunkSize);
networkStream.Write(buffer, 0, chunkSize);
transferred += chunkSize;
// here you can even report some progress to adjust a ProgressBar or something
}
}

You can use Stream.CopyTo() to copy data from a source stream into another.
The CopyTo() method uses a temporary buffer as suggested by taffer.
Do not forget to Flush() the network stream afterwards.
using (Stream fileStream = File.OpenRead(tbFilename.Text))
{
TcpClient clientSocket = new TcpClient(tbServer.Text, 8080);
NetworkStream networkStream = clientSocket.GetStream();
fileStream.CopyTo(networkStream);
networkStream.Flush();
networkStream.Close();
clientSocket.Close();
}

C# Networkstream reads nothing

Just trying to use Networkstream, and this is a simple code i wrote:
Client side:
TcpClient c = new TcpClient();
c.Connect("10.0.0.4", 10);
NetworkStream ns = c.GetStream();
byte[] buffer = System.Text.Encoding.UTF8.GetBytes("first");
byte[] buffer2 = System.Text.Encoding.UTF8.GetBytes("second");
MemoryStream stream = new MemoryStream();
stream.Write(buffer, 0, buffer.Length);
stream.Write(buffer2, 0, buffer2.Length);
stream.CopyTo(ns);
This is the server side:
TcpListener tl = new TcpListener(IPAddress.Any, 10);
tl.Start();
TcpClient c = tl.AcceptTcpClient();
NetworkStream ns = new NetworkStream(c.Client);
byte[] buff = new byte[5];
ns.Read(buff,0,buff.Length);
string result = System.Text.Encoding.UTF8.GetString(buff);
MessageBox.Show(result);
only when i close the entire application the MessageBox line is executed , and im always getting a blank messagebox! which mean result doesnt contain nothing...
Any help?

On the client stream is positioned at the very end of the stream. Therefore, CopyTo has nothing left to copy.
Use stream.Position = 0; before copying.
Also, you don't seem to be aware of the fact that socket reads (in fact any stream read) can return less bytes than were requested (at least one). Your reading code must account for that. TCP does not preserve message boundaries.

TCPListener File Transfer

I have a little complication i encounter.
I may not be expert in TCP Connections but i hope someone here would help me.
This is my Client Code:
void Connect(String server, String message)
{
try
{
Int32 port = 8968;
TcpClient client = new TcpClient(server, port);
Byte[] data = File.ReadAllBytes(curSelectedFile);
NetworkStream stream = client.GetStream();
Byte[] fileData = File.ReadAllBytes(curSelectedFile);
Byte[] msgData = Encoding.ASCII.GetBytes("SendFile");
Byte[] sendData = new byte[fileData.Length + msgData.Length];
// Copy data to send package.
msgData.CopyTo(sendData, 0);
fileData.CopyTo(sendData, 4);
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
Console.WriteLine("Sent: {0}", message);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
Console.WriteLine("Received: {0}", responseData);
// Close everything.
stream.Close();
client.Close();
}
catch (ArgumentNullException e)
{
Console.WriteLine("ArgumentNullException: {0}", e);
}
catch (SocketException e)
{
Console.WriteLine("SocketException: {0}", e);
}
Console.WriteLine("\n Press Enter to continue...");
Console.Read();
}
This is my server one:
// Listen loop.
while(true)
{
using (TcpClient tcpClient = myListener.AcceptTcpClient())
{
Console.WriteLine("[Server] Acceptam client.");
using (NetworkStream networkStream = tcpClient.GetStream())
{
// Buffer for reading data
Byte[] bytes = new Byte[1024];
var data = new List<byte>();
int length;
while ((length = networkStream.Read(bytes, 0, bytes.Length)) != 0)
{
var copy = new byte[length];
Array.Copy(bytes, 0, copy, 0, length);
data.AddRange(copy);
}
// Incercam sa vedem ce doreste clientul.
string msg = Encoding.ASCII.GetString(data[0], 0, length);
if(msg.StartsWith("SendFile"))
{
using (Stream stream = new FileStream(#"C:\test.mp3", FileMode.Create, FileAccess.ReadWrite))
{
BinaryFormatter binaryFormatter = new BinaryFormatter();
networkStream.Position = 4;
binaryFormatter.Serialize(networkStream, data.ToArray());
}
}
}
}
}
What i'm trying to do here:
- I want the client to send a Message.. like "SaveFile" & after this string to be the filedata.
- The server should read the client message, and to process stuff according to the Client sentstring, before doing something with the file.
I believe that i don't know how to do it.
May i have an example on how to send/receive and read certain strings from the beggining of the file? How i can put them in the byte array and how to read it... It's quite overwhelming..
PS: The current Server Code is reading the data and CAN write as i coded it, without losing any packages. But also he's writing the aditional packets i sent before i converted the bytes of the file.

networkStream.Position = 4; isn't legal, because NetworkStream is not seekable.
I would discourage you from mixing text and binary data, just because of the complication it makes in the application protocol. But if you really want to do that, you should use BinaryWriter and BinaryReader, because it can write strings to a stream which can then be read later without consuming the bytes after the string.
Then you can do something like this...
In the client:
BinaryWriter writer = new BinaryWriter(networkStream);
writer.Write("SendFile");
writer.Write(fileData.Length);
writer.Write(fileData);
In the server:
BinaryReader reader = new BinaryReader(networkStream);
switch (reader.ReadString())
{
case "SendFile":
{
int length = reader.ReadInt32();
byte[] fileData = reader.ReadBytes(length);
// ... then do whatever with fileData, like write to a file
break;
}
}
The BinaryWriter/Reader implementation of the length-counted string is non-standard, so if you wanted to interact with any other non-.NET code using this technique, it would be more complicated because you have to replicate/reimplement the non-standard length-counting logic yourself.
IMHO a better approach is to encode commands as fixed-length data, e.g. an 8-, 16-, or 32-bit value, which is just some integer that specifies the command. Then you can list your commands in the code as an enum type, casting to/from the network stream for the I/O. This would be more portable, easier to implement on non-.NET platforms.

create a "listener" for an incoming socket stream that will contain a byte[] array

I'm creating a socket server that needs to continuously listen for incoming messages from the connected clients. Those messages will be sent in a byte[] array. I had the server working great with a StreamReader but StreamReader only works with textual represenations of the data being sent...not byte[] arrays.
Here's what I had:
StreamReader reader = new StreamReader(Client.GetStream());
string line = "";
while (true)
{
line = reader.ReadLine();
if (!string.IsNullOrEmpty(line))
{
parentForm.ApplyText(line + "\r\n");
SocketServer.SendBroadcast(line);
}
}
I need to now convert that into a raw stream somehow that will convert the stream contents into a byte[] array but I can't seem to get a handle on it.
I tried this:
while (true)
{
var bytes = default(byte[]);
using (var memstream = new MemoryStream())
{
var buffer = new byte[512];
var bytesRead = default(int);
while ((bytesRead = reader.BaseStream.Read(buffer, 0, buffer.Length)) > 0)
memstream.Write(buffer, 0, bytesRead);
bytes = memstream.ToArray();
}
//parentForm.ApplyText(bytes.Length + "\r\n");
}
but as you might guess, the while(true) loop doesn't quite work how I need it to. Can anyone help me with some code adjustment to make this work as I need it to. It needs to continuously listen for incoming messages, then when a message is received, it needs to do something with that message (the byte[] array) then go back to listening again.
TIA

I guess "listening continuously" is not task of reader its a task of listener. I ran into same problem when i was writing server using TcpListener. I am not sure what you want to do but i am posting solution for your "listening continuous" and reading into byte[] problem. I guess this code might help you:
TcpListener t = new TcpListener(IPAddress.Loopback, _port);
t.Start();
Console.WriteLine("Server is started and waiting for client\n\n");
byte[] buff = new byte[255];
NetworkStream stream;
TcpClient client;
while(true)
{
client = t.AcceptTcpClient();
if (!client.Connected)
return;
stream = client.GetStream();
while ((stream.Read(buff, 0, buff.Length)) != 0)
{
break;
}
if (0 != buff.Length)
break;
}

There's no need to convert anything. GetStream() returns a NetworkStream. See the sample Microsoft includes in the NetworkStream.Read Method. All you have to do is replace the myCompleteMessage.AppendFormat("{0}", Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead)); line with an appropriate storage mechanism.

Ensure streamreader doesn't hang waiting for data

The code below reads everything there is to read from tcp client stream, and on the next iteration it will just sit there on the Read() (i'm assuming waiting for data). How can I ensure it doesn't and just returns when there's nothing there to read? Do I have to set low timeout, and respond to an exception when it fails out? Or there's a better way?
TcpClient tcpclnt = new TcpClient();
tcpclnt.Connect(ip, port);
Stream stm = tcpclnt.GetStream();
stm.Write(cmdBuffer, 0, cmdBuffer.Length);
byte[] response = new Byte[2048];
MemoryStream ms = new MemoryStream();
byte[] buffer = new Byte[2048];
int length;
while ((length = stm.Read(buffer, 0, buffer.Length)) > 0)
ms.Write(buffer, 0, length);
byte[] response = ms.ToArray();
tcpclnt.Close();

TcpClient has two properties that I would play with.
NoDelay Gets or sets a value that disables a delay when send or receive buffers are not full.
ReceiveTimeout Gets or sets the amount of time a TcpClient will wait to receive data once a read operation is initiated

Maybe you should use multithreading. One thread should wait for data, another should process received data.

Does the server have "Keep Connections Alive" on ?
I have had servers which will send you a stream of empty bytes as long as you request them.
Turning "Keep Connections Alive" stopped it doing this.

var socket = new System.Net.Sockets.TcpClient();
socket.NoDelay = true;
socket.Connect(uri.Host, port);
var ns = socket.GetStream();
int timeout = 500; //ms
DateTime lastReceived = DateTime.Now;
string buffer = "";
while (true)
{
if (ns.DataAvailable)
{
var b = ns.ReadByte();
buffer += b + ", "; //customise this line yourself
lastReceived = DateTime.Now;
continue;
}
if (lastReceived.AddMilliseconds(timeout) < DateTime.Now) break;
System.Threading.Thread.Sleep(timeout / 5);
}