Sending large file consuming a lot of memory in c# - c#

I am sending very large file over TCP/IP network which is working as expected.
Just one concern that how to send it efficiently so that memory consumption should be optimized.
Below is the working code
Stream fileStream = File.OpenRead(tbFilename.Text);
byte[] fileBuffer = new byte[fileStream.Length];
fileStream.Read(fileBuffer, 0, (int)fileStream.Length);
// Open a TCP/IP Connection and send the data
TcpClient clientSocket = new TcpClient(tbServer.Text,8080);
NetworkStream networkStream = clientSocket.GetStream();
networkStream.Write(fileBuffer,0,fileBuffer.GetLength(0));
networkStream.Close();

Firstly, Stream.Read will probably not read the whole file just a chunk, but you omit the return value, which is the number of read bytes.
Secondly, you should use a smaller buffer (eg. 4K) and use that to send the file.
const int BUFSIZE = 4096;
long transferred = 0L;
long length = fileStream.Length;
using (BinaryReader br = new BinaryReader(fileStream))
{
while (transferred < length)
{
int chunkSize = Math.Min(length - transferred, BUFSIZE);
byte[] buffer = br.ReadBytes(chunkSize);
networkStream.Write(buffer, 0, chunkSize);
transferred += chunkSize;
// here you can even report some progress to adjust a ProgressBar or something
}
}

You can use Stream.CopyTo() to copy data from a source stream into another.
The CopyTo() method uses a temporary buffer as suggested by taffer.
Do not forget to Flush() the network stream afterwards.
using (Stream fileStream = File.OpenRead(tbFilename.Text))
{
TcpClient clientSocket = new TcpClient(tbServer.Text, 8080);
NetworkStream networkStream = clientSocket.GetStream();
fileStream.CopyTo(networkStream);
networkStream.Flush();
networkStream.Close();
clientSocket.Close();
}

Related

C# Networkstream reads nothing

Just trying to use Networkstream, and this is a simple code i wrote:
Client side:
TcpClient c = new TcpClient();
c.Connect("10.0.0.4", 10);
NetworkStream ns = c.GetStream();
byte[] buffer = System.Text.Encoding.UTF8.GetBytes("first");
byte[] buffer2 = System.Text.Encoding.UTF8.GetBytes("second");
MemoryStream stream = new MemoryStream();
stream.Write(buffer, 0, buffer.Length);
stream.Write(buffer2, 0, buffer2.Length);
stream.CopyTo(ns);
This is the server side:
TcpListener tl = new TcpListener(IPAddress.Any, 10);
tl.Start();
TcpClient c = tl.AcceptTcpClient();
NetworkStream ns = new NetworkStream(c.Client);
byte[] buff = new byte[5];
ns.Read(buff,0,buff.Length);
string result = System.Text.Encoding.UTF8.GetString(buff);
MessageBox.Show(result);
only when i close the entire application the MessageBox line is executed , and im always getting a blank messagebox! which mean result doesnt contain nothing...
Any help?
On the client stream is positioned at the very end of the stream. Therefore, CopyTo has nothing left to copy.
Use stream.Position = 0; before copying.
Also, you don't seem to be aware of the fact that socket reads (in fact any stream read) can return less bytes than were requested (at least one). Your reading code must account for that. TCP does not preserve message boundaries.

TCPListener File Transfer

I have a little complication i encounter.
I may not be expert in TCP Connections but i hope someone here would help me.
This is my Client Code:
void Connect(String server, String message)
{
try
{
Int32 port = 8968;
TcpClient client = new TcpClient(server, port);
Byte[] data = File.ReadAllBytes(curSelectedFile);
NetworkStream stream = client.GetStream();
Byte[] fileData = File.ReadAllBytes(curSelectedFile);
Byte[] msgData = Encoding.ASCII.GetBytes("SendFile");
Byte[] sendData = new byte[fileData.Length + msgData.Length];
// Copy data to send package.
msgData.CopyTo(sendData, 0);
fileData.CopyTo(sendData, 4);
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
Console.WriteLine("Sent: {0}", message);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
Console.WriteLine("Received: {0}", responseData);
// Close everything.
stream.Close();
client.Close();
}
catch (ArgumentNullException e)
{
Console.WriteLine("ArgumentNullException: {0}", e);
}
catch (SocketException e)
{
Console.WriteLine("SocketException: {0}", e);
}
Console.WriteLine("\n Press Enter to continue...");
Console.Read();
}
This is my server one:
// Listen loop.
while(true)
{
using (TcpClient tcpClient = myListener.AcceptTcpClient())
{
Console.WriteLine("[Server] Acceptam client.");
using (NetworkStream networkStream = tcpClient.GetStream())
{
// Buffer for reading data
Byte[] bytes = new Byte[1024];
var data = new List<byte>();
int length;
while ((length = networkStream.Read(bytes, 0, bytes.Length)) != 0)
{
var copy = new byte[length];
Array.Copy(bytes, 0, copy, 0, length);
data.AddRange(copy);
}
// Incercam sa vedem ce doreste clientul.
string msg = Encoding.ASCII.GetString(data[0], 0, length);
if(msg.StartsWith("SendFile"))
{
using (Stream stream = new FileStream(#"C:\test.mp3", FileMode.Create, FileAccess.ReadWrite))
{
BinaryFormatter binaryFormatter = new BinaryFormatter();
networkStream.Position = 4;
binaryFormatter.Serialize(networkStream, data.ToArray());
}
}
}
}
}
What i'm trying to do here:
- I want the client to send a Message.. like "SaveFile" & after this string to be the filedata.
- The server should read the client message, and to process stuff according to the Client sentstring, before doing something with the file.
I believe that i don't know how to do it.
May i have an example on how to send/receive and read certain strings from the beggining of the file? How i can put them in the byte array and how to read it... It's quite overwhelming..
PS: The current Server Code is reading the data and CAN write as i coded it, without losing any packages. But also he's writing the aditional packets i sent before i converted the bytes of the file.
networkStream.Position = 4; isn't legal, because NetworkStream is not seekable.
I would discourage you from mixing text and binary data, just because of the complication it makes in the application protocol. But if you really want to do that, you should use BinaryWriter and BinaryReader, because it can write strings to a stream which can then be read later without consuming the bytes after the string.
Then you can do something like this...
In the client:
BinaryWriter writer = new BinaryWriter(networkStream);
writer.Write("SendFile");
writer.Write(fileData.Length);
writer.Write(fileData);
In the server:
BinaryReader reader = new BinaryReader(networkStream);
switch (reader.ReadString())
{
case "SendFile":
{
int length = reader.ReadInt32();
byte[] fileData = reader.ReadBytes(length);
// ... then do whatever with fileData, like write to a file
break;
}
}
The BinaryWriter/Reader implementation of the length-counted string is non-standard, so if you wanted to interact with any other non-.NET code using this technique, it would be more complicated because you have to replicate/reimplement the non-standard length-counting logic yourself.
IMHO a better approach is to encode commands as fixed-length data, e.g. an 8-, 16-, or 32-bit value, which is just some integer that specifies the command. Then you can list your commands in the code as an enum type, casting to/from the network stream for the I/O. This would be more portable, easier to implement on non-.NET platforms.

C# read all bytes

I am trying to write a simple client/server application in C#. The following is an example server reply sent to my client:
reply {20}<entry name="test"/>
where {20} indicates number of chars that full reply contains.
In the code I wrote below how can I use this number to loop and read ALL chars?
TcpClient tcpClient = new TcpClient(host, port);
NetworkStream networkStream = tcpClient.GetStream();
...
// Server Reply
if (networkStream.CanRead)
{
// Buffer to store the response bytes.
byte[] readBuffer = new byte[tcpClient.ReceiveBufferSize];
// String that will contain full server reply
StringBuilder fullServerReply = new StringBuilder();
int numberOfBytesRead = 0;
do
{
numberOfBytesRead = networkStream.Read(readBuffer, 0, readBuffer.Length);
fullServerReply.AppendFormat("{0}", Encoding.UTF8.GetString(readBuffer, 0, tcpClient.ReceiveBufferSize));
} while (networkStream.DataAvailable);
}
You're not using numberOfBytesRead. It is fascinating to me that every 2nd TCP question has this same issue as its answer.
Apart from that, you cannot split UTF-8 encoded string at arbitrary boundaries. Encoding.UTF8.GetString will return garbage. Use StreamReader.
The code is just horribly wrong. #usr already pinpointed two big mistakes.
Here is corrected code:
// Server Reply
if (networkStream.CanRead) {
// Buffer to store the response bytes.
byte[] readBuffer = new byte[tcpClient.ReceiveBufferSize];
string fullServerReply = null;
using (var writer = new MemoryStream()) {
while (networkStream.DataAvailable) {
int numberOfBytesRead = networkStream.Read(readBuffer, 0, readBuffer.Length);
if (numberOfBytesRead <= 0) {
break;
}
writer.Write(readBuffer, 0, numberOfBytesRead);
}
fullServerReply = Encoding.UTF8.GetString(writer.ToArray());
}
}

create a "listener" for an incoming socket stream that will contain a byte[] array

I'm creating a socket server that needs to continuously listen for incoming messages from the connected clients. Those messages will be sent in a byte[] array. I had the server working great with a StreamReader but StreamReader only works with textual represenations of the data being sent...not byte[] arrays.
Here's what I had:
StreamReader reader = new StreamReader(Client.GetStream());
string line = "";
while (true)
{
line = reader.ReadLine();
if (!string.IsNullOrEmpty(line))
{
parentForm.ApplyText(line + "\r\n");
SocketServer.SendBroadcast(line);
}
}
I need to now convert that into a raw stream somehow that will convert the stream contents into a byte[] array but I can't seem to get a handle on it.
I tried this:
while (true)
{
var bytes = default(byte[]);
using (var memstream = new MemoryStream())
{
var buffer = new byte[512];
var bytesRead = default(int);
while ((bytesRead = reader.BaseStream.Read(buffer, 0, buffer.Length)) > 0)
memstream.Write(buffer, 0, bytesRead);
bytes = memstream.ToArray();
}
//parentForm.ApplyText(bytes.Length + "\r\n");
}
but as you might guess, the while(true) loop doesn't quite work how I need it to. Can anyone help me with some code adjustment to make this work as I need it to. It needs to continuously listen for incoming messages, then when a message is received, it needs to do something with that message (the byte[] array) then go back to listening again.
TIA
I guess "listening continuously" is not task of reader its a task of listener. I ran into same problem when i was writing server using TcpListener. I am not sure what you want to do but i am posting solution for your "listening continuous" and reading into byte[] problem. I guess this code might help you:
TcpListener t = new TcpListener(IPAddress.Loopback, _port);
t.Start();
Console.WriteLine("Server is started and waiting for client\n\n");
byte[] buff = new byte[255];
NetworkStream stream;
TcpClient client;
while(true)
{
client = t.AcceptTcpClient();
if (!client.Connected)
return;
stream = client.GetStream();
while ((stream.Read(buff, 0, buff.Length)) != 0)
{
break;
}
if (0 != buff.Length)
break;
}
There's no need to convert anything. GetStream() returns a NetworkStream. See the sample Microsoft includes in the NetworkStream.Read Method. All you have to do is replace the myCompleteMessage.AppendFormat("{0}", Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead)); line with an appropriate storage mechanism.

data loss when file is transfer from server to client using stream in c#

server side
stream.BeginWrite(clientData, 0, clientData.Length,
new AsyncCallback(CompleteWrite), stream);
client side
int tot = s.Read(clientData, 0, clientData.Length);
I have used TCPClient,TCPlistener classes
clientData is a byte array.Size of ClientData is 2682 in server side.I have used NetworkStream class to write data
but in client side received data contains only 1642 bytes.I have used stream class to read data in client side
What's wrong?
The Read method is permitted to return fewer bytes than you requested. You need to call Read repeatedly until you have received the number of bytes you want.
Use this method to read properly from a stream:
public static void ReadWholeArray (Stream stream, byte[] data)
{
int offset=0;
int remaining = data.Length;
while (remaining > 0)
{
int read = stream.Read(data, offset, remaining);
if (read <= 0)
throw new EndOfStreamException
(String.Format("End of stream reached with {0} bytes left to read", remaining));
remaining -= read;
offset += read;
}
}
You may want to Write the length of the file into the stream first (say as an int)
eg,
server side:
server.Write(clientData.Length)
server.Write(clientData);
client side:
byte[] size = new byte[4];
ReadWholeArray(stream, size);
int fileSize = BitConverter.ToInt32(size, 0);
byte[] fileBytes = new byte[fileSize];
ReadWholeArray(stream, fileBytes);
see http://www.yoda.arachsys.com/csharp/readbinary.html for more info on reading from streams.

Categories