We Keep Coding

TCP download directly to storage

I've written a program that was initially intended for very basic text communication over the internet using the .net TCPClient class in C#. I decided to try setting up a procedure to read a file from one computer, break it up into smaller pieces which are each sent to the receiving computer, and have it reassembled and saved there. Essentially a file transfer.
I then realized that all the data I'm transferring is going into the memory of the receiving computer and then onto the storage in the next step. I am now wondering, is this the best way to do it? If data can be transferred and immediately written to the storage location where it's headed (bypassing the RAM step), is this the way a program like Google Chrome would handle downloads? Or are there usually important reasons for the data to be stored in memory first?
By the way, for clarity, let's all agree that "storage" would be like a hard drive and "memory" refers to RAM. Thanks.

Th way it is done usually is you open a FileStream read data in byte[] from TcpClient and write the number of bytes read from NetworkStream to FileStream.
Here is a pseduso example :
TcpClient tcp;
FileStream fileStream = File.Open("WHERE_TO_SAVE", FileMode.Open, FileAccess.Write);
NetworkStream tcpStream = tcp.GetStream();
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = tcpStream.Read(buffer, 0, buffer.Length)) > 0)
{
fileStream.Write(buffer, 0, bytesRead);
}
tcpStream.Dispose();
fileStream.Dispose();

Send Multiple Object through TCP without Closing or Disposing Stream

I had using the BinaryFormatter to Serialize an object through NetworkStream
The code like this
//OpenConnection ...
TCPClient client = server.AcceptTCPConnection();
Message message = new Message("bla bla"); // This is the serializable class
NetworkStream stream = client.GetStream(); // Get Stream
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(stream, message);
stream.Flush();
stream.Close(); //Close Connection
And in client Code, we just need to Read from stream
bf.Deserialize(stream) as Message
to get the object we just sent from Server.
But there is a problem here, if I delete the line stream.Close(); the client cannot read this Object. Or I can change to stream.Dispose();
However, I want to use this stream again to send another Message, how I can do? Please help, it make me feel so headache ##
UPDATE:
I found the reason of this issue. Because I used one machine to run both client and server. It definitely worked well in two different machines. Someone can tell me why? Get big problem with this for a couple day ago.

Sending multiple separate messages involves "framing" - splitting the single channel into separate chunks that don't ever require the client to "read to end". Oddly, though, I was under the impression that BinaryFormatter already implemented basic framing - but: I could be wrong. In the general case, when working with a binary protocol, the most common approach is to prefix each message with the length of the payload, i.e.
using(var ms = new MemoryStream()) {
while(...)
{
// not shown: serialize to ms
var len BitConverter.GetBytes((int)ms.Length);
output.Write(len, 0, 4);
output.Write(ms.GetBuffer(), 0, (int) ms.Length);
ms.SetLength(0); // ready for next cycle
}
}
the caller has to:
read exactly 4 bytes (at least, for the above), or detect EOF
determine the length
read exactly that many bytes
deserialize
repeat
If that sounds like a lot of work, maybe just use a serializer that does all this for you; for example, with protobuf-net, this would be:
while(...) { // each item
Serializer.SerializeWithLengthPrefix(output, PrefixStyle.Base128, 1);
}
and the reader would be:
foreach(var msg in Serializer.DeserializeItems<Message>(
input, PrefixStyle.Base128, 1))
{
// ...
}
(note: this does not use the same format / rules as BinaryFormatter)

Sending a image file using a socket in C# (Large Data)

i want to send a large data (image) approx . 1MB file though a socket connection
Question 1
following code snippet of the socket client which i currently use to send a a text message .. how i can modify this to send a file ?
NetworkStream serverStream = clientSocket.GetStream();
byte[] outStream = System.Text.Encoding.ASCII.GetBytes(richTextBox1.Text+"$");
serverStream.Write(outStream, 0, outStream.Length);
serverStream.Flush();
Question 2 : What modifications that required in both socket client and server to send and get large files ?

For large data portions you will probably need some kind of transmitting by portions. In your snippet you get all of the data in the array which couldn't be possible if file is large enough (or at least not suitable if it is not just one file to send).
Sending side will be something like that:
const int bufsize = 8192;
var buffer = new byte[bufsize];
NetworkStream ns = socket.GetStream();
using (var s = File.OpenRead("path"))
{
int actuallyRead;
while ((actuallyRead = s.Read(buffer, 0, bufsize)) > 0)
{
ns.Write(buffer, 0, actuallyRead);
}
}
ns.Flush();
Receiving site is just symmetric.

A socket doesn't care if you send text or binary data. Or how much you send. If there's any trouble then it is at the receiving end, code you didn't post. A classic mistake is to forget that a NetworkStream is a stream and not a sequence of packets. The Read() call at the receiving end can return any number of bytes. It won't be the number of bytes you wrote in the Write() call, depending on how routers in between the two machines broke up the IP packets and how much data is buffered in the receiver. You are probably getting away with calling Read only once because the string is short. That is definitely not going to work when you send a lot of data.
You need a protocol to help the receiver figure out when it received all the data. A simple way to do this is by first sending the length of the data. The receiver can then first read that length, then know how long to keep calling Read() to get the rest of the data.
You can arbitrarily extend this protocol by, say, sending the name of the file. Etcetera. Although that by the time you're done, you'd be close to having re-invented FTP.

If all you want to do is to send an image and you don't need any metadata, you can use code like this:
Server:
var listener = new TcpListener(address, port);
listener.Start();
using (var incoming = listener.AcceptTcpClient())
using (var networkStream = incoming.GetStream())
using (var fileStream = File.OpenWrite(imagePath))
{
networkStream.CopyTo(fileStream);
}
listener.Stop();
Client:
var client = new TcpClient();
client.Connect(address, port);
using (var networkStream = client.GetStream())
using (var fileStream = File.OpenRead(imagePath))
{
fileStream.CopyTo(networkStream);
}
client.Close();
If you want to compress the file, you can use GZipStream.
This code uses CopyTo() method, which is available in .Net 4, but you can write it yourself in earlier versions.

Wcf multiple services read file

I'm using wcf to implement a distributed application using multiple services.
One of my services must use MSMQ binding in order to receive requests asynchronously, so the requests aren't ignored when it's offline. The problem with MSMQ binding is that it is one way only and therefore I can't have methods returning within the contract.
So I had to make another endpoint with a wshttpbinding.
Now I have a problem that it might even not bee related to wcf services.
The service with the msmq binding writes in a file which is read by the other service. The problem is that when the first service makes changes to the file I must restart both in order to the second one read those changes.
I'm writing the information from the file using the following:
string filename = "holder.txt";
if (!File.Exists(filename))
File.Create(filename);
Stream stream = File.Open(filename, FileMode.Create);
BinaryFormatter bFormatter = new BinaryFormatter();
bFormatter.Serialize(stream, requests_list);
stream.Close();
I close the stream so shouldn't the changes made been seen from the other service when reading it? using:
string filename = "holder.txt";
if (!File.Exists(filename))
return null;
Requests requests_list;
Stream stream = File.Open(filename, FileMode.Open);
BinaryFormatter bFormatter = new BinaryFormatter();
try
{
requests_list = (Requests)bFormatter.Deserialize(stream);
}
catch (Exception)
{
stream.Close();
return null;
}
stream.Close();
return requests_list;
Btw both methods are in a class shared by both services.
Thanks in advance

Download and save file from http server using .Net Socket

Hello everybody.
I would like to know how to download and save a file to my hard drive, specifically a zip file from a HTTP server using the System.Net.Socket.Sockets class.
I know there are allot easier ways to download a file with .Net, but i would like to know how to do it with Sockets, if possible of course although I'm pretty sure it is.
I've tried a few things, nothing worked once i don't have any background experience with sockets.
Your help satisfying my curiosity is appreciated.
Any question just ask. Thank you.
Note:
The file is a standard zip file, however i would like a way that would work with any file type.
The file size is different every day.
The file is downloaded every minute, caching of such file must be disabled to get a accurate and update file version from the server.
File url sample: www.somewhere.com/files/feed/list.zip

You could do this directly with a .NET socket, but it would require parsing and understanding the HTTP request.
The standard way to do this would just be to use the higher level System.Net classes. For example, this can be done in two lines of code via WebClient.DownloadFile - why make life more difficult for yourself?
If you really must do this from raw sockets, it will just take a lot of work. At it's core, you can connect to port 80 (assuming http) via a TCP connection, write the correct strings to the socket, and start receiving data.
That being said, getting everything correct, and handling all of the issues required is far beyond a standard StackOverflow answer's scope. If you want to go down this road, take a look at the HTTP Protocol specifications - you'll need to implement the proper aspects of this specification.

For this you can simply use the "HttpWebRequest" and "HttpWebResponse" classes in .net.
Below is a sample console app I wrote to demonstrate how easy this is.
using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.IO;
namespace Test
{
class Program
{
static void Main(string[] args)
{
string url = "www.somewhere.com/files/feed/list.zip";
string fileName = #"C:\list.zip";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Timeout = 5000;
try
{
using (WebResponse response = (HttpWebResponse)request.GetResponse())
{
using (FileStream stream = new FileStream(fileName, FileMode.Create, FileAccess.Write))
{
byte[] bytes = ReadFully(response.GetResponseStream());
stream.Write(bytes, 0, bytes.Length);
}
}
}
catch (WebException)
{
Console.WriteLine("Error Occured");
}
}
public static byte[] ReadFully(Stream input)
{
byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
}
}
Enjoy!