I made a client & server that establishes a TCP connection through sockets, I'm trying to send binary data over the socket but I could only send txt or pdf files, I had no luck with exe files, I use fread and fseek to read the file and split it into a buffer. When I read the whole exe file it gets sent successfully but when I split it, it gets sent corrupted!
I read some books about sockets but I still don't know much and I have some questions.
is it okay to send the whole file in one send()?? or I should continue with sending it in small chunks?
Also, why exe files get corrupted when I send them in chunks?
Thank you!
Client Code (in c):
int bufSize = 10000;
int sentBytes = 0;
FILE * pFile;
long remainingBytes;
char * buffer;
size_t result;
pFile = fopen ( "D:\\file.exe" , "rb" );
fseek (pFile , 0 , SEEK_END);
remainingBytes = ftell (pFile);
rewind (pFile);
int bufferSize = remainingBytes > bufSize ? bufSize : remainingBytes;
buffer = (char*) malloc (sizeof(char)*bufferSize);
send(Socket, (char*)&remainingBytes, 4, 0);
while(remainingBytes > 0)
{
fseek (pFile , sentBytes , SEEK_SET);
result = fread(buffer,1,bufferSize,pFile);
if(bufferSize < remainingBytes)
{
send(Socket, buffer, bufferSize, 0);
}
else
{
send(Socket, buffer, remainingBytes, 0);
bufferSize = remainingBytes;
}
remainingBytes -= bufferSize;
sentBytes += bufferSize;
}
Server code (in c#)
try
{
int bufferSize = 200;
int len = 0;
int receivedBytes = 0;
int remainingBytes = len;
byte[] length = new byte[4];
//byte[] imgBuf = new byte[bufferSize];
int current = 0;
List<byte[]> coming = new List<byte[]>();
sockets[number].Receive(length,4,0);
len = BitConverter.ToInt32(length, 0);
remainingBytes = len;
bufferSize = len < bufferSize ? len : bufferSize;
while(receivedBytes < len)
{
if (remainingBytes > bufferSize)
{
coming.Add(new byte[bufferSize]);
//imgBuf = new byte[bufferSize];
sockets[number].Receive(coming[current], bufferSize, 0);
}
else
{
coming.Add(new byte[remainingBytes]);
//imgBuf = new byte[remainingBytes];
sockets[number].Receive(coming[current], remainingBytes, 0);
bufferSize = remainingBytes;
}
remainingBytes -= bufferSize;
receivedBytes += bufferSize;
current++;
//Array.Clear(imgBuf, 0, imgBuf.Length);
}
using (var stream = new FileStream(#"C:\receivedFile.exe",FileMode.Create))
{
using (var binaryWriter = new BinaryWriter(stream))
{
foreach (byte[] buffer in coming)
{
binaryWriter.Write(buffer);
}
}
}
}
catch (Exception ex)
{ this.setText(ex.Message, textBox2); }
Edit: Thanks for the help, I got it working :)
try
{
int bufferSize = 1024 * 100;
int len = 0;
int receivedBytes = 0;
int remainingBytes = len;
int reached = 0;
byte[] length = new byte[4];
byte[] imgBuf = new byte[bufferSize];
int current = 0;
sockets[number].Receive(length,4,0);
len = BitConverter.ToInt32(length, 0);
remainingBytes = len;
bufferSize = len < bufferSize ? len : bufferSize;
imgBuf = new byte[len];
while (reached < len)
{
reached += sockets[number].Receive(imgBuf, receivedBytes, remainingBytes, 0);
remainingBytes = len - reached;
receivedBytes = reached;
current++;
//Array.Clear(imgBuf, 0, imgBuf.Length);
}
using (var stream = new FileStream(#"C:\putty.exe",FileMode.Create))
{
using (var binaryWriter = new BinaryWriter(stream))
{
binaryWriter.Write(imgBuf);
}
}
Array.Clear(imgBuf, 0, imgBuf.Length);
}
catch (Exception ex)
{ this.setText(ex.Message, textBox2); }
You don't check how many bytes you actually received in sockets[number].Receive(coming[current], bufferSize, 0); . It doesn't have to be equal to the buffer size you have declared.
Additionaly, as said in comments, keeping whole file in memory is not a good idea.
In addition to checking the number of bytes received, you need to check the number of bytes sent by each send call -- if your TCP transmit window fills up, a send call might send less data than you requested, in which case the you'll need to resend the unsent data.
In general, you ALWAYS need to check the return value of your system calls to check for the all the various odd corner cases that can occur. Read the man pages for send(2) and recv(2) for a full list of everything that can happen.
Related
I have a wpf gui that I want to upload files from and send to a C client .
I want to send 3 files and for some reason 1 of them is being sent and written (but it adds 8 nulls in the end and removes 4 of the first letters in the file)
and in the other two when I try to receive the size it says their size is 0
I've been stuck on this problem for a while now and i'm becoming depserate as i'm probably missing a small thing , if any of u could give a hand that'll mean a lot ! I really wanna know whats the problem in my code.
I have the files paths in an array and sending it in main like so
C# Main
IPEndPoint ipPoint = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 5555);//switch the port
Socket listenSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
listenSocket.Bind(ipPoint);
listenSocket.Listen(1);
Socket clientSocket = listenSocket.Accept();
for (int i = 0; i < 1; i++)
{
SendFile(clientSocket, filePaths[i]);
}
clientSocket.Shutdown(SocketShutdown.Both);
clientSocket.Close();
SendFile(C# side)
public static void SendFile(Socket clientSocket, string filePath)
{
if (File.Exists(filePath))
{
FileInfo fi = new FileInfo(filePath);
long file_size = fi.Length;
byte[] preBuffer;
using (var memoryStream = new MemoryStream())
{
using (BinaryWriter writer = new BinaryWriter(memoryStream))
{
writer.Write(file_size);
}
preBuffer = memoryStream.ToArray();
byte[] fixedBuffer = new byte[4];
Array.Copy(preBuffer, 0, fixedBuffer, 0, 4);
Console.WriteLine(BitConverter.ToString(preBuffer));
Console.WriteLine(BitConverter.ToString(fixedBuffer)); //fixing the problem i had with the converting to array that it added 4 useless zeros.
clientSocket.Send(fixedBuffer); // sending size
}
byte[] data = new Byte[4096];
using (FileStream fs = new FileStream(filePath, FileMode.Open))
{
int actualRead;
do
{
actualRead = fs.Read(data, 0, data.Length);
clientSocket.Send(data);
file_size -= actualRead;
} while (file_size - filePath.Length > 0);
}
}
else
{
MessageBox.Show("File for the program is missing! lua/pcap/csv");
}
}
C Receive(built from 3 functions)
/*
============================================
General : function is responsible for receiving a length of data from the client
Parameters : sock - client socket to receive the data from
*buf - holds a pointer to the buffer that needs to update
bufsize - the length of the buffer
Return Value : returns TRUE when the data is read correctly
else, FALSE when there was a socket error or no bytes are received.
============================================
*/
bool recv_raw(SOCKET sock, void* buf, int bufsize)
{
unsigned char* pbuf = (unsigned char*)buf;
while (bufsize > 0) {
int num = recv(sock, pbuf, bufsize, 0);
if (num <= 0) { return false; }
pbuf += num;
bufsize -= num;
}
return true;
}
/*
===================================================
General : receives the length of the file and updates it
Parameters : sock - client socket to receive the data from
*filesize - holds a pointer to the size of the buffer that needs to update
filesize_len - the length of the file size pointer
Return Value : returns TRUE when the size is read correctly
else, FALSE when there was a socket error or no bytes are received.
===================================================
*/
bool recv_file_len(SOCKET sock, long* filesize)
{
if (!recv_raw(sock, filesize, sizeof(*filesize))) { return false; }
return true;
}
/*
================================================== =
General : writes to the lua file the data from the file
that was received in the socket
Parameters : sock - the socket between the client and server
*f - the file to write the data received to
Return Value : returns TRUE when everything was written to the file.
returns FALSE if there's no data received or detected a socket problem.
================================================== =
*/
bool write_data(SOCKET sock, FILE *f)
{
long filesize;//size of address
char buffer[BUFFER_SIZE];
if (!recv_file_len(sock, &filesize)) { return false; }
printf("file size (From C#) : %ld\n", filesize);
int n = recv_raw(sock, buffer, 8); // need to get the size of the name
if (filesize > 0)
{
do {
int num = min(filesize, BUFFER_SIZE);
if (!recv_raw(sock, buffer, num)) {
return false;
}
int offset = 0;
do
{
size_t written = fwrite(&buffer[offset], 1, num - offset, f);
if (written < 1) { return false; }
offset += written;
} while (offset < num);
filesize -= num;
} while (filesize > 0);
}
return true;
}
C Main
FILE* luafhandler = fopen("test.lua", "wb");//the new lua file
if (luafhandler == NULL)
{
fclose(luafhandler);
printf("GUI CONNECT lua file failed to open!\n");
}
FILE* pcapfhandler = fopen("test.pcap", "wb");//the new lua file
if (pcapfhandler == NULL)
{
fclose(pcapfhandler);
printf("GUI CONNECT pcap file failed to open!\n");
}
FILE* csvfhandler = fopen("AlgoTest.csv", "wb");//the new lua file
if (csvfhandler == NULL)
{
fclose(csvfhandler);
printf("GUI CONNECT csv file failed to open!\n");
}
else {
SOCKET sock1 = open_socket(5555, SERVER_IP);
bool check = write_data(sock1, luafhandler);
bool check1 = write_data(sock1, pcapfhandler);
bool check2 = write_data(sock1, csvfhandler);
fclose(luafhandler);
fclose(pcapfhandler);
fclose(csvfhandler);
}
I transfer a video stream from one c# app to another. When I set the frame rate high I don't get the right data after some time.
//Receiver
Socket s;
IPAddress ipAd = IPAddress.Parse("127.0.0.1");
var ip = IPAddress.Any;
myList = new TcpListener(ipAd, 8001);
myList.Start();
s = myList.AcceptSocket();
while (runThread)
{
int buffertype = -1; MemoryStream img = null; int size = 0; int
pos = 0;
try
{
// length of frame
byte[] lengthb = new byte[32];
int k = s.Receive(lengthb, SocketFlags.None);
size = BitConverter.ToInt32(lengthb, 0);
pos = 0;
byte[] lengthframe = new byte[size];
// chunking of data
int chunksize = 100;
while (pos < size - chunksize)
{
int rec = s.Receive(lengthframe, pos, chunksize, SocketFlags.None);
pos += rec;
}
s.Receive(lengthframe, pos, size - pos, SocketFlags.None);
img = new MemoryStream(lengthframe);
}
}
// Other app: Sender (all is send)
MemoryStream ms = new MemoryStream();
img.CopyTo(ms);
byte[] ba = ms.ToArray();
var buffer = BitConverter.GetBytes(ba.Length);
stm.Write(buffer, 0, buffer.Length);
stm.Write(ba, 0, ba.Length);
How can I be safe that all packages are received?
sender:
stm.Write(buffer, 0, buffer.Length);
receiver:
byte[] lengthb = new byte[32];
int k = s.Receive(lengthb, SocketFlags.None);
are you sure buffer.Length type is 32 bytes long (256 bits)? I suppose it's rather 4 or 8 bytes long. So you're potentially receiving 32 bytes buffer which includes your length field in the beggining and the rest is a part of your encoded video stream. Then you throw away that part. Doesn't sound good.
With windows Azure Storage Client Library, CloudBlob.OpenRead() method reads only 4 mb of data. How can I read full stream using OpenRead method.
CloudBlob blob = container.GetBlobReference(filename);
Stream stream = blob.OpenRead();
I must use OpenRead method. Can't use DownloadToFile or DownloadToStream.
EDIT :
The consumer code which is outside my scope calls
stream.CopyTo(readIntoStream);
CopyTo is an extension method.
public static int CopyTo(this Stream source, Stream destination)
{
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead;
int bytesCopied = 0;
do
{
bytesRead = source.Read(buffer, 0, BUFFER_SIZE);
if (bytesRead > 0)
destination.Write(buffer, 0, bytesRead);
bytesCopied += bytesRead;
}
while (bytesRead > 0);
return bytesCopied;
}
EDIT 2 :
I have observed that when file is uploaded using blob.UploadText() method, it works fine but when blob is uploaded using OpenWrite method as done in following code sample, the OpenRead Method reads only 4194304 bytes (4 mb).
using(var input = File.OpenRead(#"C:\testFile.txt")) //5000000 bytes
using (var output = blob.OpenWrite())
{
input.CopyTo(output);
}
EDIT 3 :
Complete code that produces the issue at my end.
static void Main(string[] args)
{
var blobContainer = GetContainer("tier1");
blobContainer.CreateIfNotExist();
var containerPermissions = new BlobContainerPermissions();
containerPermissions.PublicAccess = BlobContainerPublicAccessType.Blob;
blobContainer.SetPermissions(containerPermissions);
var blob = blobContainer.GetBlobReference("testfileWithOpenWrite1.txt");
using (var input = File.OpenRead(#"C:\testFile.txt")) //5000000 bytes
using (var output = blob.OpenWrite(new BlobRequestOptions()))
input.CopyTo(output);
Console.WriteLine("uploaded");
int bytesDownloaded = 0;
byte[] buffer = new byte[65536];
using (BlobStream bs = blob.OpenRead())
{
int chunkLength;
do
{
chunkLength = bs.Read(buffer, 0, buffer.Length);
bytesDownloaded += chunkLength;
} while (chunkLength > 0);
}
Console.WriteLine(bytesDownloaded);
}
public static int CopyTo(this Stream source, Stream destination)
{
int BUFFER_SIZE = 65536;
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead;
int bytesCopied = 0;
do
{
bytesRead = source.Read(buffer, 0, BUFFER_SIZE);
if (bytesRead > 0)
destination.Write(buffer, 0, bytesRead);
bytesCopied += bytesRead;
}
while (bytesRead > 0);
return bytesCopied;
}
EDIT 4 - Conclusion:
This was probably a bug in Microsoft.WindowsAzure.StorageClient.dll (assembly version 6.0.6002.17043) that comes with SDK v1.2. It works with latest Microsoft.WindowsAzure.StorageClient.dll (assembly version 6.0.6002.18312 - SDK v1.4).
Thanks smarx :)
I can't reproduce what you're seeing. Things seem to work as expected:
static void Main(string[] args)
{
// I also tried a real cloud storage account. Same result.
var container = CloudStorageAccount.DevelopmentStorageAccount.CreateCloudBlobClient().GetContainerReference("testcontainer");
container.CreateIfNotExist();
var blob = container.GetBlobReference("testblob.txt");
blob.UploadText(new String('x', 5000000));
var source = blob.OpenRead();
int BUFFER_SIZE = 4000000;
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead;
int bytesCopied = 0;
do
{
bytesRead = source.Read(buffer, 0, BUFFER_SIZE);
bytesCopied += bytesRead;
} while (bytesRead > 0);
Console.WriteLine(bytesCopied); // prints 5000000
}
EDIT:
I've also (in response to the edited question) now tried uploading the blob using OpenWrite, and the result is the same. (I get the full blob back.) I used this code in place of blob.UploadText(...):
using (var input = File.OpenRead(#"testblob.txt")) //5000000 bytes
using (var output = blob.OpenWrite())
{
input.CopyTo(output);
}
The final WriteLine still prints 5000000.
EDIT 2:
This is getting a bit tiresome. Changing the BUFFER_SIZE to 65536 didn't change anything. The results still seem correct to me. Here's my full application for comparison:
static void Main(string[] args)
{
// I also tried a real cloud storage account. Same result.
var container = CloudStorageAccount.DevelopmentStorageAccount.CreateCloudBlobClient().GetContainerReference("testcontainer");
container.CreateIfNotExist();
var blob = container.GetBlobReference("testblob.txt");
using (var input = File.OpenRead(#"testblob.txt")) //5000000 bytes
using (var output = blob.OpenWrite())
{
input.CopyTo(output);
}
var source = blob.OpenRead();
int BUFFER_SIZE = 65536;
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead;
int bytesCopied = 0;
do
{
bytesRead = source.Read(buffer, 0, BUFFER_SIZE);
bytesCopied += bytesRead;
} while (bytesRead > 0);
Console.WriteLine(bytesCopied); // prints 5000000
}
I'm making a simple file transfer sender and receiver app through the wire. What I have so far is that the sender converts the file into a byte array and sends chunks of that array to the receiver.
This works with file of up to 256mb, but this line throws a "System out of memory" exception for anything above:
byte[] buffer = StreamFile(fileName); //This is where I convert the file
I'm looking for a way to read the file in chunks then write that chunk instead of loading the whole file into a byte. How can I do this with a FileStream?
EDIT:
Sorry, heres my crappy code so far:
private void btnSend(object sender, EventArgs e)
{
Socket clientSock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
byte[] fileName = Encoding.UTF8.GetBytes(fName); //file name
byte[] fileData = null;
try
{
fileData = StreamFile(textBox1.Text); //file
}
catch (OutOfMemoryException ex)
{
MessageBox.Show("Out of memory");
return;
}
byte[] fileNameLen = BitConverter.GetBytes(fileName.Length); //length of file name
clientData = new byte[4 + fileName.Length + fileData.Length];
fileNameLen.CopyTo(clientData, 0);
fileName.CopyTo(clientData, 4);
fileData.CopyTo(clientData, 4 + fileName.Length);
clientSock.Connect("172.16.12.91", 9050);
clientSock.Send(clientData, 0, 4 + fileName.Length, SocketFlags.None);
for (int i = 4 + fileName.Length; i < clientData.Length; i++)
{
clientSock.Send(clientData, i, 1 , SocketFlags.None);
}
clientSock.Close();
}
And here's how I receive (the code was from a tutorial)
public void ReadCallback(IAsyncResult ar)
{
int fileNameLen = 1;
String content = String.Empty;
StateObject state = (StateObject)ar.AsyncState;
Socket handler = state.workSocket;
int bytesRead = handler.EndReceive(ar);
if (bytesRead > 0)
{
if (flag == 0)
{
Thread.Sleep(1000);
fileNameLen = BitConverter.ToInt32(state.buffer, 0);
string fileName = Encoding.UTF8.GetString(state.buffer, 4, fileNameLen);
receivedPath = fileName;
flag++;
}
if (flag >= 1)
{
BinaryWriter writer = new BinaryWriter(File.Open(receivedPath, FileMode.Append));
if (flag == 1)
{
writer.Write(state.buffer, 4 + fileNameLen, bytesRead - (4 + fileNameLen));
flag++;
}
else
writer.Write(state.buffer, 0, bytesRead);
writer.Close();
handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReadCallback), state);
}
}
else
{
Invoke(new MyDelegate(LabelWriter));
}
}
I just really want to know how I can read the file in chunks so that I dont need to convert it to a byte.
Thanks for the responses so far, I think I'm starting to get it :D
Just call Read repeatedly with a small buffer (I tend to use something like 16K). Note that the call to Read may end up reading a smaller amount than you request. If you're using a fixed chunk size and need the whole chunk in memory, you could just use an array of that size of course.
Without knowing how you're sending the file, it's hard to give much advice about how to structure your code, but it could be something like this:
byte[] chunk = new byte[MaxChunkSize];
while (true)
{
int index = 0;
// There are various different ways of structuring this bit of code.
// Fundamentally we're trying to keep reading in to our chunk until
// either we reach the end of the stream, or we've read everything we need.
while (index < chunk.Length)
{
int bytesRead = stream.Read(chunk, index, chunk.Length - index);
if (bytesRead == 0)
{
break;
}
index += bytesRead;
}
if (index != 0) // Our previous chunk may have been the last one
{
SendChunk(chunk, index); // index is the number of bytes in the chunk
}
if (index != chunk.Length) // We didn't read a full chunk: we're done
{
return;
}
}
If I was more awake I'd probably find a more readable way of writing this, but it'll do for now. One option is to extract another method from the middle section:
// Attempts to read an entire chunk into the given array; returns the size of
// chunk actually read.
int ReadChunk(Stream stream, byte[] chunk)
{
int index = 0;
while (index < chunk.Length)
{
int bytesRead = stream.Read(chunk, index, chunk.Length - index);
if (bytesRead == 0)
{
break;
}
index += bytesRead;
}
return index;
}
var b = new byte[1<<15]; // 32k
while((count = inStream.Read(b, 0, b.Length)) > 0)
{
outStream.Write(b, 0, count);
}
public static IEnumerable<byte[]> SplitStreamIntoChunks(Stream stream, int chunkSize)
{
var bytesRemaining = stream.Length;
while (bytesRemaining > 0)
{
var size = Math.Min((int) bytesRemaining, chunkSize);
var buffer = new byte[size];
var bytesRead = stream.Read(buffer, 0, size);
if (bytesRead <= 0)
break;
yield return buffer;
bytesRemaining -= bytesRead;
}
}
int bufferlength = 12488;
int pointer = 1;
int offset = 0;
int length = 0;
FileStream fstwrite = new FileStream("D:\\Movie.wmv", FileMode.Create);
while (pointer != 0)
{
byte[] buff = new byte[bufferlength];
FileStream fst = new FileStream("E:\\Movie.wmv", FileMode.Open);
pointer = fst.Read(buff, 0, bufferlength);
fst.Close();
fstwrite.Write(buff, offset , pointer);
offset += pointer;
}
I used the above code for splitting a file and place it in other drive.Im not able to set the correct offset and length for this routine can anyone help me to fix this
splitting in the sense ,i split it in "x" kbs and pass it somewhere make the same file in some other location
I find it atlast ,thanks to evry one who gave their valueble responses.
Currently you're always reading from the start of the file... and even if you weren't you'd just be copying the whole file.
Here's some code which will actually split a single file into multiple files:
public static void SplitFile(string inputFile,
string outputPrefix,
int chunkSize)
{
byte[] buffer = new byte[chunkSize];
using (Stream input = File.OpenRead(inputFile))
{
int index = 0;
while (input.Position < input.Length)
{
using (Stream output = File.Create(outputPrefix + index))
{
int chunkBytesRead = 0;
while (chunkBytesRead < chunkSize)
{
int bytesRead = input.Read(buffer,
chunkBytesRead,
chunkSize - chunkBytesRead);
// End of input
if (bytesRead == 0)
{
break;
}
chunkBytesRead += bytesRead;
}
output.Write(buffer, 0, chunkBytesRead);
}
index++;
}
}
}
Your reading bufferlength of bytes. Shouldn't you set the offset like this then?
offset += bufferlength;
Don't open your source file inside the loop, or you'll always read the first chunk.
Open it before the loop, then make sure your offset is applied to the read.