I am transferring files using c#. I have used this code. The problem is small files like .txt files are transferred correctly but not big files like images, documents, pdf, ppt. Sometimes code works fine but most of the times it transfers less amount of data.
Server Code:
Socket clientSock = sock.Accept();
byte[] clientData = new byte[1024 * 50000];
int receivedBytesLen = clientSock.Receive(clientData);
int fileNameLen = BitConverter.ToInt32(clientData, 0);
string fileName = Encoding.ASCII.GetString(clientData, 4, fileNameLen);
BinaryWriter bWrite = new BinaryWriter(File.Open(receivedPath + "/" + fileName, FileMode.Append));
bWrite.Write(clientData, 4 + fileNameLen, receivedBytesLen - 4 - fileNameLen);
bWrite.Close();
clientSock.Close();
Client Code:
byte[] fileNameByte = Encoding.ASCII.GetBytes(fileName);
byte[] fileData = File.ReadAllBytes(filePath + fileName);
byte[] clientData = new byte[4 + fileNameByte.Length + fileData.Length];
byte[] fileNameLen = BitConverter.GetBytes(fileNameByte.Length);
fileNameLen.CopyTo(clientData, 0);
fileNameByte.CopyTo(clientData, 4);
fileData.CopyTo(clientData, 4 + fileNameByte.Length);
clientSock.Connect(ipEnd);
clientSock.Send(clientData);
clientSock.Close();
Complete code is given in the above link. I have also seen this post but this is not helpful.
Sometimes code works fine but most of the times it transfers less amount of data.
That's the nature of Socket.Receive(), it doesn't always return all data that gets sent to it.
You'll have to do a Receive(clientData, 4, 0) first to receive the bytes that indicate the size , then call Receive(clientData) in a loop until you've received size bytes. But beware that the Receive(buffer[], length, offset) overload just as easily as any other overload can return less than the expected amount of bytes. So you'll also have to call that in a loop:
Something like this:
// First receive the size
int sizeSize = 4; // Size of Int32 in bytes
int sizeOffset = 0;
var sizeBytes = new byte[sizeSize];
while (sizeOffset < sizeSize)
{
sizeOffset += clientSocket.Receive(sizeBytes, sizeSize - sizeOffset, sizeOffset);
}
var size = BitConverter.ToInt32(sizeBytes, 0);
// Then receive the data
byte[] fileData = new byte[size];
byte[] clientData = new byte[8192];
int totalBytes = 0;
while (totalBytes < size)
{
// This may return anything between 0 and 8192, even if not all sent data has been received yet. It may be in a buffer somewhere, waiting to be picked up. Check for 0, since that's when the client disconnects.
int bytesReceived = clientSocket.Receive(clientData);
// You now have received a chunk of data of bytesReceived length. Append it into the fileData array.
Buffer.BlockCopy(clientData, 0, fileData, totalBytes, bytesReceived);
totalBytes += bytesReceived;
}
As CodeCaster's answered that Socket.Receive(), it doesn't always return all data that gets sent to it. This is 100% correct and tested but the next step of sending the file size is not working, I found an easy and correct solution.
Socket.Receive() method takes the byte array in which received data will be copied and returns the number of bytes received. So we can easily loop it till bytes received are 0.
byte[] tempData = new byte[1024 * 5000];
BinaryWriter bWrite = null;
int bytes_received;
int fileNamelength = 0;
bool isFirstPacket = true;
do
{
bytes_received = clientSock.Receive(tempData);
if(isFirstPacket)
{
isFirstPacket = false;
int fileNameLen = BitConverter.ToInt32(tempData, 0);
string fileName = Encoding.ASCII.GetString(clientData, 4, fileNameLen);
bWrite = new BinaryWriter(File.Open(receivedPath + fileName, FileMode.Append))
bWrite.Write(tempData, 4 + fileNameLength, bytes_received - 4 - fileNamelength);
}
else
bWrite.Write(tempData, 0, bytes_received);
}
while (bytes_received != 0);
Related
Could someone be kind enough to explain how I get my files the same size after copying it using a chunked stream? I presume it is because the last chunk still has a buffer size of 2048 so it is putting empty bytes at the end, but I am unsure how I would adjust the last read?
Original size: 15.1 MB (15,835,745 bytes)
New size: 15.1 MB (15,837,184 bytes)
static FileStream incomingFile;
static void Main(string[] args)
{
incomingFile = new FileStream(
#"D:\temp\" + Guid.NewGuid().ToString() + ".png",
FileMode.Create,
FileAccess.Write);
FileCopy();
}
private static void FileCopy()
{
using (Stream source = File.OpenRead(#"D:\temp\test.png"))
{
byte[] buffer = new byte[2048];
var chunkCount = source.Length;
for (int i = 0; i < (chunkCount / 2048) + 1; i++)
{
source.Position = i * 2048;
source.Read(buffer, 0, 2048);
WriteFile(buffer);
}
incomingFile.Close();
}
}
private static void WriteFile(byte[] buffer)
{
incomingFile.Write(buffer, 0, buffer.Length);
}
The last buffer read does not necessary contain exactly 2048 bytes (it can well be incomplete). Imagine, we have a file of 5000 bytes; in this case will read 3 chunks: 2 complete and 1 incomplete
2048
2048
904 the last incomplete buffer
Code:
using (Stream source = File.OpenRead(#"D:\temp\test.png"))
{
byte[] buffer = new byte[2048];
var chunkCount = source.Length;
for (int i = 0; i < (chunkCount / 2048) + 1; i++)
{
source.Position = i * 2048;
// size - number of actually bytes read
int size = source.Read(buffer, 0, 2048);
// if we bytes to write, do it
if (size > 0)
WriteFile(buffer, size);
}
incomingFile.Close();
}
...
private static void WriteFile(byte[] buffer, int size = -1)
{
incomingFile.Write(buffer, 0, size < 0 ? buffer.Length : size);
}
In your case you write 15837184 == 7733 * 2048 bytes (7733 complete chunks) when you should write 15835745 == 7732 * 2048 + 609 bytes - 7732 complete chunks and the last incomplete one of 609 bytes
TCP is stream-based protocol. To convert that stream into my messages, I send the size of each message with the message itself. At server side, I first read the first two bytes of message, which have the size. Then I create a byte array, of size equal to the size which was just read. Then I read the bytes into that array. But for some reason, more bytes are being read than specified. How can I read exactly the same number of bytes as I specify?
Here is my code:
while (true)
{
data = null;
length = null;
size = new byte[2];
handler.Receive(size);
length += Encoding.ASCII.GetString(size, 0, 2);
System.Console.WriteLine("Size: " + Int32.Parse(length));
bufferSize = Int32.Parse(length) + 2;
bytes = new byte[bufferSize];
handler.Receive(bytes);
data += Encoding.ASCII.GetString(bytes, 0, bufferSize);
System.Console.WriteLine("Data: " + data);
}
This is my server running in Windows PC, written in C#. My client is running in android phone, written in Java.
It's unclear why you're adding two to the size that's been transmitted - you've already accounted for the two additional bytes for storing the length during your previous receive. So I'd get rid of the +2.
You also need to respect the fact already stated in your question - TCP is a sequence of bytes, not messages. As such, you're never guaranteed whether a call to Receive is going to retrieve an entire "message" or just part of one (or, possible, parts of multiple messages). As such, you need to make sure that you respect the return value from Receive.
We can probably re-write your code as:
while (true)
{
data = null;
length = null;
size = ReceiveExactly(handler,2);
length = Encoding.ASCII.GetString(size, 0, 2); //Why +=?
bufferSize = Int32.Parse(length); //Why + 2?
System.Console.WriteLine("Size: " + bufferSize);
bytes = ReceiveExactly(handler,bufferSize);
data += Encoding.ASCII.GetString(bytes, 0, bufferSize);
System.Console.WriteLine("Data: " + data);
}
Where ReceiveExactly is defined something like this:
private byte[] ReceiveExactly(Socket handler, int length)
{
var buffer = new byte[length];
var receivedLength = 0;
while(receivedLength < length)
{
var nextLength = handler.Receive(buffer,receivedLength,length-receivedLength);
if(nextLength==0)
{
//Throw an exception? Something else?
//The socket's never going to receive more data
}
receivedLength += nextLength;
}
return buffer;
}
to receive a specific amount of bytes use the method
Socket.Receive(Byte[], Int32, Int32, SocketFlags)
rather than Socket.Receive(Byte[]). see spec here
I suspect you want something like
int len = Socket.Receive(bytes, 0, bufferSize, SocketFlags.None);
data += Encoding.ASCII.GetString(bytes, 0, len);
System.Console.WriteLine("Data: " + data);
I am currently working on a networking project where I worked out a binary protocol. My packets look like this:
[1 byte TYPE][2 bytes INDEX][2 bytes LENGTH][LENGTH bytes DATA]
And here's the code where I am receiving the packets:
NetworkStream clientStream= Client.GetStream();
while (Client.Connected)
{
Thread.Sleep(10);
try
{
if (clientStream.DataAvailable)
{
byte[] infobuffer = new byte[5];
int inforead = clientStream.Read(infobuffer, 0, 5);
if (inforead < 5) { continue; }
byte[] rawclient = new byte[2];
Array.Copy(infobuffer, 1, rawclient, 0, 2);
PacketType type = (PacketType)Convert.ToSByte(infobuffer[0]);
int clientIndex = BitConverter.ToInt16(rawclient, 0);
int readLength = BitConverter.ToInt16(infobuffer, 3);
byte[] readbuffer = new byte[readLength];
int count_read = clientStream.Read(readbuffer, 0, readLength);
byte[] read_data = new byte[count_read];
Array.Copy(readbuffer, read_data, count_read);
HandleData(read_data, type, clientIndex);
}
}
catch (Exception ex)
{
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine("[E] " + ex.GetType().ToString());
Console.ResetColor();
break;
}
}
Well, and everything works fine... as long as I run it on 127.0.0.1. As soon as I try testing it over long distance, packets somehow get lost, and I am getting an overflow-exception on the line where I convert the first byte to PacketType. Also, if I try to convert the other values to int16, I get very strange values.
I assume the stream somehow looses some bytes on its way to the server, but can this be? Or is it just a little mistake of mine somewhere in the code?
edit:
I now edited the code, now it reads till it gets its 5 bytes. But I still get the same exception over long distance...
NetworkStream clientStream = Client.GetStream();
while (Client.Connected)
{
Thread.Sleep(10);
try
{
if (clientStream.DataAvailable)
{
int totalread = 0;
byte[] infobuffer = new byte[5];
while (totalread < 5)
{
int inforead = clientStream.Read(infobuffer, totalread, 5 - totalread);
if (inforead == 0)
{ break; }
totalread += inforead;
}
byte[] rawclient = new byte[2];
Array.Copy(infobuffer, 1, rawclient, 0, 2);
PacketType type = (PacketType)Convert.ToSByte(infobuffer[0]);
int clientIndex = BitConverter.ToInt16(rawclient, 0);
int readLength = BitConverter.ToInt16(infobuffer, 3);
byte[] readbuffer = new byte[readLength];
int count_read = clientStream.Read(readbuffer, 0, readLength);
byte[] read_data = new byte[count_read];
Array.Copy(readbuffer, read_data, count_read);
HandleData(read_data, type, clientIndex);
}
}
catch (Exception ex)
{
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine("[E] " + ex.GetType().ToString());
Console.ResetColor();
break;
}
}
PacketType is an enum:
public enum PacketType
{
AddressSocks5 = 0,
Status = 1,
Data = 2,
Disconnect = 3,
AddressSocks4 = 4
}
So many things you're doing wrong here... so many bugs... where to even start...
First Network polling? Really? That's just a naïve way of doing network activity in this day and age.. but I won't harp on that.
Second, with this type of protocol, it's pretty easy to get "out of sync" and once you do, you have no way to get back in sync. This is typically accomplished with some kind of "framing protocol" which provides a unique sequence of bytes that you can use to indicate the start and end of a frame, so that if you ever find yourself out of sync you can read data until you get back in sync. Yes, you will lose data, but you've already lost it if you're out of sync.
Third, you're not really doing anything huge here, so I shamelessly stole the "ReadWholeArray" code from here, it's not the most efficient, but it works and there is other code there that might help:
http://www.yoda.arachsys.com/csharp/readbinary.html
Note: you don't mention how you are serializing the length, type and index values on the other side. So using the BitConverter may be the wrong thing depending on how that was done.
if (clientStream.DataAvailable)
{
byte[] data = new byte[5];
// if it can't read all 5 bytes, it throws an exception
ReadWholeArray(clientStream, data);
PacketType type = (PacketType)Convert.ToSByte(data[0]);
int clientIndex = BitConverter.ToInt16(data, 1);
int readLength = BitConverter.ToInt16(data, 3);
byte[] rawdata = new byte[readLength];
ReadWholeArray(clientStream, rawdata);
HandleData(rawdata, type, clientIndex);
}
/// <summary>
/// Reads data into a complete array, throwing an EndOfStreamException
/// if the stream runs out of data first, or if an IOException
/// naturally occurs.
/// </summary>
/// <param name="stream">The stream to read data from</param>
/// <param name="data">The array to read bytes into. The array
/// will be completely filled from the stream, so an appropriate
/// size must be given.</param>
public static void ReadWholeArray (Stream stream, byte[] data)
{
int offset=0;
int remaining = data.Length;
while (remaining > 0)
{
int read = stream.Read(data, offset, remaining);
if (read <= 0)
throw new EndOfStreamException
(String.Format("End of stream reached with {0} bytes left to read", remaining));
remaining -= read;
offset += read;
}
}
I think the problem is in these lines
int inforead = clientStream.Read(infobuffer, 0, 5);
if (inforead < 5) { continue; }
what happen to your previously read data if the length is under 5 byte? you should save the bytes you have read so far and append next bytes so you can have the header completely
You Read 5 - totalRead.
let totalRead equal 5 or more. When that happens you read nothing, and in cases of 1 - 4 you read that many arbitrary bytes. Not 5. You also then discard any result of less then 5.
You also copy at a offset 1 or another offset without really knowing the offset.
BitConverter.ToInt16(infobuffer, 3);
Is an example of this, what is at off 2?
So if it's not that (decoding error) and and not the structure of your data then unless you change the structure of your loop its you who's losing the bytes not the NetworkStream.
Calculate totalRead by increments of justRead when you recieve so you can handle any size of data as well as receiving it at the correct offset.
static void Main(string[] args)
{
FileStream fs = File.Open(#"C:\Skrillex - Rock n' Roll (Will Take You to the Mountain).mp3", FileMode.Open);
BinaryReader br = new BinaryReader(fs);
byte[] tag = new byte[3];
byte[] version = new byte[2];
byte[] flags = new byte[1];
byte[] size = new byte[4];
byte[] frameId = new byte[4];
byte[] frameSize = new byte[4];
byte[] frameFlags = new byte[2];
br.Read(tag, 0, tag.Length);
br.Read(version, 0, version.Length);
br.Read(flags, 0, flags.Length);
br.Read(size, 0, size.Length);
br.Read(frameId, 0, frameId.Length);
br.Read(frameSize, 0, frameSize.Length);
br.Read(frameFlags, 0, frameFlags.Length);
ulong iSize = (ulong)frameSize[0] << 21 | (ulong)frameSize[1] << 14 | (ulong)frameSize[2] << 7 | (ulong)frameSize[3];
Console.WriteLine("Frame Data Size : " + iSize.ToString());
byte[] body = new byte[iSize];
br.Read(body, 0, body.Length);
Console.WriteLine(BitConverter.ToString(body));
Console.WriteLine(ConvertHexToString(BitConverter.ToString(body)));
br.Close();
}
public string ConvertHexToString(string HexValue)
{
string StrValue = "";
HexValue = HexValue.Replace("-", "");
while (HexValue.Length > 0)
{
StrValue += Convert.ToChar(Convert.ToUInt32(HexValue.Substring(0, 2), 16)).ToString();
HexValue = HexValue.Substring(2, HexValue.Length - 2);
}
return StrValue;
}
I am writing the code for reading ID3v2.3 tags without external library or Shell32.
The above code is that code, but it seems not to work properly.
The following is the result when I run the code:
Frame Data Size : 91
01-FF-FE-52-00-6F-00-63-00-6B-00-20-00-6E-00-27-00-20-00-52-00-6F-00-6C-00-6C-00-20-00-28-> 00-57-00-69-00-6C-00-6C-00-20-00-54-00-61-00-6B-00-65-00-20-00-59-00-6F-00-75-00-20-00-74-> 00-6F-00-20-00-74-00-68-00-65-00-20-00-4D-00-6F-00-75-00-6E-00-74-00-61-00-69-00-6E-00-29-00
ÿþR
It is not returning the song title "Rock n' Roll (Will Take You to the Mountain)" that was recorded in the tag.
What is problem?
The 01 at the start indicates that it is encoded as UTF-16 (2 bytes per character). The next two bytes, FF FE, are the byte order mark so you can tell whether to interpret the byte pairs as most significant first or least significant first. After that you have the actual text data.
0052 - R
006F - o
0063 - c
006B - k
etc.
Hello I'm doing an encryption algorithm which reads bytes from file (any type) and outputs them into a file. The problem is my encryption program takes only blocks of 16 bytes so if the file is bigger it has to be split into blocks of 16, or if there's a way to read 16 bytes from the file each time it's fine.
The algorithm is working fine with hard coded input of 16 bytes. The ciphered result has to be saved in a list or array because it has to be deciphered the same way later. I can't post all my program but here's what I do in main so far and cannot get results
static void Main(String[] args)
{
byte[] bytes = File.ReadAllBytes("path to file");
var stream = new StreamReader(new MemoryStream(bytes));
byte[] cipherText = new byte[16];
byte[] decipheredText = new byte[16];
Console.WriteLine("\nThe message is: ");
Console.WriteLine(stream.ReadToEnd());
AES a = new AES(keyInput);
var list1 = new List<byte[]>();
for (int i = 0; i < bytes.Length; i+=16)
{
a.Cipher(bytes, cipherText);
list1.Add(cipherText);
}
Console.WriteLine("\nThe resulting ciphertext is: ");
foreach (byte[] b in list1)
{
ToBytes(b);
}
}
I know that my loops always add the first 16 bytes from the byte array but I tried many ways and nothing work. It won't let me index the bytes array or copy an item to a temp variable like temp = bytes[i]. The ToBytes method is irrelevant, it just prints the elements as bytes.
I would like to recommend you to change the interface for your Cipher() method: instead of passing the entire array, it would be better to pass the source and destination arrays and offset - block by block encryption.
Pseudo-code is below.
void Cipher(byte[] source, int srcOffset, byte[] dest, int destOffset)
{
// Cipher these bytes from (source + offset) to (source + offset + 16),
// write the cipher to (dest + offset) to (dest + offset + 16)
// Also I'd recommend to check that the source and dest Length is less equal to (offset + 16)!
}
Usage:
For small files (one memory allocation for destination buffer, block by block encryption):
// You can allocate the entire destination buffer before encryption!
byte[] sourceBuffer = File.ReadAllBytes("path to file");
byte[] destBuffer = new byte[sourceBuffer.Length];
// Encrypt each block.
for (int offset = 0; i < sourceBuffer.Length; offset += 16)
{
Cipher(sourceBuffer, offset, destBuffer, offset);
}
So, the main advantage of this approach - it elimitates additional memory allocations: the destination array is allocated at once. There is also no copy-memory operations.
For files of any size (streams, block by block encryption):
byte[] inputBlock = new byte[16];
byte[] outputBlock = new byte[16];
using (var inputStream = File.OpenRead("input path"))
using (var outputStream = File.Create("output path"))
{
int bytesRead;
while ((bytesRead = inputStream.Read(inputBlock, 0, inputBlock.Length)) > 0)
{
if (bytesRead < 16)
{
// Throw or use padding technique.
throw new InvalidOperationException("Read block size is not equal to 16 bytes");
// Fill the remaining bytes of input block with some bytes.
// This operation for last block is called "padding".
// See http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation#Padding
}
Cipher(inputBlock, 0, outputBlock, 0);
outputStream.Write(outputBlock, 0, outputBlock.Length);
}
}
No need to read the whole mess into memory if you can only process it a bit at a time...
var filename = #"c:\temp\foo.bin";
using(var fileStream = new FileStream(filename, FileMode.Open))
{
var buffer = new byte[16];
var bytesRead = 0;
while((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) > 0)
{
// do whatever you need to with the next 16-byte block
Console.WriteLine("Read {0} bytes: {1}",
bytesRead,
string.Join(",", buffer));
}
}
You can use Array.Copy
byte[] temp = new byte[16];
Array.Copy(bytes, i, temp, 0, 16);