I am try to do some code using BinaryWriter and Then BinaryReader.
When I wanna write I use method Write().
But the problem is that between two lines of Write method there appears a new byte which is in ASCII table in decimal 31 (sometines 24).
You can see it on this image:
You can see that byte at index 4 (5th byte) is of ASCII decimal value 31. I didnt insert it there. As you can see 1st 4 bytes are reserved for a number (Int32), next are other data (some text mostly - this is not important now).
As you can see from the code i write:
- into 1st line a number 10
- into 2nd line text "This is some text..."
How come came that 5th byte (dec 31) in between??
And this is the code I have:
static void Main(string[] args)
{
//
//// SEND - RECEIVE:
//
SendingData();
Console.ReadLine();
}
private static void SendingData()
{
int[] commandNumbers = { 1, 5, 10 }; //10 is for the users (when they send some text)!
for (int i = 0; i < commandNumbers.Length; i++)
{
//convert to byte[]
byte[] allBytes;
using (MemoryStream ms = new MemoryStream())
{
using (BinaryWriter bw = new BinaryWriter(ms))
{
bw.Write(commandNumbers[i]); //allocates 1st 4 bytes - FOR MAIN COMMANDS!
if (commandNumbers[i] == 10)
bw.Write("This is some text at command " + commandNumbers[i]); //HERE ON THIS LINE IS MY QUESTION!!!
}
allBytes = ms.ToArray();
}
//convert back:
int valueA = 0;
StringBuilder sb = new StringBuilder();
foreach (var b in GetData(allBytes).Select((a, b) => new { Value = a, Index = b }))
{
if (b.Index == 0) //1st num
valueA = BitConverter.ToInt32(b.Value, 0);
else //other text
{
foreach (byte _byte in b.Value)
sb.Append(Convert.ToChar(_byte));
}
}
if (sb.ToString().Length == 0)
sb.Append("ONLY COMMAND");
Console.WriteLine("Command = {0} and Text is \"{1}\".", valueA, sb.ToString());
}
}
private static IEnumerable<byte[]> GetData(byte[] data)
{
using (MemoryStream ms = new MemoryStream(data))
{
using (BinaryReader br = new BinaryReader(ms))
{
int j = 0;
byte[] buffer = new byte[4];
for (int i = 0; i < data.Length; i++)
{
buffer[j++] = data[i];
if (i == 3) //SENDING COMMAND DATA
{
yield return buffer;
buffer = new byte[1];
j = 0;
}
else if (i > 3) //SENDING TEXT
{
yield return buffer;
j = 0;
}
}
}
}
}
If you look at the documentation for Write(string), you'll see that it writes a length-prefixed string. So the 31 is the number of characters in your string -- perfectly normal.
You should probably be using Encoding.GetBytes and then write the bytes instead of writing a string
for example
bw.Write(
Encoding.UTF8.GetBytes("This is some text at command " + commandNumbers[i])
);
When a string is written to a binary stream, the first thing it does is write the length of the string. The string "This is some text at command 10" has 31 characters, which is the value you're seeing.
You should check the documentation of methods you use before asking questions about them:
A length-prefixed string represents the string length by prefixing to
the string a single byte or word that contains the length of that
string. This method first writes the length of the string as a UTF-7
encoded unsigned integer, and then writes that many characters to the
stream by using the BinaryWriter instance's current encoding.
;-)
(Though in fact it is an LEB128 and not UTF-7, according to Wikipedia).
The reason this byte is there because you're adding a variable amount of information, so the length is needed. If you were to add two strings, where would you know where the first ended and the second began?
If you really don't want or need that length byte, you can always convert the string to a byte array and use that.
Ok, here is my edited code. I removed BinaryWriter (while BinaryReader is still there!!), and now it works very well - no more extra bytes.
What do you thing? Is there anytihng to do better, to make it run faster?
Expecially Im interesting for that foreach loop, which read from another method that is yield return type!!
New Code:
static void Main(string[] args)
{
//
//// SEND - RECEIVE:
//
SendingData();
Console.ReadLine();
}
private static void SendingData()
{
int[] commands = { 1, 2, 3 };
// 1 - user text
// 2 - new game
// 3 - join game
// ...
for (int i = 0; i < commands.Length; i++)
{
//convert to byte[]
byte[] allBytes;
using (MemoryStream ms = new MemoryStream())
{
// 1.st - write a command:
ms.Write(BitConverter.GetBytes(commands[i]), 0, 4);
// 2nd - write a text:
if (commands[i] == 1)
{
//some example text (like that user sends it):
string myText = "This is some text at command " + commands[i];
byte[] myBytes = Encoding.UTF8.GetBytes(myText);
ms.Write(myBytes, 0, myBytes.Length);
}
allBytes = ms.ToArray();
}
//convert back:
int valueA = 0;
StringBuilder sb = new StringBuilder();
foreach (var b in ReadingData(allBytes).Select((a, b) => new { Value = a, Index = b }))
{
if (b.Index == 0)
{
valueA = BitConverter.ToInt32(b.Value, 0);
}
else
{
sb.Append(Convert.ToChar(b.Value[0]));
}
}
if (sb.ToString().Length == 0)
sb.Append("ONLY COMMAND");
Console.WriteLine("Command = {0} and Text is \"{1}\".", valueA, sb.ToString());
}
}
private static IEnumerable<byte[]> ReadingData(byte[] data)
{
using (MemoryStream ms = new MemoryStream(data))
{
using (BinaryReader br = new BinaryReader(ms))
{
int j = 0;
byte[] buffer = new byte[4];
for (int i = 0; i < data.Length; i++)
{
buffer[j++] = data[i];
if (i == 3) //SENDING COMMAND DATA
{
yield return buffer;
buffer = new byte[1];
j = 0;
}
else if (i > 3) //SENDING TEXT
{
yield return buffer;
j = 0;
}
}
}
}
}
Related
I read a file as binary, convert to hex string, convert back to binary, and write to a new file.
I expect a duplicate, but get a corrupted file.
I have been trying different ways to convert the binary into the hex string but can't seem to retain the entire file.
byte[] binary1 = File.ReadAllBytes(#"....Input.jpg");
string hexString = "";
int counter1 = 0;
foreach (byte b in binary1)
{
counter1++;
hexString += (Convert.ToString(b, 16));
}
List<byte> bytelist = new List<byte>();
int counter2 = 0;
for (int i = 0; i < hexString.Length/2; i++)
{
counter2++;
string ch = hexString.Substring(i*2,2);
bytelist.Add(Convert.ToByte(ch, 16));
}
byte[] binary2 = bytelist.ToArray();
File.WriteAllBytes(#"....Output.jpg", binary2);
Counter 1 and counter 2 should be the same count, but counter 2 is always about 10% smaller.
I want to get a duplicate file output that I could have transferred around via that string value.
Converting byte 10 will give a single char, and not 2 characters. Your convert back method (logically) build on 2 chars per byte.
this case works
byte[] binary1 = new byte[] { 100 }; // convert will result in "64"
and this case fails
byte[] binary1 = new byte[] { 10 }; // convert will result in "a"
I quick fixed your code, by padding with a "0" in case of a single char.
so working code:
byte[] binary1 = new byte[] { 100 };
string hexString = "";
int counter1 = 0;
foreach (byte b in binary1)
{
counter1++;
var s = (Convert.ToString(b, 16));
// new
if (s.Length < 2)
{
hexString += "0";
}
// end new
hexString += s;
}
List<byte> bytelist = new List<byte>();
int counter2 = 0;
for (int i = 0; i < hexString.Length / 2; i++)
{
counter2++;
string ch = hexString.Substring(i * 2, 2);
var item = Convert.ToByte(ch, 16);
bytelist.Add(item);
}
byte[] binary2 = bytelist.ToArray();
Please note, your code could use some refactoring, e.g. don't string concat in a loop and maybe check the Single Responsibility Principle.
Update, got it fixed, but there are better solutions here: How do you convert a byte array to a hexadecimal string, and vice versa?
I have the following function:
public void SetTagData(string _data)
{
string data = _data;
byte[] ba = Encoding.Default.GetBytes(data);
string hexString = BitConverter.ToString(ba);
hexString = hexString.Replace("-", "");
var blockStart = 0;
var bufferHexBlocks = String.Empty;
try
{
for (var i = 0; i < hexString.Length; i++)
{
var byteList = new List<byte>();
byte[] datablockKey = ConvertHelpers.ConvertHexStringToByteArray(i.ToString().PadLeft(2, '0'));
var block = hexString.Substring(blockStart, 8);
byte[] datablockValue = ConvertHelpers.ConvertHexStringToByteArray(block);
byteList.AddRange(datablockKey);
byteList.AddRange(datablockValue);
_reader.Protocol("wb", byteList.ToArray());
blockStart += 8;
}
}
catch (Exception ex)
{
console.log(ex.message);
}
}
The data coming in is a bunch of hex as a string. I need to split this hex string into batches of 8 characters, append an incrementing 0 padded hex number from 00 to 1f and send this new string as a byte array to the _reader.Protocol function, which accepts a string wb as first parameter and the block as the second.
For example incoming data is:
string data = "3930313B36313B5350542D53504C3B3830303B3B352E373B3B303B303B3B3B34353036383B4E3B4E3B"
I need to send the following to the _reader.Protocol object:
(incremented padded hex 01, 02, 03, ... , 0f) and the first 8 characters of the data string, then the next, and so on as a byte array.
[013930313B], [0236313B53], etc.
I think I'm getting close... but missing something...
My problem at the moment is that I can't figure out how to loop in blocks of 8 and if the hex string is say 82 characters instead of 80 (multiple of 8), then how would I grab the last two characters without getting a IndexOutofRange exception.
Note: This is for a Windows CE application, so no new C# features please.
This below will work fine in conjunction with this answer and the sample string data given.
public static byte[] Parse(string data)
{
var count = data.Length / 8; //Might be worth throwing exception with any remainders unless you trust the source.
var needle = 0;
List<byte> result = new List<byte>(); //Inefficient but I'm being lazy
for (int i = 0; i < count; i++)
{
char[] buffer = new char[8];
data.CopyTo(needle, buffer, 0, buffer.Length);
//To get around the odd number when adding the prefixed count byte, send the hex string to the convert method separately.
var bytes = ConvertHexStringToByteArray(new string(buffer)); //Taken From https://stackoverflow.com/a/8235530/6574422
//As the count is less than 255, seems safe to parse to single byte
result.Add(byte.Parse((i + 1).ToString()));
result.AddRange(bytes);
needle += 8;
}
return result.ToArray();
}
I'm figured it out. It might not be the most efficient solution but it works just fine. I did it using a for loop inside a for loop.
In case anyone is interested here is the final code:
public void SetTagData(string _data)
{
string data = _data;
byte[] ba = Encoding.Default.GetBytes(data);
string hexString = BitConverter.ToString(ba);
hexString = hexString.Replace("-", "");
var blockStart = 0;
try
{
_reader.Protocol("s");
for(var count = 0; count < 16; count++)
{
var byteList = new List<byte>();
byte[] datablockKey = ConvertHelpers.ConvertHexStringToByteArray(count.ToString("X2"));
byteList.AddRange(datablockKey);
for (var innerCount = 0; innerCount < 4; innerCount++)
{
var block = String.Empty;
if (!String.IsNullOrEmpty(hexString.Substring(blockStart, 2)))
{
block = hexString.Substring(blockStart, 2);
}
else
{
block = "20";
}
byte[] datablockValue = ConvertHelpers.ConvertHexStringToByteArray(block);
byteList.AddRange(datablockValue);
blockStart += 2;
}
_reader.Protocol("wb", byteList.ToArray());
}
}
catch (Exception)
{
}
}
I read binary file to hex by block.
It is diffrent when I use FileStream.Read and File.ReadAllBytes
FileSteram.Read
int limit = 0;
if (openFileDlg.FileName.Length > 0)
{
fileName = openFileDlg.FileName;
FileStream fs = new FileStream(fileName, FileMode.Open, FileAccess.Read);
fsLen = (int)fs.Length;
int count = 0;
limit = 100;
byte[] read_buff = new byte[limit];
StringBuilder sb = new StringBuilder();
while ( (count = fs.Read(read_buff, 0, limit)) > 0)
{
foreach (byte b in read_buff)
{
sb.Append(Convert.ToString(b, 16).PadLeft(2, '0'));
}
}
rtxb_bin.AppendText(sb.ToString() + "\n");
}
File.ReadAllBytes
if (openFileDlg.FileName.Length > 0)
{
fileName = openFileDlg.FileName;
byte[] fileBytes = File.ReadAllBytes(fileName);
StringBuilder sb2 = new StringBuilder();
foreach (byte b2 in fileBytes)
{
sb2.Append(Convert.ToString(b2, 16).PadLeft(2, '0'));
}
rtxb_allbin.AppendText(sb2.ToString());
}
case 1, reasult is ...
........04c0020f00452a00421346108129844f2138448500208020250405250043188510812e0
and case 2 is
.......04c0020f00452a00421346108129844f2138448500208020250405250043188510812e044f212cc48120c24125404f2069c2c0008bff35f8f401efbd17047
FileStream.Read doesn't read after '12e0'
'44f212cc48120c24125404f2069c2c0008bff35f8f401efbd17047' is missing
How can I read all bytes using FileStream.Read?
Why FileStream.Read doesn't read last block?
Most likely it appears to you that it does not read last block. Suppose you have file of length 102. First iteration of you loop reads first 100 bytes, all is fine. But what happens on second (last) one? You read two bytes into read_buff, which is of length 100. Now that buffer contains 2 bytes of last block and 98 bytes of previous (first) block, because Read doesn't clear the buffer. Then you proceed with:
foreach (byte b in read_buff)
{
sb.Append(Convert.ToString(b, 16).PadLeft(2, '0'));
}
In result, sb has 100 bytes of first block, 2 bytes of last block, and then again 98 bytes of first block. If you don't look too closely, it might appear that it just skipped last block, while in reality it duplicated part of the previous one.
To fix, use count (indicating how much bytes were really read into the buffer) to work only with valid part of read_buff:
for (int i = 0; i < count; i++) {
sb.Append(Convert.ToString(read_buff[i], 16).PadLeft(2, '0'));
}
You need update offset and count.
Sintaxis
public override int Read(
byte[] array,
int offset,
int count
)
Example
public static byte[] ReadFile(string filePath)
{
byte[] buffer;
FileStream fileStream = new FileStream(filePath, FileMode.Open, FileAccess.Read);
try
{
int length = (int)fileStream.Length; // get file length
buffer = new byte[length]; // create buffer
int count; // actual number of bytes read
int sum = 0; // total number of bytes read
// read until Read method returns 0 (end of the stream has been reached)
while ((count = fileStream.Read(buffer, sum, length - sum)) > 0)
sum += count; // sum is a buffer offset for next reading
}
finally
{
fileStream.Close();
}
return buffer;
}
Reference
public static void ReadAndProcessLargeFile(string theFilename, long whereToStartReading = 0)
{
FileInfo info = new FileInfo(theFilename);
long fileLength = info.Length;
long timesToRead = (fileLength / megabyte);
long ctr = 0;
long timesRead = 0;
FileStream fileStram = new FileStream(theFilename, FileMode.Open, FileAccess.Read);
using (fileStram)
{
byte[] buffer = new byte[megabyte];
fileStram.Seek(whereToStartReading, SeekOrigin.Begin);
int bytesRead = 0;
//bytesRead = fileStram.Read(buffer, 0, megabyte);
//ctr = ctr + 1;
while ((bytesRead = fileStram.Read(buffer, 0, megabyte)) > 0)
{
ProcessChunk(buffer, bytesRead);
buffer = new byte[megabyte]; // This solves last read prob
}
}
}
private static void ProcessChunk(byte[] buffer, int bytesRead)
{
// Do the processing here
string utfString = Encoding.UTF8.GetString(buffer, 0, bytesRead);
Console.Write(utfString);
}
I'm working on c# windows service that handles firebird database requests. My problem occurs at random moments (sometimes after 5 minutes, sometimes after just 4 calls to database), when I try to deserialize object on client application. It happens though only at specific position (stops at 18th byte in 54 byte array). Rest of the time the function returns a proper result.
I'm using this function to serialize single object
public byte[] ObjectToByteArray(Object obj)
{
if (obj == null)
return null;
MemoryStream fs = new MemoryStream();
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(fs, obj);
fs.Seek(0, SeekOrigin.Begin);
byte[] rval = fs.ToArray();
fs.Close();
return rval;
}
I am not serializing any custom classes, only strings and numeric types (firebird api returns them as objects though).
I use this to deserialize:
public object ByteArrayToObject(Byte[] Buffer)
{
BinaryFormatter formatter = new BinaryFormatter();
MemoryStream stream = new MemoryStream(Buffer);
stream.Position = 0;
object rval = formatter.Deserialize(stream); <--- this thing drives me nuts.
stream.Close();
return rval;
}
and main fnct in client aplication. Sorry for ugly code,
public List<object[]> ByteToList(byte[] data, int[] pomocnicza)
{
//pomocnicza table contains size of (original) particular column of list in bytes
int size_row = 0;
foreach (int i in pomocnicza)
{ size_row += i; }
List<object[]> result = new List<object[]>();
int iterator = 0;
for (int i = 0; i < data.Length / size_row ; i++)
{
object[] zxc = new object[3];
int l = pomocnicza.Length/4;
for (int j = 0; j < l; j++)
{
byte[] tmp = new byte[pomocnicza[j*4]];
System.Array.Copy(data, iterator, tmp, 0, pomocnicza[j*4]);
object ffs = ByteArrayToObject(tmp);
zxc[j] = ffs;
iterator += pomocnicza[j*4];
}
result.Add(zxc);
}
return result;
}
What is baffling me is that it works in most cases, but inevitably causes to throw an error. Thing that it happens on random makes pinpointing it harder. Please help.
#EDIT
This is how I read the input:
public List<object[]> RetrieveSelectData(FbConnection dbConn, string SQLCommand)
{
using (var command = dbConn.CreateCommand())
{
command.CommandText = SQLCommand;
using (var reader = command.ExecuteReader())
{
var rows = new List<object[]>();
while (reader.Read())
{
var columns = new object[reader.FieldCount];
reader.GetValues(columns);
rows.Add(columns);
}
return rows;
}
}
}
and then serialize with this function
public byte[] ListToByte(List<object[]> lista, out int[] rozmiary)
{
int size= 0;
rozmiary = new int[lista[0].Length];
for (int i = 0; i < lista[0].Length; i++)
{
byte[] test = this.ObjectToByteArray(lista[0][i]);
size+= test.Length;
rozmiary[i] = test.Length;
}
size*= lista.Count;
byte[] result = new byte[size];
int index = 0;
for (int i = 0; i < lista.Count; i++)
{
for (int j = 0; j < lista[i].Length; j++)
{
byte[] tmp = this.ObjectToByteArray(lista[i][j]);
tmp.CopyTo(result, index);
index += tmp.Length;
}
}
return result;
}
If you are using above deserializing methods & also call them while getting stream from clientstream OR other streams.... skip it. try to use directly those streams with formatter. Like Below :
NetworkStream clientStream = client.GetStream();
Object src = (Object)formatter.Deserialize(clientStream);
I have found the bug. The code above works fine, but care for encoding in some cases(!), so feel free to use it.
The problem laying in another part of a program, where I mistyped and send 4 bytes BUT the client app was told to receive 8, so in most cases it filled it in with zeros, but sometimes it got it from next pack of data.
It was #Marc Gravell and his blog that made me look over and over again to eventually find the source.
Hi i was able to convert a ASCII string to binary using a binarywriter .. as 10101011 . im required back to convert Binary ---> ASCII string .. any idea how to do it ?
This should do the trick... or at least get you started...
public Byte[] GetBytesFromBinaryString(String binary)
{
var list = new List<Byte>();
for (int i = 0; i < binary.Length; i += 8)
{
String t = binary.Substring(i, 8);
list.Add(Convert.ToByte(t, 2));
}
return list.ToArray();
}
Once the binary string has been converted to a byte array, finish off with
Encoding.ASCII.GetString(data);
So...
var data = GetBytesFromBinaryString("010000010100001001000011");
var text = Encoding.ASCII.GetString(data);
If you have ASCII charters only you could use Encoding.ASCII.GetBytes and Encoding.ASCII.GetString.
var text = "Test";
var bytes = Encoding.ASCII.GetBytes(text);
var newText = Encoding.ASCII.GetString(bytes);
Here is complete code for your answer
FileStream iFile = new FileStream(#"c:\test\binary.dat",
FileMode.Open);
long lengthInBytes = iFile.Length;
BinaryReader bin = new BinaryReader(aFile);
byte[] byteArray = bin.ReadBytes((int)lengthInBytes);
System.Text.Encoding encEncoder = System.Text.ASCIIEncoding.ASCII;
string str = encEncoder.GetString(byteArray);
Take this as a simple example:
public void ByteToString()
{
Byte[] arrByte = { 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0 };
string x = Convert.ToBase64String(arrByte);
}
This linked answer has interesting details about this kind of conversion:
binary file to string
Sometimes instead of using the built in tools it's better to use "custom" code.. try this function:
public string BinaryToString(string binary)
{
if (string.IsNullOrEmpty(binary))
throw new ArgumentNullException("binary");
if ((binary.Length % 8) != 0)
throw new ArgumentException("Binary string invalid (must divide by 8)", "binary");
StringBuilder builder = new StringBuilder();
for (int i = 0; i < binary.Length; i += 8)
{
string section = binary.Substring(i, 8);
int ascii = 0;
try
{
ascii = Convert.ToInt32(section, 2);
}
catch
{
throw new ArgumentException("Binary string contains invalid section: " + section, "binary");
}
builder.Append((char)ascii);
}
return builder.ToString();
}
Tested with 010000010100001001000011 it returned ABC using the "raw" ASCII values.