I have a list of a class MyData which holds byte arrays. I am creating a function to which I can pass List and a string to access the fields and do some operations on them - primarily concatenation.
The part I am struggling with is accessing a class field via string.
I am unsure if I should be using Linq or reflection or a combination of the two.
I have a lot of data, so performance is also a consideration.
Thanks
public class MyData
{
public byte[] vertices;
public byte[] indicies;
}
public byte[] ProcessData(List<MyData> inData, string fieldName)
{
byteArray = new byte[inData.Sum(x => x."fieldName").Length];
offset = 0;
for (int i = 0; i < inData.Count; i++) {
Buffer.BlockCopy(inData[i]."fieldName", 0, ret, offset, inData[i]."fieldName".Length);
offset += inData."fieldName".Length;
}
return byteArray;
}
List<MyData> AllMyData = new List<MyData>();
//Load some data (omitted)
var AllVertices = ProcessData(AllMyData, "vertices");
var AllIndicies = ProcessData(AllMyData, "indicies");
public byte[] ProcessData(List<MyData> inData, string fieldName)
{
var field = inData.GetType().GetField(fieldName);
if (field == null)
throw new ArgumentException("Invalid field name", nameof(fieldName));
byte[] bytesField = field.GetValue(inData) as byte[];
var byteArray = new byte[bytesField.Sum().Length];
var offset = 0;
for (int i = 0; i < inData.Count; i++) {
Buffer.BlockCopy(bytesField[i], 0, ret, offset, bytesField.Length);
offset += bytesField.Length;
}
return byteArray;
}
If would be faster (than reflection) if it is OK to represent the field selection not as string, but as function:
public byte[] ProcessData(List<MyData> inData, Func<MyData, byte[]> field)
{
var byteArray = new byte[inData.Sum(x => field(x).Length)];
var offset = 0;
for (int i = 0; i < inData.Count; i++) {
Buffer.BlockCopy(field(inData[i]), 0, byteArray, offset, field(inData[i]).Length);
offset += field(inData[i]).Length;
}
return byteArray;
}
var allVertices = ProcessData(AllMyData, x => x.vertices);
var allIndicies = ProcessData(AllMyData, x => x.indicies);
Related
I created small not finishd Packet Builder class.
AddString() working without problems, but if i use AddInt() the console output looks very weird. Any can tell me why the integer not display correctly?
Main
Packet packet = new Packet();
packet.builder.AddString(Constants.Requests.GET_RESOURCES);
packet.builder.AddString("Another_String");
packet.builder.AddInt(500);
byte[] byteArray = packet.builder.GetByteBuffer();
Console.WriteLine(ByteArrayToString(byteArray));
ByteArray Output: Get_Resources:Another_String:?☺:
47-65-74-5F-52-65-73-6F-75-72-63-65-73-00-3A-41-6E-6F-74-68-65-72-5F-53-74-72-69-6E-67-00-3A-F4-01-00-00-00-3A
As you can see: ?☺ is definitly wrong. The functions are almost the same.
Class
class Packet
{
public Builder builder;
public Packet()
{
builder = new Builder();
}
private static string ByteArrayToString(byte[] arr)
{
System.Text.ASCIIEncoding enc = new System.Text.ASCIIEncoding();
return enc.GetString(arr);
}
public static string[] Read(byte[] _recievedData)
{
string data = ByteArrayToString(_recievedData).Trim();
string[] result = data.Split(':');
return result;
}
public class Builder
{
private byte[] buffer;
private int offset;
//Makes very easy on client to filter packets...
private byte[] seperator;
public Builder()
{
offset = 0;
buffer = new byte[4096];
seperator = BitConverter.GetBytes(':');
}
public void AddInt(int intValue)
{
byte[] byteArray = BitConverter.GetBytes(intValue);
for (int x = 0; x < byteArray.Length; x++)
{
buffer[x + offset] = byteArray[x];
}
for (int y = 0; y < seperator.Length; y++)
{
buffer[byteArray.Length + (y + 1) + offset] = seperator[y];
}
offset += (byteArray.Length + seperator.Length);
}
public void AddString(string str)
{
byte[] byteArray = Encoding.ASCII.GetBytes(str);
for (int x = 0; x < byteArray.Length; x++)
{
buffer[x + offset] = byteArray[x];
}
for (int y = 0; y < seperator.Length; y++)
{
buffer[byteArray.Length + (y + 1) + offset] = seperator[y];
}
offset += (byteArray.Length + seperator.Length);
}
public byte[] GetByteBuffer()
{
return buffer;
}
public void Reset()
{
buffer = null;
offset = 0;
}
}
}
Your code is working perfectly fine. Possibly it is not what you want but following code converts an int in 4 bytes because it is a 32-bit integer.
byte[] byteArray = BitConverter.GetBytes(intValue);
at the end of your output, you see those 4 bytes as expected in little endian format F4-01-00-00 because 500 in hexadecimal is 0x01F4. This explains why you are getting, what you are getting.
Now I am assuming that you are expecting 500 instead of ?☺. Following code should fetch you desired result:
byte[] byteArray = BitConverter.GetBytes(intValue.ToString());
This will add a string representation of the number instead of binary representation. Based on the return type of Read function, the need seems to be a string representation.
I have the following function:
public void SetTagData(string _data)
{
string data = _data;
byte[] ba = Encoding.Default.GetBytes(data);
string hexString = BitConverter.ToString(ba);
hexString = hexString.Replace("-", "");
var blockStart = 0;
var bufferHexBlocks = String.Empty;
try
{
for (var i = 0; i < hexString.Length; i++)
{
var byteList = new List<byte>();
byte[] datablockKey = ConvertHelpers.ConvertHexStringToByteArray(i.ToString().PadLeft(2, '0'));
var block = hexString.Substring(blockStart, 8);
byte[] datablockValue = ConvertHelpers.ConvertHexStringToByteArray(block);
byteList.AddRange(datablockKey);
byteList.AddRange(datablockValue);
_reader.Protocol("wb", byteList.ToArray());
blockStart += 8;
}
}
catch (Exception ex)
{
console.log(ex.message);
}
}
The data coming in is a bunch of hex as a string. I need to split this hex string into batches of 8 characters, append an incrementing 0 padded hex number from 00 to 1f and send this new string as a byte array to the _reader.Protocol function, which accepts a string wb as first parameter and the block as the second.
For example incoming data is:
string data = "3930313B36313B5350542D53504C3B3830303B3B352E373B3B303B303B3B3B34353036383B4E3B4E3B"
I need to send the following to the _reader.Protocol object:
(incremented padded hex 01, 02, 03, ... , 0f) and the first 8 characters of the data string, then the next, and so on as a byte array.
[013930313B], [0236313B53], etc.
I think I'm getting close... but missing something...
My problem at the moment is that I can't figure out how to loop in blocks of 8 and if the hex string is say 82 characters instead of 80 (multiple of 8), then how would I grab the last two characters without getting a IndexOutofRange exception.
Note: This is for a Windows CE application, so no new C# features please.
This below will work fine in conjunction with this answer and the sample string data given.
public static byte[] Parse(string data)
{
var count = data.Length / 8; //Might be worth throwing exception with any remainders unless you trust the source.
var needle = 0;
List<byte> result = new List<byte>(); //Inefficient but I'm being lazy
for (int i = 0; i < count; i++)
{
char[] buffer = new char[8];
data.CopyTo(needle, buffer, 0, buffer.Length);
//To get around the odd number when adding the prefixed count byte, send the hex string to the convert method separately.
var bytes = ConvertHexStringToByteArray(new string(buffer)); //Taken From https://stackoverflow.com/a/8235530/6574422
//As the count is less than 255, seems safe to parse to single byte
result.Add(byte.Parse((i + 1).ToString()));
result.AddRange(bytes);
needle += 8;
}
return result.ToArray();
}
I'm figured it out. It might not be the most efficient solution but it works just fine. I did it using a for loop inside a for loop.
In case anyone is interested here is the final code:
public void SetTagData(string _data)
{
string data = _data;
byte[] ba = Encoding.Default.GetBytes(data);
string hexString = BitConverter.ToString(ba);
hexString = hexString.Replace("-", "");
var blockStart = 0;
try
{
_reader.Protocol("s");
for(var count = 0; count < 16; count++)
{
var byteList = new List<byte>();
byte[] datablockKey = ConvertHelpers.ConvertHexStringToByteArray(count.ToString("X2"));
byteList.AddRange(datablockKey);
for (var innerCount = 0; innerCount < 4; innerCount++)
{
var block = String.Empty;
if (!String.IsNullOrEmpty(hexString.Substring(blockStart, 2)))
{
block = hexString.Substring(blockStart, 2);
}
else
{
block = "20";
}
byte[] datablockValue = ConvertHelpers.ConvertHexStringToByteArray(block);
byteList.AddRange(datablockValue);
blockStart += 2;
}
_reader.Protocol("wb", byteList.ToArray());
}
}
catch (Exception)
{
}
}
a.
static void onMessage(IntPtr str)
{
string message = Marshal.PtrToStringAnsi(str);
Console.Write(message);
}
， its Return Chinese garbled。
b.
public static void onMessage(IntPtr str)
{
int nAnsiLength = Marshal.PtrToStringAnsi(str).Length;
int nUniLength = Marshal.PtrToStringUni(str).Length;
int nMaxLength = (nAnsiLength > nUniLength) ? nAnsiLength : nUniLength;
int length = 0;//循环查找字符串的长度
for (int i = 0; i < nAnsiLength; i++)
{
byte[] strbuf1 = new byte[1];
Marshal.Copy(str + i, strbuf1, 0, 1);
if (strbuf1[0] == 0)
{
break;
}
length++;
}
byte[] strbuf = new byte[length];
Marshal.Copy(str, strbuf, 0, length);
string message = System.Text.UTF8Encoding.UTF8.GetString(strbuf);
}
， Chinese display, but the length of the string returned。
I need help！
Here there are various codepages that are chinese... Try the one that seems to fit what you expect. I've even simplified the code to copy the IntPtr buffer to a byte[] buffer.
public static void onMessage(IntPtr str) {
int length = 0;//循环查找字符串的长度
while (Marshal.ReadByte(str + length) != 0) {
length++;
}
byte[] strbuf = new byte[length];
Marshal.Copy(str, strbuf, 0, length);
// Taken from https://msdn.microsoft.com/it-it/library/system.text.encodinginfo.getencoding(v=vs.110).aspx
string message1 = Encoding.UTF8.GetString(strbuf);
string message2 = Encoding.GetEncoding(54936).GetString(strbuf);
string message3 = Encoding.GetEncoding(936).GetString(strbuf);
string message4 = Encoding.GetEncoding(950).GetString(strbuf);
string message5 = Encoding.GetEncoding(10002).GetString(strbuf);
string message6 = Encoding.GetEncoding(10008).GetString(strbuf);
string message7 = Encoding.GetEncoding(20000).GetString(strbuf);
string message8 = Encoding.GetEncoding(20002).GetString(strbuf);
string message9 = Encoding.GetEncoding(20936).GetString(strbuf);
string message10 = Encoding.GetEncoding(50227).GetString(strbuf);
string message11 = Encoding.GetEncoding(51936).GetString(strbuf);
string message12 = Encoding.GetEncoding(52936).GetString(strbuf);
}
I'm working on c# windows service that handles firebird database requests. My problem occurs at random moments (sometimes after 5 minutes, sometimes after just 4 calls to database), when I try to deserialize object on client application. It happens though only at specific position (stops at 18th byte in 54 byte array). Rest of the time the function returns a proper result.
I'm using this function to serialize single object
public byte[] ObjectToByteArray(Object obj)
{
if (obj == null)
return null;
MemoryStream fs = new MemoryStream();
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(fs, obj);
fs.Seek(0, SeekOrigin.Begin);
byte[] rval = fs.ToArray();
fs.Close();
return rval;
}
I am not serializing any custom classes, only strings and numeric types (firebird api returns them as objects though).
I use this to deserialize:
public object ByteArrayToObject(Byte[] Buffer)
{
BinaryFormatter formatter = new BinaryFormatter();
MemoryStream stream = new MemoryStream(Buffer);
stream.Position = 0;
object rval = formatter.Deserialize(stream); <--- this thing drives me nuts.
stream.Close();
return rval;
}
and main fnct in client aplication. Sorry for ugly code,
public List<object[]> ByteToList(byte[] data, int[] pomocnicza)
{
//pomocnicza table contains size of (original) particular column of list in bytes
int size_row = 0;
foreach (int i in pomocnicza)
{ size_row += i; }
List<object[]> result = new List<object[]>();
int iterator = 0;
for (int i = 0; i < data.Length / size_row ; i++)
{
object[] zxc = new object[3];
int l = pomocnicza.Length/4;
for (int j = 0; j < l; j++)
{
byte[] tmp = new byte[pomocnicza[j*4]];
System.Array.Copy(data, iterator, tmp, 0, pomocnicza[j*4]);
object ffs = ByteArrayToObject(tmp);
zxc[j] = ffs;
iterator += pomocnicza[j*4];
}
result.Add(zxc);
}
return result;
}
What is baffling me is that it works in most cases, but inevitably causes to throw an error. Thing that it happens on random makes pinpointing it harder. Please help.
#EDIT
This is how I read the input:
public List<object[]> RetrieveSelectData(FbConnection dbConn, string SQLCommand)
{
using (var command = dbConn.CreateCommand())
{
command.CommandText = SQLCommand;
using (var reader = command.ExecuteReader())
{
var rows = new List<object[]>();
while (reader.Read())
{
var columns = new object[reader.FieldCount];
reader.GetValues(columns);
rows.Add(columns);
}
return rows;
}
}
}
and then serialize with this function
public byte[] ListToByte(List<object[]> lista, out int[] rozmiary)
{
int size= 0;
rozmiary = new int[lista[0].Length];
for (int i = 0; i < lista[0].Length; i++)
{
byte[] test = this.ObjectToByteArray(lista[0][i]);
size+= test.Length;
rozmiary[i] = test.Length;
}
size*= lista.Count;
byte[] result = new byte[size];
int index = 0;
for (int i = 0; i < lista.Count; i++)
{
for (int j = 0; j < lista[i].Length; j++)
{
byte[] tmp = this.ObjectToByteArray(lista[i][j]);
tmp.CopyTo(result, index);
index += tmp.Length;
}
}
return result;
}
If you are using above deserializing methods & also call them while getting stream from clientstream OR other streams.... skip it. try to use directly those streams with formatter. Like Below :
NetworkStream clientStream = client.GetStream();
Object src = (Object)formatter.Deserialize(clientStream);
I have found the bug. The code above works fine, but care for encoding in some cases(!), so feel free to use it.
The problem laying in another part of a program, where I mistyped and send 4 bytes BUT the client app was told to receive 8, so in most cases it filled it in with zeros, but sometimes it got it from next pack of data.
It was #Marc Gravell and his blog that made me look over and over again to eventually find the source.
I am try to do some code using BinaryWriter and Then BinaryReader.
When I wanna write I use method Write().
But the problem is that between two lines of Write method there appears a new byte which is in ASCII table in decimal 31 (sometines 24).
You can see it on this image:
You can see that byte at index 4 (5th byte) is of ASCII decimal value 31. I didnt insert it there. As you can see 1st 4 bytes are reserved for a number (Int32), next are other data (some text mostly - this is not important now).
As you can see from the code i write:
- into 1st line a number 10
- into 2nd line text "This is some text..."
How come came that 5th byte (dec 31) in between??
And this is the code I have:
static void Main(string[] args)
{
//
//// SEND - RECEIVE:
//
SendingData();
Console.ReadLine();
}
private static void SendingData()
{
int[] commandNumbers = { 1, 5, 10 }; //10 is for the users (when they send some text)!
for (int i = 0; i < commandNumbers.Length; i++)
{
//convert to byte[]
byte[] allBytes;
using (MemoryStream ms = new MemoryStream())
{
using (BinaryWriter bw = new BinaryWriter(ms))
{
bw.Write(commandNumbers[i]); //allocates 1st 4 bytes - FOR MAIN COMMANDS!
if (commandNumbers[i] == 10)
bw.Write("This is some text at command " + commandNumbers[i]); //HERE ON THIS LINE IS MY QUESTION!!!
}
allBytes = ms.ToArray();
}
//convert back:
int valueA = 0;
StringBuilder sb = new StringBuilder();
foreach (var b in GetData(allBytes).Select((a, b) => new { Value = a, Index = b }))
{
if (b.Index == 0) //1st num
valueA = BitConverter.ToInt32(b.Value, 0);
else //other text
{
foreach (byte _byte in b.Value)
sb.Append(Convert.ToChar(_byte));
}
}
if (sb.ToString().Length == 0)
sb.Append("ONLY COMMAND");
Console.WriteLine("Command = {0} and Text is \"{1}\".", valueA, sb.ToString());
}
}
private static IEnumerable<byte[]> GetData(byte[] data)
{
using (MemoryStream ms = new MemoryStream(data))
{
using (BinaryReader br = new BinaryReader(ms))
{
int j = 0;
byte[] buffer = new byte[4];
for (int i = 0; i < data.Length; i++)
{
buffer[j++] = data[i];
if (i == 3) //SENDING COMMAND DATA
{
yield return buffer;
buffer = new byte[1];
j = 0;
}
else if (i > 3) //SENDING TEXT
{
yield return buffer;
j = 0;
}
}
}
}
}
If you look at the documentation for Write(string), you'll see that it writes a length-prefixed string. So the 31 is the number of characters in your string -- perfectly normal.
You should probably be using Encoding.GetBytes and then write the bytes instead of writing a string
for example
bw.Write(
Encoding.UTF8.GetBytes("This is some text at command " + commandNumbers[i])
);
When a string is written to a binary stream, the first thing it does is write the length of the string. The string "This is some text at command 10" has 31 characters, which is the value you're seeing.
You should check the documentation of methods you use before asking questions about them:
A length-prefixed string represents the string length by prefixing to
the string a single byte or word that contains the length of that
string. This method first writes the length of the string as a UTF-7
encoded unsigned integer, and then writes that many characters to the
stream by using the BinaryWriter instance's current encoding.
;-)
(Though in fact it is an LEB128 and not UTF-7, according to Wikipedia).
The reason this byte is there because you're adding a variable amount of information, so the length is needed. If you were to add two strings, where would you know where the first ended and the second began?
If you really don't want or need that length byte, you can always convert the string to a byte array and use that.
Ok, here is my edited code. I removed BinaryWriter (while BinaryReader is still there!!), and now it works very well - no more extra bytes.
What do you thing? Is there anytihng to do better, to make it run faster?
Expecially Im interesting for that foreach loop, which read from another method that is yield return type!!
New Code:
static void Main(string[] args)
{
//
//// SEND - RECEIVE:
//
SendingData();
Console.ReadLine();
}
private static void SendingData()
{
int[] commands = { 1, 2, 3 };
// 1 - user text
// 2 - new game
// 3 - join game
// ...
for (int i = 0; i < commands.Length; i++)
{
//convert to byte[]
byte[] allBytes;
using (MemoryStream ms = new MemoryStream())
{
// 1.st - write a command:
ms.Write(BitConverter.GetBytes(commands[i]), 0, 4);
// 2nd - write a text:
if (commands[i] == 1)
{
//some example text (like that user sends it):
string myText = "This is some text at command " + commands[i];
byte[] myBytes = Encoding.UTF8.GetBytes(myText);
ms.Write(myBytes, 0, myBytes.Length);
}
allBytes = ms.ToArray();
}
//convert back:
int valueA = 0;
StringBuilder sb = new StringBuilder();
foreach (var b in ReadingData(allBytes).Select((a, b) => new { Value = a, Index = b }))
{
if (b.Index == 0)
{
valueA = BitConverter.ToInt32(b.Value, 0);
}
else
{
sb.Append(Convert.ToChar(b.Value[0]));
}
}
if (sb.ToString().Length == 0)
sb.Append("ONLY COMMAND");
Console.WriteLine("Command = {0} and Text is \"{1}\".", valueA, sb.ToString());
}
}
private static IEnumerable<byte[]> ReadingData(byte[] data)
{
using (MemoryStream ms = new MemoryStream(data))
{
using (BinaryReader br = new BinaryReader(ms))
{
int j = 0;
byte[] buffer = new byte[4];
for (int i = 0; i < data.Length; i++)
{
buffer[j++] = data[i];
if (i == 3) //SENDING COMMAND DATA
{
yield return buffer;
buffer = new byte[1];
j = 0;
}
else if (i > 3) //SENDING TEXT
{
yield return buffer;
j = 0;
}
}
}
}
}