We Keep Coding

Unicode Hex String to String

I have a unicode string like this:
0030003100320033
Which should turn into 0123.
This is a simple case of 0123 string, but there are some string and unicode chars as well. How can I turn this type of unicode hex string to string in C#?
For normal US charset, first part is always 00, so 0031 is "1" in ASCII, 0032 is "2" and so on.
When its actual unicode char, like Arabic and Chinese, first part is not 00, for instance for Arabic its 06XX, like 0663.
I need to be able to turn this type of Hex string into C# decimal string.

There are several encodings that can represent Unicode, of which UTF-8 is today's de facto standard. However, your example is actually a string representation of UTF-16 using the big-endian byte order. You can convert your hex string back into bytes, then use Encoding.BigEndianUnicode to decode this:
public static void Main()
{
var bytes = StringToByteArray("0030003100320033");
var decoded = System.Text.Encoding.BigEndianUnicode.GetString(bytes);
Console.WriteLine(decoded); // gives "0123"
}
// https://stackoverflow.com/a/311179/1149773
public static byte[] StringToByteArray(string hex)
{
byte[] bytes = new byte[hex.Length / 2];
for (int i = 0; i < hex.Length; i += 2)
bytes[i / 2] = Convert.ToByte(hex.Substring(i, 2), 16);
return bytes;
}
Since Char in .NET represents a UTF-16 code unit, this answer should give identical results to Slai's, including for surrogate pairs.

Shorter less efficient alternative:
Regex.Replace("0030003100320033", "....", m => (char)Convert.ToInt32(m + "", 16) + "");

You should try this solution
public static void Main()
{
string hexString = "0030003100320033"; //Hexa pair numeric values
//string hexStrWithDash = "00-30-00-31-00-32-00-33"; //Hexa pair numeric values separated by dashed. This occurs using BitConverter.ToString()
byte[] data = ParseHex(hexString);
string result = System.Text.Encoding.BigEndianUnicode.GetString(data);
Console.Write("Data: {0}", result);
}
public static byte[] ParseHex(string hexString)
{
hexString = hexString.Replace("-", "");
byte[] output = new byte[hexString.Length / 2];
for (int i = 0; i < output.Length; i++)
{
output[i] = Convert.ToByte(hexString.Substring(i * 2, 2), 16);
}
return output;
}

Convert character to CP1252 Hex in c#

I wrote a function to convert a string to a hex string. So "abc" would be converted to "616263".
Here is the function:
private string StringToHex(string myString)
{
int ctr, li_max;
string ls_hex = "";
li_max = myString.Length;
for(ctr = 0; ctr < li_max; ctr++)
ls_hex = ls_hex + (Convert.ToInt16(myString[ctr])).ToString("x2");
return ls_hex;
}
However, when I have unicode characters like "‹abc" I get "2039616263". So my first character is converted to 2039 (unicode). I wanted to get "8B616263" (CP1252). See image below:
How can I get the hex value for the correct encoding?
ie: 8B instead of 2039

You need to get use the CP1252 encoding to get the bytes of the string:
private string StringToHex(string myString)
{
byte[] bytes = Encoding.GetEncoding(1252).GetBytes(myString);
return string.Concat(bytes.Select(b => b.ToString("X2")));
}
My output is
8B616263

How to sum existing byte array value with another HEX value in C#?

i need to decode a string in C#. Algorithm peformed in HEX values. So in C# i think i need to convert to byte array?am i right?. So I did a byte array from a string:
string Encoded = "ENCODEDSTRINGSOMETHING";
byte[] ba = Encoding.Default.GetBytes (Encoded);
Now i need to modify each byte in byte array first starting from summing hex value (0x20) to first byte and for the each next byte in array i should substitute 0x01 hex from starting 0x20 hex value and sum it with following bytes in my ba array. Then i need to convert my byte array result to string again and print. In Python this is very easy:
def decode ():
strEncoded = "ENCODEDSTRINGSOMETHING"
strDecoded = ""
counter = 0x20
for ch in strEncoded:
ch_mod = ord(ch) + counter
counter -= 1
strDecoded += chr(ch_mod)
print ("%s" % strDecoded)
if __name__ == '__main__':
decode()
How can i do it in C#? Thank you very much.

Here's a rough outline of how to do what you are trying to do. Might need to change it a bit to fit your problem/solution.
public string Encode(string input, int initialOffset = 0x20)
{
string result = "";
foreach(var c in input)
{
result += (char)(c + (initialOffset --));
}
return result;
}

Try this code:
string Encoded = "ENCODEDSTRINGSOMETHING";
byte[] ba = Encoding.Default.GetBytes(Encoded);
string strDecoded = "";
int counter = 0x20;
foreach (char c in Encoded)
{
int ch_mod = (int)c+counter;
counter -= 1;
strDecoded += (char)ch_mod;
}

Decoding string Char.Parse issue

I have the following code to encode a plain text:
int AddNumber;
int AsciiNumber;
string OneChar;
string String1 = "DAVE SMITH";
string String2 = "";
for (int i = 0; i < String1.Length; i++)
{
AddNumber = i + 95;
AsciiNumber = (int)Char.Parse(String1.Substring(i,1));
byte[] NewAscii = new byte[] { Convert.ToByte( AsciiNumber + AddNumber ) };
// Get string of the NewAscii
OneChar = Encoding.GetEncoding(1252).GetString(NewAscii);
String2 = String2 + OneChar;
}
The problem I have is how to decode the string back to plain text. Here is my attempt code:
String1 = "";
for (int i = 0; i < String2.Length; i++)
{
AddNumber = i + 95;
AsciiNumber = (int)Char.Parse(String2.Substring(i,1));
byte[] NewAscii = new byte[] { Convert.ToByte( AsciiNumber - AddNumber ) };
// Get string of the NewAscii
OneChar = Encoding.GetEncoding(1252).GetString(NewAscii);
String1 = String1 + OneChar;
}
The problem is that above, on processing the encoded empty space (between DAVE and SMITH), the value AsciiNumber = (int)Char.Parse(String2.Substring(i,1)) is 402 where it should be 131.
Do you see what I am misunderstanding?

By adding 95 to a space (ASCII 36) you end up with byte 131. You then ask for the Windows-1252 text at 131, which is a Latin ƒ and store that into C#'s native Unicode string. C# is going to map that Latin ƒ back to UTF-16 for storage into memory. Later, you ask for that character back - it's Unicode code point is U+0192; convert that from hex and you get decimal 402. Trying to get that back to Windows-1252 will obviously fail, since it's not a byte.
What you probably want to do, is to use Encoding.GetBytes to have the Unicode text converted to Windows-1252 before manipulating the characters.

For the decoding part
String1 = "";
for (int i = 0; i < String2.Length; i++)
{
var charByte = System.Text.Encoding.GetEncoding(1252).GetBytes(String2.Substring(i, 1));
AddNumber = i + 95;
AsciiNumber = Convert.ToInt32(charByte[0]) - AddNumber;
String1 += Convert.ToChar(AsciiNumber);
}

Expressing byte values > 127 in .Net Strings

I'm writing some binary protocol messages in .Net using strings, and it mostly works, except for one particular case.
The message I'm trying to send is:
String cmdPacket = "\xFD\x0B\x16MBEPEXE1.";
myDevice.Write(Encoding.ASCII.GetBytes(cmdPacket));
(to help decode, those bytes are 253, 11, 22, then the ASCII chars: "MBEPEXE1.").
Except when I do the Encoding.ASCII.GetBytes, the 0xFD comes out as byte 0x3F
(value 253 changed to 63).
(I should point out that the \x0B and \x16 are interpreted correctly as Hex 0B & Hex 16)
I've also tried Encoding.UTF8 and Encoding.UTF7, to no avail.
I feel there is probably a good simple way to express values above 128 in Strings, and convert them to bytes, but I'm missing it.
Any guidance?

Ignoring if it's good or bad what you are doing, the encoding ISO-8859-1 maps all its characters to the characters with the same code in Unicode.
// Bytes with all the possible values 0-255
var bytes = Enumerable.Range(0, 256).Select(p => (byte)p).ToArray();
// String containing the values
var all1bytechars = new string(bytes.Select(p => (char)p).ToArray());
// Sanity check
Debug.Assert(all1bytechars.Length == 256);
// The encoder, you could make it static readonly
var enc = Encoding.GetEncoding("ISO-8859-1"); // It is the codepage 28591
// string-to-bytes
var bytes2 = enc.GetBytes(all1bytechars);
// bytes-to-string
var all1bytechars2 = enc.GetString(bytes);
// check string-to-bytes
Debug.Assert(bytes.SequenceEqual(bytes2));
// check bytes-to-string
Debug.Assert(all1bytechars.SequenceEqual(all1bytechars2));
From the wiki:
ISO-8859-1 was incorporated as the first 256 code points of ISO/IEC 10646 and Unicode.
Or a simple and fast method to convert a string to a byte[] (with unchecked and checked variant)
public static byte[] StringToBytes(string str)
{
var bytes = new byte[str.Length];
for (int i = 0; i < str.Length; i++)
{
bytes[i] = checked((byte)str[i]); // Slower but throws OverflowException if there is an invalid character
//bytes[i] = unchecked((byte)str[i]); // Faster
}
return bytes;
}

ASCII is a 7-bit code. The high-order bit used to be used as a parity bit, so "ASCII" could have even, odd or no parity. You may notice that 0x3F (decimal 63) is the ASCII character ?. That is what non-ASCII octets (those greater than 0x7F/decimal 127) are converted to by the CLR's ASCII encoding. The reason is that there is no standard ASCII character representation of the code points in the range 0x80–0xFF.
C# strings are UTF-16 encoded Unicode internally. If what you care about are the byte values of the strings, and you know that the strings are, in fact, characters whose Unicode code points are in the range U+0000 through U+00FF, then its easy. Unicode's first 256 codepoints (0x00–0xFF), the Unicode blocks C0 Controls and Basic Latin (\x00-\x7F) and C1 Controls and Latin Supplement (\x80-\xFF) are the "normal" ISO-8859-1 characters. A simple incantation like this:
String cmdPacket = "\xFD\x0B\x16MBEPEXE1.";
byte[] buffer = cmdPacket.Select(c=>(byte)c).ToArray() ;
myDevice.Write(buffer);
will get you the byte[] you want, in this case
// \xFD \x0B \x16 M B E P E X E 1 .
[ 0xFD , 0x0B , 0x16 , 0x4d , 0x42 , 0x45, 0x50 , 0x45 , 0x58 , 0x45 , 0x31 , 0x2E ]

With LINQ, you could do something like this:
String cmdPacket = "\xFD\x0B\x16MBEPEXE1.";
myDevice.Write(cmdPacket.Select(Convert.ToByte).ToArray());
Edit: Added an explanation
First, you recognize that your string is really just an array of characters. What you want is an "equivalent" array of bytes, where each byte corresponds to a character.
To get the array, you have to "map" each character of the original array as a byte in the new array. To do that, you can use the built-in System.Convert.ToByte(char) method.
Once you've described your mapping from characters to bytes, it's as simple as projecting the input string, through the mapping, into an array.
Hope that helps!

I use Windows-1252 as it seems to give the most bang for the byte
And is compatible with all .NET string values
You will probably want to comment out the ToLower
This was built for compatibility with SQL char (single byte)
namespace String1byte
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
String8bit s1 = new String8bit("cat");
String8bit s2 = new String8bit("cat");
String8bit s3 = new String8bit("\xFD\x0B\x16MBEPEXE1.");
HashSet<String8bit> hs = new HashSet<String8bit>();
hs.Add(s1);
hs.Add(s2);
hs.Add(s3);
System.Diagnostics.Debug.WriteLine(hs.Count.ToString());
System.Diagnostics.Debug.WriteLine(s1.Value + " " + s1.GetHashCode().ToString());
System.Diagnostics.Debug.WriteLine(s2.Value + " " + s2.GetHashCode().ToString());
System.Diagnostics.Debug.WriteLine(s3.Value + " " + s3.GetHashCode().ToString());
System.Diagnostics.Debug.WriteLine(s1.Equals(s2).ToString());
System.Diagnostics.Debug.WriteLine(s1.Equals(s3).ToString());
System.Diagnostics.Debug.WriteLine(s1.MatchStart("ca").ToString());
System.Diagnostics.Debug.WriteLine(s3.MatchStart("ca").ToString());
}
}
public struct String8bit
{
private static Encoding EncodingUnicode = Encoding.Unicode;
private static Encoding EncodingWin1252 = System.Text.Encoding.GetEncoding("Windows-1252");
private byte[] bytes;
public override bool Equals(Object obj)
{
// Check for null values and compare run-time types.
if (obj == null) return false;
if (!(obj is String8bit)) return false;
String8bit comp = (String8bit)obj;
if (comp.Bytes.Length != this.Bytes.Length) return false;
for (Int32 i = 0; i < comp.Bytes.Length; i++)
{
if (comp.Bytes[i] != this.Bytes[i])
return false;
}
return true;
}
public override int GetHashCode()
{
UInt32 hash = (UInt32)(Bytes[0]);
for (Int32 i = 1; i < Bytes.Length; i++) hash = hash ^ (UInt32)(Bytes[0] << (i%4)*8);
return (Int32)hash;
}
public bool MatchStart(string start)
{
if (string.IsNullOrEmpty(start)) return false;
if (start.Length > this.Length) return false;
start = start.ToLowerInvariant(); // SQL is case insensitive
// Convert the string into a byte array
byte[] unicodeBytes = EncodingUnicode.GetBytes(start);
// Perform the conversion from one encoding to the other
byte[] win1252Bytes = Encoding.Convert(EncodingUnicode, EncodingWin1252, unicodeBytes);
for (Int32 i = 0; i < win1252Bytes.Length; i++) if (Bytes[i] != win1252Bytes[i]) return false;
return true;
}
public byte[] Bytes { get { return bytes; } }
public String Value { get { return EncodingWin1252.GetString(Bytes); } }
public Int32 Length { get { return Bytes.Count(); } }
public String8bit(string word)
{
word = word.ToLowerInvariant(); // SQL is case insensitive
// Convert the string into a byte array
byte[] unicodeBytes = EncodingUnicode.GetBytes(word);
// Perform the conversion from one encoding to the other
bytes = Encoding.Convert(EncodingUnicode, EncodingWin1252, unicodeBytes);
}
public String8bit(Byte[] win1252bytes)
{ // if reading from SQL char then read as System.Data.SqlTypes.SqlBytes
bytes = win1252bytes;
}
}
}