We Keep Coding

How to convert unicode to utf-8 encoding in c#

I want to convert unicode string to UTF8 string. I want to use this UTF8 string in SMS API to send unicode SMS.
I want conversion like this tool
https://cafewebmaster.com/online_tools/utf8_encode
eg. I have unicode string "हैलो फ़्रेंड्स" and it should be converted into "हà¥à¤²à¥ à¥à¥à¤°à¥à¤à¤¡à¥à¤¸"
I have tried this but not getting expected output
private string UnicodeToUTF8(string strFrom)
{
byte[] bytes = Encoding.Default.GetBytes(strFrom);
return Encoding.UTF8.GetString(bytes);
}
and calling function like this
string myUTF8String = UnicodeToUTF8("हैलो फ़्रेंड्स");

I don't think this is possible to answer concretely without knowing more about the SMS API you want to use. The string type in C# is UTF-16. If you want a different encoding, it's given to you as a byte[] (because a string is UTF-16, always).
You could 'cast' that into a string by doing something like this:
static string UnicodeToUTF8(string from) {
var bytes = Encoding.UTF8.GetBytes(from);
return new string(bytes.Select(b => (char)b).ToArray());
}
As far as I can tell this yields the same output as the website you linked. However, without knowing what API you're handing this string off to, I can't guarantee that this will ultimately work.
The point of string is that we don't need to worry about its underlying encoding, but this casting operation is kind of a giant hack and makes no guarantees that string represents a well-formed string anymore.
If something expects a UTF-8 encoding, it should accept a byte[], not a string.

Try this:
string output = "hello world";
byte[] bytes1 = Encoding.Convert(Encoding.Unicode, Encoding.UTF8, Encoding.Unicode.GetBytes(output));
byte[] bytes2 = Encoding.Convert(Encoding.Unicode, Encoding.Unicode, Encoding.Unicode.GetBytes(output));
var output1 = Encoding.UTF8.GetString(bytes1);
var output2 = Encoding.Unicode.GetString(bytes2);
You will see that bytes1 is 11 bytes (1 byte per char UTF-8) and bytes2 is 22 bytes (2 bytes per char for unicode)

string to byte array (to string to XML) and back again

i know there are 1million questions about "string - byte array" conversion out there but none of them fit my problem.
For the installation of my software i need to save some informations from the user (serveraddress, userID, password and so on). Some of these informations need do be protected (encrypted using DPAPI). For that i have to convert the string (SecureString) to byte[]
public static byte[] StringToByte(string s)
{
return Convert.FromBase64String(s);
}
where i get my first problem. If the strings lenght is a not a multiple of 4 (s.lenght % 4 == 0) i get a "Invalid length for a Base-64 char array" error. I've read that i can (have to) add "=" to the end of the string but some of these strings may be passwords (which may contain "="). I need to store the (encrypted) data in a XML-file why i can't use Unicode encoding (i don't know why but it corrupts the XML file ... because of encoding i would suppose).
As last step i have to go back the way to get the stored data on app startup.
Does someone of you can help me solving this problem ? I don't care the output in the XML as long as it is "readable".
best regards Alex

where i get my first problem. If the strings lenght is a not a multiple of 4 (s.lenght % 4 == 0) i get a "Invalid length for a Base-64 char array" error.
That suggests that it's not base64 to start with. It sounds like you're going in the wrong direction here - base64 is used to convert binary data into text. To convert text into a binary form, you should normally just use Encoding.GetBytes:
return Encoding.UTF8.GetBytes(text);
Now if you needed to encode the result of the encryption (which will be binary data) as text, then you'd use base64. (Because the result of encrypting UTF-8-encoded text is not UTF-8-encoded text.)
So something like:
public static string EncryptText(string input)
{
byte[] unencryptedBytes = Encoding.UTF8.GetBytes(input);
byte[] encryptedBytes = EncryptBytes(unencryptedBytes); // Not shown here
return Convert.ToBase64String(encryptedBytes);
}
public static string DecryptText(string input)
{
byte[] encryptedBytes = Convert.FromBase64String(input);
byte[] unencryptedBytes = DecryptBytes(encryptedBytes); // Not shown here
return Encoding.UTF8.GetString(unencryptedBytes);
}

Error in converting from Encrypted value

I have a encryption method GetDecryptedSSN(). I tested it’s correctness by the following test. It works fine
//////////TEST 2//////////
byte[] encryptedByteWithIBMEncoding2 = DecryptionServiceHelper.GetEncryptedSSN("123456789");
string clearTextSSN2 = DecryptionServiceHelper.GetDecryptedSSN(encryptedByteWithIBMEncoding2);
But when I do a conversion to ASCII String and then back, it is not working correctly. What is the problem in the conversion logic?
//////////TEST 1//////////
//String -- > ASCII Byte --> IBM Byte -- > encryptedByteWithIBMEncoding
byte[] encryptedByteWithIBMEncoding = DecryptionServiceHelper.GetEncryptedSSN("123456789");
//encryptedByteWithIBMEncoding --> Encrypted Byte ASCII
string EncodingFormat = "IBM037";
byte[] encryptedByteWithASCIIEncoding = Encoding.Convert(Encoding.GetEncoding(EncodingFormat), Encoding.ASCII,
encryptedByteWithIBMEncoding);
//Encrypted Byte ASCII - ASCII Encoded string
string encodedEncryptedStringInASCII = System.Text.ASCIIEncoding.ASCII.GetString(encryptedByteWithASCIIEncoding);
//UpdateSSN(encodedEncryptedStringInASCII);
byte[] dataInBytesASCII = System.Text.ASCIIEncoding.ASCII.GetBytes(encodedEncryptedStringInASCII);
byte[] bytesInIBM = Encoding.Convert(Encoding.ASCII, Encoding.GetEncoding(EncodingFormat),
dataInBytesASCII);
string clearTextSSN = DecryptionServiceHelper.GetDecryptedSSN(bytesInIBM);
Helper Class
public static class DecryptionServiceHelper
{
public const string EncodingFormat = "IBM037";
public const string SSNPrefix = "0000000";
public const string Encryption = "E";
public const string Decryption = "D";
public static byte[] GetEncryptedSSN(string clearTextSSN)
{
return GetEncryptedID(SSNPrefix + clearTextSSN);
}
public static string GetDecryptedSSN(byte[] encryptedSSN)
{
return GetDecryptedID(encryptedSSN);
}
private static byte[] GetEncryptedID(string id)
{
ServiceProgram input = new ServiceProgram();
input.RequestText = Encodeto64(id);
input.RequestType = Encryption;
ProgramInterface inputRequest = new ProgramInterface();
inputRequest.Test__Request = input;
using (MY_Service operation = new MY_Service())
{
return ((operation.MY_Operation(inputRequest)).Test__Response.ResponseText);
}
}
private static string GetDecryptedID(byte[] id)
{
ServiceProgram input = new ServiceProgram();
input.RequestText = id;
input.RequestType = Decryption;
ProgramInterface request = new ProgramInterface();
request.Test__Request = input;
using (MY_Service operationD = new MY_Service())
{
ProgramInterface1 response = operationD.MY_Operation(request);
byte[] encodedBytes = Encoding.Convert(Encoding.GetEncoding(EncodingFormat), Encoding.ASCII,
response.Test__Response.ResponseText);
return System.Text.ASCIIEncoding.ASCII.GetString(encodedBytes);
}
}
private static byte[] Encodeto64(string toEncode)
{
byte[] dataInBytes = System.Text.ASCIIEncoding.ASCII.GetBytes(toEncode);
Encoding encoding = Encoding.GetEncoding(EncodingFormat);
return Encoding.Convert(Encoding.ASCII, encoding, dataInBytes);
}
}
REFERENCE:
Getting incorrect decryption value using AesCryptoServiceProvider

This is the problem, I suspect:
string encodedEncryptedStringInASCII =
System.Text.ASCIIEncoding.ASCII.GetString(encryptedByteWithASCIIEncoding);
(It's not entirely clear because of all the messing around with encodings beforehand, which seems pointless to me, but...)
The result of encryption is not "text encoded in ASCII" - so you shouldn't try to treat it that way. (You haven't said what kind of encryption you're using, but it would be very odd for it to produce ASCII text.)
It's just an arbitrary byte array. In order to represent that in text only using the ASCII character set, the most common approach is to use base64. So the above code would become:
string encryptedText = Convert.ToBase64(encryptedByteWithIBMEncoding);
Then later, you'd convert it back to a byte array ready for decryption as:
encryptedByteWithIBMEncoding = Convert.FromBase64String(encryptedText);
I would strongly advise you to avoid messing around with the encodings like this if you can help it though. It's not clear why ASCII needs to get involved at all. If you really want to encode your original text as IBM037 before encryption, you should just use:
Encoding encoding = Encoding.GetEncoding("IBM037");
string unencryptedBinary = encoding.GetBytes(textInput);
Personally I'd usually use UTF-8, as an encoding which can handle any character data rather than just a limited subset, but that's up to you. I think you're making the whole thing much more complicated than it needs to be though.
A typical "encrypt a string, getting a string result" workflow is:
Convert input text to bytes using UTF-8. The result is a byte array.
Encrypt result of step 1. The result is a byte array.
Convert result of step 2 into base64. The result is a string.
To decrypt:
Convert the string from base64. The result is a byte array.
Decrypt the result of step 1. The result is a byte array.
Convert the result of step 2 back to a string using the same encoding as step 1 of the encryption process.

In DecryptionServiceHelper.GetEncryptedSSN you are encoding the text in IBM037 format BEFORE encrypting.
So the following piece of code is not correct as you are converting the encrypted bytes to ASCII assuming that its in the IBM037 format. That's wrong as the encrypted bytes is not in IBM037 format (the text was encoded before encryption)
//encryptedByteWithIBMEncoding --> Encrypted Byte ASCII
string EncodingFormat = "IBM037";
byte[] encryptedByteWithASCIIEncoding = Encoding.Convert(Encoding.GetEncoding(EncodingFormat), Encoding.ASCII,
encryptedByteWithIBMEncoding);
One possible solution is to encode the encrypted text using IBM037 format, that should fix the issue I guess.

Encoding strings to and from base-64

I'm trying to encode some strings back and forth from base-64 string and I'm having truble to get the right result.
string text = base64string.... //Here I have a base-64 string.
byte[] encodedByte = System.Text.ASCIIEncoding.ASCII.GetBytes(text);
string base64Encoded = Convert.ToBase64String(encodedByte);
if (text == base64Encoded) //If the new encoded string is equal to its original value
return base64Encoded;
I have tried my ways to do this and I don't seem to get the right result. I have tried both with System.Text.Encoding.Unicode and System.Text.Encoding.UTF8
What could be the problem? Does anyone have a proper solution?

string text = base64string.... //Here I have a base-64 string.
byte[] encodedByte = System.Text.ASCIIEncoding.ASCII.GetBytes(text);
string base64Encoded = Convert.ToBase64String(encodedByte);
You are double encoding the string. You begin with a base64 string, get the bytes, and then encode it again. If you want to compare you will need to begin with the original string.

If text is a base-64 string, then you are doing it backwards:
byte[] raw = Convert.FromBase64String(text); // unpack the base-64 to a blob
string s = Encoding.UTF8.GetString(raw); // assume the blob is UTF-8, and
// decode to a string
which will get you it as a string. Note, though, that this scenario is only useful for representing unicode text in an ascii format. Normally you wouldn't base-64 encode it if the original contents are string.

Convert whatever it is that you need in Base64 into a Byte array then use the FromBase64String and ToBase64String to convert to and from Base64:
Byte[] buffer = Convert.FromBase64String(myBase64String1);
myBase64String2 = Convert.ToBase64String(buffer);
myBase64String1 will be equal to myBase64String2. You will need to use other methods to get your data type into a Byte array and the reverse to get your data type back. I have used this to convert the content of a class into a byte array and then to Base64 string and write the string to the filesystem. Later I read it back into a class instance by reversing the process.

You have the encoding code correctly laid out. To confirm whether the base64-encoded string is correct, you can try decoding it and comparing the decoded contents to the original:
var decodedBytes = Convert.FromBase64String(base64encoded);
var compareText = System.Text.Encoding.ASCII.GetString(decodedText);
if (text == compareText)
{
// carry on...
return base64encoded;
}

Invalid length for a Base-64 char array

I'm getting a "Invalid length for a Base-64 char array." inside of the IF(){...} are variations i have tried to get it to work. it fails in the first line without calling decrypt(...) proving it's not that functions problem. i get the same error inside with the first decrypt(...) call. the last one using the encoding.ascii... will get me inside the function, but then it fails inside the function. I'm getting the proper encrypted info from the database to string SSnum. it's value is: 4+mFeTp3tPF
try
{
string SSnum = dr.GetString(dr.GetOrdinal("Social Security"));
if (isEncrypted)
{
byte[] temp = Convert.FromBase64String(SSnum);
//SSnum = decrypt(Convert.FromBase64String(SSnum), Key, IV);
//SSnum = decrypt(Encoding.ASCII.GetBytes(SSnum), Key, IV);
}
txt_Social_Security.Text = SSnum;
}
catch { txt_Social_Security.Text = ""; }
I've been told to use the Convert.FromBase64String() and not the ASCII method...so why is it failing, how can i fix it?

Base64 data length should be multiple of 4 and with padding char '='
You can change your data as valid base64 data.
string dummyData = imgData.Trim().Replace(" ", "+");
if (dummyData.Length % 4 > 0)
dummyData = dummyData.PadRight(dummyData.Length + 4 - dummyData.Length % 4, '=');
byte[] byteArray = Convert.FromBase64String(dummyData);
https://stackoverflow.com/a/9301545/2024022
This will help you , try once.
Thanks
suribabu.

it's value is: 4+mFeTp3tPF
You are receiving this error because that value, 4+mFeTp3tPF, is in fact not valid Base64.
Is it possible you are simply missing the required padding character, as so 4+mFeTp3tPF=?

Are you certain that you have a Base64 string? Base64 is a means of encoding binary data into a string while only using standard 7-bit ASCII characters. It's not a string encoding like ASCII and has some control bytes present. You have a Base64 string if you're using Convert.ToBase64String to obtain the value (which, if you're trying to store binary data as a string, is your best bet)
Judging by your error (and your example data), I'm assuming that you do not have a Base64 string. If you need to store binary data in the database, you can either create a column using a binary type or encode the string into Base64 by using Convert.ToBase64String.
byte[] inputData = ...;
string base64String = Convert.ToBase64String(inputData);
byte[] outputData = Convert.FromBase64String(base64String);
Here, outputData should contain the same data as inputData.
If what you have is just an ASCII-encoded string, then your original practice of using System.Text.Encoding.ASCII.GetBytes() is correct, but you should change this to use a Base64 string if you can.

Are you sure that string 4+mFeTp3tPF is well-formed Base64 string?
I've tried some online services - no one could convert it.

replace
byte[] temp = Convert.FromBase64String(SSnum);
to this
var temp = UTF8Encoding.UTF8.GetBytes(SSnum);