I'm using the TIBCO EMS library TIBCO.EMS.dll to send xml messages to a queue on a TIBCO EMS server. The application receiving those messages requires the XML to be UTF-8 encoded. Generating the UTF-8 xml in itself isn't a problem, however I can see no method of sending a TextMessage to the queue while keeping the data in UTF-8 format.
To serialize the objects to UTF-8 XML I use the following (simplified here):
XmlSerializer serializer = new XmlSerializer(data.GetType());
MemoryStream ms = new MemoryStream();
StreamWriter sw = new StreamWriter(ms, System.Text.Encoding.UTF8);
serializer.Serialize(sw, data);
byte[] result = ms.ToArray();
Which leaves me with a byte array containing the utf-8 encoded xml. I can write this to a BytesMessage to send to the EMS queue..
BytesMessage message = _queueSession.CreateBytesMessage();
message.WriteBytes(result);
_queueSender.Send(message);
_queueSession.Commit();
But that results in a BytesMessage on the queue. The only way I can see to get a TextMessage is to use the TextMessage class but the text property of that class is a standard Unicode string which would result in the xml loosing its utf-8 encoding.
Anyone know of a way to send a UTF-8 encoded text message?
You may want to try call the Tibems.setEncoding("UTF-8") method before you send the message;
Please note that this method will impact the message encoding globally.
It seems that, by default, the TIBCO API converts C# unicode strings to UTF-8 when a message is submitted to a queue. Fine for text, but if the string uses XML and include an encoding type option, you must manually change the option to utf-8.
Related
I need to upload/create a file on an FTP server with UCS-2 LE BOM encoding. I'm using C#.
Based on Microsoft's documentation, I have to use UnicodeEncoding with bigEndian:false and byteOrderMark:true. Here is the code:
using (WebClient client = new WebClient())
{
client.Encoding = new UnicodeEncoding(false, true);
client.Credentials = myCredentials;
client.UploadString(path, WebRequestMethods.Ftp.UploadFile, myCsvInString);
}
The created file on the FTP server actually has UCS-2 Little Endian. For test purposes, I tried to switch the byteOrderMark to false and I got the same result.
Why? What am I missing?
I know that I could add '\uFEFF' but why isn't it done automatically?
The interface and description of UnicodeEncoding could be improved regarding the handling of the byte order mark. UnicodeEncoding has a byte order mark attribute but the only method (aside from Equals and GetHashCode) using it is GetPreamble. All other methods and in particular the core method GetBytes don't.
The idea is to ensure the byte order mark is only ever written at the beginning of a file. UnicodeEncoding has no knowledge of the context. So it's up to the caller to add the preamble if needed (ie. byte order mark).
Based on this concept, WebClient.UploadString cannot assume it's uploading a file. It might be some other Unicode content. So it doesn't add the preamble.
You will have to add the preamble yourself. UnicodeEncoding.GetPreamble will return it.
I am trying to read some protobuf data from a collegue - he created it in C++ and set the encoding to unicode, and transfer mode for protobuf is binary.
In python, this works perfectly:
with open('test.out', 'rb') as f:
dfile = protoclass_pb2.DataFile()
dfile.ParseFromString(f.read())
print(dfile.MetaData.author)
just like a charm.
In C# howevern, I try:
string filepath = "test.out"
FileStream fst = new FileStream(filepath,
DataFile data = DataFile.Parser.ParseDelimitedFrom(fst);
fst.Close();
and get the Exception:
Google.Protobuf.InvalidProtocolBufferException: 'Protocol message contained a tag with an invalid wire type.'
I tired somehow setting an encoding to the stream, but as far I can tell, I can only set encoding to StreamReader but not Stream itself.
Just reading the file content to an array produces the same in both languages.
How can I read the data in to C#?
Found the problem: the collegue serialized the message with SerializeToOstream and I tried reading with ParseDelimitedFrom.
Easlily fixed by either using SerializeDelimitedToOstream to serialize or ParseFrom to read.
After lots of efforts I created my own mail parser. Now successfully able to parse and display emails. But few mails especially sent from apple or Iphone appear like this after parsing. I have no idea why this is happening. Please help.
=D8=AA=D9=88=D8= =A7=D8=AC=D9=87=D9=86=D9=8A =D9=85=D8=B4=D9=83=D9=84=D8=A9 =D8=A5=D8=B4=D8= =A7=D8=B1=D8=A9 =D9=84=D9=84=D9=83=D8=B1=D8=AA =D8=B1=D9=82=D9=85 410814189= 68 =D8=B9=D9=84=D9=85=D8=A7=D9=8B =D8=A8=D8=A3=D9=86 =D8=A5=D8=B4=D8=
It would appear that you mail parser does not handle decoding of Quoted Printable content.
I imagine that if you looked at the headers, you'd find a header like this:
Content-Transfer-Encoding: quoted-printable
I've written several email clients and multiple mime parsers and am currently working on writing a new mime parser in C# (the others were in C) called MimeKit here: http://github.com/jstedfast/MimeKit. This may be of interest to you...
I've got a filterable stream class that you can add a QuotedPrintableDecoder to (which I've also implemented), then pass your data through that to decode it. Or you could just pass it through the QuotedPrintableDecoder directly, depending on whatever is easiest for you.
Example usage:
var decoder = new QuotedPrintableDecoder ();
var output = new byte[decoder.EstimateOutputLength (input.Length)];
var outputLength = decoder.Decode (input, 0, input.Length, output);
// convert the output into a string displayable to the user...
var text = System.Text.Encoding.UTF8.GetString (output, 0, outputLength);
Obviously you'd use the proper System.Text.Encoding for the content (by looking at the "charset" parameter in the Content-Type header) instead of blindly using System.Text.Encoding.UTF8.
I have a PHP SOAP server (using nuSOAP with wsdl) that send the content of a html page. Of course, the HTML can be coded with differents encoding and here is when the problems appear. If I used a PHP SOAP client I can send the encoding like this:
$clienteSOAP = new SoapClient ("http://test.mine.com/wsdl/filemanager?wsdl",
array ('encoding' => 'ISO-8859-15'));
$clienteSOAP->__soapCall ("Test.uploadHTML",
array (file_get_contents ('/home/КОЛЛЕКЦИЯ_РОДНИК_ПРЕМИУМ.html')));
And if I put the correct encoding, has never failed so far. But when I use a C# client, how can I put the encoding in the web service petition? In C# the code is:
System.IO.StreamReader html = new System.IO.StreamReader (
"C:\\Documents and Settings\\КОЛЛЕКЦИЯ_РОДНИК_ПРЕМИУМ.html"
,System.Text.Encoding.GetEncoding("iso-8859-15"));
string contenido = html.ReadToEnd();
html.Close();
Test.FileManager upload = new Test.FileManager();
string resultado = upload.TestUploadHTML (contenido);
Test.FileManager is a Web reference of the wsdl, and when I see the "upload html" some characters aren't correct.
Thanks in advance.
nusoap internally uses the php function xml_parser_create, that only supports: ISO-8859-1, UTF-8 and US-ASCII. For this reason, this library don't works well with other encoding. Great PacHecoPe...
UPDATE: The best option, in my case, is read the archive in its original encoding and transform it to utf-8:
System.IO.StreamReader html = new System.IO.StreamReader (
"C:\\Documents and Settings\\КОЛЛЕКЦИЯ_РОДНИК_ПРЕМИУМ.html"
,System.Text.Encoding.GetEncoding("iso-8859-15"));
string contenido = html.ReadToEnd();
html.Close();
System.Text.UTF8Encoding encoder = new System.Text.UTF8Encoding();
byte[] bytes = System.Text.Encoding.UTF8.GetBytes (contenido);
string contenidoUTF8 = encoder.GetString(bytes);
upload.RequestEncoding = System.Text.Encoding.GetEncoding("UTF-8");
Test.FileManager upload = new Test.FileManager();
string resultado = upload.TestUploadHTML (contenidoUTF8);
UPDATE2: With encoding that not supported in UTF-8 like big5, don't work very well the above code. For this reason, it's better don't make the transform to UTF-8 and set the parameter with the content of html like base64Binary, in the wsdl.
I am working on C# sockets and using XMLSerializer to send and receive data.
The XML data are sent from a server to a client over a network connection using TCP/IP protocol. The XML.Serializer.Serialize(stream) serializes the XML data and send them over the socket connection but when I want to use the XMLSerializer.Deserialize(stream) to read. The sent data returns a xml parse error.
Here is how I'm serializing:
Memory Stream ms = new MemoryStream();
FrameClass frame= new FrameClass ();
frame.string1 = "hello";
frame.string2 = "world";
XmlSerializer xmlSerializer = new XmlSerializer(frame.GetType());
xmlSerializer.Serialize(ms, frame);
socket.Send(ms.GetBuffer(), (int)ms.Length, SocketFlags.None);
Deserializing:
FrameClass frame;
XmlSerializer xml = new XmlSerializer(typeof(FrameClass));
frame= (FrameClass)xml.Deserialize(new MemoryStream(sockCom.SocketBuffer));
listbox1.Items.Add(frame.string1);
listbox2.Items.Add(frame.string2);
I think it has something to do with sending the data one right after another.
Can anyone teach me how to do this properly?
Have you received all of the data before attempting to deserialize (it's not clear from your code). I'd be inclined to receive all of the data into a local string and the deserialize from that rather than attempting to directly deserialize from the socket. It would also allow you to actually look at the data in the debugger before deserializing it.
Try this:
using (MemoryStream ms = new MemoryStream())
{
FrameClass frame= new FrameClass ();
frame.string1 = "hello";
frame.string2 = "world";
XmlSerializer xmlSerializer = new XmlSerializer(frame.GetType());
xmlSerializer.Serialize(ms, frame);
ms.Flush();
socket.Send(ms.GetBuffer(), (int)ms.Length, SocketFlags.None);
}
If you're sending the Frame XML one right after the other, then you're not sending an XML document. The XML Serializer will attempt to deserialize your entire document!
I don't have time to research this now, but look into the XmlReaderSettings property for reading XML fragments. You would then create an XmlReader over the memorystream with those settings, and call it in a loop.
The important thing is to flush the stream. It's also useful to put the stream in a using block to ensure it's cleaned up quickly.
Besides what #John said about the Flush call, your code looks alright.
You say you're sending multiple FrameClass data pieces, then the code should work sending just a single piece of data.
If you need to send multiple data objects, then you cannot send them all in one go, otherwise the deserialization process will stumble over the data.
You could setup some communication between the server & the client so the server knows what it's getting.
client: I have some data
Server: ok I'm ready, send it
client: sends
Server: done processing
repeat process...