We Keep Coding

XMLSerializer - issue with UTF-8 vs UTF-16 Code

I am trying to serialize a simple object (5 string properties) into XML to save to a DB Image field. Then I need to DeSerialize it back into a string later in the program.
However, I am getting some errors - caused by the XML being saved thinking it is in UTF-16 - however, when I load it from the DB back into a string - it thinks it is a UTF 8 String.
The error I get is
InnerException {"There is no Unicode byte order mark. Cannot switch to Unicode."} System.Exception {System.Xml.XmlException}
-- Message "There is an error in XML document (0, 0)." string
Is this happening because of the two different ways I save and load the string to/from the DB? On the save I am using a StringBuilder - but on the load from DB I am using just a String.
Thoughts?
Serialize and Save to DB
// Now Save the OBject XML to the Query Tables
var serializer = new XmlSerializer(ExportConfig.GetType());
StringBuilder StringResult = new StringBuilder();
using (var writer = XmlWriter.Create(StringResult))
{
serializer.Serialize(writer, ExportConfig);
}
//MessageBox.Show("XML : " + StringResult);
// Now Save to the Query
try
{
string UpdateSQL = "Update ZQryRpt "
+ " Set ExportConfig = " + TAGlobal.QuotedStr(StringResult.ToString())
+ " where QryId = " + TAGlobal.QuotedStr(((DataRowView)bindingSource_zQryRpt.Current).Row["QryID"].ToString())
;
ExecNonSelectSQL(UpdateSQL, uniConnection_Config);
}
catch (Exception Error)
{
MessageBox.Show("Error Setting ExportConfig: " + Error.Message);
}
Load from DB And Deserialize
byte[] binaryData = (byte[])((DataRowView)bindingSource_zQryRpt.Current).Row["ExportConfig"];
string XMLStored = System.Text.Encoding.UTF8.GetString(binaryData, 0, binaryData.Length);
if (XMLStored.Length > 0)
{
IIDExportObject ExportConfig = new IIDExportObject();
var serializer = new XmlSerializer(ExportConfig.GetType());
//StringBuilder StringResult = new StringBuilder(XMLStored);
// Load the XML from the Query into the StringBuilder
// Now we need to build a Stream from the String to use in the XMLReader
byte[] byteArray = Encoding.UTF8.GetBytes(XMLStored);
MemoryStream stream = new MemoryStream(byteArray);
using (var reader = XmlReader.Create(stream))
{
ExportConfig = (IIDExportObject)serializer.Deserialize(reader);
}
}

John - thank you very much for the comment! It allowed me to complete the code and find a solution.
As you noted - using a stream reader was the solution - but I could not read the first line because there was only one 'line' in my string. However, I could use the line
using (StreamReader sr = new StreamReader(stream, false))
Which allows me to read the stream and ignore the "Byte Order Mark Detection" set to false.
string XMLStored = MainFormRef.GetExportConfigForCurrentQuery();
if (XMLStored.Length > 0)
{
IIDExportObject ExportConfig = new IIDExportObject();
try
{
var serializer = new XmlSerializer(ExportConfig.GetType());
// Now we need to build a Stream from the String to use in the XMLReader
byte[] byteArray = Encoding.UTF8.GetBytes(XMLStored);
MemoryStream stream = new MemoryStream(byteArray);
// Now we need to use a StreamReader to get around UTF8 vs UTF16 issues
// A little cumbersome - but it works
using (StreamReader sr = new StreamReader(stream, false))
{
using (var reader = XmlReader.Create(sr))
{
ExportConfig = (IIDExportObject)serializer.Deserialize(reader);
}
}
}
catch
{
}
I am not sure this is the best solution - but it works. I will be curious to see if anyone else has a better way of dealing with this.

Thanks to G Bradley, I took his answer and generalized it a bit to make it a bit easier to call.
public static string SerializeToXmlString<T>(T objectToSerialize)
{
XmlSerializer serializer = new XmlSerializer(typeof(T));
XmlWriterSettings settings = new XmlWriterSettings();
settings.Indent = false;
settings.Encoding = Encoding.UTF8;
StringBuilder builder = new StringBuilder();
using (XmlWriter writer = XmlWriter.Create(builder, settings))
{
serializer.Serialize(writer, objectToSerialize);
}
return builder.ToString();
}
public static T DeserializeFromXmlString<T>(string xmlString)
{
if (string.IsNullOrWhiteSpace(xmlString))
return default;
var serializer = new XmlSerializer(typeof(T));
byte[] byteArray = Encoding.UTF8.GetBytes(xmlString);
MemoryStream stream = new MemoryStream(byteArray);
using (StreamReader sr = new StreamReader(stream, false))
{
using (var reader = XmlReader.Create(sr))
{
return (T)serializer.Deserialize(reader);
}
}
}

How to make use of memorystream instead of filestream

So what im trying to do is read a Select store procedure from my database save the data in a csv file and make it that the user is able to download it through the web application. I was able to get the requested result by saving the file temporary into my program foldel and using filestream. What i want to do now is skip the part where the file is saved onto my computer and temporary save it in the RAM memory instead. From what i understood i have to make use of memory stream instead of file stream but i dont really understand how i can do that. From what i understood from what i read is that instead of me making use of a file i need to convert my data to bytes make a memorystream out of it and then use it in my FileStreamResult. Am i correct here?
Method when i read from procedure and save to a csvfile:
public static String StoreApproved ()
{
string path1 = HttpRuntime.AppDomainAppPath + "Report.csv";
SqlConnection sqlConnection1 = new SqlConnection("CONNECTIONSTRING");
SqlCommand cmd = new SqlCommand();
SqlDataReader reader;
cmd.CommandText = "ExportApproved";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Connection = sqlConnection1;
sqlConnection1.Open();
reader = cmd.ExecuteReader();
List<ModelStoreProcedureApproved> TestList = new List<ModelStoreProcedureApproved>();
ModelStoreProcedureApproved test ;
while (reader.Read())
{
test = new ModelStoreProcedureApproved();
// test.Id = int.Parse(reader["IdTimeTracker"].ToString());
test.Month = reader["Month"].ToString();
test.EmailUser = reader["Email"].ToString();
test.Project = reader["Name"].ToString();
test.Approved = reader["Description"].ToString();
test.Month = reader["Month"].ToString();
test.Year = reader["Year"].ToString();
TestList.Add(test);
}
File.Create(path1).Close();
var i = TestList.FirstOrDefault();
using (TextWriter fileReader = new StreamWriter(path1))
{
var csv = new CsvWriter(fileReader);
csv.Configuration.Encoding = Encoding.UTF8;
foreach (var value in TestList)
{
csv.WriteRecord(value);
}
fileReader.Close();
}
sqlConnection1.Close();
return path1;
}
Controller code:
public ActionResult ExportToCSV()
{
string path = Repositories.UserRepository.StoreApproved();
var fileStream = new FileStream(path,
FileMode.Open,
FileAccess.Read);
return new FileStreamResult(fileStream, "text/csv") { FileDownloadName = "export.csv" };
}
Can someone explain me what the best way to do this is?
Other posts i have read
Serialize and Deserialize using BinaryFormatter
BinaryFormatter and Deserialization Complex objects
Using CSVHelper to output stream to browser

You can make it like this:
public static byte[] StoreApproved ()
{
string path1 = HttpRuntime.AppDomainAppPath + "Report.csv";
SqlConnection sqlConnection1 = new SqlConnection("CONNECTIONSTRING");
SqlCommand cmd = new SqlCommand();
SqlDataReader reader;
cmd.CommandText = "ExportApproved";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Connection = sqlConnection1;
sqlConnection1.Open();
reader = cmd.ExecuteReader();
List<ModelStoreProcedureApproved> TestList = new List<ModelStoreProcedureApproved>();
ModelStoreProcedureApproved test ;
while (reader.Read())
{
test = new ModelStoreProcedureApproved();
// test.Id = int.Parse(reader["IdTimeTracker"].ToString());
test.Month = reader["Month"].ToString();
test.EmailUser = reader["Email"].ToString();
test.Project = reader["Name"].ToString();
test.Approved = reader["Description"].ToString();
test.Month = reader["Month"].ToString();
test.Year = reader["Year"].ToString();
TestList.Add(test);
}
var i = TestList.FirstOrDefault();
var mem = new MemoryStream();
using (TextWriter fileReader = new StreamWriter(mem))
{
var csv = new CsvWriter(fileReader);
csv.Configuration.Encoding = Encoding.UTF8;
foreach (var value in TestList)
{
csv.WriteRecord(value);
}
}
sqlConnection1.Close();
return mem.ToArray();
}
public ActionResult ExportToCSV()
{
byte[] bytes = Repositories.UserRepository.StoreApproved();
Stream stream = new MemoryStream(bytes);
return new FileStreamResult(stream, "text/csv") { FileDownloadName = "export.csv" };
}

I suggest you make clean separation of concerns since you are also using Asp.Net MVC. Instead of reading and creating memory stream inside same method, first read/get the data collection you need and just return the data out of the method. Then inside the action method you can decorate it with required format(binding to UI or returning a file etc.) based on your requirement
Though this is not be a straight answer to your question, and if all that you are looking for is using a memory stream, there are plenty of examples available to use for example as shown here and the answer you accepted etc.
Hope this help you.

using (var ms = new MemoryStream())
{
using (var writer = new StreamWriter(ms))
using (var csv = new CsvWriter(writer))
{
csv.WriteRecords({A list here});
}
ms.ToArray() // here is your actual data in memory stream
}

Using DataContractJsonSerializer to create a Non XML Json file

I want to use the DataContractJsonSerializer to serialize to file in JsonFormat.
The problem is that the WriteObjectmethod only has 3 options XmlWriter, XmlDictionaryWriter and Stream.
To get what I want I used the following code:
var js = new DataContractJsonSerializer(typeof(T), _knownTypes);
using (var ms = new MemoryStream())
{
js.WriteObject(ms, item);
ms.Position = 0;
using (var sr = new StreamReader(ms))
{
using (var writer = new StreamWriter(path, false))
{
string jsonData = sr.ReadToEnd();
writer.Write(jsonData);
}
}
}
Is this the only way or have I missed something?

Assuming you're just trying to write the text to a file, it's not clear why you're writing it to a MemoryStream first. You can just use:
var js = new DataContractJsonSerializer(typeof(T), _knownTypes);
using (var stream = File.Create(path))
{
js.WriteObject(stream, item);
}
That's rather simpler, and should do what you want...

I am actually quite terrified to claim to know something that Jon Skeet doesn't, but I have used code similar to the following which produces the Json text file and maintains proper indentation:
var js = new DataContractJsonSerializer(typeof(T), _knownTypes);
using (var stream = File.Create(path))
{
using (var writer = JsonReaderWriterFactory.CreateJsonWriter(stream, Encoding.UTF8, true, true, "\t"))
{
js.WriteObject(writer, item);
writer.Flush();
}
}
(as suggested here.)

Binary Serialization to ResultBuffer in C#

I have a working XML Serializer which serializes a C# object to an entity in AutoCAD. I'd like to be able to do the same thing but with Binary Serialization for the cases in which XML does not work. So far my serialization method looks like this:
public static void BinarySave(Entity entityToWriteTo, Object objToSerialize, string key = "default")
{
using (MemoryStream stream = new MemoryStream())
{
BinaryFormatter serializer = new BinaryFormatter();
serializer.Serialize(stream, objToSerialize);
stream.Position = 0;
ResultBuffer data = new ResultBuffer();
/*Code to get binary serialization into result buffer*/
using (Transaction tr = db.TransactionManager.StartTransaction())
{
using (DocumentLock docLock = doc.LockDocument())
{
if (!entityToWriteTo.IsWriteEnabled)
{
entityToWriteTo = tr.GetObject(entityToWriteTo.Id, OpenMode.ForWrite) as Entity;
}
if (entityToWriteTo.ExtensionDictionary == ObjectId.Null)
{
entityToWriteTo.CreateExtensionDictionary();
}
using (DBDictionary dict = tr.GetObject(entityToWriteTo.ExtensionDictionary, OpenMode.ForWrite, false) as DBDictionary)
{
Xrecord xrec;
if (dict.Contains(key))
{
xrec = tr.GetObject(dict.GetAt(key), OpenMode.ForWrite) as Xrecord;
xrec.Data = data;
}
else
{
xrec = new Xrecord();
xrec.Data = data;
dict.SetAt(key, xrec);
tr.AddNewlyCreatedDBObject(xrec, true);
}
xrec.Dispose();
}
tr.Commit();
}
data.Dispose();
}
}
}
It's heavily based on my XML Serializer except I have no idea how to get the serialized object into a resultbuffer to be added to the Xrecord of entityToWriteTo.

If XML isn't working for you for some reason, I'd suggest trying a different textual data format such as JSON. The free and open-source JSON formatter Json.NET has some support for situations that can trip up XmlSerializer, including
Dictionaries.
Classes lacking default constructors.
polymorphic types.
Complex data conversion and remapping.
Plus, JSON is quite readable so you may be able to diagnose problems in your data by visual examination.
That being said, you can convert the output stream from BinaryFormatter to a base64 string using the following helper methods:
public static class BinaryFormatterHelper
{
public static string ToBase64String<T>(T obj)
{
using (var stream = new MemoryStream())
{
new BinaryFormatter().Serialize(stream, obj);
return Convert.ToBase64String(stream.GetBuffer(), 0, checked((int)stream.Length)); // Throw an exception on overflow.
}
}
public static T FromBase64String<T>(string data)
{
using (var stream = new MemoryStream(Convert.FromBase64String(data)))
{
var formatter = new BinaryFormatter();
var obj = formatter.Deserialize(stream);
if (obj is T)
return (T)obj;
return default(T);
}
}
}
The resulting string can then be stored in a ResultBuffer as you would store an XML string.

Create SqlXml object instead of string object

I have a method:
public static string UnZipStr(byte[] input)
{
if (input == null){
return null;
}
using (MemoryStream inputStream = new MemoryStream(input))
using (DeflateStream gzip = new DeflateStream(inputStream, CompressionMode.Decompress))
using (StreamReader reader = new StreamReader(gzip, System.Text.Encoding.UTF8))
{
return reader.ReadToEnd();
}
}
But I always get xml text unzipping, it is fact.
I need to change this method in order to return SqlXml object.
Unfortunate I'm java developer and cannot solve this task.

Do you need a SqlXml object or an XmlDocument/XDocument object? This post about converting a SqlXml object into an XmlDocument may be related to your needs.
You may be able to do the following:
public static string SqlXmlFromZippedBytes(byte[] input)
{
if (input == null){
return null;
}
using (MemoryStream inputStream = new MemoryStream(input))
using (DeflateStream gzip = new DeflateStream(inputStream, CompressionMode.Decompress))
using (StreamReader reader = new StreamReader(gzip, System.Text.Encoding.UTF8))
{
return new SqlXml(reader); // From System.Data.SqlTypes
}
}
Here is the documentation on the SqlXml constructor.