We Keep Coding

How do I append CSV using CSV helper?

I went through documentation and I didn't see anything for it. I'm using the same exact write function from documentation but the problem is I also need to append. So far I've tried reading it than adding the CSV into a list than I write the list to CSV. Is there a better way I could append?
The exact write function I use with my own variables
var records = new List<Foo>
{
new Foo { Id = 1, Name = "one" },
};
using (var writer = new StreamWriter("path\\to\\file.csv"))
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.WriteRecords(records);
}
What I use to read but with my own variables:
using (var reader = new StreamReader("path\\to\\file.csv"))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
var records = csv.GetRecords<Foo>();
}
Hopefully someone can help!

Use the StreamWriter(String, Boolean) constructor to specify you want to append to an existing file:
Initializes a new instance of the StreamWriter class for the specified file by using the default encoding and buffer size. If the file exists, it can be either overwritten or appended to. If the file does not exist, this constructor creates a new file.
Parameters
path
String
The complete file path to write to.
append
Boolean
true to append data to the file; false to overwrite the file. If the specified file does not exist, this parameter has no effect, and the constructor creates a new file.
As for your CsvHelper CsvWriter, you need to configure it to omit the header depending on if you are appending or creating:
bool append = true;
var config = new CsvConfiguration(CultureInfo.InvariantCulture);
config.HasHeaderRecord = !append;
using (var writer = new StreamWriter("path\\to\\file.csv", append))
{
using (var csv = new CsvWriter(writer, config))
{
csv.WriteRecords(records);
}
}

Write to a File using CsvHelper in C#

I tried to write to CSV file using CsvHelper in C#.
This is the link to the library http://joshclose.github.io/CsvHelper/
I used the code in web site.
Here is my code:
var csv = new CsvWriter(writer);
csv.Configuration.Encoding = Encoding.UTF8;
foreach (var value in valuess)
{
csv.WriteRecord(value);
}
It writes only a part of data to csv file.
Last rows were missing.
Could you please help with this.

You need to flush the stream. The Using statement will flush when out of scope.
using (TextWriter writer = new StreamWriter(#"C:\test.csv", false, System.Text.Encoding.UTF8))
{
var csv = new CsvWriter(writer);
csv.WriteRecords(values); // where values implements IEnumerable
}

when, I added this code after the loop code is working well
var csv = new CsvWriter(writer);
csv.Configuration.Encoding = Encoding.UTF8;
foreach (var value in valuess)
{
csv.WriteRecord(value);
}
writer.Close();
The problem occurred because I did not close the Connection

Assuming that writer is some kind of TextWriter, you should add a call to flush the contents before closing the writer:
writer.Flush()
If the last lines are missing, this is the most likely reason.

Adding to #greg's answer:
using (var sr = new StreamWriter(#"C:\out.csv", false, Encoding.UTF8)) {
using (var csv = new CsvWriter(sr)) {
csv.WriteRecords(values);
}
}

How to use XSLT to transform XML output of DataContractSerializer

I have become confused due to all of the samples using DataContractSerializer only handling one single object. I have a collection of objects, let's call it List<Ticket> tickets. I can get a the DataContractSerializer to write each object using a foreach (var ticket in tickets), but afterward I need to run a transform on the XML in order to be sure it is properly formatted. However, when using the Transform method of a XmlCompiledTransform I continue receiving the error "Unexpected end of file while parsing Name has occurred. Line 447, position 28."
Below is my code, all constructive criticism is welcome.
using (var ms = new MemoryStream())
{
using (var writer = XmlWriter.Create(ms, settings))
{
var ser = new DataContractSerializer(tickets.GetType());
writer.WriteStartDocument(true);
writer.WriteStartElement("Tickets");
foreach (var ticket in tickets)
{
ser.WriteObject(writer, ticket);
}
writer.WriteEndElement();
writer.WriteEndDocument();
ms.Position = 0;
var xslt = new XslCompiledTransform();
xslt.Load(xsltFp);
using (var output = new FileStream(xmlFp, FileMode.Create))
{
xslt.Transform(XmlReader.Create(ms), null, output);
output.Position = 0;
}
}
}

I figured it out. At the end of the foreach loop, I needed to call writer.Flush();. This effectively flushes the stream buffer before we start writing another object.

How to use JsonTextReader twice

I am given a stream of json data which contains a field named "type". This type field describes the type of object that needs to be created at runtime. It looks like I am unable to use the JsonTextReader twice and I cannot find away to reset the text reader to the beginning.
using (var streamReader = new StreamReader(stream, Encoding))
using (var jsonTextReader = new JsonTextReader(streamReader))
{
JToken token = JObject.Load(jsonTextReader);
var type = (string) token.SelectToken("type");
var modelType = Type.GetType("Project." + type + ", Project");
// Fails here
var obj = serializer.Deserialize(jsonTextReader, modelType);
}
I get this error message.
Unexpected token while deserializing object: EndObject.

You can create a JsonReader from the JToken.
JsonReader reader = token.CreateReader();

To reset your reader to the begginning, set the Position property of the underlying stream to 0.
streamReader.BaseStream.Position = 0;
Edit:
While this will reset your underlying stream, the jsonTextReader is forward-only by definition, which means its line number and position are readonly. For this to work you would have to reset the streamReader position, then feed it into a new JsonTextReader object.
So unfortunately Phil, there is no way to read the JsonTextReader twice since it is forward-only.
Reference:
http://james.newtonking.com/projects/json/help/html/T_Newtonsoft_Json_JsonTextReader.htm
"Represents a reader that provides fast, non-cached, forward-only access to serialized Json data."

I cover using the JsonTextReader in a memory-efficient format, avoiding the Large Object Heap, etc., in my blog, as per James Newton King's recommendations. You can leverage this and the supplied code to read your JSON multiple times without worrying about the underlying implementation of JsonTextReader.
Comments and feedback always welcome.

I did some more testing and found that the following works.
Set JsonTextReader.CloseInput = false
Destroy the JsonTextReader (by closing the using statement)
Set StreamReader.BaseStream.Position = 0
Create a new JsonTextReader
It would look something like this:
using (var streamReader = new StreamReader(stream, encoding))
{
Type modelType = null;
using (var jsonTextReader = new JsonTextReader(streamReader))
{
jsonTextReader.CloseInput = false;
JToken token = JObject.Load(jsonTextReader);
string type = (string)token.SelectToken("type");
modelType = Type.GetType("Project." + type + ", Project");
}
streamReader.BaseStream.Position = 0;
using (var jsonTextReader = new JsonTextReader(streamReader))
{
var obj = serializer.Deserialize(jsonTextReader, modelType);
}
}

BOM encoding for database storage

I'm using the following code to serialise an object:
public static string Serialise(IMessageSerializer messageSerializer, DelayMessage message)
{
using (var stream = new MemoryStream())
{
messageSerializer.Serialize(new[] { message }, stream);
return Encoding.UTF8.GetString(stream.ToArray());
}
}
Unfortunately, when I save it to a database (using LINQ to SQL), then query the database, the string appears to start with a question mark:
?<z:anyType xmlns...
How do I get rid of that? When I try to de-serialise using the following:
public static DelayMessage Deserialise(IMessageSerializer messageSerializer, string data)
{
using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(data)))
{
return (DelayMessage)messageSerializer.Deserialize(stream)[0];
}
}
I get the following exception:
"Error in line 1 position 1. Expecting
element 'anyType' from namespace
'http://schemas.microsoft.com/2003/10/Serialization/'..
Encountered 'Text' with name '',
namespace ''. "
The implementations of the messageSerializer use the DataContractSerializer as follows:
public void Serialize(IMessage[] messages, Stream stream)
{
var xws = new XmlWriterSettings { ConformanceLevel = ConformanceLevel.Fragment };
using (var xmlWriter = XmlWriter.Create(stream, xws))
{
var dcs = new DataContractSerializer(typeof(IMessage), knownTypes);
foreach (var message in messages)
{
dcs.WriteObject(xmlWriter, message);
}
}
}
public IMessage[] Deserialize(Stream stream)
{
var xrs = new XmlReaderSettings { ConformanceLevel = ConformanceLevel.Fragment };
using (var xmlReader = XmlReader.Create(stream, xrs))
{
var dcs = new DataContractSerializer(typeof(IMessage), knownTypes);
var messages = new List<IMessage>();
while (false == xmlReader.EOF)
{
var message = (IMessage)dcs.ReadObject(xmlReader);
messages.Add(message);
}
return messages.ToArray();
}
}

Unfortunately, when I save it to a database (using LINQ to SQL), then query the database, the string appears to start with a question mark:
?<z:anyType xmlns...
Your database is not set up to support Unicode characters. You write a string including a BOM in it, the database can't store it so mangles it into a '?'. Then when you come back to read the string as XML, the '?' is text content outside the root element and you get an error. (You can only have whitespace text outside the root element.)
Why is the BOM getting there? Because Microsoft love dropping BOMs all over the the place, even when they're not needed (and they never are, with UTF-8). The solution is to make your own instance of UTF8Encoding instead of using the built-in Encoding.UTF8, and tell it you don't want its stupid BOMs:
Encoding utf8onlynotasridiculouslysucky= new UTF8Encoding(false);
However, this is only really masking the real issue, which is the database configuration.