I've got a function that makes something equivalent to a web request, and it returns a formatted CSV. My goal is to now import this data into CsvHelper. However, I can't seem to get CSVParser to read from static text, only from a stream.
I could write the output to a file then read it back, but I feel that doesn't make much sense here.
I'm not tied down at all to CsvHelper, however I can't seem to find a CSV library that supports this behavior. How should I do this?
var csvString = functionThatReturnsCsv()
/* as string:
columnA,columnB
dataA,dataB
*/
// my goal
???.parse(csvString)
You can convert the string to a Stream in-memory and then use that as the source for your CSV reader:
public static Stream StringAsStream(string value)
{
return StringAsStream(value, System.Text.Encoding.UTF8);
}
public static Stream StringAsStream(string value, System.Text.Encoding encoding)
{
var bytes = encoding.GetBytes(value);
return new MemoryStream(bytes);
}
Usage:
using (var stream = StringAsStream("hello"))
{
// csv reading code here
}
or
using (var stream = StringAsStream("hello", Encoding.Ascii))
{
// csv reading code here
}
Try it online
Note If you are reading from a source that can return a Stream (like a web request), you should use that Stream rather than doing this.
You could use StringReader. The CsvReader constructor takes a TextReader argument rather than a Stream. If you did have a stream instead of a string, just replace StringReader with StreamReader.
public static void Main(string[] args)
{
using (var reader = new StringReader(FunctionThatReturnsCsv()))
using (var csv = new CsvReader(reader))
{
var results = csv.GetRecords<Foo>().ToList();
}
}
public static string FunctionThatReturnsCsv()
{
return "columnA,columnB\ndataA,dataB";
}
public class Foo
{
public string columnA { get; set; }
public string columnB { get; set; }
}
Related
Using Azure storage, I'm writing to a blob using a stream. I have a method something like this:
public async Task<BlobSteamContainer> GetBlobStreamAsync(string filename, string contentType = "text/csv")
{
var blob = container.GetBlockBlobReference($"{filename}--{Guid.NewGuid().ToString()}.csv");
blob.Properties.ContentType = contentType;
return new BlobSteamContainer(blob.Uri.ToString(), await blob.OpenWriteAsync());
}
Where BlobStreamContainer is just a simple object so I can keep track of the filename and the stream together:
public class BlobSteamContainer : IDisposable
{
public CloudBlobStream Stream { get; private set; }
public string Filename { get; private set; }
public BlobSteamContainer(string filename, CloudBlobStream stream)
{
Stream = stream;
Filename = filename;
}
public void Dispose()
{
Stream.Close();
Stream?.Dispose();
}
}
And then I use it something like this:
using (var blobStream = await GetBlobStreamAsync(filename))
using (var outputStream = new StreamWriter(blobStream.Stream))
using (var someInputStream = ...)
{
try
{
outputStream.WriteLine("write some stuff...");
//....processing
if (someCondition) {
throw new MyException("can't write the file");
}
//....more processing
outputStream.Flush();
}
catch(MyException e)
{
// what to do here? I want to stop writing
// and remove any trace of the file in azure
throw; // let the higher ups handle this
}
}
Where somecondition is something that I know before hand (obviously there's more going on involving processing an input stream and writing out as I go). If everything is fine, then this works great. My problem is on figuring out the best way to handle the case where an exception is thrown during writing.
I tried just deleting the file in the catch like this:
DeleteBlob(blobStream.Filename);
where:
public void DeleteBlob(string filename)
{
var blob = container.GetBlobReference(filename);
blob.Delete();
}
But the problem is that the file might not have been created yet and so this will throw a Microsoft.WindowsAzure.Storage.StorageException telling me the file wasn't found (and then the file will end up getting created anyway!).
So what would be the cleanest way to handle this?
I have code:
public Upload.UploadResponse Post(Upload.UploadRequest request)
{
Stream str = request.RequestStream; // RequestStream is System.Web.HttpInputStream
byte[] result;
using (var streamReader = new MemoryStream())
{
str.CopyTo(streamReader);
result = streamReader.ToArray();
}
return new Upload.UploadResponse() { Successed = 1 };
}
Is there any way to get file name ( with extension) from MemoryStream or stream or System.Web.HttpInputStream (part of Upload.UploadRequest request) without saving the file? I need to recognize the file without knowing what is sent to me. I've tried to cast it to FileStream but it was null. Service framework that I am using is service stack ServiceStack
edit: Maybe I need to send file info with request?
p.s sorry for my poor English any corrections are welcome
EDIT:
this is UploadClass that I am using for code above
public class Upload
{
[Route("/upload")]
public class UploadRequest : IRequiresRequestStream
{
public System.IO.Stream RequestStream { set; get; }
}
public class UploadResponse
{
public int Successed { set; get; }
}
}
You cannot extract file name from stream.
You need to add FileName property to your request.
How can I read an Excel spreadsheet that was just posted to my server?
I searched for something but I only found how to read an Excel spreadsheet with the file name path which is not my case.
I need something like that:
public ActionResult Import(HttpPostedFileBase file)
{
var excel = new ExcelQueryFactory(file); //using linq to excel
}
I was running into your same issue but I didn't want to switch to a paid service so this is what I did.
public class DataImportHelper : IDisposable
{
private readonly string _fileName;
private readonly string _tempFilePath;
public DataImportHelper(HttpPostedFileBase file, string tempFilePath)
{
_fileName = file.FileName;
_tempFilePath = Path.Combine(tempFilePath, _fileName);
(new FileInfo(_tempFilePath)).Directory.Create();
file.SaveAs(_tempFilePath);
}
public IQueryable<T> All<T>(string sheetName = "")
{
if (string.IsNullOrEmpty(sheetName))
{
sheetName = (typeof (T)).Name;
}
var excelSheet = new ExcelQueryFactory(_tempFilePath);
return from t in excelSheet.Worksheet<T>(sheetName)
select t;
}
public void Dispose()
{
File.Delete(_tempFilePath);
}
}
Here is a Test
[Fact]
public void AcceptsAMemoryStream()
{
MemoryFile file;
using (var f = File.OpenRead("SampleData.xlsx"))
{
file = new MemoryFile(f, "multipart/form-data", "SampleData.xlsx");
using (var importer = new DataImportHelper(file, "Temp/"))
{
var products = importer.All<Product>();
Assert.NotEmpty(products);
}
}
}
Here is MemoryFile.cs. This file is only used for testing. It is just an implementation of HttpPostedFileBase so you can test your controllers and my little helper. This was borrowed from another post.
public class MemoryFile : HttpPostedFileBase
{
Stream stream;
string contentType;
string fileName;
public MemoryFile(Stream stream, string contentType, string fileName)
{
this.stream = stream;
this.contentType = contentType;
this.fileName = fileName;
}
public override int ContentLength
{
get { return (int)stream.Length; }
}
public override string ContentType
{
get { return contentType; }
}
public override string FileName
{
get { return fileName; }
}
public override Stream InputStream
{
get { return stream; }
}
public override void SaveAs(string filename)
{
using (var file = File.Open(filename, FileMode.Create))
stream.CopyTo(file);
}
}
Unfortunately it's not possible to read a spreadsheet from a stream with LinqToExcel.
That's because it uses OLEDB to read from the spreadsheets and it can't read from a stream.
You can use the InputStream property of HttpPostedFileBase to read the excel spreadsheet in memory.
I use ClosedXML nuget package to read excel content from stream which is available in your case. It has a simple overload which takes stream pointing to stream for the excel file (aka workbook).
imported namespaces at the top of the code file:
using ClosedXML.Excel;
Source code:
public ActionResult Import(HttpPostedFileBase file)
{
//HttpPostedFileBase directly is of no use so commented your code
//var excel = new ExcelQueryFactory(file); //using linq to excel
var stream = file.InputStream;
if (stream.Length != 0)
{
//handle the stream here
using (XLWorkbook excelWorkbook = new XLWorkbook(stream))
{
var name = excelWorkbook.Worksheet(1).Name;
//do more things whatever you like as you now have a handle to the entire workbook.
var firstRow = excelWorkbook.Worksheet(1).Row(1);
}
}
}
You need Office Interops assemblies. Check the Excel Object Model for reference.
When writing to a stream the DataContractSerializer uses an encoding different from Unicode-16. If I could force it to write/read Unicode-16 I could store it in a SQL CE's binary column and read it with SELECT CONVERT(nchar(1000), columnName). But the way it is, I can't read it, except programatically.
Can I change the encoding used by System.Runtime.Serialization.DataContractSerializer?
The DataContractSerializer's WriteObject method has overloads which write to a Stream or to a XmlWriter (and XmlDictionaryWriter). The Stream overload will default to UTF-8, so you'll need to use another one. Using a XML Writer instance which writes the XML in UTF-16 do what you needs, so you can either do what #Phil suggested, or you can use the writer returned by XmlDictionaryWriter.CreateTextWriter for which you pass an Encoding.Unicode as a parameter.
public class StackOverflow_10089682
{
[DataContract(Name = "Person", Namespace = "http://my.namespace")]
public class Person
{
[DataMember]
public string Name { get; set; }
[DataMember]
public int Age { get; set; }
}
public static void Test()
{
MemoryStream ms = new MemoryStream();
XmlDictionaryWriter writer = XmlDictionaryWriter.CreateTextWriter(ms, Encoding.Unicode);
DataContractSerializer dcs = new DataContractSerializer(typeof(Person));
Person instance = new Person { Name = "John Doe", Age = 33 };
dcs.WriteObject(writer, instance);
writer.Flush(); // Don't forget to Flush the writer here
Console.WriteLine("Decoding using UTF-16: {0}", Encoding.Unicode.GetString(ms.ToArray()));
}
}
Have you tried using XmlWriterSettings? Something like
var s = new DataContractSerializer (typeof(Thing));
using(var wr = XmlTextWriter.Create(
#"test.xml", new XmlWriterSettings{Encoding=Encoding.UTF32}))
{
s.WriteObject(wr, new Thing{Foo="bar"});
}
public class Thing
{
public string Foo { get; set; }
}
Specify the Encoding you require.
I have the following code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
namespace ConsoleApplication28
{
class Program
{
static void Main()
{
List<string> dirs = FileHelper.GetFilesRecursive(#"c:\Documents and Settings\bob.smith\Desktop\Test");
foreach (string p in dirs)
{
Console.WriteLine(p);
}
//Write Count
Console.WriteLine("Count: {0}", dirs.Count);
Console.Read();
}
static class FileHelper
{
public static List<string> GetFilesRecursive(string b)
{
// 1.
// Store results in the file results list.
List<string> result = new List<string>();
// 2.
// Store a stack of our directories.
Stack<string> stack = new Stack<string>();
// 3.
// Add initial directory.
stack.Push(b);
// 4.
// Continue while there are directories to process
while (stack.Count > 0)
{
// A.
// Get top directory
string dir = stack.Pop();
try
{
// B
// Add all files at this directory to the result List.
result.AddRange(Directory.GetFiles(dir, "*.*"));
// C
// Add all directories at this directory.
foreach (string dn in Directory.GetDirectories(dir))
{
stack.Push(dn);
}
}
catch
{
// D
// Could not open the directory
}
}
return result;
}
}
}
}
The code above works well for recursively finding what files/directories lie in a folder on my c:.
I am trying to serialize the results of what this code does to an XML file but I am not sure how to do this.
My project is this: find all files/ directories w/in a drive, serialize into an XML file. Then, the second time i run this app, i will have two XML files to compare. I then want to deserialize the XML file from the first time i ran this app and compare differences to the current XML file and produce a report of changes (i.e. files that have been added, deleted, updated).
I was hoping to get some help as I am a beginner in C# and i am very very shaky on serializing and deserializing. I'm having lots of trouble coding. Can someone help me?
Thanks
Your result is List<string> and that is not directly serializable. You'll have to wrap it, a minimal approach:
[Serializable]
class Filelist: List<string> { }
And then the (De)Serialization goes like:
Filelist data = new Filelist(); // replaces List<string>
// fill it
using (var stream = File.Create(#".\data.xml"))
{
var formatter = new System.Runtime.Serialization.Formatters.Soap.SoapFormatter();
formatter.Serialize(stream, data);
}
data = null; // lose it
using (var stream = File.OpenRead(#".\data.xml"))
{
var formatter = new System.Runtime.Serialization.Formatters.Soap.SoapFormatter();
data = (Filelist) formatter.Deserialize(stream);
}
But note that you will not be comparing the XML in any way (not practical). You will compare (deserialzed) List instances. And the XML is SOAP formatted, take a look at it. It may not be very useful in another context.
And therefore you could easily use a different Formatter (binary is a bit more efficient and flexible).
Or maybe you just want to persist the List of files as XML. That is a different question.
For anyone who is having trouble with xml serialization and de-serialization. I have created a sample class to do this below. It works for recursive collections also (like files and directories).
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace Sample
{
[Serializable()]
[System.Xml.Serialization.XmlRootAttribute(Namespace = "", IsNullable = false, ElementName = "rootnode")]
public partial class RootNode
{
[System.Xml.Serialization.XmlElementAttribute("collection1")]
public List<OuterCollection> OuterCollections { get; set; }
}
[Serializable()]
public partial class OuterCollection
{
[XmlAttribute("attribute1")]
public string attribute1 { get; set; }
[XmlArray(ElementName = "innercollection1")]
[XmlArrayItem("text", Type = typeof(InnerCollection1))]
public List<InnerCollection1> innerCollection1Stuff { get; set; }
[XmlArray("innercollection2")]
[XmlArrayItem("text", typeof(InnerCollection2))]
public List<InnerCollection2> innerConnection2Stuff { get; set; }
}
[Serializable()]
public partial class InnerCollection2
{
[XmlText()]
public string text { get; set; }
}
public partial class InnerCollection1
{
[XmlText()]
public int number { get; set; }
}
}
This class serializes and deserializes itself....hopefully this helps.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Xml.Serialization;
using System.Xml;
namespace TestStuff
{
public class Configuration
{
#region properties
public List<string> UIComponents { get; set; }
public List<string> Settings { get; set; }
#endregion
//serialize itself
public string Serialize()
{
MemoryStream memoryStream = new MemoryStream();
XmlSerializer xs = new XmlSerializer(typeof(Configuration));
using (StreamWriter xmlTextWriter = new StreamWriter(memoryStream))
{
xs.Serialize(xmlTextWriter, this);
xmlTextWriter.Flush();
//xmlTextWriter.Close();
memoryStream = (MemoryStream)xmlTextWriter.BaseStream;
memoryStream.Seek(0, SeekOrigin.Begin);
StreamReader reader = new StreamReader(memoryStream);
return reader.ReadToEnd();
}
}
//deserialize into itself
public void Deserialize(string xmlString)
{
String XmlizedString = null;
using (MemoryStream memoryStream = new MemoryStream())
{
using (StreamWriter w = new StreamWriter(memoryStream))
{
w.Write(xmlString);
w.Flush();
XmlSerializer xs = new XmlSerializer(typeof(Configuration));
memoryStream.Seek(0, SeekOrigin.Begin);
XmlReader reader = XmlReader.Create(memoryStream);
Configuration currentConfig = (Configuration)xs.Deserialize(reader);
this.Settings = currentConfig.Settings;
this.UIComponents = currentConfig.UIComponents;
w.Close();
}
}
}
static void Main(string[] args)
{
Configuration thisConfig = new Configuration();
thisConfig.Settings = new List<string>(){
"config1", "config2"
};
thisConfig.UIComponents = new List<string>(){
"comp1", "comp2"
};
//serializing the object
string serializedString = thisConfig.Serialize();
Configuration myConfig = new Configuration();
//deserialize into myConfig object
myConfig.Deserialize(serializedString);
}
}
}
John:
May I suggest an improvement? Instead of using filenames, use the FileInfo object. This will allow you to get much more accurate information about each file rather than just if it exists under the same name.
Also, the XmlSerializer class should do you just fine. It won't serialize generic lists, so you'll have to output your List<> to an array or some such, but other than that:
XmlSerializer serial = new XmlSerializer(typeof(FileInfo[]));
StringWriter writer = new StringWriter();
FileInfo[] fileInfoArray = GetFileInfos();
serial.Serialize(writer, fileInfoArrays);
Simple and easy, unless it matters to you how the serialized XML looks.
Whatever you do, lose the empty catch block. You WILL regret swallowing exceptions. Log them or re-throw them.
Help #1. Indent code by four spaces to have it be seen as code when you post here.
2: get rid of that try/catch block, as it will eat all exceptions, including the ones you want to know about.
3: Did you make any attempt at all to serialize your results? Please edit your question to show what you tried. Hint: use the XmlSerializer class.
for xml serialisation, you have several possibilities:
Doing it manually
Using XmlSerializer as detailed above
Using System.Xml.Serialization
Using Linq to Xml
for the last two, see a code example in this answer. (See some gotchas here)
And for your recursive directory visitor, you could consider making it really recursive: here's some interesting code examples.
This question is exactly like this one. I also have a posted answer which will work for you as well:
How to serialize?