I am reading a NetworkStream for json string and then deserializing it using Newtonsoft.Json.
Sometimes, two json objects could be sent back-to-back and read at the same time on the stream. But the Newtonsoft.Json serializer gives me only one object.
For example, if I have the following string on the stream:
{"name":"John Doe","age":10}{"name":"Jane Doe","age":10}
If I deserialize the stream, the serializer reads the entire stream, but gives only the first object.
Is there a way to make the serializer read only the first object from the stream and then read the next object in the next iteration of a loop?
Code:
public static Person Deserialize(Stream stream)
{
var Serializer = new JsonSerializer();
var streamReader = new StreamReader(stream, new UTF8Encoding());
return Serializer.Deserialize<Person>(new JsonTextReader(streamReader));
}
I cannot deserialize as a list because I'm not receiving a json array.
I think you can do it like this:
public static IList<Person> Deserialize(Stream stream) {
var serializer = new JsonSerializer();
var streamReader = new StreamReader(stream, new UTF8Encoding());
var result = new List<Person>();
using (var reader = new JsonTextReader(streamReader)) {
reader.CloseInput = false;
// important part
reader.SupportMultipleContent = true;
while (reader.Read()) {
result.Add(serializer.Deserialize<Person>(reader));
}
}
return result;
}
Important part is SupportMultipleContent property, which notifies reader that there might be multiple json objects side to side.
you can try it doing like this
var httpRequest = HttpContext.Current.Request;
// This list will have all the stream objects
var persons = new List<Person>();
if (httpRequest.Files.Count > 0)
{
for (var obj = 0; doc < httpRequest.Files.Count; obj++)
{
var postedFile = httpRequest.Files[obj];
var bytes = new byte[postedFile.ContentLength];
postedFile.InputStream.Read(bytes, 0, postedFile.ContentLength);
persons.Add(Serializer.Deserialize<Person>(new JsonTextReader(new StreamReader(new MemoryStream(bytes)))));
}
}
Related
Please note this is not a duplicate question as it is about parsing (and not deserializing) a large array object by object and retrieving the raw JSON.
I am dealing with very large arrays of JSON payload (tens of GB).
The structure of each object may be different i.e. heterogeneous:
[
{"id": "foo", "value": "bar"},
{"key": "foo", "name": "bar", "age": 10},
...
]
How can I go through the stream processing each object one at the time and retrieve the raw JSON string of each object?
My current solution is to use the StreamReader together with JsonTextReader to deserialize each object with a JObject and then retrieve the JSON using the .ToString(). But I prefer to avoid the performance cost and the GC allocation pressure of having to deserialize from JSON only to retrieve the JSON back again.
var file = new FileInfo(#"C:\Very.Large.Array.json");
using (var sr = new StreamReader(file.OpenRead()))
using (var reader = new JsonTextReader(sr))
{
while (reader.Read())
{
if (reader.TokenType == JsonToken.StartObject)
{
var obj = JObject.Load(reader);
var rawJSON = obj.ToString();
}
}
}
I believe you should use JsonTextWriter along with JsonTextReader. See below the simple POC class that demonstrates the idea.
I guess that some polishing is still required to bring this code to production quality. Like you may promote the StringBuilder sb from the local variable to the instance field and clear it at each iteration instead of creating new object.
But my goal was only to show the basic idea.
public class JsonBigFileReader
{
static string ReadSingleObject(JsonTextReader reader)
{
StringBuilder sb = new StringBuilder();
using (var sw = new StringWriter(sb))
{
using (var writer = new JsonTextWriter(sw))
{
writer.WriteToken(reader, true); // writes current token including its children (meaning the whole object)
}
}
return sb.ToString();
}
public IEnumerable<string> ReadArray(string fileName)
{
var file = new FileInfo(fileName);
using (var sr = new StreamReader(file.OpenRead()))
using (var reader = new JsonTextReader(sr))
{
reader.Read();
while (reader.Read())
{
if (reader.TokenType == JsonToken.StartObject)
{
yield return ReadSingleObject(reader);
}
}
}
}
}
I'm trying to use Microsoft's Serivce Bus Brokered Message APIs and I'm encountering deserialization issues when trying to use derived types. It seems like the code does not recognize the known types, and i have no issues when serializing these types and returning them between other services. Is there something I'm missing? I've gone through a ton of SO posts and nothing seemed to work.
Here is the message structure:
[DataContract]
[KnownType(typeof(ActivityBus))]
public class BaseBusType
{
[DataMember]
public int Id { get; set; }
}
[DataContract]
public class ActivityBus : BaseBusType { }
I've tried these options so far:
// In the sending function
var brokeredMessage = new BrokeredMessage(reader.ReadToEnd());
topicClient.Send(obj);
// In the receiving function
List<Type> knownTypes = new List<Type>() { typeof(ActivityBus) };
var serializer = new DataContractSerializer(typeof(BaseBusType), knownTypes); // listed knowntypes for brevity
var messageBody = message.GetBody<BaseBusType>(serializer);
Also:
// In the sending function
using (MemoryStream memoryStream = new MemoryStream())
using (StreamReader reader = new StreamReader(memoryStream))
{
DataContractSerializer serializer = new DataContractSerializer(typeof(T));
serializer.WriteObject(memoryStream, data);
memoryStream.Position = 0;
var brokeredMessage = new BrokeredMessage(reader.ReadToEnd());
topicClient.Send(brokeredMessage);
}
// In the receiving function
using (Stream stream = new MemoryStream())
{
var str = brokeredMessage.GetBody<string>();
byte[] data = System.Text.Encoding.UTF8.GetBytes(str);
stream.Write(data, 0, data.Length);
stream.Position = 0;
List<Type> knownTypes = new List<Type>() { typeof(Activitybus) };
var deserializer = new DataContractSerializer(typeof(BaseBusType), knownTypes);
var reader = XmlReader.Create(new StringReader(str));
var obj = deserializer.ReadObject(reader);
return (T)obj;
}
I'm trying to deserialise a JSON file from local disk into an object. I've got the following, but this only seems to work when downloading it from the web:
var client = new HttpClient();
using (var s = await client.GetStreamAsync(filePath))
using (var sr = new StreamReader(s))
using (var jr = new JsonTextReader(sr))
{
var js = new JsonSerializer();
return js.Deserialize<MyObject>(jr);
}
I'm trying to find a way to do this, without first reading it into a string.
From Here
you can deserialize an object from file in two way.
Solution-1: Read file into a string and deserialize JSON to a type
string json = File.ReadAllText(#"c:\myObj.json");
MyObject myObj = JsonConvert.DeserializeObject<MyObject>(json);
Solution-2: Deserialize JSON directly from a file
using (StreamReader file = File.OpenText(#"c:\myObj.json"))
{
JsonSerializer serializer = new JsonSerializer();
MyObject myObj2 = (MyObject)serializer.Deserialize(file, typeof(MyObject));
}
You can download Newtonsoft.Json from NuGet by following command
Install-Package Newtonsoft.Json
using (var s = new StreamReader(filePath))
{
using (var jr = new JsonTextReader(s))
{
var js = new JsonSerializer();
var obj = js.Deserialize<MyObject>(jr);
return obj;
}
}
You might want to look at this : https://msdn.microsoft.com/en-us/library/bb412179(v=vs.110).aspx
It's an MSDN article called "How to: Serialize and Deserialize JSON Data"
how can I read the JSON from this site (http://www.pegelonline.wsv.de/webservices/rest-api/v2/stations.json?includeTimeseries=true&includeCurrentMeasurement=true) in C# with the Json.net library?
In this JSon there is only a array. I tested it with this code, but it doesn't work.
using (Stream stream = response.GetResponseStream())
{
JsonReader reader = new JsonTextReader(new StreamReader(stream));
dynamic info = JObject.Load(reader);
}
If i debug this, then VS says that the item is not an object. I try it with JArray.Load(reader); but then I don't know how to access the items.
You are right, JArray.Load is working correctly. The JArray then has a simple indexer to get to the individual items:
using (Stream stream = response.GetResponseStream())
{
var reader = new JsonTextReader(new StreamReader(stream));
var jsonArray = JArray.Load(reader);
var item20 = jsonArray[19];
var item20ShortName = (string)item20["shortname"];
}
I have a generic class as follows:
class myClass<T>
{
public T[] m_SomeData;
}
I want to implement a generic method to read data from a file and populate the data fields of this class. Something like:
class myClass<T>
{
public T[] m_SomeData;
public void ReadData(string fileName);
}
An implementation of the ReadData methods looks something like this (all error checking removed for brevity):
void ReadData(string fileName)
{
TextReader rdr = new StreamReader(fileName);
string line = rdr.ReadLine();
// Here I need to parse value of type T from the line
// and initialize the m_SomeData array
// How do I do that? I would like to keep it generic if possible
}
Note, I can guarantee the type T is numeric, at least by convention
Update: OP would like human readable output. I would suggest JavaScriptSerializer, then, in:
C:\Windows\Microsoft.NET\Framework\v4.0.30319\System.Web.Extensions.dll
// Serialize:
using (var fs = new FileStream(fileName, FileMode.Create))
using (var writer = new StreamWriter(fs))
{
JavaScriptSerializer serializer = new JavaScriptSerializer();
string s = serializer.Serialize(m_SomeData);
writer.Write(s);
}
// Deserialize:
using (var fs = new FileStream(fileName, FileMode.Open))
using (var reader = new StreamReader(fs))
{
JavaScriptSerializer serializer = new JavaScriptSerializer();
var s = reader.ReadToEnd();
m_SomeData = serializer.Deserialize<T[]>(s);
}
Old Answer:
This is a job for BinaryFormatter:
using (FileStream fs = new FileStream(fileName, FileMode.Open))
{
BinaryFormatter formatter = new BinaryFormatter();
m_SomeData = (T[])formatter.Deserialize(fs);
}
This of course assumes you are also using it to serialize via formatter.Serialize(fs, m_SomeData);.