I have the following object:
[Serializable]
public class ExampleImage
{
public int ID { get; set; }
public string Filename { get; set; }
public byte[] Content { get; set; }
}
I store this in a List<ExampleImage> which I then pass to the following function to serialize it to a string:
static string SerializeObjectToXmlString(object o)
{
System.Xml.Serialization.XmlSerializer serializer = new System.Xml.Serialization.XmlSerializer(o.GetType());
System.IO.StringWriter writer = new System.IO.StringWriter();
serializer.Serialize(writer, o);
return writer.ToString();
}
I then pass this serialized string to a stored procedure in SQL2000 as an NTEXT which then handled for inserting it into the database:
SELECT * INTO #TempImages
FROM OpenXML(#iDoc, '/ArrayOfExampleImage/ExampleImage')
WITH ([Filename] VARCHAR(255) './Filename', [Content] IMAGE './Content')
The problem I am having is the image is getting trashed. The btye[] is not getting saved properly to the DB. The other fields are just fine. This is the first time I have attempt to send a binary data via XML to SQL so I am most likely doing something wrong at this point. Is my SerializeObjectToXmlString function the problem and it is not handling the serialization of a byte[] properly, maybe the OpenXML SQL function or even the fact that I am sending the XML in as an NTEXT param. I would expect the serialize function to encode the binary properly but I could be wrong.
Any idea what is the issue or maybe a better approach to saving a bunch of images at once?
Edit: I think what is happening, is the serializer is making the byte[] into a base64 string, which is then getting passed along to the stored proc as base64. I am then saving this base64 string into an Image field in SQL and reading it out as a btye[]. So I think I need to somehow get it from base64 to a byte[] before inserting it in my table?
Edit: I am starting to think my only option is to change the stored proc to just do 1 image at a time and not use XML and just pass in the byte[] as an Image type and wrap all the calls in a transaction.
As Gaidin suggested, base64 is the best option. It's the usual way of writing binary data to XML. You can use the following code :
public class ExampleImage
{
public int ID { get; set; }
public string Filename { get; set; }
[XmlIgnore]
public byte[] Content { get; set; }
[XmlElement("Content")]
public string ContentBase64
{
get { return Convert.ToBase64String(Content); }
set { Content = Convert.FromBase64String(value); }
}
}
(by the way, the Serializable attribute has no meaning for XML serialization)
Instead of serializing it to XML, I would serialize it to a byte[] and store that in varbinary(MAX) type field in your DB.
You could try converting it to base64, then saving it to TEXT field or something.
Related
I asked a question a couple of days ago to collect data from MongoDB as a tree.
MongoDB create an array within an array
I am a newbie to MongoDB, but have used JSON quite substantially. I thought using a MongoDB to store my JSON would be a great benefit, but I am just experiencing immense frustration.
I am using .NET 4.5.2
I have tried a number of ways to return the output from my aggregate query to my page.
public JsonResult GetFolders()
{
IMongoCollection<BsonDocument> collection = database.GetCollection<BsonDocument>("DataStore");
PipelineDefinition<BsonDocument, BsonDocument> treeDocs = new BsonDocument[]
{
// my query, which is a series of new BsonDocument
}
var documentGroup = collection.Aggregate(treeDocs).ToList();
// Here, I have tried to add it to a JsonResult Data,
// as both documentGroup alone and documentGroup.ToJson()
// Also, loop through and add it to a List and return as a JsonResult
// Also, attempted to serialise, and even change the JsonWriterSettings.
}
When I look in the Immediate Window at documentGroup, it looks exactly like Json, but when I send to browser, it is an escaped string, with \" surrounding all my keys and values.
I have attempted to create a model...
public class FolderTree
{
public string id { get; set; }
public string text { get; set; }
public List<FolderTree> children { get; set; }
}
then loop through the documentGroup
foreach(var docItem in documentGroup)
{
myDocs.Add(BsonSerializer.Deserialize<FolderTree>(docItem));
}
but Bson complains that it cannot convert int to string. (I have to have text and id as a string, as some of the items are strings)
How do I get my MongoDB data output as Json, and delivered to my browser as Json?
Thanks for your assistance.
========= EDIT ===========
I have attempted to follow this answer as suggested by Yong Shun below, https://stackoverflow.com/a/43220477/4541217 but this failed.
I had issues, that the "id" was not all the way through the tree, so I changed the folder tree to be...
public class FolderTree
{
//[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
//public string id { get; set; }
[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
public string text { get; set; }
public List<FolderTreeChildren> children { get; set; }
}
public class FolderTreeChildren
{
[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
public string text { get; set; }
public List<FolderTreeChildren> children { get; set; }
}
Now, when I look at documentGroup, I see...
[0]: {Plugins.Models.FolderTree}
[1]: {Plugins.Models.FolderTree}
To be fair to sbc in the comments, I have made so many changes to get this to work, that I can't remember the code I had that generated it.
Because I could not send direct, my json result was handled as...
JsonResult json = new JsonResult();
json.Data = documentGroup;
//json.Data = JsonConvert.SerializeObject(documentGroup);
json.JsonRequestBehavior = JsonRequestBehavior.AllowGet;
return json;
Note, that I also tried to send it as...
json.Data = documentGroup.ToJson();
json.Data = documentGroup.ToList();
json.Data = documentGroup.ToString();
all with varying failures.
If I leave as documentGroup, I get {Current: null, WasFirstBatchEmpty: false, PostBatchResumeToken: null}
If I do .ToJson(), I get "{ \"_t\" : \"AsyncCursor`1\" }"
If I do .ToList(), I get what looks like Json in json.Data, but get an error of Unable to cast object of type 'MongoDB.Bson.BsonInt32' to type 'MongoDB.Bson.BsonBoolean'.
If I do .ToString(), I get "MongoDB.Driver.Core.Operations.AsyncCursor`1[MongoDB.Bson.BsonDocument]"
=========== EDIT 2 =================
As this way of extracting the data from MongoDB doesn't want to work, how else can I make it work?
I am using C# MVC4. (.NET 4.5.2)
I need to deliver json to the browser, hence why I am using a JsonResult return type.
I need to use an aggregate to collect from MongoDB in the format I need it.
My Newtonsoft.Json version is 11.0.2
My MongoDB.Driver is version 2.11.1
My method is the simplest it can be.
What am I missing?
I am getting JSON of Form Data from users. Below is the example of JSON Data
{
"Name":"Mike",
"Age":25,
"Gender":"Male",
"Skills":{
".Net":true,
"Mule":""
}
}
I want to save this data in a table (SQL Server). There is no table in the database, I want to define table name before sending this data to sql. Is there any approach to achieve this. Please help.
I suggest using json2csharp to convert the JSON to C# models and alter the names which are not recognized
public class Skills
{
[JsonProperty(PropertyName = ".Net")] //Add JsonProperty to include unclassified names
public bool DotNet { get; set; }
public string Mule { get; set; }
}
public class RootObject10
{
public string Name { get; set; }
public int Age { get; set; }
public string Gender { get; set; }
public Skills Skills { get; set; }
}
then, you can deserialize the json using JsonConvert
using (StreamReader r = new StreamReader(filepath))
{
string json = r.ReadToEnd();
var obj = JsonConvert.DeserializeObject<RootObject10>(json);
}
After this, based on your requirement create single data table or 2 data tables and inject the data
Assuming your question is not about how to translate json into c# object but rather about how to store it in SQL Server while still being able to query it, you can actually work with json data in sql server: https://learn.microsoft.com/en-us/sql/relational-databases/json/json-data-sql-server?view=sql-server-ver15.
I would opt for something like that if I didn’t have schema upfront or I knew it will change often. Then I would create a table called FormData with an Id and Data fields and just stored your JSON in there.
Bear in mind this is likely less performant than defining tables and properly parsing json (which is covered by other answers here) - make sure you make the call after you’ve considered all pros and cons of schema-less storage.
Upd: if you absolutely must create tables at runtime you could potentially run plain SQL DDL statements like ´CREATE TABLE´ using plain ADO.NET
I have an application that receives IoT data. I would like to change the layout (that displays the data) depending on the configuration set by the user.
Eg: The user decides that 3 bytes will be device_id, 4th byte when multiplied by a value gives temperature value,etc. How can I create such a user config file and save it for later use ?
After saving the data, how can I display the data based on these config files? I am thinking of using labels to just match the data. Is there a better way to do this ?
So I have done as #Nobody suggested.
I have created a class with details like number of bytes, device id, etc and then took the data from user input via a form. Later used Basic Serialization to save the data and deserialization to read it back the next time I open the Application as per this link.
Code :
[Serializable()]
public class Config
{
public string DeviceId { get; set; }
public string Name { get; set; }
public int Length { get; set; }
}
using (Stream testFileStream = File.Create(pathString)) // Serialization code
{
BinaryFormatter serializer = new BinaryFormatter();
serializer.Serialize(testFileStream, config);
testFileStream.Close();
}
using (Stream testFileStream = File.OpenRead(pathString))
{
BinaryFormatter deserializer = new BinaryFormatter();
config = (Config)deserializer.Deserialize(testFileStream);
testFileStream.Close();
}
I've created a successful connection to ES, and then written my json query. Now, I would like to send that query via the Serialize method.
The Serialize method requires two parameters:
1. object and
2. Stream writableStream
My question is, with the second one. When I create a stream with the following code line:
Stream wstream;
And use it to initialize my json2 variable with the following code:
var json2 = highLevelclient.Serializer.Serialize(query, wstream).Utf8String();
I get the following error on the wstream variable:
Use of unassigned local variable 'wstream'.
Am I missing something? Is it the way I create the wstream variable that is wrong? thank you.
/* \\\ edit: ///// */
There is another issue now, I use Searchblox to index and search my files, which itself calls ES 2.x to do the job. Searchblox uses a "mapping.json" file to initialize a mapping upon the creation of an index. Here's the link to that file.
As "#Russ Cam" suggested, I created my own class content with the following code (just like he did with the "questions" index and "Question" class):
public class Content
{
public string type { get; set; }
public Fields fields { get; set; }
}
public class Fields
{
public Content1 content { get; set; }
public Autocomplete autocomplete { get; set; }
}
public class Content1
{
public string type { get; set; }
public string store { get; set; }
public string index { get; set; }
public string analyzer { get; set; }
public string include_in_all { get; set; }
public string boost { get; set; }
} //got this with paste special->json class
These fields from the content class (type,store etc.) come from the mapping.json file attached above.
Now, when I (just like you showed me) execute the following code:
var searchResponse = highLevelclient.Search<Content>(s => s.Query(q => q
.Match(m => m.Field(f => f.fields.content)
.Query("service")
All I get as a response on the searchResponse variable is:
Valid NEST response built from a successful low level call on POST: /idx014/content/_search
Audit trail of this API call:
-HealthyResponse: Node: http://localhost:9200/ Took: 00:00:00.7180404
Request:
{"query":{"match":{"fields.content":{"query":"service"}}}}
Response:
{"took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}
And no documents in searchResponse.Documents.
Contradictorily, when I search for the "service" query on Searchblox or make an API call to localhost:9200 with the Sense extension of Google Chrome, I get 2 documents. (the documents that I was looking for)
In brief, all I want is to be able to :
get all the documents (no criteria)
get all the documents within a time range and based upon keywords.. such as "service"
What am I doing wrong? I can provide with more information if needed..
Thank you all for your detailed answers.
It's actually much simpler than this with NEST :) The client will serialize your requests for you and send them to Elasticsearch, you don't need to take the step to serialize them yourself and then pass them to the client to send to Elasticsearch.
Take a search request for example. Given the following POCO
public class Question
{
public string Body { get; set; }
}
We can search for questions that contain the phrase "this should never happen" in the body with
var settings = new ConnectionSettings(new Uri("http://localhost:9200"))
.InferMappingFor<Question>(m => m
.IndexName("questions")
);
var client = new ElasticClient(settings);
var searchResponse = client.Search<Question>(s => s
.Query(q => q
.MatchPhrase(m => m
.Field(f => f.Body)
.Query("this should never happen")
)
)
);
// do something with the response
foreach (var question in searchResponse.Documents)
{
Console.WriteLine(question.Body);
}
this line
My question is, with the second one. When I create a stream with the following code line:
Stream wstream;
does not create na object. It nearly declares it. You need to new-ed it.
Stream wstream = new MemoryStream(); //doesn't have to be MemoryStream here - check what does Serialize expects
Just remember to close it later or use using statement.
using(Stream stream = new MemoryStream())
{
//do operations on wstream
} //closes automatically here
You just declared wstream but never assigned an actual stream to it. Depending on how Serialize method works it could be:
you need to create a stream and pass it to the Serialize method
or you need to pass stream parameter with out prefix
I am working on a REST API for a project using Visual Studio 2013 with C# and ASP.NET, and I need some guidance.
When the webpage performs a POST, I am passing along a number of fields as a JSON object. By defining a data transfer object in my C# code, I can easily read the values from the JSON, but only if I define all the fields (with the same name).
Here is my current (working) code:
public class AgencyPostDTO
{
public string AgencyName { get; set; }
public string Address1 { get; set; }
public string Address2 { get; set; }
public string City { get; set; }
public string State { get; set; }
public string ZIP { get; set; }
}
// POST: api/Agency
public string Post(AgencyPostDTO Agency)
{
int success;
success = SQLUpdateAgency(Agency);
if (success < 1)
{
return "Failed";
}
else
{
return "Success";
}
}
So far no problems. I need to pass the data over to a second function, where I will perform some data processing (including converting the data into XML) and send the data/XML to MS SQL using a stored procedure:
public int SQLUpdateAgency(AgencyPostDTO Agency)
{
string xml = Agency.SerializeObject();
... code to call SQL stored procedure ommitted here
}
Now to my problem. I would prefer if I did not have to define the parameters of the data transfer object AgencyPostDTO in the code, and instead the code would just read the incoming JSON and pass it along to the next function, where I create the XML containing everything passed along.
As it works now, if the JSON contains for example an email address field, it will be dropped unless I define it in AgencyPostDTO.
So why do I want to do this? For future ease of maintenance. The users may come and say they want to add additional fields to the web form. I can then simply have our SQL expert add that column to the table, give me the name of it and I add an input field to the HTML form and make sure it is included in the JSON sent over. That way we never have to touch the already written, tested and working code. The new field is simply passed though the whole process.
Can this be done? If so, any suggestions on how?
If you used JSON.NET to handle the deserialisation of your objects then that has support for dynamic properties. Once you'd read your JSON string, you could convert it to a JArray or JObject and from there by using the .Children() call to get a list of all properties to convert it to any XML object you needed.
Have a look here:
Deserialize json object into dynamic object using Json.net