I am working on a project where I am trying to save certain information about the current user state to a file for later deserialization when the user decides to reopen the application or change to another saved session. The session information contains information about what image files the user was working with, attributes the user has edited on those items, and other information about things the user had done in the interface - all of this in custom classes I made.
I wrote some methods to serialize/deserialize my root "UserData" class that contains all of the other class objects into a file "user.json", which works okay, however, I noticed that certain types were not being serialized.
For example, within my UserData class, I have a List of "Sessions", each of which contain further subclasses and variables. While strings/ints in my UserData class are serialized as expected, these custom types don't serialize. For example, my user.json shows each session object as "{}" instead of the full serialization of those objects (and any sub-objects).
My issue is similar to this one: C# serialize complex object of class to json, but not exactly. I read that one and I didn't understand any better how to properly serialize/deserialize this.
I hope I am not misunderstanding the purpose of serialization, but I would like to know how these sub-objects can be fully serialized/deserialized using methods in my UserData class (which handles the serialization/deserialization of itself) or if there is a better way to do what I am trying to do.
Here are some of the classes I am working with. I sanitized some sensitive information and redundant methods from the class, most of these are to just show you the data structures being serialized:
UserData class (includes serialize/deserialize methods)
public class UserData
{
public List<appSession> userSessions { get; set;}
public DatabaseConnection lastDatabaseConnection { get; set;}
public string temp { get; set; }
public UserData() { userSessions = new List<appSession>(); }
public async Task<StorageFile> Create()
{
StorageFolder appData = ApplicationData.Current.LocalFolder;
StorageFile udFile = null;
try {
udFile = await appData.CreateFileAsync(#"UserData\user.json");
var options = new JsonSerializerOptions { WriteIndented = true };
string udData = JsonSerializer.Serialize(new UserData(), options);
await Windows.Storage.FileIO.WriteTextAsync(udFile, udData);
return udFile; }
catch (Exception ex2) { return null; }
}
public async Task<UserData> Load()
{
StorageFolder appData = ApplicationData.Current.LocalFolder;
StorageFile udFile = null;
UserData cmData;
Helper help = new Helper();
// Try to either load or create the VIPER user data file.
try {
udFile = await appData.GetFileAsync(#"UserData\user.json"); }
catch (UnauthorizedAccessException) { }
catch (FileNotFoundException){
try { udFile = await Create(); } catch {} }
if (udFile == null) {
return null;
} else {
try {
string udresult = await help.ReadAllTextFromFile(udFile);
cmData = JsonSerializer.Deserialize<UserData>(udresult);
return cmData; }
catch (Exception ex) {
try {
await udFile.RenameAsync("badUserData." + System.DateTime.Now.ToString("MM-dd-yyyy hh.mm.ss tt") + ".jsonold");
udFile = await appData.CreateFileAsync(#"UserData\user.json");
var options = new JsonSerializerOptions { WriteIndented = true };
string udData = JsonSerializer.Serialize(new UserData(), options);
await Windows.Storage.FileIO.WriteTextAsync(udFile, udData);
return await Load(); }
catch (Exception ex2){
return null;}
}
}
}
public async void Save()
{
try {
StorageFile udFile = await ApplicationData.Current.LocalFolder.GetFileAsync(#"UserData\user.json");
var options = new JsonSerializerOptions { WriteIndented = true };
var udData = JsonSerializer.Serialize(this, options);
await Windows.Storage.FileIO.WriteTextAsync(udFile, udData);
} catch(Exception ex) { }
}
}
"user.json" file contents, after execution:
{
"userSessions": [
{},
{}
],
"lastDatabaseConnection": null,
"temp": "test test test"
}
appSession Class (structural object):
public class appSession
{
public SiteConnection LinkedSite;
internal ImageUploaderData IUSession;
internal DocumentUploaderData DUSession;
public appSession(SiteConnection linkedSite)
{
LinkedSite = new SiteConnection(linkedSite);
IUSession = new ImageUploaderData();
DUSession = new DocumentUploaderData();
}
}
ImageUploaderData class (structural object)
internal class ImageUploaderData
{
List<WorkingImage> workAreaImages;
public ImageUploaderData()
{
List<WorkingImage> workAreaImages = new List<WorkingImage>();
}
}
WorkingImage class (deepest structural object)
internal class WorkingImage : WorkingFile
{
public enum TaggingState
{
Untagged,
Issue,
Tagged
}
// EXIF data
string EXIF_filename;
string EXIF_filesize;
string EXIF_rotation;
string EXIF_createDate;
string EXIF_modifyDate;
List<string> associatedTags;
CustomAreaDefinition area;
bool selected;
TaggingState tagState;
public WorkingImage() : base()
{ selected = false; }
public WorkingImage(string path) : base(path)
{ selected = false; }
}
}
JsonSerializer needs public properties with get/set, not just fields, so fix all your classes you need to serialize, for example appSession class should be
public class appSession
{
public SiteConnection LinkedSite {get;set;}
public ImageUploaderData IUSession {get;set;}
public DocumentUploaderData DUSession {get;set;}
public appSession(SiteConnection linkedSite)
{
LinkedSite = new SiteConnection(linkedSite);
IUSession = new ImageUploaderData();
DUSession = new DocumentUploaderData();
}
}
Related
I have a Json file, it contains connectionstring. I want to asynchronously read the file and deserialize it to a ConnectionString object and I always get a null result. I'm using .NET Core 6 and System.Text.Json.
Here is contents of my Json file:
{
"ConnectionStrings": {
"ConnStr": "Data Source=(local);Initial Catalog=MyData;Integrated Security=False;TrustServerCertificate=True;Persist Security Info=False;Async=True;MultipleActiveResultSets=true;User ID=sa;Password=MySecret;",
"ProviderName": "SQLServer"
}
}
Here are the contents of my classes:
internal class DBConnectionString
{
[JsonPropertyName("ConnStr")]
public string ConnStr { get; set; }
[JsonPropertyName("ProviderName")]
public string ProviderName { get; set; }
public DBConnectionString()
{
}
}
public class DBConnStr {
private static string AppSettingFilePath => "appsettings.json";
public static async Task<string> GetConnectionStringAsync()
{
string connStr = "";
if (File.Exists((DBConnStr.AppSettingFilePath)))
{
using (FileStream sr = new FileStream(AppSettingFilePath, FileMode.Open, FileAccess.Read))
{
//string json = await sr.ReadToEndAsync();
System.Text.Json.JsonDocumentOptions docOpt = new System.Text.Json.JsonDocumentOptions() { AllowTrailingCommas = true };
using (var document = await System.Text.Json.JsonDocument.ParseAsync(sr, docOpt))
{
System.Text.Json.JsonSerializerOptions opt = new System.Text.Json.JsonSerializerOptions() { AllowTrailingCommas = true, PropertyNameCaseInsensitive = true };
System.Text.Json.JsonElement root = document.RootElement;
System.Text.Json.JsonElement element = root.GetProperty("ConnectionStrings");
sr.Position = 0;
var dbConStr = await System.Text.Json.JsonSerializer.DeserializeAsync<DBConnectionString>(sr, opt);
if (dbConStr != null)
{
connStr = dbConStr.ConnStr;
}
}
}
}
return connStr;
}
}
The following is the syntax that I use to call the GetConnectionStringAsync method:
string ConnectionString = DBConnStr.GetConnectionStringAsync().Result;
When the application is running in debug mode, I checked, on line
var dbConStr = await
System.Text.Json.JsonSerializer.DeserializeAsync(sr,
opt);
The DBConnectionString object property is always empty.
I also tried the reference on the Microsoft website, https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/how-to?pivots=dotnet-6-0 but it doesn't work succeed.
using System.Text.Json;
namespace DeserializeFromFileAsync
{
public class WeatherForecast
{
public DateTimeOffset Date { get; set; }
public int TemperatureCelsius { get; set; }
public string? Summary { get; set; }
}
public class Program
{
public static async Task Main()
{
string fileName = "WeatherForecast.json";
using FileStream openStream = File.OpenRead(fileName);
WeatherForecast? weatherForecast =
await JsonSerializer.DeserializeAsync<WeatherForecast>(openStream);
Console.WriteLine($"Date: {weatherForecast?.Date}");
Console.WriteLine($"TemperatureCelsius: {weatherForecast?.TemperatureCelsius}");
Console.WriteLine($"Summary: {weatherForecast?.Summary}");
}
}
}
Do you have a solution for my problem or a better solution? I appreciate all your help. Thanks
Sorry about my English if it's not good, because I'm not fluent in English and use google translate to translate it
To begin with, if you want to read information from appSettings.json, you should explore more into reading configurations. There are helper classes provided by .Net for the same.
Coming back to your code, if you want to use your own code for Json Deserialization, then you need to make the following change to it.
var dbConStr = System.Text.Json.JsonSerializer.Deserialize<DBConnectionString>(element.GetRawText(), opt);
where, element according to code shared in the question is defined as
System.Text.Json.JsonElement element = root.GetProperty("ConnectionStrings");
This ensures the Raw Json associated with the JsonElement ConnectStrings is de-serialized.
However, I recommend you to read more into Reading configurations using the IConfiguration and related .Net helpers.
We have a console application using the Azure WebJob SDK. The WebJob relies on a WCF service using SOAP, which it accesses through a DLL we wrote that wraps the auto-generated WCF types in something a bit more friendly.
For logging purposes, we want to save the request and response XML bodies for requests that we make. These XML bodies would be saved in our database. But, because the WCF code lives in a low-level DLL, it has no concept of our database and can't save to it.
The DLL uses Microsoft's DI extensions to register types, and the WebJob calls into it like this:
class WebJobClass
{
IWCFWrapperClient _wcfWrapperClient;
public WebJobClass(IWCFWrapperClient wcfWrapperClient)
{
_wcfWrapperClient = wcfWrapperClient;
}
public async Task DoThing()
{
var callResult = await _wcfWrapperClient.CallWCFService();
}
}
IWCFWrapperClient looks like this:
class WCFWrapperClient : IWCFWrapperClient
{
IWCF _wcf; // auto-generated by VS, stored in Reference.cs
public async Task<object> CallWCFService()
{
return await _wcf.Call(); // another auto-generated method
}
}
I've implemented an IClientMessageInspector, and it works fine to get me the XML request/response, but I don't have a way to pass it back up to WCFWrapperClient.CallWCFService so that it can be returned to WebJobClass.DoThing(), who could then save it to the database.
The problem is multithreading. WebJobs, IIRC, will run multiple requests in parallel, calling into the DLL from multiple threads. This means we can't, say, share a static property LastRequestXmlBody since multiple threads could overwrite it. We also can't, say, give each call a Guid or something since there's no way to pass anything from IWCFWrapperClient.CallWCFService into the auto-generated IWCF.Call except what was auto-generated.
So, how can I return XML to WebJobClass.DoThing in a thread-safe way?
I was able to find a solution that uses ConcurrentDictionary<TKey, TValue>, but it's a bit ugly.
First, I amended the auto-generated classes in Reference.cs with a new property Guid InternalCorrelationId. Since the auto-generated classes are partial, this can be done in separate files that aren't changed when the client is regenerated.
public partial class AutoGeneratedWCFType
{
private Guid InternalCorrelationIdField;
[System.Runtime.Serialization.DataMember()]
public Guid InternalCorrelationId
{
get { return InternalCorrelationIdField; }
set { InternalCorrelationIdField = value; }
}
}
Next, I made all my request DTO types derive from a type named RequestBase, and all my response DTO types derive from a typed named ResponseBase, so I could handle them generically:
public abstract class RequestBase
{
public Guid InternalCorrelationId { get; set; }
}
public abstract class ResponseBase
{
public string RequestXml { get; set; }
public string ResponseXml { get; set; }
}
I then added a type RequestCorrelator that simply holds on to a ConcurrentDictionary<Guid, XmlRequestResponse>:
public sealed class RequestCorrelator : IRequestCorrelator
{
public ConcurrentDictionary<Guid, XmlRequestResponse> PendingCalls { get; }
public RequestCorrelator() => PendingCalls = new ConcurrentDictionary<Guid, XmlRequestResponse>();
}
public sealed class XmlRequestResponse
{
public string RequestXml { get; set; }
public string ResponseXml { get; set; }
}
RequestCorrelator is its own type for DI purposes - you may just be able to use a ConcurrentDictionary<TKey, TValue> directly.
Finally, we have the code that actually grabs the XML, a type implementing IClientMessageInspector:
public sealed class ClientMessageProvider : IClientMessageInspector
{
private readonly IRequestCorrelator _requestCorrelator;
public ClientMessageProvider(IRequestCorrelator requestCorrelator) =>
_requestCorrelator = requestCorrelator;
public object BeforeSendRequest(ref Message request, IClientChannel channel)
{
var requestXml = request.ToString();
var internalCorrelationId = GetInternalCorrelationId(requestXml);
if (internalCorrelationId != null)
{
if (_requestCorrelator.PendingCalls.TryGetValue(internalCorrelationId.Value,
out var requestResponse))
{
requestResponse.RequestXml = requestXml;
}
request = RemoveInternalCorrelationId(request);
}
return internalCorrelationId;
}
public void AfterReceiveReply(ref Message reply, object correlationState)
{
// WCF can internally correlate a request between BeforeSendRequest and
// AfterReceiveReply. We reuse the same correlation ID we added to the
// XML as our correlation state.
var responseXml = reply.ToString();
var internalCorrelationId = (correlationState is Guid guid)
? guid
: default;
if (_requestCorrelator.PendingCalls.TryGetValue(internalCorrelationId,
out var requestResponse))
{
requestResponse.ResponseXml = responseXml;
}
}
private static Guid? GetInternalCorrelationId(string requestXml)
{
var document = XDocument.Parse(requestXml);
var internalCorrelationIdElement = /* You'll have to write this yourself;
every WCF XML request is different. */
return internalCorrelationIdElement != null
? Guid.Parse(internalCorrelationIdElement.Value)
: null;
}
private static Message RemoveInternalCorrelationId(Message oldMessage)
{
// https://stackoverflow.com/a/35639900/2709212
var buffer = oldMessage.CreateBufferedCopy(2 * 1024 * 1024);
var tempMessage = buffer.CreateMessage();
var dictionaryReader = tempMessage.GetReaderAtBodyContents();
var document = new XmlDocument();
document.Load(dictionaryReader);
dictionaryReader.Close();
var internalCorrelationIdNode = /* You'll also have to write this yourself. */
var parent = internalCorrelationIdNode.ParentNode;
parent.RemoveChild(internalCorrelationIdNode);
var memoryStream = new MemoryStream();
var xmlWriter = XmlWriter.Create(memoryStream);
document.Save(xmlWriter);
xmlWriter.Flush();
xmlWriter.Close();
memoryStream.Position = 0;
var xmlReader = XmlReader.Create(memoryStream);
var newMessage = Message.CreateMessage(oldMessage.Version, null, xmlReader);
newMessage.Headers.CopyHeadersFrom(oldMessage);
newMessage.Properties.CopyProperties(oldMessage.Properties);
return newMessage;
}
}
In short, this type:
Finds the correlation ID in the XML request.
Finds the XmlRequestResponse with the same correlation ID and adds the request to it.
Removes the correlation ID element so that the service doesn't get elements they didn't expect.
After receiving a reply, uses correlationState to find the XmlRequestResponse and write the response XML to it.
Now all we have to do is change IWCFWrapperClient:
private async Task<TDtoResult> ExecuteCallWithLogging<TDtoRequest,
TWcfRequest,
TWcfResponse,
TDtoResult>(TDtoRequest request,
Func<TDtoRequest, TWcfRequest> dtoToWcfConverter,
Func<TWcfRequest, Task<TWcfResponse>> wcfCall,
Func<TWcfResponse, TDtoResult> wcfToDtoConverter)
where TDtoRequest : CorrelationBase
where TDtoResult : WcfBase
{
request.InternalCorrelationId = Guid.NewGuid();
var xmlRequestResponse = new XmlRequestResponse();
_requestCorrelator.PendingCalls.GetOrAdd(request.InternalCorrelationId,
xmlRequestResponse);
var response = await contractingCall(dtoToWcfConverter(request));
_requestCorrelator.PendingCalls.TryRemove(request.InternalCorrelationId, out _);
return wcfToDtoConverter(response).WithRequestResponse(xmlRequestResponse);
}
public async Task<DoThingResponseDto> DoThing(DoThingRequestDto request) =>
await ExecuteCallWithLogging(request,
r => r.ToWcfModel(),
async d => await _wcf.Call(d),
d => d.ToDtoModel());
WithRequestResponse is implemented as follows:
public static T WithRequestResponse<T>(this T item, XmlRequestResponse requestResponse)
where T : ResponseBase
{
item.RequestXml = requestResponse?.RequestXml;
item.ResponseXml = requestResponse?.ResponseXml;
return item;
}
And there we go. WCF calls that return their XML in the response object rather than just something you can print to console or log to a file.
This question already has answers here:
My console app shutdown prematurely when using async / await?
(4 answers)
Program exits upon calling await
(3 answers)
Closed 2 years ago.
Before you all go on a rampage about how this is a duplicate question, I have spent two days working on this issue, watching youtube tutorials on asynchronous programming, surfing similar stackoverflow posts etc, and I cannot for the life of me figure out how to apply Asynchronous Parallel Downloading of files into my project.
First things first, some background:
I am creating a program that, when given a query input via the user, will make a call to the twitch API and download clips.
My program is two parts
1- A web scraper that generates a .json file with all details needed to download files and
2 - A downloader.
Part 1 works perfectly fine and generates the .json files no trouble.
My Downloader contains reference to a Data class that is a handler for common properties and methods like my ClientID, Authentication, OutputPath, JsonFile, QueryURL. It also contains methods to give values to these properties.
Here are the two methods of my FileDownloader.cs that are the problem:
public async static void DownloadAllFiles(Data clientData)
{
data = clientData;
data.OutputFolderExists();
// Deserialize .json file and get ClipInfo list
List<ClipInfo> clips = JsonConvert.DeserializeObject<List<ClipInfo>>(File.ReadAllText(data.JsonFile));
tasks = new List<Task>();
foreach(ClipInfo clip in clips)
{
tasks.Add(DownloadFilesAsync(clip));
}
await Task.WhenAll(tasks);
}
private async static Task DownloadFilesAsync(ClipInfo clip)
{
WebClient client = new WebClient();
string url = GetClipURL(clip);
string filepath = data.OutputPath + clip.id + ".mp4";
await client.DownloadFileTaskAsync(new Uri(url), filepath);
}
This is only one of my many attempts of downloading files, one which I got the idea from this post:
stackoverflow_link
I have also tried methods like the following from a YouTube video by IAmTimCorey:
video_link
I have spent many an hour tackling this problem, and I honestly can't figure out why it won't work with any of my attempts. I would vastly appreciate your help.
Thanks,
Ben
Below is the entirety of my code, should anyone need it for any reason.
Code Structure:
The only external libraries I have downloaded is Newtonsoft.Json
ClipInfo.cs
using System;
using System.Collections.Generic;
using System.Text;
namespace Downloader
{
public class ClipInfo
{
public string id { get; set; }
public string url { get; set; }
public string embed_url { get; set; }
public string broadcaster_id { get; set; }
public string broadcaster_name { get; set; }
public string creator_id { get; set; }
public string creator_name { get; set; }
public string video_id { get; set; }
public string game_id { get; set; }
public string language { get; set; }
public string title { get; set; }
public int view_count { get; set; }
public DateTime created_at { get; set; }
public string thumbnail_url { get; set; }
}
}
Pagination.cs
namespace Downloader
{
public class Pagination
{
public string cursor { get; set; }
}
}
Root.cs
using System.Collections.Generic;
namespace Downloader
{
public class Root
{
public List<ClipInfo> data { get; set; }
public Pagination pagination { get; set; }
}
}
Data.cs
using System;
using System.IO;
namespace Downloader
{
public class Data
{
private static string directory = Directory.GetCurrentDirectory();
private readonly static string defaultJsonFile = directory + #"\clips.json";
private readonly static string defaultOutputPath = directory + #"\Clips\";
private readonly static string clipsLink = "https://api.twitch.tv/helix/clips?";
public string OutputPath { get; set; }
public string JsonFile { get; set; }
public string ClientID { get; private set; }
public string Authentication { get; private set; }
public string QueryURL { get; private set; }
public Data()
{
OutputPath = defaultOutputPath;
JsonFile = defaultJsonFile;
}
public Data(string clientID, string authentication)
{
ClientID = clientID;
Authentication = authentication;
OutputPath = defaultOutputPath;
JsonFile = defaultJsonFile;
}
public Data(string clientID, string authentication, string outputPath)
{
ClientID = clientID;
Authentication = authentication;
OutputPath = directory + #"\" + outputPath + #"\";
JsonFile = OutputPath + outputPath + ".json";
}
public void GetQuery()
{
Console.Write("Please enter your query: ");
QueryURL = clipsLink + Console.ReadLine();
}
public void GetClientID()
{
Console.WriteLine("Enter your client ID");
ClientID = Console.ReadLine();
}
public void GetAuthentication()
{
Console.WriteLine("Enter your Authentication");
Authentication = Console.ReadLine();
}
public void OutputFolderExists()
{
if (!Directory.Exists(OutputPath))
{
Directory.CreateDirectory(OutputPath);
}
}
}
}
JsonGenerator.cs
using System;
using System.IO;
using System.Net.Http.Headers;
using System.Net.Http;
using System.Threading.Tasks;
using Newtonsoft.Json;
using System.Linq;
namespace Downloader
{
public static class JsonGenerator
{
// This class has no constructor.
// You call the Generate methods, passing in all required data.
// The file will then be generated.
private static Data data;
public static async Task Generate(Data clientData)
{
data = clientData;
string responseContent = null;
// Loop that runs until the api request goes through
bool authError = true;
while (authError)
{
authError = false;
try
{
responseContent = await GetHttpResponse();
}
catch (HttpRequestException)
{
Console.WriteLine("Invalid authentication, please enter client-ID and authentication again!");
data.GetClientID();
data.GetAuthentication();
authError = true;
}
catch (Exception e)
{
Console.WriteLine(e.Message);
authError = true;
}
}
data.OutputFolderExists();
GenerateJson(responseContent);
}
// Returns the contents of the resopnse to the api call as a string
private static async Task<string> GetHttpResponse()
{
// Creating client
HttpClient client = new HttpClient();
if (data.QueryURL == null)
{
data.GetQuery();
}
// Setting up request
HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Get, data.QueryURL);
// Adding Headers to request
requestMessage.Headers.Add("client-id", data.ClientID);
requestMessage.Headers.Authorization = new AuthenticationHeaderValue("Bearer", data.Authentication);
// Receiving response to the request
HttpResponseMessage responseMessage = await client.SendAsync(requestMessage);
// Gets the content of the response as a string
string responseContent = await responseMessage.Content.ReadAsStringAsync();
return responseContent;
}
// Generates or adds to the .json file that contains data on each clip
private static void GenerateJson(string responseContent)
{
// Parses the data from the response to the api request
Root responseResult = JsonConvert.DeserializeObject<Root>(responseContent);
// If the file doesn't exist, we need to create it and add a '[' at the start
if (!File.Exists(data.JsonFile))
{
FileStream file = File.Create(data.JsonFile);
file.Close();
// The array of json objects needs to be wrapped inside []
File.AppendAllText(data.JsonFile, "[\n");
}
else
{
// For a pre-existing .json file, The last object won't have a comma at the
// end of it so we need to add it now, before we add more objects
string[] jsonLines = File.ReadAllLines(data.JsonFile);
File.WriteAllLines(data.JsonFile, jsonLines.Take(jsonLines.Length - 1).ToArray());
File.AppendAllText(data.JsonFile, ",");
}
// If the file already exists, but there was no [ at the start for whatever reason,
// we need to add it
if (File.ReadAllText(data.JsonFile).Length == 0 || File.ReadAllText(data.JsonFile)[0] != '[')
{
File.WriteAllText(data.JsonFile, "[\n" + File.ReadAllText(data.JsonFile));
}
string json;
// Loops through each ClipInfo object that the api returned
for (int i = 0; i < responseResult.data.Count; i++)
{
// Serializes the ClipInfo object into a json style string
json = JsonConvert.SerializeObject(responseResult.data[i]);
// Adds the serialized contents of ClipInfo to the .json file
File.AppendAllText(data.JsonFile, json);
if (i != responseResult.data.Count - 1)
{
// All objects except the last require a comma at the end of the
// object in order to correctly format the array of json objects
File.AppendAllText(data.JsonFile, ",");
}
// Adds new line after object entry
File.AppendAllText(data.JsonFile, "\n");
}
// Adds the ] at the end of the file to close off the json objects array
File.AppendAllText(data.JsonFile, "]");
}
}
}
FileDownloader.cs
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Threading.Tasks;
namespace Downloader
{
public class FileDownloader
{
private static Data data;
private static List<Task> tasks;
public async static void DownloadAllFiles(Data clientData)
{
data = clientData;
data.OutputFolderExists();
// Deserialize .json file and get ClipInfo list
List<ClipInfo> clips = JsonConvert.DeserializeObject<List<ClipInfo>>(File.ReadAllText(data.JsonFile));
tasks = new List<Task>();
foreach (ClipInfo clip in clips)
{
tasks.Add(DownloadFilesAsync(clip));
}
await Task.WhenAll(tasks);
}
private static void GetData()
{
if (data.ClientID == null)
{
data.GetClientID();
}
if (data.Authentication == null)
{
data.GetAuthentication();
}
if (data.QueryURL == null)
{
data.GetQuery();
}
}
private static string GetClipURL(ClipInfo clip)
{
// Example thumbnail URL:
// https://clips-media-assets2.twitch.tv/AT-cm%7C902106752-preview-480x272.jpg
// You can get the URL of the location of clip.mp4
// by removing the -preview.... from the thumbnail url */
string url = clip.thumbnail_url;
url = url.Substring(0, url.IndexOf("-preview")) + ".mp4";
return url;
}
private async static Task DownloadFilesAsync(ClipInfo clip)
{
WebClient client = new WebClient();
string url = GetClipURL(clip);
string filepath = data.OutputPath + clip.id + ".mp4";
await client.DownloadFileTaskAsync(new Uri(url), filepath);
}
private static void FileDownloadComplete(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
{
tasks.Remove((Task)sender);
}
}
}
Program.cs
using System;
using System.Threading.Tasks;
using Downloader;
namespace ClipDownloader
{
class Program
{
private static string clientID = "{your_client_id}";
private static string authentication = "{your_authentication}";
async static Task Main(string[] args)
{
Console.WriteLine("Enter your output path");
string outputPath = Console.ReadLine();
Data data = new Data(clientID, authentication, outputPath);
Console.WriteLine(data.OutputPath);
//await JsonGenerator.Generate(data);
FileDownloader.DownloadAllFiles(data);
}
}
}
The example query I usually type in is "game_id=510218"
async void is your problem
Change
public static async void DownloadAllFiles(Data clientData)
To
public static async Task DownloadAllFiles(Data clientData)
Then you can await it
await FileDownloader.DownloadAllFiles(data);
The longer story:
async void runs unobserved (fire and forget). You can't wait for them to finish. In essence as soon as your program starts the task, it finishes, and tears down the App Domain and all your sub tasks, leading you to believe nothing is working.
I'm trying to stay on topic here as best as I can, but when using JsonConvert.DeserializeObject{T}, isn't T suppose to be an encapsulating root object type? I have never used it the way you're using it, so I'm just curious if that might be your bug. I could be completely wrong, and spare me if i am, but JSON is key:value based. Deserializing directly to a List doesn't really make sense. Unless there is a special case in the deserializer? List would be a file that's purely an array of ClipInfo values being deserialized into the members of List{T}(private T[] _items, private int _size, etc.) It needs a parent root object.
// current JSON file format implication(which i dont think is valid JSON?(correct me please)
clips:
[
// clip 1
{ "id": "", "url": "" },
// clip N
{ "id": "", "url": "" },
]
// correct(?) JSON file format
{ // { } is the outer encasing object
clips:
[
// clip 1
{ "id": "", "url": "" },
// clip N
{ "id": "", "url": "" },
]
}
class ClipInfoJSONFile
{
public List<ClipInfo> Info { get; set; }
}
var clipInfoList = JsonConverter.DeserializeObject<ClipInfoJSONFile>(...);
I'm trying to XML serialize a class containing a enum property. If the property is declared using a specific enum, it works just fine. But I need the property to be of type Enum, so I can set it to different enum types. However, when doing this I get an exception.
The type [namespace].Simple may not be used in this context.
I've tried different attributes on the enum definition, but haven't gotten it right so far. Is there a way to do this?
public enum Simple : byte
{
one = 0x01,
two = 0x02,
three = 0x03
}
public class Foo
{
public Enum Simple { get; set; }
}
public class Program
{
static void Main(string[] args)
{
using (var writer = XmlWriter.Create(Console.OpenStandardOutput()))
{
try
{
var foo = new Foo
{
Simple = Simple.three
};
var serializer = new XmlSerializer(foo.GetType());
serializer.Serialize(writer, foo);
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
Console.ReadLine();
}
}
You can try to set EnumMember attrubute on your DataContract that you want to serialize, for more specific info visit
https://msdn.microsoft.com/en-us/library/system.runtime.serialization.enummemberattribute(v=vs.110).aspx
Enum is abstract and cannot be serialized. A possible approach to solve is presented in this answer.
The common primitive base type of enum is int (by default, can also be byte or long for instance).
So you could as well simply use this integer base type instead, like byte Simple in your Foo class.
In case you need the string representation to appear in xml (identical to the enum field name), expose it as string Simple.
Based on dlatikay's idea about splitting the enum into two strings for enum type and member name, I've come up with the following solution. The example converts to from a Foo object to XML string, and back to a new Foo object again.
public enum SimpleEnum : byte
{
one = 0x01,
two = 0x02,
three = 0x03
}
public class Foo
{
private Enum _simple;
[XmlIgnore]
public Enum Simple
{
get { return _simple; }
set {
_simple = value;
var type = Simple.GetType();
var underlyingType = Enum.GetUnderlyingType(type);
EnumType = Simple.GetType().FullName;
EnumMember = Simple.ToString();
}
}
private string _enumType;
public string EnumType
{
get { return _enumType; }
set { _enumType = value; }
}
private string _enumMember;
public string EnumMember
{
get { return _enumMember; }
set {
_enumMember = value;
_simple = (Enum)Enum.Parse(Type.GetType(EnumType), EnumMember);
}
}
}
public class Program
{
static void Main(string[] args)
{
var str = new StringBuilder();
using (var writer = XmlWriter.Create(str))
{
try
{
var foo = new Foo
{
Simple = SimpleEnum.three
};
var serializer = new XmlSerializer(typeof(Foo));
serializer.Serialize(writer, foo);
Console.WriteLine(str.ToString());
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
using (TextReader reader = new StringReader(str.ToString()))
{
try
{
var serializer = new XmlSerializer(typeof(Foo));
var foo = (Foo)serializer.Deserialize(reader);
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
Console.ReadLine();
}
}
I have written an Interface for writing a very very simple Plugin. In fact it is just a class that is loaded at runtime out of a dll file and is stored as Property in another class. That class that stores the interface has to get serialized. As example this is my serialized object:
<?xml version="1.0" encoding="utf-8"?><MD5HashMapper xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.namespace.net" />
But now If i want to load that Object I get an Exception:
As example :
{"<MD5HashMapper xmlns='http://www.vrz.net/Vrz.Map'> was not expected."}
So does anyone has an idea how to solve that problem?
Code:
I have an Interface named IMap that is shared in a dll file to create Addins based on that interface:
public interface IMap
{
object Map(object input);
}
I also have different Mappers (you can pass an input through them and they modify the output). All Mappers are derived from:
[XmlInclude(typeof(ConstMapper))]
[XmlInclude(typeof(FuncMapper))]
[XmlInclude(typeof(IdentMapper))]
[XmlInclude(typeof(NullMapper))]
[XmlInclude(typeof(RefMapper))]
[XmlInclude(typeof(VarMapper))]
[XmlInclude(typeof(TableMapper))]
[XmlInclude(typeof(AddinMapper))]
public class MapperBase:ComponentBase,IMap
{ public virtual object Map(object input) {
throw new NotImplementedException("Diese Methode muss überschrieben werden");
}
public override string ToString() {
return ShortDisplayName;
}
}
Just forget ComponentBase. It is not important for this...
Now i also have a AddinMapper. The main function of that mapper is to cast create MapperBase Object out of the IMap object:
And that is exactly that class I want to seralize including the properties of the Mapper Property (type IMap).
public class AddinMapper : MapperBase
{
private static MapperBase[] _mappers;
const string addinDirectory = #"Addin\Mappers\";
//Mappers from *.dll files are loaded here:
[XmlIgnore]
public static MapperBase[] Mappers
{
get
{
if (_mappers == null)
{
List<MapperBase> maps = new List<MapperBase>();
foreach (string dll in Directory.GetFiles(addinDirectory, "*.dll"))
{
if (Path.GetFileName(dll) != "IMap.dll")
{
var absolutePath = Path.Combine(Environment.CurrentDirectory, dll);
Assembly asm = Assembly.LoadFile(absolutePath);
foreach (Type type in asm.GetTypes().ToList().Where(p => p.GetInterface("IMap") != null))
{
maps.Add(new AddinMapper((IMap)Activator.CreateInstance(type)));
}
}
}
Mappers = maps.ToArray();
}
return _mappers;
}
set
{
_mappers = value;
}
}
IMap _base;
public string MapperString { get; set; }
[XmlIgnore()]
public IMap Mapper
{
get
{
if (_base == null)
{
Type type = null;
foreach (MapperBase mapperBase in Mappers)
{
if (mapperBase is AddinMapper && ((AddinMapper)mapperBase).Mapper.GetType().FullName == _mapperName)
{
type = (mapperBase as AddinMapper).Mapper.GetType();
break;
}
}
if (type != null)
{
XmlSerializer serializer = new XmlSerializer(type);
using (StringReader reader = new StringReader(MapperString))
{
Mapper = (IMap)serializer.Deserialize(reader);
}
}
}
return _base;
}
private set
{
_base = value;
StoreMapperString();
}
}
string _mapperName;
[System.ComponentModel.Browsable(false)]
public string MapperName
{
get
{
return _mapperName;
}
set
{
_mapperName = value;
}
}
public AddinMapper(IMap baseInterface) : this()
{
Mapper = baseInterface;
_mapperName = baseInterface.GetType().FullName;
}
public AddinMapper()
{
}
public override object Map(object input)
{
return Mapper.Map(input);
}
public override string ToString()
{
return Mapper.ToString();
}
private void StoreMapperString()
{
MemoryStream memstream = new MemoryStream();
XmlStore.SaveObject(memstream, Mapper);
using (StreamReader reader = new StreamReader(memstream))
{
memstream.Position = 0;
MapperString = reader.ReadToEnd();
}
}
}
An example for such a addin would be:
public class ReplaceMapper : IMap
{
public string StringToReplace { get; set; }
public string StringToInsert { get; set; }
public object Map(object input)
{
if (input is string)
{
input = (input as string).Replace(StringToReplace, StringToInsert);
}
return input;
}
}
And the Problem is I want to save the Settings like StringToReplace,... as xml
I ve solved my problem:
I really don t know why but take a look at this article: http://www.calvinirwin.net/2011/02/10/xmlserialization-deserialize-causes-xmlns-was-not-expected/
(if link is dead later)
XmlRootAttribute xRoot = new XmlRootAttribute();
xRoot.ElementName = elementName;
xRoot.IsNullable = true;
XmlSerializer ser = new XmlSerializer(typeof(MyObject), xRoot);
XmlReader xRdr = XmlReader.Create(new StringReader(xmlData));
MyObject tvd = (MyObject)ser.Deserialize(xRdr);
Now the important thing: It does not matter if you don t get an excption on serialization. You have to add the XmlRootAttribute on both ways: Serialisation and Deserialization.