We Keep Coding

Parsing multiple JSON objects sent via Express client-side in C#

I'm trying to Parse two objects sent via Express to my unity client into c# objects.
Here I am sending the search results of a mongoose.model.find
res.send({account: accountData, score: scoreData})
Over in my Unity client this is the function that grabs the data above from my node.js server. I recieve the information here as the www.downloadHandler.text
using (UnityWebRequest www = UnityWebRequest.Post(hostURL + "Login", loginForm))
{
yield return www.SendWebRequest();
if (www.result != UnityWebRequest.Result.Success)
{
Debug.Log("Error Sending Login Detail: " + www.error);
}
else
{
if (www.responseCode == 200)
{
Debug.Log(www.downloadHandler.text);
thisUser = username;
menu.LoginPassed();
leaderboard.LoadPlayerScoreData(www.downloadHandler.text);
}
else
{
Debug.Log("Login Unsuccessful. Reason: " + www.downloadHandler.text);
}
}
}
}
for reference here is the output of the Debug.Log(www.downloadHandler.text) so we can see what we are working with:
{
"account":[
{
"_id":"62fc40709bc9c241c7f650cd",
"username":"BaggyJ",
"password":"12345",
"email":"#hotmail",
"hhs":0,
"level":1,
"xp":0,
"currency":0,
"premiumCurrency":0,
"__v":0
}
],
"score":[
{
"_id":"62fb4efe6e3a942138405b3e",
"username":"BaggyJ",
"score":420,
"__v":0
}, // ...
]
}
Here I am recieving the information as intended, the account data contains the player account informaton and the score data is an array of the players best scores. So far so good!
The next step obviously, is to parse this information into a usable C# object. I attempted to do this in the same way I had previously accomplished in this project.
I will show you here the solution for this I'm using that does not work, and an example of the solution when it does work.
Firstly, what I have thats not working:
I created two C# serialized classes to represent the information.
[System.Serializable]
public class DataPackage
{
public string account;
public string score;
}
And
[System.Serializable]
public class DataPackages
{
public DataPackage[] dataPackages;
}
Finally, I use the JSON Utility function to parse the data string included above
public DataPackages ParseData(string jsonText)
{
DataPackages dataPackages = JsonUtility.FromJson<DataPackages>("{\"dataPackages\":" + jsonText + "}");
return dataPackages;
}
If I attempt to use the above code the dataPackages.dataPackages object is always null.
The function is meant to result in an object with two strings, the first to represent the account data and the second to represent the score data.
These two string will then be parsed a second time into their final C# objects for use.
It is worth noting here that the parsing tools for account data and score data are built the same way and function as expected. I will include one below for reference:
[System.Serializable]
public class ScorePacket
{
public string username;
public string score;
}
[System.Serializable]
public class ScorePackets
{
public ScorePacket[] scorePackets;
}
public ScorePackets ParseScore(string jsonText)
{
ScorePackets scorePackets = JsonUtility.FromJson<ScorePackets>("{\"scorePackets\":" + jsonText + "}");
return scorePackets;
}
The above code will return an c# object where I can access its data as such: scorePackets.scorePackets[0].score
Any insight into why I cannot accomplish this with DataPackages would be greatly appreciated!
EDIT
So I've gotten it a little closer but not quite.
I have replaced:
DataPackages dataPackages = JsonUtility.FromJson<DataPackages>("{\"dataPackages\":" + jsonText + "}");
With:
DataPackages dataPackages = JsonUtility.FromJson<DataPackages>("{\"dataPackages\":[" + jsonText + "]}");
This successfully creates a C# object with the correct fields HOWEVER both of these field are empty.
Debug.Log(dataPackages.dataPackages[0].score)
Debug.Log(dataPackages.dataPackages[0].account)
Both return empty strings.
Edit#2
We are getting so close I can taste it.
So in addition to the above changes I have also made the following change to my node.js server:
res.send({account: accountData.toString(), score: scoreData.toString()})
Now if I:
Debug.Log(dataPackages.dataPackages[0].score.ToString());
Debug.Log(dataPackages.dataPackages[0].account.ToString());
My output is:
[object Object],[object Object],[object Object],[object Object],[object Object]
[object Object]
.ToString() failed to get the json text for these objects

I figured it out!
I needed to change the way I declared my DataPacket class from containing strings to containing object arrays.
[System.Serializable]
public class DataPackage
{
public AccountPacket[] account;
public ScorePacket[] score;
}
Solved my problems

datacollection program not correctly collecting

I created an data collection app for our company which collect data from our remote devices.
The data is collected from a datamailbox which is comparable with an database that works like an 10 day buffer to store the data. this is all correctly working.
The data is collected through post api requests. for example :
var url = BuildUrl("syncdata");
var response = webClient.CallApi(url, new NameValueCollection() { { "createTransaction","" }, { "lastTransactionId", transactionId } });
var data = DynamicJson.Parse(response);
transactionId = data.transactionId;
I've been trying to collect multiple devices at one time but the problem is that it starts running and collect the data from the first device which works. Than our second device will start collecting the data but it only starts from where device one ended so i've been losing 12hours of data each run. For performance we use transactionId's.(each set of data has its own Id)
The workflow should be like this :
When the data is retrieved for the first time, the user specifies only
the createTransaction filter. The DataMailbox returns all the data of
all devices gateways – with historical data – of the account along a
transaction ID. For the next calls to the API, the client specifies
both createTransaction and lastTransactionId filters. The
lastTransactionId is the ID of the transaction that was returned by
the latest request. The system returns all the historical
data that has been received by the DataMailbox since the last
transaction and a new transaction ID. deviceIds is an additional
filter on the returned result. You must be cautious when using the
combination of lastTransactionId, createTransaction and deviceIds.
lastTransactionId is first used to determine what set of data — newer
than this transaction ID and from all the Device gateways — must be
returned from the DataMailbox, then deviceIds filters this set of data
to send data only from the desired device gateways. If a first request
is called with lastTransactionId, createTransaction and deviceIds, the
following request — implying a new lastTransactionId — does not
contain values history from the previous lastTransactionId of the
device gateways that were not in the deviceId from previous request.
I'm really struggling with the data collection and have no clue how to use the TransactionId and the LastTransActionId.This is the code for now
try
{
CheckLogin();
using (var webClient = new MyWebClient())
{
bool moreDataAvailable;
int samplesCount = 0;
string transactionId = Properties.Settings.Default.TransactionId;
string lastTransactionId = Properties.Settings.Default.LastTransactionId;
do
{
var url = BuildUrl("syncdata");
var response = webClient.CallApi(url, new NameValueCollection() { { "createTransaction","" }, { "lastTransactionId", transactionId } });
var data = DynamicJson.Parse(response);
transactionId = data.transactionId;
var talk2MMessage = getTalk2MMessageHeader(webClient);
if (talk2MMessage != null)
{
}
foreach (var ewon in data.ewons)
{
Directory.CreateDirectory(ewon.name);
foreach (var tag in ewon.tags)
{
try
{
Console.WriteLine(Path.Combine(ewon.name, tag.name + ""));
foreach (var sample in tag.history)
{
Console.WriteLine(ewon.name + " " + tag.name + " " + tag.description);
Console.WriteLine(transactionId);
samplesCount++;
}
}
catch (RuntimeBinderException)
{ // Tag has no history. If it's in the transaction, it's most likely because it has alarm history
Console.WriteLine("Tag {0}.{1} has no history.", ewon.name, tag.name);
}
}
}
Console.WriteLine("{0} samples written to disk", samplesCount);
// Flush data received in this transaction
if (Properties.Settings.Default.DeleteData)
{
//Console.WriteLine("Flushing received data from the DataMailbox...");
url = BuildUrl("delete");
webClient.CallApi(url, new NameValueCollection() { { "transactionId", transactionId } });
Console.WriteLine("DataMailbox flushed.");
}
//save the transaction id for next run of this program
Properties.Settings.Default.LastTransactionId = lastTransactionId;
Properties.Settings.Default.Save();
// Did we receive all data?
try
{
moreDataAvailable = data.moreDataAvailable;
}
catch (RuntimeBinderException)
{ // The moreDataAvailable flag is not specified in the server response
moreDataAvailable = false;
}
if (moreDataAvailable)
Console.WriteLine("There's more data available. Let's get the next part...");
}
while (moreDataAvailable);
Here are my credentials for starting the collection like all parameters
static void CheckLogin()
{
if (string.IsNullOrEmpty(Properties.Settings.Default.Talk2MDevId))
{
Properties.Settings.Default.Talk2MDevId = Prompt("Talk2MDevId");
Properties.Settings.Default.APIToken = Prompt("API Token");
string deleteInputString = Prompt("Delete data after synchronization? (yes/no)");
Properties.Settings.Default.DeleteData = deleteInputString.ToLower().StartsWith("y");
Properties.Settings.Default.TransactionId = "";
Properties.Settings.Default.LastTransactionId = "";
Properties.Settings.Default.Save();
}
I think it's something with the transactionId and LastTransaction id but i have no clue.
More information can be found here: https://developer.ewon.biz/system/files_force/rg-0005-00-en-reference-guide-for-dmweb-api.pdf

As I understand your question, you problem is that for the first few transactionIds, you only get data from device 1 and then only data from device 2.
I'm assuming the following in my answer:
You didn't specify somewhere else in code the filter on "ewonid"
When you say you lose 12 hours of data , you are assuming it because "device 2" data are streamed after "device 1" data.
You did try without the /delete call with no change
/syncdata is an endpoint that returns a block of data for an account since a given transactionId (or oldest block if you didn't provide a transactionID). This data is sorted by storage date by the server, which depends on multiple factors:
when was the device last "vpn online"
at which frequency the device is pushing data to datamailbox
when was that device packet digested by datamailbox service
You could technically have 1 year old data pushed by a device that gets connected back to vpn now, and those data would be registered in the most recent blocks.
For those reasons, the order of data block is not the order of device recording timestamp. You always have to look at the field ewons[].tags[].history[].date to known when that measure was made.
foreach (var sample in tag.history)
{
Console.WriteLine(ewon.name + " " + tag.name + " " + tag.description);
Console.WriteLine(sample.value + " at " + sample.date);
Console.WriteLine(transactionId);
samplesCount++;
}
In your case, I would assume both devices are configured to push their data once a day, one pushing it's backlog, let's say, at 6AM and the other at 6PM.

gRPC: How to use dynamically typed values?

I'm trying using gRPC dynamically typed values but with the little information about their usefulness, It's almost impossible to do this... So I will show the image/code that I have problems and the questions that are eating my brain
gRPC Method I'm doing:
public override Task<HelloReply2> TestObject(Status request, ServerCallContext context) {
//The part I may have problems
var status = new Status();
//here I want a User that corresponds to my request.Data
//example -> request.Data = User1 (gives me null if User1 don`t exist in db)
// request.Data = 14 (gives me null if 14 don`t exist in db)
// request.Data = true (gives me null if true don`t exist in db)
var a1 = _context.Users_5.FirstOrDefault(x => x.Username.Equals(request.Data));
var b1 = _context.Users_5.FirstOrDefault(x => x.Email.Equals(request.Data));
var c1 = _context.Users_5.FirstOrDefault(x => x.Age.Equals(request.Data));
var d1 = _context.Users_5.FirstOrDefault(x => x.Test.Equals(request.Data));
//is a bool
//here i want too Create dynamic values
status.Data = Value.ForStruct(new Struct {
Fields =
{
["Integer"] = Value.ForNumber(c1!.Age),
["StringName"] = Value.ForString(a1!.Username),
["StringEmail"] = Value.ForString(b1!.Email),
["Boolean"] = Value.ForBool(d1!.Test)
}
});
//Below is just a simple string who gives different string (depending on the
//data Status (also how to read the message from the status.Data ?)
HelloReply2 hello = new();
if(a1 != null)
{
hello.Message = "There is a User with the Username " + request.Data + ". His Email is " + a1.Email;
} else if (b1 != null) {
hello.Message = "There is a User with the Email " + request.Data + ". His Username is " + b1.Username;
}
else if (c1 != null)
{
hello.Message = "There is at least one User with that Age of " + request.Data + ". His Username is " + c1.Username;
}
else if (d1 != null)
{
if(d1.Test == true)
{
hello.Message = "There is at least one User who dislikes chocolate: " + request.Data + ". His Username is " + d1.Username;
} else
{
hello.Message = hello.Message = "There is at least one User who likes chocolate: " + request.Data + ". His Username is " + d1.Username;
}
}
else
{
hello.Message = "We didn't find something with the value that the User put in. Value:" + request.Data;
}
return Task.FromResult(hello);
}
Questions: How to Get the one Value from my gRPC? How to convert a "Object" in c# (one string, one integer or one List) into a ONE value of google.protobuf.Value (so it not give me errors like this Controller from a Web Api below)? Is something wrong with my gRPC Service Method (is something wrong reading the dynamic values? Can I do that calls for getting a User for a DB? How to read dynamic values?)
// I try using Google.Protobuf.WellKnownTypes.Value obj but
//not workings because gives me a lot of values to put
[HttpGet("TypeObject/{obj}")]
public async Task<ActionResult<HelloReply2>> TypeObject([FromRoute] Object obj){
Status objRequest = new Status { Data = (Google.Protobuf.WellKnownTypes.Value)
obj };
//cannot do this (gives me error of casting but the ideia is doing something
//similar to this)
var hello = await _greetClient.TestObjectAsync(objRequest);
return Ok(hello);
}
Any help on how to resolve this error of using Value gRPC or if is something wrong with the code is always welcome.
Edit:
One day after this question I don't have any solutions / progress. I was think of doing Any or OneOf for testing but it also gives me errors (who don't make sense at all). This code from Microsoft (C# Format part is not recognize) doesn't work in my project with the protos reload (the problem is not in the protos)
Link: https://learn.microsoft.com/en-us/dotnet/architecture/grpc-for-wcf-developers/protobuf-any-oneof
How I can use Any / OneOf without give me error in the Formating? What is the difference between Value and this two? Can the three (Value, Any, OneOf) be dynamic/Object values (if yes how to convert the types)?
Edit 2:
Still have problems, I'm trying using gRPC Any , and maybe have some progress (not all).
So with Any I have my method in the server gRPC and it is like this
public override Task<HelloReply2> TestObject3(AnyMessage request, ServerCallContext context){
HelloReply2 anyMessageResponse;
var y = request.TypeUrl;
switch (request.TypeUrl)
{
case "type.googleapis.com/any.HelloRequest":
var string_1 = request.Unpack<HelloRequest>();
anyMessageResponse = new HelloReply2{
Message = "You type String: " + $"{string_1.Name}"
};
break;
case "type.googleapis.com/any.TestInteger1":
var integer_1 = request.Unpack<TestInteger1>();
anyMessageResponse = new HelloReply2{
Message = "You type Integer: " + $"{integer_1.Message}"
};
break;
case "type.googleapis.com/any.TestBool1":
var bool_1 = request.Unpack<TestInteger1>();
anyMessageResponse = new HelloReply2{
Message = "You type Bool: " + $"{bool_1.Message}"
};
break;
default:
throw new InvalidOperationException("Unexpected type URL.");}
return Task.FromResult(anyMessageResponse);
}
This ideia comes from here (https://github.com/grpc/grpc-dotnet/issues/917), but the client part their don't have any much info or I don't understand that part
This is what I did in the WebApi (who is my client and the code is similar to the above one)
using AnyMessage = Google.Protobuf.WellKnownTypes.Any;
[HttpGet("TypeObject3/{obj3}")]
public async Task<ActionResult<HelloReply2>> TypeObject3([FromRoute] string obj3)
{
AnyMessage objRequest = new() { TypeUrl = obj3 };
var hello = await _greetClient.TestObject3Async(objRequest);
var l = hello.Message;
return Ok(hello);
}
First I had the variable Any declared in the method instead of string but as you can only put string and stringBytes so I preferred to put it like this (with the string as an obj3 variable) but my goal is to see if the variable is of type TestBool1 or TestInteger1 as I have declared in the protos and not be a string that I will be able to see, and the biggest problem was if I had more variables inside the messages how to proceed? So my secondary question is how to use Any on the client side via the Web-Api? I forgot to say but I'm using .Net 6 Core and for testing I'm using Swagger, where at this moment my error hits the Exception dictated by the Server method.
Questions: Why TypeUrl is a string and not object? How to fix my problem? How to test the object type (or string) for more values if the messages was with 1 more types?
Also I will show my test proto too show how I'm doing this
import "google/protobuf/struct.proto";
import "google/protobuf/any.proto";
package greet;
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayNormalHello (Empty_2) returns (HelloReply);
rpc SayHello (HelloRequest) returns (HelloReply2);
rpc TestInt (TestInteger1) returns (HelloReply2);
rpc TestBoolean (TestBool1) returns (HelloReply2);
rpc TestObject (Status) returns (HelloReply2); //Not working
rpc TestObject2 (Status2) returns (HelloReply2); //Not working
rpc TestObject3 (google.protobuf.Any) returns (HelloReply2); //Also
//Not working
}
message Empty_2{
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings.
message HelloReply {
string message = 1;
}
// The response message containing the greetings.
message HelloReply2 {
string message = 1;
}
message TestInteger1 {
int32 message = 1;
}
message TestBool1 {
bool message = 1;
}
message Status {
google.protobuf.Value data = 1;
}
message Status2 {
google.protobuf.Any data = 1;
}
Any help is welcome.

In the project I developed, I needed an infrastructure where I could perform dynamic operations such as REST service using gRPC. As I understand it, what you want is something similar to this.
I have developed a solution for this. It might work for you too.
You can gain some flexibility by making a definition with a single String field for Requests and Responses that you want to be Dynamic, and then using this field to hold JSON data as strings.
For example, instead of defining different responses for different types, you can solve it by making one definition like this.
message HelloReply {
// Stringified JSON Data
string data = 1;
}

can't load scores from leaderboard

i'm setting up a leaderboard system in my unity game, using the google play games services plugin.
i want to load score in order to integrate them in my custom LeaderbordUI,
I followed the documentation and used the ILeaderboard.LoadScores but it's not working.
when i check the logcat i get this :
02-04 11:03:56.580: W/Unity(18969): !!! [Play Games Plugin DLL] 02/04/19 11:03:56 +01:00 WARNING: Error returned from fetch: -108
I have tried to loadScore with the method "Social.LoadScores" and "PlayGamesPlatform.Instance.LoadScores", but i'm getting the same warning.
PS: when i use Social.ShowLeaderboardUI() it shows me the leaderboard.
but when i use PlayGamesPlatform.Instance.ShowLeaderboardUI(LB_Stars.id) to show a specific leaderboard it gives me "hmm,something went wrong in play games"
public void LoadLeaderboard()
{
LB_Stars.LoadScores(ok =>
{
if (ok)
{
LoadUsersAndDisplay(LB_Stars);
}
else
{
Debug.Log("Error retrieving STARS leaderboard");
}
});
}
internal void LoadUsersAndDisplay(ILeaderboard lbStar)
{
Debug.Log("gonna load user and display them");
List<string> userIds = new List<string>();
foreach (IScore score in lbStar.scores)
{
userIds.Add(score.userID);
}
Social.LoadUsers(userIds.ToArray(), (users) =>
{
string status = "Leaderboard loading: " + lbStar.title + " count = " +
lbStar.scores.Length;
foreach (IScore score in lbStar.scores)
{
IUserProfile user = FindUser(users, score.userID);
if (user != null)
{
UserLeaderboardClone = Instantiate(UserLeaderboardPrefab);
UserLeaderboardClone.name = score.rank.ToString();
LeaderboardUserScript lbUser = UserLeaderboardClone.GetComponent<LeaderboardUserScript>();
lbUser.transform.SetParent(LBScrollview.content.transform, false);
FillUserInfo(lbUser, user, score);
}
}
});
}

Okay i've figured it out, based on a comment on github https://github.com/playgameservices/play-games-plugin-for-unity/issues/2045#issuecomment-350335234

After spending a couple of hours on this, my problem turned out to be my game was using leaderboard ids from another app.
So it failed with this "unauthorized error", however the exact cause was not given.
The reason why it happened was because the play games configuration in unity was caching old values. When you open it - it has the correct ids, however in the generated GPGSids.cs file - wrong ids are present.
The solution was simply to regenerate that by re-saving the configuration.

Cannot cast from source type to destination type Json

I'm retrieving a json file from my server and then using Json to deserialize the content. How ever I keep receiving this error:
Cannot cast from source type to destination type
I was following the steps of the Minijson script but yet this error still comes up. Some help would be appreciated.
void Start () {
//creating url
image1Request = new WWW("http://development.someurl.com/MoreGames/MoreGames.json");
StartCoroutine(ImageOne(image1Request));
}
IEnumerator ImageOne(WWW www)
{
//wait until url is loaded
yield return www;
//load image into texture slot
if (www.error == null)
{
//assigning URLS
var dict = Json.Deserialize(www.text) as Dictionary<string,object>;
Debug.Log(www.text);
Debug.Log("deserialized: " + dict.GetType());
Debug.Log("dict['string']: " + (string)dict["widget"]);
}
else
{
Debug.Log("WWW Error: " + www.error);
}
}

Instead of using
var dict = Json.Deserialize(www.text) as Dictionary<string,object>;
I would suggest you to use the following code snippet
var dict = new System.Web.Script.Serialization.JavaScriptSerializer(); Dictionary<string, object> dictObject =(Dictionary<string,object>)jsSerializer.DeserializeObject(www.text);

Your Dictionary's value type is object. You are trying to cast an object type to string. It should be corrected to :
(object)dict["widget"]
I'm not sure what you are trying to log there, but if it is the widget object's name it should go like this :
Debug.Log("dict['string']: " + ((object)dict["widget"]).ToString());