I am creating a solution in SSIS to get data from an API/url, example below:
URL is obviously is different in actual project
https://{your_server}}/api/v1/{your_account}/internalqueries?queryguid=F6AC1BB3-DC03-4E64-BC46-30DEDE3BD27E&filterguid=AAED4F63-76CC-4E09-8C97-2C425D4E48EF&viewguid=DD4F7938-951F-4279-953B-A6EB075FCB35&max_rows=3
Expected JSON Result:
{
"HREF": "https://{your_server}/api/v1/{your_account}/internalqueries?queryguid=%7B6938436D-B024-4B9E-9815-A41C1D7C7A0E%7D&filterguid=%7B2FAC2998-B0EC-45BF-9B98-1A09B1F8C343%7D&viewguid=%7B631587D5-EED1-49C7-9252-54E08A398CDE%7D&max_rows=3",
"recordcount": 3,
"previouspage": 0,
"nextpage": 0,
"records": [
{
"Creation Date": "1/9/2017 11:07:34 am",
"Incident Number": "I181218_000003",
"Category": "Get Help",
"Title": "",
"Recipient": "William",
"Description": "",
"Manager": "",
"Status": "In Progress",
"Time Status": "4",
"Priority": "2",
"SLA Target": "1/11/2017 11:07:34 am",
"Cost": "70.00",
"End Date": ""
},
]
}
Created a class called ResultAPI:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace SC_b15eeeff0e3544d3a882578b3eb9bbba
{
public class ResultAPI
{
public string HREF { get; set; }
public int recordcount { get; set; }
public int previouspage { get; set; }
public string Recipient { get; set }
public string records { get; set; }
}
}
Another class called GenericResult:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace SC_b15eeeff0e3544d3a882578b3eb9bbba
{
public class GenericResponse
{
public ResultAPI[] ListData { get; set; }
}
public class ResultGen
{
public GenericResponse Result { get; set; }
}
}
Main:
public override void CreateNewOutputRows()
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
HttpClient client = new HttpClient();
client.BaseAddress = new Uri("https://{your_server}}/api/v1/{your_account}/internalqueries?queryguid=F6AC1BB3-DC03-4E64-BC46-30DEDE3BD27E&filterguid=AAED4F63-76CC-4E09-8C97-2C425D4E48EF&viewguid=DD4F7938-951F-4279-953B-A6EB075FCB35&max_rows=3");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
var cred = Convert.ToBase64String(Encoding.ASCII.GetBytes(string.Format("{0}:{1}", "username", "Password")));
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", cred);
string APIUrl = ("https://{your_server}}/api/v1/{your_account}/internalqueries?queryguid=F6AC1BB3-DC03-4E64-BC46-30DEDE3BD27E&filterguid=AAED4F63-76CC-4E09-8C97-2C425D4E48EF&viewguid=DD4F7938-951F-4279-953B-A6EB075FCB35&max_rows=3");
var response = client.GetAsync(APIUrl).Result;
if (response.IsSuccessStatusCode)
{
var result = response.Content.ReadAsStringAsync().Result;
var serializer = new JavaScriptSerializer();
serializer.MaxJsonLength = Int32.MaxValue;
var data = serializer.Deserialize<ResultAPI>(result);
APIResultsBuffer.AddRow();
APIResultsBuffer.previouspage = data.previouspage;
APIResultsBuffer.recordcount = data.recordcount;
APIResultsBuffer.HREF = data.HREF;
// Getting error here Property of indexer cannot be assigned to---it is read only**
APIResultsBuffer.records = data.records;
// this is just for illustration if I want to get actual values of the Recipient field in the SQL table**
// *APIResultsBuffer.Recipient= data.Recipient*
}
}
}
I have SQL Server DB as an OLE Destination, what I am trying to figure out is how do I get actual records in SQL Server table, all I am getting is HREF, previouspage and recordcount correctly and everything else is blank but what should be the data type for records in Script Component--Input and Output and what changes I need to make in order to get the actual Recipient data from the JSON above.
Any help appreciated.
Tried changing data type for records but nothing seems to be working
To update:
I updated the CreatedNewOutput section:
JavaScriptSerializer serializer = new JavaScriptSerializer();
serializer.MaxJsonLength = Int32.MaxValue;
List<ResultAPI> suppliers = serializer.Deserialize<List<ResultAPI>>(APIUrl);
foreach (ResultAPI resultAPI in suppliers)
APIResultsBuffer.HREF = resultAPI.Recipient;
I am getting an error: Invalid JSON primitive: https.
To address
// Getting error here Property of indexer cannot be assigned to---it is read only**
APIResultsBuffer.records = data.records;
I'm guessing you defined the records column in SSIS as DT_TEXT or DT_NTEXT which is why you're getting that error. You cannot directly assign a value to a blob type in SSIS. Instead, you need to call the AddBlobData but you can only pass in a byte array so you'll need to crib from How do I convert a String to a BlobColumn in SSIS
Output0Buffer.AddRow();
string aString = "X".PadRight(10000) + "X";
// https://stackoverflow.com/questions/35703025/how-do-i-convert-a-string-to-a-blobcolumn-in-ssis
byte[] bytes = new byte[aString.Length * sizeof(char)];
System.Buffer.BlockCopy(aString.ToCharArray(), 0, bytes, 0, bytes.Length);
Output0Buffer.RecordsText.AddBlobData(bytes);
Related
I am consuming a Web API, but when serializing I get this error:
Cannot deserialize the current JSON object (e.g. {"name":"value"}) into type 'System.Collections.Generic.List`1[Covid.Models.RegionsModel]' because the type requires a JSON array (e.g. [1,2,3]) to deserialize correctly.
To fix this error either change the JSON to a JSON array (e.g. [1,2,3]) or change the deserialized type so that it is a normal .NET type (e.g. not a primitive type like integer, not a collection type like an array or List) that can be deserialized from a JSON object. JsonObjectAttribute can also be added to the type to force it to deserialize from a JSON object.
Path 'data', line 1, position 8.
I read several possible solutions, in questions that have already been asked but could not get it to work.
JSON.NET DeserializeObject to List of Objects
Cannot deserialize the current JSON object (e.g. {"name":"value"})
If you could help me please, I have really tried for 3 days but I can't solve it.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Web;
using System.Web.Mvc;
using System.Net.Http;
using Covid.Models;
using System.Net.Http.Headers;
using Newtonsoft.Json;
namespace Covid.Controllers
{
public class HomeController : Controller
{
string Baseurl = "https://covid-19-statistics.p.rapidapi.com/regions";
public async Task<ActionResult> Index()
{
try
{
List<RegionsModel> region = new List<RegionsModel>();
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(Baseurl);
client.DefaultRequestHeaders.Add("x-rapidapi-key", "32ac4c9518msh7fb8f6c7a05eb7bp1178dbjsn193f09c779d6");
client.DefaultRequestHeaders.Add("x-rapidapi-host", "covid-19-statistics.p.rapidapi.com");
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage Res = await client.GetAsync("regions");
if (Res.IsSuccessStatusCode)
{
var response = Res.Content.ReadAsStringAsync().Result;
//var ObjMovmientos = JsonConvert.DeserializeObject<List<RegionsModel>>(response);
var objResponse1 = JsonConvert.DeserializeObject<List<RegionsModel>>(response);
region = objResponse1;
}
return View(region);
}
}
catch (Exception ex)
{
throw ex;
}
}
}
}
The model class is:
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
namespace Covid.Models
{
public class RegionsModel
{
public RegionModel[] data { get; set; }
public class RegionModel
{
[JsonProperty("iso")]
public string Iso { get; set; }
[JsonProperty("name")]
public string Name { get; set; }
}
}
}
The JSON is:
{
"data": [
{
"iso": "CHN",
"name": "China"
},
{
"iso": "TWN",
"name": "Taipei and environs"
},
{
"iso": "USA",
"name": "US"
},
{
"iso": "JPN",
"name": "Japan"
},
{
"iso": "WSM",
"name": "Samoa"
}
]
}
RegionsModel already contains a property that defines an array, so this is wrong:
var objResponse1 = JsonConvert.DeserializeObject<List<RegionsModel>>(response);
It should be:
var objResponse1 = JsonConvert.DeserializeObject<RegionsModel>(response);
Also, your method is already asynchronous. So instead of this:
var response = Res.Content.ReadAsStringAsync().Result;
Do this:
var response = await Res.Content.ReadAsStringAsync();
Finally, change this:
List<RegionsModel> region = new List<RegionsModel>();
// ...
region = objResponse1;
To this:
List<RegionModel> region = new List<RegionModel>();
// ...
region = objResponse1.data.ToList();
Change this line,
var objResponse1 = JsonConvert.DeserializeObject<List<RegionsModel>>(response);
To
var objResponse1 = JsonConvert.DeserializeObject<RegionsModel>(response);
Reason is because your json is of an object, data ,that has an array of objects...
I have C# object
Step1Members step1Members = new Step1Members()
{
businessName = "Test business",
contactName = "Test User"
};
I want to convert step1Members into DynamoDB json like below.
{"businessName" : { "S" : "Test business" }, "contactName" : { "S" : "Test User" }}
Please help me on this.
You can actually do it using the EfficientDynamoDb.
Based on your example, let's imagine you have a class like this:
public class Step1Members
{
[DynamoDbProperty("businessName")]
public string BusinessName { get; set; }
[DynamoDbProperty("contactName")]
public string ContactName { get; set; }
}
In order to generate a DynamoDb JSON, you need to:
// Create DynamoDb context, credentials are not required for JSON generation, only for real database calls
var context = new DynamoDbContext(new DynamoDbContextConfig(RegionEndpoint.EUCenteral1, new AwsCredentials("public_key", "private_key")));
var step1Members = new Step1Members()
{
BusinessName = "Test business",
ContactName = "Test User"
};
await using var memoryStream = new MemoryStream();
await using var jsonWriter = new Utf8JsonWriter(memoryStream);
// Convert a class object into the document
var document = context.ToDocument(step1Members);
// Write the document using Utf8JsonWriter
jsonWriter.WriteStartObject();
foreach (var attribute in document)
{
jsonWriter.WritePropertyName(attribute.Key);
attribute.Value.Write(jsonWriter);
}
jsonWriter.WriteEndObject();
jsonWriter.Flush();
// Read the final JSON string from the memory stream
var jsonString = Encoding.UTF8.GetString(memoryStream.ToArray());
If you don't want to create a C# class for every single use-case, you can just use the plain Document class, which is basically a dictionary.
I have a web api controller in .NET Core 2.1, which receives
JToken jsonBody
The json has the following structure
{
"id": "xxx",
"payload": {
"TelephoneNumber": "1111",
"Name": "Hans"
}
}
and more fields, but it's irrelevant.
I want to retrieve the Number and Name elegantly. Currently, I do the following, which I'm sure could be done in a nicer way:
var payload = JObject.Parse(jsonBody.SelectToken("Payload").ToString());
telephoneNumber = new TelephoneNumber(payload.SelectToken("TelephoneNumber").ToString());
I've tried just doing
jsonBody.SelectToken("Payload.TelephoneNumber")
but that doesn't work. I think that it's because somehow the jsonBody, that the controller receives, has only parsed the top nodes as json, hence it could be that it regards the value of
jsonBody.SelectToken("Payload")
as a string.
As per official documentation - you can do something like this:
var phone = jsonBody["payload"]["TelephoneNumber"].ToString();
var name = jsonBody["payload"]["Name"].ToString();
See a live demo on rextester.
This is at least a little bit more elegant:
var jsonBody = JObject.Parse(#"{
'id': 'xxx',
'payload': {
'TelephoneNumber': '1111',
'Name': 'Hans'
}
}");
var phone = jsonBody["payload"]["TelephoneNumber"].Value<string>();
var name = jsonBody["payload"]["Name"].Value<string>();
If you don't want to deserialize your full json, you can create a class with the properties you need
public class Payload
{
public string TelephoneNumber { get; set; }
public string Name { get; set; }
}
And then use JsonTextReader to deserialize the string:
private static Payload DeserializePayload(JToken token)
{
var serializer = new JsonSerializer();
using (JsonTextReader reader = new JsonTextReader(new StringReader(token.ToString())))
{
reader.CloseInput = true;
while (reader.Read())
{
if (reader.TokenType == JsonToken.StartObject && reader.Path.Equals("payload"))
{
var payload = serializer.Deserialize<Payload>(reader);
return payload;
}
}
}
// not found - return null? throw exception?
return null;
}
Testing the code:
var token = JToken.Parse(#"{
""id"": ""xxx"",
""payload"": {
""TelephoneNumber"": ""1111"",
""Name"": ""Hans""
}
}");
Payload payload = DeserializePayload(token);
Console.WriteLine($"Name: {payload.Name}, Phone number: {payload.TelephoneNumber}");
I'm having an issue deserializing a JSON string into a RootObject class with 1 string property and a list of custom objects.
When I debug the application and the code deserializes the json I get my 'ErrorCode' property, "test", is populated in the RootObject class but the 'meets' property is always null.
The code sits in a cross platform Xamarin forms application currently, but I've pulled the code and class definitions out and ran them in a simple console application that worked first time with no issues. I'm struggling to figure out what I'm doing wrong.
My simple JSON object is the following:
{"meets": [
{
"VenueName": "O2"
},
{
"VenueName": "wembly"
},
{
"VenueName": "NEC"
}
],
"ErrorCode": "test"}
My class definitions:
[JsonObject(Id = "Meets")]
public class Meets
{
[JsonProperty(PropertyName = "VenueName")]
public string VenueName { get; set; }
}
public class RootObject
{
public List<Meets> Meets { get; set; }
public string ErrorCode { get; set; }
}
The code to hit the api and get the json object (which is blanked out and a smaller simple object in its place):
using System;
using System.Collections.ObjectModel;
using System.Diagnostics;
using System.Threading.Tasks;
using Xamarin.Forms;
using EliteNfcBet.Models;
using System.Net.Http;
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
namespace EliteNfcBet.ViewModels
{
public class ItemsViewModel : BaseViewModel
{
public ObservableCollection<Meets> RaceMeets { get; set; }
public Command LoadItemsCommand { get; set; }
public ItemsViewModel ()
{
Title = "Select Meeting";
RaceMeets = new ObservableCollection<Meets>();
LoadItemsCommand = new Command(async () => await ExecuteLoadItemsCommand());
}
async Task ExecuteLoadItemsCommand()
{
if (IsBusy)
return;
IsBusy = true;
try
{
RaceMeets.Clear();
var result = await GetMeetingsAsync();
//RaceMeets = result;
//foreach (var meet in meets.Meets)
//{
// RaceMeets.Add(meet);
//}
}
catch (Exception ex)
{
Debug.WriteLine(ex);
}
finally
{
IsBusy = false;
}
}
public static async Task<RootObject> GetMeetingsAsync()
{
RootObject meet = new RootObject();
//List<Meet> meets = new List<Meet>();
HttpClient client = new HttpClient();
client.MaxResponseContentBufferSize = 256000;
var uri = new Uri(string.Format(Constants.RestUrl, string.Empty) + "/api/GetAvailablemeetings");
try
{
var response = await client.GetAsync(uri);
if (response.IsSuccessStatusCode)
{
//var content = await response.Content.ReadAsStringAsync();
var content = "{\"meets\":[{\"VenueName\":\"O2\"},{\"VenueName\":\"wembly\"},{\"VenueName\":\"NEC\"}],\"ErrorCode\":\"test\"}";
if (!string.IsNullOrEmpty(content))
{
ITraceWriter traceWriter = new MemoryTraceWriter();
var settings = new JsonSerializerSettings
{
Error = (sender, args) =>
{
if (System.Diagnostics.Debugger.IsAttached)
{
System.Diagnostics.Debugger.Break();
}
},
TraceWriter = traceWriter
};
//result = JsonConvert.DeserializeObject<T>(json, settings);
meet = JsonConvert.DeserializeObject<RootObject>(content, settings);
Console.WriteLine(traceWriter);
}
//if (meets.Count > 0)
//{
// meet.MeetList = meets;
//}
}
}
catch (Exception ex)
{
meet.ErrorCode = "INTERNAL_ERROR";
}
return meet;
}
}
}
EDIT:
I've made some minor changes suggested which are below.
My Json string is now this:
"{\"Meets\":[{\"VenueName\":\"O2\"},{\"VenueName\":\"wembly\"},{\"VenueName\":\"NEC\"}],\"ErrorCode\":\"test\"}"
My classes are below. One thing to note is they are defined within a 'models' namespace in a seperate code file that is doing the deserializing.
namespace EliteNfcBet.Models
{
public class Meets
{
public string VenueName { get; set; }
}
public class RootObject
{
public List<Meets> Meets { get; set; }
public string ErrorCode { get; set; }
}
}
Again i have debugging output when deserializing which looks like it points to the fact that the Meets class member not being found?
{2018-05-28T23:12:22.987 Info Started deserializing EliteNfcBet.Models.RootObject. Path 'Meets', line 1, position 9.
2018-05-28T23:12:22.993 Info Started deserializing System.Collections.Generic.List`1[[NInterpret.InterpretedObject, NInterpret.Xamarin.Droid, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]. Path 'Meets', line 1, position 10.
2018-05-28T23:12:22.994 Info Started deserializing EliteNfcBet.Models.RootObject. Path 'Meets[0].VenueName', line 1, position 23.
2018-05-28T23:12:22.994 Verbose Could not find member 'VenueName' on EliteNfcBet.Models.RootObject. Path 'Meets[0].VenueName', line 1, position 23.
2018-05-28T23:12:22.994 Info Finished deserializing EliteNfcBet.Models.RootObject. Path 'Meets[0]', line 1, position 28.
2018-05-28T23:12:22.995 Info Started deserializing EliteNfcBet.Models.RootObject. Path 'Meets[1].VenueName', line 1, position 42.
2018-05-28T23:12:22.995 Verbose Could not find member 'VenueName' on EliteNfcBet.Models.RootObject. Path 'Meets[1].VenueName', line 1, position 42.
2018-05-28T23:12:22.996 Info Finished deserializing EliteNfcBet.Models.RootObject. Path 'Meets[1]', line 1, position 51.
2018-05-28T23:12:22.997 Info Started deserializing EliteNfcBet.Models.RootObject. Path 'Meets[2].VenueName', line 1, position 65.
2018-05-28T23:12:22.997 Verbose Could not find member 'VenueName' on EliteNfcBet.Models.RootObject. Path 'Meets[2].VenueName', line 1, position 65.
2018-05-28T23:12:22.997 Info Finished deserializing EliteNfcBet.Models.RootObject. Path 'Meets[2]', line 1, position 71.
2018-05-28T23:12:22.998 Info Finished deserializing System.Collections.Generic.List`1[[NInterpret.InterpretedObject, NInterpret.Xamarin.Droid, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]. Path 'Meets', line 1, position 72.
2018-05-28T23:12:23.009 Info Finished deserializing EliteNfcBet.Models.RootObject. Path '', line 1, position 92.
2018-05-28T23:12:23.009 Verbose Deserialized JSON:
{
"Meets": [
{
"VenueName": "O2"
},
{
"VenueName": "wembly"
},
{
"VenueName": "NEC"
}
],
"ErrorCode": "test"
}}
edit 3:
I've just came across this post which explains a lot! Seems like xamarin.forms has a known problem currently with reflection and so certain packages may not work correctly.
JsonConvert.SerializeObject always return {} in XamarinForms
Does anyone have any insight as to why this happens and when this will be resolved. Also a work around that I could implement so I can use debugging. Thanks.
{"meets": [
{
"VenueName": "O2"
},
{
"VenueName": "wembly"
},
{
"VenueName": "NEC"
} ],
"ErrorCode": "test"}
Your problem is that you have an object called "meets" in javascript, and in your object in C# you have "Meets". Change it to "Meets" and it should work.
You also don't need the JsonObject attribute, I believe. .NET should be able to handle the serialization from JSON automatically:
public class Meets
{
public string VenueName { get; set; }
}
public class RootObject
{
public List<Meets> Meets { get; set; }
public string ErrorCode { get; set; }
}
Spend some time looking at these resources to understand why your error is occurring:
https://learn.microsoft.com/en-us/dotnet/standard/serialization/
EDIT
The above classes produces the following json:
Meets:
{
"VenueName": null
}
Root Object:
{
"Meets": [
{
"VenueName": null
},
{
"VenueName": null
},
{
"VenueName": null
}
],
"ErrorCode": null
}
In one of my apps, I am getting the response from a webrequest. The service is Restful service and will return a result similar to the JSON format below:
{
"id" : "1lad07",
"text" : "test",
"url" : "http:\/\/twitpic.com\/1lacuz",
"width" : 220,
"height" : 84,
"size" : 8722,
"type" : "png",
"timestamp" : "Wed, 05 May 2010 16:11:48 +0000",
"user" : {
"id" : 12345,
"screen_name" : "twitpicuser"
}
}
and here is my current code:
byte[] bytes = Encoding.GetEncoding(contentEncoding).GetBytes(contents.ToString());
request.ContentLength = bytes.Length;
using (var requestStream = request.GetRequestStream()) {
requestStream.Write(bytes, 0, bytes.Length);
using (var twitpicResponse = (HttpWebResponse)request.GetResponse()) {
using (var reader = new StreamReader(twitpicResponse.GetResponseStream())) {
//What should I do here?
}
}
}
How can I read the response? I want the url and the username.
First you need an object
public class MyObject {
public string Id {get;set;}
public string Text {get;set;}
...
}
Then in here
using (var twitpicResponse = (HttpWebResponse)request.GetResponse()) {
using (var reader = new StreamReader(twitpicResponse.GetResponseStream())) {
JavaScriptSerializer js = new JavaScriptSerializer();
var objText = reader.ReadToEnd();
MyObject myojb = (MyObject)js.Deserialize(objText,typeof(MyObject));
}
}
I haven't tested with the hierarchical object you have, but this should give you access to the properties you want.
JavaScriptSerializer System.Web.Script.Serialization
I'd use RestSharp - https://github.com/restsharp/RestSharp
Create class to deserialize to:
public class MyObject {
public string Id { get; set; }
public string Text { get; set; }
...
}
And the code to get that object:
RestClient client = new RestClient("http://whatever.com");
RestRequest request = new RestRequest("path/to/object");
request.AddParameter("id", "123");
// The above code will make a request URL of
// "http://whatever.com/path/to/object?id=123"
// You can pick and choose what you need
var response = client.Execute<MyObject>(request);
MyObject obj = response.Data;
Check out http://restsharp.org/ to get started.
If you're getting source in Content
Use the following method
try
{
var response = restClient.Execute<List<EmpModel>>(restRequest);
var jsonContent = response.Content;
var data = JsonConvert.DeserializeObject<List<EmpModel>>(jsonContent);
foreach (EmpModel item in data)
{
listPassingData?.Add(item);
}
}
catch (Exception ex)
{
Console.WriteLine($"Data get mathod problem {ex} ");
}