We Keep Coding

Get List from JSON [duplicate]

This question already has answers here:
Parse JSON response where the object starts with a number in c#
(2 answers)
Closed 1 year ago.
I am trying to get the List from a JSON string from a 3rd party API.
I am not able to understand how should I parse it,as the key name starts with numeric.
I do not need the key but I do require value, but I require the value part in my code to be stored in DB.
JSON
{
"5MIN": [
{
"SETTLEMENTDATE": "2021-08-16T00:30:00",
"REGIONID": "NSW1",
"REGION": "NSW1",
"RRP": 39.27,
"TOTALDEMAND": 7416.02,
"PERIODTYPE": "ACTUAL",
"NETINTERCHANGE": -788.69,
"SCHEDULEDGENERATION": 5518.17,
"SEMISCHEDULEDGENERATION": 1076.47
},
{
"SETTLEMENTDATE": "2021-08-16T01:00:00",
"REGIONID": "NSW1",
"REGION": "NSW1",
"RRP": 36.51,
"TOTALDEMAND": 7288.89,
"PERIODTYPE": "ACTUAL",
"NETINTERCHANGE": -828.1,
"SCHEDULEDGENERATION": 5362.3,
"SEMISCHEDULEDGENERATION": 1064.35
}
]
}
I think I am over complicating the issue, but I am confused

As mentioned in the comments above, you can paste the json as code.
Next you add a reference to Newtonsoft.Json:
dotnet add package newtonsoft.json
You then call JsonConvert.Deserialize<T>() as in the example below:
using System;
using System.Collections.Generic;
using System.IO;
using Newtonsoft.Json;
using StackOverflow;
using System.Linq;
//In this example I load the JSON from disk
var json = File.ReadAllText("/home/timothy/data.json");
var record = JsonConvert.DeserializeObject<ServiceResponse>(json);
//No need to convert to List<T> if you're not going to filter it
var results = record.The5Min.ToList();
foreach(var item in results)
{
Console.WriteLine($"{item.Settlementdate}, {item.Regionid}");
}
namespace StackOverflow
{
using System;
using System.Collections.Generic;
using System.Globalization;
using Newtonsoft.Json;
using Newtonsoft.Json.Converters;
public partial class ServiceResponse
{
[JsonProperty("5MIN")]
public The5Min[] The5Min { get; set; }
}
public partial class The5Min
{
[JsonProperty("SETTLEMENTDATE")]
public DateTimeOffset Settlementdate { get; set; }
[JsonProperty("REGIONID")]
public string Regionid { get; set; }
[JsonProperty("REGION")]
public string Region { get; set; }
[JsonProperty("RRP")]
public double Rrp { get; set; }
[JsonProperty("TOTALDEMAND")]
public double Totaldemand { get; set; }
[JsonProperty("PERIODTYPE")]
public string Periodtype { get; set; }
[JsonProperty("NETINTERCHANGE")]
public double Netinterchange { get; set; }
[JsonProperty("SCHEDULEDGENERATION")]
public double Scheduledgeneration { get; set; }
[JsonProperty("SEMISCHEDULEDGENERATION")]
public double Semischeduledgeneration { get; set; }
}
}

Avoiding duplicates when inserting multiple data into MongoDB

Hi I want to insert multiple data and avoid duplicates in my programme. I have implemented the code to insert multiple data to mongoDB using asp.net core web api and it's working fine. I tried so many things but still I couldn't find a way to avoid duplicate records when inserting multiple records. Can you guys help me please :)
I want to avoid inserting duplicates by using employeeid.
this is my controller
using HelloApi.Models;
using HelloApi.Services;
using Microsoft.AspNetCore.Mvc;
using System.Collections.Generic;
using System.Threading.Tasks;
namespace HelloApi.Controllers
{
[Route("api/[controller]")]
[ApiController]
[Produces("application/json")]
public class EmployeeController : ControllerBase
{
private readonly EmployeeService _employeeService;
public EmployeeController(EmployeeService employeeService)
{
_employeeService = employeeService;
}
//.....................................Create..............................................
public Task Create(IEnumerable<Employee> employees)
{
return _employeeService.Create(employees);
}
}
}
This is my model class
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;
using System.ComponentModel.DataAnnotations;
namespace HelloApi.Models
{
public class Employee
{
[BsonId]
[BsonRepresentation(BsonType.ObjectId)]
[Key]
public string Id { get; set; }
[Required]
[BsonElement("employeeid")]
public string employeeid { get; set; }
[Required]
[BsonElement("firstname")]
public string firstname { get; set; }
[Required]
[BsonElement("lastname")]
public string lastname { get; set; }
[Required]
[BsonElement("age")]
public int age { get; set; }
[Required]
[BsonElement("address")]
public string address { get; set; }
[Required]
[BsonElement("telephone")]
public int telephone { get; set; }
}
}
This is my service class
using HelloApi.Models;
using MongoDB.Driver;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
namespace HelloApi.Services
{
public class EmployeeService
{
private readonly IMongoCollection<Employee> _employee;
public EmployeeService(IHelloApiDatabaseSettings settings)
{
var client = new MongoClient(settings.ConnectionString);
var database = client.GetDatabase(settings.DatabaseName);
_employee = database.GetCollection<Employee>(settings.employeeCollectionName);
}
//.....................................Create..............................................
public Task Create(IEnumerable<Employee> employees)
{
return _employee.InsertManyAsync(employees);
}
}
}
This is the POST request in Postman and it is working fine but submit duplicates.
Really appreciate if you guys can help me with this :)

This Might be helpful. Use below InsertORUpdateEmployee function to find and update using employeeid.
public Task<string> Create(IEnumerable<Employee> employees)
{
return Task.Run(() =>
{
return InsertORUpdateEmployee(employees);
});
}
//Insert or Update Employees
private string InsertORUpdateEmployee(IEnumerable<Employee> employees)
{
try
{
foreach (Employee emp in employees)
{
var empId = emp.employeeid;
var DB = Client.GetDatabase("Employee");
var collection = DB.GetCollection<Employee>("EmployeeDetails");
//Find Employee using employeeid
var filter_id = Builders<Employee>.Filter.Eq("employeeid", empId);
var entity = collection.Find(filter_id).FirstOrDefault();
//Insert
if (entity == null)
{
collection.InsertOne(emp);
}
else
{
//Update
var update = collection.FindOneAndUpdateAsync(filter_id, Builders<Employee>.Update
.Set("firstname", emp.firstname)
.Set("lastname", emp.lastname)
.Set("age", emp.age)
.Set("address", emp.address)
.Set("telephone", emp.telephone));
}
}
return "Insert or Updated Succesfully";
}
catch (Exception ex)
{
return ex.ToString();
}
}

I think you need to add an extra step to declare your Id field as a Primary Key. [Key] should only work for RDBMS. For MongoDB i found another post here to define a field as a unique key.
(Actually MongoDB doesn't really allow for a Primary Key besides the auto-generated '_id' field, afaik, but you can create unique keys. )
Try looking through the answers posted here:
How to set a primary key in MongoDB?

It seems you need to create unique index on employeeId field to avoid duplicates :
db.employee.createIndex( { "employeeId": 1 }, { unique: true } )

Import Json data from URL to SQL server

I have a .Json URL that contains Json format data.
HTTPS://XXXX/fetch-transaction?fromdate=2020-10-13&toDate=2020-10-20 (Not working just an example)
[
{
"orderId": 110,
"consignmentNumber": "TEST",
"itemNumbers": [
"TEST"
],
"country": "UK",
"orderType": "ORDER_HOME_DELIVERY",
"paymentTransactionId": "395611",
"priceInOre": 5900,
"paidAt": "2020-10-16 10:51:08",
"orderNumber": "7000067718",
"articleName": "SOUTH-2"
}
]
I would like to insert data into a SQL server table and wonder if it's possible to use SQL server and t-SQL directly here or should I go for VS and C#?
If C# is the preferred choice can someone pleae guide me on how I would accomplish it?
I have created a Console application in Visual studio (however it might be a better solution to use something els then to create a command line applcation?) or guide me in the right direction.

You can try the following code to get the value from the json txt and transfer it to the sql server table.
using Newtonsoft.Json;
using System.Collections.Generic;
using System.Data.SqlClient;
using System.IO;
using System.Linq;
class Program
{
static void Main(string[] args)
{
string json = File.ReadAllText("D:\\test1.txt");
List<Example> list = JsonConvert.DeserializeObject<List<Example>>(json);
string strcon = #"Connstr";
SqlConnection connection = new SqlConnection(strcon);
connection.Open();
string sql = "Insert into JsonData(orderId,consignmentNumber,itemNumbers,country,orderType,paymentTransactionId,priceInOre,paidAt,orderNumber,articleName) values(#orderId,#consignmentNumber,#itemNumbers,#country,#orderType,#paymentTransactionId,#priceInOre,#paidAt,#orderNumber,#articleName)";
SqlCommand command = new SqlCommand(sql, connection);
foreach (Example item in list)
{
command.Parameters.AddWithValue("#orderId", item.orderId);
command.Parameters.AddWithValue("#consignmentNumber", item.consignmentNumber);
command.Parameters.AddWithValue("#itemNumbers", item.itemNumbers.First());
command.Parameters.AddWithValue("#country", item.country);
command.Parameters.AddWithValue("#orderType", item.orderType);
command.Parameters.AddWithValue("#paidAt", item.paidAt);
command.Parameters.AddWithValue("#paymentTransactionId", item.paymentTransactionId);
command.Parameters.AddWithValue("#priceInOre", item.priceInOre);
command.Parameters.AddWithValue("#articleName", item.articleName);
command.Parameters.AddWithValue("#orderNumber", item.orderNumber);
}
command.ExecuteNonQuery();
connection.Close();
}
}
public class Example
{
public int orderId { get; set; }
public string consignmentNumber { get; set; }
public List<string> itemNumbers { get; set; }
public string country { get; set; }
public string orderType { get; set; }
public string paymentTransactionId { get; set; }
public int priceInOre { get; set; }
public string paidAt { get; set; }
public string orderNumber { get; set; }
public string articleName { get; set; }
}
Final result:
Edit:
var json = new WebClient().DownloadString("URL");

Raw queries with overridden column names

I'm trying to retrieve some entities using Entity Framework by querying an XML column. Entity Framework doesn't support this so I had to use raw SQL.
var people = context.People.SqlQuery("SELECT * FROM [People] WHERE [DataXML].value('Properties/Age', 'int') = 21").AsQueryable().AsNoTracking();
My person class:
public class Person
{
public int Id { get; set; }
public string Name { get; set; }
[Column("YearsSinceBirth")]
public int Age { get; set; }
[Column(TypeName = "xml")]
public string DataXML { get; set; }
}
This should work, however, it falls over when trying to map it back to an object. Specifically, it's falling over on the Age property, which has it's column name overridden to "YearsSinceBirth".
'The data reader is incompatible with the specified
'MyProject.CodeBase.DataModel.DbEntities.Person'. A member of the
type, 'Age', does not have a corresponding column in the data reader
with the same name.'
I'm guessing that Entity Framework doesn't map database column names to object property names and therefore is expecting the column to be named 'Age' rather than 'YearsSinceBirth'.
I don't want to have to list each column and their mapping in the SQL query (like SELECT YearsSinceBirth As Age) as the actual project I'm working on which has this column has a lot more columns and that would mean this query would break every time the schema changed (kinda defeating the purpose of Entity Framework).

If this is EF Core, your problem is not that SqlQuery() doesn't support mapping column names (it does). Rather your problem is that your table doesn't contain a column called YearsSinceBirth, and you are returning 'select *'.
If you have a column called YearsSinceBirth, this works fine. Although you will be retrieving the value in the YearsSinceBirth column, not the value in the XML document. EG
using Microsoft.EntityFrameworkCore;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
//using Microsoft.Samples.EFLogging;
using System.ComponentModel.DataAnnotations.Schema;
using System.ComponentModel.DataAnnotations;
using System.Data.SqlClient;
namespace EFCore2Test
{
public class Person
{
public int Id { get; set; }
public string Name { get; set; }
[Column("YearsSinceBirth")]
public int Age { get; set; }
[Column(TypeName = "xml")]
public string DataXML { get; set; }
}
public class Location
{
public string LocationId { get; set; }
}
public class Db : DbContext
{
public DbSet<Person> People { get; set; }
public DbSet<Location> Locations { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer("Server=(local);Database=EFCoreTest;Trusted_Connection=True;MultipleActiveResultSets=true");
base.OnConfiguring(optionsBuilder);
}
}
class Program
{
static void Main(string[] args)
{
using (var db = new Db())
{
db.Database.EnsureDeleted();
//db.ConfigureLogging(s => Console.WriteLine(s));
db.Database.EnsureCreated();
var p = new Person()
{
Name = "joe",
Age = 2,
DataXML = "<Properties><Age>21</Age></Properties>"
};
db.People.Add(p);
db.SaveChanges();
}
using (var db = new Db())
{
var people = db.People.FromSql("SELECT * FROM [People] WHERE [DataXML].value('(/Properties/Age)[1]', 'int') = 21").AsNoTracking().ToList() ;
Console.WriteLine(people.First().Age);
Console.ReadLine();
}
Console.WriteLine("Hit any key to exit");
Console.ReadKey();
}
}
}
You can use a pattern similar to this to project entity attributes from an XML or JSON column:
public class Person
{
private XDocument xml;
public int Id { get; set; }
public string Name { get; set; }
[NotMapped]
public int Age
{
get
{
return int.Parse(xml.Element("Properties").Element("Age").Value);
}
set
{
xml.Element("Properties").Element("Age").Value = value.ToString();
}
}
[Column(TypeName = "xml")]
public string DataXML
{
get
{
return xml.ToString();
}
set
{
xml = XDocument.Parse(value);
}
}
}

You can dynamically create select query with aliases, if they needed, with the help of reflection and ColumnAttribute checking:
public string SelectQuery<T>() where T : class
{
var selectQuery = new List<string>();
foreach (var prop in typeof(T).GetProperties(BindingFlags.Public | BindingFlags.Instance))
{
var attr = prop.GetAttribute<ColumnAttribute>();
selectQuery.Add(attr != null ? $"{attr.Name} as {prop.Name}" : prop.Name);
}
return string.Join(", ", selectQuery);
}
Usage:
var people = context.People.SqlQuery($"SELECT {SelectQuery<Person>()} FROM [People] WHERE [DataXML].value('Properties/Age', 'int') = 21")
.AsQueryable().AsNoTracking();

Entity Framework 5 Won't Fetch Relationships With Include()

I am quite certain that questions like this have been answered a number of times before, but I can't get any of the suggestions to work.
I am building a MVC 4 application with Entity Framework 5, where the entities were generated from existing tables. I have entity classes that look like this:
namespace RebuildingModel
{
using System;
using System.Collections.Generic;
public partial class StandardCodeTable
{
public StandardCodeTable()
{
this.StandardCodeTableTexts = new HashSet<StandardCodeTableText>();
}
public int TableCode { get; set; }
public string RefTableName { get; set; }
public virtual ICollection<StandardCodeTableText> StandardCodeTableTexts { get; set; }
}
}
namespace RebuildingModel
{
using System;
using System.Collections.Generic;
public partial class StandardCodeTableText
{
public int TableCode { get; set; }
public string LanguageCode { get; set; }
public string TextVal { get; set; }
public virtual StandardCodeTable StandardCodeTable { get; set; }
}
}
namespace RebuildingSite.Models
{
public class CodeTableJoined
{
public int TableCode { get; set; }
public string ReferenceTableName { get; set; }
public string LanguageCode { get; set; }
public string TextValue { get; set; }
}
}
I have a DAO that looks like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace RebuildingModel.Dao
{
public class CodeTableDao
{
public CodeTableDao() { }
public ISet<StandardCodeTableText> GetCode(string refTableName)
{
HashSet<StandardCodeTableText> codes = new HashSet<StandardCodeTableText>();
using (var db = new RebuildingTogetherEntities())
{
db.StandardCodeTableTexts.Include("StandardCodeTables");
var query = from c in db.StandardCodeTableTexts
where c.StandardCodeTable.RefTableName == refTableName
orderby c.TableCode
select c;
foreach (var item in query)
{
codes.Add(item);
}
}
return codes;
}
}
I have a controller that looks like this:
namespace RebuildingSite.Controllers
{
public class CodeTableController : Controller
{
public ActionResult Index(string refTableName)
{
CodeTableDao dao = new CodeTableDao();
ICollection<StandardCodeTableText> codes = dao.GetCode(refTableName);
HashSet<CodeTableJoined> joins = new HashSet<CodeTableJoined>();
foreach (var code in codes)
{
CodeTableJoined join = new CodeTableJoined();
join.TableCode = code.TableCode;
join.LanguageCode = code.LanguageCode;
join.TextValue = code.TextVal;
join.ReferenceTableName = code.StandardCodeTable.RefTableName;
joins.Add(join);
}
ISet<string> refTableNames = dao.GetReferenceTables();
ViewBag.RefTableNames = refTableNames;
return View(joins);
}
}
}
When I run the view attached to the controller, an ObjectDisposedException is thrown at this line, where the relationship is used:
join.ReferenceTableName = code.StandardCodeTable.RefTableName;
This has to be something simple. What am I doing wrong? I have tried adding that Include() call in from the context in many different places, even multiple times.
I've also tried adding an explicit join in the Linq query. I can't get EF to fetch that relationship.

Copying my comment to an answer - Put the include be in the actual query
var query = from c in
db.StandardCodeTableTexts.include("StandardCodeTables"). where
c.StandardCodeTable.RefTableName == refTableName orderby c.TableCode
select c;