We Keep Coding

Unsupported Media Type when consuming text/plain

I receive the following response when trying to consume text/plain:
{
"type": "https://tools.ietf.org/html/rfc7231#section-6.5.13",
"title": "Unsupported Media Type",
"status": 415,
"traceId": "|b28d0526-4ca38d2ff7036964."
}
Controller definition:
[HttpPost]
[Consumes("text/plain")]
public async Task<IActionResult> PostTrace([FromBody]string body)
{ ... }
HTTP message:
POST /api/traces HTTP/1.1
Content-Type: text/plain
User-Agent: PostmanRuntime/7.19.0
Accept: */*
Cache-Control: no-cache
Postman-Token: 37d27eb6-92a0-4a6a-8b39-adf2c93955ee
Host: 0.0.0.0:6677
Accept-Encoding: gzip, deflate
Content-Length: 3
Connection: keep-alive
I am able to consume JSON or XML just fine. What am I missing?

Reference: Accepting Raw Request Body Content in ASP.NET Core API Controllers:
Unfortunately ASP.NET Core doesn't let you just capture 'raw' data in any meaningful way just by way of method parameters. One way or another you need to do some custom processing of the Request.Body to get the raw data out and then deserialize it.
You can capture the raw Request.Body and read the raw buffer out of that which is pretty straight forward.
The easiest and least intrusive, but not so obvious way to do this is to have a method that accepts POST or PUT data without parameters and then read the raw data from Request.Body:
[HttpPost]
[Route("api/BodyTypes/ReadStringDataManual")]
public async Task<string> ReadStringDataManual()
{
using (StreamReader reader = new StreamReader(Request.Body, Encoding.UTF8))
{
return await reader.ReadToEndAsync();
}
}
Request:
POST http://localhost:5000/api/BodyTypes/ReadStringDataManual HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/plain
Host: localhost:5000
Content-Length: 37
Expect: 100-continue
Connection: Keep-Alive
Windy Rivers with Waves are the best!

Requesting to github-api using .net HttpClient says Forbidden

I am trying to getting data from this github api link https://api.github.com/users/arif2009 . It works fine using posman get request.
But if i try to get data using .net HttpClient then it says Forbidden.
C# code:
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("https://api.github.com/");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var response = await client.GetAsync("users/arif2009");
if (response.IsSuccessStatusCode)
{
var data = response.Content.ReadAsAsync<GithubUser>();
}
}
Response:
Can anybody tell me where i made the mistake?

I took a look at your requests using Telerik's Fiddler and found the following.
Request with your code:
GET https://api.github.com/users/arif2009 HTTP/1.1
Accept: application/json
Host: api.github.com
Request from Postman:
GET https://api.github.com/users/arif2009 HTTP/1.1
Host: api.github.com
Connection: keep-alive
Accept: application/json
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
Postman-Token: d560ee28-b1e8-ece5-2612-87371ddcb295
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,ja-JP;q=0.8,ja;q=0.7,en-US;q=0.6
The obvious missing header seemed to be "User-Agent", so I added this:
client.DefaultRequestHeaders.UserAgent.Add(new ProductInfoHeaderValue("product", "1")); // set your own values here
Which produced the following request:
GET https://api.github.com/users/arif2009 HTTP/1.1
Accept: application/json
User-Agent: product/1
Host: api.github.com
And returned the following response:
HTTP/1.1 200 OK
Date: Wed, 17 Jul 2019 15:19:35 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 1249
Server: GitHub.com
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 52
X-RateLimit-Reset: 1563380375
Cache-Control: public, max-age=60, s-maxage=60
Vary: Accept
ETag: "1df3e0be6e824ca684f27963806533da"
Last-Modified: Tue, 16 Jul 2019 05:58:59 GMT
X-GitHub-Media-Type: github.v3
Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
Vary: Accept-Encoding
X-GitHub-Request-Id: D5E0:8D63:8BE92C:A799AE:5D2F3C86
{"login":"arif2009","id":6396346,"node_id":"MDQ6VXNlcjYzOTYzNDY=","avatar_url":"https://avatars0.githubusercontent.com/u/6396346?v=4","gravatar_id":"","url":"https://api.github.com/users/arif2009","html_url":"https://github.com/arif2009","followers_url":"https://api.github.com/users/arif2009/followers","following_url":"https://api.github.com/users/arif2009/following{/other_user}","gists_url":"https://api.github.com/users/arif2009/gists{/gist_id}","starred_url":"https://api.github.com/users/arif2009/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arif2009/subscriptions","organizations_url":"https://api.github.com/users/arif2009/orgs","repos_url":"https://api.github.com/users/arif2009/repos","events_url":"https://api.github.com/users/arif2009/events{/privacy}","received_events_url":"https://api.github.com/users/arif2009/received_events","type":"User","site_admin":false,"name":"Arif","company":"#BrainStation-23 ","blog":"https://arif2009.github.io/","location":"Bangladesh","email":null,"hireable":true,"bio":"Software Engineer | Full Stack | Web Developer | Technical Writer","public_repos":15,"public_gists":2,"followers":9,"following":7,"created_at":"2014-01-14T05:03:47Z","updated_at":"2019-07-16T05:58:59Z"}

this API reference from github says that 'User-Agent' is a required header:
All API requests MUST include a valid User-Agent header. Requests with no User-Agent header will be rejected.
Postman automatically adds its own User-Agent to the call when this is not provided by the user. (This is nicely demonstrated by #John's answer).
simply adding this header will resolve your issue:
client.DefaultRequestHeaders.UserAgent.Add(new ProductInfoHeaderValue("yourAppName", "yourVersionNumber"));

You may try to put the complete URI into the GetAsync instead using the BaseAddress, as you do not give the GetAsync an URI, but only a string.
When sending a HttpRequestMessage with a relative Uri, the message Uri will be added to the BaseAddress property to create an absolute Uri.
https://learn.microsoft.com/de-de/dotnet/api/system.net.http.httpclient.baseaddress?view=netframework-4.8

Content type in call to web service

Please correct me if I'm wrong in defining contet type:
Content-Type: application/json; charset=utf-8
{"my_id":"975","my_a_id":"Test66","param":"4","amount":"66"}
and
Content-Type: application/x-www-form-urlencoded
my_id=123&my_a_id=Test66&param=4&amount=66
What other content types might be?
What content type supports RESTLet NetSuite services?
I have POST to web service:
POST https://some.netsuite.uri?deploy=1 HTTP/1.1
Authorization: NLAuth nlauth_account=3624292_SB3, nlauth_email=xduh31#gmail.com, nlauth_signature=Pass987654, nlauth_role=3
Accept: application/json
Content-Type: application/json; charset=utf-8
Host: rest.eu1.netsuite.com
Cookie: JSESSIONID=**************************************; lastUser=1234567_SB9_1282_3; NS_ROUTING_VERSION=LAGGING; NS_VER=2017.1.0
Content-Length: 69
Expect: 100-continue
{"my_id":"975","my_a_id":"Test66","param":"4","amount":"66"}
Is it good that in URL I have something like x-www-form-urlencoded - ?deploy=1
https://some.netsuite.uri?deploy=1
and data is json style:
{"my_id":"975","my_a_id":"Test66","param":"4","amount":"66"}
UPD
I got answer that I suppose should return JSON data:
HTTP/1.1 200 OK
Date: Sat, 03 Jun 2017 06:21:46 GMT
Server: Apache
Cache-Control: No-Cache
Pragma: No-Cache
Content-Length: 41
Expires: 0
Edge-Control: no-store
X-N-OperationId: 486c2d20-099d-446b-9788-4816db59a1fd
Set-Cookie: .......................; path=/
NS_RTIMER_COMPOSITE: 1688996695:706172746E6572733030312E70726F642E6475622E6E65746C65646765722E636F6D:80
P3P: CP="CAO PSAa OUR BUS PUR"
Vary: User-Agent
Content-Type: application/json; charset=utf-8
org.mozilla.javascript.Undefined#54b896b0
But this answer not looks like JSON?

One problem might be your URL. Have you tried changing
https://some.netsuite.uri&deploy=1
to
https://some.netsuite.uri?deploy=1
Edit:
Also, your Accept header says you are looking for an application/json response, is that what the server is returning?

You can find supported content types in NS Help Center:
RESTlets support JSON and plain text content types for input and
output. For each RESTlet, output content type is the same as input
content type.
You must set the content type in the HTTP Content-Type header. You can
use the following values to specify the input/output content type for
a RESTlet:
application/json
text/plain
If you specify a content type other than JSON or text, a 415 error is
returned with the following message:
Invalid content type. You can only use application/json or text/plain
with RESTlets.
Documentation
Besides that the issue in the url mentioned by shazmodan, the script Id is missing. Below is an example of a valid restlet url:
https://rest.netsuite.com/app/site/hosting/restlet.nl?deploy=1&script=1
You can pass additional parameters in the url:
https://rest.netsuite.com/app/site/hosting/restlet.nl?deploy=1&script=1&my_id=123&my_a_id=Test66&param=4&amount=66
or in the request body if you are using POST and PUT methods but it needs to be a valid JSON. But the Content Type must always be application/json or text/plain.

How to read and process a HTTP POST

I have a HTTP handler which receives a HTTP POST. HTTP POST message with header is -
POST /ibe/example.ashx HTTP/1.1
Content-Length: 41
Content-Type: application/x-www-form-urlencoded; text/html; charset=GBK
Host: 202.177.46.142
User-Agent: Mozilla/4.0
param1=value1&param2=value2&param3=value3
Handler in my code is -
var V1 = context.Request["param1"];
var V2 = context.Request["param2"];
But values returning are null

if it's a post I think you'd want to use a stream reader to get the body, then deserialize if needed.

Parsing from web - script changes source code content?

I have been trying to parse a php generated web page(not site) for some time. I tried parsing using xpath through HTMLAgility in C# as well as PHP. At first I thought I was not parsing correctly due to incorrect values.
Later, I found that actually I am parsing it correctly. But there is a script in that page which is changing the value when loading. how, I don't know.
I am new to parsing, so here is what happening according to me:
I am downloading the source code of the content. The part I want to parse is somewhat like this:
<b id="solved_b">0</b>
When the page loads, the script in the source code changes the value to something other than 0.
When I parse using xpath, the original value, i.e. 0 is parsed, instead of the script changed value.
So, how can I parse the changed value instead of the original one?
the page I am trying to parse is
http://felix-halim.net/uva/hunting.php?id=59756
here is the snippet in HTMLAgility:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using HtmlAgilityPack;
namespace ParseFelix {
class WebParser {
string webUrl;
public WebParser(string url) {
webUrl = "http://felix-halim.net/uva/hunting.php?id=59756";
HtmlWeb htmlWeb = new HtmlWeb();
HtmlDocument htmldoc = htmlWeb.Load(webUrl);
var username = htmldoc.DocumentNode.SelectSingleNode("/html/body/div/h2/i");
var submittedStats = htmldoc.DocumentNode.SelectSingleNode(".//*[#id=\"submissions_b\"]");
string content = htmldoc.DocumentNode.InnerHtml;
//System.IO.File.WriteAllText("D:\\exp\\felix\\parsed.txt", content);
var acceptedStats = htmldoc.DocumentNode.SelectSingleNode(".//*[#id=\"solved_b\"]");
Console.WriteLine("Username is {0}, you submitted {1} solutions, and {2} were accepted", username.InnerText, submittedStats.InnerText, acceptedStats.InnerText);
}
}
}

Well what you are trying to do is parse javascript right?
And that is AFAIK not possible (aside from writing your own parser or using an existing one).
What you want is to read the manipulated DOM, and that is not trivial at all

Use Fiddler - you will see that site makes ajax queries and some values gets from json:
POST http://felix-halim.net/uva/service2.php HTTP/1.1
Host: felix-halim.net
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:2.0) Gecko/20100101 Firefox/4.0
Accept: */*
Accept-Language: lt
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Proxy-Connection: keep-alive
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: http://felix-halim.net/uva/hunting.php?id=59756
Content-Length: 73
Cookie: PHPSESSID=o6if4t4vqadv7ia6vbqcfcvi75
Pragma: no-cache
Cache-Control: no-cache
{"method":"uva2.chat_update","params":[12150,59756,"guest",3127,8744317]}
And response:
HTTP/1.0 200 OK
Date: Fri, 15 Apr 2011 05:32:08 GMT
Server: Apache
X-Powered-By: PHP/5.2.15
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Content-Type: text/html
Content-Length: 2069
[null,[["0","guest","1302842169"],["0","guest","1302793161"]<SKIPPED>