We Keep Coding

C# API call from MarketStack and print values - Error 403

I would like to make a successful API call, then print the values in order to see if it works. My main goal is to analyze the data, after I can make a successful API call, and build a systematic strategy for trading.
System.Net.Http.HttpRequestException: "Response status code does not indicate success: 403 (Forbidden)
namespace marketstacktest
{
class Program
{
static async Task Main(string[] args)
{
Console.WriteLine("Hello World!");
var options = Options.Create(new MarketstackOptions() { ApiToken = "secretTokenHere" });
var marketstackService = new MarketstackService(options, NullLogger<MarketstackService>.Instance);
var appleSymbol = "AAPL";
var fromDate = DateTime.Now.AddDays(-200);
var toDate = DateTime.Now;
//error at the await System.Net.Http.HttpRequestException: "Response status code does not indicate success: 403 (Forbidden)."
List<Marketstack.Entities.Stocks.StockBar> stock = await marketstackService.GetStockEodBars(appleSymbol, fromDate, toDate);
foreach (var stock_i in stock)
{
Console.WriteLine($"close: {stock_i.Close}");
}
}
}
}

In the API manual, which is directly linked from the github, it gives information about all of the error codes. The relevant ones here are these two:
Code
Type
Description
403
https_access_restricted
HTTPS access is not supported on the current subscription plan.
403
function_access_restricted
The given API endpoint is not supported on the current subscription plan.
Their class library on github is just wrapping a json REST api. Every call to the API is just an http request, returning data as json objects. The 403 error indicates that your request was accepted as a valid request, but intentionally rejected by the server for some reason. And according to the docs, the error was because your account is not allowed access to either https or to the type of request.
Their free-tier subscription only includes end-of-day data, which is what you requested, so it wouldn't make sense for that not to be allowed. So, your app is almost certainly making an https call.
I went to the examples at the very beginning of their quick start guide, and chose the end-of-day example to match your app, and clicked on the link. It worked, and gave a bunch of json records. But, the request they made was using 'http' not 'https'.
Changing the requst to 'https' elicited a 403 response with this content (formatted for readability):
{
"error":
{
"code": "https_access_restricted",
"message": "Access Restricted - Your current Subscription Plan does not support HTTPS Encryption."
}
}
At this point we have enough to be almost certain that this is your issue. The final thing is to go look up how to turn https requests off in their class library. To avoid having to go through the code, I checked the help at the bottom of the page one more time, and found this (formatted for readability):
var options = Options.Create(new MarketstackOptions(){
ApiToken = apiKey,
MaxRequestsPerSecond = 3,
Https = true
});
Welp. This should probably be in their first example, since that's what people are most likely to try first, but it's not. So, to stop trying to make http requests, you just need to set the Https option to false in your code. You just need to add that to the options in your code, like so:
var options = Options.Create(new MarketstackOptions(){
ApiToken = "secretTokenHere",
Https = false
});
I will leave the testing to you, but from the browser test, we know that the request should work, unless there's a bug in their library. Given the information that was available, this is almost certainly the issue.

How to handle long time request in Angular 4?

I am having thouble with a request, in Angular 4, that takes really long.
The backend is in .Net Core 2.0 that connects to an Oracle DB. It should wait for a long query to run in the database to get the data and send it to the client. However, it seems that the client is unable to wait until the end of the process (the return of all records), what I get is the error:
Failed to load resource: the server responded with a status of 502
(Bad Gateway)
This error occurs when a CGI application does not return a valid set
of HTTP headers, or when a proxy or gateway was unable to send the
request to a parent gateway. You may need to get a network trace or
contact the proxy server administrator, if it is not a CGI problem.
It is not a Timeout error, as I thought it'd be.
Here is the code of my request:
exportExcel(dadosConsulta: DadosConsultaModel) : Promise<any>{
let url ="/api/Relatorios/exportToExcel";
return this.http
.post(url, JSON.stringify(dadosConsulta),{ headers: this.headers })
.timeout(500000)
.toPromise()
.then(data => data.json())
.catch(this.handleError);
}
How can I prevent this from happening?

I worked with oracle, long time request querys, and observables and didn't have that problem.
Something like this
login(account: string, password: string, user: string): Observable <any> {
let body = new URLSearchParams();
body.set('account', account);
body.set('password', password);
body.set('user', user);
return this.http.post(this.globalVars.apiUrl + 'login', body)
.map((res: any) => {
let result = res.json().data.result;
let data = res.json().data;
this.client.auth = {
authCode: result.authCode,
token: result.token,
};
this.firstLogin = data.firstLogin;
return this.client;
})
.catch((error: any) => {
console.log('error', error._body);
return Observable.throw(error._body);
})
.timeoutWith(25000, Observable.throw(new Error('Connection Error')))
}}

The problem was actually coming from IIS, and to solve this I had to add a web.config to my project (which is not created by default) and modify the 'aspNetCore' tag like this:
<aspNetCore **requestTimeout="00:20:00"** processPath="%LAUNCHER_PATH%" arguments="%LAUNCHER_ARGS%" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout" forwardWindowsAuthToken="false"/>
Got it from this post: Timeouts with long running ASP.NET MVC Core Controller HTTPPost Method

HTTP POST not working in OWIN Self-Host Web API

I am self self hosting a Web API. Any Get Reqeust I make from my integration tests works fine. However any POST request throws connection refused. I can't seem to get a handle on what is happening.
Error Message
system.Net.HttpRequestException: An error occured while sending the request to the remote server. SocketException: no connection could be made because the target machine actively refused.
Code
using (WebApp.Start<App_Start.TestConfiguration>("http:/localhost:8216"))
{
var client = new HttpClient();
client.BaseAddress = new System.Uri("http://127.0.0.1:8216");
var response = await client.PostAsync("/api/MyController", new StringContent("something"));
}
controller
public string Post(string value)
{
return "Hello!";
}

I've got the same issue and found that it is necessary to use [FromBody] attribute before your method params. In this case it can parse request payload and you can reach your methods
Hope that helps

Could it be you're missing a / on:
using (WebApp.Start<App_Start.TestConfiguration>("http:/localhost:8216"))
it should be http://localhost:8216.

Why is my Windows CE device unable to successfully invoke my server's REST methods?

This is related/sort of a followup to this question but contains more and more specific information.
I have a REST app running on my PC. I need to call some of the methods on the server from a handheld WindowsCE device that uses the compact framework.
I can contact the server's methods from Postman, so it's not a problem that way. In Postman, I use:
http://localhost:21609/api/inventory/sendXML/duckbill/platypus/bloo
...and with this I reach the breakpoint in this REST method on my server app:
[HttpPost]
[Route("api/inventory/sendxml/{userId}/{pwd}/{filename}")]
public async Task SendInventoryXML(String userId, String pwd, String fileName)
However, although using "PPP_PEER" is the way to contact the PC from the handheld device (127.0.0.1 doesn't fly, because the handheld sees that as being itself, and commits egregious acts of cannibalism when trying to contact that address) - as can be seen by this TCP code that does work from the handheld device:
string pingString = "PING|";
TcpClient client = new TcpClient("PPP_PEER", 7727);
try
{
try
{
Stream s = client.GetStream();
StreamReader sr = new StreamReader(s);
StreamWriter sw = new StreamWriter(s) { AutoFlush = true };
String response = String.Empty;
if (firstRecord)
{
sw.WriteLine(pingString);
. . .
...trying to use PPP_PEER to call the REST method fails.
Here is the code I use to try to do that:
//HHSConsts
public static string BASE_REST_URL = "http://PPP_PEER:21609/api/";
. . .
//frmMain
private void SendInventories()
{
try
{
foreach (String tblname in listBoxWork.Items)
{
String xmlData = hhsdbutils.GetINVDataAsXMLFromTable(tblname, fileName);
String uri = String.Format("{0}inventory/sendXML/duckbill/platypus/{1}",
HHSConsts.BASE_REST_URL, fileName);
fileXferImp = HHSConsts.GetFileTransferMethodology();
fileXferImp.SendDataContentsAsXML(uri, xmlData, tblname);
. . .
// FileXferREST.cs
public void SendDataContentsAsXML(String destinationPath, String data, String fileName,
String siteNumber, bool firstRecord, bool lastRecord)
{
SendHTTPRequestNoCredentials(destinationPath, HttpMethods.POST, data, "application/xml");
}
public static HttpWebRequest SendHTTPRequestNoCredentials(string uri, HttpMethods method,
string data, string contentType)
{
WebRequest request = WebRequest.Create(uri);
try
{
request.Method = Enum.ToObject(typeof(HttpMethods), method).ToString();
request.ContentType = contentType;
((HttpWebRequest)request).Accept = contentType;
((HttpWebRequest)request).KeepAlive = false;
((HttpWebRequest)request).ProtocolVersion = HttpVersion.Version10;
if (method != HttpMethods.GET && method != HttpMethods.DELETE)
{
byte[] arrData = Encoding.UTF8.GetBytes(data);
request.ContentLength = arrData.Length;
using (Stream oS = request.GetRequestStream())
{
oS.Write(arrData, 0, arrData.Length);
}
}
else
{
request.ContentLength = 0;
}
}
catch (WebException webex)
{
HttpWebResponse hwr = (HttpWebResponse)webex.Response;
HttpStatusCode hsc = hwr.StatusCode;
String webExMsgAndStatusCode = String.Format("{0} Status code == {1}", webex.Message,
hsc.ToString());
ExceptionLoggingService.Instance.WriteLog(String.Format("From
FileXferREST.SendHTTPRequestNoCredentials: {0}", webExMsgAndStatusCode));
}
return request as HttpWebRequest;
}
No exception is thrown; it simply doens't work.
When attempting this call, I run rawcap to capture the packages being sent over the network with these command line args:
rawcap 127.0.0.1 [fileName].pcap
I then open the .pcap file in Wireshark, search for "21609" and get this TCP stream:
...I added the screen shot above to show the red/blue coloration for the request/response, but here is the entire rawcap/Wireshark conversation involving port 21609:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
POST /api//inventory/sendXML/duckbill/platypus/INV_3_20090307181658000.xml HTTP/1.0
Content-Type: application/xml
Accept: application/xml
Connection: Close
Content-Length: 388
Host: ppp_peer:21609
112209003343742SOME DESC2.2testVendorID]6161.51.995.58HTTP/1.1 404 Not Found
Cache-Control: private
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/8.0
X-SourceFiles: =?UTF-8?B?QzpccHJvamVjdFxnaXRcQ1N0b3JlXEhIUy5BUElcYXBpXGludmVudG9yeVxzZW5kWE1MXGR1Y2tiaWxsXHBsYXR5cHVzXElOVl8zXzIwMDkwMzA3MTgxNjU4MDAwLnhtbA==?=
X-Powered-By: ASP.NET
Date: Thu, 18 Dec 2014 17:23:01 GMT
Connection: close
Content-Length: 5016
IIS 8.0 Detailed Error - 404.0 - Not Found
HTTP Error 404.0 - Not Found
The resource you are looking for has been removed, had its name changed, or is temporarily unavailable.
Most likely causes:
.The directory or file specified does not exist on the Web server. .The URL contains a typographical error. .A custom filter or module, such as URLScan, restricts access to the file.
Things you can try:
.Create the content on the Web server. .Review the browser URL. .Check the failed request tracing log and see which module is calling SetStatus. For more information, click here.
Detailed Error Information:
Module IIS Web Core
Notification MapRequestHandler
Handler StaticFile
Error Code 0x80070002
Requested URL http://ppp_peer:21609/api/inventory/sendXML/duckbill/platypus/INV_3_20090307181658000.xml
Physical Path C:\project\git\CStore\HHS.API\api\inventory\sendXML\duckbill\platypus\INV_3_20090307181658000.xml
Logon Method Anonymous
Logon User Anonymous
Request Tracing Directory C:\Users\clay\Documents\IISExpress\TraceLogFiles\HHS.API
More Information:
This error means that the file or directory does not exist on the server. Create the file or directory and try the request again.
View more information »
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
So to recap, "PPP_PEER" does work as a way for the handheld device to describe/contact the PC (it works with the TCP code), and the REST method is reachable from another process (such as Postman), but trying to call the method using PPP_PEER from the handheld device fail with the "404.0 - Not Found" error.
Why?
UPDATE
When I replace "PPP_PEER" with the machine name of the PC, the error changes from "404 - Not Found" to "503 - Service Unavailable":
But that is a bit of a rompecabeza, because the service is obviously available, as it is running and its breakpoint is hit when I call it from Postman.
UPDATE 2
Now this is odd: If I use the full machine name instead of just the "truncated" name, the error goes back to being 404 instead of 503:
So if I use PPP_PEER as the host name, I get a 404; if I use shannon2, I get a 503; if I use shannon2.sscs.ad, I again get 404. Should I blame this on Tim Berners-Lee, Al Gore, Andy Warhol, or someone else?
SO...those are my results on trying to follow Eric Law's first suggestion; there are two others, but: what would I edit the Host header to that might make a difference? Or overwrite it to what?

It seems as if the basic problem is trying to access a "server" app running locally on my PC (even though the handheld != the PC, it is "sort of" the same thing, after a fashion).
Now that the server app is running on a different machine on the network altogether, I'm able to hit it; albeit not yet totally unexceptionally, as can be seen here.

400 bad request invalid hostname only in android application

I have a asp.net mvc website deployed on a server, providing a few web interfaces to others. For example, getting the current user's information, my test C# console application looks like this:
using (var client = new WebClient())
{
try
{
var url = "http://api.fake.mysite.com/v1.0/user/current";
var token = "e0034e1c082de62b74e361b15f9c6471";
var encoded = Convert.ToBase64String(Encoding.UTF8.GetBytes(token));
client.Headers["Authorization"] = encoded;
client.Headers["Content-Type"] = "application/json";
Console.WriteLine(client.DownloadString(url));
}
catch (WebException e)
{
//log the exception
}
}
You can see the usage is pretty simple, just request the url via HTTP_GET, set the Authorization header to the encoded token. Actually it works fine in my machine. But some one else meets a strange issue when visiting this url in an android application, here is the java code:
HttpClient httpClient = new DefaultHttpClient();
String token = "e0034e1c082de62b74e361b15f9c6471";
String url = "http://api.fake.mysite.com/v1.0/user/current";
HttpGet httpGet = new HttpGet(url);
String encoded = Base64.encodeToString(token.getBytes(), Base64.DEFAULT);
httpGet.addHeader("Authorization", encoded);
httpGet.addHeader("Content-Type", "application/json");
try {
HttpResponse httpResponse = httpClient.execute(httpGet);
int responseCode = httpResponse.getStatusLine().getStatusCode();
String response = EntityUtils.toString(httpResponse.getEntity());
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
then he got "400 bad request invalid host name" error. I've tried:
(1) make sure the variable "encoded" has the same value in C# and Java code.
(2) make sure the website's domain name is correctly set in server IIS
(3) all PCs/mobile phones can visit the test index page(http://api.fake.mysite.com)
(4) ping api.fake.mysite.com works fine
(5) if removing httpGet.addHeader("Authorization", encoded);, the Java program got a 401 Unauthorized result as expected(the server code under my control returns the result)
(6) some other applications using C# and PHP can use the web methods well, only android application can't(tested in two totally different android mobile phones, the android emulator got 400 invalid host name either)
(7) use IP instead of domain name http://xx.xx.xx.xx/v1.0/user/current, everything is the same. (xx.xx.xx.xx stands for the ip address)
(8) checked the IIS log, all requests to /v1.0/user/current returns 200/401/500, no 400 results.
(9) make sure the android application has internet permissions(actually we've added all permissions)
Does anyone know the reason or help to find the reason? Thank you very much, this issue is driving me crazy.

Should be httpGet.addHeader("Authorization", "basic " + encoded); and String encoded = Base64.encodeToString(token.getBytes(), Base64.NO_WRAP);

I struggled the very same problem. I can send HTTP POST from Fiddler or any other tool to my asp.net web API in debug mode but I can not access from my android application.
I tried to be sure to connect from my computer browser to
web API interface.
I tried to be sure to connect from android emulator web
browser(AEWB). And then I deployed my web api to IIS so I can get certain address to access from AEWB.
I can accessed to this adres from my AEWB
http://10.0.0.2:8088/api/tran
http://10.0.0.2 -> this is your local host address seen from Android
8088 -> this is your port of web api hosted on IIS
/api -> this is web api
/tran -> this is your controller