We Keep Coding

Browser debugger says my response.body is null but fiddler and the developer tools show it as populated [duplicate]

I can hit this endpoint, http://catfacts-api.appspot.com/api/facts?number=99 via Postman and it returns JSON
Additionally I am using create-react-app and would like to avoid setting up any server config.
In my client code I am trying to use fetch to do the same thing, but I get the error:
No 'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'http://localhost:3000' is therefore not allowed
access. If an opaque response serves your needs, set the request's
mode to 'no-cors' to fetch the resource with CORS disabled.
So I am trying to pass in an object, to my Fetch which will disable CORS, like so:
fetch('http://catfacts-api.appspot.com/api/facts?number=99', { mode: 'no-cors'})
.then(blob => blob.json())
.then(data => {
console.table(data);
return data;
})
.catch(e => {
console.log(e);
return e;
});
Interestingly enough the error I get is actually a syntax error with this function. I am not sure my actual fetch is broken, because when I remove the { mode: 'no-cors' } object, and supply it with a different URL it works just fine.
I have also tried to pass in the object { mode: 'opaque'} , but this returns the original error from above.
I belive all I need to do is disable CORS.. What am I missing?

mode: 'no-cors' won’t magically make things work. In fact it makes things worse, because one effect it has is to tell browsers, “Block my frontend JavaScript code from seeing contents of the response body and headers under all circumstances.” Of course you never want that.
What happens with cross-origin requests from frontend JavaScript is that browsers by default block frontend code from accessing resources cross-origin. If Access-Control-Allow-Origin is in a response, then browsers relax that blocking and allow your code to access the response.
But if a site sends no Access-Control-Allow-Origin in its responses, your frontend code can’t directly access responses from that site. In particular, you can’t fix it by specifying mode: 'no-cors' (in fact that’ll ensure your frontend code can’t access the response contents).
However, one thing that will work: if you send your request through a CORS proxy.
You can also easily deploy your own proxy to Heroku in just 2-3 minutes, with 5 commands:
git clone https://github.com/Rob--W/cors-anywhere.git
cd cors-anywhere/
npm install
heroku create
git push heroku master
After running those commands, you’ll end up with your own CORS Anywhere server running at, for example, https://cryptic-headland-94862.herokuapp.com/.
Prefix your request URL with your proxy URL; for example:
https://cryptic-headland-94862.herokuapp.com/https://example.com
Adding the proxy URL as a prefix causes the request to get made through your proxy, which:
Forwards the request to https://example.com.
Receives the response from https://example.com.
Adds the Access-Control-Allow-Origin header to the response.
Passes that response, with that added header, back to the requesting frontend code.
The browser then allows the frontend code to access the response, because that response with the Access-Control-Allow-Origin response header is what the browser sees.
This works even if the request is one that triggers browsers to do a CORS preflight OPTIONS request, because in that case, the proxy also sends back the Access-Control-Allow-Headers and Access-Control-Allow-Methods headers needed to make the preflight successful.
I can hit this endpoint, http://catfacts-api.appspot.com/api/facts?number=99 via Postman
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS explains why it is that even though you can access the response with Postman, browsers won’t let you access the response cross-origin from frontend JavaScript code running in a web app unless the response includes an Access-Control-Allow-Origin response header.
http://catfacts-api.appspot.com/api/facts?number=99 has no Access-Control-Allow-Origin response header, so there’s no way your frontend code can access the response cross-origin.
Your browser can get the response fine and you can see it in Postman and even in browser devtools—but that doesn’t mean browsers expose it to your code. They won’t, because it has no Access-Control-Allow-Origin response header. So you must instead use a proxy to get it.
The proxy makes the request to that site, gets the response, adds the Access-Control-Allow-Origin response header and any other CORS headers needed, then passes that back to your requesting code. And that response with the Access-Control-Allow-Origin header added is what the browser sees, so the browser lets your frontend code actually access the response.
So I am trying to pass in an object, to my Fetch which will disable CORS
You don’t want to do that. To be clear, when you say you want to “disable CORS” it seems you actually mean you want to disable the same-origin policy. CORS itself is actually a way to do that — CORS is a way to loosen the same-origin policy, not a way to restrict it.
But anyway, it’s true you can—in your local environment—do suff like give a browser runtime flags to disable security and run insecurely, or you can install a browser extension locally to get around the same-origin policy, but all that does is change the situation just for you locally.
No matter what you change locally, anybody else trying to use your app is still going to run into the same-origin policy, and there’s no way you can disable that for other users of your app.
You most likely never want to use mode: 'no-cors' in practice except in a few limited cases, and even then only if you know exactly what you’re doing and what the effects are. That’s because what setting mode: 'no-cors' actually says to the browser is, “Block my frontend JavaScript code from looking into the contents of the response body and headers under all circumstances.” In most cases that’s obviously really not what you want.
As far as the cases when you would want to consider using mode: 'no-cors', see the answer at What limitations apply to opaque responses? for the details. The gist of it is:
In the limited case when you’re using JavaScript to put content from another origin into a <script>, <link rel=stylesheet>, <img>, <video>, <audio>, <object>, <embed>, or <iframe> element (which works because embedding of resources cross-origin is allowed for those)—but for some reason you don’t want to/can’t do that just by having the markup of the document use the resource URL as the href or src attribute for the element.
When the only thing you want to do with a resource is to cache it. As alluded to in What limitations apply to opaque responses?, in practice the scenario that’s for is when you’re using Service Workers, in which case the API that’s relevant is the Cache Storage API.
But even in those limited cases, there are some important gotchas to be aware of; see the answer at What limitations apply to opaque responses? for the details.
I have also tried to pass in the object { mode: 'opaque'}
There is no 'opaque' request mode — opaque is instead just a property of the response, and browsers set that opaque property on responses from requests sent with no-cors mode.
But incidentally the word opaque is a pretty explicit signal about the nature of the response you end up with: “opaque” means you can’t see into any of its details; it blocks you from seeing.

If you are trying to address this issue temporarily on your localhost, you can use this chrome extension : Allow CORS Access-Control-Allow-Origin
https://chrome.google.com/webstore/detail/allow-cors-access-control/lhobafahddgcelffkeicbaginigeejlf

If you are using Express as back-end you just have to install cors and import and use it in app.use(cors());.
If it is not resolved then try switching ports.
It will surely resolve after switching ports

So if you're like me and developing a website on localhost where you're trying to fetch data from Laravel API and use it in your Vue front-end, and you see this problem, here is how I solved it:
In your Laravel project, run command php artisan make:middleware Cors. This will create app/Http/Middleware/Cors.php for you.
Add the following code inside the handles function in Cors.php:
return $next($request)
->header('Access-Control-Allow-Origin', '*')
->header('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS');
In app/Http/kernel.php, add the following entry in $routeMiddleware array:
‘cors’ => \App\Http\Middleware\Cors::class
(There would be other entries in the array like auth, guest etc. Also make sure you're doing this in app/Http/kernel.php because there is another kernel.php too in Laravel)
Add this middleware on route registration for all the routes where you want to allow access, like this:
Route::group(['middleware' => 'cors'], function () {
Route::get('getData', 'v1\MyController#getData');
Route::get('getData2', 'v1\MyController#getData2');
});
In Vue front-end, make sure you call this API in mounted() function and not in data(). Also make sure you use http:// or https:// with the URL in your fetch() call.
Full credits to Pete Houston's blog article.

You can also set up a reverse proxy which adds the CORS headers using a self-hosted CORS Anywhere or Just CORS if you want a managed solution.
https://justcors.com/<id>/<your-requested-resource>
http://cors-anywhere.com/<your-requested-resource>

Very easy solution (2 min to config) is to use local-ssl-proxy package from npm
The usage is straight pretty forward:
1. Install the package:
npm install -g local-ssl-proxy
2. While running your local-server mask it with the local-ssl-proxy --source 9001 --target 9000
P.S: Replace --target 9000 with the -- "number of your port" and --source 9001 with --source "number of your port +1"

Solution for me was to just do it server side
I used the C# WebClient library to get the data (in my case it was image data) and send it back to the client. There's probably something very similar in your chosen server-side language.
//Server side, api controller
[Route("api/ItemImage/GetItemImageFromURL")]
public IActionResult GetItemImageFromURL([FromQuery] string url)
{
ItemImage image = new ItemImage();
using(WebClient client = new WebClient()){
image.Bytes = client.DownloadData(url);
return Ok(image);
}
}
You can tweak it to whatever your own use case is. The main point is client.DownloadData() worked without any CORS errors. Typically CORS issues are only between websites, hence it being okay to make 'cross-site' requests from your server.
Then the React fetch call is as simple as:
//React component
fetch(`api/ItemImage/GetItemImageFromURL?url=${imageURL}`, {
method: 'GET',
})
.then(resp => resp.json() as Promise<ItemImage>)
.then(imgResponse => {
// Do more stuff....
)}

I had a similar problem with my browser debugger saying my response.body was null but fiddler and the developer tools show it as populated that turned out to be basically the same scenario as this. I was using a local Angular application hitting a Web Api service running on IISExpress. I fixed it by following the steps outlined here to find the correct applicationhost.config file to add a Access-Control-Allow-Origin header like so:
<customHeaders>
<clear />
<add name="X-Powered-By" value="ASP.NET" />
<add name="Access-Control-Allow-Origin" value="*" />
<add name="Access-Control-Allow-Headers" value="Content-Type" />
</customHeaders>

If all the above solutions don't work, probably it's because of the file permissions as sometimes even if you have fixed the non-cors problem using Heroku or another way, it throws 403 forbidden error. Set the directory/file permissions like this:
Permissions and ownership errors
A 403 Forbidden error can also be caused by incorrect ownership or permissions on your web content files and folders.
Permissions
Rule of thumb for correct permissions:
Folders: 755
Static Content: 644
Dynamic Content: 700

Stackify not using proxy in ASP.NET Core

In a ASP.Net Core website I am using NLog with a StackifyTarget to log to Stackify Retrace. Our code is running behind a corporate HTTP proxy.
I have some custom code that returns a WebProxy.
The WebProxy is then set on StackifyLib.Utils.HttpClient.CustomWebProxy, in the Configure method of the StartUp.cs.
When a log is written, it is written successfully to file, but not to the StackifyTarget.
When I investigate the network traffic using Fiddler, I can see that there are 407/Proxy AuthenticationRequired errors for outbound traffic to the Stackify servers.
Using the same StackifyTarget in a WinForms application, .Net Framework 4.7.2, it just works fine:
<system.net>
<defaultProxy useDefaultCredentials="true" />
</system.net>
When I set a breakpoint on the Log.Info, I can see that the proxy information is still set on the StackifyLib.
How can have StackifyLib work correctly with the proxy, in ASP.Net Core?

The custom code that assigns proxy to CustomWebProxy could also try and assign this first:
System.Net.WebRequest.DefaultWebProxy = webProxy;
As the very first thing in the application, before Stackify-HttpClient is created.

Stackify has 2 docs that may help with this:
https://docs.stackify.com/docs/troubleshoot-errors-and-logs-net-configurations
https://docs.stackify.com/docs/http-proxies-configure
If either of these do not work you can submit a ticket to them to support#stackify.com

No connection could be made because the target machine actively refused it 127.0.0.1:port with subdomain

I am running an ASP.NET MVC 5 application which also hosts IdentityServer3. As many others have experienced before, when I connect to this endpoint...
http://tenant1.localhost:51515/identity/.well-known/openid-configuration?client_id=backoffice&redirect_uri=http%3A%2F%2Flocalhost%3A37046%2Findex.html&response_type=id_token%20token&scope=openid%20all_claims&state=1793477650&nonce=1172967295
... I get the following error:
No connection could be made because the target machine actively refused it 127.0.0.1:51515
at System.Net.HttpWebRequest.GetResponse()
at RestSharp.Http.GetRawResponse(HttpWebRequest request)
at RestSharp.Http.GetResponse(HttpWebRequest request)
(Note: Before you decide this is a duplicate, please read until the end of the question - I have done my homework before coming here for help)
Same result when I use HttpWebRequest, cURL or even with an app in Go. Here are some code samples that I use:
C# (using RestSharp):
var client = new RestClient("http://tenant1.localhost:51515/identity/.well-known/openid-configuration?client_id=backoffice&redirect_uri=http%3A%2F%2Flocalhost%3A37046%2Findex.html&response_type=id_token%20token&scope=openid%20all_claims&state=1793477650&nonce=1172967295");
var request = new RestRequest(Method.GET);
request.AddHeader("postman-token", "a09c64d2-e0c6-a416-d5ad-92079f0676b9");
request.AddHeader("cache-control", "no-cache");
IRestResponse response = client.Execute(request);
cURL:
curl -X GET \
'http://tenant1.localhost:51515/identity/.well-known/openid-configuration?client_id=backoffice&redirect_uri=http%3A%2F%2Flocalhost%3A37046%2Findex.html&response_type=id_token%20token&scope=openid%20all_claims&state=1793477650&nonce=1172967295' \
-H 'cache-control: no-cache' \
-H 'postman-token: 271bbbe6-bcb1-b999-80e5-9193f0c134ba'
I made a few alternatives for these samples, for instance by including a host header (value tenant1.localhost:51515) or using that same uri as the proxy for the web clients. Unfortunately they all return the same error.
Strangely enough all of the requests I make with my browser or Postman succeed. JavaScript code that connects to the endpoints also works. There's one exception to this: as soon as as I have a Fiddler session running, this is the response I get from Postman:
[Fiddler] The connection to 'tenant1.localhost' failed.
<br />Error: ConnectionRefused (0x274d).
<br />System.Net.Sockets.SocketException No connection could be made because the target machine actively refused it 127.0.0.1:51515
I have been looking for a solution for days now but I can't seem to find the right one but it seems clear that only in server side code the problem occurs. Here's what I have tried so far. I included the applicationhost.config and hosts file to show how I 'enabled' subdomains for my web application (which are internally used to identify the tenants). Furthermore I use IIS Express for local development and IIS for production environments.
Applicationhost.config
<site name="MyApp" id="5">
<application path="/" applicationPool="Clr4IntegratedAppPool">
<virtualDirectory path="/" physicalPath="D:\Source\MyApp" />
</application>
<bindings>
<binding protocol="http" bindingInformation="*:51515:localhost" />
<binding protocol="http" bindingInformation="*:51515:tenant1.localhost" />
</bindings>
</site>
Hosts file
127.0.0.1 tenant.localhost
Netstat
This is the result when I execute netstat -anb:
This is the result of netstat -na | find '51515':
I am not sure what these values mean so I could use some input here. Just to make sure I disconnected from the Internet and disabled both firewall and anti virus scanners, with no result.
Proxy
These are my Internet Options settings. As you see, everything is checked out:
I tried all sorts of combinations with the proxy settings in my web/app.config files. I don't think this will play a major role in resolving the issue as I have the same problems with my Golang app (which is merely a code snippet generated by Postman). I even tried to use Fiddler as the proxy by setting the url to http://127.0.0.1:8888. As to be expected, any server side proxies for the WebRequest instances didn't help either.
<system.net>
<defaultProxy>
<proxy usesystemdefault="False"/>
</defaultProxy>
</system.net>
Visual Studio
I run both projects (host and client app) as an administrator in VS2017
I changed the port of the hosting application numerous times
Question
Given the numerous questions about this topic the only remarkable difference I see is that I use a subdomain in my url. Whenever I don't use a subdomain everything works perfectly!
If this assumption appears to be correct, how can I trick the DNS, firewall or any other blocking mechanism to accept requests from the subdomain(s)? Maybe a proxy could help?

The netstat output shows that only IP v6 addresses are used, which is not quite typical, but should be OK if for some reason IP v4 is not used on the machine. Then you cannot expect IP v4 packets (to 127.0.0.1) be processed by the server.
One quick solution is to set in hosts file a record of [::1] instead of 127.0.0.1.

Although Lex Li's answer is better, I would just like to provide an alternative solution for the issue. I extended my C# code sample from the question to this:
var client = new RestClient("http://tenant1.localhost:51515/identity/.well-known/openid-configuration?client_id=backoffice&redirect_uri=http%3A%2F%2Flocalhost%3A37046%2Findex.html&response_type=id_token%20token&scope=openid%20all_claims&state=1793477650&nonce=1172967295");
client.Proxy = new WebProxy("http://localhost:51515");
var request = new RestRequest(Method.GET);
request.AddHeader("postman-token", "a09c64d2-e0c6-a416-d5ad-92079f0676b9");
request.AddHeader("cache-control", "no-cache");
IRestResponse response = client.Execute(request);
So to answer my own question: yes a proxy could help. By adding a proxy to the RestClient with a URI that does not contain a subdomain, the webrequest works like a charm.

IIS7 URL Rewrite returns 404 for WCF requests (reverse proxy)

I am using IIS7.5, .net 4.0. I am working locally.
I have installed Application Request Routing, Web Farm Framework, WebDeploy and UrlRewrite to set up a reverse proxy. This works fine for the most part.
I have two websites:
DefaultWebSite (port 80, app pool: Default App Pool (.net 4)) and
Target (port 8085, app pool: TargetAppPool(my identity, .net 4)).
I have a rewrite rule on DefaultWebSite (created as directed on IIS.net) which redirects all localhost (port 80) traffic to localhost:8085 just as detailed in the above link. This works fine for most document types (.aspx, .xap, .htm, .ico) but a request to MyService.svc fails. It returns a 404.
To be clear:
When I paste localhost:8085/MyService.svc into a browser I get the requested WCF page.
When I paste localhost/MyService.svc into a browser I get a 404.
When I paste localhost:8085/MyIcon.ico into a browser I get the requested resource.
When I paste localhost/MyIcon.ico into a browser I get the requested resource.
.svc is the only document type that I've found that returns a 404.
I've got two pieces of info that might be of relevance.
App Pools. When I change the DefaultWebSite's app pool to TargetAppPool then the 404 becomes a 500 ("Failed to map the path '/'"). All other requests are successful when this change is made. Not sure if this relevant or not.
FREB (Failed Request Tracing) Log. I found a page (http://blogs.msdn.com/b/asiatech/archive/2011/08/25/return-404-4-not-found-when-url-rewrite.aspx) which details the steps in a FREB log when a URL rewrite is more successful than mine (it fails later on). I've not been able to find out how to generate a FREB log for a successful rewrite (if that's possible) so I can only compare my FREB log to the one on that blog. I can see that their step 21 (URL_CHANGED) in my FREB log but not 22 (URL_REWRITE_END). I've not got enough experience with these logs to notice anything more significant than that (suggestions welcomed).
My main question is: does anyone know why just URLs requesting .svc resources are not being rewritten?
A secondary question is: does anyone know how to generate a FREB log for successful request (if it's even possible)?
Thanks
Update:
I have changed the architecture to try to get more info.
I have moved the Target website to a different PC on which I have installed Microsoft Network Monitor to capture the incoming traffic.
Before I changed the url-rewrite rule to point at this new website I got the correct response when I made a request to MyService.svc on the new PC. Fine.
As soon as I changed the rewrite rule to route the request to the new Target website then it responds as before (404). I have made both POST and GET requests. There is no sign of any of the requests in the Network Monitor log (all other calls -200, 404 or otherwise- appear in this log).
This leads me to think that there is something incompatible with url-rewrites and *.svc requests. I tried making a request to MyService.asmx (having created this file) and it correctly returned a page, so it is limited to *.svc. Any ideas?

The solution to this is in the config file of the Target web site.
In web.config (in the Target application) there is a section which read:
<serviceHostingEnvironment aspNetCompatibilityEnabled="true"/>.
I changed this to read:
<serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />.
Credit must go to http://forums.iis.net/post/1956671.aspx for this (although s/he claims it is the proxy's config which needs to be changed, but I found it be the Target app, not the proxy server).

If you still can't get it running, make sure you don't have the WCF handlers on the website which acts as the reverse proxy.
I disabled this by adding this web.config of the reverse proxy:
<system.webServer>
...
<handlers>
<remove name="svc-ISAPI-4.0_64bit" />
<remove name="svc-ISAPI-4.0_32bit" />
<remove name="svc-Integrated-4.0" />
</handlers>
</system.webServer>

Because the rewrite appears to work for all resources except when the extension is .svc I would say this would be the area to concentrate on.
I would imagine that the rewrite rules are matching your other resources, but not your service, and because these are usually regular expressions (which are often complicated) I would say it would be worth testing any rules you find with your urls. Details of how to find the regular expressions for an UrlRewrite can be found here.
It is also probably also worth looking at any outbound rules with the same mindset.

How to use Fiddler to debug traffic from Any app (eg. C#/WPF app)

I am tring to debug whats wrong with my HTTP requests from another question here on SO. So i read a bit about Fiddler and wanted to use it to debug my problem. But I can't seem to get traffic from my WPF application to go through Fiddler. I believe I need to configure a proxy. I am using a WebClient for a basic example, but I think i will require a WebRequest later. But for now, with a simple WebClient, how can I get it to go through Fiddler (I believe I have to set proxy to localhost:8888)?
UPDATE:
I don't know if i did the right thing or not but I tried
var wc = new WebClient();
WebProxy proxy = new WebProxy();
proxy.Address = new Uri("http://localhost:8888");
wc.Proxy = proxy;
but failed - I don't see any traffic in Fiddler
I tried ...
var wc = new WebClient();
WebProxy proxy = new WebProxy("127.0.0.1", 8888);
wc.Proxy = proxy;
still nothing

I found the solution at this fiddler2.com page
Why don't I see traffic sent to
http://localhost or http://127.0.0.1?
Internet Explorer and the .NET Framework are hardcoded not to send
requests for Localhost through any
proxies, and as a proxy, Fiddler will
not receive such traffic.
The simplest workaround is to use your machine name as the hostname
instead of Localhost or 127.0.0.1. So,
for instance, rather than hitting
http://localhost:8081/mytestpage.aspx,
instead visit
http://machinename:8081/mytestpage.aspx.

Maybe a little late, but...
I get around this simply by appending a "dot" to localhost, so instead of accessing localhost, I try to access localhost. (notice the dot at the end of the hostname)
Credit where credit is due:
I got this unusual tip from this thread http://www.west-wind.com/weblog/posts/2009/Jan/14/Monitoring-HTTP-Output-with-Fiddler-in-NET-HTTP-Clients-and-WCF-Proxies#596591
Works fine!

You can find answer in below post
https://stackoverflow.com/a/7506427/471499
it lists that you need to add this in your web.config OR App.Config
<system.net>
<defaultProxy>
<proxy bypassonlocal="False" usesystemdefault="True" proxyaddress="http://127.0.0.1:8888" />
</defaultProxy>
</system.net>
then Start Fiddler on the same machine as the application running.
Click Tools | Fiddler Options => Connections => adjust the port as 8888.(allow remote if you need that)
Ok, then from file menu, capture the traffic.
That's all, but don't forget to remove the web.config lines after closing the fiddler, because if you don't it will make an error.
Run Fiddler for DotNet Core requests
RUN FIDDLER for .net core required "Netsh" tool https://learn.microsoft.com/en-us/windows-server/networking/technologies/netsh/netsh-contexts)
command to add proxy :
netsh winhttp set proxy 127.0.0.1:8880
After run the proxy, adjust Fiddler proxy to the same port, and enjoy
remove proxy
netsh winhttp reset proxy
Reference :
http://fiddler2.com/documentation/Configure-Fiddler/Tasks/UseFiddlerAsReverseProxy
https://docs.telerik.com/fiddler/configure-fiddler/tasks/configuredotnetapp

All the time I use below configuration to redirect the network HTTP calls to pass thru fiddler proxy from my applications.
This works in all kinds of .NET applications (which has either web.config or app.config file) and in fiddler its best to disable Capture Traffic option to avoid capturing general traffic from all the applications running. Shortcut key for this is F12.
<system.net>
<defaultProxy>
<proxy proxyaddress="http://localhost:8888/" />
</defaultProxy>
</system.net>
This is valuable configuration if you have third party assemblies in which you don't have chance of changing the code that calls URL.
I hope this helps someone.

"IIS Express won't receive traffic to machinename so instead route to localhost.fiddler fiddler2.com/documentation/Configure-Fiddler/Troubleshooting/… – robrich May 9 '13 at 6:02"
RobRich above got it right.
This is the only thing that worked as I can only use IIS Express.