I'm attempting to download image from Google Drive, but the image is not opening for some reason.
WebClient webClient = new WebClient();
webClient.Headers["User-Agent"] = "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36";
webClient.DownloadFile(url, #"path_to_download.png");
The URL looks like https://drive.google.com/file/d/1231231313+yDnEx7upHEsM4ozjh/view from this URL I take the image source. It looks like https://lh3.googleusercontent.com/drive-viewer/ASAdd1d123sT1QX_ZYlcoHMTeDBeUJaNnwxqjJzFgC8e24i1HOylgu2MnqzO9iNb41exySXsp2sJWA=w1106-h772 and after that, I run the code upper. Everything works, but I cannot open the image.
Related
i'm new in this programming world. So i got a task to download zip file from url via C# Console Application, i've tried many things but still got same error. I've already add some Headers in my code also based on Fiddler Result.
This is my code:
WebClient webclient = new WebClient();
webclient.Headers.Add("sec-ch-ua-mobile", "?0");
webclient.Headers.Add("sec-ch-ua-platform", "windows");
webclient.Headers.Add("upgrade-insecure-requests", "1");
webclient.Headers.Add("user-agent", "mozilla/5.0 (windows nt 10.0; win64; x64) applewebkit/537.36 (khtml, like gecko) chrome/98.0.4758.102 safari/537.36");
webclient.Headers.Add("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9");
webclient.Headers.Add("sec-fetch-site", "cross-site");
webclient.Headers.Add("sec-fetch-mode", "navigate");
webclient.Headers.Add("sec-fetch-user", "?1");
webclient.Headers.Add("sec-fetch-dest", "document");
webclient.Headers.Add("accept-encoding", "gzip, deflate, br");
webclient.Headers.Add("accept-language", "id-id,id;q=0.9,en-us;q=0.8,en;q=0.7,ms;q=0.6,th;q=0.5");
webclient.DownloadFile(url, #"d:\data.zip");
Is there any missed? or should i take other way?
Thanks
Regards,
Alvin
403 is mean HTTP forbidden. It's usually using authentication like an API_KEY or any authentication but refused or fail.
Make sure your API_KEY is right.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403
https://en.wikipedia.org/wiki/HTTP_403
Can you please add the URL from where you want to download that zip?
I want to send a link request to any site in asp.net. In doing so, I want the outgoing connection to be perceived as a mobile device. If I want to improve it further, I want the sent link to be at the mobile connection level. For example, 3G or 4G. I'm waiting for your help. Thank you.
The device is determined by the UserAgent header.
To set the user agent header, see How do I set a default User Agent on an HttpClient?
Regarding network speed throttling, it is possible, see Bandwidth throttling in C#
Here are some user agents, from https://deviceatlas.com/blog/mobile-browser-user-agent-strings
Safari for iOS
Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_1 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) Version/10.0 Mobile/14E304 Safari/602.1
Android Browser
Mozilla/5.0 (Linux; U; Android 4.4.2; en-us; SCH-I535 Build/KOT49H) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Chrome Mobile
Mozilla/5.0 (Linux; Android 7.0; SM-G930V Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.125 Mobile Safari/537.36
Opera Mobile (Blink rendering engine)
Mozilla/5.0 (Linux; Android 7.0; SM-A310F Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.91 Mobile Safari/537.36 OPR/42.7.2246.114996
Opera Mobile (Presto rendering engine)
Opera/9.80 (Android 4.1.2; Linux; Opera Mobi/ADR-1305251841) Presto/2.11.355 Version/12.10
Opera Mini
Opera/9.80 (J2ME/MIDP; Opera Mini/5.1.21214/28.2725; U; ru) Presto/2.8.119 Version/11.10
Opera Mini (iOS WebKit)
Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) OPiOS/10.2.0.93022 Mobile/11D257 Safari/9537.53
Firefox for Android
Mozilla/5.0 (Android 7.0; Mobile; rv:54.0) Gecko/54.0 Firefox/54.0
Firefox for iOS
Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_2 like Mac OS X) AppleWebKit/603.2.4 (KHTML, like Gecko) FxiOS/7.5b3349 Mobile/14F89 Safari/603.2.4
UC Browser
Mozilla/5.0 (Linux; U; Android 7.0; en-US; SM-G935F Build/NRD90M) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 UCBrowser/11.3.8.976 U3/0.8.0 Mobile Safari/534.30"
Dolphin
Mozilla/5.0 (Linux; Android 6.0.1; SM-G920V Build/MMB29K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.98 Mobile Safari/537.36
Puffin for Android
Mozilla/5.0 (Linux; Android 5.1.1; SM-N750K Build/LMY47X; ko-kr) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Mobile Safari/537.36 Puffin/6.0.8.15804AP
Puffin for iOS
Mozilla/5.0 (Linux; Android 5.1.1; SM-N750K Build/LMY47X; ko-kr) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Mobile Safari/537.36 Puffin/6.0.8.15804AP
Samsung Browser
Mozilla/5.0 (Linux; Android 7.0; SAMSUNG SM-G955U Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/5.4 Chrome/51.0.2704.106 Mobile Safari/537.36
Yandex Browser
Mozilla/5.0 (Linux; Android 6.0; Lenovo K50a40 Build/MRA58K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.137 YaBrowser/17.4.1.352.00 Mobile Safari/537.36
MIUI Browser
Mozilla/5.0 (Linux; U; Android 7.0; en-us; MI 5 Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/53.0.2785.146 Mobile Safari/537.36 XiaoMi/MiuiBrowser/9.0.3
IE Mobile
Mozilla/5.0 (compatible; MSIE 10.0; Windows Phone 8.0; Trident/6.0; IEMobile/10.0; ARM; Touch; Microsoft; Lumia 950)
Edge Mobile
Mozilla/5.0 (Windows Phone 10.0; Android 6.0.1; Microsoft; Lumia 950) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Mobile Safari/537.36 Edge/15.14977
BlackBerry Browser
Mozilla/5.0 (BB10; Kbd) AppleWebKit/537.35+ (KHTML, like Gecko) Version/10.3.3.2205 Mobile Safari/537.35+
I am trying to get the html source of a youtube video using cURL command line but I need it to be without https/ssl.
My problem is that I must use the compiled version of cURL with SSL/SSH.
I am using the following command:
curl --user-agent "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36" -L -x http://my.foo.proxy:8080 http://youtube.com/watch?v=youtubevideo > html.html
this works but a specific part of the html source is in https (look for a really long script string inside that file. some of the links there start with httpS)
curl --proto =http --proto-redir =http --user-agent "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36" -L -x http://my.foo.proxy:8080 http://youtube.com/watch?v=youtubevideo > html.html
this command causes an error:
protocol https not supported or disabled in libcurl.
which is really weird because the curl version I am using does have ssl and I dont even want https (see the -proto and -proto-redir args).
As a test I also tried using .NET Webclient class like:
public static void DownloadString (string address)
{
WebClient client = new WebClient ();
string reply = client.DownloadString (address);
Console.WriteLine (reply);
}
and in this case I get a html source file without https.
My question is, how do I get a html source file of a youtube video using cURL without https inside my html source file like when I use .NET/Webclient?
using an user agent without firefox fixes this issue when used inside a console:
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101
When used with a binding set SSL_VERIFYPEER to false and SSL_VERIFYHOST to 0. Allows man in the middle attack but if that is the only option...
In addition HTTPGET and FOLLOWLOCATION should also both be set to true.
i had created an empty c# web site with just one page with Request.Browser.Version & UserAgent output on it. Then hit it with different Chrome versions using "User-Agent Switcher" Chrome extension.
For time to time, though the Request.UserAgent is correct, Request.Browser.Version seems to return wrong value:
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.16 Safari/537.36" Returned Request.Browser.Version:39
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2272.16 Safari/537.36" Returned Request.Browser.Version:41
So yes, .net 4.5 caches the user agent by its first 64 chars. And that's just gets them before the version number. So the next user with the same browser but with a different version will get the wrong browser version and so.
To solve it just change the :browserCaps userAgentCacheKeyLength="...", as can be seen here:
.Net 4.0 website cannot identify some AppleWebKit based browsers
How isn't this stupid Microsoft bug on the headlines?
I've been playing around with Selenium and PhantomJS in C# but I want to be able to fake my User Agent to be this:
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0
Instead of:
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534.34 (KHTML, like Gecko) PhantomJS/1.9.1 Safari/534.34
Is it possible to modify the HTTP headers of PhantomJS with Selenium to achieve this? If so, how?
Thanks in advance.
I found the answer:
PhantomJSOptions options = new PhantomJSOptions();
options.AddAdditionalCapability("phantomjs.page.settings.userAgent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0");
IWebDriver driver = new PhantomJSDriver(options);
Thanks.