Related: how-do-i-use-webrequest-to-access-an-ssl-encrypted-site-using-https
How to send an HTTPS GET Request in C#?
Add ?var1=data1&var2=data2 to the end of url to submit values to the page via GET:
using System.Net;
using System.IO;
string url = "https://www.example.com/scriptname.php?var1=hello";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream resStream = response.GetResponseStream();
Simple Get Request using HttpClient Class
using System.Net.Http;
class Program
{
static void Main(string[] args)
{
HttpClient httpClient = new HttpClient();
var result = httpClient.GetAsync("https://www.google.com").Result;
}
}
I prefer to use WebClient, it seems to handle SSL transparently:
http://msdn.microsoft.com/en-us/library/system.net.webclient.aspx
Some troubleshooting help here:
https://clipperhouse.com/webclient-fiddler-and-ssl/
Related
Seems HttpClient is not able to perform the following post request (it throws validation errors)
POST with URI encoded parameters
no Body but Content-Type header setted.
I'm sure this is a problem of usage of C# since I was able to perform the correct request with Postman (see the image)
The service I'm trying to authenticate against is this one Fitbit OAUTH2
I tried also HttpWebRequest, but I failed to do that even with that class, however here's also the second attemp. Seems C# is performing some action different from Postman but I have to figure out which action. Does anyone ever had a similiar issue?
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;
public static class MyRequest
{
public static string Call( string uri, string headerName, string header)
{
HttpWebRequest webRequest = (HttpWebRequest)HttpWebRequest.Create( uri);
webRequest.Method = "POST";
webRequest.Headers.Add( "Authorization", $"{headerName} {header}");
webRequest.ContentType = "application/x-www-form-urlencoded";
var response = (HttpWebResponse) webRequest.GetResponse();
if(response.StatusCode == HttpStatusCode.OK || response.StatusCode == HttpStatusCode.Created)
{
}
else
{
return null;
}
using ( var stream = response.GetResponseStream())
{
return stream.ToString();
}
}
}
If I use the HttpClient with the following content
content = FormUrlEncodedContet( keyValues)
the final URL do not have the #_=_ appended, and that is requested by that service otherwise it returns 401
I have Netduino Plus and I need it to send Http requests to my server. I'm not a guru in C#, I've never tried it before, so I copy/paste code from internet and try to make it works. But even after several hours I can't get it work.
using System;
using System.IO;
using System.Net;
using System.Text;
using SecretLabs.NETMF.Hardware;
using SecretLabs.NETMF.Hardware.NetduinoPlus;
namespace NetduinoPlusApplication5
{
public class Program
{
static void Main()
{
var request = WebRequest.Create("http://example.com?variable=1");
request.Method = "GET";
var result = request.GetResponse();
}
}
}
What am I doing wrong?
You are executing a GET request so I think you want to get the response body from the server. In this case you have to use :
Stream respStream = resp.GetResponseStream();
instead of simple GetResponse(). In this way, you can read on the stream the response body.
Paolo.
I wish to automatically uncompress GZiped response.
I am using the following snippet:
mywebclient.Headers[HttpRequestHeader.AcceptEncoding] = "gzip";
mywebclient.Encoding = Encoding.UTF8;
try
{
var resp = mywebclient.DownloadData(someUrl);
}
I have checked HttpRequestHeader enum, and there is no option to do this via the Headers
How can I automatically decompress the resp? or Is there another function I should use instead of mywebclient.DownloadData ?
WebClient uses HttpWebRequest under the covers. And HttpWebRequest supports gzip/deflate decompression. See HttpWebRequest AutomaticDecompression property
However, WebClient class does not expose this property directly. So you will have to derive from it to set the property on the underlying HttpWebRequest.
class MyWebClient : WebClient
{
protected override WebRequest GetWebRequest(Uri address)
{
HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
return request;
}
}
Depending on your situation, it may be simpler to do the decompression yourself.
using System.IO.Compression;
using System.Net;
try
{
var client = new WebClient();
client.Headers[HttpRequestHeader.AcceptEncoding] = "gzip";
var responseStream = new GZipStream(client.OpenRead(myUrl), CompressionMode.Decompress);
var reader = new StreamReader(responseStream);
var textResponse = reader.ReadToEnd();
// do stuff
}
I created all the temporary variables for clarity. This can all be flattened to only client and textResponse.
Or, if simplicity is the goal, you could even do this using ServiceStack.Text by Demis Bellot:
using ServiceStack.Text;
var resp = "some url".GetJsonFromUrl();
(There are other .Get*FromUrl extension methods)
I'd like to download some data from a forum. The page containing the data is visible only to registered users. Here's an example webpage containing user data;
http://www.bikeforums.net/member.php/227664-StackOverflow
I'd like to get the data using wget or C#. I tried logging in via Firefox, then passing the cookies file (hopefully containing the login information) to wget. That was more of a makeshift hack and not a real solution, but it still failed. How do I do this properly?
I set up an account for testing if that's helpful.
User: StackOverflow
Pass: so123
Using firebug you can easily get the POST data for the login page and use it to create a WebRequest and Login to the Forum.
The Server create the cookies for authentication and we can use this cookies in the next request on the forum page so the server can authenticate the request and return all the data.
Here I've tested a simple console application that achieves this mechanism.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Globalization;
using System.Web;
using System.Net;
using System.Xml;
using System.Xml.XPath;
using System.Xml.Linq;
using System.IO;
namespace ConsoleApplication2
{
class Program
{
static void Main(string[] args)
{
CookieContainer cookieContainer = new CookieContainer();
HttpWebRequest wpost = (HttpWebRequest) HttpWebRequest.Create("http://www.bikeforums.net/login.php?do=login");
wpost.CookieContainer = cookieContainer;
wpost.Method = "POST";
string postData = "do=login&vb_login_md5password=d93bd4ce1af6a9deccaf0ea844d6c05d&vb_login_md5password_utf=d93bd4ce1af6a9deccaf0ea844d6c05d&s=&securitytoken=guest&url=%2Fmember.php%2F227664-StackOverflow&vb_login_username=StackOverflow&vb_login_password=";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
// Set the ContentType property of the WebRequest.
wpost.ContentType = "application/x-www-form-urlencoded";
// Set the ContentLength property of the WebRequest.
wpost.ContentLength = byteArray.Length;
// Get the request stream.
System.IO.Stream dataStream = wpost.GetRequestStream();
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close();
// Get the response.
HttpWebResponse response = (HttpWebResponse) wpost.GetResponse();
// Request
wpost = (HttpWebRequest)WebRequest.Create("http://www.bikeforums.net/member.php/227664-StackOverflow");
//Assing the cookies created on the server to the new request
wpost.CookieContainer = cookieContainer;
wpost.Method = "GET";
response = (HttpWebResponse)wpost.GetResponse();
Stream receiveStream = response.GetResponseStream();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8);
//Display the result to console...
Console.WriteLine(readStream.ReadToEnd());
response.Close();
readStream.Close();
Console.Read();
}
}
}
This is driving me a bit nuts. I am trying to do something quite simple, and I have done it many times before. Just trying to call a REST API.
I am trying to call GetMessage with endpoint = "http://feed.linksynergy.com/productsearch?token=717f8c8511725ea26fd5c3651f32ab187d8db9f4b208be781c292585400e682d&keyword=DVD", and it keeps returning empty string. If I pass it any other valid URL, it will work. But if I just copy and paste the original URL into the web browser, it returns fine!
Can any smart developer tell me what's going on?
Code below. Thanks in advance.
James
public string GetMessage(string endPoint)
{
HttpWebRequest request = CreateWebRequest(endPoint);
using (var response = (HttpWebResponse)request.GetResponse())
{
var responseValue = string.Empty;
if (response.StatusCode != HttpStatusCode.OK)
{
string message = String.Format("POST failed. Received HTTP {0}", response.StatusCode);
throw new ApplicationException(message);
}
// grab the response
using (var responseStream = response.GetResponseStream())
{
using (var reader = new StreamReader(responseStream))
{
responseValue = reader.ReadToEnd();
}
}
return responseValue;
}
}
private HttpWebRequest CreateWebRequest(string endPoint)
{
var request = (HttpWebRequest)WebRequest.Create(endPoint);
request.Method = "GET";
request.ContentLength = 0;
request.ContentType = "text/xml";
return request;
}
Not sure why your setting ContentLength/ContentType - that is generally for HTTP POST, where there is a request body for which you write data to via a stream.
This is a HTTP GET, so there is no request body. (just URI w/ query string)
This should work:
using System;
using System.IO;
using System.Net;
using System.Text;
// Create the web request
HttpWebRequest request = WebRequest.Create("http://www.someapi.com/") as HttpWebRequest;
// Get response
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
StreamReader reader = new StreamReader(response.GetResponseStream());
// Console application output
Console.WriteLine(reader.ReadToEnd());
}
EDIT
#Gabe is also quite right - try this on another computer, that is isn't behind any kind of firewall/proxy server.
My work PC was behind a proxy server, so in order to make REST-based HTTP calls, i needed to do this:
var proxyObject = new System.Net.WebProxy("http://myDomain:8080/", true);
System.Net.WebRequest req = System.Net.WebRequest.Create("http://www.someapi.com/");
req.Proxy = proxyObject;
proxyObject.Credentials = New System.Net.NetworkCredential("domain\username","password")