We Keep Coding

SOLVED C# HttpWebResponse delivers different response than Postman

I'm consuming a Web API of an internal system in the company.
It's getting a payload in JSON format and returning a response with data in JSON format.
When sending the request with Postman, it returns the expected response (StatusCode=200 + a response text in JSON format). That means that everything is OK with the web service.
Now I have to develop an application in C# to send this HTTP request.
The problem is, that I receive as response content "OK" and not the expected JSON response gotten with Postman.
public HttpWebResponse SendRequest(string url, string checkOutFolder, string drawingNo, string login, string password)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = "application/json";
request.Method = "GET";
request.Accept = "application/json";
string payload = GeneratePayLoad(checkOutFolder, drawingNo);
string header = CreateAuthorization(login, password);
request.Headers[HttpRequestHeader.Authorization] = header;
request.ServicePoint.Expect100Continue = false;
var type = request.GetType();
var currentMethod = type.GetProperty("CurrentMethod", BindingFlags.NonPublic | BindingFlags.Instance).GetValue(request);
var methodType = currentMethod.GetType();
methodType.GetField("ContentBodyNotAllowed", BindingFlags.NonPublic | BindingFlags.Instance).SetValue(currentMethod, false);
using (var streamWriter = new StreamWriter(request.GetRequestStream()))
{
streamWriter.Write(payload);
}
// Response
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using (StreamReader rd = new StreamReader(response.GetResponseStream()))
{
string responseContent = rd.ReadToEnd();
Console.WriteLine(responseContent);
}
return response;
}
Has anyone already experiences something similar.
Can you help me?
EDIT
Following your suggestions
1) Changed the method to POST -> result is still the same
2) Used Postman's code generator and RestSharp -> result is still the same
public void Request(string url, string checkOutFolder, string drawingNo, string login, string password)
{
var client = new RestClient(url);
var request = new RestRequest();
request.Method = Method.Post;
request.AddHeader("Content-Type", "application/json");
request.AddHeader("Authorization", "Basic **********");
var body = GeneratePayLoad(checkOutFolder, drawingNo);
request.AddParameter("application/json", body, ParameterType.RequestBody);
var response = client.Execute(request);
Console.WriteLine(response.Content);
}
Changed to HttpClient -> result still the same
using (var client = new HttpClient())
{
string uri = "******************";
string path = "destinationpath";
var endpoint = new Uri(uri);
string payload = GeneratePayLoad(path, "100-0000947591");
//FormUrlEncodedContent form = new FormUrlEncodedContent(payload);
var stringContent = new StringContent(payload);
var payload2 = new StringContent(payload, Encoding.UTF8, "application/json");
var byteArray = Encoding.ASCII.GetBytes("*******");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", Convert.ToBase64String(byteArray));
var base64EncodedAuthenticationString = Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(client.DefaultRequestHeaders.Authorization.ToString()));
var result = client.PostAsync(endpoint, stringContent).Result.Content.ReadAsStringAsync().Result;
Console.WriteLine("test");
}
Wrote a Python code using requests Package -> delivers the same as Postman. So the problem is in the C# code.
Does anyone have an idea what is going on?
SOLVED
The issue was on the payload generation!

The request needs to be an HTTP POST and not HTTP GET because it contains JSON payload.
request.Method = "GET"; should be request.Method = "POST";
That would be one of the issue(s). I am not sure if there is something else that is wrong, but try changing the request method to POST and try again.

JSON request is null when POSTing with WebClient but working with WebRequest, HttpClient

I have developed a Web API 2 application having the following Action
[Route("submit")]
[HttpPost]
public async Task<IHttpActionResult> Submit([FromBody]Request request)
{
if (request != null)
{
try
{
//Do Something
}
catch (Exception ex)
{
return this.InternalServerError(ex);
}
}
else
{
return this.BadRequest("Request is null");
}
}
In my client application, if I use WebClient to call this action, the request object in the action is coming null.
Using WebClient - Not Working
private static void CallWebAPIUsingWebClient(string dataString, TokenResponse token, string url)
{
using (WebClient client = new WebClient())
{
client.Headers[HttpRequestHeader.ContentType] = "application/json";
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers.Add("Username", "WebClient");
client.UploadString(url, "POST", dataString);
}
}
But, if i use WebRequest or HttpClient even, I am getting the complete request object in the action.
Using WebRequest - Working
private static void CallWebAPIUsingWebRequest(string dataString, TokenResponse token, string url)
{
var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.Accept = "application/json";
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
httpWebRequest.Headers.Add("Username", "WebRequest");
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(dataString);
streamWriter.Flush();
streamWriter.Close();
}
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
string result = streamReader.ReadToEnd();
}
}
Using HttpClient - Working
private async static Task CallWebAPIUsingHttpClient(string dataString, TokenResponse token, string url)
{
using (HttpClient client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.Add("Username", "HttpClient");
await client.PostAsync(url, new StringContent(dataString, System.Text.Encoding.UTF8, "application/json"));
}
}
The strange thing is, even WebClient was working till yesterday.
Please let me know why, in particular, WebClient is not working.
Thanks in advance.

Check if you are using HTTPPUT in the method below where as your original api controller action methods accepts HTTPPOST
private static void CallWebAPIUsingWebClient(string dataString, TokenResponse token, string middlewareUrl)
{
using (WebClient client = new WebClient())
{
client.Headers[HttpRequestHeader.ContentType] = "application/json";
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.Authorization] = token.TokenType.ToLowerInvariant() + " " + token.AccessToken;
client.Headers.Add("Username", "WebClient");
client.UploadString(middlewareUrl, "PUT", dataString);
}
}

C# HTTP Basic Authentication credentials being sent too early

I'm trying to log in to a server (REST API) which uses HTTP Basic Authentication. The request looks like this:
public JObject PerformLogin(string username, string password)
{
string html = string.Empty;
this.username = username;
this.password = password;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(auth_url_internal);
request.AllowAutoRedirect = true;
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
request.Method = "GET";
request.CookieContainer = cookies;
request.KeepAlive = true;
//request.ServicePoint.Expect100Continue = false;
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
request.Headers.Add("Accept-Language", "de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4");
request.PreAuthenticate = true;
request.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
string authInfo = username + ":" + password;
authInfo = Convert.ToBase64String(System.Text.Encoding.Default.GetBytes(authInfo));
request.Headers.Add("Authorization", "Basic " + authInfo);
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using (Stream stream = response.GetResponseStream())
using (StreamReader reader = new StreamReader(stream))
{
html = reader.ReadToEnd();
}
JObject jresponse = JObject.Parse(html);
sess_url_internal = jresponse["internalUrl"].ToString();
sess_url_public = jresponse["publicUrl"].ToString();
return jresponse;
}
which basically works, however the credentials are being sent too early.
First I used curl to see what the traffic looks like in detail and found a "Location:"-Header, which means that there is a redirect happening. In detail, the server redirects me from /api/rest/authenticate?version=1.0, which is the authentication URL (lets call it URL1), to /authenticationbasic/login?AlcApplicationUrl=/api/rest/authenticate%3fversion=1.0 (URL2).
However, Chrome sends the Credentials to URL2, where my program sends them to URL1 which is too early, because the server expects them at URL2, where my application doesn't send any and therefore gets a false return.
How can i change that behaviour?

So with the kind help of x... I figured out how to do it:
After the HttpWebResponse response = (HttpWebResponse)request.GetResponse(); simply add
if ((int)response.StatusCode == 302) // redirect
{
/*
Call the function recursively with the new URL, found in
response.Headers["Location"], in my case this would be:
*/
auth_url_internal = response.Headers["Location"];
return PerformLogin(username, password);
}

c# httpwebrequest for basic authentication

I've been trying to use a username, password and access key to authenticate on a server, so that I could use its API. I used to following code:
string encoded = Convert.ToBase64String(Encoding.Default.GetBytes("myusername:mypass"));
string api = "blahblahblahblabh";
string auth_string = "Basic " + encoded;
string _post_string = "username=myusername&password=mypass&grant_type=password";
HttpWebRequest request = WebRequest.Create(https_website) as HttpWebRequest;
request.Accept = "application/xml";
request.Method = "POST";
request.ContentType = "application/x-www-forms-urlencoded";
request.ContentLength = _post_string.Length;
CookieContainer cookies = new CookieContainer();
request.CookieContainer = cookies;
request.Headers["Authorization"] = auth_string;
request.Headers["api-key"] = api;
request.ServerCertificateValidationCallback += (sender1, certificate, chain, sslPolicyErrors) => { return true; };
StreamWriter writer = new StreamWriter(request.GetRequestStream());
writer.Write(_post_string);
writer.Close();
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
StreamReader reader = new StreamReader(response.GetResponseStream());
string xml_response = reader.ReadToEnd();
reader.Close();
So I sent a POST request with the username/password and access_key and tried to receive a response. I always get "The remote server returned an error: (404) Not Found". If I add more to the string of the 1st line, for example if instead of "myusername:mypass" I use "myusername:mypass:password" I get an error 401 (Unauthorized). I don't understand why this is happening...

HttpWebReponse Cookies Not Setting For Redirect

I'm trying to do the following:
send a GET request to fetch a login page (which prompts for username, password, and sets a cookie)
build a POST request that sends the cookie from #1 and a body of the username/password (this returns a Set-Cookie and redirects to the website's landing page for logged in users)
My trouble is with the 302 redirect. The webserver is returning a 302 with a Set-Cookie, but when the HttpWebRequests auto-redirects, it doesn't pass along the now updated cookie. To get around that, I'm trying to set .AllowAutoRedirect = false, saving the cookies in a CookieCollection, then building a 3rd HTTP request: a GET to the final 302 location. Unfortunately, I can't set the cookies on this request. I'm not sure why and it's driving me mad.
The HTTP requests are, in order, named request, postRequest, redirectRequest.
string loginGetUrl = "https://<..>/signin.htm";
string loginPostUrl = "https://<..>/j_acegi_security_check";
string loginRedirectUrl = "https://<..>/centraladmin/poslinks.htm";
string postData = String.Format("j_username={0}&j_password={1}", username, password);
CookieCollection cookies = new CookieCollection();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(loginGetUrl);
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
//Get the response from the server and save the cookies from the first request..
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
cookies = response.Cookies;
HttpWebRequest postRequest = (HttpWebRequest)WebRequest.Create(loginPostUrl);
postRequest.CookieContainer = new CookieContainer();
// Add the received Cookies from the HTTP Get
postRequest.CookieContainer.Add(cookies);
postRequest.Method = WebRequestMethods.Http.Post;
postRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
postRequest.AllowWriteStreamBuffering = false;
postRequest.ProtocolVersion = HttpVersion.Version11;
postRequest.AllowAutoRedirect = false;
postRequest.ContentType = "application/x-www-form-urlencoded";
byte[] byteArray = Encoding.ASCII.GetBytes(postData);
postRequest.ContentLength = byteArray.Length;
Stream newStream = postRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
newStream.Close();
HttpWebResponse postResponse = (HttpWebResponse)postRequest.GetResponse();
// Save the cookies from the POST login request, then send them on to the redirected URL
cookies = postResponse.Cookies;
HttpWebRequest redirectRequest = (HttpWebRequest)WebRequest.Create(loginRedirectUrl);
redirectRequest.CookieContainer = new CookieContainer();
// add cookies from POST
redirectRequest.CookieContainer.Add(cookies);
HttpWebResponse redirectResponse = (HttpWebResponse)redirectRequest.GetResponse();
At redirectRequest.CookieContainer.Add(cookies);, the cookies object contains the correct cookie. But when I look with Fiddler, I only see this info:
GET https://<...>/centraladmin/poslinks.htm HTTP/1.1
Host: host:port
I'm kind of banging my head on the wall at this point. Any suggestions? Am I referencing something wrong? Beware, I don't usually write C# code

I wasn't able to resolve this on my own, but did find a useful code snippet from this blog post by #malte-clasen. The code is on Github and I've attached it here for retention.
I removed the async components as it wasn't necessary in my code.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Text;
using System.Threading.Tasks;
namespace DracWake.Core
{
public class WebClient : IWebClient
{
private readonly CookieContainer _cookies = new CookieContainer();
private HttpWebRequest CreateRequest(Uri uri)
{
var request = HttpWebRequest.CreateHttp(uri);
request.AllowAutoRedirect = false;
request.CookieContainer = _cookies;
SetHeaders(request);
var defaultValidator = System.Net.ServicePointManager.ServerCertificateValidationCallback;
request.ServerCertificateValidationCallback =
(sender, certificate, chain, sslPolicyErrors) =>
certificate.Subject.Contains("O=DO_NOT_TRUST, OU=Created by http://www.fiddler2.com")
|| (certificate.Subject == "CN=DRAC5 default certificate, OU=Remote Access Group, O=Dell Inc., L=Round Rock, S=Texas, C=US")
|| (defaultValidator != null && defaultValidator(request, certificate, chain, sslPolicyErrors));
return request;
}
private async Task<string> DecodeResponse(HttpWebResponse response)
{
foreach (System.Net.Cookie cookie in response.Cookies)
{
_cookies.Add(new Uri(response.ResponseUri.GetLeftPart(UriPartial.Authority)), cookie);
}
if (response.StatusCode == HttpStatusCode.Redirect)
{
var location = response.Headers[HttpResponseHeader.Location];
if (!string.IsNullOrEmpty(location))
return await Get(new Uri(location));
}
var stream = response.GetResponseStream();
var buffer = new System.IO.MemoryStream();
var block = new byte[65536];
var blockLength = 0;
do{
blockLength = stream.Read(block, 0, block.Length);
buffer.Write(block, 0, blockLength);
}
while(blockLength == block.Length);
return Encoding.UTF8.GetString(buffer.GetBuffer());
}
public async Task<string> Get(Uri uri)
{
var request = CreateRequest(uri);
var response = (HttpWebResponse) await request.GetResponseAsync();
return await DecodeResponse(response);
}
private void SetHeaders(HttpWebRequest request)
{
request.Accept = "text/html, application/xhtml+xml, */*";
request.UserAgent = "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)";
request.ContentType = "application/x-www-form-urlencoded";
request.Headers[HttpRequestHeader.AcceptLanguage] = "en-US,en;q=0.8,de-DE;q=0.5,de;q=0.3";
request.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
request.Headers[HttpRequestHeader.CacheControl] = "no-cache";
}
public async Task<string> Post(Uri uri, byte[] data)
{
var request = CreateRequest(uri);
request.Method = "POST";
request.GetRequestStream().Write(data, 0, data.Length);
var response = (HttpWebResponse) await request.GetResponseAsync();
return await DecodeResponse(response);
}
}
}
The DecodeResponse resolved my problem.