We Keep Coding

What is the role of my machine's Application pool in Windows service development when consuming third party rest services [duplicate]

I am working on Windows Service in visual studio 2017. In the rest api's call, getting exceptions while debugging code. Sometimes first 2 3 calls working after that getting exceptions.
System.Net.WebException: 'The remote server returned an error: (503)
Server Unavailable.'
The remote server returned an error: (429)
Unable to connect to the remote server
When calling same api's from Postman, getting response successfully.
This is my code
private void timer1_Tick(object sender, ElapsedEventArgs e)
{
WriteToFile("timer1_Tick method called..");
try
{
string jsonString = "";
string jsonstring2 = "";
string prodfetchurl = HOST;
var req = WebRequest.Create(prodfetchurl) as HttpWebRequest;
req.Method = "GET";
InitializeRequest(req);
req.Accept = MIME_TYPE;
//System.Threading.Thread.Sleep(5000);
var response = (HttpWebResponse)req.GetResponse();
WriteToFile("First service called...");
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
jsonString = responseReader.ReadToEnd();
}
var deserialsseobj = JsonConvert.DeserializeObject<ProductList>(jsonString).Products.Where(i => i.Failed > 0).ToList();
foreach (var a in deserialsseobj)
{
var pid = a.ID;
string url = FailedDevicesUrl + pid.Value + "/failed";
var req2 = WebRequest.Create(url) as HttpWebRequest;
req2.Method = "GET";
InitializeRequest(req2);
req2.Timeout = 300000;
req2.Accept = MIME_TYPE;
var response1 = (HttpWebResponse)req2.GetResponse();
Stream responsestream2 = response1.GetResponseStream();
WriteToFile("Second service called...");
if (response1.StatusCode == HttpStatusCode.OK)
{
StreamReader responsereader1 = new StreamReader(responsestream2);
jsonstring2 = responsereader1.ReadToEnd();
}
var output = JsonConvert.DeserializeObject<List<FailedDeviceList>>(jsonstring2); // Will get List of the Failed devices
List<int> deviceids = new List<int>();
Reprocessdata reproc = new Reprocessdata();
Reprocessdata.DeviceId rprod = new Reprocessdata.DeviceId();
reproc.ForceFlag = true;
reproc.ProductID = pid.Value;
foreach (var dd in output)
{
rprod.ID = dd.DeviceId;
reproc.DeviceIds.Add(rprod);
}
// Reprocess the Product in Devices
var req3 = WebRequest.Create(ReprocessUrl) as HttpWebRequest;
req3.Method = "POST";
InitializeRequest(req3);
req3.Accept = MIME_TYPE;
req3.Timeout = 300000;
req3.ContentType = "application/json";
using (StreamWriter writer = new StreamWriter(req3.GetRequestStream()))
{
string json = new JavaScriptSerializer().Serialize(reproc);
writer.Write(json);
writer.Close();
}
System.Threading.Thread.Sleep(5000);
var response5 = (HttpWebResponse)req3.GetResponse();
WriteToFile("Third service called...");
if (response5.StatusCode == HttpStatusCode.OK)
{
string result;
using (StreamReader rdr = new StreamReader(response5.GetResponseStream()))
{
result = rdr.ReadToEnd();
}
}
}
response.Close();
}
catch (Exception ex)
{
WriteToFile("Simple Service Error on: {0} " + ex.Message + ex.StackTrace);
}
}
Methods used in above code
protected override void OnStart(string[] args)
{
base.OnStart(args);
timer1 = new System.Timers.Timer();
timer1.Interval = 60000; //every 1 min
timer1.Elapsed += new System.Timers.ElapsedEventHandler(timer1_Tick);
timer1.Enabled = true;
WriteToFile("Service has started..");
}
public void InitializeRequest(HttpWebRequest request)
{
request.Headers.Add("aw-tenant-code", API_TENANT_CODE);
request.Credentials = new NetworkCredential(USER_NAME, PASSWORD);
request.KeepAlive = false;
request.AddRange(1024);
}
When I contacted service provide they said everything fine from there side. Is this my code is buggy or windows service not reliable? How can I fix this issue?
Note: All APIS are working fine from Angular application using Visual Studio Code. It means my code is not working.
Edit1: Three below services I am using from this document of VMware.
private const string HOST = "https:host/api/mdm/products/search?";
private const string FailedDevicesUrl = "https:host/api/mdm/products/";
private const string ReprocessUrl = "https:host/api/mdm/products/reprocessProduct";

Response http code 429 indicates that you sending too many requests on target web service.
This means service you trying to send requests has a policies that blocks some requests by request-per-time limit.
Also I admit that external service can be manually configured to throw 403 code in specific cases that you can't know about. If that, this information can be explained in external service documentation... or not :)
What you can do with this?
Fit in limitations
You can make detailed research what limits target webservice has and set up your code to fit in this limitations. For example if service has limitation for receiving only one request per 10 minutes - you must set up your timer to send one request each 10 or more minutes. If documentation not provide such information - you can test it manually by finding some patterns with external service responses.
Use proxy
Every limitation policy based on information about requests senders. Usually this information consists of IP address of sender only. This means if you send 2 requests from two different IP addresses - limitation policy will perceive that like 2 different computers sending these requests. So you can find/buy/rent some proxy IP addresses and send requests through there on target web server.
How to connect through proxy in C# using WebRequest you can see in this answer.
Negotiate with external service provider
If you have possibility to communicate with external service developers or help center, you can ask their to reduce limitations for your IP address (if it static) or provide some mechanisms to avoid limitation policy for you. If for some reason they cannot provide this opportunity, at least you can ask detailed information about limitations.
Repetition mechanism
Some times 503 error code that is outer exception you received may be caused by service unavailable. It means that server can be under maintenance or temporary overloaded. So you can write repetition mechanism to make continious sending requests to server until it'll be accessible.
Polly library may help you with repetition mechanism creation

The inner error of that 503 is:
The remote server returned an error: (429)
HTTP 429 indicates too many requests. Maybe your upstream server can’t process all requests sent.
This can happen when you reached rate limiting / throttling value if you’re calling a third party API.
UPDATE
As per page 28 in the API docs, you could configure throttling when creating a new API. Check if the throttling is too small or maybe turn off the throttling and see if that could fix the error?

Post rest web api working from Postman but not returning any response from code in C# project

There is a POST rest api which used to work from code before. But recently it is broken and is not returning any response. However if I try to call the api from the Postman, then it works fine.
In what way can I debug this to find the root cause of the issue ?
Following is the C# code which I am using to call this post rest api
public async Task SaveToServerAsync()
{
string filePath = #"<filePath>";
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
// tried this line of code from another SO answer, but this didn't work either
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("https://<server name>/");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", "d2ebf9aefbaa416adcd0");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Add("Accept", "*/*");
client.DefaultRequestHeaders.Add("Connection", "keep-alive");
using (var fileStream = new FileStream(filePath, FileMode.Open))
{
var content = new MultipartFormDataContent();
content.Add(new StreamContent(fileStream), "file", filePath);
content.Add(new StringContent("e8d002f9-f381-44c2-bce0-13416929f14d"), "Id");
try
{
var response = await client.PostAsync("<rest api end point>", content).ConfigureAwait(false);
if (response.IsSuccessStatusCode)
{
Debug.Write("Response received");
}
}
catch (Exception ex)
{
Debug.Write("Exception occured");
Debug.Write(ex.Message);
}
finally
{
}
}
}
}
It always goes to the exception block with exception as "The task was cancelled"
Not sure how can I debug it when it anyway works from the Postman.

So the problem was related to ExpectContinue header which goes as true by default. Somehow server was not handling it properly and client was waiting for continue (100) message for indefinite time.
For the time being manually setting this header to be false worked for us:
httpClient.DefaultRequestHeaders.ExpectContinue = false;

A HttpRequest Redirect Exception in UWP Project

The code goes below
public static async Task<string> getForwardUrl(string url)
{
try
{
HttpClient client = new HttpClient();
HttpRequestMessage forwardRequest = new HttpRequestMessage();
forwardRequest.RequestUri = new Uri(url);
HttpResponseMessage Message = await client.SendAsync(forwardRequest);
return Message.RequestMessage.RequestUri.OriginalString;
}
catch (Exception ex)
{
WriteLine(ex.Message);
}
//...
}
When I run this in a uwp project, exception occurs. The message of this exception shows that the redirection request would change the safe connection to an unsafe one(After that , I checked the URL of the login page , It's https ,but the page after I logged in is http).
I find a similar question, he recommends using Windows.Web.Http instead of System.Net.Http but I get the same error message.
Thanks for your reply
EDIT:
The URL is: https://tinyurl.com /57muy (remove the space) or short a http url with tinyurl.com! The problem only occurs with a shortet http side!
Error: An error occurred while sending the request. Innermessage: Error message not found for this error

According to your description, I'd suppose you are developing a UWP app. And as you've mentioned, we got the exception here because the redirection request would change the safe connection to an unsafe one. To solve this problem, we can turn off auto-redirect and do the redirection by ourselves.
For example:
public static async Task<string> getForwardUrl(string url)
{
var handler = new System.Net.Http.HttpClientHandler();
handler.AllowAutoRedirect = false;
var client = new System.Net.Http.HttpClient(handler);
var response = await client.GetAsync(url);
if (response.StatusCode == System.Net.HttpStatusCode.Redirect || response.StatusCode == System.Net.HttpStatusCode.Moved)
{
return response.Headers.Location.AbsoluteUri;
}
return url;
}

Making Http Call from c#

There is a classic asp application which makes calls to .shtml files using AspHttp.Conn. It makes the request by appending all the input and also reads the response by reading the values in the response by length.
Here is the example
strMessage= "test.shtml"
Set HttpObj = Server.CreateObject("AspHTTP.Conn")
HttpObj.Url = url & strMessage
HttpObj.PostData = "testarea=" & strRequestData
HttpObj.TimeOut = 60
HttpObj.RequestMethod = "post"
strResponseData = HttpObj.GetURL
Response.Write Mid(strResponseData,3,1)
Response.Write Mid(strResponseData,4,3)
If I need to rewrite this, what will be the best way to do this. I will be using MVC and will be rewriting the UI. what will be the best approach to make httpcall from c#?. The backend to which the request will be sent will not be changed. Please suggest.

Using .Net Framework 4.5 you can make request as
public static async Task<int> HtmlLoadAsync(string url/*, bool addUserAgent = false*/)
{
try
{
var client = new HttpClient();
//if (addUserAgent) OPTIONAL
//{
// client.DefaultRequestHeaders.UserAgent.ParseAdd(UserAgent);
//}
//client.Timeout = TimeOut;
var response = client.GetStringAsync(url); //here you can change client method according to your required outpu
var urlContents = await response;
// process urlContents now
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
return 0;
}
Now call it as
private async void Process()
{
await HtmlLoadAsync("http://....");
}
Note: You must have to add reference System.Net.Http

401 Unauthorized on SECOND HttpClient/HttpWebRequest call

I have a application that uses the SharePoint 2010 REST API.
In the process of creating an Item there are multiple request done after each other:
1 Call: Getting Items from List: Succes
2 Call: Create Item: 401 Unauthorized
This is the same if I do it like this:
1 Call: Create Item: Succes
2 Call: Delete Item: 401 Unauthorized
What I know is that my functions work separately they DON'T work when they are called after each other.
When I close the application (Windows Phone 8.1 app) after creating a item and when restarted try to delete the item it works.
First I thought it had to do with the way I handle my fields so I changed them to NULL in a finally statement but that didn't work.
public async Task<bool> CreateNewItem(NewItem myNewItem)
{
try
{
StatusBar statusBar = await MyStatusBar.ShowStatusBar("Creating new List Item.");
//Retrieving Settings from Saved file
mySettings = await MyCredentials.GetMySettings();
myCred = new NetworkCredential(mySettings.UserName, mySettings.Password, mySettings.Domain);
using (var handler = new HttpClientHandler { Credentials = myCred })
{
HttpClient client = new HttpClient(handler);
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
NewItem newItem = myNewItem;
var jsonObject = JsonConvert.SerializeObject(newItem);
HttpResponseMessage response = await client.PostAsync(new Uri(baseUrl + listNameHourRegistration), new StringContent(jsonObject.ToString(), Encoding.Unicode, "application/json"));
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
response.EnsureSuccessStatusCode();
string responseMessage = await response.Content.ReadAsStringAsync();
client.Dispose();
if (responseMessage.Length > 0)
return true;
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
return false;
}
finally
{
request = null;
response = null;
myCred = null;
mySettings = null;
}
return false;
}

Just run into the same problem.
Anyway, the 2nd request does not follow the same authentication procedure. Even if you initialize a new HttpClient object. I sniffed the HTTP traffic.
After the 1st request I am doing another with different credentials. This is also ending in a 401. I am really confused...
Seems the NTLM Handshake stucks at the 2nd of 6 steps
http://www.innovation.ch/personal/ronald/ntlm.html
Edit:
You may want to use the CSOM.
http://social.msdn.microsoft.com/Forums/office/en-US/efd12f11-cdb3-4b28-a9e0-32bfab71a419/windows-phone-81-sdk-for-sharepoint-csom?forum=sharepointdevelopment

While I still don't know what the actual problem is, at least I found a workaround: Use the WebRequest class instead of HttpClient.

I was running into this same error when I realized I was adding the headers each time I was calling the endpoint. Hopefully this will help someone.
Instead I initialized the HttpClient instance in my class constructor and set the headers there. Also I learned it is better practice to only use 1 instance instead of recreating with "using" (See this article https://www.aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/)
I'm invoking CallApiAsync from another class in a loop.
Here's my final solution:
class ApiShared
{
private HttpClient client;
public ApiShared() {
client = new HttpClient();
client.DefaultRequestHeaders.Add("x-api-key", ConfigurationManager.AppSettings["ApiKey"]);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
public async Task<ApiResponse_Root> CallApiAsync(string endpoint)
{
// Make API call
Uri endpointUri = new Uri(endpoint);
var stringTask = client.GetStringAsync(endpointUri);
var data = JsonConvert.DeserializeObject<ApiResponse_Root>(await stringTask);
return data;
}
}

On a windows machine you can resolve this with this registry setting change:
Go to the following Registry entry:
Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
Now add a new DWORD to the Lsa folder called: DisableLoopBackCheck and set this to 1

I see that this question has been posted long back. But I don't see a correctly working solution posted yet to this thread.
I faced exactly the same issue where the next requests kept on failing returning me 401 UnAuthorized.
I figured out using fiddler that from SECOND request onwards, there was a Cookie added to the request which was possibly a result of Set-Cookie response sent by the server along with first response.
So here's how I tackled the situation - Make UseCookies false:
new HttpClientHandler { Credentials = myCred, UseCookies = false }
This should resolve your issue. Hope this helps someone who's looking for a solution to a similar issue.