We Keep Coding

What is the role of my machine's Application pool in Windows service development when consuming third party rest services [duplicate]

I am working on Windows Service in visual studio 2017. In the rest api's call, getting exceptions while debugging code. Sometimes first 2 3 calls working after that getting exceptions.
System.Net.WebException: 'The remote server returned an error: (503)
Server Unavailable.'
The remote server returned an error: (429)
Unable to connect to the remote server
When calling same api's from Postman, getting response successfully.
This is my code
private void timer1_Tick(object sender, ElapsedEventArgs e)
{
WriteToFile("timer1_Tick method called..");
try
{
string jsonString = "";
string jsonstring2 = "";
string prodfetchurl = HOST;
var req = WebRequest.Create(prodfetchurl) as HttpWebRequest;
req.Method = "GET";
InitializeRequest(req);
req.Accept = MIME_TYPE;
//System.Threading.Thread.Sleep(5000);
var response = (HttpWebResponse)req.GetResponse();
WriteToFile("First service called...");
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
jsonString = responseReader.ReadToEnd();
}
var deserialsseobj = JsonConvert.DeserializeObject<ProductList>(jsonString).Products.Where(i => i.Failed > 0).ToList();
foreach (var a in deserialsseobj)
{
var pid = a.ID;
string url = FailedDevicesUrl + pid.Value + "/failed";
var req2 = WebRequest.Create(url) as HttpWebRequest;
req2.Method = "GET";
InitializeRequest(req2);
req2.Timeout = 300000;
req2.Accept = MIME_TYPE;
var response1 = (HttpWebResponse)req2.GetResponse();
Stream responsestream2 = response1.GetResponseStream();
WriteToFile("Second service called...");
if (response1.StatusCode == HttpStatusCode.OK)
{
StreamReader responsereader1 = new StreamReader(responsestream2);
jsonstring2 = responsereader1.ReadToEnd();
}
var output = JsonConvert.DeserializeObject<List<FailedDeviceList>>(jsonstring2); // Will get List of the Failed devices
List<int> deviceids = new List<int>();
Reprocessdata reproc = new Reprocessdata();
Reprocessdata.DeviceId rprod = new Reprocessdata.DeviceId();
reproc.ForceFlag = true;
reproc.ProductID = pid.Value;
foreach (var dd in output)
{
rprod.ID = dd.DeviceId;
reproc.DeviceIds.Add(rprod);
}
// Reprocess the Product in Devices
var req3 = WebRequest.Create(ReprocessUrl) as HttpWebRequest;
req3.Method = "POST";
InitializeRequest(req3);
req3.Accept = MIME_TYPE;
req3.Timeout = 300000;
req3.ContentType = "application/json";
using (StreamWriter writer = new StreamWriter(req3.GetRequestStream()))
{
string json = new JavaScriptSerializer().Serialize(reproc);
writer.Write(json);
writer.Close();
}
System.Threading.Thread.Sleep(5000);
var response5 = (HttpWebResponse)req3.GetResponse();
WriteToFile("Third service called...");
if (response5.StatusCode == HttpStatusCode.OK)
{
string result;
using (StreamReader rdr = new StreamReader(response5.GetResponseStream()))
{
result = rdr.ReadToEnd();
}
}
}
response.Close();
}
catch (Exception ex)
{
WriteToFile("Simple Service Error on: {0} " + ex.Message + ex.StackTrace);
}
}
Methods used in above code
protected override void OnStart(string[] args)
{
base.OnStart(args);
timer1 = new System.Timers.Timer();
timer1.Interval = 60000; //every 1 min
timer1.Elapsed += new System.Timers.ElapsedEventHandler(timer1_Tick);
timer1.Enabled = true;
WriteToFile("Service has started..");
}
public void InitializeRequest(HttpWebRequest request)
{
request.Headers.Add("aw-tenant-code", API_TENANT_CODE);
request.Credentials = new NetworkCredential(USER_NAME, PASSWORD);
request.KeepAlive = false;
request.AddRange(1024);
}
When I contacted service provide they said everything fine from there side. Is this my code is buggy or windows service not reliable? How can I fix this issue?
Note: All APIS are working fine from Angular application using Visual Studio Code. It means my code is not working.
Edit1: Three below services I am using from this document of VMware.
private const string HOST = "https:host/api/mdm/products/search?";
private const string FailedDevicesUrl = "https:host/api/mdm/products/";
private const string ReprocessUrl = "https:host/api/mdm/products/reprocessProduct";

Response http code 429 indicates that you sending too many requests on target web service.
This means service you trying to send requests has a policies that blocks some requests by request-per-time limit.
Also I admit that external service can be manually configured to throw 403 code in specific cases that you can't know about. If that, this information can be explained in external service documentation... or not :)
What you can do with this?
Fit in limitations
You can make detailed research what limits target webservice has and set up your code to fit in this limitations. For example if service has limitation for receiving only one request per 10 minutes - you must set up your timer to send one request each 10 or more minutes. If documentation not provide such information - you can test it manually by finding some patterns with external service responses.
Use proxy
Every limitation policy based on information about requests senders. Usually this information consists of IP address of sender only. This means if you send 2 requests from two different IP addresses - limitation policy will perceive that like 2 different computers sending these requests. So you can find/buy/rent some proxy IP addresses and send requests through there on target web server.
How to connect through proxy in C# using WebRequest you can see in this answer.
Negotiate with external service provider
If you have possibility to communicate with external service developers or help center, you can ask their to reduce limitations for your IP address (if it static) or provide some mechanisms to avoid limitation policy for you. If for some reason they cannot provide this opportunity, at least you can ask detailed information about limitations.
Repetition mechanism
Some times 503 error code that is outer exception you received may be caused by service unavailable. It means that server can be under maintenance or temporary overloaded. So you can write repetition mechanism to make continious sending requests to server until it'll be accessible.
Polly library may help you with repetition mechanism creation

The inner error of that 503 is:
The remote server returned an error: (429)
HTTP 429 indicates too many requests. Maybe your upstream server can’t process all requests sent.
This can happen when you reached rate limiting / throttling value if you’re calling a third party API.
UPDATE
As per page 28 in the API docs, you could configure throttling when creating a new API. Check if the throttling is too small or maybe turn off the throttling and see if that could fix the error?

Sending HTTP request from Netduino

I have Netduino Plus and I need it to send Http requests to my server. I'm not a guru in C#, I've never tried it before, so I copy/paste code from internet and try to make it works. But even after several hours I can't get it work.
using System;
using System.IO;
using System.Net;
using System.Text;
using SecretLabs.NETMF.Hardware;
using SecretLabs.NETMF.Hardware.NetduinoPlus;
namespace NetduinoPlusApplication5
{
public class Program
{
static void Main()
{
var request = WebRequest.Create("http://example.com?variable=1");
request.Method = "GET";
var result = request.GetResponse();
}
}
}
What am I doing wrong?

You are executing a GET request so I think you want to get the response body from the server. In this case you have to use :
Stream respStream = resp.GetResponseStream();
instead of simple GetResponse(). In this way, you can read on the stream the response body.
Paolo.

Retrying POST request loses body c# winforms

I faced with the issue that after retrying the request my POST data got lost somehow.
Code sample below. (Please note that request.timeout = 1 set for testing purposes to reproduce the behavior shown in the code below):
//post_data_final getting
private void request_3()
{
for(int i=1; i<=5; i++)
{
byte[] byteArray = Encoding.ASCII.GetBytes(post_data_final);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(site_URI);
request.Method = "POST";
//some headers info
request.Timeout = 1;
request.ContentLength = byteArray.Length;
using (Stream os = request.GetRequestStream())
{
os.Write(byteArray, 0, byteArray.Length);
}
try
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
//some code about response
}
catch (WebException wex)
{
if (wex.Status == WebExceptionStatus.Timeout)
{
continue;
}
//some additional checks
}
}
}
The magic is that first request (until Request timeout error) goes well. Further requests are going without POST data, but content length is counted properly (i.e. stays the same as in previous request).
Updated:
post_data_final getting is separate function. It is not used (except byteArray) or changed in request_3() function.
Request works fine if it got into for loop and Timeout exception has not occured. So if I just put my request into for loop it will do particular number of valid requests. As soon as I'm getting Timeout exception, the next request will be without POST data.
Source code is edited for those who thinks that recursion is a bad idea. The edited code still doesn't work.
Any suggestions are appreciated

I cant find anything wrong in your code, so provide mode details, as the comments mentioned.
private void request_3()
{
bool sendData = true;
int numberOfTimeOuts = 0;
// The follwing only needs to be done only once, unless you alter post_data_final after each timeout.
byte[] dataToSend = Encoding.ASCII.GetBytes(post_data_final);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(site_URI);
using (Stream outputStream = request.GetRequestStream())
outputStream.Write(dataToSend, 0, dataToSend.Length);
// request.TimeOut = 1000 * 15; would mean 15 Seconds.
while(sendData && numberOfTimeOuts < MAX_NUMBER_OF_TIMEOUTS)
{
try
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if(response != null)
processResponse(response);
else
{
//You should handle this case aswell.
}
sendData = false;
}
catch(WebException wex)
{
if (wex.Status == WebExceptionStatus.Timeout)
numberOfTimeOuts++;
else
throw;
}
}
}

The issue was because of using Fiddler2 - analogue of Wireshark (i.e. intercepting traffic tool).
Requested site uses https protocol. For debugging purposes I installed Fiddler2 and Fiddler2 certificate to be able to see all incoming and outcoming responses. For some magic reason when I turned off Fiddler2 and added some additional logging into console, i figured out that requests are appeared to be valid (i.e. POST body data still exists after first request).
So with Fiddler2 code above doesn't work while we're having Timeout exception. Without Fiddler2 everything works fine under the same circumstances and using the same code.
I didn't dig deep into Fiddler2, but seems to me issue could be only with compatibility of VS2010 and internal proxy for Error Codes (taking into account that using point 2 under "Updates" area (The Fiddler2 was also used there) for success codes (i.e. 2xx - 3xx) worked fine)
Thanks everyone for getting attention into this.

.NET service responds 500 internal error and "missing parameter" to HttpWebRequest POSTS but test form works fine

I am using a simple .NET service (asmx) that works fine when invoking via the test form (POST). When invoking via a HttpWebRequest object, I get a WebException "System.Net.WebException: The remote server returned an error: (500) Internal Server Error." Digging deeper, reading the WebException.Response.GetResponseStream() I get the message: "Missing parameter: serviceType." but I've clearly included this parameter.
I'm at a loss here, and its worse that I don't have access to debug the service itself.
Here is the code being used to make the request:
string postData = String.Format("serviceType={0}&SaleID={1}&Zip={2}", request.service, request.saleId, request.postalCode);
byte[] data = (new ASCIIEncoding()).GetBytes(postData);
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.Timeout = 60000;
httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.ContentLength = data.Length;
using (Stream newStream = httpWebRequest.GetRequestStream())
{
newStream.Write(data, 0, data.Length);
}
try
{
using (response = (HttpWebResponse)httpWebRequest.GetResponse())
{
if (response.StatusCode != HttpStatusCode.OK)
throw new Exception("There was an error with the shipping freight service.");
string responseData;
using (StreamReader responseStream = new StreamReader(httpWebRequest.GetResponse().GetResponseStream(), System.Text.Encoding.GetEncoding("iso-8859-1")))
{
responseData = responseStream.ReadToEnd();
responseStream.Close();
}
if (string.IsNullOrEmpty(responseData))
throw new Exception("There was an error with the shipping freight service. Request went through but response is empty.");
XmlDocument providerResponse = new XmlDocument();
providerResponse.LoadXml(responseData);
return providerResponse;
}
}
catch (WebException webExp)
{
string exMessage = webExp.Message;
if (webExp.Response != null)
{
using (StreamReader responseReader = new StreamReader(webExp.Response.GetResponseStream()))
{
exMessage = responseReader.ReadToEnd();
}
}
throw new Exception(exMessage);
}
Anyone have an idea what could be happening?
Thanks.
UPDATE
Stepping through the debugger, I see the parameters are correct. I also see the parameters are correct in fiddler.
Examining fiddler, I get 2 requests each time this code executes. The first request is a post that sends the parameters. It gets a 301 response code with a "Document Moved Object Moved This document may be found here" message. The second request is a GET to the same URL with no body. It gets a 500 server error with "Missing parameter: serviceType." message.

It seems like you found your problem when you looked at the requests in Fiddler. Taking an excerpt from http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html:
10.3.2 301 Moved Permanently
The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible.
.....
Note: When automatically redirecting a POST request after
receiving a 301 status code, some existing HTTP/1.0 user agents
will erroneously change it into a GET request.
Here's a couple options that you can take:
Hard-code your program to use the new Url that you see in the 301 response in Fiddler
Adjust your code to retrieve the 301 response, parse out the new Url from the response, and build a new response with the new Url.
The latter option would be ideal if you're dealing with user-based input on the Url (like a web browser), since you don't know where the user is going to want your program to go.

Web server: reading http request from stream

Greetings!
I've been fooling around with C# (again) and now i've stucked with simple HTTP web server implementation. Honestly, i don't want to get along with HTTP specification - i just need to write a very tiny (read as simple) HTTP web server. And i've encouraged this problem: client sends request to the server and then server parses it, runs some actions, builds response and sends it back to client. That seems to be obvious (for me at least).
Here's what i've got so far:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.IO;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
TcpListener listener = new TcpListener(IPAddress.Any, 80);
listener.Start();
Socket sock = listener.AcceptSocket();
try
{
Stream s = new NetworkStream(sock);
s.ReadTimeout = 300;
StreamReader reader = new StreamReader(s);
StreamWriter writer = new StreamWriter(s);
writer.AutoFlush = true;
Console.WriteLine("Client stream read:\r\n");
string str = "none";
while (sock.Connected && !reader.EndOfStream && str.Length > 0) // here's where i'm stuck
{
str = reader.ReadLine();
Console.WriteLine("{0} ({1})", str, str.Length);
}
Console.WriteLine("Sending response...\r\n");
{
string response = "<h1>404: Page Not Found</h1>";
writer.WriteLine("HTTP / 1.1 404 Not Found");
writer.WriteLine("Content-Type: text/html; charset=utf-8");
writer.WriteLine("Content-Length: {0}", response.Length);
writer.WriteLine("\r\n{0}", response);
}
Console.WriteLine("Client: over\r\n");
s.Close();
sock.Close();
}
catch (Exception e)
{
Console.WriteLine("Exception: {0}\r\nTrace: \r\n{1}", e.Message, e.StackTrace);
}
Console.ReadKey();
}
}
}
But i've met an "underwater stone": i'm reading request via input stream, so input data flow will be terminated when client close page in his browser (let's talk about the most obvious actions, excluding curl, w3 and other "geek-stuff").
So, the question is: how to determine request' end? E.g. when should i stop reading request data and start sending response?

Why not use HttpListener? You can have a simple HTTP server in 5 lines of code.

This Wikipedia article explains pretty succinctly about the request message format.
The request line and headers must all end with <CR><LF> (that is, a carriage return followed by a line feed). The empty line must consist of only <CR><LF> and no other whitespace. In the HTTP/1.1 protocol, all headers except Host are optional.
Basically, watch for a blank line after the headers and/or a potential message body.
According to the HTTP specification, you can use certain headers to determine if there is a message body:
The presence of a message-body in a request is signaled by the inclusion of a Content-Length or Transfer-Encoding header field in the request's message-headers.

Another option is this one:
http://webserver.codeplex.com/
even if you don't want to use it, you can stole ideas because it implements the complete request lifecycle.