We Keep Coding

SSIS Script Task using client.GetAsync(url) not waiting for response

I have an API call using client.GetAsync(url) within a SSIS script task but for some reason its not waiting for response from API and jumping back to the entry point for the script task which is public void Main(). Done some reading and found out that it might result in a deadlock for some reason but tried all the variations I could find to get it to work but with no luck. Something else that I don't understand is the exact same code is running on a webpage and that works perfect and waits for response from the api and continuing the flow.
Script Task entry point
The response here for payload is: ID =5, Status = WaitingForActivation, Method = "{null}", Result = "{Not yet computed}"
Here if in debug mode and moving the process back to go through the process again I noticed there 2 threads one current executing and the old one with the response I was expecting on the first call but not too sure what this means.
public void Main() {
// TODO: Add your code here
try {
PackageDetails packageInfo = new PackageDetails {
PackageNumber = 1234567891, Env = "Development", UserName = "USER"
};
var payload = API.ListHeadByPackAsync(packageInfo);
//var test = GetResponse();
Dts.TaskResult = (int) ScriptResults.Success;
} catch (Exception ex) {
System.Console.Write(ex.Message);
Dts.TaskResult = (int) ScriptResults.Failure;
}
}
API Call
public static class API {
public static async Task<PackageDetails> ListHeadByPackAsync(PackageDetails package) {
PackageDetails packageInfo = new PackageDetails();
try {
using(var client = new ApiClient(requestUrl, authToken)) {
var response = await client.GetAsync(); //-> not waiting for response
}
} catch (Exception err) {
switch (err.Message) {
//TODO:
}
}
return packageInfo;
}
}
Client
public class ApiClient: IDisposable {
private readonly TimeSpan _timeout;
private HttpClient _httpClient;
private HttpClientHandler _httpClientHandler;
private readonly string _baseUrl;
private readonly string _credentials;
//private const string MediaTypeXml = "application/csv";
public ApiClient(string baseUrl, string authToken, TimeSpan ? timeout = null) {
_baseUrl = baseUrl;
_credentials = Base64Encode(authToken);
_timeout = timeout ?? TimeSpan.FromSeconds(90);
}
public async Task < string > GetAsync() {
EnsureHttpClientCreated();
using(var response = await _httpClient.GetAsync(_baseUrl).ConfigureAwait(continueOnCapturedContext: false))
//-> after executing above line it will go straight to public void Main(), dose not wait for response
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
public void Dispose() {
_httpClientHandler?.Dispose();
_httpClient?.Dispose();
}
private void CreateHttpClient() {
_httpClientHandler = new HttpClientHandler {
AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip
};
_httpClient = new HttpClient(_httpClientHandler, false) {
Timeout = _timeout
};
if (!string.IsNullOrWhiteSpace(_baseUrl)) {
_httpClient.BaseAddress = new Uri(_baseUrl);
}
_httpClient.DefaultRequestHeaders.Add("Authorization", "Basic" + " " + _credentials);
}
private void EnsureHttpClientCreated() {
if (_httpClient == null) {
//ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
CreateHttpClient();
}
}
public static string Base64Encode(string token) {
var tokenBytes = System.Text.Encoding.UTF8.GetBytes(token);
return System.Convert.ToBase64String(tokenBytes);
}
}

Memory increase gradually after every HTTPClient Call in Xamarin iOS. GC collection is not effective

I have monitor the Application Allocation using Instrument. I have noticed that Memory is gradually increase. I just need release the memory which is used by HTTPClient.
I have tried using Dispose(),used using statement,used GC.collect; but those are not reducing the memory. It looks like not effective.
Below is code used in the program.
namespace MemoryManagementStudy
{
public partial class ViewController : UIViewController
{
protected ViewController(IntPtr handle) : base(handle)
{
}
NetworkStaticCall networkStaticCall;
public override void ViewDidLoad()
{
//lblStatusUpdate.Text = "Success Count: 0, Fail Count: 0 -- Last Update: " + DateTime.Now;
base.ViewDidLoad();
//string status = "";
networkStaticCall = new NetworkStaticCall();
networkStaticCall.init();
Timer timer = new Timer(1000 * 30);
timer.AutoReset = true;
timer.Elapsed += async (sender, e) => {
Console.WriteLine("Memory " + GC.GetTotalMemory(true));
await networkStaticCall.MakeSMWebService();
Console.WriteLine("Memory " + GC.GetTotalMemory(true));
};
timer.Start();
}
public override void DidReceiveMemoryWarning()
{
base.DidReceiveMemoryWarning();
}
}
class NetworkStaticCall
{
//int successCount = 0;
//int failCount = 0;
private static HttpClient httpClient;
private static HttpResponseMessage httpResponseMessage;
private StringContent dataContent;
private string xmlString;
private static HttpClientHandler httpClientHandler;
private NSUrlSessionHandler nsUrlSessionHandler;
public void init()
{
httpClientHandler = new HttpClientHandler();
httpClientHandler.Credentials = new NetworkCredential("userName", "Password"); ;
httpClient = new HttpClient(httpClientHandler);
httpClient.DefaultRequestHeaders.Accept.Clear();
dataContent = new StringContent("Xml String Content",
Encoding.UTF8, "text/xml");
}
public async Task<string> MakeSMWebService()
{
Console.WriteLine("Call Processing");
try
{
httpClient.DefaultRequestHeaders.Accept.Clear();
httpResponseMessage = await httpClient.PostAsync("url", dataContent);
Console.WriteLine("Call success");
xmlString = await httpResponseMessage.Content.ReadAsStringAsync();
//successCount++;
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
return "";
}
}
}
We have tried https://stackoverflow.com/a/27830926 but it is not working.
I have tried WebRequest but it still increasing
using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using System.Timers;
using UIKit;
namespace MemoryManagementStudy
{
public partial class ViewController : UIViewController
{
protected ViewController(IntPtr handle) : base(handle)
{
}
CustomWebRequest customWebRequest;
public override void ViewDidLoad()
{
customWebRequest = new CustomWebRequest();
lblStatusUpdate.Text = "Success Count: 0, Fail Count: 0 -- Last Update: " + DateTime.Now;
base.ViewDidLoad();
string status = "";
//networkStaticCall = new NetworkStaticCall();
//networkStaticCall.init();
//
Timer timer = new Timer(1000 * 60);
timer.AutoReset = true;
timer.Elapsed += async (sender, e) => {
status = await customWebRequest.Post();
InvokeOnMainThread(()=>{
lblStatusUpdate.Text =status + " -- Last Update: " + DateTime.Now;
});
GC.Collect();
};
timer.Start();
}
}
class CustomWebRequest{
string finalUrl = "URL";
//StreamReader reader;
int successCount = 0;
int failCount = 0;
async public Task<string> Post()
{
var request = WebRequest.Create(finalUrl);
request.Method = "GET";
request.Timeout = 10000;
try
{
using (var response = await request.GetResponseAsync())
{
using (var reader = new StreamReader(response.GetResponseStream()))
{
Console.WriteLine(reader.ReadToEnd());
}
}
successCount++;
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
failCount++;
}
finally
{
if(request != null){
request.Abort();
request = null;
}
}
return string.Format("Success Count: {0}, Fail Count: {1}", successCount, failCount);
}
}
}

Programmatically trigger listener.GetContext()

Is it possible to trigger the below code by using a trigger URL?
As opposed to triggering by visiting the URL in the browser.
var context = listener.GetContext();
Something like this?
var triggerURL = "http://www.google.ie/";
var request = (HttpWebRequest)WebRequest.Create(triggerURL);
Or is it possible to use a do while loop? I.E do create trigger while get context

Instead of using listener.GetContext(), I was able to satisfy my requirement by using listener.BeginGetContext(new AsyncCallback(ListenerCallback), listener) and listener.EndGetContext(result), utilising the Asynchronous call, GetAsync.
public static string RunServerAsync(Action<string> triggerPost)
{
var triggerURL = "";
CommonCode(ref triggerURL);
if (listener.IsListening)
{
triggerPost(triggerURL);
}
while (listener.IsListening)
{
var context = listener.BeginGetContext(new AsyncCallback(ListenerCallback), listener);
context.AsyncWaitHandle.WaitOne(20000, true); //Stop listening after 20 seconds (20 * 1000).
listener.Close();
}
return plateString;
}
private static async void TriggerURL(string url)
{
var r = await DownloadPage(url);
}
static async Task<string> DownloadPage(string url)
{
using (var client = new HttpClient())
{
using (var r = await client.GetAsync(new Uri(url)))
{
if (r.IsSuccessStatusCode)
{
string result = await r.Content.ReadAsStringAsync();
return result;
}
else
{
return r.StatusCode.ToString();
}
}
}
}
private static void ListenerCallback(IAsyncResult result)
{
try
{
HttpListener listener = (HttpListener)result.AsyncState;
// Use EndGetContext to complete the asynchronous operation.
HttpListenerContext context = listener.EndGetContext(result);
if (context != null)
{
plateString = ProcessRequest(context);
}
else
{
plateString = "No response received!";
}
}
catch (Exception ex)
{
NLogManager.LogException(ex);
}
}

Validating URLs from Multiple Threads and Updating DataGridView

I'm trying to validate URLs from Multiple threads and update a DataTable.
The validation works fine when a single thread is used
Works Fine--Single Thread
foreach (string url in urllist)
{
Boolean valid = CheckURL(url);
this.Invoke((MethodInvoker)delegate()
{
if (valid)
{
dt.Rows[counter][2] = "Valid";
validcount++;
}
else
{
dt.Rows[counter][2] = statusCode;
invalidcount++;
}
counter++;
});
}
But when i try to do this using multiple threads Some Valid URls are reported as Invalid and vice versa.
Multi-Threads -Not Working
Parallel.ForEach(urllist, ProcessUrl);
private void ProcessUrl(string url)
{
Boolean valid = CheckURL(url);
this.Invoke((MethodInvoker)delegate()
{
if (valid)
{
dt.Rows[counter][2] = "Valid";
validcount++;
}
else
{
dt.Rows[counter][2] = statusCode;
invalidcount++;
}
counter++;
});
}
Associated Method and Class
private Boolean CheckURL(string url)
{
using (MyClient myclient = new MyClient())
{
try
{
myclient.HeadOnly = true;
myclient.Headers.Add(HttpRequestHeader.UserAgent, "My app.");
//fine, no content downloaded
string s1 = myclient.DownloadString(url);
statusCode = null;
return true;
}
catch (WebException error)
{
if (error.Response != null)
{
HttpStatusCode scode = ((HttpWebResponse)error.Response).StatusCode;
if (scode != null)
{
statusCode = scode.ToString();
}
}
else
{
statusCode = "Unknown Error";
}
return false;
}
}
}
class MyClient : WebClient
{
public bool HeadOnly { get; set; }
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest req = base.GetWebRequest(address);
req.Timeout = 10000;
if (HeadOnly && req.Method == "GET")
{
req.Method = "HEAD";
}
return req;
}
}
What i'm i doing wrong ? Please advice..
UPDATE:
How i start the task -->
var ts = new CancellationTokenSource();
CancellationToken ct = ts.Token;
Task.Factory.StartNew(() =>
{
if (nameresfailcount > 10)
{
if (ct.IsCancellationRequested)
{
// another thread decided to cancel
Console.WriteLine("task canceled");
break;
}
}
//stuff
},ct).ContinueWith(task =>
{
_benchmark.Stop();
}

ServerSentEvents, HttpTaskAsyncHandler and waiting

I'm trying to create a quite simple notifications system (don't want to use SignalIR or something else). I have the following testing code:
Client side:
var source = new EventSource('/notifications.axd');
source.onopen = function () {
Console.log("Connection open");
};
source.onerror = function () {
Console.log("Connection error");
};
source.onmessage = function (event) {
Console.log("Message: " + event.data);
};
Server side:
public class NotificationMessage {
public NotificationMessage() {
Id = Guid.NewGuid().ToString();
}
public string Id { get; private set; }
}
public class NotificationsHandler : HttpTaskAsyncHandler {
private const string CONTENT_TYPE = "text/event-stream";
private sealed class NotificationItem {
public ConcurrentQueue<NotificationMessage> Messages;
public CancellationTokenSource CancellationTokenSource;
}
private static readonly ConcurrentDictionary<string, NotificationItem> _tasks =
new ConcurrentDictionary<string, NotificationItem>();
public static void Notify(string hostId, string userId, NotificationMessage message) {
NotificationItem item;
if (!_tasks.TryGetValue(string.Format("{0}|{1}", hostId, userId), out item)) {
return;
}
var tokenSource = item.CancellationTokenSource;
item.Messages.Enqueue(message);
item.CancellationTokenSource = new CancellationTokenSource();
tokenSource.Cancel();
}
public override async Task ProcessRequestAsync(HttpContext context) {
HttpRequest request = context.Request;
NotificationItem item = _tasks.GetOrAdd(
string.Format("{0}|{1}", request.Url.Host, CsSession.Data.CurrentUser.Id),
k => new NotificationItem {
Messages = new ConcurrentQueue<NotificationMessage>(),
CancellationTokenSource = new CancellationTokenSource()
}
);
HttpResponse response = context.Response;
response.ContentType = CONTENT_TYPE;
response.CacheControl = "no-cache";
response.ContentEncoding = Encoding.UTF8;
response.AppendHeader("connection", "keep-alive");
response.BufferOutput = false;
bool supportsAsyncFlush = response.SupportsAsyncFlush;
bool shouldPing = true;
while (response.IsClientConnected) {
try {
NotificationMessage message = null;
if ((!item.Messages.IsEmpty && item.Messages.TryDequeue(out message)) || shouldPing) {
response.Write(string.Format("data:{0}\n\n", message == null ? "{}" : JsonMapper.Serialize(message)));
if (supportsAsyncFlush) {
await Task.Factory.FromAsync(response.BeginFlush, response.EndFlush, null);
} else {
response.Flush();
}
}
} catch (Exception) {
break;
}
var delay = Task.Delay(15000, item.CancellationTokenSource.Token);
await delay;
shouldPing = delay.Status == TaskStatus.RanToCompletion;
}
}
}
The problem is: the above doesn't works. I have two issues:
1) When the client connects, I receive an empty packet (that's ok). Then, if I don't enqueue any messages, after awaiting the Task.Delay, the loop tries to write an empty message again, but I don't know where. The response.Write line never returns (and nothing is being received on the client).
2) If I write to the queue, for some reason the connection is dropped. If I put a breakpoint on the line after the await delay, that line is never executed (while my logic says otherwise :) ). If I cancel the token, the delay task should quit, but it seems it is aborting the whole handler??