System.Threading.Task not starting in production environment - c#

I have a c# windows service which is doing various tasks. Its working perfectly on my local system but as soon as I start it on my product server, its doesn't perform a particular task on it.
This is how my service is structured:
public static void Execute()
{
try
{
// .... some work ....
foreach (DataRow dr in dt.Rows)
{
string cc = dr["ccode"].ToString();
Task objTask = new Task(delegate { RequestForEachCustomer(cc); });
objTask.Start();
}
}
catch (Exception ex)
{
// Logging in DB + Text File
}
}
public static void RequestForEachCustomer(object cc)
{
try
{
// .... some work ....
foreach (DataRow dr in dt.Rows)
{
WriteLog("RequestForEachCustomer - Before Task");
Task objTask = new Task(delegate { RequestProcessing(dr); });
objTask.Start();
WriteLog("RequestForEachCustomer - After Task");
}
}
catch (Exception ex)
{
// Logging in DB + Text File
}
}
public static void RequestProcessing(object dr)
{
try
{
WriteLog("Inside RequestProcessing");
// .... some work ....
}
catch (Exception ex)
{
// Logging in DB + Text File
}
}
Now what happens on the production server is that it logs the last entry in RequestForEachCustomer which is "RequestForEachCustomer - After Task" but it doesn't log the entry from RequestProcessing which mean the task is not starting at all. There are no exceptions in either database or text file.
There are no events logged in window's event viewer either. Also the service keeps working (if I insert another record in database, its processed by the service immediately so the service isn't stuck either. It just doesn't seem to process RequestProcessing task.)
I am baffled by this and it would be great if someone could point out the mistake I am making. Oh, btw did I forgot to mention that this service was working perfectly few days ago on the server and it is still working fine on my local PC.
EDIT :
WriteLog :
public static void WriteErrorLog(string Message)
{
StreamWriter sw = null;
try
{
lock (locker)
{
sw = new StreamWriter(AppDomain.CurrentDomain.BaseDirectory + "\\Logs\\LogFile.txt", true);
sw.WriteLine(DateTime.Now.ToString() + ": " + Message);
sw.Flush();
sw.Close();
}
}
catch (Exception excep)
{
try
{
// .... Inserting ErrorLog in DB ....
}
catch
{
throw excep;
}
throw excep;
}
}
I have also logged an entry on OnStop() something like "Service Stopped" and its logs every time I stop my service so the problem couldn't exist in WriteLog function.

I suggest you refactor your code as in this MSDN example. What bother me in your code is, you never wait for tasks to finish anywhere.
The following example starts 10 tasks, each of which is passed an index as a state object. Tasks with an index from two to five throw exceptions. The call to the WaitAll method wraps all exceptions in an AggregateException object and propagates it to the calling thread.
Source : Task.WaitAll Method (Task[])
This line from example might be of some importance :
Task.WaitAll(tasks.ToArray());

Related

Show exception on Hangfire Failed tasks

I'm testing Hangfire to use it in my application
the task is running as expected
now i make the task fail with try catch block the exception not showed in Failed tasks in dashboard
then i used log4net it works fine logging the exception to a text file but still not visible in hangfire dashboard what is the problem
Framework 4.7 - WebForms - log4net : 2.0.13.0 , hangfire.core : 1.1.1.0
installed by : Install-Package Hangfire_net40
Code
1- Startup.cs
public void Configuration(IAppBuilder app)
{
app.UseHangfire(config =>
{
config.UseSqlServerStorage("Hangfire_Blog");
config.UseServer();
});
}
2- web.Config
3- Global.ascx
private static readonly log4net.ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
private BackgroundJobServer _backgroundJobServer;
protected void Application_Start(object sender, EventArgs e)
{
log4net.Config.XmlConfigurator.Configure();
JobStorage.Current = new SqlServerStorage("Hangfire_Blog");
using (var connection = JobStorage.Current.GetConnection())
{
foreach (var recurringJob in connection.GetRecurringJobs())
{
RecurringJob.RemoveIfExists(recurringJob.Id);
}
}
//create an instance of BackgroundJobServer
_backgroundJobServer = new BackgroundJobServer();
//add your recurring job
RecurringJob.AddOrUpdate(() => Code(), "*/2 * * * *");
// RecurringJob. AddOrUpdate(() => Actualizacoes(), Cron.m();
}
[AutomaticRetry(Attempts = 10)]
public void Code()
{
try
{
string s = "";
int n = int.Parse(s);
}
catch (Exception ex)
{
log.Error(ex.Message);
}
}
for any one facing the same problem #madreflection Comment was a part of the answer
first Hangfire will not catch the exception in try catch block either remove try catch
or in the catch log the error the throw it to be catched by hangfire
but most important
[AutomaticRetry(Attempts = 10)]
that tag means that hangfire will not consider the task fail until it tries to execute 10 times and all the 10 times fail
hope some one find it useful
Thanks

WCF Service hosted in Windows Service hangs on stop

I have a WCF Service hosted in a Windows service as described here.
I have scheduled nightly restart of the service, but sometimes the restart fails and the service remains/hangs in Stopping state and the EXE process has to be killed manually. It looks likely that it hangs on line _ESSServiceHost.Close();, because nothing after that line is logged it the log file. It is possible but not very likely that the service gets the stop request when it is busy.
Moreover the underlying process cannot be killed because it is dependent on services.exe, so only server restart works.
What could be wrong with this approach?
protected override void OnStop()
{
try
{
if (_ESSServiceHost != null)
{
_ESSServiceHost.Close();
_ESSServiceHost = null;
//Never reaches the following line
Tools.LogInfo("Services stopped.");
}
}
catch (Exception ex)
{
Tools.LogError(ex.Message);
}
This is how I stop the service:
private bool StopService(ServiceController scESiftServer)
{
int i = 0;
if (scESiftServer.Status == ServiceControllerStatus.Running)
{
try
{
scESiftServer.Stop();
}
catch (Exception ex)
{
Tools.LogEvent("Exception ...");
return false;
}
while (scESiftServer.Status != ServiceControllerStatus.Stopped && i < 120)
{
Thread.Sleep(1000);
scESiftServer.Refresh();
i++;
}
}
if (scESiftServer.Status != ServiceControllerStatus.Stopped)
{
//This line gets executed
Tools.LogEvent("Failed within 120 sec...");
return false;
}
else
{
Tools.LogEvent("OK ...");
}
return true;
}
Could something like this help?
var task = Task.Run(() => _ESSServiceHost.Close(TimeSpan.FromSeconds(299)));
if (!task.Wait(TimeSpan.FromSeconds(300)))
{
_ESSServiceHost.Abort();
}
But _ESSServiceHost.Abort() should be called internally by the Close method if needed.
Target framework is 4.5, installed is .NET 4.7.2.
Found out that probably the service hangs after series of malformed requests. Expected record type 'Version', found '71'. etc.
I have found in the svclog file that my service hangs after series of malformed request that happen on Saturday and Sunday at approx. 5:15 AM. The error messages were Expected record type 'Version', found '71'., Error while reading message framing format at position 0 of stream (state: ReadingVersionRecord). But I could not find the cause of theese malformed request series, so I tried to fix the service to withstand the "attack".
I have modified the OnStop method as follows:
protected override void OnStop()
{
try
{
if (_ESSServiceHost != null)
{
Tools.LogInfo("Stopping ESService.");
var abortTask = Task.Run(() => _ESSServiceHost.Abort());
var closeTask = Task.Run(() => _ESSServiceHost.Close(TimeSpan.FromSeconds(300)));
try
{
if (_ESSServiceHost.State == CommunicationState.Faulted)
{
Tools.LogInfo("ESSServiceHost.State == CommunicationState.Faulted");
if (!abortTask.Wait(TimeSpan.FromSeconds(60)))
Tools.LogInfo("Failed to Abort.");
}
else
{
if (!closeTask.Wait(TimeSpan.FromSeconds(301)))
{
Tools.LogInfo("Failed to Close - trying Abort.");
if (!abortTask.Wait(TimeSpan.FromSeconds(60)))
Tools.LogInfo("Failed to Abort.");
}
}
}
catch (Exception ex)
{
Tools.LogException(ex, "ESSServiceHost.Close");
try
{
Tools.LogInfo("Abort.");
if (!abortTask.Wait(TimeSpan.FromSeconds(60)))
Tools.LogInfo("Failed to Abort.");
}
catch (Exception ex2)
{
Tools.LogException(ex2, "ESSServiceHost.Abort");
}
}
_ESSServiceHost = null;
Tools.LogInfo("ESService stopped.");
}
}
catch (Exception ex)
{
Tools.LogException(ex,"OnStop");
}
}
Today on Monday I have checked the svclog and the "attacks" with malformed request remained there but my service lived happily through it. So it seemed to be fixed. Moreover only:
Stopping ESService.
ESService stopped.
events were logged in my log file. No aborts etc. So I guess that putting the Close call on a separate thread fixed the problem but absolutely do not know why.

Terminating a function after a specified time

Im working on an EConnect integration Windows form app in C#. Im having a lot of trouble testing my connection string. Basically I give the user the option to change the connection string so I want to be able to test it. As far as I know EConnect doesnt have any built in functions that test the connection so I'm writing a pretty crude function to do it myself. Just for everyones information the connection string consists of a server name that the GP databases are located on and a database name.
The particulars of the actual test function I dont think are that important but the main issue is within that function I call an eConnect method called getEntity, this method uses the connection string and if the connection string is right it will pull information. If the database name is wrong the getEntity method will return an eConnect exception which is easy enough to catch, however if the server name is wrong the getEntity method will just spin and my app gets stuck.
Im trying to write something where I can possibly run my test function asynchronously and simultaneously check for a timeout or the econnect exception. This is where I get stuck, I cant for the life of me get it to work. Here is the first thing I tried (this is in my TestConnection method):
task = Task.Factory.StartNew(() => requester.GetEntity(GPCongfigSettings.GPConnectionString, myXmlDocument.OuterXml), token);
try
{
if (!task.Wait(timeOut, token))
{
Console.WriteLine("The server name is incorrect - task timed out");
return false;
}
}
catch (ThreadInterruptedException)
{
return false;
}
catch (AggregateException ae)
{
ae.Handle((x) =>
{
if (x is eConnectException) // This we know how to handle.
{
Console.WriteLine("Incorrect Database Name! -- " + x.Message);
return false;
}
return false; // Let anything else stop the application.
});
}
This would catch the cases where the server was wrong and if my econnect method would just time out. But it never caught the eConnect exception, Visual Studio would break the app and tell me I had an unhandled exception.
Here is what Im trying now, this is the full class I have for my form. Here I'm trying to use IAsyncResult and using a WaitHandle to check to see if the function completes or times out. This seems to work sometimes, it works for a correct string and for when the database is wrong, and sometimes it works for when the server is wrong, but once I test for a wrong server name it doesnt work correctly for anything else anymore. Is there something I'm missing or is there a better way to run the getentity method in TestGPConnection and check to see if it hasnt completed after a certain time period and if it hasnt kill that method and have the user reenter a server name?
public partial class UpdateGPConnection : Form
{
Task task;
AsyncCallback cb;
public delegate string startProcessToCall();
startProcessToCall sp2c;
public UpdateGPConnection()
{
InitializeComponent();
this.txtDatasourceName.Text = ConfigurationManager.AppSettings.Get("GPDataServer");
this.txtDatabaseName.Text = ConfigurationManager.AppSettings.Get("GPDatabase");
cb = new AsyncCallback(startProcessCallback);
sp2c = new startProcessToCall(TestGPConnection);
}
public void startProcessCallback(IAsyncResult iar)
{
startProcessToCall mc = (startProcessToCall)iar.AsyncState;
bool result = mc.EndInvoke(iar);
Console.WriteLine("Function value = {0}", result);
}
private void btnUpdate_Click(object sender, EventArgs e)
{
var config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);
config.AppSettings.Settings["GPDataServer"].Value = txtDatasourceName.Text.ToUpper();
config.AppSettings.Settings["GPDatabase"].Value = txtDatabaseName.Text.ToUpper();
config.Save(ConfigurationSaveMode.Modified);
ConfigurationManager.RefreshSection("appSettings");
GPCongfigSettings.GPConnectionString = #"data source=" + txtDatasourceName.Text.ToUpper() + ";initial catalog=" + txtDatabaseName.Text.ToUpper() + ";integrated security=SSPI;persist security info=False;packet size=4096";
IAsyncResult asyncResult = null;
asyncResult = sp2c.BeginInvoke(cb, null);
timer1.Enabled = true;
Thread.Sleep(0);
bool test = asyncResult.AsyncWaitHandle.WaitOne(15000);
if (test)
{
try
{
string testResult = sp2c.EndInvoke(asyncResult);
}
catch (Exception exc)
{
Console.WriteLine(exc.Message);
}
}
bool result = asyncResult.IsCompleted;
asyncResult.AsyncWaitHandle.Close();
this.Close();
}
public string TestGPConnection()
{
eConnectMethods requester = new eConnectMethods();
try
{
// Create an eConnect document type object
eConnectType myEConnectType = new eConnectType();
// Create a RQeConnectOutType schema object
RQeConnectOutType myReqType = new RQeConnectOutType();
// Create an eConnectOut XML node object
eConnectOut myeConnectOut = new eConnectOut();
// Populate the eConnectOut XML node elements
myeConnectOut.ACTION = 1;
myeConnectOut.DOCTYPE = "GL_Accounts";
myeConnectOut.OUTPUTTYPE = 2;
myeConnectOut.FORLIST = 1;
myeConnectOut.WhereClause = "(ACTNUMST = '99-9999-99-999')";
// Add the eConnectOut XML node object to the RQeConnectOutType schema object
myReqType.eConnectOut = myeConnectOut;
// Add the RQeConnectOutType schema object to the eConnect document object
RQeConnectOutType[] myReqOutType = { myReqType };
myEConnectType.RQeConnectOutType = myReqOutType;
// Serialize the eConnect document object to a memory stream
MemoryStream myMemStream = new MemoryStream();
XmlSerializer mySerializer = new XmlSerializer(myEConnectType.GetType());
mySerializer.Serialize(myMemStream, myEConnectType);
myMemStream.Position = 0;
// Load the serialized eConnect document object into an XML document object
XmlTextReader xmlreader = new XmlTextReader(myMemStream);
XmlDocument myXmlDocument = new XmlDocument();
myXmlDocument.Load(xmlreader);
string reqDoc = requester.GetEntity(GPCongfigSettings.GPConnectionString, myXmlDocument.OuterXml);
return "Correct Connection";
}
catch (eConnectException exc)
{
Console.WriteLine(exc.Message);
return "eConnect Excpetion";
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
return "Excpetion";
}
}
private void btnExit_Click(object sender, EventArgs e)
{
this.Close();
}
}
You are not handling the eConnectException. You are showing a message in the Console and then essentially rethrowing the exception by returning false.
If you do handle an exception then you should return true to avoid rethrowing it:
catch (AggregateException ae)
{
ae.Handle((x) =>
{
if (x is eConnectException) // This we know how to handle.
{
Console.WriteLine("Incorrect Database Name! -- " + x.Message);
}
return x is eConnectException; //rethrow anything that is not an eConnectException
});
}

Exception being thrown multiple times

Basically I am ting to catch any exception off a block of code, and fire said code one.
try {
CODE
catch (Exception e)
{
DO THIS ONCE
}
finally
{
CODE
}
In Depth
So I have been creating a TCP/SOCKET Server. Which can work with multiple clients. And send/recite (I/O) Data. It works well, and has been for a long time now. But I have found in my console that it says this:
This is bad because if it thinks the user disconnected twice it can create many problems. The way I know if a user has disconnected is by sending data to them every 200ms. And if there is a error then print they disconnected remove them from the client list, and disconnect there stream/tcp.
static bool currentlyUsing;
private static void PingClient(Object o)
{
if (!currentlyUsing)
{
if (clientsConnected.Count != 0)
{
foreach (Client c in clientsConnected)
{
try
{
c.tcp.Client.Blocking = false;
c.tcp.Client.Send(new byte[1], 0, 0);
}
catch (Exception e)
{
currentlyUsing = true;
Console.WriteLine("[INFO] Client Dissconnected: IP:" + c.ip + " PORT:" + c.port.ToString() + " Reason:" + e.Message);
clientsConnected.Remove(c);
c.tcp.Close();
break;
}
finally
{
currentlyUsing = false;
}
GC.Collect();
}
}
}
Is there a way to make it so it catches it only once, or catches it multiple times but only fires the code I want once?
If I understand your question correctly: you want to try to run the code on each iteration of the foreach block, and always run the finally code for each iteration, but only run the catch code once?
If so:
Before the foreach block, define:
bool caught = false;
And then after:
catch (Exception e)
{
if (caught == false)
{
caught = true;
...
}
}
I was making multiple timers. So it overlapped.

How to process files in directory concurrently in .net

I'm having issues processing files in parallel within a directory. I've read several similar questions and examples but I can't seem to find why my code causes exception.
My directory gets populated by other processes and will contain thousands of files at any one time. Each file has to be parsed and validated which takes time filesystem/network io etc. I need this step to be done in parallel, the rest has to be done serially.
Here's my code:
public void run()
{
XmlMessageFactory factory = new XmlMessageFactory();
DirectoryInfo dir = new DirectoryInfo(m_sourceDir);
Dictionary<string, int> retryList = new Dictionary<string, int>();
ConcurrentQueue<Tuple<XmlMsg,FileInfo>> MsgQueue = new
ConcurrentQueue<Tuple<XmlMsg,FileInfo>>();
//start worker to handle messages
System.Threading.ThreadPool.QueueUserWorkItem(o =>
{
XmlMsg msg;
Tuple<XmlMsg, FileInfo> item;
while (true)
{
if (!MsgQueue.TryDequeue(out item))
{
System.Threading.Thread.Sleep(5000);
continue;
}
try
{
msg = item.Item1;
/* processing on msg happens here */
handleMessageProcessed(item.Item2, ref retryList);
}
catch (Exception e)
{
//if this method is called it gives the
//exception below
handleMessageFailed(item.Item2, e.ToString());
}
}
}
);
while (true)
{
try
{
FileInfo[] files = dir.GetFiles(m_fileTypes);
Partitioner<FileInfo> partitioner = Partitioner.Create(files, true);
Parallel.ForEach(partitioner, f =>
{
try
{
XmlMsg msg = factory.getMessage(messageType);
try
{
msg.loadFile(f.FullName);
MsgQueue.Enqueue(new Tuple<XmlMsg, FileInfo>(msg, f));
}
catch (Exception e)
{
handleMessageFailed(f, e.ToString());
}
}
});
}
}
}
static void handleMessageFailed(FileInfo f, string message)
{
//Erorr here:
f.MoveTo(m_failedDir + f.Name);
//"The process cannot access the file because it is
//being used by another process."} System.Exception {System.IO.IOException}
}
Using ConcurrentQueue how can it end up attempting to access a file twice at the same time?
I have a test setup currently with 5000 files and this will happen at least once per run and on a different file each time. When I inspect the directory, the source file causing exception will have already been processed and is in the "processed" directory.
After a fair bit of head scratching the problem turned out to be annoyingly simple! What was happening was the parallel processing of the files in the directory was completing before the serial activity on the file, so the loop was restarting and re-adding some of the files to the Queue that were already in there.
For completeness here's the modified section of code:
while (true)
{
try
{
FileInfo[] files = dir.GetFiles(m_fileTypes);
Partitioner<FileInfo> partitioner = Partitioner.Create(files, true);
Parallel.ForEach(partitioner, f =>
{
try
{
XmlMsg msg = factory.getMessage(messageType);
try
{
msg.loadFile(f.FullName);
MsgQueue.Enqueue(new Tuple<XmlMsg, FileInfo>(msg, f));
}
catch (Exception e)
{
handleMessageFailed(f, e.ToString());
}
}
});
//Added check to wait for queue to deplete before
//re-scanning the directory
while (MsgQueue.Count > 0)
{
System.Threading.Thread.Sleep(5000);
}
}
}
I suspect a problem in XmlMsg.loadFile()
I think that you may have code like this in it:
public void loadFile(string filename)
{
FileStream file = File.OpenRead(filename);
// Do something with file
file.Close();
}
If an exception occurs in the "do something with file" part, the file won't be closed because file.Close() will never be executed. Then you'll get the "file in use" exception inside handleMessageFailed().
If so, the solution is to access the file in a using block as follows; then it will be closed even if an exception occurs:
public void loadFile(string filename)
{
using (FileStream file = File.OpenRead(filename))
{
// Do something with file
}
}
But assuming that this does turn out to be the problem, when you start using real files produced by external processes, you may have another issue if the external processes still have the files open when your worker threads try to process them.

Categories