I have a saga which checks on the status of an API calls every 30 seconds if the status returned back from the call is successful the saga is ended, if not
the saga waits 30 seconds and attempts again. If the API call has not returned a successful response within 60 minutes, the saga is timed out and ended.
I am having problems getting my 60 minute timeout to fire. The code I have is
public class MonitorSubmissionFeedSagaData: IContainSagaData
{
public Guid Id { get; set; }
public string Originator { get; set; }
public string OriginalMessageId { get; set; }
public bool TimeoutSet { get; set; }
[Unique]
public string JobId { get; set; }
}
public class MonitorSubmissionFeedSaga : Saga<MonitorSubmissionFeedSagaData>,
IAmStartedByMessages<MonitorFeedSubmissonCommand>,
IHandleMessages<StartCheckSubmissionCommand>,
IHandleTimeouts<MonitorSubmissionFeedSagaTimeout>
{
public const int SagaTimeoutInMinutes = 60;
public IEmpathyBrokerClientApi PostFileService { get; set; }
protected override void ConfigureHowToFindSaga(SagaPropertyMapper<MonitorSubmissionFeedSagaData> mapper)
{
mapper.ConfigureMapping<MonitorFeedSubmissonCommand>(x => x.JobId).ToSaga(saga => saga.JobId);
}
public void Handle(MonitorFeedSubmissonCommand message)
{
Data.JobId = message.JobId;
CheckTimeout();
Bus.Send(new StartCheckSubmissionCommand
{
JobId = Data.JobId
});
}
public void Handle(StartCheckSubmissionCommand message)
{
Log.Info("Saga with JobId {0} received", Data.JobId);
bool isCompleted = GetJobStatus(message.JobId);
while (isCompleted)
{
Thread.Sleep(30000);
isCompleted = GetJobStatus(message.JobId);
}
MarkAsComplete();
}
public void CheckTimeout()
{
RequestTimeout<MonitorSubmissionFeedSagaTimeout>(TimeSpan.FromMinutes(SagaTimeoutInMinutes));
}
public void Timeout(MonitorSubmissionFeedSagaTimeout state)
{
MarkAsComplete();
}
bool GetJobStatus(string jobId)
{
return false;
var status = PostFileService.GetJobIdStatus(jobId);
if (status.state == "FAILURE" || status.state == "DISCARDED")
{
return false;
}
return true;
}
}
Can anyone see where I am going wrong?
thanks
Your Saga should go idle. You're keeping it alive with a while loop. The timeout message arrives at some point and then you should check what should happen. Either another checkout or MarkAsComplete.
I wrote this in Notepad, so it might not compile. But it's to get an idea.
public class MonitorSubmissionFeedSagaData: IContainSagaData
{
public Guid Id { get; set; }
public string Originator { get; set; }
public string OriginalMessageId { get; set; }
[Unique]
public string JobId { get; set; }
public DateTime SagaStartTimeUtc { get; set; }
}
public class MonitorSubmissionFeedSaga : Saga<MonitorSubmissionFeedSagaData>,
IAmStartedByMessages<MonitorFeedSubmissonCommand>,
IHandleTimeouts<VerifyApiTimeOut>
{
public IEmpathyBrokerClientApi PostFileService { get; set; }
public void Handle(MonitorFeedSubmissonCommand message)
{
Data.JobId = message.JobId;
Data.SagaStartTimeUtc = DateTime.NowUtc;
CreateTimeoutRequest();
}
public void CreateTimeoutRequest()
{
RequestTimeout<VerifyApiTimeOut>(TimeSpan.FromSeconds(30));
}
public void Timeout(VerifyApiTimeOut state)
{
if (!GetJobStatus(Data.JobId) && DateTime.NowUtc < Data.SagaStartTimeUtc.AddMinutes(60))
{
CreateTimeoutRequest();
}
MarkAsComplete();
}
bool GetJobStatus(string jobId)
{
return false;
var status = PostFileService.GetJobIdStatus(jobId);
if (status.state == "FAILURE" || status.state == "DISCARDED")
{
return false;
}
return true;
}
}
Another comment could be that the Saga itself should not call out to external services. Preferably not even to some database. Delegate this to another service. Every 30 seconds, send out a message to another handler. This handler should call the WebService/WebAPI. When it can confirm that everything is correct, reply to the original Saga. When it's not correct, just let it be. The Saga will send out messages every 30 seconds to retry.
After 60 minutes, the Saga should stop sending messages and MarkAsComplete.
Related
I am trying make a post with my form along with a file. I tried see others posts but without result. If I make the post without the file, works just fine. English is not my native language, sorry for any mistakes.
This is the error:
This is my data:
My code -> .Net
Model:
public class EmailDto
{
public int Id { get; set; }
public string Name { get; set; }
public string Subject { get; set; }
public int? RecurrentDay { get; set; }
public DateTime? SendDate { get; set; }
public short SendHour { get; set; }
public int? GroupId { get; set; }
public int? RecipientTypeId { get; set; }
public int? ListTypeId { get; set; }
public bool SendForce { get; set; }
public string? Recipients { get; set; }
public bool Archived { get; set; }
public bool Active { get; set; }
public int IdTemplate { get; set; }
public TemplateDto Template { get; set; }
public int? IdRecurrence { get; set; }
public IFormFile? File { get; set; }
}
Controller:
[HttpPost("create")]
public async Task<ActionResult> CreateEmail([FromBody] EmailDto email)
Angular ->
Model:
export class EmailModel {
id!: number
idTemplate!: number
idRecurrence!: number
name!: string
subject!: string
active!: boolean
groupId!: number
recipientTypeId!: number
listTypeId! : number
recipients!: string
sendHour!: number
sendForce!: boolean
template!: TemplateModel
file!: File
}
email.component.ts:
onFileSelected(event: any) {
let fileList: FileList = event.target.files;
if (fileList.length > 0) {
this.file = fileList[0];
}
}
async onSubmit() {
if (this.file != null) {
this.email.file = this.file;
let headers = new HttpHeaders();
headers.append('Content-Type', 'multipart/form-data');
headers.append('Accept', 'application/json');
this.createEmail(this.email, headers);
} else {
this.createEmail(this.email);
}
}
createEmail(email: EmailModel, headers?: HttpHeaders) {
if (headers != null) {
this.EmailService.CreateEmail(email, headers).subscribe(() => {
this.isCreated = true;
}), (error: any) => {
console.log(error);
}
} else {
this.EmailService.CreateEmail(email).subscribe(() => {
this.isCreated = true;
}), (error: any) => {
console.log(error);
}
}
}
service:
CreateEmail(data: EmailModel, headers?: HttpHeaders) {
return this.http.post(`${this.emailApi}/create`, data, {headers: headers});
}
The last print:
If I understood your problem correctly, it would seem you are trying to send a JSON object body with a file and, on the .NET side, it can't deserialize it.
You could try to send the file as base64 instead of whatever you are attempting to send it as now:
async onSubmit()
{
if (this.file == null)
{
this.createEmail(this.email);
return;
}
// Create a file reader
const fileReader = new FileReader();
// Tell the file reader what to do once it loads
fileReader.onload = (event) =>
{
// which is to give us the base64 representation of the file
const base64File = event.target.result.toString();
// and assign it to the dto
this.email.file = base64File;
// before sending the request.
this.createEmail(this.email);
};
// Finally, read the file.
fileReader.readAsBinaryString(this.file);
}
Of course, you need to change file to a string in your DTOs.
You can then parse the base64 back to binary and do whatever you want with it. If you need IFormFile, then:
[HttpPost("create")]
public async Task<ActionResult> CreateEmail([FromBody] EmailDto email)
{
var byteArray = Convert.FromBase64String(email.File);
IFormFile file;
using(var memoryStream = new MemoryStream(byteArray))
{
file = new FormFile(memoryStream,
0,
byteArray.Length,
"someName",
"someFileName");
}
...
doWhatever
}
In my program I have a set of Links, let's say a million. The requirements are:
1) Process the links in parallel
2) Limit how many of these links will run in parallel.
In the example, we don't care about the execution order of the links but to ensure the process is finished with errors or without.
How can I properly ensure the above conditions?
class Program
{
private static void Main()
{
start_url_list();
}
static void start_url_list()
{
HashSet<string> urls = get_url_list();
Parallel.ForEach(urls, new ParallelOptions { MaxDegreeOfParallelism = 10 },
url =>
{
Console.WriteLine("Trying..:{0}", url);
var my_url = new Url_Class(url);
my_url.Start();
Console.WriteLine("Processing url is over");
//Do stuff with my_url properties
if (my_url.isError)
{
Console.WriteLine("ERROR:{0} msg:{1}",url,my_url.errorMsg);
}
else
{
Console.WriteLine("OK:{0} Size{1}", url, my_url.size);
}
});
}
}
public class Url_Class
{
public string url { get; private set; }
public string errorMsg { get; internal set; }
public bool isError { get; private set; }
public int size { get; private set; }
public Host(string url_str)
{
url = url_str;
isError = false;
}
public void Start()
{
//Long Process
}
}
I have 50 IMountCmd objects from one or more threads and drop them in a blocking collection. Each is executed and some get results or errors. They are put into a ConcurrentDictionary where I loop for ContainsKey and return the objects. Does this seems thread safe and correct way to process a blocking queue?
public class CmdGetAxisDegrees : IMountCommand
{
public long Id { get; }
public DateTime CreatedUtc { get; private set; }
public bool Successful { get; private set; }
public Exception Exception { get; private set; }
public dynamic Result { get; private set; }
private readonly AxisId _axis;
public CmdGetAxisDegrees(long id, AxisId axis)
{
Id = id;
_axis = axis;
CreatedUtc = HiResDateTime.UtcNow;
Successful = false;
Queues.AddCommand(this);
}
public void Execute(Actions actions)
{
try
{
Result = actions.GetAxisDegrees(_axis);
Successful = true;
}
catch (Exception e)
{
Successful = false;
Exception = e;
}
}
}
private static void ProcessCommandQueue(IMountCommand command)
{
command.Execute(_actions);
if (command.Id > 0)
{
_resultsDictionary.TryAdd(command.Id, command);
}
}
public static IMountCommand GetCommandResult(long id)
{
IMountCommand result;
while (true)
{
if (!_resultsDictionary.ContainsKey(id)) continue;
var success = _resultsDictionary.TryRemove(id, out result);
if (!success)
{
//log
}
break;
}
return result;
}
static Queues()
{
_actions = new Actions();
_resultsDictionary = new ConcurrentDictionary<long, IMountCommand>();
_commandBlockingCollection = new BlockingCollection<IMountCommand>();
Task.Factory.StartNew(() =>
{
foreach (var command in _commandBlockingCollection.GetConsumingEnumerable())
{
ProcessCommandQueue(command);
}
});
}
public interface IMountCommand
{
long Id { get; }
DateTime CreatedUtc { get; }
bool Successful { get; }
Exception Exception { get; }
dynamic Result { get; }
void Execute(Actions actions);
}
I am working with Sagas in ReBus and from my experience with NServiceBus you could reply to the original creator of the Saga to give updates, something like this:
Saga<>.ReplyToOriginator
I do not see an equivalent way of doing this with ReBus. Is there a way to do this, and if not what is a good pattern (other than the originator polling) I can use that will achieve the same thing? An example is trying to create a customer and the client wanting to know when it is created before trying to change it's address.
Here is a trivial example of the Customer scenario I quickly put together:
public class CreateCustomerSaga : Saga<CreateCustomerData>,
IAmInitiatedBy<CreateCustomerCommand>,
IHandleMessages<CustomerUniqunessCheckResult>
{
private readonly IBus _bus;
private readonly ICustomerResourceAccess _customerResourceAccess;
public CreateCustomerSaga(IBus bus, ICustomerResourceAccess customerResourceAccess)
{
_bus = bus;
_customerResourceAccess = customerResourceAccess;
}
public override void ConfigureHowToFindSaga()
{
Incoming<CustomerUniqunessCheckResult>(x => x.IsCustomerUnique).CorrelatesWith(y => y.CustomerId);
}
public void Handle(CreateCustomerCommand message)
{
Data.CustomerId = message.CustomerId;
Data.CustomerName = message.CustomerName;
_bus.Send(new CheckCustomerUniquenessCommand(message.CustomerId));
}
public void Handle(CustomerUniqunessCheckResult message)
{
if (message.IsCustomerUnique)
{
_customerResourceAccess.CreateCustomer(Data.CustomerId, Data.CustomerName);
// This is what seems to be missing from ReBus to reply to the original sender
_bus.?(new CustomerCreatedEvent(Data.CustomerId));
}
else
{
// This is what seems to be missing from ReBus to reply to the original sender
_bus.?(new CustomerAlreadExistsEvent(Data.CustomerId));
}
}
}
public class CustomerCreatedEvent
{
public Guid CustomerId { get; set; }
public CustomerCreatedEvent(Guid customerId)
{
CustomerId = customerId;
}
}
public class CustomerAlreadExistsEvent
{
public Guid CustomerId { get; set; }
public CustomerAlreadExistsEvent(Guid customerId)
{
CustomerId = customerId;
}
}
public class CustomerUniqunessCheckResult
{
public bool IsCustomerUnique { get; set; }
}
public class CheckCustomerUniquenessCommand
{
public CheckCustomerUniquenessCommand(Guid customerId)
{ }
}
public interface ICustomerResourceAccess
{
void CreateCustomer(Guid customerId, string customerName);
}
public class CreateCustomerCommand
{
public Guid CustomerId { get; set; }
public string CustomerName { get; set; }
}
public class CreateCustomerData : ISagaData
{
public CreateCustomerData()
{
Id = Guid.NewGuid();
}
public Guid CustomerId { get; set; }
public string CustomerName { get; set; }
public Guid Id { get; set; }
public int Revision { get; set; }
}
No, unfortunately there's no reply to originator function in Rebus' sagas at the moment. You can easily do it though, by storing the originator's endpoint when your saga is created like this (in all Handle methods of messages that can initiate the saga):
if (IsNew) {
Data.Originator = MessageContext.GetCurrent().ReturnAddress;
}
and then when you want to reply back to the originator:
bus.Advanced.Routing.Send(Data.Originator, new HelloThereMyFriend());
I've often thought about adding this to Rebus though, either as an extra field on ISagaData, or as an extra interface ISagaDataWithOriginator that you could optionally apply to your saga data, but I've never had the need (enough) myself.
If each sender is a separate (virtual) machine, like in my implementation, you could get a guaranteed reply to originator by using their machine unique ID or MAC address in the reply queue name, like this:
Bus = Configure.With(adapter)
.Transport(t => t.UseSqlServer(DbConfiguration.DatabaseConnectionString,
sInstanceId, sInstanceId + ".Error")
.EnsureTableIsCreated())
...
If each originator has a unique queue ID, the consumer can reply to originator simply by using Bus.Reply.
The unique machine id can be determined using System.Management:
string uuid = string.Empty;
ManagementClass mc = new System.Management.ManagementClass("Win32_ComputerSystemProduct");
if (mc != null)
{
ManagementObjectCollection moc = mc.GetInstances();
if (moc != null)
{
foreach (ManagementObject mo in moc)
{
PropertyData pd = mo.Properties["UUID"];
if (pd != null)
{
uuid = (string)pd.Value;
break;
}
}
}
}
Or by using the MAC address of the machine (our fallback code):
if (string.IsNullOrEmpty(uuid))
{
uuid = NetworkInterface.GetAllNetworkInterfaces()
.Where(ni => ni.OperationalStatus == OperationalStatus.Up)
.FirstOrDefault()
.GetPhysicalAddress().ToString();
}
Well i have created a job scheduler that has many capabilities; However i really want to use .Net 4.5 Async/Await feature with it in order to wait for a job to finish executing.
-Code
Scheduler.cs
public abstract class Scheduler
{
#region Fields && Properties
private readonly List<Job> _jobs = new List<Job>();
private readonly Random _rand = new Random();
private Job _currentlyExecutingJob;
private Thread _workingThread;
public bool? Parallel { get; private set; }
public DateTimeOffset NextExecutionTime { get; private set; }
public string ID { get; set; }
public abstract Task JobTrigger(Job job);
public abstract void UnobservedException(Exception exception, Job job);
#endregion
#region Ctor
protected Scheduler(bool parallel) { Parallel = parallel; }
#endregion
#region Fluent
public Scheduler Start()
{
if (Equals(_workingThread, null)) {
_workingThread = new Thread(ReviewJobs);
_workingThread.Start();
}
return this;
}
public Scheduler Stop()
{
if (!Equals(_workingThread, null)) {
_workingThread.Abort();
_workingThread = null;
}
return this;
}
#endregion
#region Events
private void ReviewJobs()
{
while (!Equals(_workingThread, null)) {
IEnumerable<Job> jobsToExecute = from job in _jobs
where job.NextExecutionTime <= DateTimeOffset.Now
orderby job.Priority
select job;
if (!jobsToExecute.Any()) {
Thread.Sleep(100);
continue;
}
try {
foreach (Job job in jobsToExecute) {
Job o = _currentlyExecutingJob = job;
if (Parallel != null && (bool)Parallel) {
JobTrigger(o);
} else {
JobTrigger(o).Wait();
}
if (!o.Repeat)
_jobs.Remove(o);
else if (o.Interval != null)
o.NextExecutionTime = DateTimeOffset.Now.Add((TimeSpan)(o.RandomizeExecution
? TimeSpan.FromSeconds(_rand.Next((int)((TimeSpan)o.Interval).TotalSeconds, ((int)((TimeSpan)o.Interval).TotalSeconds + 30)))
: o.Interval));
}
} catch (Exception exception) {
UnobservedException(exception, _currentlyExecutingJob);
} finally {
NextExecutionTime = (from job in _jobs
where job.NextExecutionTime <= DateTimeOffset.Now
orderby job.Priority
select job.NextExecutionTime).FirstOrDefault();
}
}
}
#endregion
#region Helper Methods
public Job GetJob(string id) { return _jobs.Find(job => job.ID == id); }
public Job GetJob(Job job) { return _jobs.Find(x => x == job); }
public IEnumerable<Job> GetAllJobs() { return _jobs; }
public void AddJob(Job job, bool overwrite = false)
{
Job existingJob = GetJob(job);
if (null != existingJob) {
if (overwrite) {
_jobs.RemoveAll(jobs => jobs == existingJob);
_jobs.Add(job);
} else {
_jobs.Add(job);
}
} else {
_jobs.Add(job);
}
}
public bool RemoveJob(string id)
{
Job existingJob = GetJob(id);
if (null != existingJob) {
return _jobs.Remove(existingJob);
}
return false;
}
public bool RemoveJob(Job job)
{
Job existingJob = GetJob(job);
if (null != existingJob) {
return _jobs.Remove(existingJob);
}
return false;
}
public void RemoveAllJobs() { _jobs.RemoveRange(0, _jobs.Count); }
#endregion
}
Job.cs
public class Job
{
public string ID { get; set; }
public TimeSpan Interval { get; private set; }
public DateTimeOffset NextExecutionTime { get; set; }
public int Priority { get; set; }
public bool Repeat { get; private set; }
public bool RandomizeExecution { get; set; }
public object Data { get; set; }
#region Fluent
public Job RunOnceAt(DateTimeOffset executionTime)
{
NextExecutionTime = executionTime;
Repeat = false;
return this;
}
public Job RepeatFrom(DateTimeOffset executionTime, TimeSpan interval)
{
NextExecutionTime = executionTime;
Interval = interval;
Repeat = true;
return this;
}
#endregion
}
-Usage
public class SchedulerUsage : Scheduler
{
public SchedulerUsage(bool parallel) : base(parallel) {
}
public override async Task JobTrigger(Job job)
{
switch (job.ID) {
//Handle Jobs
}
}
public override void UnobservedException(Exception exception, Job job)
{
//Handle Exceptions
}
/// <summary>
/// Example of adding job
/// </summary>
public void ExampleUsage()
{
Job job = new Job
{
ID = "ID",
Data = "ATTACH SOME DATA"
}.RunOnceAt(DateTimeOffset.Now.AddSeconds(7));
//Add the job... [HERE IS WHAT I WANT TO ACHIVE]
/*await*/base.AddJob(job);
//Start the scheduler...
base.Start();
}
}
Question: How to use async/await to await the execution of a job in my implementation above.
NB: I am sorry if my question seems to be a bit complicated but it is very necessary for me so please be patience with me.