I am experimenting with producer and consumer using AWS Kinesis and the issue is the consumer keeps receiving the first message (or record) that we produced though we have changed the data object sent multiple times . Additionally we have tried multiple ShardIteratorType and none have worked. Latest produces no results, all others produce the same original record.
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Threading.Tasks;
using Amazon;
using Amazon.Internal;
using Amazon.Kinesis;
using Amazon.Kinesis.Model;
using BenchmarkRuleSetModel.Models;
using MongoDB.Driver;
using Newtonsoft.Json;
namespace ConsoleApp7
{
internal class Program
{
private static AmazonKinesisClient _client;
private static string _streamName;
static async Task ReadFromStream()
{
var kinesisStreamName = _streamName;
var describeRequest = new DescribeStreamRequest
{
StreamName = kinesisStreamName,
};
var describeResponse = await _client.DescribeStreamAsync(describeRequest);
var shards = describeResponse.StreamDescription.Shards;
foreach (var shard in shards)
{
var iteratorRequest = new GetShardIteratorRequest
{
StreamName = kinesisStreamName,
ShardId = shard.ShardId,
ShardIteratorType = ShardIteratorType.AT_TIMESTAMP,
Timestamp = DateTime.MinValue
};
var iteratorResponse = await _client.GetShardIteratorAsync(iteratorRequest);
var iteratorId = iteratorResponse.ShardIterator;
while (!string.IsNullOrEmpty(iteratorId))
{
var getRequest = new GetRecordsRequest
{
ShardIterator = iteratorId, Limit = 10000
};
var getResponse = await _client.GetRecordsAsync(getRequest);
var nextIterator = getResponse.NextShardIterator;
var records = getResponse.Records;
if (records.Count > 0)
{
Console.WriteLine("Received {0} records. ", records.Count);
foreach (var record in records)
{
var json = Encoding.UTF8.GetString(record.Data.ToArray());
Console.WriteLine("Json string: " + json);
}
}
iteratorId = nextIterator;
}
}
}
private static async Task<string> Produce()
{
var data = new
{
Message = "Hello world!",
Author = "Amir"
};
//convert to byte array in prep for adding to stream
var oByte = Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(data));
using (var ms = new MemoryStream(oByte))
{
//create put request
var requestRecord = new PutRecordRequest
{
StreamName = _streamName,
PartitionKey = Guid.NewGuid().ToString(),
Data = ms
};
//list name of Kinesis stream
//give partition key that is used to place record in particular shard
//add record as memorystream
//PUT the record to Kinesis
var response = await _client.PutRecordAsync(requestRecord);
return response.SequenceNumber;
}
}
static void Main(string[] args)
{
_client = new AmazonKinesisClient("ExampleKey", "ExampleSecret", RegionEndpoint.EUWest2);
_streamName = "SomeStream";
Produce().Wait();
ReadFromStream().Wait();
}
}
}
First of all, as I have debugged your code, I noticed that it loops infinitely in the inner loop (while (!string.IsNullOrEmpty(iteratorId))) and never loops over all the shards in your stream (assuming you have >1). The reason is explained in https://docs.aws.amazon.com/streams/latest/dev/troubleshooting-consumers.html#getrecords-returns-empty - because the producer never called MergeShards or SplitShards, they remain open, thus NextShardIterator will never be NULL.
This is why you only ever see records put on the first shard (or at least I did when running your code) - you must read from shards in parallel.
As far as your usage pattern goes, you're using:
ShardIteratorType = ShardIteratorType.AT_TIMESTAMP,
Timestamp = DateTime.MinValue
By this, you're essentially telling Kinesis "give me all the records in the stream from the beginning of time" (or at least as far as the retention period reaches). That's why you keep seeing the same old records in addition to new ones (again, that's what I saw when I ran your code).
A GetRecords[Async] call does not actually remove records from the stream (see https://stackoverflow.com/a/25741304/4940707). The correct way of using Kinesis is to move checkpoint-to-checkpoint. If the consumer was to persist the SequenceNumber from the last record read and then restart as such:
ShardIteratorType = ShardIteratorType.AT_SEQUENCE_NUMBER,
StartingSequenceNumber = lastSeenSequenceNumber
Then you'd see only newer records.
Related
I want to send an email at a specified time in Sendgrid.
according to this document.and this question,I created a sample console application in VS2022 with c#
There is no problem sending Email. However, Email is still delivered to target even after canceling
Program.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Text;
using System.Threading.Tasks;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using SendGrid;
using SendGrid.Helpers.Mail;
namespace sendmailtest
{
internal class Program
{
private static string batchId = "";
private static string apikey = "my sendgrid apikey";
static void Main(string[] args)
{
SendEmail().Wait();
Console.WriteLine("Cancel scheduled emails. yes or not?");
var result = Console.ReadLine();
if (result == "yes")
{ //Cancel scheduled sends while inputting "yes" in console
NotSendEmail().Wait();
}
}
static async Task SendEmail()
{
var client = new SendGridClient(apikey);
var response = await client.RequestAsync(method: SendGridClient.Method.POST, urlPath: "mail/batch");
if (response.StatusCode == HttpStatusCode.Created)
{ // get batchID
JObject joResponse = JObject.Parse(response.Body.ReadAsStringAsync().Result);
batchId = (((Newtonsoft.Json.Linq.JValue)joResponse["batch_id"]).Value).ToString();
}
var dtNow = DateTime.Now;
//Automatically send emails 30 seconds after execution starts
var dtSend = dtNow.AddSeconds(30);
long sendAtUnixTime = new DateTimeOffset(dtSend).ToUnixTimeSeconds();
var msg = new SendGridMessage()
{
From = new EmailAddress("info#sample.com", "sample"),
Subject = "scheduling sending mail test",
PlainTextContent = "and easy to do anywhere, even with C#",
HtmlContent = "<strong>and easy to do anywhere, even with C#</strong>",
BatchId = batchId,
SendAt = sendAtUnixTime
};
msg.AddTo(new EmailAddress("targetEmailAddress", "targetName"));
await client.SendEmailAsync(msg);
//Email sent successfully
}
static async Task NotSendEmail()
{
var client = new SendGridClient(apikey);
string data = "{\"batch_id\":\"" + batchId + "\",\"status\": \"cancel\"}";
await client.RequestAsync(method: SendGridClient.Method.POST, urlPath: "user/scheduled_sends", requestBody: data);
}
}
}
How can achieve this problem. Please provide the solutions for it.
Thanks in advance.
ToUnixTimeSeconds is based on UTC. Your dtNow is in local time. If you set it with DateTime.UtcNow you've probably fixed the issue, assuming you're living in a country with a negative timezone (as such it's probably sending the email as soon as you create the call since the scheduled date is surpassed).
https://learn.microsoft.com/en-us/dotnet/api/system.datetimeoffset.tounixtimeseconds?view=net-7.0
Going by the example of https://docs.sendgrid.com/for-developers/sending-email/stopping-a-scheduled-send your code seems fine otherwise.
New Info:
I thought I would paste this in full as I can not seem to find any samples on the web of a c# solution for StarLink so hopefully anyone else looking for something may find this helpful and may contribute.
My New Proto File - (partial) - I took the advise of Yuri below. Thanks for the direction here. I was able to I have been using this tool and it has brought a lot of insight but I am still stuck on the c# side of the solution. I am an old VB.Net developer though I have done a bunch in c# I am by no means savvy in it and am probably missing something so simple. Again, any insight would be awesome. I can not post the full proto here as stack has char limit on posts. this is the first bit with messages etc. I can post more if it helps but trying to keep it to the important part.
syntax = "proto3";
option csharp_namespace = "SpaceX.API.Device";
package SpaceX.API.Device;
service Device {
//rpc Handle (.SpaceX.API.Device.Request) returns (.SpaceX.API.Device.Response) {}
//rpc Stream (stream .SpaceX.API.Device.ToDevice) returns (stream .SpaceX.API.Device.FromDevice) {}
rpc Handle (Request) returns (Response);
rpc Stream (Request) returns (Response);
}
message ToDevice {
string message = 1;
}
message Request {
uint64 id = 1;
string target_id = 13;
uint64 epoch_id = 14;
oneof request {
SignedData signed_request = 15;
RebootRequest reboot = 1001;
SpeedTestRequest speed_test = 1003;
GetStatusRequest get_status = 1004;
AuthenticateRequest authenticate = 1005;
GetNextIdRequest get_next_id = 1006;
GetHistoryRequest get_history = 1007;
GetDeviceInfoRequest get_device_info = 1008;
GetPingRequest get_ping = 1009;
SetTrustedKeysRequest set_trusted_keys = 1010;
FactoryResetRequest factory_reset = 1011;
GetLogRequest get_log = 1012;
SetSkuRequest set_sku = 1013;
UpdateRequest update = 1014;
GetNetworkInterfacesRequest get_network_interfaces = 1015;
PingHostRequest ping_host = 1016;
GetLocationRequest get_location = 1017;
EnableFlowRequest enable_flow = 1018;
GetHeapDumpRequest get_heap_dump = 1019;
RestartControlRequest restart_control = 1020;
FuseRequest fuse = 1021;
GetPersistentStatsRequest get_persistent_stats = 1022;
GetConnectionsRequest get_connections = 1023;
FlushTelemRequest flush_telem = 1026;
StartSpeedtestRequest start_speedtest = 1027;
GetSpeedtestStatusRequest get_speedtest_status = 1028;
ReportClientSpeedtestRequest report_client_speedtest = 1029;
InitiateRemoteSshRequest initiate_remote_ssh = 1030;
SelfTestRequest self_test = 1031;
SetTestModeRequest set_test_mode = 1032;
DishStowRequest dish_stow = 2002;
DishGetContextRequest dish_get_context = 2003;
DishSetEmcRequest dish_set_emc = 2007;
DishGetObstructionMapRequest dish_get_obstruction_map = 2008;
DishGetEmcRequest dish_get_emc = 2009;
DishSetConfigRequest dish_set_config = 2010;
DishGetConfigRequest dish_get_config = 2011;
StartDishSelfTestRequest start_dish_self_test = 2012;
WifiSetConfigRequest wifi_set_config = 3001;
WifiGetClientsRequest wifi_get_clients = 3002;
WifiSetupRequest wifi_setup = 3003;
WifiGetPingMetricsRequest wifi_get_ping_metrics = 3007;
WifiGetDiagnosticsRequest wifi_get_diagnostics = 3008;
WifiGetConfigRequest wifi_get_config = 3009;
WifiSetMeshDeviceTrustRequest wifi_set_mesh_device_trust = 3012;
WifiSetMeshConfigRequest wifi_set_mesh_config = 3013;
WifiGetClientHistoryRequest wifi_get_client_history = 3015;
TransceiverIFLoopbackTestRequest transceiver_if_loopback_test = 4001;
TransceiverGetStatusRequest transceiver_get_status = 4003;
TransceiverGetTelemetryRequest transceiver_get_telemetry = 4004;
}
reserved 1025, 3011, 3014;
}
message SignedData {
bytes data = 1;
bytes signature = 2;
}
My New .cs
I have tried many things from Microsoft's examples to thing I can gather from other samples. I simply can not get it to work and am lost. Again, any insight would be amazing and hopefully helpful to others looking for a solution in c#. You will see my commented code of this I have been playing with. Basically I am attempting to achieve three things and have made some movement in one of them.
Goals:
1 - Use Server Reflection to discover services.
I think I got this one resolved with dot-net grpc.
2 - Simply want to check available methods under a service and potentially either check or generate a new .proto file in case things change. StaLink does not publish its proto schema so I assume it could change anytime without warning.
3 - Just run any one of the available methods. I have tried the GetDeviceInfoRequest but can not seem to construct the request message properly. I have not been able to get this accomplishe in the gRPCurl tool either. I can do it on the basic service shown by Microsoft of course but these methods seem to be more complex and I simply get all kinds of errors.
Again, any insight or assistance would be amazing. Thanks to any and all in advance.
New .cs File
using System;
using System.Diagnostics;
using System.Threading.Tasks;
using Grpc.Core;
using Grpc.Net.Client;
using Grpc.Reflection.V1Alpha;
using ServerReflectionClient = Grpc.Reflection.V1Alpha.ServerReflection.ServerReflectionClient;
using SpaceX.API.Device;
public class Program
{
static async Task Main(string[] args)
{
//SETUP CHANNEL AND CLIENT
using var channel = GrpcChannel.ForAddress("http://192.168.100.1:9200");
var client = new ServerReflectionClient(channel);
var StarLinkClient = new Device.DeviceClient(channel);
//using var call = StarLinkClient.StreamAsync(new ToDevice { Request = GetDeviceInfoRequest });
//await foreach (var response in call.ResponseStream.ReadAllAsync())
//var request = Device.GetDeviceInfoRequest;
//var reply = await StarLinkClient.HandleAsync(
// new Request {'"getDeviceInfo" : {} '});
//Console.WriteLine(reply.Message);
//=============================================SERVER REFLECTION=============================================================
Console.WriteLine("Calling reflection service:");
var response = await SingleRequestAsync(client, new ServerReflectionRequest
{
ListServices = "" // Get all services
});
Console.WriteLine("Services:");
foreach (var item in response.ListServicesResponse.Service)
{
Console.WriteLine("- " + item.Name);
Console.WriteLine();
var StarLink = item.Name;
//Console.WriteLine(StarLink.getStatus());
}
//=============================================SERVER REFLECTION=============================================================
Console.WriteLine("Press any key to exit...");
Console.ReadKey();
}
void setupchannel()
{
}
private static Task SingleRequestAsync(ServerReflectionClient client, Metadata metadata)
{
throw new NotImplementedException();
}
private static async Task<ServerReflectionResponse> SingleRequestAsync(ServerReflectionClient client, ServerReflectionRequest request)
{
using var call = client.ServerReflectionInfo();
await call.RequestStream.WriteAsync(request);
Debug.Assert(await call.ResponseStream.MoveNext());
var response = call.ResponseStream.Current;
await call.RequestStream.CompleteAsync();
return response;
}
}
Again, thanks in advance to anyone willing to assist here. Hopefully this helps others as well.
I'm getting a print queue for various printers installed locally by doing:
var queueName = "Myprinterqueue";
using (var ps = new PrintServer())
using (var pq = ps.GetPrintQueue(queueName))
{
pq.Refresh();
var status = pq.QueueStatus;
var jobs = pq.NumberOfJobs;
var averagePagesPerMinute = pq.AveragePagesPerMinute;
}
Status and NumberOfJobs are retrieved correctly. But many other print capabilities, eg. AveragePagesPerMinute, are always empty or 0.
Why is this?
Method One
_AcDb.Line oLine = new _AcDb.Line(ptStart, ptEnd);
AddToModelSpace("PLOT", oLine);
Where AddToModelSpace is:
public static void AddToModelSpace(string strLayer, _AcDb.Entity oEntity)
{
_AcAp.Document acDoc = _AcAp.Application.DocumentManager.MdiActiveDocument;
_AcDb.Database acCurDb = acDoc.Database;
_AcEd.Editor ed = acDoc.Editor;
using (_AcDb.BlockTable bt = acCurDb.BlockTableId.GetObject(_AcDb.OpenMode.ForRead) as _AcDb.BlockTable)
using (_AcDb.BlockTableRecord ms = bt[_AcDb.BlockTableRecord.ModelSpace].GetObject(_AcDb.OpenMode.ForWrite) as _AcDb.BlockTableRecord)
ms.AppendEntity(oEntity);
oEntity.Layer = strLayer;
oEntity.Dispose();
}
Method Two
// Get the current document and database
_AcAp.Document docActive = _AcAp.Application.DocumentManager.MdiActiveDocument;
_AcDb.Database docDB = docActive.Database;
// Start a transaction
using (_AcDb.Transaction acTrans = docDB.TransactionManager.StartTransaction())
{
// Open the Block table for read
_AcDb.BlockTable acBlkTbl;
acBlkTbl = acTrans.GetObject(docDB.BlockTableId,
_AcDb.OpenMode.ForRead) as _AcDb.BlockTable;
// Open the Block table record Model space for write
_AcDb.BlockTableRecord acBlkTblRec;
acBlkTblRec = acTrans.GetObject(acBlkTbl[_AcDb.BlockTableRecord.ModelSpace],
_AcDb.OpenMode.ForWrite) as _AcDb.BlockTableRecord;
// Create line
using (_AcDb.Line acLine = new _AcDb.Line(ptStart, ptEnd))
{
// Add the new object to the block table record and the transaction
acBlkTblRec.AppendEntity(acLine);
acTrans.AddNewlyCreatedDBObject(acLine, true);
}
// Save the new object to the database
acTrans.Commit();
}
I have used AddToModelSpace in my project so I hope it is fine!
Method Two is the way Autodesk recommends in the developer's documentation (you can read this section).
In Method One, you use the ObjectId.GetObject() method to open the BlockTable and the model space 'BlockTableRecord'. This method uses the top transaction to open object which means that there's an active transaction you should use to add the newly created entity. You can get it with Database.TransactionManager.TopTransaction. If you don't want to use a transaction at all, you have to use the "for advanced use only" ObjectId.Open() method.
A Method Three should be using some extension methods to be called from within a transaction. Here's a simplified (non error checking) extract of the ones I use.
static class ExtensionMethods
{
public static T GetObject<T>(
this ObjectId id,
OpenMode mode = OpenMode.ForRead,
bool openErased = false,
bool forceOpenOnLockedLayer = false)
where T : DBObject
{
return (T)id.GetObject(mode, openErased, forceOpenOnLockedLayer);
}
public static BlockTableRecord GetModelSpace(this Database db, OpenMode mode = OpenMode.ForRead)
{
return SymbolUtilityServices.GetBlockModelSpaceId(db).GetObject<BlockTableRecord>(mode);
}
public static ObjectId Add (this BlockTableRecord owner, Entity entity)
{
var tr = owner.Database.TransactionManager.TopTransaction;
var id = owner.AppendEntity(entity);
tr.AddNewlyCreatedDBObject(entity, true);
return id;
}
}
Using example:
using (var tr = db.TransactionManager.StartTransaction())
{
var line = new Line(startPt, endPt) { Layer = layerName };
db.GetModelSpace(OpenMode.ForWrite).Add(line);
tr.Commit();
}
I am on Acumatica 5.30.1672 and am using the SOAP API for Screen IN508000 to try and run the Prepare Replenishment process. I am setting the WarehouseID for where I want to prepare replenishment. The API call is returning the correct InventoryIDs in the response object but when I tell it to ProcessAll, it is not. There is no error, just no processing. I feel like I am missing something trivial here but I just cannot see it. When I do this using the GUI, everything works perfectly.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using AcumaticaTest.AcumaticaWebReference;
namespace AcumaticaTest
{
class Program
{
static void Main(string[] args)
{
Screen context = new Screen();
context.CookieContainer = new System.Net.CookieContainer();
context.Timeout = 1200000;
context.Url = "url to SOAP API Endpoint";
LoginResult lresult = context.Login("<username>", "<password>");
IN508000Content IN508000 = context.IN508000GetSchema();
context.IN508000Clear();
var commands = new Command[]
{
new Value {Value = "<WarehouseID>", LinkedCommand = IN508000.Selection.Warehouse, Commit= true },
new Value {Value = "false", LinkedCommand = IN508000.Selection.Me, Commit= true },
IN508000.ItemsRequiringReplenishment.InventoryID,
IN508000.Actions.ProcessAll
};
var response = context.IN508000Submit(commands);
var status = context.IN508000GetProcessStatus();
while (status.Status == ProcessStatus.InProcess)
{
status = context.IN508000GetProcessStatus();
}
}
}
}
I tested your code and it is working just fine - the system prepares the replenishment plans for the items of the requested warehouse, and the response object contains all the items that would be listed if you opened the Prepare Replenishment screen with the parameters you set. If you want to e-mail me URL and credentials to your site (gmichaud at acumatica) I can take a look to see what's wrong.
Please note that you don't need to explicitly set Commit = true for these two fields (the schema object already has them set to true). I would also recommend to add System.Threading.Thread.Sleep(1000) in your loop to avoid hammering the server with requests while it is processing.