We Keep Coding

How to set up a .NET 7 C# rate limiter per user if the user id is stored ONLY in the entity in the request body?

There is an API that accepts an entity with a previously unknown ID. I need to configure the rate limiter so that entities with the same ID get into the queue. I figured out how to create a window and a queue. How to make a separate queue for each ID?
The entity is a JSON file. The ID is inside the file.
The following is written, but this forms one queue:
services.AddRateLimiter(options => options
.AddFixedWindowLimiter(policyName: "UserPolicy", options =>
{
options.PermitLimit = 1;
options.Window = TimeSpan.FromSeconds(10);
options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
options.QueueLimit = 3;
}));

You can try using PartitionedRateLimiter. Something along these lines (not tested):
builder.Services.AddRateLimiter(options =>
{
options.AddPolicy("myRateLimiter1", context =>
{
var request = context.Request;
var partitionKey = "";
if (request.Method == HttpMethods.Post && request.ContentLength > 0)
{
request.EnableBuffering();
var buffer = new byte[Convert.ToInt32(request.ContentLength)];
request.Body.Read(buffer, 0, buffer.Length);
//get body string here...
var requestContent = Encoding.UTF8.GetString(buffer);
// get partition key here... partitionKey = ...
request.Body.Position = 0; //rewinding the stream to 0
}
return RateLimitPartition.GetFixedWindowLimiter(
partitionKey: partitionKey,
factory: partition => new FixedWindowRateLimiterOptions
{
PermitLimit = 1,
Window = TimeSpan.FromSeconds(10),
QueueProcessingOrder = QueueProcessingOrder.OldestFirst,
QueueLimit = 3
});
});
});
Though I would suggest to consider passing Id in some other way (headers) or resolve the limiter on the handler/BL level.

WCF Websocket project fails upon Entity Framework data access attempt

I am new to WebSockets (this AM) and have set up a WCF WebSocket app that works when doing a trivial example I found online (http://www.codeproject.com/Articles/619343/Using-WebSocket-in-NET-Part).
I added Entity Framework and as soon as I add code to try to access data the process (just sending a message back and forth) no longer works.
Could there be some fundamental concept I could be missing?
Does anyone have any good ideas for troubleshooting?
namespace PBWebSocket
{
public class PBWebSocket : IBWebSocket
{
private SPEntities db = new SPEntities();
public async Task SendMessageToServer(Message msg)
{
var callback = OperationContext.Current.GetCallbackChannel<IPBCallback>();
if (msg.IsEmpty || ((IChannel)callback).State != CommunicationState.Opened)
{
return;
}
byte[] body = msg.GetBody<byte[]>();
string msgTextFromClient = Encoding.UTF8.GetString(body);
var reqId = Int32.Parse(msgTextFromClient);
// *** The below line breaks it ***
var req = db.Requests.Where(r => r.Id == 164).FirstOrDefault();
reqId = reqId + 2;
Message newMsg = ByteStreamMessage.CreateMessage(
new ArraySegment<byte>(Encoding.UTF8.GetBytes(reqId.ToString())));
newMsg.Properties["WebSocketMessageProperty"] =
new WebSocketMessageProperty
{ MessageType = WebSocketMessageType.Text };
await callback.SendMessageToClient(newMsg);
}
}
}

Configure kafka-net to stop sending latest messages

I'm using kafka 0.8.1.1 on a Red Hat VM with kafka-net plugin. How can I configure my consumer to stop receiving earlier messages from kafka?
My consumer code:
var options = new KafkaOptions(new Uri("tcp://199.53.249.150:9092"), new Uri("tcp://199.53.249.151:9092"));
Stopwatch sp = new Stopwatch();
var router = new BrokerRouter(options);
var consumer = new Consumer(new ConsumerOptions("Test", router));
ThreadStart start2 = () =>
{
while (true)
{
sp.Start();
foreach (var message in consumer.Consume())
{
if (MessageDecoderReceiver.MessageBase(message.Value) != null)
{
PrintMessage(MessageDecoderReceiver.MessageBase(message.Value).ToString());
}
else
{
Console.WriteLine(message.Value);
}
}
sp.Stop();
}
};
var thread2 = new Thread(start2);
thread2.Start();

The Consumer in Kafka-net does not currently auto track the offsets being consumed. You will have to implement the offset tracking manually.
To Store the offset in kafka version 0.8.1:
var commit = new OffsetCommitRequest
{
ConsumerGroup = consumerGroup,
OffsetCommits = new List<OffsetCommit>
{
new OffsetCommit
{
PartitionId = partitionId,
Topic = IntegrationConfig.IntegrationTopic,
Offset = offset,
Metadata = metadata
}
}
};
var commitResponse = conn.Connection.SendAsync(commit).Result.FirstOrDefault();
To set the consumer to start importing at a specific offset point:
var offsets = consumer.GetTopicOffsetAsync(IntegrationConfig.IntegrationTopic).Result
.Select(x => new OffsetPosition(x.PartitionId, x.Offsets.Max())).ToArray();
var consumer = new Consumer(new ConsumerOptions(IntegrationConfig.IntegrationTopic, router), offsets);
Note the above code will set the consumer to start consuming at the very end of the log, effectively only receiving new messages.

RabbitMq Implementation Practically

I am working on rabbitmq and I am confused with some points Like.
I have just implemented a sample from internet which creates a queue and then it fetch
the message from that queue thereby showing it on the webpage.
Now my problem is::
Suppose My server has RabbitmQ installed and multiple users are accessing this website where I
have implemented the rabbitmq.Now, first user sends a message but to whome it will send this message?
To all the users who will open the page because the code will be common for sent message and the name
of the queue will also be same.
Suppose, first user send one message="Hello" on the queue "Queue1"
now, one other user sends another message="Hello World" on the same queue
and one more user sends a message="Hello Worl World" on the same queue.
Now nth user clicks on receive message then which message will be shown to him?
first ,second or third one?
It means we will always have a single queue for my application?
Can somebody please guide me. I am pretty much confused...
Below I am pasting the code sample I will be using for my website
//For sending the messge
protected void btnSendMail_Click(object sender, EventArgs e)
{
try
{
var factory = new ConnectionFactory();
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
// ConnectionFactory factory = new ConnectionFactory() { HostName = "localhost" };
// // factory.UserName = txtUserEmail.Text.ToString();
//// factory.Password = "password";
// factory.VirtualHost = "/";
// factory.Protocol = Protocols.FromEnvironment();
// factory.HostName = "localhost";
// factory.Port = AmqpTcpEndpoint.UseDefaultPort;
// IConnection conn = factory.CreateConnection();
// using (var channel = conn.CreateModel())
// {
// channel.QueueDeclare("hello", false, false, false, null);
// string message = "Hello World!";
// var body = Encoding.UTF8.GetBytes(txtUserEmail.Text.ToString());
// channel.BasicPublish("", "hello", null, body);
// conn.Close();
// }
//Sending Message
channel.QueueDeclare("hello1", false, false, false, null);
string message = txtUserEmail.Text.ToString();
var body = Encoding.UTF8.GetBytes(message);
channel.BasicPublish("", "hello1", null, body);
//Console.WriteLine(" [x] Sent {0}", message);
//Console.ReadLine();
Label1.Text = Encoding.Default.GetString(body);
}
}
}
catch
{
}
}
//For receiving the message.
protected void btnReceive_Click(object sender, EventArgs e)
{
try
{
//var factory = new ConnectionFactory() { HostName = "localhost" };
//using (var connection = factory.CreateConnection())
//{
// using (var channel = connection.CreateModel())
// {
// channel.QueueDeclare("hello", false, false, false, null);
// BasicGetResult result = channel.BasicGet("hello", false);
// var consumer = new QueueingBasicConsumer(channel);
// channel.BasicConsume("hello", true, consumer);
// Console.WriteLine(" [*] Waiting for messages." +
// "To exit press CTRL+C");
// while (true)
// {
// var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
// var body = ea.Body;
// var message = Encoding.UTF8.GetString(body);
// Console.WriteLine(" [x] Received {0}", message);
// Label1.Text = message.ToString();
// }
// }
//}
var factory = new ConnectionFactory();
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
bool noAck = false;
BasicGetResult result = channel.BasicGet("hello1", noAck);
if (result == null)
{
}
else
{
IBasicProperties props = result.BasicProperties;
byte[] Body = result.Body;
Label1.Text = Encoding.Default.GetString(Body);
}
}
}
}
catch
{
}
}

If you are creating a messaging system using RabbitMQ you should probably publish your messages to an exchange and then attach a queue to the exchange for each user of the site. Then have the exchange route the messages to the right user/users queue.
You need a better understanding of messaging patterns associated with the use of RabbitMQ
These tutorials would be most relevant
Publish/Subscribe
http://www.rabbitmq.com/tutorials/tutorial-three-python.html
Routing
http://www.rabbitmq.com/tutorials/tutorial-four-python.html
The tutorials are also available in c# if you need it.

Reading ActiveMQ messages without deletion

I am trying to read through ActiveMQ messages and according some filters to process some of them or leave the other messages in the queue.
I use NMS API with the following code:
Uri connecturi = new Uri("activemq:tcp://model.net:61616");
IConnectionFactory factory = new NMSConnectionFactory(connecturi);
List<ModelBuilderBase> result = new List<ModelBuilderBase>();
using (IConnection connection = factory.CreateConnection())
using (ISession session = connection.CreateSession())
{
IDestination destination = SessionUtil.GetDestination(session, "queue://cidModelbuilderQ");
using (IMessageConsumer consumer = session.CreateConsumer(destination))
{
connection.Start();
ITextMessage message;
while ((message = consumer.ReceiveNoWait() as ITextMessage) != null)
{
if (message.Properties[MANDATOR] == null || message.Properties[REFCODE] == null)
continue;
var mandator = message.Properties[MANDATOR].ToString();
var refCode = message.Properties[REFCODE].ToString();
result.Add(ModelBuilderFactory.Instance.GetInstance(refCode, mandator));
}
}
Problem is that after a message is received the message is deleted. Can I somehow change this behavior and delete the messages only manually after a successful processing?
}

Although it was not easy to write a working code for that, thanks to the ARSs answer I have now this working solution:
Uri connecturi = new Uri("activemq:tcp://model.net:61616");
IConnectionFactory factory = new NMSConnectionFactory(connecturi);
List<ModelBuilderBase> result = new List<ModelBuilderBase>();
using (IConnection connection = factory.CreateConnection())
using (ISession session = connection.CreateSession())
{
IDestination destination = SessionUtil.GetDestination(session, "queue://cidModelbuilderQ");
using (IMessageConsumer consumer = session.CreateConsumer(destination))
{
connection.Start();
var q = session.GetQueue("cidModelbuilderQ");
var b = session.CreateBrowser(q);
var msgs = b.GetEnumerator();
while (msgs.MoveNext())
{
ITextMessage message = msgs.Current as ITextMessage;
if (message.Properties[MANDATOR] == null || message.Properties[REFCODE] == null)
continue;
var mandator = message.Properties[MANDATOR].ToString();
var refCode = message.Properties[REFCODE].ToString();
result.Add(ModelBuilderFactory.Instance.GetInstance(refCode, mandator));
}
}
}

Creates a QueueBrowser object to peek at the messages on the specified queue. Perform you logic on the message and then creates a QueueReceiver object to receive messages from the specified queue.