This question already has answers here:
Random.Next returns always the same values [duplicate]
(4 answers)
Closed 6 years ago.
My program is a multi threaded proxy checker and whenever I return the proxy ip addresses from my method and try to echo them out I'm getting a bunch and the threads are doing it completely unintended. It's supposed to supply each thread with a line of IP addresses. Here's a screenshot of what's echoing. After this the IP variable will return and contain null.
My plagued code (bear with me, based off a public example):
static List<String> ips = new List<String>();// this is at the start of the program class
static Random rnd = new Random();
private static String getip()
{
if (ips.Count == 0)
{
return null;
}
return ips[rnd.Next(0, ips.Count)];
}
Also the get IP is called in a while (true) loop as it's a proxy checker, I don't think that code is too necessary.
The other code:
while (true)
{
string ip = getip();
try
{
using (var client = new ProxyClient(ip, user, pass))
{
Console.WriteLine(ip, user, pass);
client.Connect();
if (client.IsConnected)
{
return true;
}
else
{
client.Disconnect();
return false;
}
}
}
catch
{
removeip(ip);
}
Thread.Sleep(30);
}
For example, thread 1 should have 127.0.0.1 (first IP from list), thread 2, 127.0.0.2 (second IP from list) etc etc, the problem at the moment is located in the screenshot.
Edit: this is not a duplicate i didn't explain what i need properly, this note from Eric J explains what i'm trying to do, it wasn't just the random issue.
NOTE
If you want each thread to get its own unique IP rather than a random one, you'll need to do something different than pick a random IP. You can after all get the same random IP more than once (if you flip a coin twice, you might get head twice or tails twice).
A good strategy would be to start from your List<String> ips and create one thread for each entry in that list.
I don't see a need to lock the code in question. List<T> is thread-safe for read access. You only need to lock it if you are adding to or modifying the list (see Thread Safety at the bottom of the MSDN entry).
The problem you are experiencing is unrelated. When you create a new Random(), the seed for the pseudo-random number generator is based on the system clock. Multiple calls in quick succession can happen on the same clock tick, meaning that they get the same number sequence.
Initialize your random outside of getip() to avoid the problem (e.g. make it a static field of your class).
static List<String> ips = new List<String>();// this is at the start of the program class
// See the discussion in comments about multithreaded access to Random()
// In particular see http://stackoverflow.com/a/19271004/141172
static Random rnd = // Get a thread safe Random instance
private static String getip()
{
if (ips.Count == 0)
{
return null;
}
return ips[rnd.Next(0, ips.Count)];
}
NOTE
If you want each thread to get its own unique IP rather than a random one, you'll need to do something different than pick a random IP. You can after all get the same random IP more than once (if you flip a coin twice, you might get head twice or tails twice).
A good strategy would be to start from your List<String> ips and create one thread for each entry in that list. Pass the IP it should be responsible for as a parameter.
Related
I am writing an application that needs to write messages to a USB HID device and read responses. For this purpose, I'm using USBHIDDRIVER.dll (https://www.leitner-fischer.com/2007/08/03/hid-usb-driver-library/ )
Now it works fine when writing many of the message types - i.e. short ones.
However, there is one type of message where I have to write a .hex file containing about 70,000 lines. The protocol requires that each line needs to be written individually and sent in a packet containing other information (start, end byte, checksum)
However I'm encountering problems with this.
I've tried something like this:
private byte[] _responseBytes;
private ManualResetEvent _readComplete;
public byte[][] WriteMessage(byte[][] message)
{
byte[][] devResponse = new List<byte[]>();
_readComplete = new ManualResetEvent(false);
for (int i = 0; i < message.Length; i++)
{
var usbHid = new USBInterface("myvid", "mypid");
usbHid.Connect();
usbHid.enableUsbBufferEvent(UsbHidReadEvent);
if (!usbHid.write(message)) {
throw new Exception ("Write Failed");
}
usbHid.startRead();
if (!_readComplete.WaitOne(10000)) {
usbHid.stopRead();
throw new Exception ("Timeout waiting for read");
}
usbHid.stopRead();
_readComplete.Reset();
devResponse.Add(_responseBytes.ToArray());
usbHid = null;
}
return devResponse;
}
private void ReadEvent()
{
if (_readComplete!= null)
{
_readComplete.Set();
}
_microHidReadBytes = (byte[])((ListWithEvent)sender)[0];
}
This appears to work. In WireShark I can see the messages going back and forth. However as you can see it's creating an instance of the USBInterface class every iteration. This seems very clunky and I can see in the TaskManager, it starts to eat up a lot of memory - current run has it above 1GB and eventually it falls over with an OutOfMemory exception. It is also very slow. Current run is not complete after about 15 mins, although I've seen another application do the same job in less than one minute.
However, if I move the creation and connection of the USBInterface out of the loop as in...
var usbHid = new USBInterface("myvid", "mypid");
usbHid.Connect();
usbHid.enableUsbBufferEvent(UsbHidReadEvent);
for (int i = 0; i < message.Length; i++)
{
if (!usbHid.write(message)) {
throw new Exception ("Write Failed");
}
usbHid.startRead();
if (!_readComplete.WaitOne(10000)) {
usbHid.stopRead();
throw new Exception ("Timeout waiting for read");
}
usbHid.stopRead();
_readComplete.Reset();
devResponse.Add(_responseBytes.ToArray());
}
usbHid = null;
... now what happens is it only allows me to do one write! I write the data, read the response and when it comes around the loop to write the second message, the application just hangs in the write() function and never returns. (Doesn't even time out)
What is the correct way to do this kind of thing?
(BTW I know it's adding a lot of data to that devResponse object but this is not the source of the issue - if I remove it, it still consumes an awful lot of memory)
UPDATE
I've found that if I don't enable reading, I can do multiple writes without having to create a new USBInterface1 object with each iteration. This is an improvement but I'd still like to be able to read each response. (I can see they are still sent down in Wireshark)
Usually I am able to resolve problems on my own, but once again I have to ask for help. I apologise in advance if something is incoherent, as it appears that I've got a bit of a fever.
I am currently making really simple game in XNA. It's a 2D overhead shooter. The only action is Player shooting Bullets. The thing is, I want to make it multiplayer (players in the same network).
I've considered following approach:
One of players hosts server
Server holds information about position of every player and bullet
Server creates new World (composed of players and bullets) and updates it in a new thread like this:
while (true)
{
lock (world) world.update();
Thread.Sleep(worldUpdatePeriod);
}
Server creates new Thread for every player that has to be handled
Players connect to the server
They should periodically receive information from server what should they draw:
class Asset
{
string type;
Vector2 position;
}
They should be able to send input from Keyboard and Mouse to move and shoot.
Now, to the point. Player is able to send three types of messages to the server:
"CAN READ"
"MOUSE"
"KEYBOARD"
string header = "CAN READ";
lock (streamWriter) { streamWriter.WriteLine(header); }
Server, in a separate Thread for every client waits for messages
streamReader = new StreamReader(networkStream);
streamWriter = new StreamWriter(networkStream);
streamWriter.AutoFlush = true;
while (true)
{
if (networkStream.DataAvailable)
{
string header;
lock (streamReader)
{
header = Convert.ToString(streamReader.ReadLine());
Console.WriteLine($"Header received: {header}");
handleHeader(header);
}
}
}
Now if the header is, for example, "CAN READ"
Server
case "CAN READ":
if (streamWriter.BaseStream.CanRead)
{
lock (streamWriter)
{
streamWriter.WriteLine(server.world.assets.Count);
foreach (var asset in server.world.assets)
{
streamWriter.WriteLine(asset.type);
streamWriter.WriteLine(asset.position.X);
streamWriter.WriteLine(asset.position.Y);
}
}
}
break;
Client
int assetCount = Convert.ToInt32(streamReader.ReadLine());
List<Asset> receivedAssets = new List<Asset>();
lock (streamReader)
{
for (int i = 0; i < assetCount; i++)
{
string type = Convert.ToString(streamReader.ReadLine());
int x = Convert.ToInt32(streamReader.ReadLine());
int y = Convert.ToInt32(streamReader.ReadLine());
receivedAssets.Add(new Asset(type, new Vector2(x, y)));
}
return receivedAssets;
}
Of course Client class and handleClient class have separate streamReaders nad streamWriters.
You get the idea. I am trying to match every WriteLine() from Server with ReadLine() from Client and so on. And it works fine! Until I am adding user input to the mixture.
Once I send keyboard input (matched as shown above), it appears to work, with occasional System.FormatException, meaning that some ReadLine() read wrong WriteLine(). I am able to catch that and retry.
And everything goes to hell once I send mouse input as well. Mismatching exceptions become waaay more often to the point where the 'game' is unplayable. So I am wondering if there's something wrong with my approach. I apologise for length of the question, I tried to be concise but also felt need to provide context. Thanks if you made it to the end, I'll be glad to answer any further questions as I am trying to solve it for the past few days.
I've read the link that Andrew provided me with and I've been able to make it work. How? Using both TCP Client (to set up connection, read input from client) and UDP Client (to broadcast assets to every client). Just posting that in case that someone encounters such problem in the future, which I highly doubt.
Cheers!
As part of an effort to automate starting/stopping some of our NServiceBus services, I'd like to know when a service has finished processing all the messages in it's input queue.
The problem is that, while the NServiceBus service is running, my C# code is reporting one less message than is actually there. So it thinks that the queue is empty when there is still one message left. If the service is stopped, it reports the "correct" number of messages. This is confusing because, when I inspect the queues myself using the Private Queues view in the Computer Management application, it displays the "correct" number.
I'm using a variant of the following C# code to find the message count:
var queue = new MessageQueue(path);
return queue.GetAllMessages().Length;
I know this will perform horribly when there are many messages. The queues I'm inspecting should only ever have a handful of messages at a time.
I have looked at
other
related
questions,
but haven't found the help I need.
Any insight or suggestions would be appreciated!
Update: I should have mentioned that this service is behind a Distributor, which is shut down before trying to shut down this service. So I have confidence that new messages will not be added to the service's input queue.
The thing is that it's not actually "one less message", but rather dependent on the number of messages currently being processed by the endpoint which, in a multi-threaded process, can be as high as the number of threads.
There's also the issue of client processes that continue to send messages to that same queue.
Probably the only "sure" way of handling this is by counting the messages multiple times with a delay in between and if the number stay zero over a certain number of attempts that you can assume the queue is empty.
WMI was the answer! Here's a first pass at the code. It could doubtless be improved.
public int GetMessageCount(string queuePath)
{
const string query = "select * from Win32_PerfRawData_MSMQ_MSMQQueue";
var query = new WqlObjectQuery(query);
var searcher = new ManagementObjectSearcher(query);
var queues = searcher.Get();
foreach (ManagementObject queue in queues)
{
var name = queue["Name"].ToString();
if (AreTheSameQueue(queuePath, name))
{
// Depending on the machine (32/64-bit), this value is a different type.
// Casting directly to UInt64 or UInt32 only works on the relative CPU architecture.
// To work around this run-time unknown, convert to string and then parse to int.
var countAsString = queue["MessagesInQueue"].ToString();
var messageCount = int.Parse(countAsString);
return messageCount;
}
}
return 0;
}
private static bool AreTheSameQueue(string path1, string path2)
{
// Tests whether two queue paths are equivalent, accounting for differences
// in case and length (if one path was truncated, for example by WMI).
string sanitizedPath1 = Sanitize(path1);
string sanitizedPath2 = Sanitize(path2);
if (sanitizedPath1.Length > sanitizedPath2.Length)
{
return sanitizedPath1.StartsWith(sanitizedPath2);
}
if (sanitizedPath1.Length < sanitizedPath2.Length)
{
return sanitizedPath2.StartsWith(sanitizedPath1);
}
return sanitizedPath1 == sanitizedPath2;
}
private static string Sanitize(string queueName)
{
var machineName = Environment.MachineName.ToLowerInvariant();
return queueName.ToLowerInvariant().Replace(machineName, ".");
}
I am seeing some dead-instance weirdness running parallelized nested-loop web stress tests using Selenium WebDriver, simple example being, say, hit 300 unique pages with 100 impressions each.
I'm "successfully" getting 4 - 8 WebDriver instances going using a ThreadLocal<FirefoxWebDriver> to isolate them per task thread, and MaxDegreeOfParallelism on a ParallelOptions instance to limit the threads. I'm partitioning and parallelizing the outer loop only (the collection of pages), and checking .IsValueCreated on the ThreadLocal<> container inside the beginning of each partition's "long running task" method. To facilitate cleanup later, I add each new instance to a ConcurrentDictionary keyed by thread id.
No matter what parallelizing or partitioning strategy I use, the WebDriver instances will occasionally do one of the following:
Launch but never show a URL or run an impression
Launch, run any number of impressions fine, then just sit idle at some point
When either of these happen, the parallel loop eventually seems to notice that a thread isn't doing anything, and it spawns a new partition. If n is the number of threads allowed, this results in having n productive threads only about 50-60% of the time.
Cleanup still works fine at the end; there may be 2n open browsers or more, but the productive and unproductive ones alike get cleaned up.
Is there a way to monitor for these useless WebDriver instances and a) scavenge them right away, plus b) get the parallel loop to replace the task segment immediately, instead of lagging behind for several minutes as it often does now?
I was having a similar problem. It turns out that WebDriver doesn't have the best method for finding open ports. As described here it gets a system wide lock on ports, finds an open port, and then starts the instance. This can starve the other instances that you're trying to start of ports.
I got around this by specifying a random port number directly in the delegate for the ThreadLocal<IWebDriver> like this:
var ports = new List<int>();
var rand = new Random((int)DateTime.Now.Ticks & 0x0000FFFF);
var driver = new ThreadLocal<IWebDriver>(() =>
{
var profile = new FirefoxProfile();
var port = rand.Next(50) + 7050;
while(ports.Contains(port) && ports.Count != 50) port = rand.Next(50) + 7050;
profile.Port = port;
ports.Add(port);
return new FirefoxDriver(profile);
});
This works pretty consistently for me, although there's the issue if you end up using all 50 in the list that is unresolved.
Since there is no OnReady event nor an IsReady property, I worked around it by sleeping the thread for several seconds after creating each instance. Doing that seems to give me 100% durable, functioning WebDriver instances.
Thanks to your suggestion, I've implemented IsReady functionality in my open-source project Webinator. Use that if you want, or use the code outlined below.
I tried instantiating 25 instances, and all of them were functional, so I'm pretty confident in the algorithm at this point (I leverage HtmlAgilityPack to see if elements exist, but I'll skip it for the sake of simplicity here):
public void WaitForReady(IWebDriver driver)
{
var js = #"{ var temp=document.createElement('div'); temp.id='browserReady';" +
#"b=document.getElementsByTagName('body')[0]; b.appendChild(temp); }";
((IJavaScriptExecutor)driver).ExecuteScript(js);
WaitForSuccess(() =>
{
IWebElement element = null;
try
{
element = driver.FindElement(By.Id("browserReady"));
}
catch
{
// element not found
}
return element != null;
},
timeoutInMilliseconds: 10000);
js = #"{var temp=document.getElementById('browserReady');" +
#" temp.parentNode.removeChild(temp);}";
((IJavaScriptExecutor)driver).ExecuteScript(js);
}
private bool WaitForSuccess(Func<bool> action, int timeoutInMilliseconds)
{
if (action == null) return false;
bool success;
const int PollRate = 250;
var maxTries = timeoutInMilliseconds / PollRate;
int tries = 0;
do
{
success = action();
tries++;
if (!success && tries <= maxTries)
{
Thread.Sleep(PollRate);
}
}
while (!success && tries < maxTries);
return success;
}
The assumption is if the browser is responding to javascript functions and is finding elements, then it's probably a reliable instance and ready to be used.
EDIT: Ok I had a problem with one of the string concatenation functions, has nothing to do with threads, but knowing that it couldn't be a problem with threading lead me to the answer thank you for answering.
I am making a simple tcp/ip chat program for practicing threads and tcp/ip. I was using asynchronous methods but had a problem with concurrency so I went to threads and blocking methods (not asynchronous). I have two private variables defined in the class, not static:
string amessage = string.Empty;
int MessageLength;
and a Thread
private Thread BeginRead;
Ok so I call a function called Listen ONCE when the client starts:
public virtual void Listen(int byteLength)
{
var state = new StateObject {Buffer = new byte[byteLength]};
BeginRead = new Thread(ReadThread);
BeginRead.Start(state);
}
and finally the function to receive commands and process them, I'm going to shorten it because it is really long:
private void ReadThread(object objectState)
{
var state = (StateObject)objectState;
int byteLength = state.Buffer.Length;
while (true)
{
var buffer = new byte[byteLength];
int len = MySocket.Receive(buffer);
if (len <= 0) return;
string content = Encoding.ASCII.GetString(buffer, 0, len);
amessage += cleanMessage.Substring(0, MessageLength);
if (OnRead != null)
{
var e = new CommandEventArgs(amessage);
OnRead(this, e);
}
}
}
Now, as I understand it only one thread at a time will enter BeginRead, I call Receive, it blocks until I get data, and then I process it. The problem: the variable amessage will change it's value between statements that do not touch or alter the variable at all, for example at the bottom of the function at: if (OnRead != null) "amessage" will be equal to 'asdf' and at if (OnRead != null) "amessage" will be equal to qwert. As I understand it this is indicative of another thread changing the value/running asynchronously. I only spawn one thread to do the receiving and the Receive function is blocking, how could there be two threads in this function and if there is only one thread how does amessage's value change between statements that don't affect it's value. As a side note sorry for spamming the site with these questions but I'm just getting a hang of this threading story and it's making me want to sip cyanide.
Thanks in advance.
EDIT:
Here is my code that calls the Listen Method in the client:
public void ConnectClient(string ip,int port)
{
client.Connect(ip,port);
client.Listen(5);
}
and in the server:
private void Accept(IAsyncResult result)
{
var client = new AbstractClient(MySocket.EndAccept(result));
var e = new CommandEventArgs(client, null);
Clients.Add(client);
client.Listen(5);
if (OnClientAdded != null)
{
var target = (Control) OnClientAdded.Target;
if (target != null && target.InvokeRequired)
target.Invoke(OnClientAdded, this, e);
else
OnClientAdded(this, e);
}
client.OnRead += OnRead;
MySocket.BeginAccept(new AsyncCallback(Accept), null);
}
All this code is in a class called AbstractClient. The client inherits the Abstract client and when the server accepts a socket it create's it's own local AbstractClient, in this case both modules access the functions above however they are different instances and I couldn't imagine threads from different instances combining especially as no variable is static.
Well, this makes no sense the way you described it. Which probably means that what you think is going on is not what is really happening. Debugging threaded code is quite difficult, very hard to capture the state of the program at the exact moment it misbehaves.
A generic approach is to add logging to your code. Sprinkle your code with Debug.WriteLine() statements that shows the current value of the variable, along with the thread's ManagedId. You get potentially a lot of output, but somewhere you'll see it going wrong. Or you get enough insight in how thread(s) are interacting to guess the source of the problem.
Just adding the logging can in itself solve the problem because it alters the timing of code. Sucks when that happens.
I assume OnRead is firing an event dispatched on a thread pool thread. If any registered event handler is writing to amessage, its value could change any time you're in the reading loop.
Still not very clear where you are gettingthe value assigned to amessage in the loop. Should cleanmessage read content?