I am working on a system that will analyze video frames from multiple cameras. Each camera will initialize a WCF session and be assigned a GUID for identification before sending video frames. The processing load is more than the server can accomplish in real time as the number of cameras grow. As such, I am writing a class to manage the data as it arrives in bursts from the motion activated cameras.
Each frame from the cameras must be analyzed in parallel with frames from the other cameras. Each of the cameras will be filling a queue with frames to manage the FIFO nature of the data.
Management of the Queues has left me searching for ideas. Initially I had intended to create a dictionary and insert each GUID-Queue pair. After working on the code for a while I am starting to think that this is not the best approach. Does anyone have any suggestions?
Some simplified sample code:
namespace DataCollection
{
[ServiceBehavior(Name = "ClientDataView", InstanceContextMode = InstanceContextMode.Single)]
public class DataCollectionService : IDataCollectionService
{
//Dictionary of Guid-Queue pairs
CameraCollection CameraArray = new CameraCollection();
public Guid RegisterCamera()
{
Guid ID = new Guid();
WorkQueue FrameQueue = new WorkQueue();
CameraArray.AddCamera(ID, FrameQueue)
return ID;
}
public bool SendData(Guid ID, FrameReady newFrame)
{
Frame dataFrame = newFrame.OpenFrame;
WorkQueue que = CameraArray.GetQueueReference(ID);
que.QueueFrame(dataFrame);
return true;
}
}
}
Currently if I hard-code everything for a set number of cameras the system works. Getting it to work with a variant number of cameras is more challenging. Any suggestions or advise would be appreciated.
.Net framewok 4 has a class called Parallel. This class has an extension method Foreach. With your collection, you can do something like Parallel.Foreach(collection,action),
Related
Problem
I'm using Firebase Realtime Database (for Unity) to manage the server side for a turn based game but I have a problem with my matchmaking... a high download usage.
Every online game have 2 base states to avoid more than 2 players join a game: Created and Joined
Created: A player try to join a game, if can't find one a new game will be Created
Joined: A player try to join a game, if find one change the state to from Created to Joined
I'm using RunTransaction to prevent more than 2 players from joining a game, but I checked that the latest data was not fetched from the database because of the local cache, adding keepSynced over my matches-{lang} child node will always have the latest data but naturally this produces a high download usage.
private DatabaseReference DatabaseReference()
{
return FirebaseDatabase.DefaultInstance.RootReference.Child(MatchesLocation(LanguageManager.Manager.GetPlayerLanguageCode()));
}
private DatabaseReference DatabaseReferenceLangMatch(Language language)
{
return FirebaseDatabase.DefaultInstance.RootReference.Child(MatchesLocation(LanguageManager.Manager.GetLanguageCode(language)));
}
public void ManageKeepSyncedMatches(Language lang)
{
DatabaseReferenceLangMatch(Language.English).KeepSynced(lang == Language.English);
}
public void JoinMatchTransaction(GameMatchOnline gameMatchOnline, UnityAction<string, bool> callback)
{
JoinTransactionAbort joinResult = JoinTransactionAbort.None;
DatabaseReference matchesListRef = DatabaseReference();
Dictionary<string, object> joinerDict = gameMatchOnline.ToJoinDictionary();
matchesListRef.Child(gameMatchOnline.matchId).RunTransaction(matchData =>
{
Dictionary<string, object> matchDict = matchData.Value as Dictionary<string, object>;
if (matchDict == null)
{
joinResult = JoinTransactionAbort.Null;
return TransactionResult.Success(null);
}
if (!matchDict.ContainsKey("state"))
{
joinResult = JoinTransactionAbort.Error;
return TransactionResult.Abort();
}
GameMatchOnline.State state = (GameMatchOnline.State)System.Convert.ToInt32(matchDict["state"]);
if (state != GameMatchOnline.State.Created)
{
joinResult = JoinTransactionAbort.Error;
return TransactionResult.Abort();
}
joinResult = JoinTransactionAbort.None;
matchDict.Add("joinerInfo", joinerDict["joinerInfo"]);
matchDict["state"] = joinerDict["state"];
matchData.Value = matchDict;
return TransactionResult.Success(matchData);
}).ContinueWith(task =>
{
// Fail
if (task.IsFaulted || task.IsCanceled)
{
UnityThread.executeInUpdate(() =>
{
if (joinResult == JoinTransactionAbort.Error)
{
callback(null, false);
}
});
}
// Can Join match
else if (task.IsCompleted)
{
UnityThread.executeInUpdate(() =>
{
if (joinResult == JoinTransactionAbort.None)
{
AddListenerResultsValueChanged(gameMatchOnline.matchId, gameMatchOnline.joinerInfo.userId, gameMatchOnline.isPrivate, gameMatchOnline.language);
callback(gameMatchOnline.matchId, true);
}
else
{
callback(null, false);
}
});
}
});
}
Question
Removing keepSynced players will have locally cached information for matches-{lang}, can I trust that by doing this there will be no more than 2 players per game? *Transactions are supposed to avoid this kind of problem.
Is there a way to avoid the local cache for a request and thus always get the updated data?
Could the best solution be to move the games to another node to reduce the size of the matches-{lang} node?
Thanks!
Removing "keepSynced" players will have locally cached information for "matches", can I trust that by doing this there will be no more than 2 players per game? *Transactions are supposed to avoid this kind of problem.
With KeepSynced off, Transactions will still hit the local cache then hit the internet. It'll probably save you some bandwidth since it's a lazy access (that's assuming you don't do something like "get all matches"), and you'll be able to make the guarantees you need. Whether or not you use KeepSynced, you should be prepared for your transaction to run multiple times (and against null data if the local cache is empty).
Is there a way to avoid the local cache for a request and thus always get the updated data?
Correction
It looks like I got this a little backwards, see this answer for more details. It will return the cached value and request an updated one. Subsequent calls will get a new value when it's available. You should always try to use ValueChanged when possible.
old answer:
You _can_ just say `GetValueAsync`, which has to bypass the cache since it will only fire once. You really should use ValueChanged listeners to listen for changes and Transactions to change data if you can to keep everything up to date and to avoid data races.
Could the best solution be to move the games to another node to reduce the size of the "matches" node?
Generally the fewer people hitting a shared resource, the better your performance. If you haven't already, check out the Loteria post to see how a team created a realtime game on Realtime Database that was resilient enough to be a Google Doodle.
The TLDR is that rather than a player being responsible for creating or finding a game, players looking for games are written to a queue. When a player is added to the matchmaking queue, a Cloud Function trigger fires which does the work of hooking the users up. A client knows that they're in a game by putting a ValueChanged listener onto a player entry in the database and waiting for a game to be written into it.
The game is further kept low latency with some manual sharding logic. They performed some profiling to see how much traffic a single database could handle, then write some quick (manual - since it was a one day thing) scaling logic to distribute players to one of a number of databases.
I hope that all helps!
--Patrick
I'm developing a Unity plugin that uses BLE to communicate with some device. I have implementation for both Windows (using P/Invoke to this device's SDK) and Android, which uses Unity's helper classes for interfacing with the JVM. The device's SDK in Java has a callback for incoming new data, the callback looks like this:
#Override
public void onDeviceData(BluetoothDevice device, byte[] data) {
}
I've implemented a class that inherits Unity's AndroidJavaProxy to register a callback handler from the C# side:
class Callbacks : AndroidJavaProxy
{
ConcurrentQueue<byte[]> IncomingMessages { get; private set; }
public Callbacks()
: base("com.example.sdk.SdkCallbacks")
{
IncomingMessages = new ConcurrentQueue<byte[]>();
}
void onDeviceData(AndroidJavaObject device, byte[] data)
{
IncomingMessages.Enqueue(data);
}
}
I have a script in the scene that deques this data in the Update() function like so:
while (!_device.IncomingMessages.IsEmpty)
{
byte[] data;
if (_device.IncomingMessages.TryDequeue(out data))
{
// do something with the data
}
}
In my specific case I'm using the data coming from the device to move a point on the screen (think "wireless mouse").
That works pretty well, until I need to get high-rate data from the device (About ~60 callbacks per second with the data parameter containing an array of ~30 bytes). At first everything looks okay, but after a few seconds the point starts to lag heavily behind the actual input from the device, and the lag only gets bigger with time.
I've tried checking if the IncomingMessages queue is getting too big which would mean the producer outpace the consumer by a lot, causing an ever-increasing delay, but that was not the case, IncomingMessages.Count always shows 0-2.
This does not happen when running the code on Windows and using the device's SDK from a regular Android app also works well.
Can this be a marshaling overhead between the JNI and the CLR runtimes? Or something similar?
If so, is there anything I can do about it? I know Unity has another way of communicating between native and the scripting runtime with UnitySendMessage but I'm not sure if that won't have the same problem.
Here is the basic idea of my code:
private void CaptureCameraFrame()
{
Capture = new VideoCapture();
CameraModel.Instance.CameraViewMat = Capture.QueryFrame();
// do stuff with queried matrix here
if(noAbortCondition)
{
CaptureCameraFrame();
}
}
The method should run in a separate thread updating my GUI with the current image after processing.
Only Problem is, that I get two different kinds of error:
Attempt to read/write protected memory: This happens on the second runthrough
of the method.
I get an null-reference error using `CameraModel.Instance.CameraViewMat right after querying the frame.
The two issues seem to be connected, seems like QueryFrame() runs asynchronously from the rest of the code and isn't done when the program jumps to the next step.
Question is: How can I make sure, that querying the image from the camera is finished, and I can use the information in the matrix as well as start a new query?
In all the examples I have found this is done by using time, but I would like to start with a new frame as soon as processing on the last frame is done.
I haven't really done much in C# when it comes to threading, but what I understand in such cases one would use the asyncand awaitkeywords to make sure a method in an asynchronous method is finished. However I wasn't able to make a working implementation in this case.
You are creating VideoCapture class instance repeatedly and even not disposing of it. Create your VideoCapture instance only once and use them for your task. At the end dispose it.
public YourConstructor()
{
Capture = new VideoCapture();
}
private void CaptureCameraFrame()
{
CameraModel.Instance.CameraViewMat = Capture.QueryFrame();
// do stuff with queried matrix here
if(noAbortCondition)
{
CaptureCameraFrame();
}
}
Hopefully, it will work for you!
I'm using WaveInEvent of NAudio to record microphone data. It works fine for a while, but after a few times, it stops providing input data- the DataAvailable callback is never called with new data.
I have tried creating a new WaveInEvent each time, but this did not resolve the problem. I've also tried using the WASAPI input, which always called DataAvailable - with zero bytes of data.
How can I record audio from the microphone reliably with NAudio?
Currently, my code looks like this:
StartRecording() {
microphone = new WaveInEvent();
microphone.DeviceNumber = 0;
microphone.WaveFormat = outformat;
microphone.BufferMilliseconds = 50;
microphone.DataAvailable += (_, recArgs) =>
{
session.OnAudioData(recArgs.Buffer, recArgs.BytesRecorded);
};
microphone.StartRecording();
}
StopRecording() {
if (microphone != null)
{
microphone.StopRecording();
microphone.Dispose();
microphone = null;
}
}
There's no other naudio code in the project except using WaveFormat to describe wave formats.
NAudio throws an access violation exception trying to call WaveInBuffer.Reuse() from a threadpool worker. I'm not sure why this doesn't do something more serious than just drop audio data.
For the condition where I did not recreate the WaveInEvent, I get an MmException instead- invalid handle calling waveInPrepareHeader, in the same place.
Frankly, the fact that I get different results heavily implies that NAudio is doing some funky shit it shouldn't to share state between instances, and looking at the source on Codeplex, I'm not really sure WTF is going on.
It seems that the drivers for the USB microphone do not behave correctly. When the buffer is sent to the user through the WIM_DATA message, it is full. However when waveInUnprepareHeader is called, it's still in the queue, even though it was literally just passed as full. So I think that the drivers for the microphone are ultimately to blame.
I've been looking more closely at the microphone and it seems that this particular unit is actually known to have been damaged.
I've been playing with XNA and want to try and make a game work over LAN but it turns out that to do this I need to use something called remoting. Anyway I managed to get this to work
public class TestObject : MarshalByRefObject
{
int testInt;
public Level()
{
this.testInt = 5.Zero;
}
public int GetNumber()
{
return testInt;
}
}
and my server
channel = new TcpChannel(4444);
ChannelServices.RegisterChannel(channel, false);
Type type = Type.GetType("Domain.Level,Domain");
RemotingConfiguration.RegisterWellKnownServiceType(type,
"FirstRemote",
WellKnownObjectMode.Singleton);
and client
this.chanel = new TcpChannel();
ChannelServices.RegisterChannel(chanel, false);
this.testObject = (TestObject)Activator.GetObject(typeof(TestObject),
"tcp://localhost:4444/FirstRemote");
so that works but the problem is that the server has no way to access the object and I cant make a constructor that takes arguments so there is no way to initialize any data on the test object. How do I make an object, and then make it use that instead of making a new object?
Unfortunately, you'll probably find that realtime network communication in games is more complex than the current direction you're taking. Most games use persistent socket connections to pass information back and forth between clients and other methods are generally too slow for realtime networking.
I recommend looking into the Lidgren networking library. Lidgren abstracts network communication and makes it much easier to serialize data into very small and fast packets that can be transferred via several reliability modes across UDP and TCP. You can find the Lidgren project here:
http://code.google.com/p/lidgren-network-gen3/
As a side note, it was very valuable for me to read how the Unreal Engine does networking here:
http://udn.epicgames.com/Three/NetworkingOverview.html
And I wrote a blog that demonstrates some specific details about implementing a client/server pattern:
http://syndicatex.com/flatredball/flatredball-and-lidgren-multiplayer-networking/