We Keep Coding

Not receiving JSON data from Microsoft FaceAPI

I am trying to get JSON data from a picture using Microsoft's FaceAPI. I am receiving a StatusCode OK, but am not getting anything significant back. I have verified that the MemoryStream has the right data (which I am getting from an Image control) by saving it to a file.
private async Task<string> GetJSON()
{
var client = new HttpClient();
var queryString = HttpUtility.ParseQueryString(string.Empty);
// Request headers
client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", "mykey");
// Request parameters
queryString["returnFaceId"] = "true";
queryString["returnFaceLandmarks"] = "false";
var uri = "https://api.projectoxford.ai/face/v1.0/detect?" + queryString;
HttpResponseMessage response;
// Request body
byte[] byteData = ImageToByte();
using (var content = new ByteArrayContent(byteData))
{
content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
response = await client.PostAsync(uri, content);
}
return "";
}
private byte[] ImageToByte()
{
using (MemoryStream stream = new MemoryStream())
{
videoBox.Dispatcher.Invoke(delegate
{
var encoder = new PngBitmapEncoder();
var flippedBitmap = new TransformedBitmap();
flippedBitmap.BeginInit();
flippedBitmap.Source = (BitmapSource)videoBox.Source;
var transform = new ScaleTransform(-1, 1);
flippedBitmap.Transform = transform;
flippedBitmap.EndInit();
encoder.Frames.Add(BitmapFrame.Create(flippedBitmap));
encoder.Save(stream);
});
using (FileStream test = new FileStream("snapshot.bmp", FileMode.Create))
{
stream.Position = 0;
stream.CopyTo(test);
}
return stream.ToArray();
}
}

You'll want to call await response.Content.ReadAsStringAsync() to get the JSON.
Alternatively, you can use the Microsoft.ProjectOxford.Face NuGet package which does the plumbing for you, plus provide C# types thereby relieving you the tedium of parsing the JSON.

I am not a c# programmer but after looking at your code, method GetJSON is returning hard coded empty string that might be the cause you are not getting anything back from the server after invoking this method or second reason could be your asynchronous server configuration is not working properly thus its returning blank first and doing actual operation later.

Two way communication using TCP Client only

When dealing with TCP Client, especially when the stream has to be encrypted, the message will not be passed unless the CryptoCtream is closed and this results in the inner stream to be not readable as it will be disposed.
For instance
TCPClient client = new TCPClient("some ip", 1234);
using(var i = new CryptoStream(client.GetStream(), myEncryptor(), CryptoStreamMode.Write) {
i.Write(some Data, 0, 1024);
}
I've tried the flush but it seems that the data can be communicated only FROM the client to server this way. Being aware of all the alternatives, I'm curious about how this could be made possible (sending data using TCP Client and receiving a response even without setting a second channel).

use an intermediate MemoryStream
var memstr = new MemoryStream();
using (var i = new CryptoStream(memstr.....)
{
i.Write(some data);
var buf = memstr.GetBuffer();
client.GetStream().Write(buf);
}
var inp = client.GetStream.Read(..);
var memstr2 = new MemoryStream(inp);
using (var o = new CryptoStream(memstr2,...))
{
var x = memstr2.Read();
}

Stream.ReadAsync() on first invoke very slow

I am writing a class to handle file downloads and i am using this code [simplified]:
var webRequest = (HttpWebRequest)WebRequest.Create(downloadOperation.Link);
webRequest.Proxy = null;
using (var webResponse = await webRequest.GetResponseAsync())
{
using (var downloadStream = webResponse.GetResponseStream())
{
using (var outputFileWriteStream = await outputFile.OpenStreamForWriteAsync())
{
var buffer = new byte[4096];
var downloadedBytes = 0;
var totalBytes = webResponse.ContentLength;
while (downloadedBytes < totalBytes)
{
//*************************THIS LINE TAKES ABOUT 32 SECONDS TO EXECUTE ON FIRST INVOKE, ALL NEXT INVOKES TAKE ABOUT 120MS***************************
var currentRead = await downloadStream.ReadAsync(buffer, 0, buffer.Length);
//*******************************************************************************************************************************************************************
await outputFileWriteStream.WriteAsync(buffer, 0, currentRead);
}
}
}
}
Can you please explain to me why is it taking that long on first invoke and not on the next ones? I am worried that it is downloading the entire file on the first read.
Note that the files are usually between 3~15MB.

I am worried that it is downloading the entire file on the first read.
That's precisely what's happening. You can change that by setting webRequest.AllowReadStreamBuffering to false.

So i found a way to fix this problem, but it doesn't use WebRequest class.
I am now using the HttpClient found in (Windows.Web.Http).
Here is the fixed code:
var client = new Windows.Web.Http.HttpClient(); // prepare the http client
//get the response from the server
using (var webResponse = await client.GetAsync(downloadOperation.Link, HttpCompletionOption.ResponseHeadersRead)) //***********Node the HttpCompletionOption.ResponseHeaderRead, this means that the operation completes as soon as the client receives the http headers instead of waiting for the entire response content to be read
{
using (var downloadStream = (await webResponse.Content.ReadAsInputStreamAsync()).AsStreamForRead() )
{
using (var outputFileWriteStream = await outputFile.OpenStreamForWriteAsync())
{
var buffer = new byte[4096];
var downloadedBytes = 0;
var totalBytes = webResponse.ContentLength;
while (downloadedBytes < totalBytes)
{
//*************************THIS LINE NO LONGER TAKES A LONG TIME TO PERFORM FIRST READ***************************
var currentRead = await downloadStream.ReadAsync(buffer, 0, buffer.Length);
//*******************************************************************************************************************************************************************
await outputFileWriteStream.WriteAsync(buffer, 0, currentRead);
}
}
}
}
Hope this will help someone out there ;)
thank you all

Streamed HTTP with GZIP being buffered by StreamReader?

Struggling to find anyone experiencing a similar issue or anything similar.
I'm currently consuming a stream over http (json) which has a GZip requirement, and I am experiencing a delay from when the data is sent, to when reader.ReadLine() reads it. It has been suggested to me that this could be related to the decoding keeping back data in a buffer?
This is what I have currently, it works fine apart from the delay.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);
request.Method = "GET";
request.PreAuthenticate = true;
request.Credentials = new NetworkCredential(username, password);
request.AutomaticDecompression = DecompressionMethods.GZip;
request.ContentType = "application/json";
request.Accept = "application/json";
request.Timeout = 30;
request.BeginGetResponse(AsyncCallback, request);
Then inside the AsyncCallback method I have:
HttpWebRequest request = result.AsyncState as HttpWebRequest;
using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(result))
using (Stream stream = response.GetResponseStream())
using (StreamReader reader = new StreamReader(stream, Encoding.UTF8))
{
while (!reader.EndOfStream)
{
string line = reader.ReadLine();
if (string.IsNullOrWhiteSpace(line)) continue;
Console.WriteLine(line);
}
}
It just sits on reader.Readline() until more data is received, and then even holds back some of that. There are also keep-alive newlines received, these are often are read out all at once when it does decide to read something.
I have tested the stream running side by side with a curl command running, the curl command receives and decompresses the data perfectly fine.
Any insight would be terrific.
Thanks,
Dan
EDIT
Had no luck using the buffer size on streamreader.
new StreamReader(stream, Encoding.UTF8, true, 1)
EDIT
Also had no luck updating to .NET 4.5 and using
request.AllowReadStreamBuffering = false;

Update: This seems to have issues over long periods of time with higher rates of volume, and should only be used on small volume where the buffer is impacting the application's functionality. I have since switched back to a StreamReader.
So this is what I ended up coming up with. This works, without the delay. This does not get buffered by automated GZip decompression.
using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(result))
using (Stream stream = response.GetResponseStream())
using (MemoryStream memory = new MemoryStream())
using (GZipStream gzip = new GZipStream(memory, CompressionMode.Decompress))
{
byte[] compressedBuffer = new byte[8192];
byte[] uncompressedBuffer = new byte[8192];
List<byte> output = new List<byte>();
while (stream.CanRead)
{
int readCount = stream.Read(compressedBuffer, 0, compressedBuffer.Length);
memory.Write(compressedBuffer.Take(readCount).ToArray(), 0, readCount);
memory.Position = 0;
int uncompressedLength = gzip.Read(uncompressedBuffer, 0, uncompressedBuffer.Length);
output.AddRange(uncompressedBuffer.Take(uncompressedLength));
if (!output.Contains(0x0A)) continue;
byte[] bytesToDecode = output.Take(output.LastIndexOf(0x0A) + 1).ToArray();
string outputString = Encoding.UTF8.GetString(bytesToDecode);
output.RemoveRange(0, bytesToDecode.Length);
string[] lines = outputString.Split(new[] { Environment.NewLine }, new StringSplitOptions());
for (int i = 0; i < (lines.Length - 1); i++)
{
Console.WriteLine(lines[i]);
}
memory.SetLength(0);
}
}

There may be something to the Delayed ACK C.Evenhuis discusses, but I've got a weird gut feeling it's the StreamReader that's causing you headaches...you might try something like this:
public void AsyncCallback(IAsyncResult result)
{
HttpWebRequest request = result.AsyncState as HttpWebRequest;
using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(result))
using (Stream stream = response.GetResponseStream())
{
var buffer = new byte[2048];
while(stream.CanRead)
{
var readCount = stream.Read(buffer, 0, buffer.Length);
var line = Encoding.UTF8.GetString(buffer.Take(readCount).ToArray());
Console.WriteLine(line);
}
}
}
EDIT: Here's the full harness I used to test this theory (maybe the difference from your situation will jump out at you)
(LINQPad-ready)
void Main()
{
Task.Factory.StartNew(() => Listener());
_blocker.WaitOne();
Request();
}
public bool _running;
public ManualResetEvent _blocker = new ManualResetEvent(false);
public void Listener()
{
var listener = new HttpListener();
listener.Prefixes.Add("http://localhost:8080/");
listener.Start();
"Listener is listening...".Dump();;
_running = true;
_blocker.Set();
var ctx = listener.GetContext();
"Listener got context".Dump();
ctx.Response.KeepAlive = true;
ctx.Response.ContentType = "application/json";
var outputStream = ctx.Response.OutputStream;
using(var zipStream = new GZipStream(outputStream, CompressionMode.Compress))
using(var writer = new StreamWriter(outputStream))
{
var lineCount = 0;
while(_running && lineCount++ < 10)
{
writer.WriteLine("{ \"foo\": \"bar\"}");
"Listener wrote line, taking a nap...".Dump();
writer.Flush();
Thread.Sleep(1000);
}
}
listener.Stop();
}
public void Request()
{
var endPoint = "http://localhost:8080";
var username = "";
var password = "";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(endPoint);
request.Method = "GET";
request.PreAuthenticate = true;
request.Credentials = new NetworkCredential(username, password);
request.AutomaticDecompression = DecompressionMethods.GZip;
request.ContentType = "application/json";
request.Accept = "application/json";
request.Timeout = 30;
request.BeginGetResponse(AsyncCallback, request);
}
public void AsyncCallback(IAsyncResult result)
{
Console.WriteLine("In AsyncCallback");
HttpWebRequest request = result.AsyncState as HttpWebRequest;
using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(result))
using (Stream stream = response.GetResponseStream())
{
while(stream.CanRead)
{
var buffer = new byte[2048];
var readCount = stream.Read(buffer, 0, buffer.Length);
var line = Encoding.UTF8.GetString(buffer.Take(readCount).ToArray());
Console.WriteLine("Reader got:" + line);
}
}
}
Output:
Listener is listening...
Listener got context
Listener wrote line, taking a nap...
In AsyncCallback
Reader got:{ "foo": "bar"}
Listener wrote line, taking a nap...
Reader got:{ "foo": "bar"}
Listener wrote line, taking a nap...
Reader got:{ "foo": "bar"}
Listener wrote line, taking a nap...
Reader got:{ "foo": "bar"}
Listener wrote line, taking a nap...
Reader got:{ "foo": "bar"}
Listener wrote line, taking a nap...
Reader got:{ "foo": "bar"}

This may have to do with Delayed ACK in combination with Nagle's algorithm. It occurs when the server sends multiple small responses in a row.
On the server side, the first response is sent, but subsequent response data chunks are only sent when the server has received an ACK from the client, or until there is enough data for a big packet to send (Nagle's algorithm).
On the client side, the first bit of response is received, but the ACK is not sent immediately - since traditional applications have a request-response-request-response behavior, it assumes it can send the ACK along with the next request - which in your case does not happen.
After a fixed amount of time (500ms?) it decides to send the ACK anyway, causing the server to send the next packages it has accumulated sofar.
The problem (if this is indeed the problem you're experiencing) can be fixed on the server side at the socket level by setting the NoDelay property, disabling Nagle's algorithm. I think you can also disable it operating system wide.
You could also temporarily disable Delayed ACK (I know windows has a registry entry for it) on the client side to see if this is indeed the problem, without having to change anything on your server. Delayed ACK prevents DDOS attacks, so make sure you restore the setting afterwards.
Sending keepalives less frequently may also help, but you'll still have a chance for the problem to occur.

C# - Unable to get server response from UDP - Black Ops Rcon

I'm creating an RCON web application for Call of Duty Black Ops. COD uses rcon and udp packets to send and receive information. Using the following code, I've been able to send and receive information with a COD4 server. Now that COD7 is out, I'm no longer receiving responses back.
Socket client = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
client.Connect(IPAddress.Parse(gameServerIP), gameServerPort);
string command;
command = password + " " + rconCommand;
byte[] bufferTemp = Encoding.ASCII.GetBytes(command);
byte[] bufferSend = new byte[bufferTemp.Length + 5];
//intial 5 characters as per standard
bufferSend[0] = byte.Parse("255");
bufferSend[1] = byte.Parse("255");
bufferSend[2] = byte.Parse("255");
bufferSend[3] = byte.Parse("255");
bufferSend[4] = byte.Parse("02");
int j = 5;
for (int i = 0; i < bufferTemp.Length; i++)
{
bufferSend[j++] = bufferTemp[i];
}
//send rcon command and get response
IPEndPoint RemoteIpEndPoint = new IPEndPoint(IPAddress.Any, 0);
client.Send(bufferSend, SocketFlags.None);
//big enough to receive response
byte[] bufferRec = new byte[65000];
client.Receive(bufferRec);
Does anyone have any ideas? Black Ops ships with its own Rcon tool that I've tried using Wireshark to capture the outgoing packets to copy. The outgoing packets between my application and theirs are next to identical, except I get no replies back when I use mine.

i knwo why cause i make a tool myself.
What's wrong in your code is that : bufferSend[4] = byte.Parse("02");
the good one is : bufferSend[4] = byte.Parse("00");
Try it, works for me!

Here my piece of code, i use a thread to run it:
how to do:
in your class set the private command, then call the worker, when thread finish, just read the private response.
private IPEndPoint ipendpoint;
private string command;
private string response;
private void worker()
{
UdpClient client = new UdpClient();
var result1 = string.Empty;
var result2 = string.Empty;
bool sent = false;
bool DoubleTrame = false;
Byte[] bufferTemp = Encoding.ASCII.GetBytes(this.command);
Byte[] bufferSend = new Byte[bufferTemp.Length + 5];
Byte[] bufferRec;
Byte[] bufferRec2;
bufferSend[0] = Byte.Parse("255");
bufferSend[1] = Byte.Parse("255");
bufferSend[2] = Byte.Parse("255");
bufferSend[3] = Byte.Parse("255");
bufferSend[4] = Byte.Parse("00");
for (int i = 0; i < bufferTemp.Length; i++)
{
bufferSend[i + 5] = bufferTemp[i];
}
while (!sent)
{
client.Send(bufferSend, bufferSend.Length, ipendpoint);
Thread.Sleep(200);
if (client.Available > 0)
{
sent = true;
if (client.Available > 1200)
{
DoubleTrame = true;
}
}
}
bufferRec = client.Receive(ref ipendpoint);
if (DoubleTrame)
{
bufferRec2 = client.Receive(ref ipendpoint);
result2 = Encoding.ASCII.GetString(bufferRec2);
if (result2.Contains("\n\n"))
{
result2 = result2.Remove(result2.IndexOf("\n\n"));
}
result2 = result2.Substring(12);
}
result1 = Encoding.ASCII.GetString(bufferRec);
this.response = result1 + result2;
}

I am doing a similar thing using vb.net
I have written rcon tools for COD MW and COD WW no problem however so far I have not been able to get a response back from my black ops server.

In fact, i have done the saame thing as you, i used WireShark to look the byte sent and received with the default rconTool provide in steam for Black Ops.
Other tip:
The command "status" give you many information about the current players but not the team of each.
You better use "teamstatus" instead, this one give you the same information but with the team of each player.
I have trouble right now to perfectly parse the response, but to give a readeable answer use this:
replace :
byte[] bufferRec = new byte[65000];
client.Receive(bufferRec);
by:
byte[] bufferRec;
while (client.Available == 0)
{
Thread.Sleep(10);
}
client.Receive(bufferRec);
string response=Encoding.ASCII.GetString(bufferRec);
var list= response.Split('\n');
This way you'll have an array with each player separate in a row.
Btw: i'm mack, i'm now registered, wasn't this night
Edit: oups, i didn't notice you've already tried the teamstatus command.
You need to look at the number return by client.Avalaible, because the server only send 1168 byte at once, so if client.Avalaible>1168, you need a second buffer to get the second flow with client.receive.
In fact there's only two possible number for client.avalaible : 1168 and 2336 (yes, the double) i don't know why, but they didn't managed to send the exact number of data, the buffer is always full or empty.
I noticed also that the second receive() is like "paste" on the first buffer.
YOu'll have at hte begginig the complementary information of the first Receive(), then the "noise" of the old one.
Just take a look, you will see what i mean.
I'm at work now, but this evening i will post my code to help.