We Keep Coding

How to have an AWS Lambda/Rekognition Function return an array of object keys

This feels like a simple question and I feel like I am overthinking it. I am doing an AWS project that will compare face(s) on an image to a database (s3bucket) of other faces. So far, I have a lambda function for the comparefacerequest, a class library which invokes the function, and an UWP that inputs the image file and outputs a result. It has worked so far being based on boolean (true or false) functions, but now I want it to instead return what face(s) are recognized via an array. I struggling at implementing this.
Below is my lambda function. I have adjusted the task to be an Array instead of a bool and changed the return to be an array. At the bottom, I have created a global variable class with a testing array so I could attempt to reference the array elsewhere.
public class Function
{
//Function
public async Task<Array> FunctionHandler(string input, ILambdaContext context)
{
//number of matched faces
int matched = 0;
//Client setup
var rekognitionclient = new AmazonRekognitionClient();
var s3client = new AmazonS3Client();
//Create list of target images
ListObjectsRequest list = new ListObjectsRequest
{
BucketName = "bucket2"
};
ListObjectsResponse listre = await s3client.ListObjectsAsync(list);
//loop of list
foreach (Amazon.S3.Model.S3Object obj in listre.S3Objects)
{
//face request with input and obj.key images
var comparefacesrequest = new CompareFacesRequest
{
SourceImage = new Image
{
S3Object = new S3Objects
{
Bucket = "bucket1",
Name = input
}
},
TargetImage = new Image
{
S3Object = new S3Objects
{
Bucket = "bucket2",
Name = obj.Key
}
},
};
//compare with confidence of 95 (subject to change) to current target image
var detectresponse = await rekognitionclient.CompareFacesAsync(comparefacesrequest);
detectresponse.FaceMatches.ForEach(match =>
{
ComparedFace face = match.Face;
if (match.Similarity > 95)
{
//if face detected, raise matched
matched++;
for(int i = 0; i < Globaltest.testingarray.Length; i++)
{
if (Globaltest.testingarray[i] == "test")
{
Globaltest.testingarray[i] = obj.Key;
}
}
}
});
}
//Return true or false depending on if it is matched
if (matched > 0)
{
return Globaltest.testingarray;
}
return Globaltest.testingarray;
}
}
public static class Globaltest
{
public static string[] testingarray = { "test", "test", "test" };
}
Next, is my invoke request in my class library. It has so far been based on the lambda outputting a boolean result, but I thought, "hey, it is parsing the result, it should be fine, right"? I do convert the result to a string, as there is no GetArray, from what I know.
public async Task<bool> IsFace(string filePath, string fileName)
{
await UploadS3(filePath, fileName);
AmazonLambdaClient client = new AmazonLambdaClient(accessKey, secretKey, Amazon.RegionEndpoint.USWest2);
InvokeRequest ir = new InvokeRequest();
ir.InvocationType = InvocationType.RequestResponse;
ir.FunctionName = "ImageTesting";
ir.Payload = "\"" + fileName + "\"";
var result = await client.InvokeAsync(ir);
var strResponse = Encoding.ASCII.GetString(result.Payload.ToArray());
if (bool.TryParse(strResponse, out bool result2))
{
return result2;
}
return false;
}
Finally, here is the section of my UWP where I perform the function. I am referencing the lambda client via "using Lambdaclienttest" (name of lamda project, and this is its only instance I use the reference though). When I run my project, I do still get a face detected when it should, but the Globaltest.testingarray[0] is still equal to "test".
var Facedetector = new FaceDetector(Credentials.accesskey, Credentials.secretkey);
try
{
var result = await Facedetector.IsFace(filepath, filename);
if (result)
{
textBox1.Text = "There is a face detected";
textBox2.Text = Globaltest.testingarray[0];
}
else
{
textBox1.Text = "Try Again";
}
}
catch
{
textBox1.Text = "Please use a photo";
}
Does anyone have any suggestions?

API is mixing up data from different devices

I have an API that has devices firing data to it at the same time or within a few milliseconds. What I am finding is that the data is getting mixed up. The data is sent every five minutes (on the clock 05, 10, 15 etc.) I have an execution filter that traps the URL data coming in so I always have a real source, then it goes to the endpoint and then onto processing. For example, there will a be random five minute period missing. When I debug step by step with the missing URL from the execution filter it works fine. By that I mean I take the URL and debug, then it inserts.
In summary, I have device id 1 and device id 2.I will get missing intervals even though, I can see the data has hit the execution filter.
I am assuming that the API is not handling these as separate transactions, but somehow mixing them up together, hence the data missing and the serial numbers appearing in the wrong place, such that data from id 1 is appearing in id 2 vice versa etc.
API End Point:
public class SomeController : ApiController
{
[HttpGet]
[ExecutionFilter]
public async Task<HttpResponseMessage> Get([FromUri] FixedDataModel fdm)
{
var reply = new HttpResponseMessage();
string url = HttpUtility.UrlDecode(HttpContext.Current.Request.QueryString.ToString());
if (url.Contains("timestamp"))
{
reply = TimeSyncValidation.TimeSync;
return reply;
}
else if (!url.Contains("timestamp"))
{
reply = await Task.Run(() => DeviceClass.DeviceApiAsync(fdm, url));
}
return reply;
}
}
Processing class:
namespace API.Services
{
public class DeviceClass
{
private static string serialNumber;
private static byte chk;
private static string channelName, channelReadingNumber, channelValue, queryString, readingDate;
private static int colonPosition, chanCountFrom, equalsPosition;
private static bool checkSumCorrect;
public static HttpResponseMessage DeviceApiAsync(FixedDataModel fdm, string urlQqueryString)
{
Guid guid = Guid.NewGuid();
//ExecutionTrackerHandler.Guid = guid;
//Remove question mark
var q = urlQqueryString;
queryString = q.Substring(0);
var items = HttpUtility.ParseQueryString(queryString);
serialNumber = items["se"];
//Store raw uri for fault finding
var rawUri = new List<RawUriModel>
{
new RawUriModel
{
UniqueId = guid,
RawUri = q,
TimeStamp = DateTime.Now
}
};
//Checksum validation
chk = Convert.ToByte(fdm.chk);
checkSumCorrect = CheckSumValidator.XorCheckSum(queryString, chk);
if (!checkSumCorrect)
{
return ValidationResponseMessage.ResponseHeaders("Checksum");
}
//Create list of items that exist in URL
var urldata = new UrlDataList
{
UrlData = queryString.Split('&').ToList(),
};
var data = new List<UriDataModel>();
//Split the URL string into its parts
foreach (var item in urldata.UrlData)
{
colonPosition = item.IndexOf(":");
chanCountFrom = colonPosition + 1;
equalsPosition = item.LastIndexOf("=");
if (colonPosition == -1)
{
channelName = item.Substring(0, equalsPosition);
channelReadingNumber = "";
channelValue = item.Substring(item.LastIndexOf("=") + 1);
}
else
{
channelName = item.Substring(0, colonPosition);
channelReadingNumber = item.Substring(chanCountFrom, equalsPosition - chanCountFrom);
channelValue = item.Substring(item.LastIndexOf("=") + 1);
if (channelName == "atime" || channelName == "adate")
{
readingDate = DateValidator.CreateDate(channelValue);
}
};
bool nullFlag = false;
if (channelValue == null)
nullFlag = true;
bool missingFlag = false;
if (channelValue == "x") {
missingFlag = true;
channelValue = "0";
}
//Add data to model ready for DB insert.
data.Add(new UriDataModel
{
uid = guid,
SerialNumber = serialNumber,
ChannelName = channelName,
ChannelReadingNumber = channelReadingNumber,
ChannelValue = channelValue.Replace(",", "."),
ReadingDate = readingDate,
TimeStamp = DateTime.Now.ToString("yyyy-MM-dd HH:mm"),
Processed = false,
NullFlag = nullFlag,
MissingFlag = missingFlag
});
};
//Validate dates
var allDates = (from x in data where x.ChannelName.Contains("atime") || x.ChannelName.Contains("adate") select x.ChannelValue).ToList();
bool dateValidation = DateValidator.IsValid(allDates);
if (!dateValidation)
{
return ValidationResponseMessage.ResponseHeaders("Date");
};
//Validate values
var channels = Enum.GetNames(typeof(Channels)).ToList();
List<string> allChannelValues = data.Where(d => channels.Contains(d.ChannelName)).Select(d => d.ChannelValue).ToList();
bool valueValidation = ValueValidator.IsValid(allChannelValues);
if (!valueValidation)
{
return ValidationResponseMessage.ResponseHeaders("Values");
};
//Insert live data
var insertData = DataInsert<UriDataModel>.InsertData(data, "Staging.UriData");
if (!insertData)
{
return ValidationResponseMessage.ResponseHeaders("Sql");
}
var content = "\r\nSUCCESS\r\n";
var reply = new HttpResponseMessage(System.Net.HttpStatusCode.OK)
{
Content = new StringContent(content)
};
return reply;
}
}
}
TIA

You are using global variables and static method to process your data.
Change your method to non-static.
Each DeviceClass worker must update only its own isolated data then push that off back to controller.

How to get Bitcoin value for corresponding USD value in ASP.NET C#?

I want to get Bitcoin value for corresponding USD value and store it in table or variable. I got this URL from which I can get a Bitcoin value for USK amount. I searched on blockchain and I found this URL.
For example:
500usd = 0.76105818 btc
I tried:
https://blockchain.info/tobtc?currency=USD&value=500
at the end, its USD value which we want to convert in Bitcoin.
I want to get the result in the variable in C# (backend).
How can I accomplish this?

You need to just make call to server and parse the response.
var uri = String.Format("https://blockchain.info/tobtc?currency=USD&value={0}", 500);
WebClient client = new WebClient();
client.UseDefaultCredentials = true;
var data = client.DownloadString(uri);
var result = Convert.ToDouble(data);

Install-Package CoinMarketCapClient
using CoinMarketCap;
public static async Task<double> GetBitcoinInUsd(double usd){
//https://api.coinmarketcap.com/v1/ticker/bitcoin/
CoinMarketCapClient client = CoinMarketCapClient.GetInstance();
var entity = await client.GetTickerAsync("bitcoin");
return entity.PriceUsd * usd;
}

var uri = String.Format(#"https://blockchain.info/tobtc?currency=USD&value={0}",1);
WebClient client = new WebClient();
client.UseDefaultCredentials = true;
var data = client.DownloadString(uri);
var result = 1/Convert.ToDouble(data.Replace('.',',')); //you will receive 1 btc = result;

There are several APIs out there that will allow you to request the prices for a list of crypto currencies, and/or fiat currencies. The problem is that all the APIs do it in a disparate way. The follow on from that is that any one could be down at any given time, so you need to have some failure tolerance built in. I.e. the code should attempt to use one API, and if that fails, move to the next. Of course, this is further complicated by the fact that price is subjective and localised to a given country, exchange and so on. So, getting an accurate value is very difficult.
Here is example Crypto Compare client from CryptoCurrency.Net (https://github.com/MelbourneDeveloper/CryptoCurrency.Net/blob/master/src/CryptoCurrency.Net/APIClients/PriceEstimationClients/CryptoCompareClient.cs):
public class CryptoCompareClient : PriceEstimationClientBase, IPriceEstimationClient
{
public CryptoCompareClient(IRestClientFactory restClientFactory) : base(restClientFactory)
{
RESTClient = restClientFactory.CreateRESTClient(new Uri("https://min-api.cryptocompare.com"));
}
protected override Func<GetPricesArgs, Task<EstimatedPricesModel>> GetPricesFunc { get; } = async a =>
{
var retVal = new EstimatedPricesModel();
if (a.Currencies.ToList().Count == 0)
{
return retVal;
}
retVal.LastUpdate = DateTime.Now;
var symbolsPart = string.Join(",", a.Currencies.Select(c => c.Name));
var priceJson = await a.RESTClient.GetAsync<string>($"data/pricemultifull?fsyms={symbolsPart}&tsyms={a.FiatCurrency}");
var jObject = (JObject)JsonConvert.DeserializeObject(priceJson);
var rawNode = (JObject)jObject.First.First;
foreach (JProperty coinNode in rawNode.Children())
{
var fiatNode = (JProperty)coinNode.First().First;
var allProperties = fiatNode.First.Children().Cast<JProperty>().ToList();
var change24HourProperty = allProperties.FirstOrDefault(p => string.Compare(p.Name, "CHANGEPCT24HOUR", true) == 0);
var priceProperty = allProperties.FirstOrDefault(p => string.Compare(p.Name, "PRICE", true) == 0);
var price = (decimal)priceProperty.Value;
var change24Hour = (decimal)change24HourProperty.Value;
retVal.Result.Add(new CoinEstimate { CurrencySymbol = new CurrencySymbol(coinNode.Name), ChangePercentage24Hour = change24Hour, FiatEstimate = price, LastUpdate = DateTime.Now });
}
//Extreme hack. It's better to show zero than nothing at all and get the coins stuck
foreach (var currency in a.Currencies)
{
if (retVal.Result.FirstOrDefault(ce => ce.CurrencySymbol.Equals(currency)) == null)
{
retVal.Result.Add(new CoinEstimate { ChangePercentage24Hour = 0, CurrencySymbol = currency, FiatEstimate = 0, LastUpdate = DateTime.Now });
}
}
return retVal;
};
}
The PriceEstimationManager will flick through APIs until it finds one that works (https://github.com/MelbourneDeveloper/CryptoCurrency.Net/blob/master/src/CryptoCurrency.Net/APIClients/PriceEstimationClients/PriceEstimationManager.cs):
public class PriceEstimationManager
{
#region Fields
private readonly Collection<IPriceEstimationClient> _Clients = new Collection<IPriceEstimationClient>();
#endregion
#region Constructor
public PriceEstimationManager(IRestClientFactory restClientFactory)
{
foreach (var typeInfo in typeof(PriceEstimationManager).GetTypeInfo().Assembly.DefinedTypes)
{
var type = typeInfo.AsType();
if (typeInfo.ImplementedInterfaces.Contains(typeof(IPriceEstimationClient)))
{
_Clients.Add((IPriceEstimationClient)Activator.CreateInstance(type, restClientFactory));
}
}
}
#endregion
#region Public Methods
/// <summary>
/// TODO: This needs to be averaged. The two current clients give wildly different values. Need to include some Australian exchanges etc.
/// </summary>
public async Task<EstimatedPricesModel> GetPrices(IEnumerable<CurrencySymbol> currencySymbols, string fiatCurrency)
{
//Lets try a client that hasn't been used before if there is one
var client = _Clients.FirstOrDefault(c => c.AverageCallTimespan.TotalMilliseconds == 0);
var currencies = currencySymbols.ToList();
if (client != null)
{
try
{
return await client.GetPrices(currencies, fiatCurrency);
}
catch
{
//Do nothing
}
}
foreach (var client2 in _Clients.OrderBy(c => c.SuccessRate).ThenBy(c => c.AverageCallTimespan).ToList())
{
try
{
return await client2.GetPrices(currencies, fiatCurrency);
}
catch (Exception ex)
{
Logger.Log("Error Getting Prices", ex, nameof(PriceEstimationManager));
}
}
throw new Exception("Can't get prices");
}
#endregion
}
At a higher level, you can use the code like this (https://github.com/MelbourneDeveloper/CryptoCurrency.Net/blob/master/src/CryptoCurrency.Net.UnitTests/PricingTests.cs):
public async Task GetUSDBitcoinPrice()
{
var priceEstimationManager = new PriceEstimationManager(new RESTClientFactory());
var estimatedPrice = await priceEstimationManager.GetPrices(new List<CurrencySymbol> { CurrencySymbol.Bitcoin }, "USD");
Console.WriteLine($"Estimate: {estimatedPrice.Result.First().FiatEstimate}");
}
As more pricing clients are added, it will get more and more reliable.

C# Nested Function Wont Trigger

Hello i would like to ask you about a small problem i have with a certain program. So basically the program(console app) reads packets from other Procees(Program) and delivers the packets so they can be analyzed. The problem i stumbeled upon is that One of my nested functions wont trigger (completely). Example:
This line is lying in the Main() function. Along with the rest , this is just a clip
captureDevice.OnPacketArrival += new PacketArrivalEventHandler(Program.device_OnPacketArrival);
After which i have some console.writeline etc. Nothing special.
device_OnPacketArrival() is Function with some other Nested Functions Inside.
private static void device_OnPacketArrival(object sender, CaptureEventArgs packet)
{
DateTime date = packet.Packet.Timeval.Date;
int length = packet.Packet.Data.Length;
..........
Program.ValidateItem(first_db_id, first_bin_id, out char_id, out user_id, out type, out valid);
............
The Function that causes problem is:ValidateItem()
public static void ValidateItem(int item_Id, int item_type, out int char_id, out int user_id, out int type, out int valid)
{
valid = type = user_id = char_id = 0;
string oString = "SELECT type,char_id,user_id FROM [DB].[dbo].[Table] WHERE id = #item_id";
SqlCommand oCmd = new SqlCommand(oString, Program.s_DB);
oCmd.Parameters.AddWithValue("#item_id", item_Id);
try
{
using (SqlDataReader oReader = oCmd.ExecuteReader())
{
if (oReader.Read())
{
type = Convert.ToInt32(oReader["type"]);
char_id = Convert.ToInt32(oReader["char_id"]);
user_id = Convert.ToInt32(oReader["user_id"]);
if (type == item_type)
{
valid = 1;
}
else
{
valid = 0;
}
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
The Problem is that only the first line of the function is initialized - valid = type = user_id = char_id = 0; But if i run the function directly in Main() its working fine.

Not sure if answering your question
but in c# there is no "nested" functions , if you need to create that "illusion" try using annonymus
functions

MongoCollection.Update() using C# to update list<T>

I am using MongoVue application to show the data preview stored in "MongoDb".
In the attached image, the database name "Energy" has collection name "DataLog". In "DataLog", there are several rows. I am adding these row to the collection by reading it from a .CSV file.
Now sometimes the column name Pings has huge data [say array of 2000 items] for a single row due to which the exception occurs i.e if "MaxDocumentSize exceeds in 16MB"
Since the Pings array was huge which threw an exception and to avoid this, I removed the collection of Pings [i.e. entered blank collection] from row and tried to Insert, it went successful.
Now I want to update the Pings for the same entry, but in case the array is something like 2000 elements or above, then I wish to update it in group of 500 items [500 x 4 = 2000] in a loop.
Can anyone help me out.
** SAMPLE CODE **
private void InsertData(Datalog xiDatalog)
{
List<Ping> tempPings = new List<Ping>();
tempPings.AddRange(xiDatalog.Pings);
xiDatalog.Pings.RemoveAll(x => x.RowId != 0);
WriteConcernResult wc = mongoCollection.Insert(xiDatalog);
counter++;
var query = new QueryDocument("_id", xiDatalog.Id);
MongoCursor<Datalog> cursor = mongoCollection.FindAs<Datalog>(query);
foreach (Datalog data in cursor)
{
AddPings(data, tempPings, mongoCollection);
break;
}
}
private void AddPings(Datalog xiDatalog, List<Ping> xiPings, MongoCollection<Datalog> mongoCollection)
{
int groupCnt = 0;
int insertCnt = 0;
foreach (Ping px in xiPings)
{
xiDatalog.Pings.Add(px);
groupCnt++;
if (((int)(groupCnt / 500)) > insertCnt)
{
UpdateDataLog(xiDatalog.Id, xiDatalog.Pings, mongoCollection);
insertCnt++;
}
}
}
private bool UpdateDataLog(BsonValue Id, List<Ping> tempPings, MongoCollection<Datalog> mongoCollection)
{
bool success = false;
try
{
var query = new QueryDocument("_id", Id);
var update = Update<Datalog>.Set(e => e.Pings, tempPings);
mongoCollection.Update(query, update);
success = true;
}
catch (Exception ex)
{
string error = ex.Message;
}
return success;
}

Answer : Just modified the code to use Update.PushAll() instead of Update.Set()
Please refer below code
private bool UpdateDataLog(BsonValue Id, List<Ping> tempPings, MongoCollection<Datalog> mongoCollection)
{
bool success = false;
try
{
var query = new QueryDocument("_id", Id);
var update = Update<Datalog>.PushAll(e => e.Pings, tempPings);
mongoCollection.Update(query, update);
success = true;
}
catch (Exception ex)
{
string error = ex.Message;
}
return success;
}