We Keep Coding

C# Multiple Web Data/Stream Reader

I'm trying to learn programming by myself the best I can but, seems like my code isn't as productive as it can be. I'm trying to learn by doing things that I would use on a normal occasion and I can't figure out how to properly manage it, so any information would be greatly appreciated.
I'm working on a discord bot for personal use at the moment, it works fine, the loadout time is just terrible when it comes to this part of the command. Maybe cause I'm trying to have it open, read, and close multiple databases? Or is there another explanation or method to doing this that can make it load within a faster time?
string NormalExp = "0";
string IronExp = "0";
string HCExp = "0";
string UIMExp = "0";
//Normal Account
try
{
WebRequest NormalScore = WebRequest.Create("https://secure.runescape.com/m=hiscore_oldschool/index_lite.ws?player=" + player);
WebResponse NormalResponse = NormalScore.GetResponse();
using (Stream NormalDStream = NormalResponse.GetResponseStream())
{
StreamReader NormalReader = new StreamReader(NormalDStream);
string NormalResponseFromServer = NormalReader.ReadToEnd();
var _Score = NormalResponseFromServer.Split('\n');
var _Total = _Score[0];
var _TotalGet = _Total.Split(',');
var _TotalRank = _TotalGet[0];
var _TotalLevel = _TotalGet[1];
var _TotalExp = _TotalGet[2];
NormalExp = _TotalExp;
}
NormalResponse.Close();
}
catch (Exception)
{
}
//Normal Ironman
try
{
WebRequest IronScore = WebRequest.Create("https://secure.runescape.com/m=hiscore_oldschool_ironman/index_lite.ws?player=" + player);
WebResponse IronResponse = IronScore.GetResponse();
using (Stream IronDStream = IronResponse.GetResponseStream())
{
StreamReader IronReader = new StreamReader(IronDStream);
string IronResponseFromServer = IronReader.ReadToEnd();
var _Score = IronResponseFromServer.Split('\n');
var _Total = _Score[0];
var _TotalGet = _Total.Split(',');
var _TotalRank = _TotalGet[0];
var _TotalLevel = _TotalGet[1];
var _TotalExp = _TotalGet[2];
IronExp = _TotalExp;
}
IronResponse.Close();
}
catch (Exception)
{
}
//Hardcore Ironman
try
{
WebRequest HCScore = WebRequest.Create("https://secure.runescape.com/m=hiscore_oldschool_hardcore_ironman/index_lite.ws?player=" + player);
WebResponse HCResponse = HCScore.GetResponse();
using (Stream HCDStream = HCResponse.GetResponseStream())
{
StreamReader HCReader = new StreamReader(HCDStream);
string HCResponseFromServer = HCReader.ReadToEnd();
var _Score = HCResponseFromServer.Split('\n');
var _Total = _Score[0];
var _TotalGet = _Total.Split(',');
var _TotalRank = _TotalGet[0];
var _TotalLevel = _TotalGet[1];
var _TotalExp = _TotalGet[2];
HCExp = _TotalExp;
}
HCResponse.Close();
}
catch (Exception)
{
}
//Ultimate Ironman
try
{
WebRequest UIMScore = WebRequest.Create("https://secure.runescape.com/m=hiscore_oldschool_ultimate/index_lite.ws?player=" + player);
WebResponse UIMResponse = UIMScore.GetResponse();
using (Stream UIMDStream = UIMResponse.GetResponseStream())
{
StreamReader UIMReader = new StreamReader(UIMDStream);
string UIMResponseFromServer = UIMReader.ReadToEnd();
var _Score = UIMResponseFromServer.Split('\n');
var _Total = _Score[0];
var _TotalGet = _Total.Split(',');
var _TotalRank = _TotalGet[0];
var _TotalLevel = _TotalGet[1];
var _TotalExp = _TotalGet[2];
UIMExp = _TotalExp;
}
UIMResponse.Close();
}
catch (Exception)
{
}
await ReplyAsync(
$"**Normal: ** {NormalExp}\n" +
$"**Ironman: ** {IronExp}\n" +
$"**Hardcore: ** {HCExp}\n" +
$"**UIM: ** {UIMExp}");
if (Convert.ToInt64(UIMExp) == Convert.ToInt64(NormalExp))
{
await ReplyAsync("Account is a UIM");
}
else if (Convert.ToInt64(HCExp) == Convert.ToInt64(NormalExp))
{
await ReplyAsync("Account is a HC");
}
else if (Convert.ToInt64(IronExp) == Convert.ToInt64(NormalExp) && Convert.ToInt64(IronExp) > Convert.ToInt64(UIMExp + HCExp))
{
if (Convert.ToInt32(UIMExp) > 1)
{
await ReplyAsync("Account is a ~~UIM~~ Normal Ironman");
}
else if (Convert.ToInt64(HCExp) > 1)
{
await ReplyAsync("Account is a ~~HC~~ Normal Ironman");
}
else
{
await ReplyAsync("Account is a Normal Ironman");
}
}
else
{
if (Convert.ToInt64(UIMExp) > 1 && Convert.ToInt64(IronExp) > 1)
{
await ReplyAsync("Account is a ~~UIM~~, ~~Ironman~~, normal player.");
}
else if (Convert.ToInt64(HCExp) > 1 && Convert.ToInt64(IronExp) > 1)
{
await ReplyAsync("Account is a ~~HC~~, ~~Ironman~~, normal player.");
}
else if (Convert.ToInt64(IronExp) > 1 && Convert.ToInt64(HCExp) == 0 && Convert.ToInt64(UIMExp) == 0)
{
await ReplyAsync("Account is a ~~Ironman~~ normal player.");
}
else
{
await ReplyAsync("Account is a Normal Player");
}
}

In general, sending requests is an expensive operation but you can improve it by changing your method.
Try to use HttpClient instead of WebRequest
and I suggest reading about Async/Sync operations

Importing files into SQLite database not using all cores

Context
I am importing IMDB database files into an SQLite database with the help of EntityFrameworkCore. In fact, two files, the titles.basics and the titles.akas (which is linked to basics via its movie ID).
At first, I had a single thread reading lines from basics and loop through akas until it changes of ID. Though, there was an issue there and most of all, it was too slow. So, I decided to create a multithread code that would read both files at the same time and another combining akas with the appropriate movie.
I am currently importing so I still do not know if my issue is fixed (probably it is). Though, it is still too much slow for me.
Issue
The combining part is still very slow, but more importantly, I can see my process is only using around 12% of CPU which corresponds to only 1/8 of total usage and I have 8 physical cores. So, it really seems the process is only using 1 core.
I am not giving any code here, as having a minimal testable code wouldn't mean anything. Though, you can see both versions here:
https://cints.net/public/Imdb-MultiThread.cs.txt
using com.cyberinternauts.all.MediaRecognizer.Database;
using com.cyberinternauts.all.MediaRecognizer.Models.Metas;
using Microsoft.EntityFrameworkCore;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Net;
using System.Threading;
using System.Threading.Tasks;
namespace com.cyberinternauts.all.MediaRecognizer.MetaSources
{
class Imdb : MediaSource
{
private const string TITLES_FILE = "title.basics.tsv.gz";
private const string AKAS_FILE = "title.akas.tsv.gz";
private readonly string temporaryFolder = #"c:\temp\";
private readonly string baseUrl = "https://datasets.imdbws.com/";
private readonly WebClient webClient = new();
MediaRecognizerContext db = new();
private IQueryable<MetaMovie> imdbMovies = null;
private async Task<bool> GatherFilesAsync()
{
var totalFilesGathered = 0;
var filesToDownload = new string[] { AKAS_FILE, TITLES_FILE };
foreach(var fileToDownload in filesToDownload)
{
var compressedFile = temporaryFolder + fileToDownload;
if (!File.Exists(compressedFile) || !File.GetLastWriteTime(compressedFile).Date.Equals(DateTime.Today))
{
await GatherFileAsync(fileToDownload);
totalFilesGathered++;
}
}
return totalFilesGathered != 0;
}
private async Task GatherFileAsync(string fileName)
{
var compressedFile = temporaryFolder + fileName;
var uncompressedFile = temporaryFolder + Path.GetFileNameWithoutExtension(compressedFile);
await webClient.DownloadFileTaskAsync(baseUrl + fileName, compressedFile);
using Stream fd = File.Create(uncompressedFile);
using Stream fs = File.OpenRead(compressedFile);
using Stream csStream = new GZipStream(fs, CompressionMode.Decompress);
var buffer = new byte[1024];
int nRead;
while ((nRead = await csStream.ReadAsync(buffer, 0, buffer.Length)) > 0)
{
await fd.WriteAsync(buffer, 0, nRead);
}
}
private async Task LoadMetaDataAsync()
{
//return; //TODO: Remove this line
//TODO: Reactivate this line
//if (!await GatherFilesAsync()) return;
var titlesFile = temporaryFolder + Path.GetFileNameWithoutExtension(TITLES_FILE);
var akasFile = temporaryFolder + Path.GetFileNameWithoutExtension(AKAS_FILE);
var dbLock = new SemaphoreSlim(1);
var akasLock = new SemaphoreSlim(1);
var currentTitlesAkasLock = new SemaphoreSlim(1);
var associateLock = new SemaphoreSlim(1);
using (var db = new MediaRecognizerContext())
{
db.ChangeTracker.AutoDetectChangesEnabled = false;
var titles = new ConcurrentDictionary<string, MetaMovie>();
var readTitles = Task.Factory.StartNew(() =>
{
Parallel.ForEach(File.ReadLines(titlesFile), (titleLine, _, readingIndex) =>
{
if (readingIndex == 0) return; // Skipping columns titles line
var movieInfos = titleLine.Split("\t", StringSplitOptions.None);
dbLock.Wait();
MetaMovie metaMovie = db.MetaMovies.Where(m => m.ExternalId == movieInfos[0]).Include(m => m.Titles).FirstOrDefault();
dbLock.Release();
if (metaMovie == null)
{
int totalMinutes = -1;
if (!int.TryParse(movieInfos[7], out totalMinutes))
{
totalMinutes = -1;
}
metaMovie = new MetaMovie
{
ExternalId = movieInfos[0],
MetaSource = nameof(Imdb),
MovieType = movieInfos[1],
Title = movieInfos[3],
TotalMinutes = totalMinutes,
Genres = movieInfos[8]
};
metaMovie.Titles = new List<MetaTitle>();
if (int.TryParse(movieInfos[5], out int startYear))
{
metaMovie.StartYear = new DateTime(startYear, 1, 1);
}
else
{
metaMovie.StartYear = new DateTime(9999, 1, 1);
}
if (int.TryParse(movieInfos[6], out int endYear))
{
metaMovie.EndYear = new DateTime(endYear, 1, 1);
}
else
{
metaMovie.EndYear = metaMovie.StartYear;
}
}
titles.TryAdd(metaMovie.ExternalId, metaMovie);
});
});
var akas = new Dictionary<string, List<MetaTitle>>();
var currentTitlesAkas = new ConcurrentDictionary<string, int>();
var readAkas = Task.Factory.StartNew(() =>
{
Parallel.ForEach(File.ReadLines(akasFile), (akaLine, _, readingIndex) =>
{
if (readingIndex == 0) return; // Skipping columns titles line
currentTitlesAkasLock.Wait();
var titleInfos = akaLine.Split("\t", StringSplitOptions.None);
var externalId = titleInfos[0];
if (!currentTitlesAkas.ContainsKey(externalId))
{
currentTitlesAkas.TryAdd(externalId, 1);
}
else
{
currentTitlesAkas[externalId]++;
}
currentTitlesAkasLock.Release();
var metaTitle = new MetaTitle
{
MetaMovie = null,
Text = titleInfos[2],
Region = titleInfos[3],
Language = titleInfos[4]
};
akasLock.Wait();
List<MetaTitle> titleAkas;
if (!akas.ContainsKey(externalId))
{
titleAkas = new List<MetaTitle>();
akas.Add(externalId, titleAkas);
}
else
{
titleAkas = akas[externalId];
}
titleAkas.Add(metaTitle);
akasLock.Release();
currentTitlesAkasLock.Wait();
currentTitlesAkas[externalId]--;
currentTitlesAkasLock.Release();
});
});
var savingCounter = 0;
var associate = Task.Factory.StartNew(() =>
{
Parallel.For(1, Environment.ProcessorCount * 10, async (_) =>
{
var isAssociating = true;
do
{
var externalId = string.Empty;
var currentTitleAkaRemoved = false;
currentTitlesAkasLock.Wait();
foreach (var curExternalId in currentTitlesAkas.Keys.OrderBy(t => t))
{
if (currentTitlesAkas[curExternalId] == 0)
{
externalId = curExternalId;
break;
}
}
if (externalId != String.Empty)
{
currentTitleAkaRemoved = currentTitlesAkas.TryRemove(externalId, out int useless0); // Removing so other threads won't take it
}
isAssociating = !readAkas.IsCompleted || !readTitles.IsCompleted || !currentTitlesAkas.IsEmpty;
currentTitlesAkasLock.Release();
if (String.IsNullOrEmpty(externalId) || !currentTitleAkaRemoved) continue;
if (titles.TryGetValue(externalId, out MetaMovie metaMovie))
{
akasLock.Wait();
var titleAkas = akas[externalId];
akas.Remove(externalId);
akasLock.Release();
var changedMovie = false;
var movieAkas = metaMovie.Titles.Select(t => t).ToList(); // Clone list
foreach (var metaTitle in titleAkas)
{
var existingTitle = movieAkas.Where(t => t.Text == metaTitle.Text && t.Region == metaTitle.Region && t.Language == metaTitle.Language).FirstOrDefault();
if (existingTitle == null)
{
changedMovie = true;
metaMovie.Titles.Add(metaTitle);
}
else
{
movieAkas.Remove(existingTitle);
}
}
foreach (var movieTitle in movieAkas)
{
changedMovie = true;
metaMovie.Titles.Remove(movieTitle);
}
dbLock.Wait();
if (metaMovie.Id == 0)
{
db.Add(metaMovie);
}
else if (changedMovie)
{
db.Update(metaMovie);
}
dbLock.Release();
currentTitlesAkasLock.Wait();
currentTitlesAkas.TryRemove(externalId, out int uselessOut); // Free memory
isAssociating = !readAkas.IsCompleted || !readTitles.IsCompleted || !currentTitlesAkas.IsEmpty;
currentTitlesAkasLock.Release();
titles.TryRemove(externalId, out MetaMovie uselessOut2); // Free memory
associateLock.Wait();
savingCounter++;
var localSavingCounter = savingCounter;
associateLock.Release();
if (localSavingCounter != 0 && localSavingCounter % 1000 == 0)
{
var ttt = currentTitlesAkas.Where(t => t.Value > 0);
dbLock.Wait();
await db.SaveChangesAsync();
dbLock.Release();
Console.WriteLine("Saved " + localSavingCounter);
}
}
else if (!readTitles.IsCompleted) // If reading titles is not ended, then maybe it was not read yet... otherwise, it doesn't exist
{
currentTitlesAkasLock.Wait();
currentTitlesAkas.TryAdd(externalId, 0); // Readd because still no movie associated
currentTitlesAkasLock.Release();
}
} while (isAssociating);
});
});
Task.WaitAll(readTitles, readAkas, associate);
await db.SaveChangesAsync();
}
}
public async override Task<IEnumerable<MetaMovie>> FindMediasAsync(DirectoryInfo directory)
{
await LoadMetaDataAsync();
var movie = await ExtractInfosAsync(directory);
if (movie == null) return null;
if (imdbMovies == null)
{
imdbMovies = db.MetaMovies.Where(m => m.MetaSource == nameof(Imdb) && m.MovieType == "movie");
}
return FindCorrespondances(imdbMovies, movie);
}
}
}
https://cints.net/public/Imdb-SingleThread.cs.txt
using com.cyberinternauts.all.MediaRecognizer.Database;
using com.cyberinternauts.all.MediaRecognizer.Models.Metas;
using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
namespace com.cyberinternauts.all.MediaRecognizer.MetaSources
{
class Imdb : MediaSource
{
private const string TITLES_FILE = "title.basics.tsv.gz";
private const string AKAS_FILE = "title.akas.tsv.gz";
private readonly string temporaryFolder = #"c:\temp\";
private readonly string baseUrl = "https://datasets.imdbws.com/";
private readonly WebClient webClient = new();
MediaRecognizerContext db = new();
private IQueryable<MetaMovie> imdbMovies = null;
private async Task<bool> GatherFilesAsync()
{
var totalFilesGathered = 0;
var filesToDownload = new string[] { AKAS_FILE, TITLES_FILE };
foreach(var fileToDownload in filesToDownload)
{
var compressedFile = temporaryFolder + fileToDownload;
if (!File.Exists(compressedFile) || !File.GetLastWriteTime(compressedFile).Date.Equals(DateTime.Today))
{
await GatherFileAsync(fileToDownload);
totalFilesGathered++;
}
}
return totalFilesGathered != 0;
}
private async Task GatherFileAsync(string fileName)
{
var compressedFile = temporaryFolder + fileName;
var uncompressedFile = temporaryFolder + Path.GetFileNameWithoutExtension(compressedFile);
await webClient.DownloadFileTaskAsync(baseUrl + fileName, compressedFile);
using Stream fd = File.Create(uncompressedFile);
using Stream fs = File.OpenRead(compressedFile);
using Stream csStream = new GZipStream(fs, CompressionMode.Decompress);
var buffer = new byte[1024];
int nRead;
while ((nRead = await csStream.ReadAsync(buffer, 0, buffer.Length)) > 0)
{
await fd.WriteAsync(buffer, 0, nRead);
}
}
private async Task LoadMetaDataAsync()
{
//return; //TODO: Remove this line
//TODO: Reactivate this line
//if (!await GatherFilesAsync()) return;
var titlesFile = temporaryFolder + Path.GetFileNameWithoutExtension(TITLES_FILE);
var akasFile = temporaryFolder + Path.GetFileNameWithoutExtension(AKAS_FILE);
var titlesLines = File.ReadLines(titlesFile);
var akasLines = File.ReadLines(akasFile);
var titlesIterator = titlesLines.GetEnumerator();
titlesIterator.MoveNext(); // Skip columns headers
var akasIterator = akasLines.GetEnumerator();
akasIterator.MoveNext();
akasIterator.MoveNext(); // Done twice to skip columns headers
var currentAka = akasIterator.Current;
var savingCounter = 0;
using (var db = new MediaRecognizerContext())
{
db.ChangeTracker.AutoDetectChangesEnabled = false;
while (titlesIterator.MoveNext())
{
var titleLine = titlesIterator.Current;
var movieInfos = titleLine.Split("\t", StringSplitOptions.None);
MetaMovie metaMovie = db.MetaMovies.Where(m => m.ExternalId == movieInfos[0]).FirstOrDefault();
var isNewMovie = false;
if (metaMovie == null)
{
int totalMinutes = -1;
if (!int.TryParse(movieInfos[7], out totalMinutes))
{
totalMinutes = -1;
}
isNewMovie = true;
metaMovie = new MetaMovie
{
ExternalId = movieInfos[0],
MetaSource = nameof(Imdb),
MovieType = movieInfos[1],
Title = movieInfos[3],
TotalMinutes = totalMinutes,
Genres = movieInfos[8]
};
metaMovie.Titles = new List<MetaTitle>();
if (int.TryParse(movieInfos[5], out int startYear))
{
metaMovie.StartYear = new DateTime(startYear, 1, 1);
}
else
{
metaMovie.StartYear = new DateTime(9999, 1, 1);
}
if (int.TryParse(movieInfos[6], out int endYear))
{
metaMovie.EndYear = new DateTime(endYear, 1, 1);
}
else
{
metaMovie.EndYear = metaMovie.StartYear;
}
}
var movieAkasIds = metaMovie.Titles.Select(t => t.Id).ToList();
var titleInfos = currentAka?.Split("\t", StringSplitOptions.None);
while (currentAka != null && int.Parse(titleInfos[0][2..]) <= int.Parse(metaMovie.ExternalId[2..]))
{
if (titleInfos[0] == metaMovie.ExternalId)
{
var metaTitle = new MetaTitle
{
MetaMovie = metaMovie,
Text = titleInfos[2],
Region = titleInfos[3],
Language = titleInfos[4]
};
var existingTitle = metaMovie.Titles.Where(t => t.Text == metaTitle.Text && t.Region == metaTitle.Region && t.Language == metaTitle.Language).FirstOrDefault();
if (existingTitle == null)
{
metaMovie.Titles.Add(metaTitle);
}
else
{
movieAkasIds.Remove(existingTitle.Id);
}
}
else
{
var a = 1;
}
akasIterator.MoveNext();
currentAka = akasIterator.Current;
titleInfos = currentAka.Split("\t", StringSplitOptions.None);
}
foreach(var movieTitleId in movieAkasIds)
{
metaMovie.Titles.Remove(metaMovie.Titles.Where(t => t.Id == movieTitleId).FirstOrDefault());
}
if (isNewMovie)
{
db.Add(metaMovie);
}
else
{
db.Update(metaMovie);
}
savingCounter++;
if (savingCounter % 10000 == 0)
{
await db.SaveChangesAsync();
Console.WriteLine("Saved " + savingCounter);
}
}
await db.SaveChangesAsync();
}
}
public async override Task<IEnumerable<MetaMovie>> FindMediasAsync(DirectoryInfo directory)
{
await LoadMetaDataAsync();
var movie = await ExtractInfosAsync(directory);
if (movie == null) return null;
if (imdbMovies == null)
{
imdbMovies = db.MetaMovies.Where(m => m.MetaSource == nameof(Imdb) && m.MovieType == "movie");
}
return FindCorrespondances(imdbMovies, movie);
}
}
}
In the multithread version, the slow part is in the method LoadMetaDataAsync and more precisely in var associate = Task.Factory.StartNew(() => code part.
This is in development and cleaning, splitting will done after I have the appropriate result/speed.

Case closed. I returned to the single thread version and I found my initial issue (my code was supposing the files were in order, which they were partially).
Thank you for all people that participated.

How to detect edited or deleted messages on telegram TLSharp

I am trying to detect which message is edited or deleted on a subscribed channel on telegram with TLSharp library in c#.
1- in a while(true) loop I am getting latest updates.
2- when I delete or edit a message for test, I receive TLUpdateChannelTooLong only.
3- then I use client.GetHistoryAsync function to get channel messages, and check their EditDate.
But I don't know how much should I go deep in history and I can not find deleted message with this code easily.
Is there any solution to find deleted/edited messages easy and safe?
Part of my code:
state = await client.SendRequestAsync<TLState>(new TLRequestGetState());
while (true)
{
await Task.Delay(1000);
var req = new TLRequestGetDifference() { Date = state.Date, Pts = state.Pts, Qts = state.Qts };
TLDifference diff = null;
try
{
diff = await client.SendRequestAsync<TLAbsDifference>(req) as TLDifference;
}
catch (Exception ex)
{
HandleThisException(ex);
}
//--
if (diff != null)
{
state = await client.SendRequestAsync<TLState>(new TLRequestGetState());
foreach (var upd in diff.OtherUpdates.OfType<TLUpdateNewChannelMessage>())
{
var tm = (upd.Message as TLMessage);
if (tm == null) { continue; } // ?
var textMessage = tm.Message;
if (tm.Media != null)
{
if (tm.Media.GetType().ToString() == "TeleSharp.TL.TLMessageMediaPhoto")
{
var tLMessageMediaPhoto = (tm.Media as TLMessageMediaPhoto);
textMessage = tLMessageMediaPhoto.Caption;
}
}
try
{
var from = (tm.ToId as TLPeerChannel).ChannelId;
long replyTo = tm.ReplyToMsgId == null ? 0 : (long)tm.ReplyToMsgId;
await AnalyzeNewMessage( ... );
}
catch (Exception exParsing)
{
HandleThisException(exParsing);
}
}
// Checking Edited/Deleted Messages
foreach(var upLong in diff.OtherUpdates.OfType<TLUpdateChannelTooLong>())
{
TLChannel theChat = null;
foreach(var chat in diff.Chats.OfType<TLChannel>())
{
if(chat.Id == upLong.ChannelId) { theChat = chat; break; }
}
if (theChat != null)
{
var x = await client.GetHistoryAsync(
new TLInputPeerChannel { ChannelId = theChat.Id, AccessHash = (long)theChat.AccessHash },
0,-1,2
); // checking only 2 last messages!
var ChMsgs = x as TLChannelMessages;
foreach (var msg in ChMsgs.Messages.OfType<TLMessage>())
{
if(msg.EditDate != null)
{
var txt = msg.Message;
if (msg.Media != null)
{
if (msg.Media.GetType().ToString() == "TeleSharp.TL.TLMessageMediaPhoto")
{
txt = (msg.Media as TLMessageMediaPhoto).Caption;
}
}
await AnalyzeEditedMessage( ... );
}
}
}
}
}
}

API face identification with detectFacesAsync Microsoft Cognitive service C#

I have a problem I can not use DetectFacesAsync to make a detection of faces in a preview that I get from the webcam of my PC. Could you help me?
private async Task GetPreviewFrameAsSoftwareBitmapAsync()
{
var previewProperties = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;
var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, (int)previewProperties.Width, (int)previewProperties.Height);
using (var currentFrame = await _mediaCapture.GetPreviewFrameAsync(videoFrame))
{
SoftwareBitmap previewFrame = currentFrame.SoftwareBitmap;
FrameInfoTextBlock.Text = String.Format("{0}x{1} {2}", previewFrame.PixelWidth, previewFrame.PixelHeight, previewFrame.BitmapPixelFormat);
if (ShowFrameCheckBox.IsChecked == true)
{
var sbSource = new SoftwareBitmapSource();
await sbSource.SetBitmapAsync(previewFrame);
PreviewFrameImage.Source = sbSource;
}
}
}
private async Task DetectAsync()
{
IList<DetectedFace> faces = null;
const BitmapPixelFormat InputPixelFormat = BitmapPixelFormat.Nv12;
using (VideoFrame destinationPreviewFrame = new VideoFrame(InputPixelFormat, 640, 480))
{
await this._mediaCapture.GetPreviewFrameAsync(destinationPreviewFrame);
if(FaceDetector.IsBitmapPixelFormatSupported(InputPixelFormat))
{
faces = await this.DetectFacesAsync(SoftwareBitmap, PreviewFrameImage);
foreach (var face in faces)
{
SoftwareBitmap convertedBitmap = SoftwareBitmap.Convert(destinationPreviewFrame.SoftwareBitmap, BitmapPixelFormat.Rgba16);
Byte[] rawBytes = await GetBytesFromBitmap(convertedBitmap, BitmapEncoder.BmpEncoderId, face.FaceBox);
using (Stream stream = rawBytes.AsBuffer().AsStream())
{
var faceAttributesToReturn = new List<FaceAttributeType>()
{
FaceAttributeType.Age,
FaceAttributeType.Emotion,
FaceAttributeType.Hair
};
Face[] detectedFaces = await this.faceServiceClient.DetectAsync(stream, true, true, faceAttributesToReturn);
Debug.Assert(detectedFaces.Length > 0);
}
}
}
}
}
private async Task<byte[]> GetBytesFromBitmap(SoftwareBitmap soft, Guid encoderId, BitmapBounds bounds)
{
byte[] array = null;
using (var ms = new InMemoryRandomAccessStream())
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(encoderId, ms);
encoder.SetSoftwareBitmap(soft);
encoder.BitmapTransform.Bounds = bounds;
await encoder.FlushAsync();
array = new byte[ms.Size];
await ms.ReadAsync(array.AsBuffer(), (uint)ms.Size, InputStreamOptions.None);
}
return array;
}
private async Task GetPreviewFrameAsD3DSurfaceAsync()
{
using (var currentFrame = await _mediaCapture.GetPreviewFrameAsync())
{
if (currentFrame.Direct3DSurface != null)
{
var surface = currentFrame.Direct3DSurface;
FrameInfoTextBlock.Text = String.Format("{0}x{1} {2}", surface.Description.Width, surface.Description.Height, surface.Description.Format);
}
{
SoftwareBitmap previewFrame = currentFrame.SoftwareBitmap;
FrameInfoTextBlock.Text = String.Format("{0}x{1} {2}", previewFrame.PixelWidth, previewFrame.PixelHeight, previewFrame.BitmapPixelFormat);
}
PreviewFrameImage.Source = null;
}
}
I don't understand where the error comes from, DetecFacesAsync is a method I can call to Windows.Media.Face.Analysis. Excuse me for my approximative english.

Issue with System.Threading.Tasks.Task.Factory.StartNew() in UWP?

I am working on Windows 10 UWP app and my requirement is to upload 5 images on the server with unique value. So, I have used System.Threading.Tasks.Task.Factory.StartNew().Now, when I checked while debugging, I found that randomly sometimes for 2 images, it sends same unique key. Can someone suggest is it better to use System.Threading.Tasks.Task.Factory.StartNew()?
All the images are sent using a web service. My sample code for this is following
WebServiceUtility serviceUtility = new WebServiceUtility();
List<System.Threading.Tasks.Task> tasks = new List<System.Threading.Tasks.Task>();
var cancelSource = new CancellationTokenSource();
cancellationToken = cancelSource.Token;
System.Threading.Tasks.Task currentTask = null;
List<System.Threading.Tasks.Task> uploadTasks = new List<System.Threading.Tasks.Task>();
List<string> uploadedImageIdList = new List<string>();
foreach (var image in _imageCollection)
{
if (!cancellationToken.IsCancellationRequested)
{
currentTask = await System.Threading.Tasks.Task.Factory.StartNew(async () =>
{
string imageName = string.Empty;
string imagePath = string.Empty;
if (image.IsEvidenceImage)
{
imageName = image.EvidencePath.Split('\\')[1];
imagePath = image.EvidencePath;
}
else
{
imageName = image.EvidencePath.Split('#')[1].Split('\\')[1];
imagePath = image.EvidencePath.Split('#')[1];
}
byte[] _imageAsByteArray = await GetEvidenceFromIsoStore(imagePath);
if (null != _imageAsByteArray && _imageAsByteArray.Length > 0)
{
IRestResponse response = await serviceUtility.UploadImage
(_imageAsByteArray, imageName,
new RequestDataGenerator().generateRequestDataForMediaUpload(
(null != _imageItem.I_IS_PRIMARY && "1".Equals(_imageItem.I_IS_PRIMARY) ? "1" : "0"),
evidenceName
));
if (response != null && response.RawBytes.Length > 0)
{
var successMessage = MCSExtensions.CheckWebserviceResponseCode(response.StatusCode);
if (successMessage.Equals(Constants.STATUS_CODE_SUCCESS))
{
byte[] decryptedevidenceresponse = WebserviceED.finaldecryptedresponse(response.RawBytes);
string responseString = Encoding.UTF8.GetString(decryptedevidenceresponse, 0, decryptedevidenceresponse.Length);
JObject reponseObject = JObject.Parse(responseString);
//Debug.WriteLine("Evidence Upload Response : " + Environment.NewLine);
uploadedimageIdList.Add(reponseObject["P_RET_ID"].ToString());
try
{
if (image.IsEvidenceImage)
{
if (await FileExists(image.EvidencePath))
{
StorageFile file = await localFolder.GetFileAsync(image.EvidencePath);
await file.DeleteAsync();
}
}
else
{
string[] evidenceMedia = image.EvidencePath.Split('#');
foreach (string evidenceItem in evidenceMedia)
{
if (await FileExists(evidenceItem))
{
StorageFile file = await localFolder.GetFileAsync(evidenceItem);
await file.DeleteAsync();
}
}
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
else
{
UserMessageUtil.ShowMessage(successMessage);
}
}
}
}, cancellationToken);
uploadTasks.Add(currentTask);
}
}
await System.Threading.Tasks.Task.WhenAll(uploadTasks.ToArray());

Just make it a separate method:
...
foreach (var image in _imageCollection)
{
if (!cancellationToken.IsCancellationRequested)
{
currentTask = UploadAsync(...);
uploadTasks.Add(currentTask);
}
}
await Task.WhenAll(uploadTasks);
async Task UploadAsync(...)
{
string imageName = string.Empty;
string imagePath = string.Empty;
...
}
Or, a bit more simply at the call site:
...
var uploadTasks = _imageCollection.Select(x => UploadAsync(...));
await Task.WhenAll(uploadTasks);