C# Roslyn CompletionService where to get method overload information - c#

I've got a method that returning back from CompletionService.GetDescriptionAsync(Document, CompletionItem) gives me the following description:
void SQL.GetSQLiteDB(string url) (+ 1 overload)
This is a method I made on a Xamarin project, here are both method signatures:
public static void GetSQLiteDB(string url);
public static string GetSQLiteDB(string url, string name);
What's the Roslyn way to get information on both?
Here's how I'm setting up completions:
async Task InitCodeCompletion()
{
host = MefHostServices.Create(MefHostServices.DefaultAssemblies);
workspace = new AdhocWorkspace(host);
Type[] types =
{
typeof(object),
typeof(System.Linq.Enumerable),
typeof(System.Collections.IEnumerable),
typeof(Console),
typeof(System.Reflection.Assembly),
typeof(List<>),
typeof(Type),
typeof(SQL)
};
imports = types.Select(x => x.Namespace).Distinct().ToImmutableArray();
assemblies = types.Select(x => x.Assembly).Distinct().ToImmutableArray();
references = assemblies.Select(t => MetadataReference.CreateFromFile(t.Location) as MetadataReference).ToImmutableArray();
compilationOptions = new CSharpCompilationOptions(
OutputKind.DynamicallyLinkedLibrary,
usings: imports);
projectInfo = ProjectInfo.Create(ProjectId.CreateNewId(), VersionStamp.Create(), "Script", "Script", LanguageNames.CSharp, isSubmission: true)
.WithMetadataReferences(references).WithCompilationOptions(compilationOptions);
project = workspace.AddProject(projectInfo);
documentInfo = DocumentInfo.Create(DocumentId.CreateNewId(project.Id), "Script", sourceCodeKind: SourceCodeKind.Script,
loader: TextLoader.From(TextAndVersion.Create(SourceText.From(""), VersionStamp.Create())));
document = workspace.AddDocument(documentInfo);
var services = workspace.Services;
completionService = CompletionService.GetService(document);
}
async Task<CodeCompletionResults> GetCompletions(string code)
{
string codeModified = "using SQL = XamTestNET5.Services.SQLiteGeneratorService; " + Environment.NewLine;
codeModified += "using HtmlSvc = XamTestNET5.Services.HtmlRetrievalService;" + Environment.NewLine;
// ^^^ The above two lines set up some simple namespace aliases in my project, if you know how to put this in a separate project document and use it in code completion please let me know in comments as otherwise doing so gives me an exception that you can't have multiple syntax trees
codeModified += code;
var source = SourceText.From(codeModified);
document = document.WithText(source);
// cursor position is at the end
var position = source.Length;
var completions = await completionService.GetCompletionsAsync(document, position);
return new CodeCompletionResults() { InputCode = code, ModifiedCode = codeModified, Completions = completions };
}
Here's how I'm getting them now and putting them in a browser control:
private async void CSharpShellEnvironment_EntryCodeCompletionEntry(object sender, CSharpShellEnvironment.EntryEventArgs e)
{
if (e.Value != "")
{
CodeCompletionResults results = await GetCompletions(e.Value);
CompletionList list = results.Completions;
if (list != null)
{
if (list.Items != null)
{
StringBuilder sb = new StringBuilder();
foreach (var item in list.Items)
{
string spanText = (item.Span.Start != item.Span.End) ? results.ModifiedCode.Substring(item.Span.Start, item.Span.Length) : "";
bool recommended = spanText == "" ? true : item.DisplayText.StartsWith(spanText);
if (recommended)
{
string fText = item.DisplayText.Substring(spanText.Length);
string props = "";
foreach(var p in item.Properties)
{
props += $"<span data-key=\"{p.Key}\" data-value=\"{p.Value}\"></span>";
}
string tags = "";
foreach(var t in item.Tags)
{
tags += $"<span data-tag=\"{t}\"></span>";
}
string descStr = "";
if (item.Tags != null)
{
if (item.Tags.Where(x => x.ToLower() == "method").FirstOrDefault() != null && item.Tags.Where(x => x.ToLower() == "public").FirstOrDefault() != null)
{
var desc = await completionService.GetDescriptionAsync(document, item);
descStr += $"<span data-desc=\"{desc.Text}\">";
foreach(var part in desc.TaggedParts)
{
descStr += $"<span data-desc-part-tag=\"{part.Tag}\" data-desc-part-text=\"{part.Text}\"></span>";
}
descStr += "</span>";
}
}
sb.AppendLine($"<div class=\"codecompleteentry\" data-display-text=\"{item.DisplayText}\" data-span-text=\"{spanText}\" data-final-text=\"{fText}\">{props}{tags}{descStr}{fText}</div>");
}
}
string scriptInputClick = "Array.prototype.forEach.call(document.getElementsByClassName('codecompleteentry'), function(el) { el.addEventListener('click', function(elem) { var text = { MessageType: 'CodeCompletion', Parameters: JSON.stringify({ DataDisplayText: el.getAttribute('data-display-text'), DataSpanText: el.getAttribute('data-span-text'), DataFinalText: el.getAttribute('data-final-text') }), Message: el.innerText }; window.chrome.webview.postMessage(text); } ); });";
sb.AppendLine($"<script type=\"text/javascript\">{scriptInputClick}</script>");
env.EnterCodeCompletionResponse(sb.ToString());
}
else
{
env.EnterCodeCompletionResponse(strNoSuggestions);
}
}
else
{
env.EnterCodeCompletionResponse(strNoSuggestions);
}
}
else
{
env.EnterCodeCompletionResponse(strNoSuggestions);
}
}

It seems on the surface that CompletionSurface has everything you need, but it doesn't, you need to reference the Document's SemanticModel in order to get all of the signature overloads of a method when the user types ( on a method during code completion.
It wasn't very obvious to me until I started looking through the RoslynPad source, which I recommend doing for a practical example: https://github.com/aelij/RoslynPad
List<IEnumerable<ReferencedSymbol>> allMethodRefs = new List<IEnumerable<ReferencedSymbol>>();
async Task<CodeCompletionResults> GetCompletions(string code)
{
string codeModified = "using SQL = XamTestNET5.Services.SQLiteGeneratorService; " + Environment.NewLine;
codeModified += "using HtmlSvc = XamTestNET5.Services.HtmlRetrievalService;" + Environment.NewLine;
// ^^^ I put my namespace aliases in the same SyntaxTree for now,
// I'd like a better solution though.
codeModified += code;
var source = SourceText.From(codeModified);
document = document.WithText(source);
// cursor position is at the end
var position = source.Length;
var completions = await completionService.GetCompletionsAsync(document, position);
syntaxRoot = await document.GetSyntaxRootAsync();
semanticModel = await document.GetSemanticModelAsync();
var methods = syntaxRoot.DescendantNodes().OfType<InvocationExpressionSyntax>();
allMethodRefs = new List<IEnumerable<ReferencedSymbol>>();
if (methods != null)
{
if (methods.Count() > 0)
{
foreach(var m in methods)
{
var info = semanticModel.GetSymbolInfo(m);
if (info.Symbol != null)
{
allMethodRefs.Add(await SymbolFinder.FindReferencesAsync(info.Symbol, solution));
}
else
{
foreach(var symbol in info.CandidateSymbols)
{
allMethodRefs.Add(await SymbolFinder.FindReferencesAsync(symbol, solution));
}
}
}
}
}
return new CodeCompletionResults() { InputCode = code, ModifiedCode = codeModified, Completions = completions };
}

Related

Import a COM register dll in application in KTM

I create a dll it contains an entry point. The whole code is working fine. I build the dll, i created a console project that import the reference and so on.
My Issue concerns importing the dll into an application that requires the DLL to be registered as COM. Based on documentation i found, I used Regasm.exe:
"C:\Windows\Microsoft.NET\Framework\v4.0.30319\RegAsm.exe" "mydll.dll" /tlb /codebase
This generates a .tlb file, so far so good. I do also successfully import the tlb. My issue is that even if the dll is correctly found, I can't access to any methods of it.
[![enter image description here][1]][1]
here is a sample of code:
namespace mynamespace
{
public class AppManager
{
public Task<SoapResponse> SoapApiCaller(String url, String val1, String val2)
{
return Utils.queryResponse(url, val1, val2);
}
}
}
my utility class is just making the call. It should be splitted too. but that's not the point;
namespace mynamespace
{
public class Utils
{
public static async Task<SoapResponse> queryResponse(String url, String val1,
String val2)
{
SoapResponse soapResponse= new SoapResponse();
// check before values
bool isValid = true;
var exc = "";
if (url == null)
{
isValid = false;
exc += "L'url est requise\n";
}
if (val1 == null)
{
isValid = false;
exc += "Va\n";
}
if (val2 == null)
{
isValid = false;
exc += "val 2 required\n";
}
if (!isValid)
{
throw new Exception(exc);
}
// SOAP enveloppe
string xmlStr = "<?xml version=\"1.0\" encoding=\"utf-8\" ?> ...";
StringContent stringContent = new StringContent(xmlStr, Encoding.UTF8, "text/xml");
try
{
var client = new HttpClient();
SoapResponse soapResponse = new SoapResponse();
var response = await client.PostAsync(url, stringContent);
var soapResponse = await response.Content.ReadAsStringAsync();
var statusCode = (int)response.StatusCode;
XDocument document = XDocument.Parse(soapResponse);
if (statusCode == 200)
{
var WebServiceFirstResult =
document.Descendants().FirstOrDefault(p => p.Name.LocalName == "WebServiceFirstResult");
var WebServiceSecondResult =
document.Descendants().FirstOrDefault(p => p.Name.LocalName == "WebServiceSecondResult");
XText xText = (XText)WebServiceSecondResult.FirstNode;
var xStr = xText.ToString().Replace(">", ">").Replace("<", "<");
xStr = xStr
.Replace("]]>", "")
.Replace("<![CDATA[<", "");
string[] strArr = xStr.Split('<');
// debugger here !
for (var i = 0; i < strArr.Length; i++)
{
// nettoyer autour de "CDATA"
Console.WriteLine(strArr[i]);
Console.Write("");
// on splitte avec >
if (strArr[i].Split('>')[0] == "VAL1")
{
response.VAL1= strArr[i].Split('>')[1];
}
if (strArr[i].Split('>')[0] == "VAL2")
{
response.VAL2= strArr[i].Split('>')[1];
}
if (strArr[i].Split('>')[0] == "VAL3")
{
response.VAL3 = strArr[i].Split('>')[1];
}
soapResponse.message = "OK";
soapResponse.statusCode = (int)response.StatusCode;
soapResponse.response= response;
}
else
{
soapResponse.statusCode = (int)response.StatusCode;
if (document.Descendants().FirstOrDefault(p => p.Name.LocalName == "faultstring") != null)
{
var errMessageElt = document.Descendants().FirstOrDefault(p => p.Name.LocalName == "faultstring");
if (errMessageElt.Value != null)
{
int index = errMessageElt.Value.IndexOfAny(new char[] { '\r', '\n' });
string firstline = index == -1 ? errMessageElt.Value : errMessageElt.Value.Substring(0, index);
var exceptionMessage = "KO : erreur " + (int)response.StatusCode + " "+ firstline;
throw new Exception(exceptionMessage);
}
}
else
{
throw new Exception("KO : erreur " + (int)response.StatusCode + " " + soapResponse);
}
}
}
catch (Exception e)
{
Console.WriteLine(e);
throw new Exception(e.Message);
}
return soapResponse;
}
}
}
the assembly information contain the following information:
[![enter image description here][2]][2]
[![enter image description here][3]][3]
It's using .NET Framework 4.8 and c# 8
[1]: https://i.stack.imgur.com/xjbeP.png
[2]: https://i.stack.imgur.com/j3wjq.png
[3]: https://i.stack.imgur.com/kVDFM.png

Sendgrid dynamic data not being sent to Template

This is how I send my email:
public async System.Threading.Tasks.Task<string> SendInternalEmail(BasicEmailStructureViewModel Structure, List<string> AdditionalVariables, TenantEmailTemplate tenantEmailTemplate, TenantCommunication tenantCommunication, string ReceiverId, string ReceiverEmail, string ReceiverName, string CampaignName)
{
try
{
var client = new SendGridClient(tenantCommunication.SendgridApiKey);
var message = new SendGridMessage();
message.SetFrom(new EmailAddress(tenantCommunication.SendgridPrimarySender, tenantCommunication.SendgridPrimarySenderTag));
message.AddTo(new EmailAddress(ReceiverEmail, $"{ReceiverName}"));
message.Subject = tenantEmailTemplate.Subject;
message.SetTemplateId(tenantEmailTemplate.TemplateId);
List<string> jsonVars = new List<string>();
var subjectString = #$"""subject"":""{tenantEmailTemplate.Subject}""";
jsonVars.Add(subjectString);
foreach (PropertyInfo prop in Structure.GetType().GetProperties())
{
var variableString = #$"""{prop.Name}"":""{prop.GetValue(Structure, null)}""";
}
for (var i = 0; i < AdditionalVariables.Count; i++)
{
jsonVars.Add(AdditionalVariables[i]);
}
var flattenList = "{" + string.Join(",", jsonVars) + "}";
var emailData = Newtonsoft.Json.JsonConvert.DeserializeObject<object>(flattenList);
message.SetTemplateData(emailData);
if (CampaignName != null && CampaignName != "")
{
message.AddCustomArg("CampaignName", CampaignName);
}
var response = await client.SendEmailAsync(message);
if (response.IsSuccessStatusCode == true)
{
return Guid.NewGuid().ToString();
}
else
{
var errorMessage = response.Body.ReadAsStringAsync().Result;
return errorMessage;
}
}
catch(Exception e)
{
if (e != null)
{
return e.Message;
}
}
return "Invalid Email";
}
A typical input to this function will be like this:
var variableString =
#$"""verification_link"":""www.website.com?Key={Input.Key}""";
My email sends normally, however, none of the variables that I have set have been sent through. This is based roughly off the template sample on github: https://github.com/sendgrid/sendgrid-csharp/blob/main/examples/templates/templates.cs
Is there another sample I can use or what is the correct way to send variables dynamically?
I don't think that is the best way to construct the JSON for your dynamic template variables.
Would it be possible for you to build the variables as an object and then serialize them. Like:
var templateData = new {
subject = tenantEmailTemplate.Subjectm
otherVariableName = otherVariable
};
string emailData = JsonConvert.SerializeObject(templateData);
message.SetTemplateData(emailData);

Importing files into SQLite database not using all cores

Context
I am importing IMDB database files into an SQLite database with the help of EntityFrameworkCore. In fact, two files, the titles.basics and the titles.akas (which is linked to basics via its movie ID).
At first, I had a single thread reading lines from basics and loop through akas until it changes of ID. Though, there was an issue there and most of all, it was too slow. So, I decided to create a multithread code that would read both files at the same time and another combining akas with the appropriate movie.
I am currently importing so I still do not know if my issue is fixed (probably it is). Though, it is still too much slow for me.
Issue
The combining part is still very slow, but more importantly, I can see my process is only using around 12% of CPU which corresponds to only 1/8 of total usage and I have 8 physical cores. So, it really seems the process is only using 1 core.
I am not giving any code here, as having a minimal testable code wouldn't mean anything. Though, you can see both versions here:
https://cints.net/public/Imdb-MultiThread.cs.txt
using com.cyberinternauts.all.MediaRecognizer.Database;
using com.cyberinternauts.all.MediaRecognizer.Models.Metas;
using Microsoft.EntityFrameworkCore;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Net;
using System.Threading;
using System.Threading.Tasks;
namespace com.cyberinternauts.all.MediaRecognizer.MetaSources
{
class Imdb : MediaSource
{
private const string TITLES_FILE = "title.basics.tsv.gz";
private const string AKAS_FILE = "title.akas.tsv.gz";
private readonly string temporaryFolder = #"c:\temp\";
private readonly string baseUrl = "https://datasets.imdbws.com/";
private readonly WebClient webClient = new();
MediaRecognizerContext db = new();
private IQueryable<MetaMovie> imdbMovies = null;
private async Task<bool> GatherFilesAsync()
{
var totalFilesGathered = 0;
var filesToDownload = new string[] { AKAS_FILE, TITLES_FILE };
foreach(var fileToDownload in filesToDownload)
{
var compressedFile = temporaryFolder + fileToDownload;
if (!File.Exists(compressedFile) || !File.GetLastWriteTime(compressedFile).Date.Equals(DateTime.Today))
{
await GatherFileAsync(fileToDownload);
totalFilesGathered++;
}
}
return totalFilesGathered != 0;
}
private async Task GatherFileAsync(string fileName)
{
var compressedFile = temporaryFolder + fileName;
var uncompressedFile = temporaryFolder + Path.GetFileNameWithoutExtension(compressedFile);
await webClient.DownloadFileTaskAsync(baseUrl + fileName, compressedFile);
using Stream fd = File.Create(uncompressedFile);
using Stream fs = File.OpenRead(compressedFile);
using Stream csStream = new GZipStream(fs, CompressionMode.Decompress);
var buffer = new byte[1024];
int nRead;
while ((nRead = await csStream.ReadAsync(buffer, 0, buffer.Length)) > 0)
{
await fd.WriteAsync(buffer, 0, nRead);
}
}
private async Task LoadMetaDataAsync()
{
//return; //TODO: Remove this line
//TODO: Reactivate this line
//if (!await GatherFilesAsync()) return;
var titlesFile = temporaryFolder + Path.GetFileNameWithoutExtension(TITLES_FILE);
var akasFile = temporaryFolder + Path.GetFileNameWithoutExtension(AKAS_FILE);
var dbLock = new SemaphoreSlim(1);
var akasLock = new SemaphoreSlim(1);
var currentTitlesAkasLock = new SemaphoreSlim(1);
var associateLock = new SemaphoreSlim(1);
using (var db = new MediaRecognizerContext())
{
db.ChangeTracker.AutoDetectChangesEnabled = false;
var titles = new ConcurrentDictionary<string, MetaMovie>();
var readTitles = Task.Factory.StartNew(() =>
{
Parallel.ForEach(File.ReadLines(titlesFile), (titleLine, _, readingIndex) =>
{
if (readingIndex == 0) return; // Skipping columns titles line
var movieInfos = titleLine.Split("\t", StringSplitOptions.None);
dbLock.Wait();
MetaMovie metaMovie = db.MetaMovies.Where(m => m.ExternalId == movieInfos[0]).Include(m => m.Titles).FirstOrDefault();
dbLock.Release();
if (metaMovie == null)
{
int totalMinutes = -1;
if (!int.TryParse(movieInfos[7], out totalMinutes))
{
totalMinutes = -1;
}
metaMovie = new MetaMovie
{
ExternalId = movieInfos[0],
MetaSource = nameof(Imdb),
MovieType = movieInfos[1],
Title = movieInfos[3],
TotalMinutes = totalMinutes,
Genres = movieInfos[8]
};
metaMovie.Titles = new List<MetaTitle>();
if (int.TryParse(movieInfos[5], out int startYear))
{
metaMovie.StartYear = new DateTime(startYear, 1, 1);
}
else
{
metaMovie.StartYear = new DateTime(9999, 1, 1);
}
if (int.TryParse(movieInfos[6], out int endYear))
{
metaMovie.EndYear = new DateTime(endYear, 1, 1);
}
else
{
metaMovie.EndYear = metaMovie.StartYear;
}
}
titles.TryAdd(metaMovie.ExternalId, metaMovie);
});
});
var akas = new Dictionary<string, List<MetaTitle>>();
var currentTitlesAkas = new ConcurrentDictionary<string, int>();
var readAkas = Task.Factory.StartNew(() =>
{
Parallel.ForEach(File.ReadLines(akasFile), (akaLine, _, readingIndex) =>
{
if (readingIndex == 0) return; // Skipping columns titles line
currentTitlesAkasLock.Wait();
var titleInfos = akaLine.Split("\t", StringSplitOptions.None);
var externalId = titleInfos[0];
if (!currentTitlesAkas.ContainsKey(externalId))
{
currentTitlesAkas.TryAdd(externalId, 1);
}
else
{
currentTitlesAkas[externalId]++;
}
currentTitlesAkasLock.Release();
var metaTitle = new MetaTitle
{
MetaMovie = null,
Text = titleInfos[2],
Region = titleInfos[3],
Language = titleInfos[4]
};
akasLock.Wait();
List<MetaTitle> titleAkas;
if (!akas.ContainsKey(externalId))
{
titleAkas = new List<MetaTitle>();
akas.Add(externalId, titleAkas);
}
else
{
titleAkas = akas[externalId];
}
titleAkas.Add(metaTitle);
akasLock.Release();
currentTitlesAkasLock.Wait();
currentTitlesAkas[externalId]--;
currentTitlesAkasLock.Release();
});
});
var savingCounter = 0;
var associate = Task.Factory.StartNew(() =>
{
Parallel.For(1, Environment.ProcessorCount * 10, async (_) =>
{
var isAssociating = true;
do
{
var externalId = string.Empty;
var currentTitleAkaRemoved = false;
currentTitlesAkasLock.Wait();
foreach (var curExternalId in currentTitlesAkas.Keys.OrderBy(t => t))
{
if (currentTitlesAkas[curExternalId] == 0)
{
externalId = curExternalId;
break;
}
}
if (externalId != String.Empty)
{
currentTitleAkaRemoved = currentTitlesAkas.TryRemove(externalId, out int useless0); // Removing so other threads won't take it
}
isAssociating = !readAkas.IsCompleted || !readTitles.IsCompleted || !currentTitlesAkas.IsEmpty;
currentTitlesAkasLock.Release();
if (String.IsNullOrEmpty(externalId) || !currentTitleAkaRemoved) continue;
if (titles.TryGetValue(externalId, out MetaMovie metaMovie))
{
akasLock.Wait();
var titleAkas = akas[externalId];
akas.Remove(externalId);
akasLock.Release();
var changedMovie = false;
var movieAkas = metaMovie.Titles.Select(t => t).ToList(); // Clone list
foreach (var metaTitle in titleAkas)
{
var existingTitle = movieAkas.Where(t => t.Text == metaTitle.Text && t.Region == metaTitle.Region && t.Language == metaTitle.Language).FirstOrDefault();
if (existingTitle == null)
{
changedMovie = true;
metaMovie.Titles.Add(metaTitle);
}
else
{
movieAkas.Remove(existingTitle);
}
}
foreach (var movieTitle in movieAkas)
{
changedMovie = true;
metaMovie.Titles.Remove(movieTitle);
}
dbLock.Wait();
if (metaMovie.Id == 0)
{
db.Add(metaMovie);
}
else if (changedMovie)
{
db.Update(metaMovie);
}
dbLock.Release();
currentTitlesAkasLock.Wait();
currentTitlesAkas.TryRemove(externalId, out int uselessOut); // Free memory
isAssociating = !readAkas.IsCompleted || !readTitles.IsCompleted || !currentTitlesAkas.IsEmpty;
currentTitlesAkasLock.Release();
titles.TryRemove(externalId, out MetaMovie uselessOut2); // Free memory
associateLock.Wait();
savingCounter++;
var localSavingCounter = savingCounter;
associateLock.Release();
if (localSavingCounter != 0 && localSavingCounter % 1000 == 0)
{
var ttt = currentTitlesAkas.Where(t => t.Value > 0);
dbLock.Wait();
await db.SaveChangesAsync();
dbLock.Release();
Console.WriteLine("Saved " + localSavingCounter);
}
}
else if (!readTitles.IsCompleted) // If reading titles is not ended, then maybe it was not read yet... otherwise, it doesn't exist
{
currentTitlesAkasLock.Wait();
currentTitlesAkas.TryAdd(externalId, 0); // Readd because still no movie associated
currentTitlesAkasLock.Release();
}
} while (isAssociating);
});
});
Task.WaitAll(readTitles, readAkas, associate);
await db.SaveChangesAsync();
}
}
public async override Task<IEnumerable<MetaMovie>> FindMediasAsync(DirectoryInfo directory)
{
await LoadMetaDataAsync();
var movie = await ExtractInfosAsync(directory);
if (movie == null) return null;
if (imdbMovies == null)
{
imdbMovies = db.MetaMovies.Where(m => m.MetaSource == nameof(Imdb) && m.MovieType == "movie");
}
return FindCorrespondances(imdbMovies, movie);
}
}
}
https://cints.net/public/Imdb-SingleThread.cs.txt
using com.cyberinternauts.all.MediaRecognizer.Database;
using com.cyberinternauts.all.MediaRecognizer.Models.Metas;
using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
namespace com.cyberinternauts.all.MediaRecognizer.MetaSources
{
class Imdb : MediaSource
{
private const string TITLES_FILE = "title.basics.tsv.gz";
private const string AKAS_FILE = "title.akas.tsv.gz";
private readonly string temporaryFolder = #"c:\temp\";
private readonly string baseUrl = "https://datasets.imdbws.com/";
private readonly WebClient webClient = new();
MediaRecognizerContext db = new();
private IQueryable<MetaMovie> imdbMovies = null;
private async Task<bool> GatherFilesAsync()
{
var totalFilesGathered = 0;
var filesToDownload = new string[] { AKAS_FILE, TITLES_FILE };
foreach(var fileToDownload in filesToDownload)
{
var compressedFile = temporaryFolder + fileToDownload;
if (!File.Exists(compressedFile) || !File.GetLastWriteTime(compressedFile).Date.Equals(DateTime.Today))
{
await GatherFileAsync(fileToDownload);
totalFilesGathered++;
}
}
return totalFilesGathered != 0;
}
private async Task GatherFileAsync(string fileName)
{
var compressedFile = temporaryFolder + fileName;
var uncompressedFile = temporaryFolder + Path.GetFileNameWithoutExtension(compressedFile);
await webClient.DownloadFileTaskAsync(baseUrl + fileName, compressedFile);
using Stream fd = File.Create(uncompressedFile);
using Stream fs = File.OpenRead(compressedFile);
using Stream csStream = new GZipStream(fs, CompressionMode.Decompress);
var buffer = new byte[1024];
int nRead;
while ((nRead = await csStream.ReadAsync(buffer, 0, buffer.Length)) > 0)
{
await fd.WriteAsync(buffer, 0, nRead);
}
}
private async Task LoadMetaDataAsync()
{
//return; //TODO: Remove this line
//TODO: Reactivate this line
//if (!await GatherFilesAsync()) return;
var titlesFile = temporaryFolder + Path.GetFileNameWithoutExtension(TITLES_FILE);
var akasFile = temporaryFolder + Path.GetFileNameWithoutExtension(AKAS_FILE);
var titlesLines = File.ReadLines(titlesFile);
var akasLines = File.ReadLines(akasFile);
var titlesIterator = titlesLines.GetEnumerator();
titlesIterator.MoveNext(); // Skip columns headers
var akasIterator = akasLines.GetEnumerator();
akasIterator.MoveNext();
akasIterator.MoveNext(); // Done twice to skip columns headers
var currentAka = akasIterator.Current;
var savingCounter = 0;
using (var db = new MediaRecognizerContext())
{
db.ChangeTracker.AutoDetectChangesEnabled = false;
while (titlesIterator.MoveNext())
{
var titleLine = titlesIterator.Current;
var movieInfos = titleLine.Split("\t", StringSplitOptions.None);
MetaMovie metaMovie = db.MetaMovies.Where(m => m.ExternalId == movieInfos[0]).FirstOrDefault();
var isNewMovie = false;
if (metaMovie == null)
{
int totalMinutes = -1;
if (!int.TryParse(movieInfos[7], out totalMinutes))
{
totalMinutes = -1;
}
isNewMovie = true;
metaMovie = new MetaMovie
{
ExternalId = movieInfos[0],
MetaSource = nameof(Imdb),
MovieType = movieInfos[1],
Title = movieInfos[3],
TotalMinutes = totalMinutes,
Genres = movieInfos[8]
};
metaMovie.Titles = new List<MetaTitle>();
if (int.TryParse(movieInfos[5], out int startYear))
{
metaMovie.StartYear = new DateTime(startYear, 1, 1);
}
else
{
metaMovie.StartYear = new DateTime(9999, 1, 1);
}
if (int.TryParse(movieInfos[6], out int endYear))
{
metaMovie.EndYear = new DateTime(endYear, 1, 1);
}
else
{
metaMovie.EndYear = metaMovie.StartYear;
}
}
var movieAkasIds = metaMovie.Titles.Select(t => t.Id).ToList();
var titleInfos = currentAka?.Split("\t", StringSplitOptions.None);
while (currentAka != null && int.Parse(titleInfos[0][2..]) <= int.Parse(metaMovie.ExternalId[2..]))
{
if (titleInfos[0] == metaMovie.ExternalId)
{
var metaTitle = new MetaTitle
{
MetaMovie = metaMovie,
Text = titleInfos[2],
Region = titleInfos[3],
Language = titleInfos[4]
};
var existingTitle = metaMovie.Titles.Where(t => t.Text == metaTitle.Text && t.Region == metaTitle.Region && t.Language == metaTitle.Language).FirstOrDefault();
if (existingTitle == null)
{
metaMovie.Titles.Add(metaTitle);
}
else
{
movieAkasIds.Remove(existingTitle.Id);
}
}
else
{
var a = 1;
}
akasIterator.MoveNext();
currentAka = akasIterator.Current;
titleInfos = currentAka.Split("\t", StringSplitOptions.None);
}
foreach(var movieTitleId in movieAkasIds)
{
metaMovie.Titles.Remove(metaMovie.Titles.Where(t => t.Id == movieTitleId).FirstOrDefault());
}
if (isNewMovie)
{
db.Add(metaMovie);
}
else
{
db.Update(metaMovie);
}
savingCounter++;
if (savingCounter % 10000 == 0)
{
await db.SaveChangesAsync();
Console.WriteLine("Saved " + savingCounter);
}
}
await db.SaveChangesAsync();
}
}
public async override Task<IEnumerable<MetaMovie>> FindMediasAsync(DirectoryInfo directory)
{
await LoadMetaDataAsync();
var movie = await ExtractInfosAsync(directory);
if (movie == null) return null;
if (imdbMovies == null)
{
imdbMovies = db.MetaMovies.Where(m => m.MetaSource == nameof(Imdb) && m.MovieType == "movie");
}
return FindCorrespondances(imdbMovies, movie);
}
}
}
In the multithread version, the slow part is in the method LoadMetaDataAsync and more precisely in var associate = Task.Factory.StartNew(() => code part.
This is in development and cleaning, splitting will done after I have the appropriate result/speed.
Case closed. I returned to the single thread version and I found my initial issue (my code was supposing the files were in order, which they were partially).
Thank you for all people that participated.

PHAssetChangeRequest Cocoa error -1

I need to save downloaded video to gallery on iPhone, but getting error:
The operation couldnt be completed. (Cocoa error -1/)
Tried also to do this through webClient.DownloadDataAsync(), getting same error. Here is my listing:
public async Task<string> DownloadFile(string fileUri)
{
var tcs = new TaskCompletionSource<string>();
string fileName = fileUri.Split('/').Last();
var documentsDirectory = Environment.GetFolderPath(Environment.SpecialFolder.Personal);
string videoFileName = System.IO.Path.Combine(documentsDirectory, fileName);
var webClient = new WebClient();
webClient.DownloadFileCompleted += (s, e) =>
{
var authStatus = await PHPhotoLibrary.RequestAuthorizationAsync();
if(authStatus == PHAuthorizationStatus.Authorized){
var fetchOptions = new PHFetchOptions();
var collections = PHAssetCollection.FetchAssetCollections(PHAssetCollectionType.Album, PHAssetCollectionSubtype.Any, fetchOptions);
collection = collections.firstObject as PHAssetCollection;
PHPhotoLibrary.SharedPhotoLibrary.PerformChanges(() => {
var assetCreationRequest = PHAssetChangeRequest.FromVideo(NSUrl.FromFileName(videoFileName));
var assetPlaceholder = assetCreationRequest.PlaceholderForCreatedAsset;
var albumChangeRequest = PHAssetCollectionChangeRequest.ChangeRequest(collection);
albumChangeRequest.AddAssets(new PHObject[] { assetPlaceholder });
}, delegate (bool status, NSError error) {
if (status)
{
Console.Write("Video added");
tcs.SetResult("success");
}
});
}
try
{
webClient.DownloadFileAsync(new Uri(fileUri), videoFileName);
}
catch (Exception e)
{
tcs.SetException(e);
}
return await tcs.Task;
}
Any help would be appreciated. Thanks.
(Cocoa error -1/)
Are you sure that you actually have valid data/mp4 from your download?
Are you using SSL (https), otherwise have you applied for an ATS exception in your info.plist?
Check the phone/simulator console output for errors concerning ATS
Note: I typically use NSUrlSession directly to avoid the HttpClient wrapper...
Example using NSUrlSession task:
var videoURL = NSUrl.FromString(urlString);
var videoPath = NSFileManager.DefaultManager.GetUrls(NSSearchPathDirectory.DocumentDirectory, NSSearchPathDomain.User)[0];
NSUrlSession.SharedSession.CreateDownloadTask(videoURL, (location, response, createTaskError) =>
{
if (location != null && createTaskError == null)
{
var destinationURL = videoPath.Append(response?.SuggestedFilename ?? videoURL.LastPathComponent, false);
// If file exists, it is already downloaded, but for debugging, delete it...
if (NSFileManager.DefaultManager.FileExists(destinationURL.Path)) NSFileManager.DefaultManager.Remove(destinationURL, out var deleteError);
NSFileManager.DefaultManager.Move(location, destinationURL, out var moveError);
if (moveError == null)
{
PHPhotoLibrary.RequestAuthorization((status) =>
{
if (status.HasFlag(PHAuthorizationStatus.Authorized))
{
PHPhotoLibrary.SharedPhotoLibrary.PerformChanges(() =>
{
PHAssetChangeRequest.FromVideo(destinationURL);
}, (complete, requestError) =>
{
if (!complete && requestError != null)
Console.WriteLine(requestError);
});
}
});
}
else
Console.WriteLine(moveError);
}
else
Console.WriteLine(createTaskError);
}).Resume();
Note: To confirm your code, try using a known valid secure URL source:
https://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4 via “Video For Everybody” Test Page

call back query data from inline keyboard

I'm trying to get the data from an inline keyboard, I've searched a lot but unfortunately didn't get my answer and non of codes worked for me. Here's my code please help me
static void Main(string[] args){
InlineKeyboardButton[][] buttons =
new InlineKeyboardButton[][]{
new InlineKeyboardButton[]{newInlineKeyboardButton() { Text = "show Channel", CallbackData = "Data1" }},
new InlineKeyboardButton[]{new InlineKeyboardButton() { Text = "show Website", CallbackData = "Data2" }}};
inlineKeyboardMarkup = new InlineKeyboardMarkup() { InlineKeyboard = buttons };
Task.Run(() => RunBot());
Console.ReadLine();
} // End of main method
public static async Task RunBot(){
while (true){
var u = await bot.MakeRequestAsync(new GetUpdates() { Offset
= offset });
foreach (var update in u)
{
offset = update.UpdateId + 1;
var text = update.Message.Text;
// here I want to get the data like this, but it doesn't work
if (update.ChosenInlineResult != null){
Console.WriteLine("Chosen Inline Result: " +
update.ChosenInlineResult.ToString());
}
switch(text){
case "Something":{
var req = new SendMessage(update.Message.Chat.Id, "راهنما") { ReplyMarkup = inlineKeyboardMarkup };
await bot.MakeRequestAsync(req);
break;
}
}
}
}
}
you must replace this
if (update.ChosenInlineResult != null){
Console.WriteLine("Chosen Inline Result: " +
update.ChosenInlineResult.ToString());
}
with something like This :
if (update.CallbackQuery != null)
{
Console.WriteLine(val.CallbackQuery.Message+"-"+val.CallbackQuery.Data);
}

Categories