We Keep Coding

How can I make the CompletionService aware of other documents in the project?

I'm building an application that allows users to define, edit and execute C# scripts.
The definition consists of a method name, an array of parameter names and the method's inner code, e.g:
Name: Script1
Parameter Names: arg1, arg2
Code: return $"Arg1: {arg1}, Arg2: {arg2}";
Based on this definition the following code can be generated:
public static object Script1(object arg1, object arg2)
{
return $"Arg1: {arg1}, Arg2: {arg2}";
}
I've successfully set up an AdhocWorkspace and a Project like this:
private readonly CSharpCompilationOptions _options = new CSharpCompilationOptions(OutputKind.ConsoleApplication,
moduleName: "MyModule",
mainTypeName: "MyMainType",
scriptClassName: "MyScriptClass"
)
.WithUsings("System");
private readonly MetadataReference[] _references = {
MetadataReference.CreateFromFile(typeof(object).Assembly.Location)
};
private void InitializeWorkspaceAndProject(out AdhocWorkspace ws, out ProjectId projectId)
{
var assemblies = new[]
{
Assembly.Load("Microsoft.CodeAnalysis"),
Assembly.Load("Microsoft.CodeAnalysis.CSharp"),
Assembly.Load("Microsoft.CodeAnalysis.Features"),
Assembly.Load("Microsoft.CodeAnalysis.CSharp.Features")
};
var partTypes = MefHostServices.DefaultAssemblies.Concat(assemblies)
.Distinct()
.SelectMany(x => x.GetTypes())
.ToArray();
var compositionContext = new ContainerConfiguration()
.WithParts(partTypes)
.CreateContainer();
var host = MefHostServices.Create(compositionContext);
ws = new AdhocWorkspace(host);
var projectInfo = ProjectInfo.Create(
ProjectId.CreateNewId(),
VersionStamp.Create(),
"MyProject",
"MyProject",
LanguageNames.CSharp,
compilationOptions: _options, parseOptions: new CSharpParseOptions(LanguageVersion.CSharp7_3, DocumentationMode.None, SourceCodeKind.Script)).
WithMetadataReferences(_references);
projectId = ws.AddProject(projectInfo).Id;
}
And I can create documents like this:
var document = _workspace.AddDocument(_projectId, "MyFile.cs", SourceText.From(code)).WithSourceCodeKind(SourceCodeKind.Script);
For each script the user defines, I'm currently creating a separate Document.
Executing the code works as well, using the following methods:
First, to compile all documents:
public async Task<Compilation> GetCompilations(params Document[] documents)
{
var treeTasks = documents.Select(async (d) => await d.GetSyntaxTreeAsync());
var trees = await Task.WhenAll(treeTasks);
return CSharpCompilation.Create("MyAssembly", trees, _references, _options);
}
Then, to create an assembly out of the compilation:
public Assembly GetAssembly(Compilation compilation)
{
try
{
using (MemoryStream ms = new MemoryStream())
{
var emitResult = compilation.Emit(ms);
if (!emitResult.Success)
{
foreach (Diagnostic diagnostic in emitResult.Diagnostics)
{
Console.Error.WriteLine("{0}: {1}", diagnostic.Id, diagnostic.GetMessage());
}
}
else
{
ms.Seek(0, SeekOrigin.Begin);
var buffer = ms.GetBuffer();
var assembly = Assembly.Load(buffer);
return assembly;
}
return null;
}
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
And, finally, to execute the script:
public async Task<object> Execute(string method, object[] params)
{
var compilation = await GetCompilations(_documents);
var a = GetAssembly(compilation);
try
{
Type t = a.GetTypes().First();
var res = t.GetMethod(method)?.Invoke(null, params);
return res;
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
So far, so good. This allows users to define scripts that can all each other.
For editing I would like to offer code completion and am currently doing this:
public async Task<CompletionList> GetCompletionList(Document doc, string code, int offset)
{
var newDoc = doc.WithText(SourceText.From(code));
_workspace.TryApplyChanges(newDoc.Project.Solution);
var completionService = CompletionService.GetService(newDoc);
return await completionService.GetCompletionsAsync(newDoc, offset);
}
NOTE: The code snippet above was updated to fix the error Jason mentioned in his answer regarding the use of doc and document. That was, indeed, due to the fact that the code shown here was extracted (and thereby modified) from my actual application code. You can find the original erroneous snippet I posted in his answer and, also, further below a new version which addresses the actual issue that was causing my problems.
The problem now is that GetCompletionsAsync is only aware of definitions within the same Document and the references used when creating the workspace and project, but it apparently does not have any reference to the other documents within the same project. So the CompletionList does not contain symbols for the other user scripts.
This seems strange, because in a "live" Visual Studio project, of course, all files within a project are aware of each other.
What am I missing? Are the project and/or workspace set up incorrectly? Is there another way of calling the CompletionService? Are the generated document codes missing something, like a common namespace?
My last resort would be to merge all methods generated from users' script definitions into one file - is there another way?
FYI, here are a few useful links that helped me get this far:
https://www.strathweb.com/2018/12/using-roslyn-c-completion-service-programmatically/
Roslyn throws The language 'C#' is not supported
Roslyn service is null
Updating AdHocWorkspace is slow
Roslyn: is it possible to pass variables to documents (with SourceCodeKind.Script)
UPDATE 1:
Thanks to Jason's answer I've updated the GetCompletionList method as follows:
public async Task<CompletionList> GetCompletionList(Document doc, string code, int offset)
{
var docId = doc.Id;
var newDoc = doc.WithText(SourceText.From(code));
_workspace.TryApplyChanges(newDoc.Project.Solution);
var currentDoc = _workspace.CurrentSolution.GetDocument(docId);
var completionService = CompletionService.GetService(currentDoc);
return await completionService.GetCompletionsAsync(currentDoc, offset);
}
As Jason pointed out, the cardinal error was not fully taking the immutability of the project and it's documents into account. The Document instance I need for calling CompletionService.GetService(doc) must be the actual instance contained in the current solution - and not the instance created by doc.WithText(...), because that instance has no knowledge of anything.
By storing the DocumentId of the original instance and using it to retrieve the updated instance within the solution, currentDoc, after applying the changes, the completion service can (as in "live" solutions) reference the other documents.
UPDATE 2: In my original question the code snippets used SourceCodeKind.Regular, but - at least in this case - it must be SourceCodeKind.Script, because otherwise the compiler will complain that top-level static methods are not allowed (when using C# 7.3). I've now updated the post.

So one thing looks a bit fishy here:
public async Task<CompletionList> GetCompletionList(Document doc, string code, int offset)
{
var newDoc = document.WithText(SourceText.From(code));
_workspace.TryApplyChanges(newDoc.Project.Solution);
var completionService = CompletionService.GetService(newDoc);
return await completionService.GetCompletionsAsync(document, offset);
}
(Note: your parameter name is "doc" but you're using "document" so I'm guessing this code is something you pared down from the full example. But just wanted to call that out since you might have introduced errors while doing that.)
So main fishy bit: Roslyn Documents are snapshots; a document is a pointer within the entire snapshot of the entire solution. Your "newDoc" is a new document with the text that you've substituted, and you're updating thew workspace to contain that. You're however still handing in the original document to GetCompletionsAsync, which means you're still asking for the old document in that case, which might have stale code. Furthermore, because it's all a snapshot, the changes made to the main workspace by calling TryApplyChanges won't in any way be reflected in your new document objects. So what I'm guessing might be happening here is you're passing in a Document object that doesn't actually have all the text documents updated at once, but most of them are still empty or something similar.

CallerMemberName parameter is blank for a specific method

I have the following Log method (for a Unit Test project):
public static void WriteLogFile<T>(T obj, [System.Runtime.CompilerServices.CallerMemberName] string callingMethod = "")
{
#if WRITELOGFILE
var path = Path.Combine(LogFileLocation, callingMethod + ".json");
var content = (typeof(T) == typeof(string)) ? (obj as string) : JsonConvert.SerializeObject(obj);
File.WriteAllText(path, content);
#endif
}
And I use it in the following TestMethod:
[TestMethod]
public async Task TestGetUserInfoAsync()
{
//var tokenFile = TestUtils.GetLogFileLocation(nameof(this.TestRedeemTokensAsync2));
var tokenFile = #"D:\Temp\TestLog\TestRedeemTokensAsync2.json";
var accessToken = JsonConvert.DeserializeObject<dynamic>(File.ReadAllText(tokenFile))
.access_token.ToString();
var result = await GoogleService.GetUserInfoAsync(this.systemConfig,
accessToken);
TestUtils.WriteLogFile(result);
Assert.IsNotNull(result);
}
I have other 3 Test Methods and they all runs correctly, and they all have async/await Tasks, writing log files with the filenames taken from the method names.
However, for the above specific method, the parameter callingMethod is an empty string (not null). The result file is a file named .json (it is a valid file in Windows). I am not sure if this is related to Unit Test project only.
Why is it happening? How can I debug in this case?
Additional information: I thought it may be because I added the following method, and the 1st calling mess up:
public static string GetLogFileLocation([System.Runtime.CompilerServices.CallerMemberName] string callingMethod = "")
{
var path = Path.Combine(LogFileLocation, callingMethod + ".json");
return path;
}
However, I tried removing it and use hard string for file path (as you see in my code above) instead, but the problem persist.
EDIT: I added the Stack Trace that may be useful. I notice there is a [External Code] row between the code. I don't know what causes it.
EDIT2: The IL code indeed has problem: the 4 (sorry, 4, not 3) other TestMethods are running fine, and their IL code is correct, like this:
However, the method I am having problem with, I don't see any function call. The only WriteLogFile I can see is this one, but it is a string:
Full IL code here: http://pasted.co/cf4b0ea7

The problem seems to be happening with async and dynamic going together. Remove either one (switch to synchronous or casting into a strongly typed variable) fix the problem:
[TestMethod]
public async Task TestGetUserInfoAsync()
{
//var tokenFile = TestUtils.GetLogFileLocation(nameof(this.TestRedeemTokensAsync2));
var tokenFile = #"D:\Temp\TestLog\TestRedeemTokensAsync2.json";
var accessToken = (JsonConvert.DeserializeObject<dynamic>(File.ReadAllText(tokenFile))
.access_token.ToString()) as string; // Here, cast this to string instead of keeping it as dynamic
var result = await GoogleService.GetUserInfoAsync(this.systemConfig,
accessToken);
TestUtils.WriteLogFile(result);
Assert.IsNotNull(result);
}

Assert.Pass made me out of loop. unable to generate out come

I am automating a Web application using C#, VSTS2010, Nunit and .Resx file as data source.
able to pick the data successfully from Resource file. while generating results, upon using Assert.Pass("") it made me out of loop if passed. No idea how to produce the outcome and where ? Please assist me.
foreach (string from1 in from)
{
foreach (string to1 in To)
{
driver.Navigate().GoToUrl(www.xyz.com);
System.Threading.Thread.Sleep(5000);
driver.FindElement(By.Id("tbFrom")).Clear();
driver.FindElement(By.Id("tbFrom")).SendKeys(from1);
driver.FindElement(By.Id("tbTo")).Clear();
driver.FindElement(By.Id("tbTo")).SendKeys(to1);
if (driver.FindElement(By.Id("trAirLines")).Text.Contains("Air Canada"))
{
Assert.Pass("Air Canada Results are available")
}
else
{
Assert.Fail("AC not found")
}
}
all iteration get completed when I remove Assert statement else it get failed when it become pass

You must not call Assert.Pass(). This will abort the execution of the test immediately. So Assert.Fail() does.
I think the best thing you can get along is:
foreach (string from1 in from)
{
foreach (string to1 in To)
{
driver.Navigate().GoToUrl(www.xyz.com);
System.Threading.Thread.Sleep(5000);
driver.FindElement(By.Id("tbFrom")).Clear();
driver.FindElement(By.Id("tbFrom")).SendKeys(from1);
driver.FindElement(By.Id("tbTo")).Clear();
driver.FindElement(By.Id("tbTo")).SendKeys(to1);
var infoElement = driver.FindElement(By.Id("trAirLines"));
Assert.That(infoElement.Text, Is.StringContaining("Air Canada"));
}
}
The Assert.That ... Is.StringContaining checks whether the text of the referenced element contains "Air Canada". The test will now fail on the first element that does not fulfill this assertion - otherwise pass obviously.

Foreach loop and error that not all code paths return a value: async-await issue?

I am not sure why the "not all code paths return a value" error appears (see commented code below). Is it because the foreach loop is considered as not one path but many, and it's possible that an iteration might not return a value and yet the loop would continue?
[BTW, my goal is to process 100K urls, creating many web requests so they are active simultaneously, and be able to capture the status of each url (alive, moved, timed out) while updating the UI as the set of urls is being processed with real-time information about the number of bad urls found so far and the number or urls processed so far. I've tried Parallel.ForEach approach with synchronous web requests but the UI became unresponsive.]
EDITED to include a test if Rows.Count == 0.
public async Task<UrlInfo> ProcessUrls(DataTable urls)
{
if (urls.Rows.Count == 0)
{
return new UrlInfo();
}
else
{
foreach (DataRow r in urls.Rows)
{
UrlInfo info = new UrlInfo()
{
Url = (string)r["url"],
status = UrlStatusCode.untested,
articleid = (int)r["articleid"]
};
return await Foo(info);
}
}
//return new UrlInfo(); // error unless this line is uncommented
}
public async Task<UrlInfo> Foo(UrlInfo info) {
<snip>
}

The compiler is complaining about the situation where urls.Rows does not contain any elements. In that case, the method will never encounter a return statement.
On a side note, I do not think that method means what you think it means. A method only returns once, so it would only process the first url.

Awesomnium Post Parameters

currently i am working with Awsomnium 1.7 in the C# environment.
I'm just using the Core and trying to define custom post parameters.
Now, i googled a lot and i even posted at the awsomnium forums, but there was no answer.
I understand the concept, but the recent changes just dropped the suggested mechanic and examples.
What i found:
http://support.awesomium.com/kb/general-use/how-do-i-send-form-values-post-data
The problem with this is, that the WebView Class does not contain "OnResourceRequest" Event anymore.
So far, i have implemented the IResourceInterceptor and have the "OnRequest"-Function overwritten
public ResourceResponse OnRequest(ResourceRequest request)
is the signature, but i have no chance to reach in there in order to add request headers.
Anyone here any idea? I tried to look in the documentation, but i didn't find anything on that.....

You need to attach your IResourceInterceptor to WebCore, not WebView. Here's a working example:
Resource interceptor:
public class CustomResourceInterceptor : ResourceInterceptor
{
protected override ResourceResponse OnRequest(ResourceRequest request)
{
request.Method = "POST";
var bytes = "Appending some text to the request";
request.AppendUploadBytes(bytes, (uint) bytes.Length);
request.AppendExtraHeader("custom-header", "this is a custom header");
return null;
}
}
Main application:
public MainWindow()
{
WebCore.Started += WebCoreOnStarted;
InitializeComponent();
}
private void WebCoreOnStarted(object sender, CoreStartEventArgs coreStartEventArgs)
{
var interceptor = new CustomResourceInterceptor();
WebCore.ResourceInterceptor = interceptor;
//webView is a WebControl on my UI, but you should be able to create your own WebView off WebCore
webView.Source = new Uri("http://www.google.com");
}

HotN's answer above is good; in fact, it's what I based my answer on. However, I spent a week searching for this information and putting together something that will work. (The answer above has a couple of issues which, at the very least, make it unworkable with v1.7 of Awesomium.) What I was looking for was something that would work right out of the box.
And here is that solution. It needs improvement, but it suits my needs at the moment. I hope this helps someone else.
// CRI.CustomResourceInterceptor
//
// Author: Garison E Piatt
// Contact: {removed}
// Created: 11/17/14
// Version: 1.0.0
//
// Apparently, when Awesomium was first created, the programmers did not understand that someone would
// eventually want to post data from the application. So they made it incredibly difficult to upload
// POST parameters to the remote web site. We have to jump through hoops to get that done.
//
// This module provides that hoop-jumping in a simple-to-understand fashion. We hope. It overrides
// the current resource interceptor (if any), replacing both the OnRequest and OnFilterNavigation
// methods (we aren't using the latter yet).
//
// It also provides settable parameters. Once this module is attached to the WebCore, it is *always*
// attached; therefore, we can simply change the parameters before posting to the web site.
//
// File uploads are currently unhandled, and, once handled, will probably only upload one file. We
// will deal with that issue later.
//
// To incoroprate this into your application, follow these steps:
// 1. Add this file to your project. You know how to do that.
// 2. Edit your MainWindow.cs file.
// a. At the top, add:
// using CRI;
// b. inside the main class declaration, near the top, add:
// private CustomResourceInterceptor cri;
// c. In the MainWindow method, add:
// WebCore.Started += OnWebCoreOnStarted;
// cri = new CustomResourceInterceptor();
// and (set *before* you set the Source value for the Web Control):
// cri.Enabled = true;
// cri.Parameters = String.Format("login={0}&password={1}", login, pw);
// (Choose your own parameters, but format them like a GET query.)
// d. Add the following method:
// private void OnWebCoreOnStarted(object sender, CoreStartEventArgs coreStartEventArgs) {
// WebCore.ResourceInterceptor = cri;
// }
// 3. Compile your application. It should work.
using System;
using System.Runtime.InteropServices;
using System.Text;
using Awesomium.Core;
using Awesomium.Windows.Controls;
namespace CRI {
//* CustomResourceInterceptor
// This object replaces the standard Resource Interceptor (if any; we still don't know) with something
// that allows posting data to the remote web site. It overrides both the OnRequest and OnFilterNavigation
// methods. Public variables allow for run-time configuration.
public class CustomResourceInterceptor : IResourceInterceptor {
// Since the default interceptor remains overridden for the remainder of the session, we need to disable
// the methods herein unless we are actually using them. Note that both methods are disabled by default.
public bool RequestEnabled = false;
public bool FilterEnabled = false;
// These are the parameters we send to the remote site. They are empty by default; another safeguard
// against sending POST data unnecessarily. Currently, both values allow for only one string. POST
// variables can be combined (by the caller) into one string, but this limits us to only one file
// upload at a time. Someday, we will have to fix that. And make it backward-compatible.
public String Parameters = null;
public String FilePath = null;
/** OnRequest
** This ovverrides the default OnRequest method of the standard resource interceptor. It receives
** the resource request object as a parameter.
**
** It first checks whether or not it is enabled, and returns NULL if not. Next it sees if any
** parameters are defined. If so, it converst them to a byte stream and appends them to the request.
** Currently, files are not handled, but we hope to add that someday.
*/
public ResourceResponse OnRequest(ResourceRequest request) {
// We do nothing at all if we aren't enabled. This is a stopgap that prevents us from sending
// POST data with every request.
if (RequestEnabled == false) return null;
// If the Parameters are defined, convert them to a byte stream and append them to the request.
if (Parameters != null) {
var str = Encoding.Default.GetBytes(Parameters);
var bytes = Encoding.UTF8.GetString(str);
request.AppendUploadBytes(bytes, (uint)bytes.Length);
}
// If either the parameters or file path are defined, this is a POST request. Someday, we'll
// figure out how to get Awesomium to understand Multipart Form data.
if (Parameters != null || FilePath != null) {
request.Method = "POST";
request.AppendExtraHeader("Content-Type", "application/x-www-form-urlencoded"); //"multipart/form-data");
}
// Once the data has been appended to the page request, we need to disable this process. Otherwise,
// it will keep adding the data to every request, including those that come from the web site.
RequestEnabled = false;
Parameters = null;
FilePath = null;
return null;
}
/** OnFilterNavigation
** Not currently used, but needed to keep VisualStudio happy.
*/
public bool OnFilterNavigation(NavigationRequest request) {
return false;
}
}
}