In-memory CSharpCompilation cannot resolve attributes - c#

I am trying to run C# source generators in-memory using the following code snippet:
var syntaxTree = await SyntaxTreeFromRelativeFile("testdata/IMyInterface.cs");
var compilation = CSharpCompilation.Create("compilation", ImmutableArray.Create(syntaxTree), References);
var generator = new ProxyGenerator();
GeneratorDriver driver = CSharpGeneratorDriver.Create(generator);
driver = driver.RunGenerators(compilation);
References is set to the necessary sources to compile the code:
public static readonly ImmutableArray<MetadataReference> References = ImmutableArray.Create<MetadataReference>(
// System
MetadataReference.CreateFromFile(typeof(object).Assembly.Location),
MetadataReference.CreateFromFile(typeof(GCSettings).Assembly.Location),
MetadataReference.CreateFromFile(typeof(Attribute).Assembly.Location),
MetadataReference.CreateFromFile(typeof(Enumerable).Assembly.Location),
// JetBrains
MetadataReference.CreateFromFile(typeof(UsedImplicitlyAttribute).Assembly.Location),
// some custom ones
);
While the generator runs just fine this way, it sadly doesn't have the desired affect, because the source generator relies on an attribute to know whether to generate source for the specified type. The exact source is something along the following:
[MyAttribute]
public interface IMyInterface { /* ... */ }
The source generator picks up the attribute correctly, but it gets resolved to an ExtendedErrorTypeSymbol with the result kind being NotAnAttributeType. However, the extended error type symbol also has a candidate symbol, which is the exact symbol I expect it to match.
This is surprising to me, because clearly the type is an attribute, and running the source generator as part of the normal compilation actually does generate all the right types. This seems to imply that there is something strange going on because of the in-memory nature of this run specifically.
As far as I can tell, my list of References covers everything that is needed to correctly realise that something is an attribute (mscorlib, System.Runtime, netstandard, and System.Core), though perhaps there is another MetadataReference missing?
I did find this GitHub issue which seems to describe a very similar, if not the same problem.
I'd love to know if I did something wrong here, if there are other references I am missing, or whether I am missing something completely else altogether.

By decompiling the source and stepping through how the attribute become an ExtendedErrorTypeSymbol, I found out that alongside the resultKind and candidate type, you could also find a DiagnosticBag. In this bag, the actual problem was shown:
error CS0012: The type 'Attribute' is defined in an assembly that is not referenced. You must add a reference to assembly 'System.Runtime, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
I was under the impression that my code added that correctly, but using this error I was luckily able to further my search, and ran into this Stackoverflow question. This seems to imply that for some reason (I am not 100% sure why), the following code will not actually add the right reference to System.Runtime:
MetadataReference.CreateFromFile(typeof(GCSettings).Assembly.Location)
Instead, I followed the example of the answer linked above, and changed the code to:
private static readonly string dotNetAssemblyPath = Path.GetDirectoryName(typeof(object).Assembly.Location);
public static readonly ImmutableArray<MetadataReference> References = ImmutableArray.Create<MetadataReference>(
// .NET assemblies are finicky and need to be loaded in a special way.
MetadataReference.CreateFromFile(Path.Combine(dotNetAssemblyPath, "mscorlib.dll")),
MetadataReference.CreateFromFile(Path.Combine(dotNetAssemblyPath, "System.dll")),
MetadataReference.CreateFromFile(Path.Combine(dotNetAssemblyPath, "System.Core.dll")),
MetadataReference.CreateFromFile(Path.Combine(dotNetAssemblyPath, "System.Private.CoreLib.dll")),
MetadataReference.CreateFromFile(Path.Combine(dotNetAssemblyPath, "System.Runtime.dll")),
// more references, loaded as before
);
For some reason, this made all the difference. I had to also add the reference to System.Private.CoreLib.dll, but now that I knew where to find the diagnostics bag with additional information about what is actually wrong, this was an easy fix.

Related

How can I fake Assembly.LoadFile and Assembly.GetTypes?

I'm struggling to fake these two lines of code:
Assembly asm = Assembly.LoadFile(fileName);
Type[] asmTypes = loggerAssembly.GetTypes();
When I type System.Reflection.ShimAssembly there is no such type as ShimAssembly like for example in the case of System.IO.ShimFile but only a StubAssembly.
I could refactor the code and do it with a static helper method like LoadAssemblyAndGetTypes but it seems to be an unnecessary workaround. I'd prefer the official solution.
If it only worked like this:
var shimAsm = System.Reflection.Fakes.ShimAssembly.LoadFile = (fileName) =>
{
return ???
};
var types = shimAsm.GetTypes = () =>
{
return new Type[] { new object() };
};
Why does it work for System.IO.File and not for System.Reflection.Assembly. Is this because Assembly is an abstract class?
In my unit test I want to check if my assembly loader correctly checks if the loaded assembly contains types that implement some interfaces so that I can instantiate them later later.
I think I found the answer and it doesn't look good:
#Patrick Tseng - Visual Studio team - writes at Shim mscorlib and system limitations:
...we DO have a list of type we purposely not allowing them to be
shimmed. Reason being that it could potentially cause recurisvely
calling into your detour delegate from CLR runtime itself. E.g If CLR
runtime uses a type in System.Reflection during runtime and you happen
to detour functions in this type. You might end up causing expected
behavior since runtime behavior will totally be changed.
In short, we don't shim value type, System.Reflection.,
System.Runtime., XamlGeneratedNamespace, and a few other types which
we deem is important and will not shim them.
so after all it seems I'll need to keep the static helper method.

Get Symbol for ReferenceLocation

I'm using the SymbolFinder to find all references to a certain type in my solution like this:
ISymbol typeOfInterest = compilation.GetTypeByMetadataName(
"System.Reflection.PropertyInfo");
var references = SymbolFinder.FindReferencesAsync(typeOfInterest, solution).Result;
foreach (var reference in references)
{
// reference.Locations => symbol?
}
This part is working fine, the SymbolFinder returns correct ReferenceLocations (upon manual inspection). I'm actually interested in the symbols at these locations to get more (semantic) information about the references, so I can filter upon / work with it (e.g. only work on properties).
There seems to be very little public information on Roslyn yet and I couldn't find anything working with the results of SymbolFinder in the samples of the SDK Preview. So here is my question: Is it possible to get the symbol corresponding to a ReferenceLocation? How?
So, there isn't strictly a "symbol" at any of these locations, at least no innate concept of that. What you can do is take that Location, and find the enclosing symbol. You can take the location's SyntaxTree and get a Document. From there, call GetSemanticModelAsync, and then call ISemanticModel.GetEnclosingSymbol.
As an example, here's some (internal) code that does this for FAR itself: https://github.com/dotnet/roslyn/blob/748d6ab1b504ceee0c29f132fdcbe2a777aa88ea/src/Workspaces/Core/Portable/FindSymbols/ReferenceLocationExtensions.cs#L67-L101

Adding a lambda expression has resulted in a strange error when attempting to compile

So currently there is a piece of code which looks like this...
string name = GetValues(sequenceOfCodes, 0, IDtoMatch, 1)[0];
I just updated the following line to be
string name = sequenceOfCodes
.Select(x => x[0])
.Where(x => x == IDtoMatch)
.FirstOrDefault();
Which should hopefully return the same thing.
sequenceOfCodes is a List<List<String>> and the IDtoMatch is also a string.
So hopefully this all seems fine.
However when I go to compile I get an odd error
The type 'System.Windows.Forms.ComboBox' is defined in an assembly
that is not referenced.
You must add a reference to assembly 'System.Windows.Forms, Version=4.0.0.0,
Culture=neutral, PublicKeyToken=b77a5c561934e089'
And when I take my newly added code away it compiles fine and runs... So why is it just because I have added a lambda expression does it think that I it needs a reference to System.Windows.Forms.ComboBox?
Just to state that this is a console application. Not a winforms application.
-----------UPDATE----------
Ok, So I have found that a one of the references does reference the System.Windows.Forms, which I is really disappointing as this is core code and should not have dependencies like this :(
However I still wonder why the error did not appear before until after I added my line of code.
To confirm, If I remove my code I can close VS down and restart and rebuild and all is fine.
If I add my line of code and close down and restart, etc. The error will reappear on rebuild.
Very strange error to me.
Thanks guys for all your help
You mention that one of the other projects does reference windows forms. My guess is that this project also declares some extension methods that are in scope (given your using directives), and which the compiler needs to explore for overload resolution - presumably of the Where, Select or FirstOrDefault methods; meaning: it can't decide that the best overload of these is the System.Linq.Enumerable one until it has compared it to the other candidates, and it can't do that without being able to understand the types used in the competing method signatures.
Or in other words: is there a Select, Where or FirstOrDefault custom extension method that mentions ComboBox ?

Adding a TypeDefinition from another Assembly

This one is driving me crazy.
AssemblyDefinition asm1 = AssemblyDefinition.ReadAssembly(example);
AssemblyDefinition asm2 = AssemblyDefinition.ReadAssembly(example2);
asm2.MainModule.Types.Add(asm1.MainModule.Types[0]);
Whenever I try to execute the above code I get this error 'Type already attached'
I decided to look this error at MonoCecil source and I found it throws this error because the Type's MainMoudle isn't asm2 MainModules. So I decided to Copy that Type to a new one.
TypeDefinition type2 = new TypeDefinition("", "type2", Mono.Cecil.TypeAttributes.Class);
foreach (MethodDefinition md in asm2.Methods )
{
type2.Methods.Add(md);
}
And then add this type to my assembly normally but this throws another error, 'Specified method is not supported.'.
Any thoughts why I am getting this error?
Edit: Just to add, the type I'm trying to add contains some methods which uses pointers. Might this be the problem? As far as I know mono supports that but not mixed mode.
I'm afraid there's no built in, easy way to do this.
When you read an assembly with Cecil, every piece of metadata is glued together by the Module the metadata is defined in. You can't simply take a method from a module, and add it into another one.
To achieve this, you need to clone the MethodDefinition into a MethodDefinition tied to the other module. Again, there's nothing built-in yet for this.
I suggest you have a look at IL-Repack, which is an open-source ILMerge clone. It does exactly that, it takes types from different modules, and clone them into another one.

The as operator rejects the object even though the object appears to be of the correct type in the debugger

The following code throws an exception. If there is no easy answer or stuff to check, I'll try to produce something that reproduces the error (though I don't know where to upload it).
public static XMLobj Load(string FileName)
{
if (File.Exists(FileName) == false)
{
return null;
}
IRDnet.XMLobj def;
XmlSerializer xmlser = new XmlSerializer(typeof(IRDnet.XMLobj));
System.IO.Stream stream = File.OpenRead(FileName);
object o = xmlser.Deserialize(stream);
// o appears to be of correct type in the quick watch.
IRDnet.XMLobj def2 = o as IRDnet.XMLobj;
// def2 is "undefined" (as operator rejected o)
def = (IRDnet.XMLobj)o;
// Throws InvalidCastException with no InnerException.
stream.Close();
return def;
}
The strange thing is that "o" appears to be of correct type if I break just before the exception is thrown:
o {IRDnet.XMLobj} System.Object
And the object casts just fine in the quickwatch window. Values are easily inspected.
It is executed from the same project that it is part of. So, no loading contexts.
FYI: the static method is part of the XMLobj class if that's relevant.
Is there some other criteria for a successful cast that I'm not aware of? Any code that gets implicitly executed?
I've checked that reflector produces the equivalent code to make sure that nothing was lost in compiler optimization.
Any clues, people? I'm stumped. I even hoped that just writing this question would make me think twice on something completely obvious.
Chances are this is a versioning issue. That is, the deserialized XMLobj is a different version to the one you've compiled with. Check the fully qualified name of each type.
The .NET Serializer produces {Assembly}.Serializer.dll assemblies to speed up XML serialization/deserialization. Try to delete every Assembly and compile from scratch.
If the assemblies does not match exactly an InvalidCast Exception is throw.
EDIT: Look in your debugger output to see what assemblies have been loaded.
UPDATE:
string tmp = o.GetType().AssemblyQualifiedName;
string tmp2 = typeof(XMLobj).AssemblyQualifiedName;
produces:
tmp "IRDnet.XMLobj, IRDnet, Version=1.0.3600.18887, Culture=neutral, PublicKeyToken=null" string
tmp2 "IRDnet.XMLobj, IRDnet, Version=1.0.3601.27699, Culture=neutral, PublicKeyToken=null" string
So there is definitely a legitimate type mismatch. I highly appreciate this help. I was completely stuck. Now I have a lead. Now to find how the old type survived the rebuilding.
Maybe the XML file has something to say about it....
Or maybe it is the secretive Serialize-assembly somewhere that was mentioned....
I'm not stuck. I'm still digging, but I thought I'd inform of this new development. If anybody has more tips about interrogating a type about its declaration, then please chip in. Otherwise, thank you all!
ANSWER
The first thing to do if you ever get into this exception for seeming identical types, is to check that the two types are actually the same.
System.Type t1 = typeof(XMLobj);
System.Type t2 = o.GetType();
And then check them in the debugger. Take especially note of the "AssemblyQualifiedName". My two types turned out to be different versions of the same class.
Then do pay careful attention to the output window in Visual Studio as you are single stepping through. It will tell you when which assemblies are loaded. In my example, the culprit came in clear text in the debug output just as I stepped over the line
XmlSerializer xmlser = new XmlSerializer(typeof(IRDnet.XMLobj));
There, the debugger revealed that another, older, version of my assembly was loaded from a location that I wasn't aware of.
I neglected to tell the guys here that I debug using an external exe that loads my assembly. It turns out that the CLR, I assume on behalf of the XmlSerializer class constructor to resolve the "typeof", looks in the exe folder and its subfolders (which is what CLR does). And, lo and behold, deep down there there was a bastard of the original version of the IRDnet.dll file that I was fixing.
I still do not see how the two "typeof" statements produce different results: One results in the object type currently within context, whereas the other causes the CLR do go out looking for it in the execution folder. One possible theory is that the typeof statement is somehow lazily evalutated. I wouldn't know.

Categories