We use protocol buffers for communication between native C++ apps, but also between native C++ app and .NET application (all is VS2012) via protobuf-net r666.
We rely in C++ heavily on the has_ functions that are available for an optional element.
E.g. if we have a message with a field optional bool, it can be that it is not set, it is set to true, or it is set to false.
In C++ this can be checked with the function has_field and if set then the content can be fetched with get_field function. If not set, and get_field is called, then the get returns the default, which if not explicitly set is false (for a boolean).
This works perfectly in C++, but, in protobuf-net however, we cannot seem to find the equivalent of the has_ function, and, when the message is received, the field is added to the message and it's content is set to the default, being false. It's not a disaster that the field is there with the default, but the problem is that there is no has_ function to check whether it was set in the message.
Please advise whether this is a bug or whether we missed something in protobuf-net and that this actually is possible
Thx in advance.
Wim
(I know we already covered this in the issue tracker - this is purely for visibility etc)
This relates to generating classes from .proto files, which in the case of protobuf-net is via the protogen tool. By default, it does not create the equivalent of has_* methods, but this can be enabled with the -p:detectMissing switch - which causes it to create *Specified accessors. The naming here is a .NET idiom, with *Specified recognised by some other .NET serializers and internal code. It also generates a private ShouldSerialize* method, which again helps some internal .NET code.
In this specific case, there was a secondary issue with a member called value causing confusion; the csharp.xslt file has now been updated to account for this.
Update: in the fully managed rewrite, the ShouldSerialize*() method is generated by default when using proto2 syntax (the default). No additional parameter is required. The *Specified member is not added (it serves no additional purpose over ShouldSerialize*().
Note that when using proto3, the changes to serialization rules mean that this concept no longer has meaning. A value is serialized if and only if it is not the default, which is always null/false/zero/empty. There is no concept of "the default value but specified". Because of this, the ShouldSerialize*() methods are generally no longer useful, and are not generated. I'm open to making them optionally generated for proto3, with them basically meaning "non-default", if that helps some genuine coding scenario.
Related
I am writing a library in F#, with some interfaces and base classes that are publicly visible. Generally, I avoid specifying [<AllowNullLiteral>] on my custom types as this complicates the validation logic in my F# code (see this nice post for goods and bads of null handing in F# to get a picture ), and also, F# does not initially allow null for F# types. So, I validate for nulls only for types that accept the null value as valid.
However, an issues arises when my library is used from another .NET language, such as C#. More particularly, I worry how should I implement methods that accept F#-declared interfaces, when called by C# code. The interface types are nullable in C#, and I suspect that there will not be an issue for the C# code to pass a null to my F# method.
I fear that the caller would crash and burn with a NPE, and the problem is that I am not even allowed to properly handle that in the F# code -- say throw an ArgumentNullException -- because the respective interface lack the AllowNullLiteral attribute. I fear that I would have to use the attribute and add the related null-checking logic in my F# code to eventually prevent such a disaster.
Are my fears reasonable? I am a little confused as I initially attempt to stick to the good F# practices and avoid null as much as possible. How does this change if among my goals is to allow C# code to subclass and implement the interfaces I created in F#? Do I have to allow nulls for all non-value types coming from my F# code if they are public and can be accessed from any CLR language? Is there a best practice or a good advice to follow?
There are two basic approaches you can take:
Document in your API design that passing null to your library is not allowed, and that the calling code is responsible for ensuring that your library never receives a null. Then ignore the problem, and when your code throws NullReferenceExceptions and users complain about it, point them to the documentation.
Assume that the input your library receives from "outside" cannot be trusted, and put a validation layer around the "outside-facing" edge of your library. That validation layer would be responsible for checking for null and throwing ArgumentNullExceptions. (And pointing to the documentation that says "No nulls allowed" in the exception message).
As you can probably guess, I favor approach #2, even though it takes more time. But you can usually make a single function, used everywhere, to do that for you:
let nullArg name message =
raise new System.ArgumentNullException(name, message)
let guardAgainstNull value name =
if isNull value then nullArg name "Nulls not allowed in Foo library functions"
let libraryFunc a b c =
guardAgainstNull a nameof(a)
guardAgainstNull b nameof(b)
guardAgainstNull c nameof(c)
// Do your function's work here
Or, if you have a more complicated data structure that you have to inspect for internal nulls, then treat it like a validation problem in HTML forms. Your validation functions will either throw an exception, or else they will return valid data structures. So the rest of your library can ignore nulls completely, and be written in a nice, simple, idiomatic-F# way. And your validation functions can handle the interface between your domain functions and the untrusted "outside world", just as you would with user input in an HTML form.
Update: See also the advice given near the bottom of https://fsharpforfunandprofit.com/posts/the-option-type/ (in the "F# and null" section), where Scott Wlaschin writes, "As a general rule, nulls are never created in "pure" F#, but only by interacting with the .NET libraries or other external systems. [...] In these cases, it is good practice to immediately check for nulls and convert them into an option type!" Your library code, which expects to get data from other .NET libraries, would be in a similar situation. If you want to allow nulls, you'd convert them to the None value of an Option type. If you want to disallow them and throw ArgumentNullExceptions when you get passed a null, you'd also do that at the boundaries of your library.
Based on #rmunn's advice I ended up creating a simple null2option function:
let null2option arg = if obj.ReferenceEquals(arg, null) then None else Some arg
It solved most of my cases alone. If I expect a null argument to be coming for the calling code I would simply use this idioms:
match null2option arg with | None -> nullArg "arg" "Message" | _ -> ()
I'd like to use C#'s reflection and custom attributes to simplify registering a series of types with a central management class (i.e. it provides static methods taking a string key and invoking/retrieving the proper method/parameter for the associated type). Looking at other questions here and a couple places elsewhere, it seems like the best way of doing so is to simply iterate through all public types of the assembly -- since it's intended to be a library -- and check if each type has the proper attribute before adding the relevant values to the underlying Dictionaries. The reflection and iteration will definitely be slow, but I can live with it since it should only occur once.
Unfortunately, I can't figure out how to get an attribute from a type. For methods and assemblies, I can use CustomAttributeExtensions.GetCustomAttribute<MyAttribute>(base) from System.Reflection.Extensions, but that doesn't provide an overload for Type; the same for Assembly.GetCustomAttribute(Assembly, Type) and the .IsDefined(...) methods used in this question. Other suggestions use methods on the Type itself that, from the documentation, seem to be loaded from mscorelib.dll, but it didn't seem to be showing up in Intellisense even after adding the reference and I'm not sure how that .dll interacts with .NET Standard, anyway (as in, does it reduce the ability to run on arbitrary platforms at all?)
Am I missing something obvious, or is it really this hard to get an Attribute back off of a Type?
Try typeof(YourType).GetTypeInfo().GetCustomAttributes();
So today I was browsing around ILSpy to get a better understanding of how .NET performs DllImports on external methods, when I came across something odd:
When searching for references to the enum value PInvokeImpl, which is defined in the System.Reflection.MethodAttributes enumeration, I noticed a matching definition in System.Reflection.FieldAttributes.
Well sure enough, this appears to be more than just a behind-the-scenes, re-use of enumeration values: System.Reflection.FieldInfo has an publicly defined property called IsPinvokeImpl, which specifically checks if this implementation flag is set.
Interestingly enough, the MethodInfo class doesn't even have this property - it must be determined from the MethodImplementationFlags property instead.
Question:
Is it actually possible for a field to be PInvoke implemented, or is this just a stub implementation in the .NET framework for balance between field decorations and method decorations?
If it is possible, can it be done in C#, or is this a feature that requires C++/CLI?
When you look at the MSDN description for FieldAttributes then you'll see it documented as "Reserved for future use". That future has not arrived yet so its intention is not nailed down.
Types like FieldAttributes are not arbitrary, they follow the CLI specification. Ecma-335 nails down what the metadata in a .NET assembly needs to look like and how it should be interpreted. This document does reveal an interesting quirk.
Chapter II.16.1 describes the field attributes, you'll see a close match between the metadata tokens and the FieldAttributes enum. Note however that pinvokeimpl is missing in that chapter.
Chapter II.23.1.5 gives the specific values of the attributes, it has PInvokeImpl with the value 0x2000. Description is "Implementation is forwarded through PInvoke". Compare to II.23.1.10, describes method attributes. It has many values in common with field attributes.
This looks a lot like a copy/paste bug :)
Digging a bit deeper through the .NET Framework source code, the CLR and jitter only ever considers pinvokeimpl on methods. The C# compiler however appears to be based on the CLI spec and actually sets the attribute. Appears in emit.cpp, RegMeta::_DefinePinvokeMap() function, it will set the attribute if this function is called for a field instead of a method. That never actually happens.
From FieldAttributes:
PinvokeImpl Reserved for future use.
So I would say:
Q: Is it actually possible for a field to be PInvoke implemented
A: No
I find this question in Stack Overflow when googleing, but it has been deleted. So I list this question again.
As I can't find the LcidAttribute or RetvalAttribute in BCL, I guess C# hasn't provided the support for locale identifier parameter and return value parameter.
Is that it?
Thanks all.
They are associated with the ParameterAttributes enumeration. Which is used in metadata for the parameter of a method, only a compiler can emit the [modopt].
I do not know of a compiler that actually does this. I have a decent guess at the background though, these attributes are also used in IDL. Which is an interface description language that is used in COM and RPC. Having this option ensures that .NET metadata can also describe the kind of declarations that are written in IDL and can appear in type libraries.
The [lcid] attribute is described here. It doesn't actually describe usage and I've never used it myself. No real idea why you'd use it.
The [retval] attribute is described here. Very important in COM automation method declarations, it marks the parameter that returns the method value. And used by a tool like Tlbimp.exe, it rewrites the method to make that parameter the return value type.
I ran into, what I thought was a bug is actually, a feature detailed in This post. Can anyone explain to me why this is allowed? It seems like a legacy quirk/bug that became useful.
I'm not sure which part of that you think is a bug, but it has always been possible to access the internals of a class via reflection when you cannot do so at compile time. This is by design. Numerous aspects of the CLR rely on reflection to access fields, such as serialization. The compiled IL needs to be able to access all fields of all objects, or else you couldn't set private fields from within your class.
The access modifiers in C# are not a security mechanism. If you are relying on a field being private to prevent anyone from setting it from the outside, you're doing something wrong. They exist to clearly delineate which parts of your class are it's public contract (and thus, in theory, stable) from those parts that are implementation details (and thus can change without notice.)
If you choose to use reflection to change the internal state of an object, despite all indications that you should leave it alone, you are taking the stability of your application into your own hands, and you get what deserve.
Reflection is allowed only for Full Trust code, so the code already able to do anything (including directly poking in memory of the process). So having supported way of changing values even for private properties does not make code any less secure. It makes reflection API consistent and allows useful scenarios especially for testing.