We Keep Coding

.NET Core 3.1 v .NET 6.0

I got all excited on the release of Visual Studio 2022, C# 10 and .NET 6.0 and downloaded and installed the community edition and tested a project I am working on. I changed the target framework to 6.0 and performed a clean build. Great, everything built as expected.
So, onwards and upwards and ran the project. The very first test failed. I must say I was surprised. I started digging around and was really surprised to find a difference between .NET Core 3.1 and .NET 6.0.
Here is a sample program:
public class Program
{
public static readonly string CTCPDelimiterString = "\x0001";
public static readonly char CTCPDelimiterChar = '\x0001';
public static void Main(string[] args)
{
string text = "!sampletext";
Console.Write(" Using a char: ");
if (text.StartsWith(CTCPDelimiterChar) && text.EndsWith(CTCPDelimiterChar))
{
Console.WriteLine("got CTCP delimiters");
}
else
{
Console.WriteLine("did not get CTCP delimiters");
}
Console.Write("Using a string: ");
if (text.StartsWith(CTCPDelimiterString) && text.EndsWith(CTCPDelimiterString))
{
Console.WriteLine("got CTCP delimiters");
}
else
{
Console.WriteLine("did not get CTCP delimiters");
}
}
}
Using a target framework of 'netcoreapp3.1' I got the following output:
Using a char: did not get CTCP delimiters
Using a string: did not get CTCP delimiters
Using a target framework of 'net6.0' I got the following output:
Using a char: did not get CTCP delimiters
Using a string: got CTCP delimiters
So, I can only assume that it is a unicode setting but I cannot find it anywhere (if there is one). My understanding is that all strings are UTF16 but why the difference between frameworks.
And yes, I can see the bug in my code, it should be a char anyway but it was working fine using 'netcoreapp3.1'.
Can anyone shed some light on this please.

After .Net Core 3, you must highlight your comparison mode by StringComparison code.
change
if (text.StartsWith(CTCPDelimiterString) && text.EndsWith(CTCPDelimiterString))
with
if (text.StartsWith(CTCPDelimiterString, StringComparison.Ordinal) && text.EndsWith(CTCPDelimiterString, StringComparison.Ordinal))

ML.NET in Unity

I cannot figure out how to use ML.NET in Unity.
What I did:
Converted my project to be compatible with framework 4.x.
Converted api compatibility level to framework 4.x.
Made assets/plugins/ml folder and droped in Microsoft.ML apis with corresponding xmls.
Marked all ml.dlls platform settings to be only 86_64 compatible (this was redundant).
I can now:
Call ML apis and create MlContext, TextLoader, and do the training of a model. When a model is trained I can also evaluate the trained model, but...
I cannot:
When trying to get a prediction out of the model I get an error:
github comment on issue from 28.12.18 (there is also a whole project attached there, you can see the code there)
The same code works in visual studio solution.
public float TestSinglePrediction(List<double> signal, MLContext mlContext, string modelPath)
{
ITransformer loadedModel;
using (var stream = new FileStream(modelPath, FileMode.Open, FileAccess.Read, FileShare.Read))
{
loadedModel = mlContext.Model.Load(stream);
}
var predictionFunction = loadedModel.MakePredictionFunction<AbstractSignal, PredictedRfd>(mlContext);
var abstractSignal = new AbstractSignal()
{
Sig1 = (float)signal[0],
Sig2 = (float)signal[1],
Sig3 = (float)signal[2],
Sig4 = (float)signal[3],
Sig5 = (float)signal[4],
Sig6 = (float)signal[5],
Sig7 = (float)signal[6],
Sig8 = (float)signal[7],
Sig9 = (float)signal[8],
Sig10 = (float)signal[9],
Sig11 = (float)signal[10],
Sig12 = (float)signal[11],
Sig13 = (float)signal[12],
Sig14 = (float)signal[13],
Sig15 = (float)signal[14],
Sig16 = (float)signal[15],
Sig17 = (float)signal[16],
Sig18 = (float)signal[17],
Sig19 = (float)signal[18],
Sig20 = (float)signal[19],
RfdX = 0
};
var prediction = predictionFunction.Predict(abstractSignal);
return prediction.RfdX;
}
This is the method that returns an error line:
var predictionFunction = loadedModel.MakePredictionFunction<AbstractSignal, PredictedRfd>(mlContext);

Starting with Unity 2018.1, unity can target .net 4.x. So you would need to set the .net version to .NET 4.x Equivalent, or .net standard 2.0 (https://blogs.unity3d.com/2018/03/28/updated-scripting-runtime-in-unity-2018-1-what-does-the-future-hold/) and make sure you add your dll to the project as a reference in visual studio. If you don't add it as a reference, then visual sudio doesn't know about it.

As Nick said in his post**, it should work with Unity if you follow those steps.
However, at the time I am writing this post, the ML.NET team has not yet done comprehensive testing with Unity, so it's not completely surprising that it's not working out of the box. This issue has been opened on the ML.NET Github repository. I suggest keeping an eye on that issue for the status of Unity support.
** Nick:
Starting with Unity 2018.1, unity can target .net 4.x. So you would need to set the .net version to .NET 4.x Equivalent, or .net standard 2.0 (https://blogs.unity3d.com/2018/03/28/updated-scripting-runtime-in-unity-2018-1-what-does-the-future-hold/) and make sure you add your dll to the project as a reference in visual studio. If you don't add it as a reference, then visual sudio doesn't know about it.

As follows is a bit modyfied Iris Example from https://learn.microsoft.com/en-us/dotnet/machine-learning/tutorials/iris-clustering (that one does not work anymore due to some ML API changes)
First make sure that you have the latest .net version installed and that your Unity version is at least 2019.2.0f1 (this was a preview version) or higher.
Creste a new unity project. Create a Plugins folder inside your Assets folder. Import all ML .Net APIs into that folder (Might be a foolish thing to do, but I have forehand created a visual studio soution and added all those APIs to that solution via nuget, and than just copied those dll files to Assets/Plugins folder in my unity project).
In Assets folder create an Data folder and paste iris.data file from https://github.com/dotnet/machinelearning/blob/master/test/data/iris.data into it.
Create a script named MLuTest and paste into it the following code:
public class MLuTest : MonoBehaviour{
static readonly string _dataPath = Path.Combine(Environment.CurrentDirectory, "Assets", "Data", "iris.data");
static readonly string _modelPath = Path.Combine(Environment.CurrentDirectory, "Assets", "Data", "IrisClusteringModel.zip");
MLContext mlContext;
void Start()
{
Debug.Log("starting...");
mlContext = new MLContext(seed: 0);
IDataView dataView = mlContext.Data.ReadFromTextFile<IrisData>(_dataPath, hasHeader: false, separatorChar: ',');
string featuresColumnName = "Features";
var pipeline = mlContext.Transforms
.Concatenate(featuresColumnName, "SepalLength", "SepalWidth", "PetalLength", "PetalWidth")
.Append(mlContext.Clustering.Trainers.KMeans(featuresColumnName, clustersCount: 3));//read and format flowery data
var model = pipeline.Fit(dataView);//train
using (var fileStream = new FileStream(_modelPath, FileMode.Create, FileAccess.Write, FileShare.Write))//save trained model
{
mlContext.Model.Save(model, fileStream);
}
var predictor = mlContext.Model.CreatePredictionEngine<IrisData, ClusterPrediction>(model);//predict
IrisData Setosa = new IrisData
{
SepalLength = 5.1f,
SepalWidth = 3.5f,
PetalLength = 1.4f,
PetalWidth = 0.2f
};
Debug.Log(predictor.Predict(Setosa).PredictedClusterId);
Debug.Log("...done predicting, now do what u like with it");
}
}
public class IrisData
{
[LoadColumn(0)]
public float SepalLength;
[LoadColumn(1)]
public float SepalWidth;
[LoadColumn(2)]
public float PetalLength;
[LoadColumn(3)]
public float PetalWidth;
}
public class ClusterPrediction
{
[ColumnName("PredictedLabel")]
public uint PredictedClusterId;
[ColumnName("Score")]
public float[] Distances;
}
This should work right out of the box ... well it did for me. Where you could mess up is when getting api files, they could be different version from mine or just some .net framework compatible. So get the content of my Plugins folder (mind you all those apis may not be necesery, do the cherrypicking yourselve):https://github.com/dotnet/machinelearning/issues/1886
It used to be (in previous unity versions) that some player settings had to be changed but i did not have to do it. But anhoo here are mine:
I hope this helps, since Unity update 19.2 I have not had any problems mentioned in previous posts in this thread.

.Net Standard 4.7.1 Could not load System.Private.CoreLib during serialization

I'm working on migrating most of my project to .Net Standard 2.0.
Last part of the project in .net standard 4.7.1 is a WCF front end.
This executable communicate with a .net core application throught a library Akka.net by ClientServiceReceptionist.
Network communication between application by akka.net work.
Except when an it try to serialize a ReadOnlyCollection.
In this case the system try to load a "System.Private.CoreLib.dll" that is not available.
I readed many issues with .net 4.6 using .net standard 2.0 libraries that must have been corrected in 4.7.1. I pass from package.config to PackageReference.
I tryed to used NewtonSoft as a serializer in place of Hyperion without progress.
Does anyone have an idea, a solution ?
Edit : 07-05-2018
The issue is throw in the WCF Entry Points when i sent a ClusterClient.Send object throught the ClusterClientReceptionist.
The object sent contains only boolean, string, Guid and array of string properties.
Edit 08-05-2018
The object sent look like this :
{
(string) "Path": "/user/ProcessOrderWorkflow",
"Message": {
"Order": {
"Data": {
(string)"Sentence": "i love my name",
"Options": {
(boolean)"Simplify" : true,
(IReadOnlyCollection<string>) LanguageAffinity : [ "FR", "EN" ]
}
},
"OrderQuery": {
"Verb": {
(string)"Verb": "Extract"
},
"Arguments": [
{
(string)"Argument": "Sentence"
},
{
(string)"Argument": "Meaning"
}
]
},
(Guid)"ProcessId": "0bfef4a5-c8a4-4816-81d1-6f7bf1477f65"
},
(string)"ReturnTypeFullName": "Viki.Api.Server.Data.SentenceMeaningMsg,, Viki.Api.Server.Data, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
},
(boolean)"LocalAffinity": false
}
Each of the object used in the hierachi is builded using the constructor.
All the properties are in readonly.
I tryed to serialize and deserialize the result on the WCF part before it's sent and it works.
var serializer = this._system.Serialization.FindSerializerFor(result);
var bytes = serializer.ToBinary(result);
var data = serializer.FromBinary(bytes, result.GetType());
The strange think is that it try to deserialize the object in the WCF part where it is sent and not in the LightHouse where the element should be received and transfert to a agent for processing.

This is a top result for request "Could not load System.Private.CoreLib" so I post the workaround for ASP.NET Core APIs and .NET clients.
If json serializer type handling set to auto
settings.TypeNameHandling = TypeNameHandling.Auto;
serializer will include type information for polymorphic types. After migration to .NET Core some clients reported exception and the response body contained following type descriptor:
"$type":"System.String[], System.Private.CoreLib"
In the API model property type was defined as IEnumerable<string> which forced serializer to include actual type for an Array. The solution was to replace IEnumerable<string> with string[] or any other concrete type which allowed serializer to omit the type descriptor.
If the above does not work, for instance when you use Dictionary<string, object> you can implement custom SerializationBinder:
public class NetCoreSerializationBinder : DefaultSerializationBinder
{
private static readonly Regex regex = new Regex(
#"System\.Private\.CoreLib(, Version=[\d\.]+)?(, Culture=[\w-]+)(, PublicKeyToken=[\w\d]+)?");
private static readonly ConcurrentDictionary<Type, (string assembly, string type)> cache =
new ConcurrentDictionary<Type, (string, string)>();
public override void BindToName(Type serializedType, out string assemblyName, out string typeName)
{
base.BindToName(serializedType, out assemblyName, out typeName);
if (cache.TryGetValue(serializedType, out var name))
{
assemblyName = name.assembly;
typeName = name.type;
}
else
{
if (assemblyName.AsSpan().Contains("System.Private.CoreLib".AsSpan(), StringComparison.OrdinalIgnoreCase))
assemblyName = regex.Replace(assemblyName, "mscorlib");
if (typeName.AsSpan().Contains("System.Private.CoreLib".AsSpan(), StringComparison.OrdinalIgnoreCase))
typeName = regex.Replace(typeName, "mscorlib");
cache.TryAdd(serializedType, (assemblyName, typeName));
}
}
}
And register it in JsonSerializerSettings:
settings.SerializationBinder = new NetCoreSerializationBinder();
Note: .AsSpan() for string comparison was added for backwards compatibility with .NET Framework. It doesn't harm, but not required for .NET Core 3.1+, feel free to drop it.

So this is what I thought the issue might be... The problem is that transmitting serialized content between a .NET Core application and a .NET Framework application using a polymorphic serializer like JSON.NET or Hyperion is an unsupported operation in Akka.NET at least, but also in any other runtime where the users uses CLR types as the wire format of the messages.
A brief explanation as to why we don't support this in Akka.NET, from the v1.3.0 release notes:
As a side note: Akka.NET on .NET 4.5 is not wire compatible with Akka.NET on .NET Core; this is due to fundamental changes made to the base types in the CLR on .NET Core. It's a common problem facing many different serializers and networking libraries in .NET at the moment. You can use a X-plat serializer we've developed here: #2947 - please comment on that thread if you're considering building hybrid .NET and .NET Core clusters.
The "fundamental changes" in this case being that the namespaces for types like string are different on .NET 4.* and .NET Core, thus a polymorphic serializer that doesn't account for those differences will throw this type of exception.
We do have a workaround available here if you'd like to use it:
https://github.com/akkadotnet/akka.net/pull/2947
You'll need to register that serializer compat layer with JSON.NET inside Lighthouse and your .NET 4.7.1 apps.

If this issue happens during deserialization in .NET Framework application and you can't do anything about the JSON you receive, then you can extend DefaultSerializationBinder and use it in JsonSerializerSettings to deserialize JSON generated by .NET Core application:
internal sealed class DotNetCompatibleSerializationBinder : DefaultSerializationBinder
{
private const string CoreLibAssembly = "System.Private.CoreLib";
private const string MscorlibAssembly = "mscorlib";
public override Type BindToType(string assemblyName, string typeName)
{
if (assemblyName == CoreLibAssembly)
{
assemblyName = MscorlibAssembly;
typeName = typeName.Replace(CoreLibAssembly, MscorlibAssembly);
}
return base.BindToType(assemblyName, typeName);
}
}
and then:
var settings = new JsonSerializerSettings()
{
SerializationBinder = new DotNetCompatibleSerializationBinder()
};

for the base Type try to use
serializer.TypeNameHandling = TypeNameHandling.Auto
can ignore the System.Private.CoreLib namespace in Json

.Net Core Library in .Net Standard: Core Long Path Handling

How would one use a .Net Core library in a regular .Net application? I came across this Hanselman article that explains I should be using a .Net Standard library. Problem is, my code only works in a Core Library (afaik).
The below method works in Core but not in a Standard library. It appears that Core handles long paths (over the 250 char limit) better than regular .Net solutions (Background).
Is there a way to use a Core library in a regular .Net library?
private static List<FileInfo> FastGetFileList(string myBaseDirectory, string searchString = "*.*")
{
var dirInfo = new DirectoryInfo(myBaseDirectory);
return dirInfo.EnumerateDirectories()
.AsParallel()
.SelectMany(di => di.EnumerateFiles(searchString, SearchOption.AllDirectories)).ToList()
.Union(dirInfo.EnumerateFiles(searchString)).ToList();
}
Full Solution for Testing
This will fail in a regular console app and also doesn't work when called from a .Net Standard 2.0 library. In a Core console app, it will work.
private string path =
#"\\GREATER THAN 250 CHARACTERS NETWORK FOLDER LOCATION"
;
static void Main(string[] args)
{
var temp = FastGetFileList(path);
Console.WriteLine("Success");
Console.ReadLine();
}
private static List<FileInfo> FastGetFileList(string myBaseDirectory, string searchString = "*.*")
{
var dirInfo = new DirectoryInfo(myBaseDirectory);
return dirInfo.EnumerateDirectories()
.AsParallel()
.SelectMany(di => di.EnumerateFiles(searchString, SearchOption.AllDirectories)).ToList()
.Union(dirInfo.EnumerateFiles(searchString)).ToList();
}

Using System.Json for non-Silverlight projects?

Any idea on how to do it? If not possible, what's a good JSON library for C#?

System.Json is now available in non-Silverlight projects via NuGet (.Net's package management system) and is hopefully going to be released as part of the core framework in vnext. The NuGet package is named JsonValue.
Imagine that we have the following JSON in the string variable json:
[{"a":"foo","b":"bar"},{"a":"another foo","b":"another bar"}]
We can get write the value "another bar" to the console using the following code:
using System.Json;
dynamic jsonObj = JsonValue.Parse(json);
var node = jsonObj[1].b;
System.Console.WriteLine(node.Value);

Here's an extenstion method to serialize any object instance to JSON:
public static class GenericExtensions
{
public static string ToJsonString<T>(this T input)
{
string json;
DataContractJsonSerializer ser = new DataContractJsonSerializer(input.GetType());
using (MemoryStream ms = new MemoryStream())
{
ser.WriteObject(ms, input);
json = Encoding.Default.GetString(ms.ToArray());
}
return json;
}
}
You'll need to add a reference to System.ServiceModel.Web to use the DataContractSerializer.

Scott Guthrie blogged about this
http://weblogs.asp.net/scottgu/archive/2007/10/01/tip-trick-building-a-tojson-extension-method-using-net-3-5.aspx

If you're just looking for JSON encoding/decoding, there is an official System.Web extension library from Microsoft that does it, odds are you probably already have this assembly (System.Web.Extensions):
http://msdn.microsoft.com/en-us/library/system.web.script.serialization.javascriptserializer.aspx
Example:
using System;
using System.Web.Script.Serialization;
class App
{
static void Main(string[] args = null)
{
JavaScriptSerializer jss = new JavaScriptSerializer();
String sJson = "{\"Name\": \"Your name\"}";
DesJson json = jss.Deserialize<DesJson>(sJson);
Console.WriteLine(json.Name);
}
}
class DesJson {
public string Name {get; set;}
}

Another option is to use Mono's implementation of System.Json,
I was able to backport it to C# 2.0 with a few minor changes.
You can simply download my C# 2.0 project from here.