We Keep Coding

Py.GIL() is giving error pythonnet embedded python in C#

I have the following C# code:
static void Main()
{
string pythonpath1 = #"C:\Users\user\Documents\pynet_test\Python\Python37";
string pythonpath2 = #"C:\Users\user\Documents\pynet_test\Python\Python37\lib";
string envpythonhome = #"C:\Users\user\Documents\pynet_test\Python\Python37\python37.dll";
Environment.SetEnvironmentVariable("PYTHONNET_PYDLL", envpythonhome, EnvironmentVariableTarget.Process);
Environment.SetEnvironmentVariable("PATH", pythonpath1, EnvironmentVariableTarget.Process);
Environment.SetEnvironmentVariable("PYTHONHOME", pythonpath1, EnvironmentVariableTarget.Process);
Environment.SetEnvironmentVariable("PYTHONPATH", pythonpath2, EnvironmentVariableTarget.Process);
using (Py.GIL())
{
dynamic np = Py.Import("numpy");
Console.WriteLine(np.cos(np.pi * 2));
dynamic sin = np.sin;
Console.WriteLine(sin(5));
double c = np.cos(5) + sin(5);
Console.WriteLine(c);
dynamic a = np.array(new List<float> { 1, 2, 3 });
Console.WriteLine(a.dtype);
dynamic b = np.array(new List<float> { 6, 5, 4 }, dtype: np.int32);
Console.WriteLine(b.dtype);
Console.WriteLine(a * b);
}
Console.WriteLine(DateTime.Now);
}
The error I am getting is:
System.MissingMethodException
HResult=0x80131513
Message=Method not found: 'System.Reflection.Emit.AssemblyBuilder System.AppDomain.DefineDynamicAssembly(System.Reflection.AssemblyName, System.Reflection.Emit.AssemblyBuilderAccess)'.
Source=Python.Runtime
StackTrace:
at Python.Runtime.CodeGenerator..ctor()
at Python.Runtime.DelegateManager..ctor()
at Python.Runtime.PythonEngine.Initialize(IEnumerable`1 args, Boolean setSysArgv, Boolean initSigs)
at Python.Runtime.PythonEngine.Initialize(Boolean setSysArgv, Boolean initSigs)
at Python.Runtime.PythonEngine.Initialize()
at Python.Runtime.Py.GIL()
at WrapperPython.Program.Main() in C:\Users\user\Documents\pynet_test\pynet_test\Program.cs:line 50
The Python environment is in the Project folder and this is the following specifications:
python version used : 3.7 (x64)
pythonnet version: 2.5.2-cp37-cp37m-win_amd64
OS : Windows Server 2019
Reference has been made to the python.Runtime.dll under site-packaged. CSproj looks like following:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.1</TargetFramework>
<LangVersion>latest</LangVersion>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<ItemGroup>
<Reference Include="Python.Runtime">
<HintPath>..\Python\Python37\Lib\site-packages\Python.Runtime.dll</HintPath>
</Reference>
</ItemGroup>
</Project>
I have tried everything provided online but can't seem to find the issue. I have a hunch it's based on the environment variables but not sure how to proceed.

okay so python.net installation is really not well documented and the folks maintaining the python.net repository don't really help a lot since it's not a "support forum".
I solved this issue by installing the python.runtime.AllPlatflorms nuget package and pointing the environment variables to the right python folders/ files.
This works with python3.8 as well.

Azure function binding to container error when unzipping file to blob storage

I have been trying to set up a Blobtrigger function that when a zip file is uploaded to a container it unzips and uploads it to the same container. I got the solution from Azure Function: Unzip file works in debug but not in production.
public static class Function1
{
[FunctionName("Function1")]
public static async Task Run([BlobTrigger("raw-nppes/{inputBlobName}", Connection = "key")] Stream inputBlob, string inputBlobName,
Binder binder,
ILogger log)
{
log.LogInformation($"Blob trigger function received blob\n Name:{inputBlobName} \n Size: {inputBlob.Length} Bytes");
if (Path.GetExtension(inputBlobName)?.ToLower() == ".zip")
{
// We use the first char of the input file name as a dynamic part in the container. (Note: You should check if this is a valid char for the container name)
var container = $"my-dynamic-container-{inputBlobName.Substring(0, 1).ToLower()}";
var attributes = new Attribute[]
{
new BlobAttribute($"{container}", FileAccess.ReadWrite),
new StorageAccountAttribute("AzureWebJobsStorage")
};
var outputContainer = await binder.BindAsync<CloudBlobContainer>(attributes);
await outputContainer.CreateIfNotExistsAsync();
var archive = new ZipArchive(inputBlob);
foreach (var entry in archive.Entries)
{
// we write the output files to a directory with the same name as the input blob. Change as required
var blockBlob = outputContainer.GetBlockBlobReference($"{inputBlobName}/{entry.FullName}");
using (var fileStream = entry.Open())
{
if (entry.Length > 0)
{
log.LogInformation($"Extracting - {entry.FullName} to - {blockBlob.Name}");
await blockBlob.UploadFromStreamAsync(fileStream);
}
}
}
}
else
{
log.LogInformation("Not a zip file. Ignoring");
}
}
}
}
I got this error: Microsoft.Azure.WebJobs.Host: Can't bind Blob to type 'Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer'
with this possible cause: Tried binding to 'Microsoft.Azure.Storage.Blob.CloudBlobDirectory, Microsoft.Azure.Storage.Blob, Version=11.1.7.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' but user type assembly was 'Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer, Microsoft.WindowsAzure.Storage, Version=9.3.1.0
I don't know what that possible cause actually means. I have looked EVERYWHERE on here, github, and social.msdn but no answer. I have no idea what the issue is.

It seems that your assembly is too old, please install the latest Microsoft.Azure.Storage.Blob assembly:
https://www.nuget.org/packages/Microsoft.Azure.Storage.Blob/11.2.2
Then use using Microsoft.Azure.Storage.Blob; instead of using Microsoft.WindowsAzure.Storage.Blob;.
The csproj file like this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<AzureFunctionsVersion>v3</AzureFunctionsVersion>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Azure.Storage.Blob" Version="11.2.2" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.Storage" Version="4.0.3" />
<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="3.0.11" />
</ItemGroup>
<ItemGroup>
<None Update="host.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
<None Update="local.settings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<CopyToPublishDirectory>Never</CopyToPublishDirectory>
</None>
</ItemGroup>
</Project>
Regarding the difference between them, you can refer to this post：
What is the difference between the Microsoft.Azure.Storage and WindowsAzure.Storage Nuget packages?

Pay attention on the "Important" section of the Changelog!
I was doing the same error.
https://github.com/Azure/azure-webjobs-sdk/releases/tag/storage-v4.0.0
Important Storage namespaces have changed, so you may need to update
your code if you are using any of the Storage types directly. Your
code may continue to compile as you may have an implicit reference to
the old Storage SDK, but it will fail at runtime if these have not
been updated in your code:
For blobs and queues, Microsoft.WindowsAzure.Storage.* namespaces are
now Microsoft.Azure.Storage.. For tables, the namespaces are now
Microsoft.Azure.Cosmos.Table..
Check your using ;)

IronPython: Error when launching .py script from c#

I'm running into a similar problem as this guy: IronPython : Microsoft.Scripting.SyntaxErrorException: 'unexpected token '=''
Unfortunately, there we no answers on that thread.
This is my code:
var engine = Python.CreateEngine();
var scope = engine.CreateScope();
try
{
engine.ExecuteFile(String.Concat(Directory.GetParent(Environment.CurrentDirectory).Parent.Parent.FullName, "\\Client.py"), scope);
}
catch(Exception e)
{
Console.WriteLine(e.Message);
}
And then here's the .py (that doesn't do anything yet, really):
#imports
import os
import tempfile
#Test
print("Here we go.")
The abomination to get the full path for my python file was an attempt to check if it got the path wrong or couldn't find the file, wasn't the case but I left it there. Debugger shows that the path is correct. However, it always fails on engine.ExecuteFile(...). and catches an exception that, according to the debugger is null. I got this error:
Microsoft.Scripting.SyntaxErrorException
and then goofed around with settings, changing Tools > Options > Debugging > General > "Enable just my Code" from checked to unchecked which lead to me not getting the SyntaxErrorException anymore but instead it's now this, but it still fails at the same line, with an exception that is still null:
IronPython.Runtime.Exceptions.ImportException in Microsoft.Dynamic.dll
At this point I don't know if I made a step in the right direction or went one back. Can anyone help with this?
EDIT: I need to correct this. There currently is an exception that states: "No module named os" instead of being just null which makes sense considering the exception type.

I moved the Lib folder to my project folder and had the search path extended by it like this:
String projectPath = Directory.GetParent(Environment.CurrentDirectory).Parent.Parent.FullName;
var engine = Python.CreateEngine();
var libs = new[]
{
String.Concat(projectPath, "\\Lib")
};
var pySP = engine.GetSearchPaths();
foreach (String resource in libs)
{
pySP.Add(resource);
}
following this thread: IronPython: No module named json.
Then I undid the changes I made to the debugging settings, and lastly added NuGet packages. Maybe there is now redundance with the added search path and the newly added packages but I am not willing to test my luck and undo any of the changes. Here is my .csproj file, for anyone who might need it when in the same position:
<Project Sdk="Microsoft.NET.Sdk.WindowsDesktop">
<PropertyGroup>
<OutputType>WinExe</OutputType>
<TargetFramework>netcoreapp3.1</TargetFramework>
<UseWPF>true</UseWPF>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="IronPython" Version="2.7.10" />
<PackageReference Include="IronPython.Interpreter" Version="2.7.4" />
<PackageReference Include="IronPython.StdLib" Version="2.7.10" />
</ItemGroup>
</Project>

FileNotFoundException: Could not load file or assembly 'System.Data.Entity'

I am trying to parse a dynamic Expression string using Linq
var x = Expression.Parameter(typeof(T), "x");
var e = Dynamic.DynamicExpression.ParseLambda(new[] { x }, null, "x.Id > 1");
var compiledDelegate = exp.Compile();
var values = new List<T>
{
new T
{
Id = 1
},
new T
{
Id = 2
}
};
var result = values.Where((Func<T, bool>)compiledDelegate);
When trying to execute the ParseLambda line, the code throws the below exception
I am using net standard 2.0 and Visual Studio 15.3.5.
I am pulling the below two packages with all the necessary ASP.NET core stuff.
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<NeutralLanguage>en</NeutralLanguage>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="System.Linq.Dynamic" Version="1.0.7" />
<PackageReference Include="System.Linq.Expressions" Version="4.3.0" />
</ItemGroup>
<PropertyGroup>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
</PropertyGroup>

With more googling I found that System.Data is not yet ported (if it will ever be) to .net standard 2.0 APIs so System.Linq.Dynamic was actually not fully compatible with .net standard 2.0.
Instead I found another port of the same assembly System.Linq.Dynamic.Core which did the job.

Embedded resource in .Net Core libraries

I just have started looking into .Net Core, and I don't see classical resources and anything what looks like resources. In classical .Net class libraries I was able to add, for example, text filtes with some script to my project, than I can add these files to project's resources. After that I could easily use that by the following way:
Connection.Execure(Properties.Resources.MySuperScript);
I see that there isn't such feature in .Net Core libraries, at least I don't see.
Is there an alternative in .Net Core to store some statical data as an embedded resource in libraries? And how to use that if it exists?

UPDATE:
.NET Core 1.1 and later have dropped project.json and returned to .csproj files.
This changes Step 2, but not all that much. The necessary lines are very similar:
<ItemGroup>
<Content Remove="_fonts/OpenSans.ttf" />
<Content Remove="_fonts/OpenSans-Bold.ttf" />
<Content Remove="_fonts/OpenSans-Italic.ttf" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="_fonts/OpenSans.ttf" />
<EmbeddedResource Include="_fonts/OpenSans-Bold.ttf" />
<EmbeddedResource Include="_fonts/OpenSans-Italic.ttf" />
</ItemGroup>
There may be a similar *.tff form; unconfirmed.
Steps 1 and 3 are unchanged.
To use embedded resources in .NET Core 1.0 project do the following:
Add your embedded file(s) as usual.
Example: some FONT files on a directory named "_fonts"
Modify "project.json" to include the related resources.
In my case:
"buildOptions": {
"embed": {
"include": [
"_fonts/*.ttf"
]
}
},
Access the embedded resource in code.
var assembly = typeof(MyLibrary.MyClass).GetTypeInfo().Assembly;
Stream resource = assembly.GetManifestResourceStream("MyLibrary._fonts.OpenSans.ttf");
The key point is to use the right name on GetManifestResourceStream call. You have to use [assembly name].[directory].[file name].

Now that project.json is deprecated, you have to specify this in the .csproj file.
<ItemGroup>
<EmbeddedResource Include="_fonts\*.ttf" />
</ItemGroup>
You can use a wildcard as shown, or just list out the files explicitly.

With newer versions of .Net Core - 2.0 or greater - there's a specialized class EmbeddedFileProvider that abstract the embedded file reading. To use it, add Microsoft.Extensions.FileProviders.Embedded package to your application:
dotnet add package Microsoft.Extensions.FileProviders.Embedded
The EmbeddedFileProvider allows you to create a stream reader, and use according to your scenario:
var embeddedProvider = new EmbeddedFileProvider(Assembly.GetExecutingAssembly());
using (var reader = embeddedProvider.GetFileInfo("yourfile.ext").CreateReadStream())
{
// some logic with stream reader
}

People have already generally answered this, so this is a rendering of the answers into something simple.
Before using the following, the file should be added as an embedded resource to the .csproj / project.json
Usage
var myJsonFile = ReadManifestData<Tests>("myJsonFile.json");
Parameter: embedded filename name; Type: any class from the target resource's assembly
looks for an embedded resource with that name
returns the string value
Method
public static string ReadManifestData<TSource>(string embeddedFileName) where TSource : class
{
var assembly = typeof(TSource).GetTypeInfo().Assembly;
var resourceName = assembly.GetManifestResourceNames().First(s => s.EndsWith(embeddedFileName,StringComparison.CurrentCultureIgnoreCase));
using (var stream = assembly.GetManifestResourceStream(resourceName))
{
if (stream == null)
{
throw new InvalidOperationException("Could not load manifest resource stream.");
}
using (var reader = new StreamReader(stream))
{
return reader.ReadToEnd();
}
}
}

I have not confirmed this in documentation, but for me, it would appear the auto-generated Resource code that retrieves embedded files found in Resource.Designer.cs is now functioning again in .NET Core 3.1. I can now retrieve an embedded jpg simply by calling the Properties.Resources.MyImageName which returns a Bitmap object.