AWS Lambda access local resource or storage C# - c#

How can I store and access a file in Lambda using C# I used the tmp folder that is available for the lambda but I get an error of could not load file or assembly. How can I solve the error? I used the ADP nuget.
using (WebClient webClient = new WebClient())
{
webClient.DownloadFile(reportLine, Path.GetTempPath() +
"sample_auth.key");
}
I used this to download the file into the tmp folder of the lambda. I did not include the other config in the string confidential but you can check the github below for the exact same sample.
string config = #"{
""sslCertPath"": ""/tmp/sample.pfx"",
""sslKeyPath"": ""/tmp/sample_auth.key"",
}";
ADPAccessToken token = null;
if (String.IsNullOrEmpty(clientconfig))
{
Console.WriteLine("Settings file or default options not available.");
}
else
{
ClientCredentialConfiguration connectionCfg = JSONUtil.Deserialize<ClientCredentialConfiguration>(clientconfig);
ClientCredentialConnection connection = (ClientCredentialConnection)ADPApiConnectionFactory.createConnection(connectionCfg);
//context.Logger.Log(ADPApiConnection.certificatepath);
//context.Logger.Log(clientconfig);
try
{
connection.connect();
if (connection.isConnectedIndicator())
{
token = connection.accessToken;
// context.Logger.Log("Connected to API end point");
// //Console.WriteLine("Token: ");
// //Console.WriteLine(" AccessToken: {0} ", token.AccessToken);
// //Console.WriteLine(" TokenType: {0} ", token.TokenType);
// //Console.WriteLine(" ExpiresIn: {0} ", token.ExpiresIn);
// //Console.WriteLine(" Scope: {0} ", token.Scope);
// //Console.WriteLine(" ExpiresOn: {0} ", token.ExpiresOn);
// //Console.ReadLine();
}
}
catch (ADPConnectionException e)
{
context.Logger.Log(e.Message);
}
//catch (Exception e)
//{
// context.Logger.Log(e.Message);
//}
//Console.Read();
}
return "Ok";
}
I get an error of I think lambda check the /var/task folder
errorMessage": "One or more errors occurred. (Could not load file or
assembly 'System.Net.Http.WebRequest, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file
specified.\n)",
"cause": {
"errorType": "FileNotFoundException",
"errorMessage": "Could not load file or assembly
'System.Net.Http.WebRequest, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file
specified.\n",
Here is the sample program: https://github.com/adplabs/adp-connection-NET/blob/master/ADPClientDemo/Program.cs
I can run the program on console but when I try to do it in lambda I get an error. Is it because of the NuGet from AWS?
I have the following NuGet
Amazon Lambda Core
Amazon Lambda S3 Events
Amazon lambda Serialization json
AWS SDK Core
Microsoft Asp Net Web Api Client
ADP library connection NET

Could not load file or assembly: System.Net.Http.WebRequest
The error seems to be caused by a versioning issue, I think you need to use a .Net core version of System.Net.Http.WebRequest dll or a later version than .Net 4.0 to work with .NET Core 2.0.
Actually please see this answer you might be out of luck: The libraries you use need to ship targeting .NET Core: https://github.com/dotnet/corefx/issues/28267#issuecomment-396349873
Also see https://stackoverflow.com/a/41683787/495455 and Could not load file or assembly "System.Net.Http.Webrequest" in .NET AWSSDK for mono for similar versioning issues and fixes.
If that doesn't fix it, please consider using the AWS API. You could put your sample_auth.key file on an S3 bucket and read it, eg
https://docs.aws.amazon.com/lambda/latest/dg/with-s3.html
Or as per the example you linked to they package the json file with the Lambda:
https://github.com/adplabs/adp-connection-NET/tree/master/ADPClientDemo/Content/config
And they read it using StreamReader, perhaps this will work using the System.IO dll instead of trying to find the System.Net.Http.WebRequest dll:
string configFileName = "default.json";
StreamReader sr = new StreamReader("..\\..\\Content\\config\\" + configFileName);
string clientconfig = sr.ReadToEnd();

Related

.net C# jdbc driver connector exception

I'm trying to connect to a hyperSQL database (HSQLDB) on my local machine from a c# application in visual studio.
I've ran through the steps to create the .net JDBC driver to build the dll, and also downloading the demo console application in the "download" section from the following url: http://nikolaiklimov.de/query-java-HyperSQL-database-with-csharp/
The demo application works!! i can connect to the database and query its contents . Next I have converted the console app into a class library then call the class library to query the DB but this is where it falls over and I get initialization errors of System.TypeInitializationException. any idea why the project falls over after converting to a class library. (if i convert the class library straight back a console app it works again).
The code and connection string are:
namespace HyperSQL
{
public static class sqlconnector
{
readonly static string CONNECTION_STRING =
ConfigurationManager.ConnectionStrings["HyperSQL"].ConnectionString;
const string SQL = "SELECT * FROM meeting";
public static void getdata()
{
try
{
java.sql.DriverManager.registerDriver(new org.hsqldb.jdbcDriver());
using (java.sql.Connection conn = java.sql.DriverManager.getConnection(CONNECTION_STRING))
{
java.sql.PreparedStatement ps = conn.prepareStatement(SQL);
using (java.sql.ResultSet rs = ps.executeQuery())
{
while (rs.next())
{
Console.WriteLine($"MEETING_NO={rs.getInt("MEETING_NO")}");
Console.WriteLine("------------------");
}
}
}
}
catch (Exception ex)
{
}
Console.ReadLine();
}
}
}
jdbc:hsqldb:hsql://localhost:3458/elitedb;crypt_key=DADADADADAADDADAD;crypt_type=AES;shutdown=true;write_delay=false;user=****;password=****
Ive added a console app to my solution after the conversion to a class libary. The console app code is below:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
try
{
HyperSQL.sqlconnector.getdata();
}
catch (Exception ex)
{
}
}
}
}
Here is a link to download my solution created in visual studio. It contains a console application that calls a function in a class library. The class library (HyperSQL) itself is simply converted from a console application and can be converted back. You will need a running instance of HyperSQL on your machine to be able to connect successfully.
http://eliteservicedev.azurewebsites.net/DemoHyperSQL.zip
Have you seen the error listed in the original site you posted?
If your forget to add runtime dependencies to the build folder of your
project then you can get these errors: Could not load file or
assembly 'IKVM.OpenJDK.Util, Version=8.1.5717.0, Culture=neutral,
PublicKeyToken=13235d27fcbfff58' or one of its dependencies. The
system cannot find the file specified. TypeInitializationException:
The type initializer for 'org.hsqldb.jdbc.JDBCDriver' threw an
exception: FileNotFoundException: Could not load file or assembly
'IKVM.Runtime, Version=8.1.5717.0, Culture=neutral,
PublicKeyToken=13235d27fcbfff58' or one of its dependencies. 'The
type initializer for 'sun.util.locale.provider.LocaleProviderAdapter'
threw an exception.' FileNotFoundException: Could not load file or
assembly 'IKVM.OpenJDK.Text, Version=8.1.5717.0, Culture=neutral,
PublicKeyToken=13235d27fcbfff58' or one of its dependencies. 'The
type initializer for 'org.hsqldb.HsqlDateTime' threw an exception.'
InvalidCastException: Unable to cast object of type
'java.util.PropertyResourceBundle' to type
'sun.util.resources.OpenListResourceBundle'.
***** UPDATE
With the source project i see an error, looking in the bin\debug folder i see that non all the referenced file are copied to ConsoleApp1.exe bin folder: this is the compare result:
Confronto in corso dei file consoleapp1.txt e DEMOCSHARPTHYPERSQL.TXT
***** consoleapp1.txt
ConsoleApp1.exe
ConsoleApp1.exe.config
ConsoleApp1.pdb
hsqldb.dll
HyperSQL.dll
***** DEMOCSHARPTHYPERSQL.TXT
hsqldb.dll
HyperSQL.dll
*****
***** consoleapp1.txt
IKVM.OpenJDK.Jdbc.dll
IKVM.OpenJDK.Text.dll
***** DEMOCSHARPTHYPERSQL.TXT
IKVM.OpenJDK.Jdbc.dll
IKVM.OpenJDK.Localedata.dll --> Missing in ConsoleApp1.exe folder
IKVM.OpenJDK.Text.dll
*****
If i manually copy the missing dll on ConsoleApp1.exe folder the program run correctly. I think that is a visual studio bug
Seems VS is not copying all IKVM DLLs to output folder due to not resolving they are needed. Please see this SourceForge issue and this GitHub issue, the later one with my comments.

Run Mono.Cecil in .NET Core

I ran the HelloWorld console app example from a SO anwser compiled with .NET Core 2 and Mono.Cecil 0.10.0-beta7:
var myHelloWorldApp = AssemblyDefinition.CreateAssembly(
new AssemblyNameDefinition("HelloWorld", new Version(1, 0, 0, 0)), "HelloWorld", ModuleKind.Console);
var module = myHelloWorldApp.MainModule;
// create the program type and add it to the module
var programType = new TypeDefinition("HelloWorld", "Program",
Mono.Cecil.TypeAttributes.Class | Mono.Cecil.TypeAttributes.Public, module.TypeSystem.Object);
module.Types.Add(programType);
// add an empty constructor
var ctor = new MethodDefinition(".ctor", Mono.Cecil.MethodAttributes.Public | Mono.Cecil.MethodAttributes.HideBySig
| Mono.Cecil.MethodAttributes.SpecialName | Mono.Cecil.MethodAttributes.RTSpecialName, module.TypeSystem.Void);
// create the constructor's method body
var il = ctor.Body.GetILProcessor();
il.Append(il.Create(OpCodes.Ldarg_0));
// call the base constructor
il.Append(il.Create(OpCodes.Call, module.Import(typeof(object).GetConstructor(Array.Empty<Type>()))));
il.Append(il.Create(OpCodes.Nop));
il.Append(il.Create(OpCodes.Ret));
programType.Methods.Add(ctor);
// define the 'Main' method and add it to 'Program'
var mainMethod = new MethodDefinition("Main",
Mono.Cecil.MethodAttributes.Public | Mono.Cecil.MethodAttributes.Static, module.TypeSystem.Void);
programType.Methods.Add(mainMethod);
// add the 'args' parameter
var argsParameter = new ParameterDefinition("args",
Mono.Cecil.ParameterAttributes.None, module.Import(typeof(string[])));
mainMethod.Parameters.Add(argsParameter);
// create the method body
il = mainMethod.Body.GetILProcessor();
il.Append(il.Create(OpCodes.Nop));
il.Append(il.Create(OpCodes.Ldstr, "Hello World"));
var writeLineMethod = il.Create(OpCodes.Call,
module.Import(typeof(Console).GetMethod("WriteLine", new[] { typeof(string) })));
// call the method
il.Append(writeLineMethod);
il.Append(il.Create(OpCodes.Nop));
il.Append(il.Create(OpCodes.Ret));
// set the entry point and save the module
myHelloWorldApp.EntryPoint = mainMethod;
myHelloWorldApp.Write("HelloWorld.exe");
The above code executes fine when compiling with .NET Framework, but when compiling with .NET Core resulted in the error:
Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'System.Console, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
at HelloWorld.Program.Main(String[] args)
I'd like to ask, why can't the HelloWorld app locate the assembly file mscorlib.dll? What should I do to fix it?
I solved this by creating a HelloWorld.runtimeconfig.json at the same folder of HelloWorld.exe and then ran dotnet ./HelloWorld.exe. It would print Hello World to the console.
The Json file:
{
"runtimeOptions": {
"tfm": "netcoreapp2.0",
"framework": {
"name": "Microsoft.NETCore.App",
"version": "2.0.0"
}
}
}
you must be performing the assembly manipulation from within .net classic, ie .net 4.x. so this means when u do typeof(Console) is imports the Console from the full runtime, no from the NET Core runtime
If you use typeof(...), you are importing types from the current runtime (.NET Core 2). If you want to make a .NET Framework application from .NET Core, you need to import mscorlib from .NET Framework on Windows, or mono, and use the types from it. See my other answer: https://stackoverflow.com/a/73434552/4205390

Creating a signalR client HubConnection with Powershell

I am trying to connect to a SignalR hub using a powershell script. I am very new to powershell, so please excuse any rookie mistake.
I have set up a minimal not working example of what I have tried here :
Gist
Relevant code:
Load dlls
$dllFolder = -join((Get-Item -Path ".\" -Verbose).FullName, "\bin\Debug\")
[string[]] $dllPathsToLoad = #("\Newtonsoft.Json.dll", "\Microsoft.AspNet.SignalR.Client.dll")
$token = "insertyourtokenhere"
function LoadDllPaths($dlls)
{
foreach ($dll in $dlls)
{
$dllpath = $dllFolder + $dll
[System.Reflection.Assembly]::LoadFrom($dllpath)
}
}
[...]
LoadDllPaths($dllPathsToLoad)
Create HubConnection:
$server = "https://localhost/rest/"
[...]
$hub = New-Object Microsoft.AspNet.SignalR.Client.HubConnection($server)
Steps:
Create a new Visual Studio project
Add Newtonsoft.Json v10.0.2 Nuget package (latest)
Add Microsoft.AspNet.SignalR.Client v2.2.2 Nuget package (latest)
Add powershell script to the root of the project
With powershell (run as admin), type .\HubConnectionTestsScript.ps1
Result:
View on imgur
Error : System.Management.Automation.MethodInvocationException: Exception calling ".ctor" with "1" argument(s): "Could not load file or assembly 'Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file specified." ---> System.IO.FileNotFoundException: Could not load file or assembly 'Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file specified.
at Microsoft.AspNet.SignalR.Client.Connection..ctor(String url, String queryString)
at Microsoft.AspNet.SignalR.Client.HubConnection..ctor(String url, Boolean useDefaultUrl)
--- End of inner exception stack trace ---
at System.Management.Automation.DotNetAdapter.AuxiliaryConstructorInvoke(MethodInformation methodInformation, Object[] arguments, Object[] originalArguments)
at System.Management.Automation.DotNetAdapter.ConstructorInvokeDotNet(Type type, ConstructorInfo[] constructors, Object[] arguments)
at Microsoft.PowerShell.Commands.NewObjectCommand.CallConstructor(Type type, ConstructorInfo[] constructors, Object[] args)
This signalR source code object seems to be the problem, I just don't see what part of it can be throwing this error.
Question:
Why does the error mention Newtonsoft.Json v6.0.0 when signalR dependencies say >=6.0.4, and I have 10.0.2?
Am I doing anything wrong in my Powershell script which could be causing this?
Thank you very much! Any help is appreciated at this point
I managed to solve this issue with some help from a colleague. Sharing the solution here in case anyone ever struggles on the same problem.
It appears that one of SignalR dependencies tries to load an old version of Newtonsoft.Json. We can force it to redirect him to our own instance of Newtonsoft.Json
Inspired by this gist, here is the idea :
When you load your Json Assembly, store it in a variable
$newtonsoftAssembly = [System.Reflection.Assembly]::LoadFrom($dllFolder + "\Newtonsoft.Json.dll")
Afterwards, setup the redirect bindings. My best guess is that this intercepts any call to load an assembly, giving us the opportunity to return our own Json assembly instead of letting him fail to find the version he wants (6.0.0 in my case).
function RedirectJsonBindings()
{
$onAssemblyResolveEventHandler = [System.ResolveEventHandler] {
param($sender, $e)
# You can make this condition more or less version specific as suits your requirements
if ($e.Name.StartsWith("Newtonsoft.Json")) {
Write-Host "Newtonsoft assembly" $e.Name -ForegroundColor DarkGreen
return $newtonsoftAssembly
}
foreach($assembly in [System.AppDomain]::CurrentDomain.GetAssemblies()) {
if ($assembly.FullName -eq $e.Name) {
return $assembly
}
}
return $null
}
[System.AppDomain]::CurrentDomain.add_AssemblyResolve($onAssemblyResolveEventHandler)
}
And finally, at the end of your script, unbind
# Detach the event handler (not detaching can lead to stack overflow issues when closing PS)
[System.AppDomain]::CurrentDomain.remove_AssemblyResolve($onAssemblyResolveEventHandler)

Load latest assembly version dynamically from GAC

I'd like to dynamically load the latest installed version of an assembly in the GAC using reflections.
So far I've found multiple working ways to accomplish this but they're all having their specific downsides.
The easiest solution is using the Assembly.LoadWithPartialName() method. But this method is obsolete since .NET Framework 2:
var assembly = Assembly.LoadWithPartialName("Microsoft.WindowsAzure.ServiceRuntime");
Another possibility is to use Assembly.Load() (as recommended by the obsolete warning) and call the different assembly versions with their fully qualified assembly name in a try/catch block to get the latest installed version. This screams for maintenance and just looks dirty:
Assembly assembly = null;
try
{
assembly = Assembly.Load("Microsoft.WindowsAzure.ServiceRuntime, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35");
}
catch (FileNotFoundException) { }
try
{
assembly = Assembly.Load("Microsoft.WindowsAzure.ServiceRuntime, Version=1.7.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35");
}
catch (FileNotFoundException) { }
Last but not least there's another solution I found here on SO using the Assembly.LoadFrom() method and then basically importing the assembly manager module fusion.dll to figure the path of the latest version. This seems to be way too much for such a "simple" task.
Isn't there any better solution to accomplish that without using an obsolete method, creating a maintenance hell with magic strings or by calling unmanaged code?
Thanks in advance!
Fusion.dll... ugh. I would find your target framework version and enumerate all of the gas directories and simply query them based on the root assembly name so that your only magic string is the assembly name. Then, use Assembly.ReflectionOnlyLoadFrom static method to compare the versions of until you find the latest one and then use the Assembly.Load method with the appropriate file path arguments. You may have trouble with this using click-once download unless you have given the app the appropriate trust.
// List of all the different types of GAC folders for both 32bit and 64bit
// environments.
List<string> gacFolders = new List<string>() {
"GAC", "GAC_32", "GAC_64", "GAC_MSIL",
"NativeImages_v2.0.50727_32",
"NativeImages_v2.0.50727_64"
};
foreach (string folder in gacFolders)
{
string path = Path.Combine(#"c:\windows\assembly", folder);
if(Directory.Exists(path))
{
Response.Write("<hr/>" + folder + "<hr/>");
string[] assemblyFolders = Directory.GetDirectories(path);
foreach (string assemblyFolder in assemblyFolders)
{
Response.Write(assemblyFolder + "<br/>");
}
}
}
See this other answer here on stack overflow. I used this to enumerate the gac successfully.

How to get access to embedded assembly's metadata using IMetaDataDispenser.OpenScope?

I have a .NET solution which consists of several projects. It's possible to say that one of these projects is logically a primary one and all others are secondary. Our team has decided to build the project the next way. The main project will produce an assembly (I'll refer it to as Primary). All other projects' assemblies are Secondary and they will be embedded as a resource into the Primary one.
The SourceCodeForExceptionHelper class in the Primary project is responsible for getting the original source code using PDB files on every encountered exception. To do that I use the approach described here. It worked correctly in my separate proof of concept project. But when I tried to move that class into the real solution I've encountered a problem: the IMetaDataDispenser.OpenScope method requires not null reference to assembly file's path. Surely, I haven't such a reference for any of Secondary assembly (because their files are embedded in the Primary). For that reason I can't create an object of the type ISymbolReader and read the source code. How can I solve that problem? By the way, the problem is even worse because we embed only Secondary assemblies without their PDB files (though we will do it if it is necessary).
Thanks in advance for any help and advice!
I don't think you can do this using the .NET Framework builtin functions, as they rely on physical files. However, there is a solution using the Mono Cecil library, as it has an overloads that takes a Stream as input instead of a file path for its symbols reader.
Here is an example of a Console app named "TestPdb" which dumps its IL code to the console, including PDB information:
using System;
using System.IO;
using Mono.Cecil;
using Mono.Cecil.Cil;
using Mono.Cecil.Pdb;
namespace TestPdb
{
class Program
{
static void Main(string[] args)
{
// we use a Stream for the assembly
AssemblyDefinition asm;
using (FileStream asmStream = new FileStream("testpdb.exe", FileMode.Open, FileAccess.Read, FileShare.Read))
{
asm = AssemblyDefinition.ReadAssembly(asmStream);
}
// we use a Stream for the PDB
using (FileStream symbolStream = new FileStream("testpdb.pdb", FileMode.Open, FileAccess.Read, FileShare.Read))
{
asm.MainModule.ReadSymbols(new PdbReaderProvider().GetSymbolReader(asm.MainModule, symbolStream));
}
TypeDefinition type = asm.MainModule.GetType("TestPdb.Program");
foreach (MethodDefinition method in type.Methods)
{
Console.WriteLine("Method:" + method.Name);
foreach (Instruction ins in method.Body.Instructions)
{
Console.WriteLine(" " + ins);
if (ins.SequencePoint != null)
{
Console.WriteLine(" Url:" + ins.SequencePoint.Document.Url);
// see http://blogs.msdn.com/b/jmstall/archive/2005/06/19/feefee-sequencepoints.aspx
if (ins.SequencePoint.StartLine != 0xFEEFEE)
{
Console.WriteLine(" StartLine:" + ins.SequencePoint.StartLine + " StartColumn:" + ins.SequencePoint.StartColumn);
Console.WriteLine(" EndLine:" + ins.SequencePoint.EndLine + " EndColumn:" + ins.SequencePoint.EndColumn);
}
// etc...
}
}
}
}
}
}
NOTE: since you only need to read from PDBs, you can recompile the Cecil library defining the READ_ONLY conditional symbol to save some bytes. You can also embed Cecil source code directly in your assemblies, no need to use the .DLL versions.

Categories