Trying to build OpenNN for use in C# with Visual Studio - c#

Link to the source. Link to the build instructions.
I create a new DLL C++ library project for .NET 4.7.
I go into the properties and add the eigen directory (a dependency) to the "Additional Include Directories" and "Additional #using Directories".
I add all of the .h files in the opennn directory to the project's "Header Files" directory and all of the .cpp files to the project's "Source Files" directory.
I attempt to build the project and get a ton of the same error. I find the solution here, selecting every file in the Source Files directory, going into properties, and setting "Precompiled Header" to "Not Using Precompiled Headers".
The build is running fine for a bit, with a bunch of signed/unsigned mismatch warnings I can ignore.
I get a boatload of errors.
missing type specifier - int assumed. Note: C++ does not support default-int ~ informedness_optimization_threshold.cpp ~ 172
syntax error: missing ';' before '*' ~ informedness_optimization_threshold.cpp ~ 172
'performance_functional_pointer': undeclared identifier ~ informedness_optimization_threshold.cpp ~ 172
'get_performance_functional_pointer': is not a member of 'OpenNN::TrainingStrategy' ~ informedness_optimization_threshold.cpp ~ 172
'performance_functional_pointer': undeclared identifier ~ informedness_optimization_threshold.cpp ~ 174
left of '->get_neural_network_pointer' must point to class/struct/union/generic type ~ informedness_optimization_threshold.cpp ~ 174
'maximum_iterations_number': undeclared identifier ~ informedness_optimization_threshold.cpp ~ 238
'MaximumIterations': is not a member of 'OpenNN::ThresholdSelectionAlgorithm' ~ informedness_optimization_threshold.cpp ~ 247
'MaximumIterations': undeclared identifier ~ informedness_optimization_threshold.cpp ~ 247
'final_binary_classification_test': is not a member of 'OpenNN::InformednessOptimizationThreshold::InformednessOptimizationThresholdResults' ~ informedness_optimization_threshold.cpp ~ 275
'maximum_iterations_number': undeclared identifier ~ informedness_optimization_threshold.cpp ~ 316
'maximum_time': undeclared identifier ~ informedness_optimization_threshold.cpp ~ 325
'maximum_time': undeclared identifier ~ informedness_optimization_threshold.cpp ~ 449
'set_maximum_iterations_number': identifier not found ~ informedness_optimization_threshold.cpp ~ 527
'set_maximum_time': identifier not found ~ informedness_optimization_threshold.cpp ~ 546
How do I achieve my end goal of calling OpenNN functions from C#?

I have compiled OpenNN with flowing procedure.
remove informedness_optimization_threshold.cpp, informedness_optimization_threshold.h from project.
Add tinyxml2.cpp, tinyxml2.h in tinyxml2 folder to project.
I think it's not a good solution but may be we need more update of OpenNN.

Related

Using Numpy.NET NuGet package in C# class library making .exe file too big

I'm using Numpy.NET NuGet package in C# class library and it's making .exe files on release build too big (from ~10 to ~30 MB). What is the exact reason for that and is there any solution to this problem?
I used ILDASM to get stats of .exe file and this is what it showed:
File size : 31318016
PE header size : 512 (472 used) ( 0.00%)
PE additional info : 366928 ( 1.17%)
Num.of PE sections : 2
CLR header size : 72 ( 0.00%)
CLR meta-data size : 157972 ( 0.50%)
CLR additional info : 30706176 (-39.09%)
CLR method headers : 8741 ( 0.03%)
Managed code : 77296 ( 0.25%)
Data : 367104 ( 1.17%)
Unaccounted : -366785 (-1.17%)
Num.of PE sections : 2
.text - 30950400
.rsrc - 367104
CLR meta-data size : 157972
Module - 1 (10 bytes)
TypeDef - 275 (3850 bytes) 8 interfaces, 0 explicit layout
TypeRef - 436 (2616 bytes)
MethodDef - 1598 (22372 bytes) 29 abstract, 0 native, 1537 bodies
FieldDef - 1013 (6078 bytes) 20 constant
MemberRef - 1210 (7260 bytes)
ParamDef - 1212 (7272 bytes)
MethodImpl - 65 (390 bytes)
Constant - 189 (1134 bytes)
CustomAttribute- 1041 (6246 bytes)
StandAloneSig - 210 (420 bytes)
InterfaceImpl - 77 (308 bytes)
PropertyMap - 104 (416 bytes)
Property - 400 (2400 bytes)
MethodSemantic- 731 (4386 bytes)
TypeSpec - 255 (510 bytes)
Assembly - 1 (22 bytes)
AssemblyRef - 27 (540 bytes)
ManifestResource- 48 (576 bytes)
NestedClass - 68 (272 bytes)
EventMap - 19 (76 bytes)
Event - 28 (168 bytes)
GenericParam - 12 (96 bytes)
MethodSpec - 173 (692 bytes)
GenericParamConstraint- 1 (4 bytes)
Strings - 50364 bytes
Blobs - 18320 bytes
UserStrings - 20924 bytes
Guids - 16 bytes
Uncategorized - 234 bytes
CLR additional info : 30706176
Resources - 30706176
CLR method headers : 8741
Num.of method bodies - 1537
Num.of fat headers - 413
Num.of tiny headers - 1124
Num.of fat sections - 14
Num.of small sections - 81
Managed code : 77296
Ave method size - 50
Turns out the issue was in .csproj file. It has following lines, that embeds all .dll's on "AfterResolveReferences" event (don't know if it's VS default settings or previous dev's did this on purpouse).
<Target Name="AfterResolveReferences">
<ItemGroup>
<EmbeddedResource Include="#(ReferenceCopyLocalPaths)" Condition="'%(ReferenceCopyLocalPaths.Extension)' == '.dll'">
<LogicalName>%(ReferenceCopyLocalPaths.DestinationSubDirectory)%(ReferenceCopyLocalPaths.Filename)%(ReferenceCopyLocalPaths.Extension)</LogicalName>
</EmbeddedResource>
</ItemGroup>
</Target>
My sln have multiple projects and main project doesn't need all libraries that this solutions use, so i added condition to exclude all python-related libraries and it did the trick. This didn't hurt the programm, because code library project that use Numpy.net still working great after installer build and .exe file now 20 MB lighter.
Below the modified code:
<Target Name="AfterResolveReferences">
<ItemGroup>
<EmbeddedResource Include="#(ReferenceCopyLocalPaths)" Condition="'%(ReferenceCopyLocalPaths.Extension)' == '.dll' And '%(ReferenceCopyLocalPaths.Filename)' != 'Numpy' And '%(ReferenceCopyLocalPaths.Filename)' != 'Python.Runtime' And '%(ReferenceCopyLocalPaths.Filename)' != 'Python.Included'
And '%(ReferenceCopyLocalPaths.Filename)' != 'Python.Deployment'">
<LogicalName>%(ReferenceCopyLocalPaths.DestinationSubDirectory)%(ReferenceCopyLocalPaths.Filename)%(ReferenceCopyLocalPaths.Extension)</LogicalName>
</EmbeddedResource>
</ItemGroup>
</Target>

WinDbg "invalid access to memory location" - InitializeProcThreadAttributeList

I'm relatively new to WinDbg, so I'm hoping this is just something obvious I'm missing. I have a .NET assembly that contains the function calls InitializeProcThreadAttributeList, UpdateProcThreadAttribute and DeleteProcThreadAttributeList, defined using P/Invoke. All three of those functions are imported from the "kernel32.dll" library. My goal is to trace these functions to understand the ntdll sys calls being used.
I tried to set a breakpoint with this command:
bp KERNEL32!InitializeProcThreadAttributeList
, but got this error:
Couldn't resolve error at 'KERNEL32!InitializeProcThreadAttributeList'
Next, I searched for any sign of this function using the following command:
x kernel32!*procthread*
I got this reply:
00007ffd`c7598588 KERNEL32!_imp_InitializeProcThreadAttributeList = <no type information>
I then tried to set a breakpoint on this stub? function like this:
bp kernel32!_imp_InitializeProcThreadAttributeList
Although it says the breakpoint has been defined, when I continue execution, it errors with this:
Unable to insert breakpoint 0 at 00007ffd`c7598588, Win32 error 0n998
"Invalid access to memory location."
I also tried setting the breakpoint with "bu", but that produced the same error.
Does anyone know how to set a breakpoint in this situation? Thank you in advance!
Most of the functions in kernel32 have been implemented in kernelbase.dll
and kernel32 only contains a thunk
the telltale sign is the IMP denoting imported from
_imp_InitializeProcThreadAttributeList
try setting bp kernelbase!InitializeProcThreadAttributeList
0:000> x /v kernel32!InitializeProcThreadAttributeList
pub func 761f4fc1 0 kernel32!InitializeProcThreadAttributeList (<no parameter info>)
0:000> u kernel32!InitializeProcThreadAttributeList l1
kernel32!InitializeProcThreadAttributeList:
761f4fc1 ff25c0181476 jmp dword ptr [kernel32!_imp__InitializeProcThreadAttributeList (761418c0)]
0:000> ? poi(kernel32!_imp__InitializeProcThreadAttributeList)
Evaluate expression: 1978493618 = 75ed6ab2
0:000> ln poi(kernel32!_imp__InitializeProcThreadAttributeList)
Exact matches:
KERNELBASE!InitializeProcThreadAttributeList (<no parameter info>)
0:000> bp poi(kernel32!_imp__InitializeProcThreadAttributeList)
0:000> bl
0 e 75ed6ab2 0001 (0001) 0:**** KERNELBASE!InitializeProcThreadAttributeList

Using gulp-sass-glob in Visual Studio

In Visual Studio (Task Runner Explorer), I am trying to use gulp-sass-glob in my SCSS:
gulpfile.js
var gulp = require('gulp'),
sass = require('gulp-sass'),
sassGlob = require('gulp-sass-glob');
gulp.task('css', function () {
return gulp.src('./gulp/scss/**/*.scss')
.pipe(sassGlob())
.pipe(sourcemaps.init())
.pipe(sass({ outputStyle: 'compressed' }))
.pipe(sourcemaps.write('.'))
.pipe(gulp.dest('./css'));
});
site.scss
#import "modules/**/*.*";
But in Task Runner Explorer, I get the following error:
Error: File to import not found or unreadable: modules/**/*.*
Parent style sheet: C:/Users/.../scss/site.scss
on line 95 of gulp/scss/site.scss
>> #import "modules/**/*.*";
Does anybody know how to fix this error or if this is even achievable when using SCSS and gulp in Visual Studio?
gulp-sass-glob uses the glob package for path globbing. Its documentation has this to say about **:
** If a "globstar" is alone in a path portion, then it matches zero or more directories and subdirectories searching for matches.
Notice the bolded part. It means ** only has special meaning when its the only thing between two slashes /.
Try the following in your site.scss:
#import "modules/**/*.*";

Can't resolve entity, yet it exists

I'm parsing an XML file that contains entities. I receive the following exception:
Reference to undeclared entity 'deg'. Line 283, position 53.
This is my DOCTYPE:
<!DOCTYPE dmodule [
<!ENTITY % ISOEntities PUBLIC "ISO 8879-1986//ENTITIES ISO Character Entities 20030531//EN//XML" "ent/ISOEntities">
%ISOEntities;
]>
Inside the ISOEntities file I have this line:
<!ENTITY deg "°"> <!-- DEGREE SIGN -->
The entity definition exists. I don't understand why this particular entity is causing an error, when everything else is being resolved.

Determine line number of InnerException from minidump using WinDbg

I'm trying to track down a NullReferenceException from a dump. The NullReferenceException is not the crashing exception, rather the crashing exception is a TargetInvocationException with an InnerException which is the NullReferenceException.
I'm using Windbg with SOS, I use the the command analyze -v and this gives me the call stack of the NullReferenceException:
EXCEPTION_OBJECT: !pe f6cb150
Exception object: 000000000f6cb150
Exception type: System.NullReferenceException
Message: Object reference not set to an instance of an object.
InnerException: <none>
StackTrace (generated):
SP IP Function
000000002CD9D8C0 000007FF01E7C639 MyDll!DoSomething2()+0xe99
000000002CD9DBE0 000007FF01E7B11D MyDll!DoSomething1()+0x43d
000000002CD9DD20 000007FF01E7AB11 MyDll!WorkerDoWork(System.Object, System.ComponentModel.DoWorkEventArgs)+0x51
000000002CD9DD80 000007FEEA68A0F2 System_ni!System.ComponentModel.BackgroundWorker.WorkerThreadStart(System.Object)+0x62
Notice, that I get the method names with byte offsets, but no line numbers. DoSomething2 is a large function, so it's not obvious where the NullReferenceException occurred.
I attempted to follow the instructions in Tess Ferrandez's blog:
.Net exceptions - Tracking down where in the code the exceptions occurred
But I got stuck early on where I attempt to determine the method descriptor for the method DoSomething2 using !ip2md with the IP of DoSomething2: 7FF01E7C639:
> !ip2md 7FF01E7C639
Failed to request MethodData, not in JIT code range
Note that the !ip2md command succeeds on the IP of the method where the TargetInvocationException occurred.
Question:
Where can I go from here to narrow down what line in DoSomething2 is crashing?
Note that I cannot reproduce the crash, so all I have this (and several duplicate) dumps.
Additional Notes:
.NET 4.0
Windbg Version: 6.12.0002.633 AMD64
I'm new to Windbg: so the more info, the better
Edit 1
When I don't have symbols set up properly, I get the following:
STACK_TEXT:
00000000`2cd9d8c0 00000000`ffffffff MyDll!Unknown_0xe99+0xe99
00000000`2cd9dbe0 00000000`ffffffff MyDll!Unknown_0x43d+0x43d
00000000`2cd9dd20 00000000`ffffffff MyDll!Unknown_0x51+0x51
00000000`2cd9dd80 00000000`ffffffff system_ni! System.ComponentModel.BackgroundWorker.WorkerThreadStart+0x62
When I set it up to point to my symbol server and turn on !sym noisy, it appears to load symbols properly:
0:000> ld MyDll
DBGHELP: C:\Program Files\Debugging Tools for Windows (x64)\MyDll.dll - file not found
SYMSRV: c:\symbols\MyDll.dll\4F3D6F4B154000\MyDll.dll not found
SYMSRV: http://msdl.microsoft.com/download/symbols/MyDll.dll/4F3D6F4B154000/MyDll.dll not found
SYMSRV: \\mysymbolserver\store\Mydll.dll\4F3D6F4B154000\file.ptr
SYMSRV: MyDll.dl_ from \\mysymbolserver\store: uncompressed
DBGHELP: c:\symbols\MyDll.dll\4F3D6F4B154000\MyDll.dll - OK
DBGENG: c:\symbols\MyDll.dll\4F3D6F4B154000\MyDll.dll - Mapped image memory
SYMSRV: c:\symbols\MyDll.pdb\8AFC2BE7529A41289FA9FBCEDB6836161\Mydll.pdb not found
SYMSRV: http://msdl.microsoft.com/download/symbols/Mydll.pdb/8AFC2BE7529A41289FA9FBCEDB6836161/MyDll.pdb not found
SYMSRV: \\mysymbolserver\store\MyDll.pdb\8AFC2BE7529A41289FA9FBCEDB6836161\file.ptr
SYMSRV: MyDll.pd_ from \\mysymbolserver\store: uncompressed
DBGHELP: MyDll - private symbols & lines
c:\symbols\MyDll.pdb\8AFC2BE7529A41289FA9FBCEDB6836161\MyDll.pdb
Symbols loaded for MyDll
Edit 2
I tried using !name2ee as follows:
0:000> !name2ee MyDll!MyType.DoSomething2
Module: 000007ff004995b8
Assembly: Autodesk.DataManagement.Client.Framework.Vault.dll
<invalid module token>
So, no luck there. But then I almost seemed to get somewhere with this:
0:000> !name2ee MyDll.dll!MyNamespace.MyType
Module: 000007ff004995b8
Assembly: MyDll.dll
Token: 000000000200008c
MethodTable: 000007ff01b2e258
EEClass: 000007ff01b415e0
Name: MyNamespace.MyType
0:000> !dumpmt -md 7ff01b2e258
EEClass: 000007ff01b415e0
Module: 000007ff004995b8
Name: MyNamspace.MyType
mdToken: 000000000200008c
File: C:\Program Files\MyCompany\MyProduct\Bin\MyDll.dll
BaseSize: 0x30
ComponentSize: 0x0
Slots in VTable: 31
Number of IFaces in IFaceMap: 2
--------------------------------------
MethodDesc Table
Entry MethodDesc JIT Name
000007feeb31a2c0 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007feeb3689f0 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007feeb3688c0 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007feeb353440 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01b01300 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01e89140 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01b9c080 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01f45f40 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01a9b358 000007ff01b2e128 NONE MyType.DoSomething3()
000007ff01a9b360 000007ff01b2e130 NONE MyType.DoSomething4()
000007ff01a9b368 000007ff01b2e138 NONE MyType.DoSomething5()
000007ff01e79800 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff020fea80 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01a9b3b0 000007ff01b2e1b0 NONE MyType.DoSomething6()
000007ff01a9b3b8 000007ff01b2e1b8 NONE MyType.DoSomething7()
000007ff01a9b328 000007ff01b2e0f0 NONE MyType..ctor()
000007ff01b01280 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01e7a810 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01e7aac0 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01e83240 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01f19520 000007ff01b2e178 JIT MyType.RunWorkerCompleted(System.Object, System.ComponentModel.RunWorkerCompletedEventArgs)
000007ff01e7ace0 0000000000000000 JIT 0000000000000000 is not a MethodDesc
000007ff01e7b7a0 0000000000000000 JIT 0000000000000000 is not a MethodDesc
000007ff01e7b710 0000000000000000 JIT 0000000000000000 is not a MethodDesc
000007ff01e7d2b0 0000000000000000 JIT 0000000000000000 is not a MethodDesc
000007ff01b015f0 0000000000000000 JIT 0000000000000000 is not a MethodDesc
000007ff01b88ce0 0000000000000000 JIT 0000000000000000 is not a MethodDesc
000007ff01a9b3e0 000007ff01b2e200 NONE MyType.DoSomething8()
000007ff01b921e0 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01b933b0 0000000000000000 NONE 0000000000000000 is not a MethodDesc
000007ff01b93870 0000000000000000 NONE 0000000000000000 is not a MethodDesc
I'm guessing all the missing entries (those listed with "is not a MethodDesc") are due to this not being a full mini-dump. Is that right?
It looks like WinDbg doesn't pick up symbols for your DLL. You can look into that by setting the symbol path and using !sym noisy to troubleshoot as necessary.
I can't say why !ip2md doesn't work in this case, but there are other ways to get the code for DoSomething2. Try !name2ee on the method name, e.g. !name2ee *!TypeName.DoSomething2 or you can get it via the type itself like this !name2ee *!Namespace.TypeName and then !dumpmt -md <method table> on the method table you get from !name2ee.
Once you have the code, the !u command can show you an .NET annotated dump of the assembly code. By using the offset from the exception, you may be able to determine the nature of the NullReferenceException.
You are probably using a stripped PDB file, the default generated for a project in the Release build. All file and line number info is removed from such a file.
Switch to the Release configuration, Project + Properties, Build tab, Advanced, Debug Info = "full".
This is intentional btw, line number info is not very accurate for the Release build. The jitter optimizer moves code around so you'll need to keep in mind that a displayed line number is an approximation.
In a comment above, you mention that the || command yields "User mini dump". In order to properly debug .NET code, you need a full dump, which would indicate "Full memory user mini dump" from the || command. I think this is your problem. Without access to the full loader heaps, it's not possible to map a code address back to a .NET method, so you can't get a stack trace. If you can reproduce this problem, capture a full dump. You can use ADPlus, ProcDump or DebugDiag to capture a dump on crash.

Categories