We Keep Coding

How to calculate the "training dataset error" in Encog Machine Learning Framework

I am using CalculateGradients() (source) on a test dataset (30% of my data). See code below.
The goal is to get the error on the testingDataSet, using the parameters calculated by trainingDataSet (70% of my data).
My question: is CalculateGradients() independent? (thus just allowing me to calculate the error as intended), or is it adding another iteration to the shared network thus spoiling the isolation of the "training dataset"?
Source code for Iteration() here: source.
Code:
public void Train(BasicNetwork network, IMLDataSet trainingDataSet, IMLDataSet testingDataSet)
{
var trainPropagation = new ResilientPropagation(network, trainingDataSet);
var testPropagation = new ResilientPropagation(network, testingDataSet);
var epoch = 1;
do
{
trainPropagation.Iteration();
// !!! Trying to calculate the error on `testingDataSet` using the parameters from `trainPropagation`
testPropagation.CalculateGradients();
Console.WriteLine($"Epoch #{epoch}, Train error: {trainPropagation.Error}, Test error: {testPropagation.Error}" );
if (double.IsNaN(trainPropagation.Error))
throw new Exception("Train error was NaN");
epoch++;
} while (trainPropagation.Error > _maxError);
}

Using GDAL.OpenEx() in C# with CSV file

I'm trying to read all fields in a CSV file using GDAL v2.1.3
Using this command it is working:
ogrinfo -ro -al -so test.csv -oo X_POSSIBLE_NAMES=Lon* -oo Y_POSSIBLE_NAMES=Lat* -oo KEEP_GEOM_COLUMNS=NO
Now I need to do this in my C# application.
Because I need to send -oo (open options) I cannot use Ogr.Open(), but I should use Gdal.OpenEx(), right?
OpenEx returns a Dataset, which I somehow need to convert to an ogr datasource so I can use GetLayerByIndex(), GetFieldCount() and GetFieldDefn() but I can't get it to work.
Looking at numerous Python example it seems in Python no conversion is needed.
This is my C# code:
var openOptions = new[]
{
"X_POSSIBLE_NAMES", "Lon*",
"Y_POSSIBLE_NAMES", "Lat*",
"KEEP_GEOM_COLUMNS", "NO"
};
using (var ds = Gdal.OpenEx(input, 4, null, openOptions, null))
{
if (ds == null)
{
throw new Exception("Can't open OGR dataset: " + input);
}
// var layer = ds.GetLayerByIndex(0); <----- COMPILE ERROR
}
Any help is much appreciated.

I am no swig expert at all, but it looks like the vector (OGR) part of the Dataset is specifically NOT defined for C# in the current GDAL code

I was facing similar issues with C++, trying to switch from ogr2ogr CLI to GDAL C++. Maybe this will help someone.
I had to format my options array differently to get it to work :
const char *optionsArray[5] = {
"X_POSSIBLE_NAMES=lon*",
"Y_POSSIBLE_NAMES=lat*",
"Z_POSSIBLE_NAMES=alt*",
"KEEP_GEOM_COLUMNS=NO",
NULL
};
In C++ I have to add the ending NULL as stated by GDALOpenEx documentation.

Delete Record from C# using ManagedInterop

I am working with a C# project that uses the dll, Microsoft.Dynamics.AX.ManagedInterop to work with an AX 2012 environment. From within the code, I need to find a SalesQuotationLine based on specific criteria and delete it. So far, I can get the record I need but I am unable to delete it because I am not using a TTSBEGIN/TTSCOMMIT statement and I am not using FORUPDATE. This is my code:
DictTable dictTable = new DictTable(Global.tableName2Id("SalesQuotationLine"));
int quotationIdFieldId = (int)dictTable.Call("fieldName2Id", "QuotationId");
int bdcParentRecIdFieldId = (int)dictTable.Call("fieldName2Id", "BDCParentRecId");
var query = new Query();
var datasource = query.addDataSource(Global.tableName2Id("SalesQuotationLine"));
var queryRange1 = datasource.addRange(quotationIdFieldId);
queryRange1.value = "=" + line.QuotationId;
QueryRun queryRun = new QueryRun(query as object);
while (queryRun.next())
{
var result = queryRun.get(Global.tableName2Id("SalesQuotationLine"));
result.Delete();
}
I also looked at the code here, http://msdn.microsoft.com/en-us/library/cc197113.aspx but I found that I cannot use it since the code I am working with does not use the .NET Business Connector (I am not sure when one dll should be used over the other).

Use TTSBegin() and TTSCommit() methods on Microsoft.Dynamics.AX.ManagedInterop.RuntimeContext.Current. The forUpdate flag can be set by QueryBuildDataSource's update().
It may be easier (and better for maintenance) to write it in an X++ method just call the method from C#.

How to get a list of defects in QC11.0 via C# OTA using BugFilter

I have successfully connected to QC using VBscript via the OTA interface. In VbScript I had the following code to filter out defects and get load them in a list.
BugFilter.Filter("BG_STATUS") = "Not Canceled and NOT Closed"
BugFilter.Filter("BG_PROJECT") = "Business*"
Set BugList = BugFilter.NewList()
The above worked flawlessly in Vbscript.
In C#.NET (4.0), I am able to connect to QC successfully but when I try to apply the filter , it give me a error..
TDConnection qcc = new TDConnection();
qcc.InitConnectionEx(sr);
qcc.ConnectProjectEx("XXXX", "------", "----", "-----");
if (qcc.Connected)
{
Console.WriteLine("connected");
BugFactory bf = (BugFactory)qcc.BugFactory;
bf.Filter["BG_STATUS"] = "Not Canceled and NOT Closed";
bf.Filter["BG_PROJECT"] = "Business*";
List bugs = (List)bf.NewList(bf.Filter);
on the last line of code , it gives me the following error "Could not convert argument 0 for call to NewList."
I am relative new to C#, Can anybody help me here?

Try bg.Filter.text()
You'd need to check the method, 'cause I do that in java. But there is a method by that name. How I normally do that is like this:
List bugs = (List)bg.NewList();

I usually pass a string into the bug factory by using the .Text property of the Filter object rather than the filter object itself.
For example, I've had success with handling the filtering like this:
var tdFilter = (TDFilter)bf_filter;
tdFilter["BG_STATUS"] = "Not Canceled and NOT Closed";
tdFilter["BG_PROJECT"] = "Business*";
var bugs = bf.NewList(tdFilter.Text);

ImportXmlWithProgress not updating result attribute of importjob

I tried to write some code to import a large customization containing 50+ entities. I used the microsoft article 'ImportXmlWithProgress Message (CrmService) as a bases, but was not getting the output I expected.
The 'job.data' in the following code was not changing from the original parameterxml data. So this implies to me that the import was not sucessful. I imported the same compressed importexportxml using the microsoft web ui, and it worked fine. So I'm wondering why my job.data is not being updated with 'result' attributes for each entity that is imported.
Below is my method to import.
private void ImportEntitySchema()
{
const string parameterXml = #"<importexportxml>
<entities>
{0}
</entities>
<nodes/>
<securityroles/>
<settings/>
<workflows/>
</importexportxml>";
var importRequest = new ImportCompressedXmlWithProgressRequest
{
ImportJobId = Guid.NewGuid(),
CompressedCustomizationXml = GetCompressedCustomizationXmlFromEmbeddedResource(),
ParameterXml = string.Format(parameterXml, string.Join("\n", _entitySchemaEntityNames.Select(item => string.Format("<entity>{0}</entity>", item)).ToArray()))
};
try
{
_crmService.Execute(importRequest);
}
catch (Exception e)
{
//Error resulted from import request
}
// Retrieve the results of the import.
XmlNode node;
do
{
Thread.Sleep(2000);
var job = (importjob)_crmService.Retrieve(EntityName.importjob.ToString(), importRequest.ImportJobId, new AllColumns());
var data = new XmlDocument();
data.LoadXml(job.data);
node = data.SelectSingleNode("importexportxml/entities/entity/#result");
} while (node == null);
//code below here never gets executed because the loop continues infinitely
}
I've been looking, but haven't found any/many [useful] examples on the net of the ImportXmlWithProgress being used. Hopefully someone has used it and has an idea of how to get it working.

I remember having trouble with this message, I just can't remember exactly what the trouble was. How big is your import file? We also brewed an import utility for importing our customizations and I use the ImportCompressedAllXmlRequest synchronously (no timeout) on a BackgroundWorker thread. For large amounts of customizations you may have to look at: http://support.microsoft.com/kb/918609. We typically split up our customizations into a bunch of small imports to avoid this.
Should the XPath be "importexportxml/entities/entity[#result]"?