We Keep Coding

VSTS Task that changes project files and checks in the changes

With TFS 2015, we had made a vNext (VSTS) task that would find a selected file, substitute a token (version number is where it started,) wrote out the changes to the file, and checked the file in with a comment as to the nature of the change. It did this all through automation during the build process.
We are about to upgrade to DevOps 2020 and the TFS management tools from 2015 were deprecated, which in itself would be fine, however, we still need to automate these file changes during the build process, including the check-ins noting the nature of the change.
The old task fails miserably ported over directly. I have re-written the process as a C# console app project and planned to call it during a PowerShell script, but I am encountering a number of road blocks to this plan.
What I have done so far.
I have written a task.json for the VSTS task that accepts parameters that it passes to a PowerShell script.
I have written a PowerShell script that call a C# console application to both locate and change the tokens in the file(s) as specified by the task. After changing the file content, it overwrites the original file.
I appear to have two issues that, as yet, I am unable to solve.
I am expecting (if task parameters dictate,) to alter the pipeline environment variable $env:BUILD_BUILDNUMBER to a new value, from the C# code. I am using the following C# command using the third parameter expecting that this will allow the pipeline to see the changes it has made to the variable: Environment.SetEnvironmentVariable("BUILD_BUILDNUMBER", buildNumber, EnvironmentVariableTarget.Machine ); (I have also tried without the parameter and also User and Process, to no avail.) The variable will not 'set' for the pipeline to see outside of the console app.
I need to check the changes made in step 2 from the C# code, back into the code repository with a short comment. The initial 'Get' that the pipeline does calls tf, but I have not had the same success. I have found that if I call VS2019's copy of tf.exe or the agents copy in 'externals' on the build machine, whether from C# or from the later PowerShell script, I get "##[error]Unable to determine the workspace. You may be able to correct this by running 'tf workspaces /collection:TeamProjectCollectionUrl'" Needless to say, this later instruction to run tf workspaces does not help out.
I am hoping these things have simple solutions. I have searched for, but not found an API call to DevOps 2020 that will check in the code. Perhaps that does not exist. As for the Environment Variable, I am at a little bit of mystery as to why the EnvironmentVariableTarget.Machine is not working. I suspect it has something to do with the ##vso[] method not being called but I am unsure of how to pass out my findings from the console app (which also needs to return 0 for success, else failure,) to the PowerShell script that can change the variable more easily.
If there are any bright ideas out there about this, I would really appreciate the insight you might lend. I have been at this for a while and I am not sure what else to consider to make this work.

Using TFVC rather than Git, and a multitude of other little legacy technology items, makes working with the new tools a bit harder to figure out as it was in this case.
So my first problem item, updating the environment Build Number in the build tasks before the build starts. I was able to set the variable from my C# code, by opening a PowerShell process that takes an argument to update the variable with the ##vso[Build.UpdateBuildNumber] function in PowerShell. The C# code to do this looks like the following:
var ps = #$"C:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.exe";
var script = $#"SetEnvironmentBuildNumber.ps1";
if (File.Exists( script )){
if ( File.Exists( ps ) )
{
await Task.Run( () =>
{
var scriptWithArguments = string.Concat( Path.Combine( Directory.GetCurrentDirectory(), script ), " -newBuildNumber ", newBuildNumber);
Process.Start( ps, scriptWithArguments );
} );
}
else
{
Console.WriteLine( $"PowerShell File Not found at: {ps}" );
}
}
else
{
Console.WriteLine( $"The current directory is: {Directory.GetCurrentDirectory()}");
Console.WriteLine($"Script File Not found at: {script}");
}
The PowerShell script in SetEnvronemntBuildNumber.ps1 looks like this:
[CmdletBinding()]
param(
[string][Parameter(Mandatory=$true)]$newBuildNumber
)
Write-Host( "Setting the new build number - " + $newBuildNumber )
Write-Host( "##vso[Build.UpdateBuildNumber]$newBuildNumber" )
As it turns out, this exercise helped me understand the Undetermined Workspace, my problem #2.
If you look at the if statements of my C# code, you'll note a check to see just what directory is the current working directory. Finding out that my new task changed the current working directory to the location of my Task and not the pipeline's working directory gave me reason to pause.
Even though, I fall out of the C# code, back into my PowerShell script, I am not looking at the solution's directory, I am looking at the Task directory. If I call tf from this view, there is no source code, and hence no way to determine the WorkSpace.
The solution was simple, there is a pipeline System environment variable that will get me back home, so to speak. Calling the PS Set-Location as below, just before I call tf vc checkin... allowed tf to find the Workspace and save my changes.
Set-Location $env:System_DefaultWorkingDirectory
This was all it took, all these tweaks were innocent enough, but very troublesome to diagnose without any formal training in the ways of the DevOps pipeline. I look forward to more 'fun' with DevOps, when we try to figure out how to get to Git from TFVC... I am sure not all our legacy will see such significant changes.
Thanks for the suggestions, and I hope this can help some other poor soul trapped in a legacy world and trying to get out of the hole.

Executing Python Script in C# or Unity

I used Keras in Python to design a neural network calculating something like a noise-reducing-function. It works pretty good so far, and now I want to use this network to clean the data inside a Unity-Project of mine.
I would not have thought that this could be so difficult. I could only find one python interpreter in the asset store, which does not support external python librarys. IronPython is not an option either, because I need to include the Keras Packages.
I found a KerasSharp Project on GitHub, but there is no documentation on how to load an already trained network, and training it at the beginning is not an option. Furthermore it seems like there is no one working on the project anymore, due to the commit history and unanswered questions. Accessing the script via network APIs is probably not an good option either, due to the latency. I need the calculation for every frame.
So my question is: Is there any way I can load a Keras/Tensorflow model in C# or Unity
OR
Can I somehow access the python script which is calculating the noise-reducing-function using the Keras model?

If your situation allows for you to start the python script after Unity, you can try starting the python script as a subprocess as described here:
http://answers.unity.com/answers/14156/view.html
If you do not require the other process to be running before the Unity
one, you could have your Unity project launch that via Process and
then redirect stdin/out to streams and communicate through these.
Example:
Process otherProcess = new Process ();
otherProcess.StartInfo.FileName = path;
otherProcess.StartInfo.CreateNoWindow = true;
otherProcess.StartInfo.UseShellExecute = false;
otherProcess.StartInfo.RedirectStandardInput = true;
otherProcess.StartInfo.RedirectStandardOutput = true;
// Now communicate via streams
// otherProcess.StandardOutput
// and
// otherProcess.StandardInput
It is also possible that grabbing a running process by name or pid and
then setting the forwarding would work, but I've not tested this and
it does seem rather doubtful.
This setup would require that your python script be able to take in data from standard in, and output its results over standard out.

Issue calling a python script from an iron python script (which is called from c# script engine)

I'm calling an iron python script from c# using
ScriptRuntimeSetup setup = Python.CreateRuntimeSetup(null);
ScriptRuntime runtime = new ScriptRuntime(setup);
ScriptEngine engine = Python.GetEngine(runtime);
ScriptSource source = engine.CreateScriptSourceFromFile("C:/KitGenerator/KitGenerator/LoadKitsFromDatabase.py");
ScriptScope scope = engine.CreateScope();
source.Execute(scope);
This works fine and using print outs i've proven that the code runs correctly.
However inside the iron python script is a call to another python script
os.system("ipython C:\KitGenerator\Kit-Generator\GenerateKitImages.py")
(i have tried it with just (ipython "GenerateKitImages.py")
THE ISSUE:
When i run the first iron python script using Ipy.exe both run perfectly fine and do as expected.
HOWEVER when i run the first iron python script using the c# script engine it only runs the first script and the second script is never called.
i'm lost on this one. I can confirm it is nothing to do with : permissions, dependencies.
a gold medal and pizza for the man who can fix this problem.

First, the os.system call should use a raw (r"") string:
os.system(r"ipython C:\KitGenerator\Kit-Generator\GenerateKitImages.py")
Otherwise the backslashes will cause issues.
However, since you said it works from ipy.exe, my guess is that ipython is on your PATH in the first case and not the second case. You can set the PATH environment variable from C#, include the complete path to ipython in the os.system call, or ensure that the PATH is set before calling the C# program, by changing it globally.

How to launch a JVM from C#

I would like to launch a java application from C# but am unsure of the proper way to do it. I can do it from a bat file:
java -cp ".;other_classes_location" classname
How does this get turned into C#?

What you have posted is a command line (likely executed in a command shell)
you cannot turn this into C# code
I guess you are looking for a way to execute that command using C#. In this case the answer is the below code
System.Diagnostics.Process.Start("yourPath\java.exe", "Command Line Arguments");

Use Process.Start.
Examples here.

C# Application Calling Powershell Script Issues

I have a C# Winforms application which is calling a simple powershell script using the following method:
Process process = new Process();
process.StartInfo.FileName = #"powershell.exe";
process.StartInfo.Arguments = String.Format("-noexit \"C:\\Develop\\{1}\"", scriptName);
process.Start();
The powershell script simply reads a registry key and outputs the subkeys.
$items = get-childitem -literalPath hklm:\software
foreach($item in $items)
{
Write-Host $item
}
The problem I have is that when I run the script from the C# application I get one set of results, but when I run the script standalone (from the powershell command line) I get a different set of results entirely.
The results from running from the c# app are:
HKEY_LOCAL_MACHINE\software\Adobe
HKEY_LOCAL_MACHINE\software\Business Objects
HKEY_LOCAL_MACHINE\software\Helios
HKEY_LOCAL_MACHINE\software\InstallShield
HKEY_LOCAL_MACHINE\software\Macrovision
HKEY_LOCAL_MACHINE\software\Microsoft
HKEY_LOCAL_MACHINE\software\MozillaPlugins
HKEY_LOCAL_MACHINE\software\ODBC
HKEY_LOCAL_MACHINE\software\Classes
HKEY_LOCAL_MACHINE\software\Clients
HKEY_LOCAL_MACHINE\software\Policies
HKEY_LOCAL_MACHINE\software\RegisteredApplications
PS C:\Develop\RnD\SiriusPatcher\Sirius.Patcher.UI\bin\Debug>
When run from the powershell command line I get:
PS M:\> C:\Develop\RegistryAccess.ps1
HKEY_LOCAL_MACHINE\software\ATI Technologies
HKEY_LOCAL_MACHINE\software\Classes
HKEY_LOCAL_MACHINE\software\Clients
HKEY_LOCAL_MACHINE\software\Equiniti
HKEY_LOCAL_MACHINE\software\Microsoft
HKEY_LOCAL_MACHINE\software\ODBC
HKEY_LOCAL_MACHINE\software\Policies
HKEY_LOCAL_MACHINE\software\RegisteredApplications
HKEY_LOCAL_MACHINE\software\Wow6432Node
PS M:\>
The second set of results match what I have in the registry, but the first set of results (which came from the c# app) don't.
Any help or pointers would be greatly apreciated :)
Ben

This is actually not a particularly good way to embed PowerShell within a C# api. There are APIs for that.
You can find an example of them on MSDN, but in your case the could would look something like
PowerShell.Create().AddScript("get-childitem -literalPath hklm:\software").Invoke()
You can also check out this blog post, which will show you how to dot source inside of the API and how to use this API to get at the other data streams in PowerShell.
Hope this helps

Are you running a 64bit version of Windows by chance? It could be a difference in how the two "hives" are being shown. Try forcing your C# app to compile to x86/x64 instead of "Any" in the Project properties. See if that makes any difference.
Also, your command line syntax is a little strange, see the following thread for better details, but you may want to adjust your syntax:
String cmd = "-Command "& { . \"" + scriptName + "\" }";
Process process = new Process();
process.StartInfo.FileName = #"powershell.exe";
process.StartInfo.Arguments = cmd;
process.Start();
Calling a specific PowerShell function from the command line

I did look at alternative methods for calling Powershell and came across that API.
Am I right in thinking though that they rely on a Microsoft SDK??
I'm not really a fan of dependencies on external SDKs. I work in a rather large company and ensuring that the SDK is installed on all of the developers machines would be a nightmare.
If I'm wrong in my thinking, I am open to a better way of calling Powershell. I didn't particularly like calling the script as a separate process and would like the ability to have values returned from the script.
Ben