I'm trying to put together a web-scraping app, using Selenium and .NET Core, but I'm having trouble getting my WebDriver exes to be found.
I have one .csproj that will run the API for the project, which calls out to (amongst others) another .csproj that will handle the webscraping.
All are in a single .sln, and all are running .NET Core 2.1
In the scraping proj, I've nuget-installed Selenium.WebDriver and Selenium.WebDriver.ChromeDriver.
I've created an endpoint in the API, which calls out to the scraping project, and runs a method that attempts to invoke new ChromeDriver(). It doesn't work :( Specifically, I get:
The chromedriver.exe file does not exist in the current directory or in a directory on the PATH environment variable. The driver can be downloaded at ... <url>
Seems fairly clear (although it dissappointingly doesn't tell you what "current directory" means. I'll be submitting a PR for that imminently)
By observing changes during a rebuild, and other research online, I see that:
All the dlls and exes from the nuget packages are stored in the Global Nuget cache, rather than a nuget packages folder in the solution directory.
This appears to be expected behaviour: "Bug" raised in dotnet Std; MSDN migration docs.
The chromedriver.exe appears to get copied to <solutionFolder>\<ScrapingProjectFolder>\bin\Debug\chromeDriver.exe.
I assume that this is what the ChromeDriver Nuget package does; certainly I haven't configured it myself.
This superficially feels like a reasonable thing for that ChromeDriver package to be doing as an attempt at "install this to make new ChromeDriver() JustWork."
Digging into the WebDriver codebase, reveals that the "currentDirectory" that it's looking at is "the location of WebDriver.dll".
In my case, that's "<globalNugetPackagesCache>\selenium.webdriver\3.141.0\lib\netstandard2.0"
It doesn't seem like I should be trying to get the chromedriver.exe to end up in this folder - copying it into a different package's global cache seems wrong? (Do people agree?)
This article seems to have reached broadly the same conclusion and says that the solution is to invoke the driver as:
new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location))
Unfortunately, that path takes me to <solutionFolder>\<APIProjectFolder>\bin\Debug\<ScrapingProjectFolder>.dll, because the dll gets copied over the the API project's folder.
A couple of solutions occur to me, none of which really appeal:
I could install Selenium.WebDriver.ChromeDriver into the API project.
Eww... the API project doesn't know about WebDriver or Selenium, and now the Scraping project doesn't have the driver exe.
I could manually explictly copy the exe into the right place.
Doesn't really feel right, and feels fragile. I suspect this will make deployment painful.
I could manually point the ChromeDriver constructor to a hard-coded path, that I just happen to know contains the current exe.
Seems similar to the above; though not quite as bad.
??? Is there some way to make all the DLLs etc. of a project get compiled into a single common folder? ???
Is there a good, non-hacky way to solve this problem. Which will result in a git repo that JustWorks, and is going to be relatively painless to deploy to a server in the future?
Are any of the things I've described above wrong, or mis-configured?
From what I understand you have an API project that depends on a Scraping project.
Scraping.csproj:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<LangVersion>7.2</LangVersion>
<PublishChromeDriver>true</PublishChromeDriver>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Selenium.WebDriver" Version="3.141.0" />
<PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="2.46.0" />
</ItemGroup>
</Project>
API.csproj:
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<ProjectReference Include="..\Scraping\Scraping.csproj" />
</ItemGroup>
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<LangVersion>7.2</LangVersion>
</PropertyGroup>
</Project>
The trick is adding <PublishChromeDriver>true</PublishChromeDriver> to the transitive project to make it publish the chromedriver when running dotnet publish API.csproj The ChromeDriver package has custom build targets in the NuGet package so it is custom.
You can now use
new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location));
and dotnet run API.csproj
Please correct me if I'm wrong. You have some kind of Class Library that has reference to Selenium and you would like to use ChromeDriver.exe but you are getting an error that it cannot be found under the following location. This is fairly simple.
Currently you are referencing Class Library lets say Foo to API. Your Assembly Location will point to API bin location, whereas chromedriver.exe is located under Class library bin.
If this is the case the only thing you would have to do is copy following chromedriver.exe to final bin directory which is API.
Add following Post Build Event to your API project to copy chromedriver:
<Target Name="PostBuild" AfterTargets="PostBuildEvent">
<Exec Command="copy $(SolutionDir)\ClassLibrary\bin\Debug\netstandard2.0\chromedriver.exe $(TargetDir)" />
</Target>
This will copy your chromedriver.exe to API bin. Later while initializing ChromeDriver use:
var options = new ChromeOptions();
var service = ChromeDriverService.CreateDefaultService(AppDomain.CurrentDomain.BaseDirectory);
WebDriver = new ChromeDriver(service, options);
While AppDomain.CurrentDomain.BaseDirectory will point to your API bin directory.
Related
When I publish my ABP project I get the following error:
C:\Program Files\dotnet\sdk\6.0.100-rc.1.21458.32\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ConflictResolution.targets(112,5): error NETSDK1152: Found multiple publish output files with the same relative path:
D:\Github\volo\abp\lepton-theme\src\Volo.Abp.AspNetCore.Mvc.UI.Theme.Lepton\compilerconfig.json,
D:\Github\volo\abp\bookstore\src\Acme.BookStore.Theme\compilerconfig.json,
D:\Github\volo\abp\lepton-theme\src\Volo.Abp.AspNetCore.Mvc.UI.Theme.Lepton\package.json,
D:\Github\volo\abp\bookstore\src\Acme.BookStore.Web\package.json.
D:\Github\volo\abp\bookstore\src\Acme.BookStore.Web\Acme.BookStore.Web.csproj
Issue:
The issue raises after .NET 6 migration.
There's a new feature that blocks multiple files from being copied to the same target directory with the same file name.
See https://learn.microsoft.com/en-us/dotnet/core/compatibility/sdk/6.0/duplicate-files-in-output
Solution #1 (workaround):
You can add the following build property to all your publishable (*.Web) projects' *.csproj files.
This property will bypass this check and works as previously, in .NET5.
<PropertyGroup>
<ErrorOnDuplicatePublishOutputFiles>false</ErrorOnDuplicatePublishOutputFiles>
</PropertyGroup>
Solution #2:
Exclude the problematic files to be copied to the output folder.
In this example we'll exclude these files: compilerconfig.json and package.json.
Add the following lines to your common.props (located in the root directory of your solution):
<Content Remove="compilerconfig.json;package.json"/>
<None Include="compilerconfig.json;package.json">
<ExcludeFromSingleFile>true</ExcludeFromSingleFile>
<CopyToPublishDirectory>Never</CopyToPublishDirectory>
</None>
The above answers led me to my solution. My case is a self-building Entity Framework library project that was now copying over its appsettings.json when building the website that used it.
My solution was to let it copy to output folder (when I am doing migration actions in VS**) but prevent it from publishing using the "Never" value because it is only ever published as a library under a website or web service.
<ItemGroup>
<Content Include="appsettings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<ExcludeFromSingleFile>true</ExcludeFromSingleFile>
<CopyToPublishDirectory>Never</CopyToPublishDirectory>
</Content>
</ItemGroup>
** My EF library project builds itself according to the pattern in this data-seeding article.
Thus do I eat my cake and keep it.
If you are getting this in a azure devops pipleline you can add the following task to specify the SDK version for your build
- task: UseDotNet#2
displayName: 'Install .Net SDK version'
inputs:
packageType: sdk
version: x.x.xxx //example (3.1.416)
installationPath: $(Agent.ToolsDirectory)/dotnet
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/tool/dotnet-core-tool-installer?view=azure-devops
I ran into this with a Blazor WebAssembly project and an associated integration test project which both had appsettings.json files while I was dotnet publish'ing out via a GitHub action. I found two additional ways that worked for me (along with the accepted answer):
Add <IsPublishable>false</IsPublishable > to the test project
In the dotnet publish commands, specify the .csproj directly via arguments
I ran into this issue with a web application that had a Razor Class Library. The culprit file was LIBMAN.JSON.
Right click on the file and change the properties of the file to:
Build Action: NONE
Copy to Output Directory: DO NOT COPY
Other files that are used for tooling only could possibly be changes the same way.
This is caused by a breaking change in the .NET 6 SDK, and is independent of the .NET version your projects target. For example if you install Visual Studio 2022 it will install the .NET 6 SDK and use that for builds and deploys.
You can force VS to use an older SDK toolchain by generating a global.json file by running dotnet new globaljson in your solution root, then replacing the "version" property value with the desired SDK version (use dotnet --list-sdks to list installed versions).
I guess this means if you have a project dependency A->B where A and B are both executable and have their own appsettings.json, it would be preferable to split project B into B1 as a shell project with the appsettings.json and B2 as a library with all of B's functionality. Then dependencies A->B2 and B1->B2 would avoid the "multiple publish output files" issue.
I have also used compilerconfig.json for compiling scss to css.
And the easiest fix through UI is to:
Open Solution Explorer->compilerconfig.json->right click->properties
and there set:
Build Action: None
Copy to Output Directory: Do not copy
Do this for all compiler.config files (in my case on client project as well as on the server)
The reason behind this is that this compiler config is only used locally in building process but it is not required later on while app is running.
If your projects (All part of the same solution) uses a different version of the same nuget pacage, you will see this error. Now you can either find a workaround as others mentioned in the answers if for some reason you have to keep both versions (which is not a good practice).
Or do the right thing and make sure all project using same version of the package. to do that just open Visual studio's NuGet package manager for solution as shown in the screenshot
A window opens which will have a consolidate tab at the top, click on the consolidate tab. if you have a version conflict, you will be able to see lisr=t of NuGet packages on the left side. If that is the case it means you have conflicts. Click on any package and you will be able to see the list of your solution's projects on the right side just like the following screenshot
in my example (screenshot), I have 2 versions of Microsoft.Net.Sdk.Functions
one with 3.0.13 and 3.0.11.
All you need to do is to select your preferred version and click install and both projects will be updated to the same version.
Push the changes and devops build again and enjoy
I have two projects, API and Hangfire.
The duplication was in publishing hangfire since it uses both API and Hangfire projects and I solved it by removing appsettings files before the publish step.
COPY . .
RUN find ${API} -iname "appsettings*.json" -exec rm {} \;
RUN dotnet publish ${HANGFIRE}/*.csproj --configuration Release --output out --no-restore
I was able to resolve it by setting the Microsoft.NET.ConflictResolution.targets file under the <NETSdkError Condition="'$(_ResolvedFileToPublishContainsDuplicates)' == 'false'" <= this was originally true.
This file is located in "\Program Files\dotnet\sdk\6.0.100\Sdks\Microsoft.NET.Sdk\targets"
The project (netcore3.1) is being packaged with a Windows Application Packaging Project (.wapproj + .appxmanifest) and published for sideloading using:
right click packaging project->Publish->Create App Packages
The error is:
"It is not supported to build or publish a self-contained application without specifying a RuntimeIdentifier, Please either specify a RuntimeIdentifier or set SelfContained to false."
Other answers describe workarounds and methods to get the publishing system to recognise a RuntimeIdentifier, but none explains how to disable self-contained packaging. Surely there is a flag or property that can be set to create a package without the runtime libraries.
Try to play with SelfContained=True or DesktopBridgeSelfContained params of your *.wapproj
<ItemGroup>
<ProjectReference Include="..\WpfApp1\WpfApp.csproj">
<DesktopBridgeSelfContained>False</DesktopBridgeSelfContained>
<DesktopBridgeIdentifiers>win-x64;win-x86</DesktopBridgeIdentifiers>
<RuntimeIdentifier>win10-x64</RuntimeIdentifier>
<Properties>SelfContained=True;RuntimeIdentifier=win-x64;PublishReadyToRun=true</Properties>
<SkipGetTargetFrameworkProperties>True</SkipGetTargetFrameworkProperties>
</ProjectReference>
</ItemGroup>
I'm trying to publish my C# project to an executable in order to distribute it. However, I've referenced 'Interop.IWshRuntimeLibrary' and since including it, my project publishes but then crashes on execution with:
An assembly specified in the application dependencies manifest was not found:
package: 'Interop.IWshRuntimeLibrary', version: '1.0.0.0'
path: 'Interop.IWshRuntimeLibrary.dll'
So far I have tried:
Setting the Copy Local and Embed Interlop Types properties for Interop.IWshRuntimeLibrary to every combination of true/false.
Setting the below tag to true/false in the .csproj file.
<PublishWithAspNetCoreTargetManifest>
Installing the System.Runtime.InteropServices NuGet package.
My .csproj file currently looks like this:
<PropertyGroup>
<OutputType>WinExe</OutputType>
<TargetFramework>netcoreapp3.1</TargetFramework>
<RootNamespace>Project</RootNamespace>
<UseWPF>true</UseWPF>
<UseWindowsForms>true</UseWindowsForms>
</PropertyGroup>
<ItemGroup>
<COMReference Include="IWshRuntimeLibrary.dll">
<Guid>f935dc20-1cf0-11d0-adb9-00c04fd58a0b</Guid>
<VersionMajor>1</VersionMajor>
<VersionMinor>0</VersionMinor>
<WrapperTool>tlbimp</WrapperTool>
<Lcid>0</Lcid>
<Isolated>false</Isolated>
</COMReference>
</ItemGroup>
Other references in the project, like to Newtonsoft work fine. I've consulted every thread I can find across the web pertaining to this. The closest I came to another thread describing my problem was Could not load file or assembly Interop.IWshRuntimeLibrary? but I found no useful info there either.
Ideally, I want to just click publish and publish to a folder on my desktop - preferably as the single .exe but the whole folder is fine if it works. I am unaware if I am perhaps missing a step somewhere as I've never used the publish function before. I'm at a loss for what to try next. Thanks.
How are you publishing at the moment?
You should be able to run the publish command with a flag to tell it to publish as a single file :
dotnet publish -r win-x64 -c Release /p:PublishSingleFile=true
More info here : https://dotnetcoretutorials.com/2019/06/20/publishing-a-single-exe-file-in-net-core-3-0/
But as for your particular issue. When publishing as a single file (Which you may already be doing), there is some level of treeshaking involved to try and limit which dependencies it's publishing. In some cases, if you are referencing a library that is loaded using reflection or similar, then ILLinker doesn't know that it's actually being referenced and used.
To get around this, you can add to your csproj file the following :
<ItemGroup>
<TrimmerRootAssembly Include="Interop.IWshRuntimeLibrary" />
</ItemGroup>
Then publish your project like so :
dotnet publish -r win-x64 -c Release /p:PublishSingleFile=true /p:PublishTrimmed=true
More info on how ILLinker works here : https://dotnetcoretutorials.com/2019/06/27/the-publishtrimmed-flag-with-il-linker/
I'm trying to pack a UnitTest project as a Nuget package and I always get the following warning(s) if I build my project:
The assembly
'content\SpecFlow.MSDependencyInjection.SpecFlowPlugin.dll' is not
inside the 'lib' folder and hence it won't be added as a reference
when the package is installed into a project. Move it into the 'lib'
folder if it needs to be referenced.
My csproj file looks like this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
<Version>0.1.0</Version>
<IsPackable>true</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Http" Version="2.2.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.0.1" />
<PackageReference Include="MSTest.TestAdapter" Version="1.4.0" />
<PackageReference Include="MSTest.TestFramework" Version="1.4.0" />
<PackageReference Include="SpecFlow" Version="3.0.188" />
<PackageReference Include="SpecFlow.MSDependencyInjection.SpecFlowPlugin" Version="1.0.2" />
<PackageReference Include="SpecFlow.MsTest" Version="3.0.188" />
<PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.0.188" />
</ItemGroup>
</Project>
The error still appears if I copy the files into the lib folder of the Nuget package. I'm at a loss what I need to change for this warning to disappear. To be frank I'm not even sure why it appears in the first place because I have a different project that works fine without this error.
Update 1:
After the detailed answer from #zivkan I changed my project structure so it is not a UnitTest project anymore.
Sadly the errors still appear if my project is a class library...
Screenshot with all Nuget-Packages that I need for my project to work
If I only add my own Nuget-Package that consists of two dependencies (Microsoft.Extensions.DependencyInjection and SpecFlow) it still produces this error but the two dependencies in this Nuget-Package don't. To me this seems to be a problem with the Nuget-Packages...
I'm not 100% sure, but my guess is that since with SDK style csproj files, when you build, only your assembly's dll is normally written to the output directory. When you run a non-test netcoreapp, the dotnet cli looks at what project references and nuget references you have, and configures the assembly loader to load from their "original" locations, rather than having all the assemblies copied to your app's bin folder. Perhaps the unit test framework doesn't support loading assemblies in this way and creates Content items out of each dll, which tells the build step to copy the content (in this case dlls) into the output directory (bin\$(Configuration)\$(TargetFramework)). Therefore, when you run unit tests, the unit test framework has all the required assemblies in the single directory, whereas that's normally not true for non-test projects.
Next you need to understand that when NuGet packs a project, it looks for MSBuild items of the type Content, and puts copies of them in the nupkg's content and contentFiles directories. Due to how NuGet works, only dlls in the lib\ or ref\ directories within the nupkg are given to the compiler, therefore any dlls you have a content directory will not be passed to the compiler, so your project that references this nupkg cannot use classes in those dlls. This is not how people usually intend to use NuGet packages and therefore NuGet generates a warning.
So, I believe the reason you're getting this warning is because you're packing a project type that is not intended to be packable. The project type does some uncommon things in order to work, which triggers NuGet warnings because typically when this uncommon thing is done, it's a mistake.
I believe this to be a case of a XY problem. I assume you're packing a unit test project because you want to share some utility code useful for tests, maybe some mocks or object initialisation code. In this case, I recommend you create a new classlib project, put your shared code in there, leaving all your test cases in your netcoreapp test project, even if it's nothing more than a single method call into the classlib. This way you can pack and share the classlib without warnings. Packing a unit test seems unusual and it would be interesting to discuss why you want to do this, what problem do you intend to solve and if packing a test is really the best way to achieve it. Unfortunately Stack Overflow isn't a good place to have discussions and is often actively discouraged.
perhaps you have missed a file, please follow this link for full details : https://learn.microsoft.com/en-us/nuget/create-packages/creating-a-package#Package_Conventions
I'm using Visual Studio Code for a .NET Core Project.
I have an ASP.NET Core Project for which i want to create a separate unit tests project, i created a sub folder and ran
dotnet new xunit
dotnet restore
When i tried to run it from the cli "dotnet build" "dotnet run" it ran successfully, however in visual studio, it says that it can't find the namespace Xunit
This is very strange because Visual Studio code has worked fine for me so far, it never had problem with dependencies, it only has this problem with Xunit.
Anyone familiar with this issue?
I had the same issue. It was solved by typing "Restart Omnisharp" in the Command Palette.
Adding the xunit reference to the root csproj is likely undesirable.
The Issue
The idea of having tests in a separate csproj is that they and their dependencies won't be included in the main project. Adding xunit to the root csproj will however cause the main project to reference xunit. Depending on how the product is bundled, this will cause the xunit dlls and all the dlls it depends on to be included unnecessarily.
Unfortunately, Omnisharp has an issue with nested csproj. It appears the root csproj will claim all source files in all subdirectories, even if there are nested csproj. This causes the missing reference error.
The Solution
Do not nest csproj. Note how Microsoft's xunit setup guide puts the main csproj and its tests csproj into different subdirectories of the sln.
I first ran into this error as well and after restructuring the project to not nest csproj, the error was resolved; with only the tests csproj referencing xunit.
I tried opening the test project directly with vscode (instead of opening the root project that contains the test project in a sub folder) and now vscode recognizes Xunit.
I then opened the root project with vscode again, added Xunit to the root project's csproj file, ran "dotnet restore" in the root project and now Xunit is recognized in the test project.
The thing is that vscode (or is it Omnisharp? i'm not sure) probably looks only at the root csproj file and ignores any csproj file that happens to be in a sub folder.
Prelude
None of the above worked for me.
Turns out I should have checked my notes from before, as this has been a recurring issue here with a project that uses Xunit:
Make sure all installed Xunit.xxxxxxxxx packages have the EXACT SAME version.
The problem occurs while/after having updated my NuGet packages automatically, which will, at the time of writing, install Xunit 2.4.1 (and several other xunit packages at version 2.4.1) plus xunit.runner.visualstudio at version 2.4.3 (!)
While nothing untoward is reported during this NuGet update, the result is a permanently failing build, where Fact and Assert are suddenly unknown, e.g.
Error CS0246: The type or namespace name 'Fact' could not be found (are you missing a using directive or an assembly reference?) Imazen.Test.Webp
The fix
What did work out for me was to go and revert that xunit.runner.visualstudio update, re-installing 2.4.1.
BTW, in Visual Studio, this would then look something like this (after the revert/re-install):
Note that the package manager there is hinting that an update is available and as soon as you apply that 2.4.3 update again, in any way, you're back to square one: a curiously failing build.
The key to the solution is to have all installed xunit packages with the same version. -- if only a few have updates available on NuGet, wait until all xunit packages are available for that same version.
Postscript
Don't know why this is so finicky, as I've only observed this brittle behaviour with xunit.*, but this is what has worked earlier this year (I had forgotten) and now had happen to me again, with the same outcome: the mandatory revert of a NuGet xunit package update.
#areller's solution solved my issue. I just want to expand on his answer to provide some sample code, in case there are other people who doesn't know exactly what to do (especially if you just started on C# like I did).
The Issue
First, I followed instructions here to start writing unit tests for a .NET Core application:
https://learn.microsoft.com/en-gb/dotnet/core/testing/unit-testing-with-dotnet-test
Then I encountered the same issue mentioned by OP.
Solution
First, I tried Claus' solution by restarting OmniSharp, but it doesn't fix the issue.
Then, following #areller's suggestion, I found these lines in Tests.csproj:
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.7.0" />
<PackageReference Include="xunit" Version="2.3.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
<DotNetCliToolReference Include="dotnet-xunit" Version="2.3.1" />
</ItemGroup>
I copied that, and pasted in my root directory's .csproj file, so the root .csproj looks like this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.7.0" />
<PackageReference Include="xunit" Version="2.3.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
<DotNetCliToolReference Include="dotnet-xunit" Version="2.3.1" />
</ItemGroup>
</Project>
After that, VS Code asks me to restore. The VS Code dialog came with a "Restore" button; I clicked on that; several seconds later the issue is gone.
Good luck!
I had the same issue. I installed xunit.extensibility.core (2.4.1) for my .Net 5 solution and it solved the issue. xunit.runner.visualstudio (2.4.3) was already present.