We have an MVC 5 application that is based on modules (a groupe of functionalities). We also have teams dedicated to each module. So my question is: is it possible to do some sort of partial deploy where we only publish the changes we need?
Maybe is better for me to explain how we are working today:
Right now, when each member of any team commits code we run the CI build and the unit tests and the site gets deployed to a test server where the dev that made the commit tests it's own code and once they are happy with that we deploy that version to testing. But that brings a problem as THAT developer was happy with HIS functionality but other dev may not be at the same stage with other part of the development in other module, so that unwanted change gets deployed to testing.
When I was using ASP.Net Webforms with other project we usually deployed at a Page/Code behind level and it was just fine but now everything (Controllers, helpers, etc.) gets bundled into one assembly and also if we change something on the Bin folder the AppDomain gets recycled so the app basically restarts.
Of course that we are taking care of this testing deploy issues, just by communicating with the rest of the teams before publishing something to test but I was wondering if there is something more that we could do, that's why I thought about a partial deploy.
If you have different teams working on the same project, you should probably take a look on branch strategies. In your particular scenario, branch per feature (or per module) might work fine.
In your case CI has to be enabled on "main" branch. When a team starts a feature, they create a branch. During development they commit to that branch (you can enable automated test execution and deploy to dev on "feature" branches as well). After finishing a feature, they merge changes into "main", CI create a build and deploy it to server. However there is one drawback, possible conflicts during merging, so you need to make sure that your teams commit to main branch as often as possible to minimize these problems.
Related
How would I go about releasing or deploying (do those mean the same thing?) my WPF application (.Net Framework 4.7.2) through Azure Pipelines?
I'm a little confused on what to use. There's all these terminologies like artifacts, release, publish, staging. I'm not sure what task I need to use.
Let's go over a typical scenario so you best understand the terminology. This, of course, is just an example.
You have multiple git repositories in Azure DevOps Repos. One of those git repositories is a common library which is shared across multiple projects in your company. Another one is the companies flagship application, written in WPF, and used internally by 100 employees. Lastly, there is a web portal for external clients to submit orders. Each of these git repositories has branch policies in place to prevent checking in directly to the master branch (a pull request is required).
Development Team A works on the core functionality of your technology team, they update the common libraries. As part of the build pipeline, the library is compiled and a NuGet package is generated. The pipeline then publishes the package to your private Azure DevOps Artifacts feed, so that the other development teams can use the package.
Development Team B works on the flagship WPF application. As part of the build pipeline, the pipeline compiles the application. It creates a build artifact of the bin directory. Another stage in the pipeline takes uses the artifact and publishes it to the test environment, and sets a flag on the database that the current test version is the version that was just published. When the development team launches the test application, it is automatically updated because the version was incremented in the database.
Development Team C works on the web application. As part of the build pipeline, the pipeline compiles the application. It deploys the web application to the test web server. The test web application is available immediately for the development team to test.
At a later time, when the iteration cycle is completed, there is a code freeze. The code freeze involves the DevOps team manually running the build pipelines. Because the pipelines were manually run, the first pipeline will deploy the WPF application to the staging environment, and the version number is incremented in the database. The second pipeline deploys the web application to a green-blue server.
The development teams do a final round of testing to ensure there are no bugs which cause a stoppage. Once they are good to go, the WPF application is released from the staging environment by having it pushed out to all of the employee desktops. The release process for the web application involves flipping a switch so that the green server (updated web application) goes live, the network switch points to the green server, then the blue server (older web application) goes down.
If any major issues arise, things can be rolled back by flipping the network switch from the green server back to the blue, and deploying the older version of the WPF application to the employee desktops, etc.
This is just an example, it is not the only way to do this of course.
The question in short form and then the explanation
We want to create patches and include only files which have changed in the build due to some bugfixes for a dotnet application. The patches should get automatically built in the Continuous Integration process involving SVN, Cruisecontrol.net and msbuild.
We have a scenario here:
We want to maintain a .net application which runs on remote servers using continuous integration. The source code is in a SVN and has 3 different repositories for DEV, QA and PROD.
Our developers do new bug fixes almost everyday and merge the changes into the dev repository after their initial testing and satisfaction.
The code after a certain problem is solved or a feature has been added is then merged into the QA repository.
The QA code is built and tested on QA machines manually.
After the QA testing we merge it into PROD. With it the QA also makes new patches for the files which have to be replaced or changed manually. Then the patches are deployed on a staging server. On which it is tested until perfect and then the patches are deployed on actual remote servers.
In search of continuous integration we are now trying to use a mix of CruiseControl.net and msbuild to do the process. The process is good until the stage where we have to generate patches from the QA builds automatically. After the patches are generated we will put them on a ftp server and from their they will be downloaded to the staging server to be tested.
The problem i.e. the generation of patches from the new build has a few aspects. The solution file for the application has many projects and the dlls are copied using postbuild events to the startup app bin folder. So we have a specific directory structure in the actual application which itself is a combination of 6 solutions which are more or less independent of each other.
The way we are trying to create the patches is we are searching the logs of svn to find which files have changed. Then we are parsing it finding the project name. Then we are copying all the files from the bin directory of that project to the patch folder in the specific manner in which the release has them by using a mapping file which has all the files of the application in it.
So can anyone please suggest a much better or easier way to make a patch provided we have svn and the cruisecontrol.net. Or any other opensource tool to do that.
hope the problem is clear
This whole process, in general, goes against established best practices. This is not necessarily a bad thing if you have good reasons for it, but I don't see them here.
In essence, you are not using QA and DEV environments to secure the stability of production. Worse, you use different source trees to build code for them. This introduces new points of possible failures into the deployment process.
A standard way of approaching this would be to have a single SVN code tree - tag a version when it is released to QA (using already built binaries!), possibly tag it again when releasing to PROD. Don't re-build the binaries, use the ones that you actually tested!
If your Msbuild task is performing a build instead of a rebuild then the date/time of unaffected dlls will not have their modified date changed.
I would suggest this for the following reasons:
Msbuild will update the modified date for any assembly which has been affected by a change - I.e. an interface change?
the assembly is what you want to deploy, so check this rather than the source for modifications. Otherwise you need to know the build process (won't change) - I.e. Souce file locations, references etc.
Your deployment would just include the dlls from the build directories where the modified date >= BuildDate.
I agree with skolima about the recommanded build & patch process. You should create your patches from your initial tags plus the modifications and then create a new version which have to be deployed in all environments.
In my company, we are using this method :
Each successfull build are automatically tagged by our CI Server
When a patch is needed for a specific version, the programmers copy
& check-out the tagged version and apply fixes on it
Then we have a specific "Patch" build on our CI Server which do
exactly the same thing as a normal build with a "Patch" flag and point to the patched sources
The deployment target is the same, the build process is the same, only sources changes.
The plus is the patches have their own build history on the CI because they are builded separately but are treated as normal builds.
Anyway, if you want to automate a patching processes between two repositories via your CI, ihmo you have to create specific MSBuild tasks to do this. You can either try to merge the changes between them or check the SVN diff & patch commands.
I'm looking at migrating from TFS (Team Foundation Server) to Git, but can't find anything matching TFS' support for gated check-ins (also called pre-tested or delayed commits).
Atlassian Bamboo has no support for gated check-ins. TeamCity does support it ("delayed commits" using their terminology), but not for Git. Using Jenkins by itself or Jenkins+Gerrit has huge drawbacks and doesn't come close to the gated check-in functionality in TFS. (Drawbacks explained by the creator of Jenkins himself in this video: http://www.youtube.com/watch?v=LvCVw5gnAo0)
Git is very popular (for good reason), so how are people solving this problem? What is currently the best solution?
We have just started using git and have implemented pretested commits using workflows (I finished testing this just today).
basically each dev has a personal repository which they have read/write access. The build server TeamCity in our case, builds using these personal repositories, and then if successful pushes the changes to the 'green' repository. Devs have no write access to 'green', only TeamCity build agents can write to that, but devs pull common updates from 'green'.
So dev pulls from 'green', pushes to personal, TeamCity builds from personal, pushes to green.
This blog post shows the basic model we are using, with GitHub forks for the personal repositories (using forks means that the number of repositories doesn't get out of hand and end up costing more, and means that the developers can manage the personal builds, as they can fork and then create the team city build jobs to get their code pushed to 'green'):
This is more work to set up in TeamCity as each developer has to have their own build configuration. Which actually has to be 2 configurations as TeamCity seems to execute all build steps (including the final 'push to green' step) even if the previous build steps fail (like the tests :)), which meant that we had to have a personal build for the developer, then a another build config which was dependent on that, which would just do the push assuming the build worked.
Check out Verigreen - A lightweight, server side gated check-in system. It verifies each commit before it finds its way into the branches the system protects. Verigreen will not allow any failed CI commit to break the integration, release, or any branch that need be protected.
Moreover – it's a free, open-source project.
How it works:
Verigreen intercepts check-ins and runs verification in an ad-hoc branch - so that in case of failed commit, only the relevant developer is affected.
A pre-receive hook intercepts and creates an ad-hoc branch of the code.
Verification is run via a Jenkins job. The verification job content is fully configurable.
The verified code is merged back into the protected branch whereas a failed commit is blocked with a notification sent to the developer.
Decisions are made based on the following flow:
For more information, please see the wiki or Verigreen.io site
I think that after October 23, 2013 the answer can be - Automatic Merge in TeamCity.
git has different philosophy - commits are easy, commit as you wish. If something wrong, you can change it later. And merges are easy. So, you could organize a appropriate workflow, e.g. developer(s) could commit in a separate branch(es). When a branch is tested, it could be merged into a main branch.
Why not use TFS as the central repository and make use of GIT as a local DVCS solution?
This would allow you to build and commit locally and then push what you want to the TFS server and do a gated build.
Sometimes it is good to have the best of both worlds...
With VS Team Services (fka Visual Studio Online) and TFS 2015 or newer you can use branch policies requiring a passing build for a pull request as a gated checkin workflow with Git.
In my team we create assemblies to attach to extensible released software created and published elsewhere in my company. These assemblies are often specific for an individual client, though some are reused. I want to introduce a couple of standards into this environment - version numbers and installers.
Currently, many assemblies go to clients without adequate versioning. I want to institute automated version number updates so when a client has a problem we can be sure which source code was used in their software.
Currently, assemblies are installed by the individual copying them manually to the correct path and performing any necessary registration. I want to force people to use an installer package so the path and registration is handled automatically.
I could implement the first step by getting people to use:
[assembly: AssemblyVersion("1.0.*")]
But I'd prefer to update the AssemblyFileVersion rather than the AssemblyVersion. This is because I understand that advancing AssemblyVersion combined with our manual installation can lead to multiple versions of an assembly being registered. AssemblyFileVersion doesn't update automatically, and I'm wary of a solution that requires developers install 3rd party tools. If we had a proper installation process, the problem would multiple versions would go away.
For the second step, if I use a Visual Studio setup project then adding the assembly causes it to try to add other assemblies from the original published software, which I don't want. I assume I can create this as a patch somehow, but I've not worked that out yet. Of course, an installer will require reliable version numbers or things will go badly.
It seems clear having written this that I need to advance both issues simultaneously, but I'd really rather approach one at a time.
Any thoughts for the best way to get over these two issues?
I don't have nearly enough information to point you to a solution. What are you using to build your application and installers? Desktop F5 build? Team Foundation Server? Cruise Control?
Things to realize:
1) Visual Studio Deployment Projects suck. Yes, I'll stick by that comment. In your case, the dependency scanning problem you have is unfixable. Even if you right click | exclude the dependency it could scan a new dependency at build time. We even wrote visual studio automation to open the project, right click | exclude everything and then save it on the build machine to avoid this problem. Trust me, it's a horrible road to go down. Even Microsoft knows it sucks and that's why it won't be in the next release of Visual Studio anyways. Use other tools such as Windows Installer XML or InstallShield Limited Edition or Professional.
2) You must update AssemblyFileVersion. This is such a core/foundational tenant of Change Management and it's critical in getting Windows Installer upgrades and patches to work. AssemblyVersion can be changed at your discretion and is only applicable to Strong Naming and IoC scenarios such as Prism where you write rules on what constitutes a valid class for injection.
3) 1.0.* isn't what you want. You want a system that increments your version and passes it into your build automation. What you use will depend on what you are using for build automation. I use Team Foundation Server and a project in CodePlex to do my versioining.
4) You should never be building on a developers machine. You should always be using a clean build machine with automated scripts and not F5.
If these are released applications, then the installer method is fine. If you are adding libraries through this method, and not necessarily the actual application, then something like NuGet (package manager) is an option. NuGet itself is a bit infant and needs to grow up a bit, but I think it should fit your basic scenario.
If you have published software, a bootstrap on the client that calls for updates and then runs the update installer is a good pattern.
The basic answer is you have options, depending what bits you are employing and should take advantage of the one(s) that fit your needs.
I have developed a web application also that has been deployed in many client area.
I have created a installer to install this application at client area for first time.
If i done any bug fix or add any new functionality to this site, should i create new installer for site? or How to update the existing site without creating new installer with latest bug fix?
Or, Is there any other option to update existing application?
Or, we can create a installer only install the latest updates?
Please guide me
You can start your journey using the built in web installer (part of Visual Studio).
Important thing is it will only install to a Virtual Directory of an existing web site.
If virtual directories are fine, you then need to worry about bug fixing and updates.
here is a solution...
If you are at the client or have easy access to the web sites via FTP.
You can map drives on your development / test environment to the production FTP sites, then just create a batch to copy stuff over.... best part is - if you're hosting on IIS, if you make any changes to the web.config or any assembly in \Bin you'll trigger an IIS Reset, so this means you shouldn't need to have remote access to their servers.
Make sure that batch updates all web applications in the farm.
You can also fully rely on the application installer - IF YOU'VE FULLY TESTED THAT IT DOES INFACT UPDATE YOUR FILES, LIKE IT SAYS IT SHOULD. Don't just assume it will, test that installer, and retest it... because I have had lots of problems with upgrading a web site, and you can't expect your client to uninstall, reinstall every time.
I wouldn't worry about creating a "patch installer" Web apps are relatively small anyways - and copying unchanged files won't make a huge differance.
Just be sure when you create the installer, you add project outputs to the installer file... and work with project outputs, rather than having to copy stuff in manually to the installer every time you make a change.
Hope this is at least a good starting point.