For the local development we have added all the customizations in a separate solution project so that they are taken when the build is done. They are also copied when publishing the project. When doing a sitecore upgrade you would need to modify the files only in this project.
For the higher environments we handled this by having an override folder separate from the vanilla instance files. We modified the CI pipeline so that during deploy it will copy the files from the override folder over the vanilla sitecore files. This would also be needed in case you want to only change the .xml file in CM instance only.