Visual Studio: preventing unecessary recompiles of static libs
Pages: 12
MSVC Version 16.11.0 Preview 4.0, Windows 10.
I have two MSVC C++ apps which share a number of static libraries. When I switch from one solution to the other and try to run the app, MSVC always recompiles the static libraries. To try to stop this, I recreated the .sln and .vcxproj files from scratch for one of the apps, copied them to the 2nd app, manually edited the filenames and then added/removed source files. (The apps are actually quite similar and mostly use the same names for the source files.) Both apps run fine but MSVC thinks the libraries are out of date and always recompiles them when switching from one to the other. (Repeated 'build' doesn't trigger recompile for either app by itself.)
I have been assuming the vcxproj files for the static libs can't be causing this because there is only one for each static lib and they all live in the library directory which is separate from the app directories.
The .obj files are very slightly different e.g. 499921 bytes vs. 499913 bytes and nm shows a few differences. Since this is the exact same library with the same settings being used in two very similar apps, I cannot understand why there should be any differences at all.
Is there a way to find out why MSVC thinks a recompile is necessary? Or is there some feature of MSVC which requires a recompilation? I'm thinking if there is some sort of global optimization, it may always want to recompile everything. But this problem occurs even with the debug build.
I have two MSVC C++ apps which share a number of static libraries. When I switch from one solution to the other and try to run the app, MSVC always recompiles the static libraries. To try to stop this, I recreated the .sln and .vcxproj files from scratch for one of the apps, copied them to the 2nd app, manually edited the filenames and then added/removed source files. (The apps are actually quite similar and mostly use the same names for the source files.) Both apps run fine but MSVC thinks the libraries are out of date and always recompiles them when switching from one to the other. (Repeated 'build' doesn't trigger recompile for either app by itself.)
I have been assuming the vcxproj files for the static libs can't be causing this because there is only one for each static lib and they all live in the library directory which is separate from the app directories.
The .obj files are very slightly different e.g. 499921 bytes vs. 499913 bytes and nm shows a few differences. Since this is the exact same library with the same settings being used in two very similar apps, I cannot understand why there should be any differences at all.
Is there a way to find out why MSVC thinks a recompile is necessary? Or is there some feature of MSVC which requires a recompilation? I'm thinking if there is some sort of global optimization, it may always want to recompile everything. But this problem occurs even with the debug build.
Last edited on
Why are you using a preview version of VS2019? The current version is 16.11.2. MS don't guarantee anything for a preview version.
Are you using precompiled headers? They can be twitchy, causing stuff to be recompiled when it doesn't require it.
Interestingly PCH was actually enabled. Not by me though so I guess it happened during the upgrade from vc15. Unfortunately turning it off hasn't made any difference.
Precompiled headers are by default used, unless you use a wizard check box to NOT use them.
I personally despise them, so any new VS project I start I click the check box to not use the feature.
Easy enough to stop using them in an existing project/solution.
I personally despise them, so any new VS project I start I click the check box to not use the feature.
Easy enough to stop using them in an existing project/solution.
Agree 100%. They've been nothing but trouble as far as I'm concerned. This code is pretty old though, most dates back to vc11 days and I definitely had PCH turned off originally. I recall removing all the stdafx.h files to make sure it was disabled. I also never had this recompile problem so something strange is going on. Starting to suspect it's just a bug so if I can recreate it with a small example I'll report it to MS.
Tried new .vcxproj files for the static libs but still the same problem.
Tried new .vcxproj files for the static libs but still the same problem.
Last edited on
Rob190, do you have incremental linking enabled in Configuration Properties -> Linker -> General?
I don't remember what the normal VS defaults are, I have set as default incremental linking enabled for debug builds and disabled for release builds.
I don't remember what the normal VS defaults are, I have set as default incremental linking enabled for debug builds and disabled for release builds.
No it's disabled. In both projects. I seem to remember it caused an issue with something but in any case, these are new(ish) project files so I guess it must be the default.
I do know that updating the IDE to a newer minor version -- 16.11.0/1 to 16.11.2 for example -- can force a full recompile of a solution/project created in an earlier version.
Are your static libs' source code part of your projects? Your dependency chain of the static libs for both projects may be the culprit. There may be some obscure setting that differs between the two.
I'm just spitballin', I've never experienced a situation like you describe.
Custom created static libs I use I place in a separate project and compile/link them on their own. Just add the resulting lib files for any project that uses them.
Are your static libs' source code part of your projects? Your dependency chain of the static libs for both projects may be the culprit. There may be some obscure setting that differs between the two.
I'm just spitballin', I've never experienced a situation like you describe.
Custom created static libs I use I place in a separate project and compile/link them on their own. Just add the resulting lib files for any project that uses them.
Yes, the source files are part of the static lib projects. Everything is integrated. The reason being that I'm making some changes to the APIs so it's much easier to have the app and libraries all under the IDE. I used to have the .lib files explicitly listed so I know that works. It's just too easy to forget to recompile something.
Also, yes, updating the IDE does force a recompile. That's mostly automatic, except for wxWidgets which has some special build steps so that needs to be done first.
Appreciate you taking the time to suggest some ideas. I'm sure it's something simple but it's very difficult to understand what VS is doing, even in 'verbose' mode.
Also, yes, updating the IDE does force a recompile. That's mostly automatic, except for wxWidgets which has some special build steps so that needs to be done first.
Appreciate you taking the time to suggest some ideas. I'm sure it's something simple but it's very difficult to understand what VS is doing, even in 'verbose' mode.
Have you tried removing all files except the .sln/.vcxproj and source files and the re-compiling?
Your static lib projects must be referenced by dependent projects.
You add a reference by expanding project file.
When you do this, make sure to click on every reference and in property window set all fields to false, this forces you to manually set input libraries and library search path.
The only purpose of a reference is to let MSBuild know the order, but linking to libs must be explicit. (explicit linking means MSBuild does not copy *.obj files to target project and does not automatically handle referenced libs beyond knowing the build order)
You add a reference by expanding project file.
When you do this, make sure to click on every reference and in property window set all fields to false, this forces you to manually set input libraries and library search path.
The only purpose of a reference is to let MSBuild know the order, but linking to libs must be explicit. (explicit linking means MSBuild does not copy *.obj files to target project and does not automatically handle referenced libs beyond knowing the build order)
Last edited on
@seeplus: I saw the same problem when I checked out the project on another computer to test if it would compile with vc17 (VS 2022) preview so I think that's equivalent to removing all files except for .sln and .vcxproj (which are the only project files I have in Git).
@malibor: It isn't copying the .obj files. I've just done a desktop search to make absolutely sure and there is only one of each .obj files on my system. It is copying the .lib files. However, that would not explain why it is recompiling the source files in the library. I will try adding the .lib files explicitly. Thanks for the suggestion.
@malibor: It isn't copying the .obj files. I've just done a desktop search to make absolutely sure and there is only one of each .obj files on my system. It is copying the .lib files. However, that would not explain why it is recompiling the source files in the library. I will try adding the .lib files explicitly. Thanks for the suggestion.
Rob190, I think you misunderstood me
Please take a look on reference properties in the screenshot below, it's all set to false:
https://i.imgur.com/crQn2M4.png
Most important, this means MSBuild doesn't automatically link and manage referenced libraries.
To set project reference, you do it after expanding project file in solution explorer, example screenshot contains 4 references (ex. 4 lib projects):
https://i.imgur.com/ealoZSP.png
After you do this specify input libraries (LIB files) under:
Property manager -> <your configuration> -> Linker -> Input
where "your configuration" is target configuration and platform.
And also library search path under:
Linker -> General -> Additional library directories
which is the path to LIB files you want to link.
When you do this it's all set up, now what remain is to make sure you build target project only, ex. not "Build solution" because that builds everything including LIB projects.
If your LIB files are not part of solution, then skip references part, and just explicitly link in linker properties.
I see no reason why would LIB files compile without changes after you do this.
Please take a look on reference properties in the screenshot below, it's all set to false:
https://i.imgur.com/crQn2M4.png
Most important, this means MSBuild doesn't automatically link and manage referenced libraries.
To set project reference, you do it after expanding project file in solution explorer, example screenshot contains 4 references (ex. 4 lib projects):
https://i.imgur.com/ealoZSP.png
After you do this specify input libraries (LIB files) under:
Property manager -> <your configuration> -> Linker -> Input
where "your configuration" is target configuration and platform.
And also library search path under:
Linker -> General -> Additional library directories
which is the path to LIB files you want to link.
When you do this it's all set up, now what remain is to make sure you build target project only, ex. not "Build solution" because that builds everything including LIB projects.
If your LIB files are not part of solution, then skip references part, and just explicitly link in linker properties.
I see no reason why would LIB files compile without changes after you do this.
Last edited on
Sorry, I think I did understand, I just found it difficult to believe Microsoft would deliberately make it so counter-intuitive.
I followed your instructions above and confirm it avoids the recompile. Many thanks.
I still think this is a bug though. I'm used to the way makefiles work and if you don't change source files or anything else the .lib (or .so) is dependent on, it will not force a recompile, period.
I reached out to Microsoft and they claim they cannot reproduce the issue. I now have a simple example which shows the problem so I'll forward it to them.
Meanwhile, I'm not willing to go back to explicitly including dependencies because I used to work that way and it was far too easy to end up with out-of-date libraries. The effect is random and sometimes unpredictable runtime behaviour. Usually it results in exceptions but I've had a number of cases where memory gets corrupted and goes undetected.
I followed your instructions above and confirm it avoids the recompile. Many thanks.
I still think this is a bug though. I'm used to the way makefiles work and if you don't change source files or anything else the .lib (or .so) is dependent on, it will not force a recompile, period.
I reached out to Microsoft and they claim they cannot reproduce the issue. I now have a simple example which shows the problem so I'll forward it to them.
Meanwhile, I'm not willing to go back to explicitly including dependencies because I used to work that way and it was far too easy to end up with out-of-date libraries. The effect is random and sometimes unpredictable runtime behaviour. Usually it results in exceptions but I've had a number of cases where memory gets corrupted and goes undetected.
I would be interested to see your reproducible sample project, because I don't have these issues even if not explicitly linking.
There must be something misconfigured on your side. (your build setup)
There must be something misconfigured on your side. (your build setup)
Yes, I'm sure it's not behaving as it's supposed to.
You'd be welcome to take a look. Problem is I think even a simple example is too big to post here and I don't see a way of uploading attachments. Maybe I can set up a dropbox or something.
You'd be welcome to take a look. Problem is I think even a simple example is too big to post here and I don't see a way of uploading attachments. Maybe I can set up a dropbox or something.
ex. Link to Microsoft OneDrive or google Drive. or GitHub if you know how to use it..
Try not to upload anything personal for your privacy...
Try not to upload anything personal for your privacy...
Last edited on
Try this: https://1drv.ms/u/s!Aq_J7sG5MG9jyXjGfE2F9sKcKeaE?e=QtW9YD
It's just a simple example. Nothing confidential. The issue is switching between 'Project-1' and 'Project-2'. In all cases I see the lib.cpp source file being recompiled.
It's just a simple example. Nothing confidential. The issue is switching between 'Project-1' and 'Project-2'. In all cases I see the lib.cpp source file being recompiled.
Pages: 12