The Case of the Breaking Breakpoints
As a software engineer in the video game industry, I have encountered my fair share of challenges while debugging game servers on a Windows environment. Recently, my team and I ran into an issue where our game server could no longer be debugged using various debuggers like Visual Studio, LLDB, and WinDBG.
It was quite perplexing, to say the least. Whenever we tried to set a breakpoint within our custom module or other modules, the breakpoint would automatically disable, and the debugger would indicate that it could not be hit. This meant we were unable to debug our game server, which was a major setback for our development process.
After investigating the issue, we discovered that the debugger was no longer loading our modules, even though they had loaded correctly at runtime and had been working flawlessly just a few hours prior. We were left wondering what could have caused this sudden change in behavior.
Blame Windows
After some extensive debugging and searching for answers, my team and I were able to find the solution to the issue we were facing with our game server. It turned out to be quite anticlimactic, given the amount of time and effort we put into finding it.
Did you know that there is a kernel limitation from the Windows XP era that is still present in modern Windows systems and can cause issues with debuggers? It turns out that the maximum number of loaded modules in your debugger is capped at 500 by default!
This can be particularly problematic for large projects such as those created in Unreal Engine, where the sheer number of modules involved can easily push you towards this limit. In fact, you might be closer to the limit than you think!
We recently ran into this issue on our game server, where our custom modules and others were causing our breakpoints to automatically disable and our debugger to tell us that they could not be hit. After some investigation, we discovered that we had around 600 modules trying to load into the debugger, but only 500 were actually being loaded due to the Windows limitation.
It’s unclear why this issue had not been discovered earlier, or why it only became a problem for us recently, but it certainly provided an interesting challenge to overcome. So, next time you run into similar issues, remember to check whether you’re hitting this Windows limitation!
The Solution
Unfortunately, resolving this issue requires tweaking your Registry Keys - proceed with caution!
The solution we discovered involves adding a new DWORD key to your registry that overrides the maximum number of loaded modules that can be used by your debugger.
If you open RegEdit in Windows, you want to add a new DWORD key named:
DebuggerMaxModuleMsgs
to:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager
Set the value of the key to a high power-of-2 number, such as 2048, to ensure that it won’t have to be changed again in the future.
Remember to restart your computer after making this change.
Once again, please exercise extreme caution when editing your registry!
… Thanks Windows! 🙃
