"Close document windows" in Visual Studio doesn't clear the current context

If I open some windows in Visual Studio, they're saved to the current context. However if I right click the tab well and click "Close document windows", the windows are not removed from the context, and they reappear next time I open the branch I was working on. I have to manually update the context in Mental Snapshots to remove them.