Improve UX when clearing cache due to the stack changing (#442)

* Test that the cache is invalidated when the stack changes

* Improve UX when clearing cache due to the stack changing

Now outputs a message informing that the cache was cleared, and
clears the cache first to avoid a redundant message about removing
an old Python version.