How can I get rid of my Amphitheatre, or at least put it into storage?

You cannot store or sell any of the 3x3 coin-producing buildings. The only thing you can do is to use gold bars to select a different façade.