Is there a way to track down an enemy that continuously teleports in and attacks me?

Build your kingdom tower the higher possible. You will receive better info on his scouts and attacks. The best way is to be online of course and see his alliance position in the data. But if the b.i.t.ch has so much gold for teleports probably is a P2W ba.ss.t.aard so better for you to hide in other place.