I got lots of gems. On what should I spend it on?

As you upgrade the gems will be very useful to advance your building and research as well as speeding up troop training etc for the hourly and other events.

If you find you have a lot that you have no other use for then I recommend using them to buy materials chests that will give you materials to use for crafting the accessories for your legions. Find the chests in the items store. There are chests for each of the types of equipment sets.