Do prayer letters do anything except allow you to get kongming lantern?

They can be used individually to get a single small reward item or combined into the lantern which gives greater rewards.

You can exchange them for random crates of goods. 1 lantern = 1 create. If you exchanged 5 letters for a lantern, you get 6 crates.