I don't understand why it takes so long to move up on the leaderboard when you get perfect scores in..

"Perfect" scores only means no hints, no misses, no lapse of the bonus bar. It can be achieved without fast speed or tapping in order. To climb the leaderboard, you have to beat your previous score by being faster than before. Tapping objects in the order they are written left to right will maximize the base score (454,000 at the 5th star) and may also help if it doesn't hurt your speed.

I can only move up is to top my one score