The dots with numbers seem to take over the board. And they are never the color I need in the quest...

The fastest way is to square the color of the number dots. It will remove the same-colored non-numbered dots, and lower the number on the numbered dots. The number is how many times it has left before it stops replicating. You can also lower that number by creating rockets, or using one of your booster rockets. It's the hardest when there are pearls (oysters) or bulls involved, because the numbered dots will swallow them. Maybe someone has better methods. I hope this helps some.