When I sold an item to make room on my island I expected the total number of flowers I had would decrease..

Yes, that is correct. The counter shows how many more flowers are required for the next level. When the counter reaches zero, you will level up. So when you add items to your island, you'll get closer to zero. When you sell things, the count will go up, farther from zero. Once you reach zero, you level goes up by one, and the flower count will immediately show you how many you need to reach the next level.

The number of flowers you see is not how many flowers you already have but how many more you need to progress to the next level. If you sell an item the number of flowers needed will always go up since you are taking away the flowers of the sold item. The number will never decrease, which would mean you need less flowers while actually now you need more flowers to progress.