Do you get penalized for selecting the wrong items in the hidden object scenes?

I noticed a negagive clock icon pop up and the timer subtract from the time I think for either clicking the object twice or clicking another object too quickly. I have had it happen 2 or 3 times so far but have not figured it out more yet. Stay tuned when I know more will advise as i understand this issue.