You can find an explanation here (github of the googleapi):
My current understanding of a color's score is a combination of two things:
What is the focus of the image?
What is the color of that focus?
For example, given the following image:
The focus is clearly the cat, and therefore the color annotation for this image with the highest score (0.15) will ...