Question

In his experiment, Duncan Watts created a website where respondents could rate songs. His sample size was 14,341 respondents. If Watts selected a new sample of 14,341 different respondents and their average ratings of each song were significantly different than the first sample, what problem would Watts's research suffer from?

Answer 1

Answer: unreliability

Explanation: Some of the problems that determine the quality of an experiment are reliability and validity.  

Reliability considers the consistency of the experiment. In the example above,  the result of the second sample is inconsistent with the result of the first sample, making Watt's research unreliable. That is, it does not reflect the ratings of a larger population. This problem can be solved by increasing the sample size, such that the average ratings are more reflective of a wider audience.

Validity considers the accuracy of the experiment. A lack of reliability of an experiment will likely make the result invalid as well, but this is not always the case.