Can you really rely on online product reviews?

Product reviews on online platforms are growing in popularity1,2. Platforms like Amazon, Google or the App store use product reviews to show which products have the best experience in usage by other consumers. Most of these product reviews are extremely positive about the product3, but does this indicate that all products are extremely good and that there is no moderate product on the online market? Let’s give it a try to search on Amazon three random product reviews from books, video games and sports. The results are shown in table 1.

table 1

As can be seen from the table, two of those random reviews are extremely positive (the book and the sport watch) and one is extremely negative (the video game). An experiment done by Hu et al., (2009) asked customers to rate a music CD on a 5-star scale. This experiment shows an almost normal distribution, which can be expected if the ratings are randomly done by every buyer of the product. Most of the reviews on Amazon (table 1) show a so called J-shape distribution and not the outcome of the explained experiment. What could be the cause of those differences?

The first explanation is the purchasing bias, which states that customers with higher product valuation are more likely to purchase the product than customers with a low product valuation. Customers with a higher product valuation are more willing to write positive (and in rare cases negative) about the product, because they are biased in the valuation of the product and this give the high ratings4. The second explanation is the underreporting bias, which states that customers with extreme ratings (5 stars for very satisfied or 1 star for very disappointed) are more likely to express their opinion on the website than those customers with a moderate rating4,5. This type of bias should be taken into account in all reviews and marketing models, and ignoring this bias can result in the incorrect interpretation of the given data5. Both examples indicate that the online reviews are not as reliable as online platforms such as Amazon might argue.

Customers rely too much on extreme online reviews, which has the same impact on consumers as with positive or negative worth of mouth2,4,6. Together with the two explained biases, it indicates that online reviews, by using such a rating system, are not representative for the quality of the product2,4. This does not indicate that consumers should never look at online reviews. Today, most reviews also contain explanations of the reviewers about the rating they gave. These explanations are useful for the consumers to understand the positive and negative experiences of the reviewers, and give more insight in which products fits their expectations the most.

Thomas (400487)



  1. Chen, Y. and Xie, J. (2005). Third-Party Product Review and Firm Marketing Strategy. Marketing Science, 24 (2), pp. 218 – 240.
  2. Dellarocas, C. (2003). The Digitization of Word-of-Mouth: Promise and Challenges of Online Reputation Mechanisms. Management Science 49 (10), pp. 1407 – 1424.
  3. Kadet, A. (2007). Rah-Rah ratings. From: SmartMoney Magazine,16 (3), p. 116.
  4. Hu, N., Zhang, J. and Pavlou, P.A. (2009). Overcoming the J shaped distribution of product reviews. Communication of the ACM, 52 (10), pp. 144 – 147
  5. Yang, A., Zhao, Y. and Dhar, R. (2010). Modeling the Underreporting Bias in Panel Survey Data. Marketing Science, 29 (3), pp. 525 – 539.
  6. Chevalier, J. and Mayzlin, D. (2006). The Effect of Word of Mouth on Sales: Online Book Reviews.  Journal of Marketing  Research, 43 (3), pp. 345 – 354.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s