Skip to content
Behavioral economics

Understanding market bubbles through sentiment data

How sentiment indices can help investors make sense of conversation in the news and social media – and avoid getting caught in a speculative bubble.

Financial bubbles, it appears, are a fact of life in today’s global economy. Formerly rare, once-in-a-generation events, bubbles now inflate – and deflate – much more often, thanks in part to freer flows of money and assets across borders and broader dissemination of data about markets and investments. It’s a safe bet that anyone reading this article remembers the dot-com bubble of 1996-2000, the 2004-2006 housing bubble, the Chinese stock bubble of 2006 and the bitcoin bubble of 2013. Others attract less public notice but nevertheless entice many investors, such as the 2006-2008 bubble in the price of Rhodium, a rare chemical element.

What is a market bubble?

A “bubble” is a speculative mania – a surge in the price of an asset that’s not justified by its fundamentals and that ends when market sentiment turns and the asset’s price collapses and returns to the mean.

Bubbles are notoriously difficult to spot and tricky to invest into and out of.

Because they occur so much more often than in the past, it’s more urgent for investors to understand them and know how to respond when they appear. By harnessing the vast amount of news and discussion available from online news feeds and archives and from social media, and applying sophisticated analytics, analysts can distill a broad reading of collective sentiment about individual companies and asset classes, enabling them to spot the next bubble.

Enter sentiment indices and the “Bubbleometer”

Sentiment indices reflect market perceptions. MarketPsych LLC has collected 17 years of such market sentiment data from both news and social media. Trust is one such sentiment, and the trust that investors feel toward a company is a key driver of its stock price. By quantifying references that indicate trust in a company’s management team versus those that indicate suspicion, MarketPsych creates an index called “ManagementTrust” for thousands of companies. Investment professionals can derive new insights from visualizing and tracking public perceptions such as these. Additionally, investors develop predictive models on top of this data, enabling them to enhance their investment and trading performance.

MarketPsych developed the Thomson Reuters MarketPsych Indices (TRMI) in 2012, using real-time linguistic and psychological analysis of news and social media to quantify how the public regards various asset classes according to dozens of sentiments including optimism, fear, trust and uncertainty. Initially, the TRMI covered five asset classes: agricultural commodities, materials and energy, countries, equities and currencies. Some sentiments apply to particular asset classes more than others – for instance, InflationForecast is relevant to countries’ macroeconomic environments and may influence their fixed income and currency prices as well, while ProductionVolume is of concern specifically in relation to commodities. This spring, Thomson Reuters and MarketPsych expanded the TRMI to include 7,500 individual companies.

The TRMI are the product of a complex process of relevance and credibility testing applied to a pool of tens of thousands of online sources and authors in both news and social media. Three principal sources comprise the searchable universe:

  • Millions of social media messages daily from 1998 to the present
  • News articles crawled from thousands of Internet news sites, some since 1998
  • Live, low-latency Thomson Reuters News

Relevant and credible articles are then analyzed by parsing out English-language entities, words and phrases of potential interest to traders, investors and economists. Complex relationships between those concepts, based on part of speech and grammatical analysis, are determined algorithmically and scores assigned to a pool of important fundamental, technical and sentiment concepts. The indices themselves are constructed using a Natural Language Processing software that employs grammatical templates to extract the correct meaning from all of these text sources. For instance, MarketPsych’s software “knows” that the phrase “That trade was the bomb!” refers to a successful trade, not to warfare.

Big Data analytics like the TRMI are accumulating valuable insights about bubbles – for example, by monitoring the balance between speculative conversation (references to future price outperformance as well as the expression of positive emotions) and analytical conversation (references to accounting fundamentals and expressions of pessimism and fear).

When MarketPsych created the “Bubbleometer,” which measures the amount of speculative minus analytical conversation in the media about an asset, for instance, it found that the result was inversely correlated with the price of that asset over a one-year period: The higher the Bubbleometer, the lower the asset price one year later.

The four stages of a market bubble

The four stages of market bubbles: Reasoned optimism; greed and overconfidence; discord, disagreement and price volatility; panic pessimism and price crash

Could investors have anticipated the recent deflation of another speculative frenzy in China?

When the Chinese stock market exploded in 2006-2008 – the Shanghai Composite Index rose 130% in 2006 and 84.4% in the first half of 2007 alone – local investors said they trusted the government would not let prices collapse and cited rumors that Beijing would allow greater foreign investment. In 2008, the bubble collapsed and the index lost 65% of its value.

Shanghai A-shares rose 100% in the 12 months ending in June. Overall Chinese optimism – based on English-language references – spiked, although nowhere near the optimism levels of 2006. However, Chinese-language sources reflected substantial unrealistic optimism about stock prices. MarketPsych’s Sentiment Index showed market sentiment peaking in early June, anticipating the decline in prices that set in less than a month later.

Chinese business sentiment versus the Shanghai Composite Index

30-day and 90-day moving averages of media sentiment are depicted. Shading between the two averages is gray if the 30-day average is lower than the 90-day average, and blue if it is higher. When short-term sentiment rises, it tends to pull prices upwards; when it falls, prices tend to follow it down.

Chart shows Chinese business sentiment versus the Shanghai Composite Index

The flip side of a bubble is when an asset is undersold, often in overreaction to bad news, and investors run the risk of missing an opportunity when the asset price bounces back. The TRMI highlights how investors can overreact and underprice assets when an event takes place that contradicts their standard view of a particular market or asset. Overreaction refers to the tendency for stock prices to mean-revert after a large, significant event. As a result, it is advantageous to buy low-trust stocks and shift high-trust stocks on a monthly rotation – a practice MarketPsych calls ”trust arbitrage.”

The graph below highlights the performance of a simple investment strategy that buys the least-trusted and shorts the most-trusted stocks in the US market on a monthly rotation. To derive the equity curve shown here, MarketPsych’s software first identified the top 100 US stocks in the news over the past 12 months, then ranked these stocks by their average monthly level of Trust (quantified in the Trust TRMI). The strategy then bought the 20 least-trusted stocks and shorted the 20 most-trusted stocks at the open price the following month and held these positions for one month. Repeated monthly since February 1998, this is an absolute return strategy that produces its gains through trust arbitrage.

A trust arbitrage strategy for the top 100 US stocks in the news

In a monthly rotation, this strategy identifies the top 100 US stocks in the news over the last 12-month period. It then buys the 20 least-trusted stocks from this group and shorts the 20 most-trusted at the open price, and holds these positions for the next month.

Chart shows rise in MP news trust

Petrobras, the partly state-owned Brazilian energy company, provides a recent example. The focal point of a recent, major bribery scandal, Petrobras went from enjoying the second highest market sentiment among four leading Latin American companies that the TRMI tracked from June 2014 through May 2015, to last place. Yet, after its stock price moved sideways for four to five months during the worst of the scandals, Petrobras jumped 75% in less than two months, from March to May 2015, as the company announced a major new oil discovery off the Rio de Janeiro coast and moved closer to an agreement with Brazilian regulators on valuation of costs related to the bribery scandal.

Data from the TRMI also shed light on how investors change their strategy based on their overall view of the market: Growth strategies predominate following periods when market sentiment is optimistic, and value strategies when sentiment is more negative. These patterns underscore the impact that trailing market sentiment can have on investor behavior.

“The flip side of a bubble is when an asset is undersold, often in overreaction to bad news, and investors run the risk of missing an opportunity when the asset price bounces back.”

In a paper published last fall, Elijah DePalma, senior quantitative research analyst for Thomson Reuters Machine Readable News group, argued that market sentiment also has a dynamic influence on risk pricing. Periods of negative sentiment about a particular asset tend to be followed by high volatility in that asset, for example – creating opportunities for investors so long as they can control risk. So, when DePalma constructed equal-weighted, monthly-quintile portfolios using market beta – the volatility or systematic risk in the market – he found that beta is strongly priced following periods of negative market sentiment, suggesting that risk-adjusted returns can be improved by neutralizing the portfolio against beta. On the other hand, following periods of positive market sentiment, high-beta portfolios generated lower Sharpe Ratios than did low-beta portfolios, suggesting that low-volatility portfolios outperform high-volatility portfolios following these periods.

Insights like these can help investors to succeed at perhaps their greatest challenge: separating their own convictions about the market, and the direction the market should take, from the sometimes overwhelming conversation generated in the market and the media. Are investors following their own best practices – the sensible responses to unexpected run-ups or run-downs in asset process that they claim are part of their standard operating procedure – or just following the market? A better understanding of the patterns in the conversation can help them to answer this question clearly and confidently.


About the author

Dr. Richard PetersonDr. Richard Peterson is CEO of MarketPsych Data which produces psychological and macroeconomic data derived from text analytics of news and social media, also known as the Thomson Reuters MarketPsych Indices. MarketPsych’s data is consumed by banks, hedge funds, commodities and forex traders and economic research departments. Peterson is an award-winning financial writer, an associate editor of the Journal of Behavioral Finance, has published widely in academia and performed postdoctoral neuroeconomics research at Stanford University.


Learn more

Get more from Exchange Magazine in the Know 360 App.

  • Facebook
  • Twitter
  • Linkedin
  • Google+
  • Email

More answers