Oligopolies, algorithms and your money

Index providers, transparency and China

Ranjan here, and I want to talk to you about financial indexes.

We’ve all have heard about the S&P 500 and the Dow Jones Index, but are you familiar with the companies that manage and sell these products?

Financial index providers are quietly becoming powerful thanks to the rise of passive investing, via ETFs and roboadvisors. Trillions of dollars are being allocated and re-allocated based on the decisions of a very few, very profitable companies. The selection processes are opaque, and their business models are laden with conflicts of interest. It's worth paying attention to.


It was two recent headlines that got me thinking about this. The WSJ covered the inclusion of Chinese bonds in a major global bond index, while the FT told us "the most powerful man on Wall Street who you have never heard of is retiring" (hint: it's about the Chairman of the S&P Dow Jones Index Committee).

But my exposure to this world goes a little further back, to a college internship in the Salomon Smith Barney Index Group during the summer of 2001. For the millennial readers here, Salomon was once a somewhat legendary investment bank (it was bought by and folded into Citigroup). Like many bright-eyed college econ majors, I got into a generalist internship program and was placed in their Index Group, where I was very out of my element. It was mostly Math PhD's with extensive computer programming experience, and my fellow interns were all computer science students. I still wonder if they put me in the role because they saw an Indian name.....

At this time the Index Group felt like a side project within the bank. Some really smart, quantitative people sat on a separate floor, wrote code, ran numbers and had vigorous debates to compose the various indexes. It definitely didn’t feel like a power center within the firm. The people were really nice too, without any of the trappings you might associate with a banker.


Fast forward to 2019. Index providers are really powerful thanks to the rise of passive investing. Their decisions strongly influence trillions of dollars and passive is projected to overtake active investment management soon. From Forbes:

The thing is, we’re testing that limit more and more every day as passive mutual funds and ETFs—those that seek not to “beat the market” but track an index—take up a larger slice of the pie. The share has increased dramatically in the past 10 years, rising from only 15% in 2007 to as much as 35% by the end of 2017.

As for when passive investments will overtake the active market, Moody’s Investors Service estimates we’ll see this happen sometime between 2021 and 2024.

When a dollar flows into a product that tracks an Index, that dollar is now beholden to the allocation choices of an index provider. When you buy an ETF, choose a fund in your 401k, or crank the risk dial on your Wealthfront account, the underlying investment decision is very likely being made by one of these firms. The FT tells us that "financial indexes are arguably the most under-appreciated force shaping global markets."

So who are these companies and how do they choose what goes into an index?


The Index provider market looks a whole lot different than when I was an intern. The market is heavily concentrated among 3 players: MSCI, FTSE Russell and the S&P Dow Jones Indices. Like many industries nowadays, that concentration should be of concern, especially given the revenue model of these firms.

Index providers make money by licensing their indexes to investment firms to create financial products off of. They make a lot of it, with MSCI pulling in $1.43 billion revenue in 2018, with a Facebook-like ~36% profit margin.

An example: If I'm Blackrock, I create the iShares MSCI Developed World Index Fund for you to invest in, and pay MSCI a licensing fee for access to the data and brand. They come up with the formulas for baskets of assets that properly represent specific asset classes, and people pay them for this service. It seems clean enough.

But here come the conflicts of interest. MSCI recently announced it would be including a number of Chinese stocks in its very popular Emerging Markets Index, after years of rejection. The high-level questions which can shape decisions like these:

  • The argument for - China's stock market is so big, any representative index obviously has to include a percentage of them.

  • Those argument against - China's corporate and financial disclosure practices remain very opaque as compared to developed markets.

There are reasonable arguments on both sides, but things start to get murky when you learn the commercial factors at play:

MSCI’s discussions with several Chinese asset managers were abruptly curtailed in 2015 and 2016 after the firm didn’t add Chinese-listed stocks to the emerging-markets index following its midyear reviews, according to people close to or directly involved in the discussions. The Chinese firms communicated that they had been instructed by authorities to cut off negotiations with MSCI, the people said.

China’s two national stock exchanges also threatened to withdraw MSCI’s access to market pricing data, which the company provided to its customers all over the world, the people added. It was akin to “business blackmail,” said a person familiar with MSCI’s negotiations with Chinese regulatory authorities.

MSCI has spent years trying to become the dominant index provider for financial assets in China, which would provide them millions in licensing fees. If China is included in a major index, it means billions of new capital flowing into their financial markets. You get the picture.

So how do index providers claim to avoid these conflicts? The MSCI website gives us some perspective - everything is rules-based:

MSCI’s index methodologies are rules-based. Any exercise of discretion, which is designed to be rare and limited to situations where the rules-based methodology does not effectively address or anticipate a particular market situation, must be approved by the appropriate governance committee.

..and the WSJ gives a bit more insight into how that "discretion" works:

MSCI said all of its index decisions are made after consultations and feedback from a wide range of global market participants, including asset owners and managers, brokers, local authorities, regulators, stock exchanges and others. The consultation process is “transparent and objective to ensure MSCI indexes remain relevant and accurate investment decision support tools for its clients,” the firm said in a statement to the Journal.

The company cited safeguards including the fact that its index-inclusion process is handled by its editorial division, which is separate from its commercial operations, and based on criteria that it has made public.


Okay, I'm just going to say it. In the first draft of this newsletter I sent Can, I was evasive on my central point, but this is a newsletter so here goes….

  • We have a black box of algorithmic methodologies, built by very smart, quantitative people.

  • There is still a large element of opaque "discretion" that goes into its creation.

  • The business model is built on a potential conflict of interest

  • These black boxes are controlled by a very few powerful and profitable players.

  • These companies can literally shape the future of companies and even countries with their decisions

  • We barely hear about them and don't know how they make decisions.

  • We are entrusting them with more and more money.

It's like this weird, perfect combination of what's wrong with tech today and what was wrong with finance before the crisis (especially with regards to credit ratings agencies). It's something definitely something worth paying attention to.

A few side notes on the topic:

  • As a timely example on how this all works, the S&P and FTSE Russell both made the decision in 2017 to limit or not allow companies with dual-class voting structures. A number of investors were calling for Lyft to not pursue their dual-class structure so they could gain inclusion in the S&P 500 which would open up massive new pools of capital. They didn't and a friendly reminder that the Lyft founders own 5% of the equity and 49% of the voting rights. Community!

  • Another example of politics and indexing colliding: Last summer the MSCI made the decision to include the Saudi Arabian stocks in the MSCI EM Index, based on the modernization effortss of MBS. The killing of Jamal Khashoggi was exactly the type of event where there would be human, editorial debates over whether to maintain the inclusion. They did, and January was a record month for foreign buying in Saudi stocks.

  • If you pay them, they will build it and tie a financial product to it. There is every kind of index you can imagine out there. One of my favorites was the S&P Catholic Values Index.

  • There's a lot more media + finance history you can dive into. The S&P Dow Jones Indices JV is "between S&P Global, the CME Group, and News Corp that was announced in 2011 and later launched in 2012". S&P Global is an offshoot of McGraw-Hill publishing. They also still own "the Dow Jones Transportation Index, which is considered the first financial index and created in 1882 by Charles Dow, the founder of The Wall Street Journal." I love financial media history and this might be another post in itself.

  • One related longread from Institutional Investor, on the conflict of interest underlying bond credit ratings agencies and an effort by Jules Kroll to build a better ratings agency. For those unfamiliar, he’s the founder of Kroll, Inc. and kind of the inventor of the corporate investigations field. He’s also the father of Nick Kroll :)

On a related, more personal note:

The Index Group internship was at 390 Greenwich Street, in Tribeca, just a few blocks from the World Trade Center. After work we'd go to happy hours at Moran’s at the base of the WTC (the ultimate 2001-finance-bro-wannabe-happy-hour), and a few times went up to Windows on the World. I could see the buildings every day while sitting in the office.

I left NYC in mid-August to return to my senior year in Atlanta, but had an interview back in NYC scheduled for Sept 12th, 2001. It was rescheduled to a week later, and I flew up to interview with Bank of America for the trading role where I'd spend much of the next decade. I stopped in to see the folks from the Index group. They were all telling me how they could see everything clearly. One of the MDs brought up the sheer number of people he watched jump - and how there was almost no footage of it so it wasn't in the press. They had a number of therapist services set up at the office. And the smell.....you could still, from the 30th floor, a few blocks away, about 10 days later,still remember that burnt smell.