The Secrets of High Frequency Trading

In recent years, advances in telecommunications, computing capacity and financial software platform capabilities have seen huge growth in the field of High Frequency and Algorithmic Trading (now accounting for over 70% of all equity trades placed on US exchanges and in excess of 77% in the UK).  HFT firms (which can often make more than 80 million trades in a single day) often enter and exit trades in thousandths of a second, and are conservatively estimated to generate at least $21billion in profits every year.

To get under the skin of the world of high frequency trading, interviewed Arzhang Kamarei.  Mr. Kamarei is a Partner at Tradeworx, a quantitative investment management firm with expertise in high-frequency and medium-frequency equity market-neutral strategies.  He co-founded Thesys Technologies, a Tradeworx subsidiary, in early 2009 to address the growing technology needs of high frequency traders. What is the principal investment strategy behind High Frequency Trading?

Arzhang Kamarei: The majority of US Equity HFT is employed in the strategy of liquidity provisioning, also known as electronic market making.   Historically, such a service was provided by NYSE specialists and NASDAQ market makers but, with the advent of decimalization, human specialists and market makers were no longer able to keep up with the liquidity demands of investors and automated technology became necessary for this function.

To implement electronic market making strategies, HFTs use passive orders, which are limit orders that do not cross the spread, but stay on a limit order book until they are filled or cancelled.  This allows HFTs to profit from capturing rebates and the bid-ask spread.   These profits offset losses incurred by providing liquidity to informed traders or large traders who drive stocks directionally (i.e., adverse selection or inventory risk).  Without the rebate and spread capture employed by passive trading, the majority of HFT strategies would be unprofitable.   HFT strategies that are based on actively crossing the spread and consuming liquidity are rare, although active orders are occasionally necessary for inventory or loss management.  HFTs do not typically have enough alpha to make “all-active” strategies profitable.

A key goal in providing liquidity passively is to be on the top of the bid/offer stack, otherwise known as being at the “front of the queue” for the order book.  The rationale for this is very simple:  adverse selection is lower for passive orders at the top of the stack than for passive orders at the bottom of the stack.  For an intuitive understanding of why, consider the following:  if you are bidding at the top of the stack and are hit, the remaining orders on the book support the bid price you have just paid.  However, if you are hit on a bid at the bottom of the stack , it is very likely that the price you just paid is in the process of becoming the new offer.  Buying on the offer and selling on the bid is a characteristic of active strategies that consume liquidity; as mentioned before, these active strategies have a high hurdle rate for profitability and are not feasible for market makers who have very small alphas.  To give a sense of perspective, consider that an HFT making $0.001 per share in SPY (a typical HFT profit margin) has an alpha that is worth $0.001/$120 – or roughly 0.0008%.   A one penny spread is 10 times this amount.   In contrast, long term investors are able to effectively employ active strategies.  Their alphas are orders of magnitude greater than the cost of crossing the spread and afford them the ability to choose when to execute trades as opposed to having to wait for the contra side as market makers do.

Once one understands the disadvantage of being at the bottom of the stack, it becomes much easier to understand the importance of speed in HFT.   As new price levels form, HFTs compete with other HFTs to join the new price as quickly as possible in order to secure advantageous stack position for passive orders.  After all, shaving off microseconds is only a meaningful objective if one is in competition with other market participants who have similar response times.

HFTs are sometimes perplexed by the negative insinuations that are made against their form of market making behavior.  From their perspective, liquidity provisioning in today’s marketplace is a highly competitive enterprise that is governed by market forces.   Compare this to the dynamics of market making twenty years ago when NYSE specialists and Nasdaq market makers dominated liquidity provisioning.  In that era, specialists enjoyed monopolies on market making for a particular stock, had full knowledge of size and direction of client orders (often including the disclosure of the ticket size of the “upstairs trader” – i.e., orders not even yet on the floor), and often had the discretion to fill customer orders against their book at times and prices of their choosing.  In the old days of NYSE trading, if a large mutual fund sent a five hundred thousand share order to buy IBM to the floor of the exchange, the broker in the crowd would tell the specialist he had “size to buy.”  He might, in fact, even disclose to the specialist (and others in the crowd) the total size of his order.   On top of these disadvantages for customers, spreads were mandated to be $0.0625 per share, or what would amount to 62.5x current HFT profit margins.  Although one might argue that HFTs do not have the same stringent market making requirements of the specialist era, the fact remains that academics, research analysts, and even regulatory agencies have recognized that today’s markets are more liquid and less expensive than before the advent of HFT.  This is no doubt due in part to the improvement in liquidity provisioning that has arisen from the better risk management that computers provide over human traders. What are the key markets high frequency traders look towards and why?

Arzhang Kamarei: HFTs prefer markets which:  (1) allow for short-term strategies that can go home flat with no net exposure;  (2) have trading venues with low latency matching engines;  (3) have securities with lot sizes small enough to allow for precise risk control; and (4) are markets in which automated market making has some advantage over human market making.    For instance, if the swaps market continues to predominantly trade in very large lot sizes, it will not be an attractive asset class for HFTs.   This is because large-sized, low turnover trades have much higher risk and holding period than HFTs typically target.  As holding period and risk increase, the importance of alpha increases – and market makers must then begin to form alphas similar to those of their clients simply in order to manage their inventory.  In addition, this leads to a market dynamic where automated trading loses advantage over employing a human trader who can make long term predictions and evaluate unique, idiosyncratic transactions much better than a computer can.  In such a market, humans have an edge over machines, especially as the complexity of the factors which influence alpha increases significantly.  To take the example to an extreme, consider how many real estate or private equity transactions are currently executed by a computer versus a human being.  Being an “alpha-less market maker” in such a market is not possible.

HFT strategies perform better in volatile markets for simple reasons.  High volatility is correlated with high volume and as volumes increase, HFTs’ absolute profits increase.  To make the example more concrete, if an HFT’s profitability is approximately $0.001/share and that firm typically trades 100 million shares a day, on days when they trade 150 million shares their profitability will be greater.  An additional reason that volatility benefits HFTs comes from the competition of liquidity seekers.  In volatile markets, liquidity seekers become highly competitive in their behavior and aggressively compete for volume at the same prices.   As competition forces liquidity seekers to be more aggressive with their active orders, this can drive spreads wider and increase the margins of passive liquidity providers.  This gain in profit margin, however, is typically secondary to the gains from higher volumes.  Given the highly competitive nature of HFTs, it’s not difficult to understand why – competition drives liquidity provisioning such that wider spreads are quickly competed away.

While HFTs do see their profits increase during periods of increased volatility and volumes (as market makers commonly have throughout history), the liquidity they provide via passive orders serves to dampen this volatility, not to increase it.  Put differently, the activities in which HFTs engage during volatile markets are exactly the behaviours that reduce volatility.  However, because of the simple fact that HFT P&Ls increase during volatile markets, many assume that HFTs have somehow created this volatility to begin with.  To assume so is to confuse correlation with causation.  To make an analogy, this is similar to arguing that air conditioning manufacturers cause hot weather because whenever temperatures inside buildings rise greatly, their sales increase.   Given that HFTs have been this business for several years now, if they were able to spontaneously generate volatility at will and then benefit from it, such a perpetual profit machine would ensure that they would have significantly larger market caps than they currently enjoy.  For anyone who seeks to understand why our markets have had such volatile bouts in the past few years, one only need to look at the news headlines.  The past three years have seen talk of another Great Depression, the imminent demise of the dollar, devastation in the housing market, the potential disintegration of the euro, and a downgrade of US creditworthiness.  Finally, if this is not adequately convincing, one should consider the volatility in asset classes not dominated by HFTs over this same time period (such as credit default swaps) or instances of extreme market volatility that pre-existed current HFTs (for example, April 4, 2000, when the Nasdaq market fell nearly 15% and recovered to nearly flat in the same day).  What are the key risks in High Frequency Trading and how to traders mitigate those?

Arzhang Kamarei: The key risk in liquidity provisioning is inventory management.  Informed traders who have strong alphas or large traders who have significant size to move help create this risk.   Of course, this risk is no different than the risk faced by specialists years ago when an informed trader could be accused of “running over the specialist” by asking for a market, transacting on it, and then continuing to drive the stock aggressively in the same direction.  The tactics for controlling this risk are to keep net exposures low and diversify across different securities, just as it is in any form of portfolio management.  Although volatile regimes can be dangerous for any trader and increase the risk of adverse selection for HFTs, volatile regimes bring an increase in volume, which benefits HFTs who see a greater demand for liquidity provisioning services (i.e., it’s a “volume business” and volumes are good in volatile markets).   A positive side effect of this is that HFTs are incentivized to provide passive liquidity on volatile days, thereby dampening volatility overall.   Consider that no day in 2008 saw a loss of the same magnitude as Black Monday, despite the fact that many considered the economic risks at the time to rival those of the Great Depression.

Technology failure risk is managed by building redundant systems with human oversight at all times.  HFTs are very focused on real-time risk management and, given the fact that many trade with their principals’ own capital, this is hardly surprising.

In terms of market structure risk, significant strides have been made recently by exchanges and regulators with the advent of single stock circuit breakers.   In the past, NYSE specialists could help institute a halt when trading became anomalous.  Although HFTs replaced many of the key functions of specialists, until recently, the elimination of this feedback loop and the lost ability of market makers to pause trading during turbulent activity was not identified as a key risk factor by regulators.    Single stock circuit breakers should provide a safety cushion that addresses this issue. What are the key challenges for investors in the HFT arena?

Arzhang Kamarei: On a day to day basis, the main challenge for any HFT comes from competition with other HFTs.  The barriers to entry in the HFT space are relatively low as the cost for acquiring the appropriate technology has decreased significantly and is not greater than the cost of starting a small hedge fund.  This is amply evidenced by the high number of small prop shops in the HFT space.

As an industry, the greatest challenge for HFT comes from the risk of adverse regulation with unintended consequences.  For instance, consider the concept of a transaction-based tax.  The problem with such a tax is that it targets a specific business model as opposed to targeting profitability.  HFTs are in a low margin, high volume business.  Their expected profits per trade are extremely low at approximately $0.001/share.  On the other hand, long term investors have large alphas and expect profits in the tens of cents, if not dollars, per share.  To see how lopsided this tax is, consider that a tax of 1 basis point on SPY, the most liquid equity ticker, is more than the entire profit margin of an HFT.  To make an analogy, it would be akin to the Internal Revenue Service instituting a $1 tax on every item sold in a store.  A jewellery store would easily be able to pay a $1 tax per piece of jewellery sold with no adverse consequences.   On the other hand, a book store would be affected much more harshly by such a tax.  And a low-margin dollar store would likely see its business completely decimated.  Although some pundits do not seem to be concerned about the elimination of HFT, they have yet to give an example of a successful financial market which did not have market makers.  The marketplace is a delicate ecosystem and it seems unlikely that the elimination of 50% of trading volume and the overwhelming majority of passive orders would not have a serious impact on the health of the system.

Finally, an argument that is often repeated by adversaries of HFT is that HFT is not socially useful.  There are serious questions about the philosophical underpinnings of this line of reasoning.  Free market economies are based on equal opportunity, low barriers to entry, and competition.  Free markets are not based on social utility.   Legislators do not seek to tax mutual funds that underperform ETFs for not achieving any social utility.  Similarly, regulators are not seeking to ban hedge fund managers who have a high correlation to beta for being socially useless.   That HFTs are willing to engage the debate on these terms is likely a testament to their confidence that the generally accepted view that HFTs improve trading costs and liquidity provision for all market participants is true.

What does this mean for traders?

Any innovation, regardless of sector is met by those who embrace, those who resist, and those who misunderstand.  HFT has experienced all of these.  The truth is that HFT combines rigorous intellectual disciplines including finance, economics, mathematics, physics and computing – and creates highly profitable opportunities for investors.  Alongside this, high frequency firms provide essential liquidity and counterparty availability in the market.  In some senses, you could see them as the lubricant to the efficient functioning of the markets.

This is a huge growth sector- and with scientists entering the fold from fields as diverse as biology, neuroscience and even astrophysics- the future of HFT is going to be very interesting.

Be Sociable, Share!


  1. Mike O'Hara
    September 7, 2011 at 4:52 am

    Excellent article. I congratulate Mr Kamerei on providing a very clear explanation of what high frequency trading actually is and how it works.

    Anyone who is asking the question “do HFTs serve a purpose?” should read this article.

    Great use of analogy too, really helps to clarify the issues.

  2. Ian
    September 7, 2011 at 6:08 am

    A very fine article. I have always disliked HFTs having the possibly misguided opinion that they are pinching pennies from the average investor. Maybe they are in fact not that bad

  3. mlr
    September 7, 2011 at 8:40 am

    Impressive article explaining HFT, although it would help me and the general public, lawmakers, and the mainstream media if was explained more in plain English.

    I fear that many want HFT abolished or taxed out of existence not realizing very long-term buy and hold investors will have annual yields reduced as some estimate as much as two or three percent with a tax and resulting increased spreads and put most short-term or human traders out of business.

  4. What Da?
    September 7, 2011 at 10:53 pm

    The author writes this article as though HFT is a needed spoke on the wheel of markets. In reality the spoke is now the wheel and the wheel is now a rectangle.

    Our markets are no longer properly functioning markets. The life cycle of an entry onto our markets should be:
    (1) Initial Public Offering
    (2) Company Growth
    (3) Job Creation
    (4) Investor Participation
    (5) Overall Economic Growth

    Where does the HFT guru or author mention any of these factors? In fact, if you look at each one of these factors, since the creation of HFT, every listed factor is in a negative stage of existence.

    It is obvious that the current structure of our markets is not working. 3% rises and declines on a daily basis is not something that investors will stomach. Without investors, the markets are just HFT playgrounds. This is what we have and the SEC needs to address this. The problem is not HFT, the problem is that the SEC has enabled a structure that allows them to exist and profit such as they do.

    Wake up SEC, your markets are no long functioning properly.

  5. mlr
    September 8, 2011 at 8:18 pm

    Volatility created by HFT? During the Great Depression the average monthly range was +20% to -20%. Should have been HFT back then for less volatility.

    HFT created negative impact on your (1)-(5)? All of a sudden HFT was born the day after the bull market ended March 24, 2000? The tech and dot com boom is over. The government artificially created a housing bull market to take its place and ended in failure and the economy we have today.

  6. Scott
    October 19, 2011 at 2:26 am

    “impressive article”?? I’m a HFT and all he said is that HFT are making markets. With like a dozen exchanges there is definitely a need for market makers.

    The $21billion profit figure seems way too high. That is not made from high frequency trading.

  7. Trading Market Online
    June 3, 2012 at 9:29 pm

    it’s not that hard to trade by automated system with high frequency. it is simply necessary to find an edge and use it over and over again. even if the chance of profit would be 51% + commisions.. then system can be used live

Leave A Reply

← High Frequency Trading Dressing your pension fund: One leg at a time →