Quant Funds: Model Risk and Error Disclosure Missing From Many Radar Screens

By Christopher Faille

A settlement early this year between the Securities and Exchange Commission (SEC) and the AXA Rosenberg Group LLC (ARG), along with other entities affiliated with ARG, took much of the asset-management industry by surprise, according to a recent benchmarking survey of market practice on model risk prepared by Capital Market Risk Advisers and the International Association of Financial Engineers.

Specifically, 42 percent of the respondents to the CMRA/IAFE survey said they were surprised either by the fact that the SEC concerned itself with models unrelated to financial reporting or disclosure or by the implication that model error ought to have been disclosed.

On February 3, 2011, the Securities and Exchange Commission charged ARG with securities fraud.  ARG is a holding company, which owns and governs AXA Rosenberg Investment Management LLC (ARIM), an institutional money manager, and Barr Rosenberg Research Center LLC (Barr), the company that has developed and periodically updates a Risk Model used by ARIM.

The specifics of the problem alleged by the SEC turn on the distinction between Barr’s Risk Model proper, and a separate system, called the Optimizer, a program that took data generated by the Risk Model and used it to recommend an optimal portfolio for a particular client based on a benchmark chosen by that client, such as the S&P 500.

SEC charged that after the Risk Model update in 2007, two programmers goofed.  They were assigned the task of writing code that would link the new version of that program with the Optimizer.  Their coding reported some information to the Optimizer in a decimal form, though other information was expressed as percentages.  As a consequence, the Risk Model was working at a less than optimal level from April 2007 onward.

There was no independent quality control of their work.  Matters seem to have rolled along in their sub-optimal way until June 2009, when another version of the Risk Model was to be introduced.  A new Barr employee “noticed certain unexpected results” when comparing the 2009 model then under preparation to the older 2007 model.

A No-Escalation Decision

He presented his findings to a Senior Official of Barr later that month and advocated that the error be fixed immediately.  But the Senior Official said that it would be fixed with the new model was implemented, that September, and in the meantime told other Barr employees to keep quiet about the discovery, and in particular not to inform ARG’s Global Chief Investment Officer.

It wasn’t until late November 2009 that a Barr employee informed ARG’s Global CEO that there ever had been such an error.  Thereafter, that company conducted an internal investigation and disclosed the situation to the SEC examination staff.  In April 2010 it took the next step, informing its clients.

None of these entities (ARG, ARIM, and Barr) have admitted or denied any wrongdoing.  Together, though, they consented to the entry of an SEC order that assigned joint and several liabilities of $25 million and that separately demanded that they pay $217 million to the clients of ARIM and other advisers affiliated with ARG to redress harm from the coding error.

Bruce Karpati, Co-Chief of the Asset Management Unit in the SEC’s Division of Enforcement, point a dot on the “i” in the SEC’s statement: “Quant managers must be fully forthcoming about the risks of their model-driven strategies, especially when errors occur and the models don’t work as predicted.”

Survey Highlights

That is the background of a recent survey by Capital Market Risk Advisors and the International Association of Financial Engineers. CMRA and the IAFE asked banks, asset managers, insurance companies, and institutional investors about their practices concerning model risk.

Among the statistical highlights:

  • 41.1 percent of respondents do not escalate model errors
  • There is no clear consensus as to when a model “change” or “enhancement” becomes an error
  • 63.2 percent review models on an ongoing basis; the remaining 36.8 percent focus their reviews on the time of the model’s release
  • 49.4 percent have error disclosure policies – and only half of those who do have such policies have them approved by the board.

The survey also looked into the question of what role, if any, model risk plays in the due diligence of institutional investors (II).  It found that only 47.7 percent of II respondents said that model risk “has been traditionally part of our risk due diligence.”  Only another 27.8 percent said that they are “considering” the inclusion of such risks within due diligence.  The remaining 25 percent of the respondents, then, said simply: No.  It appears not to be on their radar at all.

Be Sociable, Share!


Leave A Reply

← Post AXA Rosenberg/SEC Enforcement Action Model Risk Benchmarking Survey Love 'em or leave 'em? It's hard to tell how institutions feel about hedge funds →