For example, you could compare the long portfolio performance vs. the performance of all the stocks in the market that increase, i.e., all the opportunities that were available to make money on the long side. And you could measure the short portfolio performance vs. all the short-able stocks in the market that decreased, i.e., all the opportunities to make money on the short side. And of course measure both over a particular time period.

This way managers would be compared against their investment strategy goal. No manager would be expected to actually achieve this perfect portfolio of longs or perfect portfolio of shorts, but managers would be measured on the basis of relative underperformance against perfection. Kind of like measuring how close they get to the bulls-eye.

I’m oversimplifying here, but you get the idea.

I think there would be a lot more information about manager risk-taking, strategy, and skill contained in this type of analysis than in benchmarking to traditional market indices.