Money Stuff: ChatGPT Can’t Pick the Stocks

I often write around here that there are two fundamental ways to use artificial intelligence to trade stocks:

1. You build a machine learning model for stock prices, you train it on historical data, and you use it to predict future stock prices. You buy the stocks that it says will go up.
2. You subscribe to ChatGPT. You prompt it, like, “You are a brilliant hedge fund portfolio manager. What stocks will go up?” You buy the stocks that it says will go up.

The first model is, approximately, a “quant hedge fund.” People have been doing some form of this for a long time, and some of them ( Renaissance Technologies comes to mind) have had a great deal of sustained success. This model works. Not always, not for everyone, and you might reasonably worry about risk or crowding or regime changes or whatever. But hiring really good machine learning engineers to build models to predict stock prices makes sense both theoretically and empirically.

The second model … I mean, when I started writing about these two models, I was joking about the second model. The second model is a joke! Come on! Like:

• Training a model on historical financial data to predict future financial data makes sense. Training a model on a corpus of, like, books and Reddit to write coherent prose, and then asking it to predict future financial data, is a non sequitur. “If we use all of the world’s electricity and microchips, we can train a model that is smarter than the smartest Ph.D.,” but the smartest Ph.D. is not necessarily good at picking stocks! In some obvious sense an omniscient superintelligence should be good at picking stocks, but that seems like way too high a standard for picking stocks. Just, you know, regress some historical stock prices against some historical data and see what you get.
• You can get ChatGPT for like $20 a month? And everyone does? And everyone is like “hey ChatGPT what stocks should I buy”? Why would you expect to get alpha from doing an easy thing that everyone else does? 

But, though I was joking about this, nobody else is. People keep trying it! For intuitive reasons: It is hard to build a machine learning model to predict stock prices, but it is easy to ask ChatGPT what stocks to buy. The frontier AI labs have done the hard work, and now by far the most accessible way to use artificial intelligence to pick stocks is by asking a chatbot. So people do. As far as I know, no big hedge fund actually has a strategy of “we ask ChatGPT what stocks to buy and then buy them,” but we are getting perilously close. Anyway Bloomberg’s Justina Lee reports:

AI isn’t ready to replace your fund manager — and the public experiments testing it are showing why.

Across a series of new trading contests between the world’s leading AI models, the verdict so far is unflattering. Most of the systems lose money. They trade too much. They make wildly different decisions when given identical instructions. And no one yet knows if these shortcomings will fade with more powerful iterations — or if they reveal something fundamental about the gap between large language models and how markets actually work.

Take Alpha Arena, run by tech startup Nof1. It pitted eight major frontier AI systems — including Anthropic’s Claude, Google’s Gemini, OpenAI’s ChatGPT and Elon Musk’s Grok — against each other in four separate competitions. Each was handed $10,000 per contest before being turned loose on US tech stocks for two weeks. The challenges involved trading on a variety of signals, acting defensively, reacting to the competition, and using high leverage.

The portfolio as a whole lost about a third of its capital. Across all 32 sets of results, a model finished in profit only six times. …

“LLMs can’t really make money by themselves,” said Jay Azhang, founder of Nof1. “You need basically a very sophisticated harness and scaffolding and data platform in order to even give them a chance.”

LLMs are good at doing research and finding and deploying the correct tools for certain tasks, he said. But they don’t yet know how much each of the many variables that swing stocks — including things like analyst ratings, insider transactions, and sentiment shifts — actually matters. They tend to mistime their trades, incorrectly size positions and buy and sell too often.

It would be crazy if it was otherwise! If you could just go to ChatGPT and type “hey tell me what stocks will go up” and it told you, then everyone would do it, and how could everyone beat the market? I definitely see the intuition here — “if AI agents can do a lot of the work of lawyers and accountants and marketing consultants, why can’t they do the work of hedge fund investors?” — but it has to be wrong. The work of investors is fundamentally adversarial; we can’t all beat the market. [1]

Meanwhile, at the Wall Street Journal, Gunjan Banerji asked ChatGPT for investing advice and got, you know, investing advice:

ChatGPT lucidly laid out the risks associated with any trade war. It suggested making shifts to my portfolio, such as potentially adding bonds or, once again, options.

It also spat out a list of companies including defense stocks like Lockheed Martin it thought might outperform. The basket has risen around 5.5% since mid-October, lagging behind the S&P 500’s roughly 8% gain.

This was market timing again, something advisers didn’t recommend. …

At times, it felt like ChatGPT responded with what I wanted to hear, which would be risky if I were putting real money to work.

Right, like, ChatGPT’s job is to (1) convincingly mimic what a human would do in the relevant situation and (2) keep you engaged and entertained and feeling good about using ChatGPT. If you asked a financial adviser for trading advice, and she said “buddy just put the phone down and take a walk, you don’t need to do anything about a trade war,” that would be probably very good advice but also sort of not what she’s there for. She’s there to make you feel good about yourself and about the service she provides. Not answering the phone is often the best service a retail adviser can provide, but it doesn’t feel that way. What feels like good service is doing stuff. Doing stuff maximizes ChatGPT’s reward function, and probably your financial adviser’s reward function, but it probably does not maximize your portfolio.

Also, just like your retail broker, ChatGPT has no earthly idea what stocks will go up, come on. If your retail broker knew that, she’d be managing a hedge fund. If ChatGPT knew that, it would be managing Renaissance.

I have been more or less entirely won over to Cliff Asness’s view that not having real-time prices is a feature of private markets, not a bug. Some days the stock market goes up and some days it goes down; your stock portfolio probably goes up in the long run but with a lot of fluctuations along the way, and each time it goes down you are sad and nervous and inclined to sell. Meanwhile your portfolio of private investments pays you some steady money and rarely changes price; when it does, it’s usually because the price has gone up. Far more pleasant.

Of course intellectually you might say “this portfolio of private stocks has the same fundamental economic risk factors as the public stock market, so if the public market is down 1% today then my private stocks would probably fetch 1% less today too.” But nobody makes you think about that, so mostly you don’t.

For long-term private investments this is simply a pleasant situation. You invest in a startup or a private credit fund, you wait a few years, eventually the startup goes public or the fund returns capital, and you cash out at a large profit. The value of the startup or the fund in the intervening years simply does not concern you very much: You made a long-term investment, you had no plan or ability to cash out before it concluded, so getting a daily statement of its value would be pointless. You cash out at the end and are happy.

But for short-term private investments it is untenable. If you invest in a startup or a private credit fund thinking “I’m gonna park this money here for a bit but I might change my mind at any time,” then you need to know the price each day so you know whether to change your mind. If you buy at $40 and the price goes to $45 next week, you might say “hey that’s a nice quick profit, I’m selling.” If you buy at $40 and the price goes to $38 next week, you might say “ooh I can’t sell here, that would mean taking a loss, I’m sure it will rebound.” Or vice versa or whatever. The important thing is that, if you do sell in a week, you have to sell for a price. You need to know what the price is, and whoever is buying from you — maybe a secondary investor, maybe the fund — has to pay you that price. The fund or startup can’t just raise money once at one price, wait five years, and conclude the trade at another price. It needs to have lots of interim prices along the way.

Historically an important distinction between public and private markets was precisely that private markets locked up investor capital for the long term, and “I’ll park money in a private asset for a week” would have been an absurd thing to think. But private markets are the new public markets, and alternative asset managers are trying to sell private equity and private credit to retail investors who want daily liquidity. Which means that private assets also need daily prices. So:

Apollo Global Management Inc. said that more than $830 billion of its credit assets will be priced daily by the end of September, a move to bring more transparency to opaque credit markets.

The firm, which manages more than $1 trillion in assets, announced the initiative on its first-quarter earnings call on Wednesday. Apollo has been increasing its efforts to provide liquidity and price transparency in the $1.8 trillion private-credit market, where assets don’t typically change hands, Bloomberg News previously reported.

“That essentially means the totality of our credit business will be 100% daily pricing,” Apollo Chief Executive Officer Marc Rowan said on the firm’s first-quarter earnings call with analysts.

If you are a retail investor in an Apollo business development company, that is probably helpful. If you’re an institutional investor in private credit funds, though, being blithely ignorant about the value of your portfolio was kind of nice, and now you are stuck knowing the price.

The way secured lending works is that I lend you $100 and you give me, say, $105 or $120 or $200 or $500 worth of your stuff as collateral. When you pay me back the $100 with interest, I give you back your collateral. The collateral is generally worth more than the money I lend you, because:

1. I want to protect myself from the risk that you don’t pay me back, the value of the collateral goes down, and I end up selling the collateral for $90. (These risks are correlated: You don’t pay back your mortgage at the exact time house prices collapse, you don’t pay back your stock margin loan at the exact time stock prices collapse, etc.) So I take extra collateral as a cushion against that risk.
2. You came to me. Like: I am in the business of lending money to people against collateral, and you need the money. I don’t need the collateral. So if you want $100 and I say “give me $200 of collateral” you’ll be like “yeah that’s fair.” Whereas if you give me $50 of collateral and say “lend me $100 against this” I’ll be like “what, no, I don’t want that.”

Occasionally this is reversed. Treasury repurchase transactions are often roughly “you borrow $100 of cash secured by $102 of Treasuries,” but sometimes they are “I borrow $100 of Treasuries secured by $102 of cash,” because I might need the Treasuries (to sell short) more than you need the cash. You will lend me the Treasuries, but you want to protect yourself against the (correlated) risk that (1) I won’t give them back and (2) Treasury prices will go up and you’ll end up having to buy them back for $104. 

The thing to notice is that the risk is symmetrical: If I lend you money and you give me collateral, I take the risk that you won’t pay me back, and you take the risk that I won’t give you back your collateral. But the transaction is, almost always, asymmetrical: I only give you $100 of cash, but you give me like $200 of collateral.

In the real world, part of the reason for this is that you “give” me the collateral only in some approximate sense. If I give you an $800,000 mortgage loan against your $1,000,000 house, you don’t give me your house to keep in a drawer. You live in the house. If you don’t pay me back, I can seize and sell the house, but it will take a long time and be a huge pain. And if you do pay me back, there is no practical way for me to run off with your house. [2]

But in modern financial markets, things are closer to the theoretical ideal. If I give you a $100 margin loan against $200 worth of stock, I do “take” the stock in a somewhat real sense. I can “rehypothecate” it, giving the stock to someone else who lends me money against it. If you don’t pay me back, I can blow out the stock in minutes. I have fairly real control of the collateral.

And so a trade that might occur to me is: “Wait, I gave you $100 for $200 of stock. Good trade for me. Later, you will give me back $100, and you will expect me to give you back $200 of stock. But that’s a bad trade for me. What if I just don’t do that? What if I just keep the $200 of stock? You can keep the $100, whatever. Then I’d have $200 of stock for which I paid $100, which is a good deal.”

This is not in fact a very good trade, because I’d go to jail, but still, you can see how someone might come up with it. The Financial Times reports:

An indictment unsealed on Tuesday alleges that Val Sklarov, a Ukrainian-born American, ran an “Astor”-branded company that promised a cash loan to an unnamed victim after inducing the victim to pledge as collateral hundreds of millions of dollars in shares that were then sold.

It comes months after the FT published an investigation into a dispute between Mexican billionaire Ricardo Salinas Pliego and Sklarov.

The indictment, filed in the Southern District of New York on April 30, matches Salinas’s allegations to the FT about Sklarov’s business. Astor Asset Management in July 2021 promised Salinas — whose Grupo Salinas has interests spanning media, telecommunications, banking and retail — a cash loan of up to $150mn secured against shares in his company Elektra worth roughly three times that amount. ...

“The guy took my stock, sold it, and gave me the money as a loan,” Salinas, who took out the loan in order to buy bitcoin, had told the FT. “Jesus, that’s as bad as it gets.” Sklarov had denied Salinas’s allegations, saying: “I certainly do not consider myself to be a fraudster, but there’s a saying: ‘It takes one to know one’.”

Rather than holding the shares as collateral, prosecutors allege that Sklarov and others sold them shortly after they were deposited with the custodians. Any loan payments to the victim were funded using proceeds from the sale of the victim’s own shares, prosecutors said, while “hundreds of millions of dollars” in remaining proceeds were kept by the scheme.

Here is the indictment, which charges both that Sklarov lied about his own identity and also that “SKLAROV agreed to lend Victim-1 at least approximately $115 million and Victim-1 agreed to post the Company Shares as collateral, on the understanding that the Company Shares would not be sold or transferred absent a default.” But he then allegedly sold all the shares — some $450 million worth — immediately, used some of the money to fund the loan, and kept the rest.

Libor, the London interbank offered rate, was for a long time “the most important number in the world,” the main reference interest rate for loans and derivatives. Libor was calculated by asking a panel of banks what rate they would have to pay to borrow money in the London interbank market. It collapsed in scandal after the 2008 financial crisis. For our purposes there were two big problems with Libor:

1. The simpler problem is that the interbank lending market that set Libor was not that active, so that by 2008 Libor was “the rate at which banks don’t lend to each other.” Banks had to guess at their Libor submissions, because they were not actually borrowing in all of the tenors and currencies that Libor used.
2. The subtler problem is that Libor was narrowly a measure of bank borrowing costs, but was loosely a measure of “the cost of money” generally. Libor was “the interest rate,” and for many applications, if you needed to plug a risk-free interest rate into some pricing model, you’d just use Libor. For much of recent history, the price that banks paid for money was a reasonable proxy for the cheapest price of money; the big Libor panel banks were approximately risk-free. And then in 2008 they weren’t, and Libor went up. And so if some perfectly fine creditworthy company had floating-rate debt, or an interest-rate swap, its interest rate went up. And the company might reasonably say: “Wait, no, I’m fine; I did nothing wrong; my credit is great. Why do I have to pay more interest just because some banks blew themselves up?” Libor was in an important sense the wrong index. It tied trillions of dollars of economic activity to the idiosyncratic risks of a handful of banks that everyone had casually assumed were risk-free benchmark borrowers, but that turned out to be quite risky.

Here’s a story about shipping charters:

Mercuria Energy Group Ltd. is suing the Baltic Exchange, Bloomberg reported on Thursday, alleging that it has suffered losses in the hundreds of millions of dollars as a result of distortions in a key benchmark for the cost of shipping oil from the Middle East to China. …

At the heart of the dispute is the rate for hiring a tanker to ship oil from the Middle East to China, which for years has been calculated based on the cost of shipping oil from Saudi Arabia’s Ras Tanura port. Crucially, it is located inside the Persian Gulf, and can only be accessed via the Strait of Hormuz.

Each day, shipbrokers convened by the Baltic Exchange submit their assessments of the price, which are used to create a rate known as TD3C. That rate is then used across the global oil industry as a benchmark to set the rate of tanker contracts, as well as to underpin a multibilllion-dollar derivatives market.

The Baltic Exchange, which is now owned by Singapore Exchange Ltd., stirred controversy in March when it told brokers that the TD3C rate should continue to be calculated based on the cost of shipping oil from Ras Tanura — even though shipments had all but ceased due to the war. 

The TD3C rate soared as a result, rising to as much as $600,000 a day compared to more typical rates between $40,000 and $100,000.

For traders and shipowners who have contracts and derivatives positions linked to the index, it has caused turmoil. In one example, Norwegian shipping company Hunter Group ASA said last month that a customer had paid it $8.3 million less than it was due for ship charters during the month of March. The customer involved was Mercuria, and it was refusing to pay charter rates that were pegged to TD3C, according to a person familiar with the matter. 

Right, the benchmark for lots of shipping charter rates is based on a standard shipping product that now (1) rarely trades and (2) is idiosyncratically risky. If you’ve chartered a ship on some entirely different route, you don’t have those idiosyncratic risks. But you might have to pay for them anyway.

I wrote yesterday about investment banking without a license, and I asked:

If I am some random celebrity or politician and I get hired by an investment bank to glad-hand clients, can I glad-hand the clients before passing my licensing exams?

A reader pointed out that the US Securities and Exchange Commission has addressed a related point: In 1991, it allowed Paul Anka to get a finder’s fee for giving a company a list of potential investors (but not contacting them himself), even though he was not licensed as a securities broker. Apparently this exemption — that even without a license, you can collect a finder’s fee for referring a company to investors — is still called the Paul Anka Exemption. So, yes, some amount of random celebrity unlicensed investment banking is allowed.

SEC Proposes to Eliminate Quarterly Reporting Requirement for Public Companies. Nvidia Inks $500 Million Deal for Shares in Fiber Maker Corning. Crypto Exchange Bullish Strikes $4.2 Billion Deal for Transfer Agent in Tokenization Push. Morgan Stanley Debuts Crypto Trading, Undercuts Rivals on Price. Watchdog Calls Out SRT ‘ Circles of Risk’ as Banks Hunger for Default Protection.  BNPL SRT. Victoria’s Secret Rebukes Major Shareholder in Proxy Fight. Viral $100 Kids Landline Startup Tin Can Debuts Bulk Order Program. James Murdoch in Talks to Buy Vox’s New York Magazine and Podcast Division. Is Insider Sports Betting a Federal Crime? The NBA Player and the Gambler Who Needed Him. What Happened When the Pope Had to Call Customer Service.

If you'd like to get Money Stuff in handy email form, right in your inbox, please subscribe at this link. Or you can subscribe to Money Stuff and other great Bloomberg newsletters here. Thanks!