Image: 123RF/ Karel Miragaya
Image: 123RF/ Karel Miragaya

Every month, the U.S. Department of Agriculture puts out a report that says how many soybeans there are. If there are more soybeans than people thought, soybeans will be worth less than people thought, and vice versa, because of supply and demand.

If everyone thought that there were, like, 600 soybeans, and the USDA reports that there are in fact 500 soybeans, then the price of a soybean will rise: If it was $100 a minute before the report, it might be $110 a minute after. (I am simplifying here—these are not the actual prices or quantities of soybeans, and the report is more complicated than this—but who cares.)

This is good. At the new higher price, soybean farmers will want to grow more soybeans in order to make more money, and soybean users will cut back on their soybean use—switching to lima beans, say—to save money. The price signal conveyed by financial markets will direct resources toward their best use, and will make the world more efficient. And so the people who create that price signal should be, and are, rewarded for making the world better.

The mechanism that rewards them for correcting the prices is quite straightforward; in fact it is the same mechanism that corrects the prices: They buy soybeans (or soybean futures) at the old wrong low price, and their buying pushes up the price of soybeans, and they keep buying until the price is correct, which makes them richer, since they now own soybeans that are worth more than they paid for them.

This is a big part of the justification for financial markets: They provide incentives for very smart people to put a lot of time and effort and creativity into making prices correct, because correct prices are a valuable social good, and the people who make them correct can therefore make a lot of money. Intuitively it makes sense that people who put a lot of time and effort and creativity into finding very hard-to-discover information, or making very clever deductions about prices based on that information, would be richly rewarded. On the other hand people who read a public government report about how many soybeans there are, realize that there are fewer than they thought, and rush to buy soybeans—those people are fine, don’t get me wrong, that’s a valuable service too, but they are not working all that hard. You could—and again I am oversimplifying all of this somewhat but not actually all that much—you could program a computer to do that.

The people whose computers are fastest to download and parse the reports make the most money. And the computers can plausibly do this more efficiently than the humans used to

And of course people have, and there are computers that scrape the USDA report every month and compare the number of soybeans in the report to the consensus expectations of the number of soybeans, and then buy or sell soybean futures based on the difference. And because this is quite straightforward to do, and there is potentially a lot of money at stake, a lot of people program computers to do this, and then they compete over who does it best: The people whose computers are fastest to download and parse the reports make the most money. And the computers can plausibly do this more efficiently than the humans used to: Instead of one human buying 10 soybeans at $101, and then another buying 10 soybeans at $102, etc. until the prices are correct, one computer might buy one soybean at $101 and the other computers might immediately react by raising their prices to $110. If everyone recognizes the information quickly, the prices can become correct with very little trading. (In practice there … tends to be a lot of trading.)

One complaint that people have about this is that it creates a socially wasteful arms race: It is good for the prices to be correct five minutes after the USDA report comes out, and maybe (?) it’s even better for the prices to be correct one second after the USDA report comes out, but these computers are competing over milliseconds and it’s not clear that it matters to any, like, actual agricultural user of soybean prices if the prices are correct 467 milliseconds or 468 milliseconds after the report comes out. Nobody plants more soybeans in that extra millisecond. This complaint strikes me as mostly right—though every so often the “socially wasteful” arms race throws off technological innovations that are useful elsewhere—but, like, what are you gonna do about it? In a system that rewards people for getting information first, someone is always going to be first; you can build a hybrid system that rewards people equally for getting information within the same millisecond/second/minute/week as each other, but that seems a bit arbitrary.

Another complaint that people have about it is that it makes markets more volatile and unstable, because the computers are not as wise and prudent as humans and have more hair-trigger reactions; they just read the report and bang out soybean trades, without a deep understanding of the nuances of the soybean business developed from a lifetime of trading soybeans. This complaint is, you know, sure, fine, I guess.

But I think that the main complaint that people really have about this is that the computers are taking jobs from people who would otherwise profit by reading the soybean report faster than other people. Here is a Bloomberg News article about these reports:

The U.S. Department of Agriculture may well be clearing the way for some Wall Street speed demons to trade on market-moving data before others. Abandoning decades of precedent, the agency has decided to only post its reports directly on the web, rather than also release them via accredited media. While that may seem like a democratic move, it actually could set the stage for a winner-takes-all arms race to grab the info first.

The development, announced Tuesday, is the latest saga for crop markets that have increasingly seen high-speed algorithms taking over and running circles around slower human counterparts, a theme popularized by Michael Lewis’s "Flash Boys." …

The irony is that the USDA’s attempt to ensure everyone gets potentially market-moving information on commodities at the same time could actually do the opposite. … The trading firm with the bot that first scrapes the USDA site will have an advantage over everyone else, a head start on placing trades in commodities markets. 

As far as I can tell the old system gave a head start to the trading firm with the bot that first scrapes its purchased news feed (disclosure, Bloomberg sells news feeds), but never mind that; here’s this:

To be sure, there’s nothing illegal about getting publicly available data first and trading on it. And traders have always sought to get an edge. But in this modern era, an edge can consist of getting important data in just a tiny sliver of a second faster than anyone else — even a millionth of a second.

There is I think a bafflingly widespread vague sense that it should be illegal to get publicly available data first and trade on it. There is a sense that the proper way to trade is to get a cup of coffee, sit down at your computer, type the USDA report’s web address into your browser (no autocomplete, type the whole thing), read the report front to back, take some notes, ponder its implications for a few minutes, and then, many minutes after the report’s release, call your broker on the telephone and ask her to buy you some soybean futures. Someone who just programs a computer to scan the report the second it comes out, and to buy soybeans electronically without even reading it to the end, is cheating.

Some of this odd sensibility comes from amateur hobbyist investors who like the illusion that they can beat the market just by native intelligence and reading the same reports everyone else gets. “Shouldn’t there be a level playing field,” they ask, “between me and a giant high-frequency trading firm that invests millions of dollars in technology and employs teams of skilled professionals whose full-time job is figuring out what financial products to buy and sell?” This is mostly dumb, and there’s no more reason to think that there should be a level playing field between amateur and professional investors than that there should be a level playing field between amateur and professional dentists.

A lot of the low-level rewards of just paying attention to the market all day get eaten up by computers whose attention is mechanical and limited but faster; the humans are freed up to perform the higher-level and more creative tasks

But some of it comes from professional investors who think that they should be rewarded for reading and reacting to the soybean report before the amateurs do, and who are annoyed that in fact the computers read and react to the soybean reports before they can. 

This is, I think, understandable. You could have a model of professional investors in which they spend all of their time thinking about the soybean market, and become very knowledgeable about it, and use that knowledge to make trades. Some of those trades are difficult high-risk trades in which the traders’ lifetime of experience and nuanced understanding of soybean ecology give them, they hope, a 51 percent edge in understanding the likely future direction of soybean prices.

Others are complete gimmes: You read the soybean report, see there are fewer soybeans than people thought, and buy soybean futures for an easy profit. A life of only making the difficult 51-percent trades would be financially risky, and also exhausting. The gimmes subsidize—financially, psychically—the hard trades. You can make a sustainable living, and lead a sustainable life, as a deep soybean thinker because sometimes your soybean experience puts you in line to make quick big easy profits. 

And then a computer comes in and takes all the easy trades from you just by being dumb and fast, and leaves you with only the hard ones as your source of income! Of course you are mad.

Of course I am not really talking about soybeans here; I don’t know much about the sociology of soybean traders, but this is all at least as true in, say, market making, or in fundamental equity investing. A lot of the low-level rewards of just paying attention to the market all day get eaten up by computers whose attention is mechanical and limited but faster; the humans are freed up to perform the higher-level and more creative tasks. 

It is hard as a human to make a profit by reading this morning’s USDA report; the only way left to make a profit is by correctly anticipating next month’s report. That is of course how it should be! Taking away the easy rote tasks and freeing up humans for more creative tasks is the point of automation, in any industry. It makes the industry more efficient, if obvious information is instantly incorporated into prices by robots instead of being laboriously incorporated by overpaid humans. 

But if you’re one of the overpaid humans it makes you sad. And this is true even if you spend a lot of your time on the higher-level creative stuff anyway. The easy stuff helped make the hard stuff profitable, and manageable; without it, everything just looks hard.

- Bloomberg