Part 3 – Artificial Intelligence and Intelligence Amplification



Artificial Intelligence and Intelligence Amplification in Financial Markets

Securities Markets are Machinery Now.

This raises the question of how to best participate in the world’s new wired markets. People who use information technology most effectively will be rewarded.

Artificial intelligence (AI) as an academic discipline began at the famous 1955 Dartmouth conference organized by John McCarthy from Stanford University and Marvin Minsky from MIT. The goal of the AI pioneers was to create a mind, a human in silicon. One key idea was that the brain was a biological computer so all the researchers had to do was figure out what the brain was doing and put it into an actual computer and they’d be done. This was something that people thought in the 1950s might take 10 years to accomplish, maybe 15 with long lunches.

So far, it hasn’t exactly worked out. In fiction we have the example of HAL, from 2001: A Space Odyssey. Letting the computer do the thinking turned out badly in that case. HAL discovered the lie in the first reel, and quickly moved on to become a paranoid serial killer.(1)

In financial circles there was a lot of sort of irrational technology exuberance as well. As seen in Chapter 2, there were some inspired magazine covers from the magazine Wall Street Computer Review(2) that showed the unrealistic expectations for AI. One, from 1987, depicts Socrates on the steps of the stock exchange surrounded by a horde of PCs, and touts: “Teaching Computers to Emulate Great Thinkers.”

Source: Wall Street Computer Review (now Wall Street & Technology), June 1987.

Source: Wall Street Computer Review (now Wall Street & Technology), June 1987.

Computers “thinking” like a person didn’t really work out as well as people had hoped. In the academic AI world, the artificial sentients were persistent no-shows. Artificial intelligence got kind of a bad rep on Wall Street as well. To get into the game, you had to buy a LISP (list processing) machine, a $100,000 machine that ran the elegant but arcane AI language of choice, LISP, and related expert system tools with names like Automated Reasoning Tool (ART) and Knowledge Engineering Environment (KEE).

You could prove a theorem or two with this stuff right out of the box, but getting a price for IBM on the screen proved to be a prodigious amount of work.

As recounted in the Introduction to this book, I worked for one of the companies that made these machines—there were two, LISP Machines and Symbolics—and came to the realization that building from the high-concept top down was a poor idea. Here is a picture I used back then to show the user reaction to AI investment systems, showing a mix of fear and rage.

Chapter 7 – A Little AI Goes a Long Way on Wall Street, describes a successful application, built in the early 1990’s, which was much more modest in scope than bringing back the great thinkers. The goal was to get computers to be decent users of other computer systems, which were overloading their users. Recently, this idea has been called intelligence amplification (IA). The systems here were market data and execution systems, and the combination was an early version of algorithmic trading.

The AI/IA-flavored approach to algo trading and market surveillance used in QuantEx and MarketMind, described in Chapter 7, was very effective in many contexts—equity strategies, quant option trading, and monitoring liquidity— but, like spreadsheets, they were clever ways to get the computer to do what you said, and didn’t have anything in the box that would let the machine learn what you should have said and adjust its behavior or models to better meet your needs.

One of the early enthusiasts for these technologies was Henry Lichstein, an MIT graduate then serving as technology adviser to John Reed, chairman of Citicorp. Henry was also on the board of the Santa Fe Institute (SFI),(3) a new high-powered research institute interested in complex interactions of complex entities. SFIdid a great deal of computer experimentation in artificial life (ALife).

The ALife people had much more modest goals than AI; all they wanted to do was build software entities that displayed lifelike behavior. And their efforts were met with early success. A bunch of random ALife birds might fl y in random directions, and not act much like birds. But giving them a few simple rules, like “fl y toward the closest bird,” “go with the flow,” and “don’t hit other birds,” could give rise to distinctly birdlike behavior.

The original ALife flock is Craig Reynolds’ “Boids,” done at Symbolics in 1986.(4) Simulated herding and fl ocking turn out to be of some commercial interest. Those massive stampedes in Disney cartoons, with thousands of , and all those schools of talking fish are descendants of the Boids. Hollywood showed its appreciation by giving Reynolds an Academy Award in 1998.

Like our early algos, the SFIartificial life was dumb, just obeying a few simple handwritten rules, without any ability to learn from mistakes. But the ALife researchers had a good answer—evolving intelligent behavior by mimicking natural evolution. Represent the programs as digital chromosomes, and simulate crossover and mutation, to breed better programs. The original version was called the genetic algorithm (GA), which later evolved into evolutionary computation (EC). Henry Lichstein suggested this approach for use in trading around 1990. The results for ALife had been striking. All sorts of tasks could be learned by the programs operating in an electronic world, which sure sounded like quant investment and trading. ALife programs evolved behavior to solve problems, avoid obstacles, find rewards, and cooperate. These are remarkable to watch.(5)

Chapter 8 – Perils and Promise of Evolutionary Computation on Wall Street, is one of the few detailed descriptions of the use of the GA and EC in finance. Actual chromosomes and fitness functions are included. If that last sentence made your eyes glaze over, skip to Chapter 9. It was satisfying research, and the power of evolutionary search truly teaches you to be careful what you ask for. Evolutionary approaches could be given rules to avoid many sins of data mining, and had many appealing features. As much as I would like to see this work, we don’t see a torrent of EC scientists fl owing toward greater Wall Street. We don’t even see a trickle. Olsen & Associates was a Zurich-based currency trading firm, founded in 1992, heavy with physicists from the Eidgenössische Technische Hochschule (ETH), Einstein’s alma mater. They were keen on genetically adaptive strategies and well funded, but vanished, and few of the principals are still keen on genetic algorithms.

After sending the GA to the back of the breakthrough line in the previous chapter, in Chapter 9 – The Text Frontier, we get to using IA, natural language processing, and Web technologies to extract and make sense of qualitative written information from news and a variety of disintermediated sources.

In Chapter 6 – Stupid Data Miner Tricks, we saw how you could fool yourself with data. When you collect data that people have put on the Web, they can try to fool you as well.

Chapter 10 – Collective Intelligence, Social Media, and Web Market Monitors and Chapter 11 – Three Hundred Years of Stock Market Manipulations, include some remarkable and egregious examples.

>>>>>> READ MORE HERE < <<<<<<

Notes for this section about Financial Artificial Intelligence, Intelligence Amplification, Genetic Algorithms, and Evolutionary Computation in Securities Markets:

1) Sci-fi buffs have a rich amount of material to consider in this context. Vernor Vinge, a computer scientist who has also won five Hugo awards, deals with the topic in much of his work, including his latest novel, Rainbow’s End. If we construct an artificial super-intelligent entity, what will it think of us?

2) Source: Wall Street Computer Review, June 1987. Since renamed Wall Street & Technology, and a useful resource for nerds on Wall Street at www.wallstreetandtech.com.

3) “The Santa Fe Institute is devoted to creating a new kind of scientific research community, one emphasizing multidisciplinary collaboration in pursuit of understanding the common themes that arise in natural, artificial, and social systems.” It was founded in 1984 (www.santafe.edu/).

4) … and still flocking after all these years at www.red3d.com/cwr/boids.

5) Until they start barcoding these into books, printed URLs are annoying, but it is actually worth typing these in. Karl Sim’s MIT video is here: www.youtube.com/watch?v=F0OHycypSG8.

Wall Street Analytics

Easily find all of the
lowest cost no load
mutual funds and ETFs
Available at major ebook stores:

Amazon -- Kindle/MOBI
index funds list investing books

Apple iBookstore -- iPad/EPUB
index mutual funds investment guide

Barnes & Noble -- Nook/EPUB
no load mutual funds investment guide

Smashwords -- EPUB, MOBI, PDF
low cost mutual funds investment guide

no load index mutual funds
by David Leinweber