Invisibilia: Do the Patterns in Your Past Predict Your Future? | Artificial intelligence
Welcome to Invisibilia Season 4! The NPR program and podcast explores the invisible forces that shape human behavior, and we here at Shots are joining in to probe the science of why we act the way we do. In Episode 4, they’re asking: are we destined to repeat our patterns or do we generally stray in surprising directions? – a question increasingly relevant in an age when algorithms are trying to predict everything about our behavior. Here’s an excerpt from the episode.
On paper, Shon Hopwood’s life doesn’t make a lot of sense, not even to him.
“I don’t have a great excuse as to why I did these things. And everybody always wants that,” he tells me. “It closes the circle for people. But that’s not really how it happened.”
To the naked eye, it looked like Shon Hopwood was born into a really good pattern. He grew up in the neighborly, low-crime community of David City, Neb., to a great Christian family that encouraged self-reliance. “My parents basically opened the door in the morning and would say, ‘See you in a few hours.’ It was a good childhood.”
Fresh air, loving family, safe community: Those are some pretty good patterns.
But for some reason, in college, Hopwood started veering off the graph. He wasn’t that interested in school, so he dropped out and returned to David City to work, and that was all going fine, until one day when his friend Tom asked him down to the bar for a drink.
“And he just asked me, he said, ‘What do you think about robbing a bank?,’ ” Hopwood says.
“And you know most people would have said, ‘No!’ Or ‘What are you talking about?’ Or walked away, or a million other responses. And my response was, ‘Yes! This is a great idea!’ “
And so Hopwood’s path forked. He would become a bank robber.
Of course Hopwood had second thoughts. He had them right up to the moment he walked into the bank dressed as a handyman. “I walk in the bank and I pull a mask up and I drop the tool box on the ground. It makes a huge noise so everyone turns and looks at me. And I unzip my coveralls, pull out a 22[-caliber] rifle and yell, ‘Everyone get down. This is a robbery!’ “
After that Hopwood recruited a small group of friends and just hit one bank after another until one day, four guys from the FBI tackled him. He was sentenced to 12 years in prison, which was bad enough, but what really cut was that a bunch of people in his hometown disowned his completely blameless parents.
“They have to have some reason for why I did these things, because otherwise it just doesn’t compute. Because it didn’t make any sense that me and the people that were involved with me had robbed these banks. I mean one of my co-defendants was the son of the town attorney,” he says.
We need to find a predictable pattern, and when it eludes us, we ache for it.
Now, because we live in the age of computers, our ability to discern patterns has expanded. Today a computer can scan more data in a minute than you or I could sift in a lifetime, and in that data see things we could never see. Beautiful things and horrible things and even — we’re told — the future.
A competition to find the patterns that determine the path of a human life
About two years ago a Princeton sociologist named Matthew Salganik decided to stage a massive computer competition.
The idea was inspired by Netflix. In 2006, the company distributed huge amounts of user data to programmers all over the world so they could write computer models that found hidden patterns that improved its predictions for which movies subscribers would like. The competition totally worked out for Netflix. The companies movie choice predictions substantially improved.
So Salganik’s plan was to do the exact same thing, except instead of staging a competition to improve predictions of movie preference, he wanted the competition to improve predictions about the things sociologists cared about: high school GPA, which child would persevere when faced with adversity, who would become homeless. Could we, he wondered, harness the pattern-finding abilities of computers to discover new things about how individual lives turned out?
“Looking at lots of people and looking at broader patterns helps us have a fuller understanding of what’s possible,” Salganik says. If his competition worked well, it could make the world a better place. After all, if computers could locate the things that predicted stuff like higher grades, policy makers could design better interventions.
So Salganik set to work. He got a massive trove of data on 5,000 kids who had been followed from the day they were born, then made that information available to data geeks and researchers across the globe. Four hundred teams were given incredibly detailed information about the kids from birth until age nine, then told to predict their grades — and a handful of other outcomes — at age 15.
One day last fall, Salganik sat down to crunch the numbers, figure out which models were best able to predict where the children in the study had ended up, and what he found deeply surprised him.
What Salganik wanted to see was at least one computer model entry able to predict with reasonable accuracy the outcomes of each child in the study.
But none of the computer models did as well as Salganik expected.
If they had, the screen in front of him would have been filled with tall, colorful towers — bars stretching from the floor of the y-axis to the top, indicating that the predictions had gotten close to 100 percent accurate. Instead what he saw was a bunch of squat bars crowded around the bottom like flattened mushrooms, indicating that the predictions were a lot closer to 0 percent accurate than 100 percent.
“I would say this is not impressive,” he tells me as he looks at the graph. “I think this is sad. Disappointing.”
But was it? Or was it just an accurate representation of how unpredictable our individual lives are?
Duncan Watts works at Microsoft Research. He does computational social science, including prediction studies similar to the one that Salganik was doing. In fact over the years, Watts says, he’s done tons. He says when it comes to predicting stuff like what will happen in a particular human life, Watts thinks the outcome that Salganik found is just the outcome.
“We find exactly the same pattern everywhere we look… when you’re talking about individual outcomes, there’s a lot of randomness,” Watts says.
“And the other half of this conversation is that people don’t like that answer, and so they keep wanting a different answer. They say nature abhors a vacuum. Humans abhor randomness. We like deterministic stories,” Duncan says.
We like the idea that patterns can tell us what will happen in life because that idea makes us feel more secure he says.
“If you think you can predict things— even if you’re wrong — it means that you get up in the morning and you feel confident,” Watts says. “And so you can invest your time and energy today in things that won’t pay off until tomorrow.”
Watts believes being more realistic about patterns is important but hard to pull off because it involves accepting something that feels like a contradiction: that patterns are important and predictive — you can identify things in the lives of kids that tend to help or hurt them — but you can’t say for sure whether those same things would influence the life of any individual kid because randomness has a lot more power over complex things like life than we like to think.
Randomness, Watts says, is as important as pattern, though obviously it doesn’t get the same respect in our algorithmic world.
A life that no one could predict turns again
Which brings us back to the unusual trajectory of Shon Hopwood.
Hopwood was working in the prison law library, checking out books, when a fellow prisoner asked him for help with his case. The guy wanted help to draft a petition the U.S. Supreme Court.
Hopwood had never studied law and only had a high school education but he wanted to be helpful so he spent two months working on an argument, then sent off the petition and basically forgot about it.
“Then one day I’m walking out to the recreation yard at 6:30,” Hopwood says, “and a friend of mine comes running and screaming out of the housing unit. And this being federal prison my first thought is, ‘What did I say to this guy yesterday that he wants to come and fight me at 6:30 in the morning?’ “
The Supreme Court had accepted the appeal he had written.
That highly unusual event led to other unusual turns in trajectory, until one fine day, Hopwood found himself moving a box of belongings into a small office at Georgetown University Law School.
The bank robber had become a law professor.
Proof that even though we yearn for predictable patterns, no matter how many computers we have, it might just be impossible to predict where any single life will go.