August 11, 2008
Big Problem or Right Problem? The Egg Freckles Saga.

Have you ever spent hours trying to solve a problem only to find you've been working on the wrong problem? Try doing it for five years. That's what Apple Computer engineers did with the Newton handheld computer over a decade ago.

From 1993-1998, Apple made a valiant effort to break open a market for portable handheld pen computers. Unfortunately, they spent most of that time working on a problem that didn't really exist for consumers. And as they labored at it, their intended market was stolen by Palm Computing's PalmPilot.newton130x.jpeg

What follows is a tale about a fatal assumption -- an obsession with a Big Problem that led to one of Silicon Valley's great product misfires.
Consider the moral first.


Solving a Big problem doesn't mean you're solving the Right problem.

Apple's team chose to tackle the biggest challenge in pen computing: high-level handwriting recognition. Newton would be the first portable computer people could write on directly using their natural hand. From anyone's scrawl, the computer would extract the standard ASCII characters computers need to work with. This posed a massive challenge in pattern recognition. Since every user's handwriting is different, the Newton would need to learn the particular way its user wrote each letter and number. IF it got all the letters in, say, the word "thing" right, Newton would compare that string of letters to words in its 10,000 word native memory. IF the word "thing" was stored there, Newton would find a match and "know" the word.

The Newton team was determined to build the world's most sophisticated pattern learning pen computer. But why were they doing it? And for who? Here they made one fatal assumption about their potential buyer, an assumption that would seal the Newton's fate.

The assumption went something like this:

"Users want to do things the way they've always done them. The user shouldn't have to learn anything new to adapt to a machine. A smart machine can and should adapt to the user (in this case, learn the user's handwriting)."

This assumption became a frame and the frame became a mindset. Without ever turning back to question their customer premise, Newton's team labored to build a noble, mind-blowing machine that could recognize the diverse scrawls of any and every human on Earth. But was this the Right Problem to solve?

When the Newton Message Pad debuted in 1993, its handwriting recognition fell way short of the mark, and a public drubbing ensued. The Doonesbury comic strip showed a character writing a six-word sentence on a Newton-like hand-held. The unit coughed up "Egg freckles?" Then The Simpsons piled on. The world laughed.

All through 1993, the Newton was skewered in the press. In October of that year, Apple CEO John Sculley left with freckled egg on his face. Humiliated, the Newton team redoubled their efforts to solve their core problem: getting Newton to learn better.

At the heart of Newton's learning challenge was the "second-stroke problem." Each time a user's pen lifted off the tablet and set back down, Newton's brain detected a pause and became uncertain. "What did that pause mean? Is this next stroke part of the current letter, or a new letter or word?" As it turns out, many alphabet characters need multiple strokes, leaving plenty of room for uncertainty. Capital "T" and "X" involve two strokes. "H" needs three. Add user hesitancy and writing quirks, and you have a thorny problem. And that's just English. Try Cyrillic or Japanese ideograms.

Because Newton's recognition engine was unsure so often, it routinely threw a list of possible words at the user. This was both inconvenient and embarrasing. Who wants their computer to say, "I'm confused. Take time out, scan these words and select the right one"? Worse, if you wanted Newton to learn a word outside its native 10,000 word database, you had to train it. You first had to write it your way, then type it letter by letter using an on-screen keyboard. All that to tell Newton, "This is what 'Hoboken' looks like when I write it."

The upshot? To "save" users from having to adapt their writing habits to machines, the Newton subjected ordinary people to drawn out and repetitive clarification and training routines; a tacit admission that Newton wasn't doing its core job cleanly.

None of this was lost on Jeff Hawkins, inventor of the Palm Pilot, who was carrying around a wooden block as a pretend pocket PDA and using a whittled down chopstick as a pen to imagine his interface.

Hawkins never lost sight of what consumers would want most in a pen computer: fast writing and true mobility - something they could fit in their shirt-pocket. He cut to the chase and questioned Apple's core assumption:"Why must the computer learn everything? Why can't users adapt? Why build a sophisticated learning machine at all? Let's get the job done. People learn faster than computers, so why can't people help the machine? People could easily get the hang of a new single-stroke alphabet. Hmm. One stroke per character and presto! No more second-stroke problem."

So that's what Jeff Hawkins did. With his Grafitti language, he simply redesigned the alphabet, turning centuries-old letters and numbers into single-stroke symbols that mostly kept the look of the original characters. Suddenly the computer had only one master rule to follow. "When the pen lifts up, the character is done. When the pen comes down again, it's a new character. Want to end a word? One stroke makes a space." Simple. And while we're at it - since each stroke is a new character, lets not even write along a line. Write letters on top of each other, in the same input space, and let them display as type in another. Presto - a smaller screen.

Hawkin's low-tech solution made Palm Pilot's pen input "good enough." (Apple even licensed Grafitti in 1995 as an input option for the Newton. Some say it kept the Newton alive.) But the real power of Grafitti was size. It shrank the screen, which shrank the box, which created a viable pocket-PDA market.

In March, 1996, when Newtons were selling as digital writing tablets for up to $1000, the first pocket-sized PalmPilots debuted for under $300. A million of them sold in the first 18 months. The Newton team countered with a much improved Newton 1000 and 2000, but by then it was too late. Two years after the PalmPilot was released, Apple cancelled the Newton product line on February 27, 1998. The project had cost the company half a billion dollars.

Hawkins "technology" was a low-tech workaround; it wasn't "handwriting recognition" in the high-level MIT sense. But while PhD's may have felt Grafitti was a cheat, ordinary people, not giving a hang about the technology issues, found PalmPilots handy and useful. While engineers rallied around solving the Big Problem, consumers swarmed to buy the solution to the Right Problem, which started with a chopstick and a block of wood.

By year 2000, Palm owned 70 percent of PDA sales and had sold well over five million units. At the peak of PDA use, white boards everywhere were covered with Grafitti symbols, which many considered faster to write for high-velocity brainstorming.

The Newton team spent five years working on the Big Problem, writing and rewriting untold lines of code to create a learning machine for the existing alphabet. Hawkins spent a few days designing a new alphabet any computer could easily understand.

Despite its truly impressive interface, Newton stumbled at the main task it promised to do - turn writing into standard ASCII characters quickly. And why did Apple paint themselves into this corner? Because they assumed consumers would want their handheld to adapt to their personal way of writing. Instead of biting into Apple's Big Problem, Jeff Hawkins assumed people would adapt. As he once put it, "It takes you weeks or months to learn how to type, so why not spend 15 minutes learning [how to talk to a computer] with a pen?"

The Lessons

In hindsight, Apple's underlying user assumptions made little sense. What makes people's standard routine (handwriting) so sacred? Who said people shouldn't adapt to machines? Who said you had to work with the existing English alphabet? Why make a program strain to recognize every possible variant of every letter and number? Who said your program had to recognize scrawled words by finding them in a limited word database? Engineers set up these problems, not users.

Great minds often get hijacked by their own brillliance and vision. They forget that simple is smart, dumb is basic and low-tech often beats high tech. We can get so obsessed with an elusive quarry and so enamored of our intelligence that we never go back up to the 20,000 foot level and see that we're hacking the wrong problem. The famous monkey trap metaphor is worth repeating here.

If a monkey reaches through a hole for a banana, but the hole is too small for her hand to withdraw with the banana, she's presented with a quandry. "Which do I want? - the banana or my freedom?" All she has to do is let go of the banana in order to be free of the trap. But the monkey doesn't let go of the banana. She sits there determined to extract it, even in the face of being captured.

Big Problems are like monkey traps. If your Solution quest starts feeling "heroic," or your Big Problem is "big" mostly because everyone is trying to solve it (big kudos await if YOU solve it), its likely you're trapped by the epic magnitude of your quest. In that mindset, the simplest options are likley to escape your notice. Check to see if your solving the Right Problem by running your mind through the following four steps:

1. Restore objectivity. Take time off and come back fresh later. Sleep on it.

2. Once you're fresh, carefully and slowly go over your assumptions about the people who will use you product or service. Put yourself in their shoes. Separate your needs from theirs. Don't underestimate their intelligence or overestimate the rightness of your point of view. Break down every assumption you have about your prospective buyer and question it.

3. Especially question your assumptions about what your "users" expect. Often they don't know what they want. They rarely see the next development much less have an opinion about it. But they are ready for a surprise, a break in routine, a new challenge. Keep in mind that IF the payoff is strong, humans will learn new tricks. Are student drivers motivated learners? You bet.

4. Review your supposed technical limitations, challenges or goals to see if you can use lower-tech or human-scale solutions. Stretch for new metaphors that can change the problem, shift the frame, reverse figure and ground.

5. Simplify. Simplify again. Keep simplifying.

Whenever you're stuck or breathing hot and heavy about a solution, you're too close to your work. It's time to step out of problem-solving mode and reassess the problem you're trying to solve.

This excerpt is from the author's book-in-progress, Big Problem or Right Problem? Innovating For Real People.

Copyright © 2007 Tim Moore. All reproduction rights except blog linking are reserved.

Posted by Tim Moore at August 11, 2008 02:03 PM

Comments

Fascinating story - something we all need to be aware of - what are our assumptions? Are we looking to solve the big problem or the right problem?

Posted by: Matt at August 31, 2007 08:12 AM

I ve heard a similar story about the problem of using a pen in outer space. It goes something like:
Astronauts could not use pens in outer space. The ink would not flow out the right way, because of zero gravity.
One nation spent a lot of time and money to develop a pen that would work in zero gravity.

The other just used a pencil instead.

Posted by: chotu [TypeKey Profile Page] at September 1, 2008 05:44 AM

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


Remember me?


Who Are We?

Idea Champions is a consulting and training company dedicated to awakening and nurturing the spirit of innovation. We help individuals, teams and entire organizations tap into their innate ability to create, develop and implement ideas that make a difference.

Top 5 Speaker
Mitch Ditkoff, the Co-Founder and President of Idea Champions, has recently been voted a top 5 speaker in the field of innovation and creativity by Speakers Platform, a leading speaker's bureau.
Webinars Powered by
Idea Champions University

Webinars for online training If you enjoy our blog, you will love our newly launched webinars! Our training is now accessible online to the whole world.

Visit IdeaChampionsUniversity.com to find out more.
Our New Book

If you want to know when Mitch Ditkoff's new book, "Wisdom at Work", will be published, enter your email address.

Your Email address:
Top 5 Speaker
Mitch Ditkoff, the Co-Founder and President of Idea Champions, has recently been voted a top 5 speaker in the field of innovation and creativity by Speakers Platform, a leading speaker's bureau.
Awake at the Wheel, Book about big ideas If you're looking for a powerful way to jump start innovation and get your creative juices flowing, Awake at the Wheel is for you. Written by Mitch Ditkoff, Co-Founder and President of Idea Champions.
See Mitch's keynote address Enjoy a 7-minute interview with Mitch at the Ethical Sourcing Forum in NYC: 3/28/11
Featured in Alltop Guy Kawasaki's Alltop "online magazine rack" has recognized Idea Champions' blog as one of the leading innovation blogs on the web. Check out The Heart of Innovation, and subscribe!
Face the Music Blues Band The world's first interactive business blues band. A great way to help your workforce go beyond complaint.

"In tune with corporate America." — CNN

Breakthrough Cafe.
A totally unique brainstorming salon. Great food. Great food for thought. Great people. Collaborate, have fun, get out of the box.

"Inno-waiters With Whine Lists" – The Breakthrough Cafe featured in January 2006 issue of
© IDEA CHAMPIONS