Bad Advice: You Just Need Agile

"Can Agile work for design?" 

When designers ask me this, what they are often actually asking is, "no, seriously, how the f*#k can we design with Agile?" because many of them have gone through crappy experiences where they have felt like victims of an Agile process rollout rather than active participants and beneficiaries of it. Agile has often been a process or change that's engineering-driven rather than design-driven, so how designers feel about it have rarely been considered. This is unfortunate. 

Despite Agile's noble beginnings as a philosophy and approach, in practice I have seen it boiled down to these commandments: 

  • Iterate often and fail fast. 
  • Deliver software as fast as possible. 
  • Aim for minimum viable product. 

I know that they mean to say that you shouldn't be afraid to fail, move fast, and don't over-design or over-engineer. Unfortunately, in practice, I have seen it often devolve to the age-old practice of throwing crap on the wall to see what sticks. If you're writing code that does something, it must mean you're making progress, right? 

"You're doing it wrong."

In theory there is no difference between theory and practice. In practice there is.
— Yogi Berra

I see. I have gone back and reread the Agile Manifesto. I like it and what it has to say. Seriously. However, like many ideologies espoused by a manifesto, it sounds great and perfect in theory, but in practice, when it meets imperfect people and complex organizations, it tends to fall apart. You can say the same thing about any ideology like Capitalism, Socialism, Communism, Libertarianism... Objectivism... whatever. 

All of us, whether working by ourselves or with others, would like to do things better, faster, and cheaper. However, Agile isn't the only way to do it. If you don't know who your customer is, what you should be building, and why, moving faster and iterating quickly will not help you. Instead of saving you time and money, it will end up costing you more in the long run. 

•••••

I worked at Zynga, the social gaming company best known for Farmville, Poker, and Words with Friends. My team was responsible for creating the company's social platform on mobile – something like an Xbox Live or Gamecenter for the Zynga mobile portfolio. Our team believed in our charter and we accepted our responsibility with gusto. We practiced Agile. We took ownership of a live game – Chess With Friends – so that we could test and iterate with real users. We did daily scrums. We used Pivotal. We released MVPs. 

Unfortunately, company leadership never really prioritized investing in building a mobile network, so there was no downwards pressure for the game studios to back our efforts. We had a difficult time getting their support because they wanted data to prove that it worked before they would integrate anything, so we were stuck in a Catch-22 trying to create network effects without a network of games and users. We also had significant technical challenges building something that could scale to dozens of games, both legacy and in development, that had completely different engines, architectures, and systems, and it would have required serious work and investment to overcome. 

If you played Chess With Friends from 2011-2013, you may have experienced some of what we did. 

If you played Chess With Friends from 2011-2013, you may have experienced some of what we did. 

What we had was a small, isolated game with users who were very different from the usual Zynga mobile customer, a legacy codebase, and no top-down pressure or bottoms-up support. We had enough support to build out a full team filled with skilled people across product management, design, development, and testing. We received a lot of encouragement. I would like to say we were able to do a lot in a short amount of time and that we were able to make significant progress given all these challenges, and it might even be true (my ego would feel better about it), but, unfortunately, it clearly wasn't enough. 

Our problem wasn't something that Agile and all our iterations and data could fix. We had fundamental strategy and priority issues, and we weren't going to quickly MVP and incrementally A/B test our way out of it. I'm sure I made a lot of mistakes, especially since this was when I dove into the deep end of product management, but I learned a tremendous amount going through this process. Regardless, we clearly did not accomplish what we set out to do and I can't really point to a product at the end that made the kind of impact we wanted it to. 

•••••

I don't believe that one should look at a process as a place to start. It dictates how without necessarily asking why and for whom. So before you go implement a process or methodology, try to think about it like this: 

  • Who are you, and what do you stand for? 
  • What is the impetus for change, and what is the desired outcome? How are you going to best achieve that change? 
  • Is the product coming out at the end of the process good? I don't mean MVP which tends to mean "mostly stable", "does what the PM/developer says it should do", "passes QA", or "didn't crash when the CEO/product owner played with it." What is your measure of success? 

Product design and development is never as neat and clean as people plan for beforehand or remember after the fact, and anyone that promises a perfect process that involves people with thoughts, ambitions, feelings, and egos is either naive or selling something. 

•••••

When we were designing the Xbox 360 in the early 2000's, there wasn't much talk about Agile quite yet. However, the Xbox team had a different culture and attitude from the Microsoft of the early aughts. We moved fast, experimented, tested often, and pretty much did what we could to design, build, and deliver a product at the end that we could all be proud of.

We understood and bought into the vision. We had a good sense of who we were, and we felt like we knew who we were designing for. We didn't have a step-by-step recipe book or set of guidelines to follow. We used our design judgment as a baseline, but we depended on dogfooding, heuristic evaluations, and in-lab usability testing to validate. It was almost a two year design and development schedule, and over that time we did dozens of heuristic evaluations and over 25 in-lab usability tests of 8-12 participants each using an iterative design-build-evaluate process. Some parts of the product had multiple rounds of testing and iteration, some had light validation testing, and some had none at all. It sometimes got messy, but we were giving it 110%. 

The Xbox 360 console, controller, and dashboard user interface at its launch in 2005.

The Xbox 360 console, controller, and dashboard user interface at its launch in 2005.

We knew we had a special culture that was enabled by everyone, from our leaders to the front line. Design was included and respected, and our processes were established with all the disciplines' involvement. We were physically separated from the rest of the Microsoft corporate behemoth which gave us not only physical buffer space, but philosophical and psychological space as well. It gave us license to do things a bit differently. We may not have created a perfect product, but we felt like we created a magical one.

To this day, I feel that we created a great product and brand because of it. Many of us who were part of this experience consider it an amazing accomplishment to be rightfully proud of. 

•••••

So the next time someone says they have a new process they want you to implement, take a step back and think about it this way: 

  • Lead with culture. 
  • Enable with process. 
  • Prove with product. 

By culture I mean the organization's people, their values, attitudes, and principles. Who are you, and why are you here? What do you want to stand for? Are all the disciplines (especially design!) involved and empowered? 

Then figure out the right process that best enables the people in that culture. If the people know what their shared goal is and they are empowered to figure out how to get there, you maximize the chances of that process having the desired effect. 

And lastly, prove it by delivering a great product. Make sure that how you're measuring success actually encourages people to make the right decisions. No matter how much you say that you have the best team and best process, if the product is crap, nobody cares. I've talked to too many people who tell me they've implemented Agile but can't show me a good product or result at the end. "Oh that's because management did this" or "marketing decided that" or blah blah blah. If any of that is true, then your development process wasn't the problem. 

The goal shouldn't be to implement Agile or whatever process du jour that the manager has read about recently. If Agile is actually the right approach for your team, then go for it, but make sure that you're going into it with the right expectations and that you don't make the mistake of following the letter of the law but ignoring its spirit. 

Your goal should be to enable your team to work together to create a great product and user experience. Create something good, but make sure that you're equally as proud of how you get there. 

Bad Advice: Trust Your Gut

"Trust your gut." 

This sounds great. Empowering, even. I have heard this often in the past, and I may have even given it in moments of excitement. But this is bad advice for many reasons. 

Homer Simpson's flawed and tiny brain. 

Homer Simpson's flawed and tiny brain. 

Our brains are extremely flawed and are prone to lead us astray. The following are just a few of the cognitive glitches that explain why you shouldn't trust your gut.

Confirmation bias is our very human tendency to favor information that supports our existing beliefs and discount those that disagree with them.

How we look for and gather information is biased, and your search engines are doing this automatically. When you do a search, it's more than likely the results are ranked or filtered based on your historical behavior – and you're not even aware of it.  

How we interpret information is biased. Two people who hold different beliefs can see the same exact information and come to different conclusions. 

How we interpret memories is biased. We view the past in the best possible light or least embarrassing way and we selectively remember those things that support what we already believe. 

The sunk-cost fallacy leads us to keep going because we have already invested a lot of time, energy, money, or effort towards something, even when it's irrational to continue. How often have you pursued a design solution beyond the point of reason because you already spent so much time on it? 

When we put too much weight on the first piece of information we get about something or someone, that's anchoring. It's one of the reasons why we dress up when we want to make a good impression, whether it's for a first date or a job interview. 

Due to the gambler's fallacy, we truly suck at estimating the odds. We expect the frequency of things to balance out in the future. If you flip a coin and get heads three times in a row, what are the chances of the next coin flip to be heads again? 

One of my favorites is the above average effect or illusory superiority where we tend to overestimate our positive qualities. Most of us believe we are exceptional; for example, the vast majority of surveyed drivers consider themselves to be of above-average skill. 

•••••

I was a designer on Crimson Skies for the original Xbox game console. You got to fly around in cool planes and shoot stuff. I was primarily responsible for multiplayer where up to sixteen people can play with and against each other, but I also had some responsibility for the single-player campaign mode. 

One of the chapters in the campaign involved you, the protagonist, having to prove yourself worthy. I devised a series of challenges including the Trial of Skill (go to 5:40 in the video above to see this in action) which involved following another plane through a series of obstacles and maneuvers. I spent a lot of time making sure the story made sense, the environment was set up properly, the flight paths and special moves were just right. Sounds fun, right? 

These are just a few of the actual online comments from customers. 

These are just a few of the actual online comments from customers. 

Mea culpa! I probably should have listened to more of the people who said that it may have been a little difficult. I should have ignored the fact that I had spent so much time on it. I should have sought out more opinions from people outside the team who weren't already experts playing the game that we had been building for a couple years.

I listened to my gut a little too much and I honestly feel like I almost broke the single-player campaign mode because of it.

•••••

Good judgment comes from experience, and experience comes from bad judgment.
— Rita Mae Brown or Mark Twain or Will Rogers or Someone else

To have a gut you can trust, you need to have good judgment. To actually gain good experience from exercising bad judgment, you need to be able to critically examine the reasons for failure. You need to work on objectively learning from the experience. 

•••••

I was on the design team for the Xbox 360 platform team (which was a fantastic experience, by the way) and I remember working on the game controller. 

The Xbox 360 controller sitting on a pile of prototypes, models, and sketches. 

The Xbox 360 controller sitting on a pile of prototypes, models, and sketches. 

6 & 9 are the left & right triggers, 5 & 10 are the left & right bumpers. 

6 & 9 are the left & right triggers, 5 & 10 are the left & right bumpers. 

After we finalized the names of the buttons, which was another interesting experience by itself, there was significant disagreement on the orientation of the labels for the Triggers and Bumpers

Most people, including the higher-ups in the organization, wanted to have them appear "right side up". It was how the Sony Playstation did it, so it should be good enough, right? 

I disagreed. I thought they should be "upside down" – it just felt right to me. At this point I had no data to back it up but it made tons of sense to me as a gamer and game designer. I gathered feedback informally by taking a prototype controller, walking around the building, and handing it to random people. I would ask them to hold it as if they're playing normally and look at the triggers to look at how they're labeled. Almost every single one of them would simply tilt both hands up and look down to see them.

I've recolored the labels to make them more visible - they're not actually colored red. 

I've recolored the labels to make them more visible - they're not actually colored red. 

Even with this, there were some people who fought us every step of the way. The Marketing department argued that they didn't want the product photos to show upside down labels! 

In the end, we ended up including a task in an official usability test and found my original hypothesis validated. We got the data we needed to make the case. And so it came to be, and this little decision has carried over to the Xbox One to the minor delight of gamers everywhere. 

•••••

As much as I'd like to say to actually trust your gut, I won't. What you should do instead: 

  • Train your gut. 
  • Lead with design. 
  • Validate with data. 

Train your gut. Don't trust your gut blindly. Recognize that we are extremely flawed humans with brains end egos that mislead us. Be aware of your biases. 

Lead with design. Put your skills, expertise, and discipline to good use. Understand the problem first – you can't solve the problem if you don't know what it is. 

Validate with data. Test it somehow. It doesn't need to be extremely formal in a usability lab with tens of users. You can do it cheap and quick, but if you can, get some feedback. 

If you have a hypothesis, write it down so you can check to see how close or far you were from actual results. Look for opportunities to do this over and over and over. Over time, you'll get better at it so your gut is informed and exercised. Instead of gut, develop good judgment. 

Getting it right once isn't necessarily attributable to skill; it could be dumb luck. But getting it right repeatedly? That's a hell of a valuable skill. 

TBT: Being Called Out

Seventh grade was my first school year after immigrating to the US. Even though I knew English, I wasn't quite fluent yet. When my classmates spoke in their normal talkasfastasyoucangetitout speed sprinkled liberally with mid-80's slang, I could barely understand what they were saying. 

Of course, my friends would try to teach me the necessary words and phrases that every 12-year old kid would need to know. This being a working-class parochial school in the suburbs of Philadelphia, one key phrase that the guys in my class thought I should understand is "being called out." When someone calls you out, they want to fight. Good to know. 

Within a couple weeks, during recess or lunch, a girl from the eighth grade came up to me to ask if I would be interested in "going out" with her friend, Missy. I look behind her, and standing about twenty feet away was Missy who was a good four or more inches taller than me. 

I was extremely confused. What did I say or do, in my ignorance, that would provoke this girl who was a year older and bigger than I was, to want to kick my ass? 

I believe I stammered, "Uh, no, I'm sorry. Please tell her I'm sorry." And I quickly turned around and ran off to hide. It wasn't until some time later that I understood what "being asked out" meant, and this entire episode made much more sense.  

Why Do We Drive on the Left/Right?

I was doing some research for a presentation a few months back, and I learned the interesting history of why we drive on the left or right side of the road. I ended up not including this in my final presentation, but I thought it interesting enough to share anyway. This is probably incomplete as most of the references I found came from the West, so if you have any other references, please share. 

A Roman road that's better than some roads in San Francisco.

A Roman road that's better than some roads in San Francisco.

As with many things in Western history, it has roots in Rome. One of ancient Rome's greatest inventions were paved roads – all roads lead to Rome and all that. Their vast network enabled their armies to travel willy nilly, relatively speaking, across their vast empire at speeds never before seen. 

Archeological evidence at a Roman mine showed deeper wear on the left side of the road. This suggested that the heavily-laden carts were coming out of the mine on the left.

The Roman Empire at its height. 

The Roman Empire at its height. 

Considering that the Roman Empire was once 2.51 million square miles (6.8 million square kilometers), this greatly influenced the rest of the continent's road habits. 

Fechtbüch of Paulus Kal, Bayerische Staatsbibliothe  

Fechtbüch of Paulus Kal, Bayerische Staatsbibliothe

 

Another thing that suggests the early European habit of riding on the left side of the road is the fact that most people were (and still are) right-handed. This meant that fighters, especially on horseback, stayed on the left side so their weapons were kept on their right. 

This habit seemed to go on for a few hundred years, probably reinforced by the near-constant warfare that seemed to be the primary form of entertainment before TV was invented. And then this dude came along with his conquering armies across Europe. 

If you don't know who this guy is, it's worth reading a little bit of Western history. 

If you don't know who this guy is, it's worth reading a little bit of Western history. 

Napoleon was reportedly left-handed, and whether it was due to this, he just wanted to make a point, or just because he wanted to piss everyone else off, he had his armies march on the right side of the road. So for a while, this Napoleonic French habit influenced the empire's colonies, occupations, and allies. Their continental rivals, the British, continued riding and driving on the left side of the road. I imagine this was partly due to cultural inertia and just because it wasn't French.  

Originally, the British colonies across the ocean adhered to the left-side rule, but due to a deep disagreement that boiled over in the late 18th century, lots of these old British habits were set aside. Anything that was anti-British probably got a decent amount of popular support during this period in what became the United States. 

Washington Crossing the Delaware by Emanuel Leutze, MMA-NYC, 185. He probably crossed on the right side of the river, but I have found no evidence of this. 

Washington Crossing the Delaware by Emanuel Leutze, MMA-NYC, 185. He probably crossed on the right side of the river, but I have found no evidence of this. 

These European and American habits spread across their respective spheres of influence over the next couple centuries and continued via cultural inertia, even as the primary mode of transportation evolved from horses to automobiles. 

Red: drive on the right | Blue: drive on the left

Red: drive on the right | Blue: drive on the left

In more recent history, some countries have swapped from one side to the other. On September 3, 1967, Sweden decided to swap from left to right. Their neighboring border-sharing countries all drove on the right, so there were good reasons to change, but it apparently took 40 years of debate before the law was finally passed.  

Sweden's "Dagen H" or "H-Day" in 1967. It was mayhem on the roads for a little while. 

Sweden's "Dagen H" or "H-Day" in 1967. It was mayhem on the roads for a little while. 

Most recently on September 7, 2009, Samoa also changed the rules of the road making the switch the other way, this time from from right to left. Their reasons for changing were primarily economic. Their closest international & economic neighbors drove on the left: Australia, New Zealand, and Japan. It was far cheaper to import cars from those countries than from the United States. As you can see below, the transition wasn't pretty. 

If you have anything to add to this, I would love to learn more about it. I will try to do additional research on Eastern cultural history to see how they started, and I will follow up once I have more to share. 

User Delight

Yes, another Venn diagram.

Here's one showing how Useful, Usable, and Beautiful together create the conditions for User Delight. Over the years I've been involved in many discussions that throw around these terms but nothing had resolved their relationship to each other in a way that fully made sense to me, so I came up with this. 

The conditions needed to deliver a delightful user experience: usable, useful, and beautiful. 

The conditions needed to deliver a delightful user experience: usable, useful, and beautiful. 

These terms are commonly used throughout the UX disciplines: 

  • Useful: It fulfills a user need.  
  • Usable: It is easy to use.   
  • Beautiful: It is aesthetically and emotionally pleasing. 

Together they create the conditions that enable delightful user experiences. You really need all three.  

  • Usable plus beautiful without useful is frivolous
  • Beautiful plus useful without usable is frustrating
  • Useful plus usable without beauty is functional

It is really much harder than people think to truly delight their users. It often requires teams of UX people with complementary skills and expertise to cover all the bases: design researchers, ethnographers, information architects, interaction designers, usability engineers, visual/graphic designers, motion designers, etc. In an organization that is trying to deliver a product or service, the relationships between Business, Experience, and Technology (BXT) need to be mutually respectful. The focus has to be on user delight. Leadership has to empower and enable it. Culture has to live it. It's difficult to do one of these well, never mind all of them. 

The great part is that there has never been a better time to be in UX. Just because it's hard doesn't mean it's not worth the attempt of doing it well.