18-April-2020 By Jeffrey Cooper

Success Isn’t Always Obvious

In many companies and in many programs, success is obvious and measuring success is easy. You set about to achieve a goal, and you measure it by how well you achieved that goal. A business’s goal is to make money, and the amount and margin is a direct measure of that success.

If you develop an application, it’s success can be determined by a combination of its popularity and performance.

But, sometimes, success is measured by what doesn’t happen. Consider companies who’s main area of business is risk mitigation. Their success is determined by how well problems didn’t occur. You have to observe the space in between.

When it is Obvious

If you are in manufacturing, and you make thousands of the same thing, then success can be easy to measure. Let’s say you have an inefficiency in the manufacturing process. The measure of success is are you now producing x% more than before?

Or let’s say you have a defect rate that is too high. If you change the design of the product in some way, and the defect rate goes down, again- you’re able to directly measure the success.

In either case, you will likely have reams of historical data to compare to.

When it is Less Obvious

There are many situations where success is not obvious. The lack of success is obvious, though. Consider the recent crisis at Boeing with the 737 MAX. It’s obvious in hindsight that there is a problem, and success is measured only by the problems not occurring, which can take time.

It’s the same with software. The concept of “production” is quite different in software systems and applications. You write the app once, and exact copies are sent out. All will be subject to the same failures, though each copy may be used differently by their users. With thousands or millions of copies distributed, if there are bugs, the odds are high someone will find them.

In some cases, you may create a white label app, with small amounts of customization. Those apps for the most part fall into the same category- it is basically the same app on all phones, whether it be for Company A or Company B.

For software development firms that develop many different products for many different customers, it’s even more difficult to measure success. Those companies will produce custom solutions for each partners, and in most cases, each product is unique.

Software Development

Defects in software, at best cause user dissatisfaction- they are detrimental to the User Experience. If they are bad enough, users will stop using your product. In the worst cases, data is lost or accidents happen. In today’s development culture, there is an emphasis on testing to find bugs, but not as much effort in preventing bugs in the first place. I often hear the word “refactor” to rework code later to be more efficient. But every time you change the code, you have yet more opportunities in introduce new errors. I consider “refactor” to be a dirty word.

I prefer waiting to code until you take the time to plan it out first. Properly architecting a system before starting coding- prefactoring, if you will, results in more reliable code and less introduced errors. Good modularity will result in reusable models, that once debugged, can be safely reused, reducing risk on future products.

Measuring the Lack of Bugs

When I managed a team of developers at a previous employer, the team developed the 3rd largest subsystem in the product, but took some time up front to design the elements. The code was extensible, highly modularized, and easily testable. They did their due diligence in testing. Out code was a little later getting released to the complete build, and management was upset with us. I stood my ground- as manager, that was my duty. But when the code released a little later, it worked.

Over the course of the product cycle (2 years), there was a total of 17 bugs found. By comparison, the largest subsystem in the phone debugged 50 errors in one week alone. Management praised the team that found the 50 bugs, but my team never even coded 50 bugs to begin with. So I would ask, who was more successful?

I’m not personally taking credit for their design. It was the team’s expertise that delivered those results. My role was simply supporting them, getting them what they needed, and defending them when I believed they were doing the right thing. I learned what I know now from this awesome team.

Careful Design

Frequently development is rushed into. But examples abound that a period of careful planning up front will reap rewards in saved time and user perception (which equals money, in the end) down the road. I’m not promoting one design methodology over another, but that there is a methodology. Planning a design fits just fine into Agile methodology if you adapt it for this. Once you’ve laid out a modularized plan, you can chart a course towards minimum viable product and iterate your way there, and then continue until the full product is finished.

Along the way you also need to regression test so that you ensure there were no introduced bugs in previously existing code. The two week cycles in Agile actually fit nicely with older, design-first approaches and synchronized release schedules.

In the example of my software development team, success was measurable when the bugs surfaced only in small numbers. Months and years later, it was obvious that they were more successful. The important thing is when you do encounter the inevitable bug, you take measures to understand how it occurred and work to prevent that in the future. Over time, your results will improve continuously. Early on, you can compare with previous efforts and peers in other companies to the best ability you have to get those measures.

When Failure Matters Most

I’ll close on a more serious note. In the past, global containment of pandemics appear to have worked. There have been documented failures more recently that coincide with the current Covid-19 pandemic. A full analysis will not happen for some time, but obviously things have gone wrong. In the larger scheme, comparing Covid-19 with previous pandemic-potentials- SARS, MERS, Ebola, how did this one stack up compared to those with respect to its “pandemic parameters”, those measures that determine, unchecked, a pathogen’s pandemic potential, is of utmost importance. Normalizing for that will help determine if past preventative measures were as effective as they seemed, or simply not effective enough this time around.

Always, in every process or product, failures need robust analysis to find ways to prevent them from occurring again. The incredible safety of air travel today is a shining example.