The Over-Engineering Problem (and How to Avoid It)

by | Software Development

One of my coworkers remarked recently that software engineers have an overengineering problem. What did he mean — and how common is this problem?

Let’s look at what over-engineering is and why it’s a problem with a real-world example we’re all familiar with. 

What Does Over-Engineering Mean?

Let’s take a car. What defines a car? Well, it has an engine, which is connected to four wheels. It has a steering wheel and seats which the passenger(s) sit on. It has doors so the passengers can get in and out, and a trunk where you can store items like luggage. Then you have windows you can see out of, which is necessary for the driver to know where to maneuver the car with the steering wheel. There are the gas and brake pedals that allow you to accelerate and slow down.

Even the earliest first generation cars that came out at the turn of the twentieth century all had these same attributes as the latest and greatest vehicles we’re producing in the twenty-first century.

Sure, nowadays we have hybrid and electric cars for better fuel efficiency and lots of onboard computers and technology that let us take advantage of GPS and radio satellites, but all in all, the basic attributes of a car have been there since the dawn of the automobile… 4 wheels, an engine, a steering wheel, gas pedal, brakes, doors, and seats.

An automobile, no matter what manufacturer produces it, will likely always be designed with these same basic car attributes.

A well-designed car is properly engineered. It has just the exact amount of design and technology put into it to properly do its job, which is to get passengers from destination A to destination B in the most time efficient and safe way possible.

What Would an Over-Engineered Car Look Like? 

What if the car had wings like an airplane? In case the car crashed through a guardrail which ended in a steep cliff? Wouldn’t you want airplane wings on the side of the car so the car would have a fighting chance of gliding down to safety?

1947 Convair 118 via Flickr

We all know that cars have four tires. But wouldn’t more tires be useful, in case one of the tires blew out or had a flat? You could still limp the car along until you found a gas station.

Cars have one engine. But wouldn’t two engines be better? Or better yet, what about three? One in the front, one placed in the middle of the car, and one in the rear? That way, in case one engine malfunctioned, you would have the other engines to act as backups.

But there’s a reason why we never see cars with all these extra features and doohickeys.

Why?

Because they’re just not worth the additional effort and cost to include them in a car.

Take the tire example. The reason cars only have four tires is that it’s the optimal number of tires needed to maneuver a car from point A to point B. Any fewer, and you’d make the car less stable for maneuverability. Too many, and again you run into both the added cost of additional tires, but the additional axles and other technology needed to accommodate more than four tires.

The same goes for the engine. Cars have one and exactly one engine. Putting an additional engine into a car adds nothing but additional fuel costs, all the extra technology and engineering needed to accommodate more than one engine, and raises the cost of the car to the point where it would be beyond the reach of most consumers.

It’s pretty easy to avoid over-engineering in a car because the cause and effects are pretty easy to measure. We know exactly what every component costs in terms of parts and labor.

The same can’t be said for software programming.

What Is Over-Engineered Software?

Based on twenty years of working with other people’s code as well as my own, I can say without a shadow of a doubt that over-engineered software is actually the NORM, and not the exception.

Back to my coworker. He was lamenting the codebase he was combing through to fix some production bugs. Every programmer on the planet has faced this same task at one time or another in their career. He was expressing his frustration at how difficult it was to fix, let alone even UNDERSTAND what the source code was doing. He concluded that the code was overengineered up the wazoo.

Surprise, surprise!

When I asked him what specifically was over-engineered, he pretty much said “everything”. My coworker concluded the original software programmers of the code designed the application with the “just in case” mentality.

The Problem with “Just in Case”

“Just in case” the code MIGHT need to swap out the Microsoft back-end database with Oracle or a NoSQL database, the original software developers wrote the code in a way to use inversion of control and dependency injection so that it would be theoretically easier and faster to do so. 

Here’s the problem with the “just in case” methodology of software development. Nine times out of ten, You Ain’t Gonna Need It … and yes, there’s an official acronym: YAGNI.

So what ended up happening with the entire codebase my coworker was struggling to work in, which was all written and designed without YAGNI in mind. In layman’s terms, it was over-engineered by at least a factor of ten when there was no legitimate reason to.

There’s a famous example of how a simple “hello world!” program that is supposed to output “hello world” to a computer screen, can be inflated and overengineered into a scary, spaghetti code mess.

A simple Java program that outputs “hello world” would look like this:

System.out.println(“hello world”);

But it’s totally possible to overengineer this simple and easy to read code with something like this and still make it completely acceptable to execute by a computer:

public interface Subject {

    public void attach(Observer observer);

    public void detach(Observer observer);

    public void notifyObservers();

} 

public interface Observer {

    public void update(Subject subject);

}

public class HelloWorldSubject implements Subject {

    private ArrayList<Observer> observers;

    private String str;

    public HelloWorldSubject() {

        super();

        observers = new ArrayList<Observer>();

    }

    public void attach(Observer observer) {

        observers.add(observer);

    }

    public void detach(Observer observer) {

        observers.remove(observer);

    }

    public void notifyObservers() {

        Iterator<Observer> iter = observers.iterator(); 

        while (iter.hasNext()) {

            Observer observer = iter.next();

            observer.update(this);

        }

    }

    public String getStr() {

        return str;

    }

    public void setStr(String str) {

        this.str = str;

        notifyObservers();

    }

}

public class HelloWorldObserver implements Observer {

    public void update(Subject subject) {

        HelloWorldSubject sub = (HelloWorldSubject)subject;

        System.out.println(sub.getStr());

    }

}

public interface Command {

    void execute();

}

public class HelloWorldCommand implements Command {

    private HelloWorldSubject subject;

    public HelloWorldCommand(Subject subject) {

        super();

        this.subject = (HelloWorldSubject)subject;

    }

    public void execute() {

        subject.setStr(“hello world”);

    }

}

public interface AbstractFactory {

    public Subject createSubject();

    public Observer createObserver();

    public Command createCommand(Subject subject);

}

public class HelloWorldFactory implements AbstractFactory { 

    public Subject createSubject() {

        return new HelloWorldSubject();

    }

    public Observer createObserver() {

        return new HelloWorldObserver();

    } 

    public Command createCommand(Subject subject) {

        return new HelloWorldCommand(subject);

    }

}

public class FactoryMakerSingleton {

    private static FactoryMakerSingleton instance = null;

    private AbstractFactory factory;

    private FactoryMakerSingleton() {

        factory = new HelloWorldFactory();

    }

    public static synchronized FactoryMakerSingleton getInstance() {

        if (instance == null) {

            instance = new FactoryMakerSingleton();

        }

        return instance;

    }

    public AbstractFactory getFactory() {

        return factory;

    }

}

And then once all of this code is in place, then you can finally make the computer say “hello world” by doing this:

public class AbuseDesignPatterns {

    public static void main(String[] args) {

        AbstractFactory factory = FactoryMakerSingleton.getInstance().getFactory();

        Subject subject = factory.createSubject();

        subject.attach(factory.createObserver());

        Command command = factory.createCommand(subject);

        command.execute();

    }

}

And voila, after you exploded the program in size at least tenfold and make it absolutely impossible to glance at and understand quickly, you have created a bloated and over-engineered mess!

And this is exactly what my coworker experienced.

So it begs the question …

Why Is So Much Code Over-Engineered?

I believe a lot of it may be due to the fact that we all get pounded in our heads the importance of properly architecting your code with various design patterns and principles.

Pretty much every design pattern and principle of software programming revolves around one concept … they’re all designed to make code easy to change.

Because if there’s one constant in software programming, it’s the fact that requirements will always change, which necessitates a software developer to go back to the original code and refactor it so it accomplishes the new requirement.

And believe me when I say there are LOTS of programming patterns, design principles and methodologies that help code become more flexible to changing program requirements.

But that’s where the crux of the problem is.

The two most important guiding principles of ANY computer source code are the following:

  1. It’s easy to understand
  2. It’s easy to change

Unfortunately, these two golden rules of programming are often in conflict with each other.

Code that is easy to read and understand often means the code is hard to change when program requirements change.

Code that is easy to read and understand often means it is hard to change when program requirements change. Click To Tweet

And the inverse is strangely true as well. When you refactor code so it’s easier to change when program requirements change, the code becomes more complex and hard to understand for the next developer who needs to take it over and make future enhancements and bug fixes.

How to Strike the Right Balance

Getting the right balance so that code is easy to understand AND easy to change, can be extremely challenging. It takes a lot of fine-tuning and experimentation.

So if my premise holds water and making code easier to change in the future results in hard to understand and readable code, what is a software developer to do?

Well, it goes back to the You Ain’t Gonna Need It, syndrome. You, as the software developer, need to constantly ask yourself if the way you’re currently writing code solves the immediate problem or program requirement OR if you’re writing it in a way to future proof it and make it more resilient to change.

If, for example, you think it’s very likely you will need to swap out your Microsoft database with a different database or some sort of NoSQL document-based database like Couch or Mongo, then it would be worth looking into the mechanisms and methodologies that are available to you in the programming language and framework you’re currently using, to make the code easier to change in the future.

However, if it’s very unlikely that you will have to swap out the database in the future, then write your code in the most straightforward and simple way possible … you ain’t gonna need the complexity that will be necessary to future proof it for a different database down the road.

When possible, follow the KISS principle … KEEP IT SIMPLE STUPID! Only when you’re confident that the portion of your code you’re writing will need to change due to future program requirements should you consider refactoring the code to make it more resilient to changing requirements in the future.

Otherwise, simple is better. You’ll save yourself time and more importantly, you won’t end up targeted as public enemy #1 by the developer in front of you who will have to take over your code.

Because we all know how dangerous programmers get when you box them into a corner with no way out.

A feral code monkey is the most dangerous species on the planet.

Ready for Your Next Job?

We can help! Send us your resume today.


Need Talent?

Submit your job order in seconds.


About ProFocus

ProFocus is an IT staffing and consulting company. We strive to connect a select few of the right technology professionals to the right jobs.

We get to know our clients and candidates in detail and only carefully introduce a small number of candidates that fit the role well.