I sat in front of Mat, idly chatting about tech and cuisine. Before now, I had known him mostly for his cooking pictures on Twitter, the kind that made me envious of suburbanites and their 75,000 BTU woks. But now he was the test subject for my new project, to see if it was going to be fruitful or a waste of time.
âWhatâs your job?â
âRight now Iâm working on microservices for a social media management platform.â
âAnd before that?â
âGeological engineering. A lot of open pit mining, some amount of underground tunnel work. Hydropower work. Earth embankment dams because they come along with mines.â
He told me a story about his old job. His firm was hired to analyze a block cave in British Columbia. Block caves are a kind of mining project where you dig tunnels underneath the deposit to destabilize it. The deposit slowly collapses and leaks material into the tunnels, and then âyou just print moneyâ, as Mat called it. The big problem here? The block cave was a quarter mile under a rival companyâs toxic waste dump. âIn the event of an earthquake, could the waste flood the mine and kill everyone?â He had to prove it was safe. A different kind of work than what he was doing now.
As for another difference? âMy personal blog has better security than some $100 million mining projects.â
This wasnât going to be a waste of time, after all.
Is software engineering âreallyâ engineering? A lot of us call ourselves software engineers. Do we deserve that title? Are we mere pretenders to the idea of engineering? This is an important question, and like all important questions, it regularly sparks arguments online. On one hand you have the people who say we are not engineers because we do not live up to âengineering standardsâ. These people point to things like The Coming Software Apocalypse as evidence that we donât have it together. They say that we need things like certification, licensing, and rigorous design if we want to earn the title engineer.
On the other end of the horseshoe, we have people like Pete McBreen and Paul Graham who say that we are not engineers because engineering cannot apply to our domain. Engineers work on predictable projects with a lot of upfront planning and rigorous requirements. Software is dynamic, constantly changing, unpredictable. If we try to apply engineering practice to software then software would be 10 times as expensive and stuck in 1970.
For a long time I refused to call myself a software engineer and thought of other people with that title as poseurs. It was reading a short story against engineering that made me question my core assumptions. It was A Bridge to Nowhere, one of the Codeless Code stories. The author argues that the techniques of bridge building donât apply to software, since software clients can change their requirements and software gravity can sometimes reverse itself.
The predictability of a true engineerâs world is an enviable thing. But ours is a world always in flux, where the laws of physics change weekly. If we did not quickly adapt to the unforeseen, the only foreseeable event would be our own destruction.
Thatâs when I realized something about everybody involved in all of these arguments.
Theyâve never built a bridge.
Nobody I read in these arguments, not one single person, ever worked as a ârealâ engineer. At best they had some classical training in the classroom, but we all know that looks nothing like reality. Nobody in this debate had anything more than stereotypes to work with. The difference between the engineering in our heads and in reality has been noticed by others before, most visibly by Glenn Vanderburg. He read books on engineering to figure out the difference. But I wanted to go further.
If I wanted to know how software development compares and contrasts with ârealâ engineering, then Iâd have to talk to ârealâ engineers. But thinking about that, I realized another problem: while ârealâ engineers could tell me what they did, they couldnât tell me how their work differed from mine. Just as we make sweeping statements about real engineering without any evidence, they could make sweeping statements about software without knowing anything about it either. Even if they use a bit of software in the day-to-day work, thatâs not the same as developing software as a job. Only a person whoâs done both software development and ârealâ engineering can truthfully speak to the differences between them.
So thatâs what I set out to find: people who used to be professional engineers and then became professional software developers. I call these people crossovers, hybrids between the two worlds. I interviewed 17 crossovers on common software misconceptions, how the two worlds relate to each other, whether we can truthfully call what we do engineering, and what the different fields can teach and learn from each other.
Thereâs a lot I want to talk about here, much more than can comfortably fit in one blog post. I divided the write up into three parts, dealing with my three core topics. Part one is about the term âengineeringâ. Is what we do engineering, and can we honestly call ourselves engineers?
Why is this a Question?
So first some ground rules: âsoftware engineeringâ is a real field. Most people would agree that the software running on spacecraft counts as âreal engineeringâ. The debate is about rest of the field. Are the people who build websites engineers? What about embedded hardware? I found in my research that people defending the title âengineerâ rarely coherently define what an engineer is, usually boiling it down to âengineers solve problemsâ. The arguments against the title tend to be a little more developed, as are the arguments about transcending it.
The backlash against âsoftware engineeringâ comes from two places. First, there are the gatekeepers. While at first we called ourselves programmers and âdevelopersâ, after 1985 more and more people started using the title âsoftware engineerâ. This âtitle bloatâ is one of the things that led to the backlash: people using it as a title without trying to live up to some form of standards.
The other side of the backlash comes from the new software movements of the 90s and early 00s. Movements like Extreme Programming and Agile were rejections of applying standard project management techniques to software development. To them, engineering represented the old way of doing things, incompatible with the new way of software development. Software engineering was the past, a failed dead end, and artisanal software was the future.
Both of these backlashes come from the connotations of engineering. In the case of gatekeepers, itâs the positive connotations they believe we have not earned. In the case of artisanal craftship, itâs the negative baggage they believe we donât deserve. Both of these are agenda-driven viewpoints, arguments based on how they want software to be. This is good for advocacy but doesnât help us figure out where software is right now. And their agendas are based on a reference point they donât understand themselves. None of the people arguing for or against software engineering as engineering have worked as engineers.
Common Misconceptions
Letâs start with common arguments that people make about whether to include software in engineering. Most people have an informal concept of what engineering but not a strict definition. Like pornography, people know what engineering is when they see it. When I asked people to explicitly list the qualities of what makes something engineering, here are the most common responses:
- Making something complex as part of a team.
- Making something physical.
- Making something according to a rigorous spec with strict principles.
- Using mathematical principles in their design.
- Working on situations with high consequences, like loss of life.
- Doing work with a certification and license.
The first quality is too broad: almost all human professions involve making something complex in a team. It doesnât capture âengineeringâ. The second quality is too restrictive: not all forms of engineering result in physical processes. In particular, industrial engineering rarely does. The third quality will be discussed in the next essay.
That leaves three claims to discuss here. All three are used to say software canât be engineering. âWe donât use any math, unlike real engineers. Software doesnât matter, unlike real engineers. We arenât licensed, unlike real engineers.â As weâll see, none of these quite hold up.
Engineering is mathematical
This one I can address directly. The claim is that engineering involves a lot of hard math, while software involves very little math. The confusion here comes from our misunderstanding of mathematics. Much of the math that mechanical engineers use is continuous math. This is where we work over a continuous domain, like real numbers. Things like calculus, trigonometry, and differential equations are in this category. This is what most people in the US learn in high school, codifying it as what they think of as âmathâ.
In software, we donât use these things, leading to the conception that we donât use math. But we actually use discrete math, where we deal exclusively with non-continuous numbers. This includes things like graph theory, logic, and combinatorics. You might not realize that you are using these, but you do. Theyâre just so internalized in software that we donât see them as math! In fact most of computer science is viewable as a branch of mathematics. Every time you simplify a conditional or work through the performance complexity of an algorithm, you are using math. Just because there are no integrals doesnât mean we are mathless.
This falls in line with the rest of engineering. Different branches use different kinds of math in different ways. Industrial engineers are concerned with very different things than mechanical engineers are. Just because we use a different branch of math doesnât mean weâre not doing engineering.
Engineering is high-consequence
Iâm not sure what first disabused me of this notion. It might have been the conversation with Mike, an ex-mechanical engineer who now designs glaciology sensors. One of his last projects before the crossover was as part of a team working on a classified medical device. Most of it went smoothly, but then came the big blocker:
The one the thing that gave us the most grief was the handle, which was a bent bit of wire with some plastic molded around it. Yeah, getting the wires to bend correctly and the plastic onto this shiny piece of anodized aluminium wire. [â¦] That was the thing that took three months in the project.
Three months of the project dedicated to getting the handle working right.
This is not to say the work wasnât important. It does mean that the engineers spent a lot of time on something thatâs low-consequence or non-safety critical. In retrospect, this shouldnât have surprised me. The world is vast and our industry is huge. Someone needs to design the bridges, yes, but someone also needs to design all of the little things we use in our day-to-day life. Much of the engineering there is low-stakes, low-consequence, just like much software is.
Thereâs a huge difference between if youâre making a cabinet for Bluetooth speakers, like ones that are on my desk, or youâre making an assembly for the landing gear of 737. Youâre using some of the same tools, but your approach is wildly different. - Nathan (mechanical)
«But there are still some things that lead to loss of life!» Yes, and the same is true of software. An integer overflow in the Intrado software lead to a several hour 911 outage for millions of people. A biased algorithm unfairly sent people to jail. Just like with ârealâ engineering, thereâs also a wide swath of software that wonât kill people but will lead to millions of dollars in damages. This can be everything from Amazon going down to a game being unplayable.
Engineers are licensed
This is the claim Iâve heard most: licensure. Thereâs a difference between doing engineering and being an engineer, just as people practice medicine at home without being doctors. Maybe software engineering is possible, but without the licenses we are not software engineers. In Canada you canât even call yourself a âSoftware Engineerâ unless youâre accredited!
At least thatâs what people in the United States say. Several Europeans I spoke to said the exact same thing about US system. Everybody seems to think the worst about their own system and the best about others. And about half of the engineers I spoke to werenât licensed but were still considered professional engineers by their peers. They didnât have the âcertificationâ, but they had the skills.
In the US, you donât need a license to practice any kind of engineering. You need a license to be a âprincipal engineerâ, a.k.a. the person who formally signs off on plans as valid. But the engineers working under the principal engineer donât need to be accredited and often are not. In fact many of them donât even have formal training as engineers.
Hereâs the problem with deciding engineering based on licenses: licenses are a political and social construct, not a fact of nature. Societies adopt licensure for reasons that are as much political as technical. To see this, letâs discuss some of the history of licensure in the United States.
As recently as 1906, not a single state in the US required licenses for any project. Wyoming was the first state to instate a licensure policy, entirely because irrigation projects kept blowing the budget. They theorized accredited engineers could give better estimates on project costs. It took 40 more years for all fifty states to get on board.
This all means that licensure isnât part of the federal government: itâs a requirement by the states. Among other things, licenses arenât necessarily transferable between states. If you get a license in Texas, you have to go through a process to be able to practice in California, in whatâs called âcomityâ.
While licensure originated as part of cost overruns, it expanded so rapidly for an entirely different reason. Regulations are written in blood. Fields become regulated when the lack of regulation kills people. Until that happens on a wide scale with arbitrary programs, itâs unlikely that weâll ever see the same licensure requirements for software. In the places where software has killed people, like Therac-25 and aircraft accidents, we see stricter regulations. Whether this leads to broader licensure requirements for software engineers remains to be seen.
You could argue that itâs immoral for us to not be licensed. This is an argument Iâm sympathetic to. But itâs a normative argument, not a positive one. By saying âwe should be licensedâ, you are saying something about how the world ought to be. You are trying to answer the question âshould we be held to higher standards?â But thatâs not the question here. I donât care right now where we should be going; I just want to know where we are right now. Whether or not we are engineers is irrelevant to whether or not we are good engineers.
In conclusion: licenses exist because we are part of society and have legal requirements, not because they are essential to what it means to do engineering. So while you might want to make software more licensed, as it stands the licensing question doesnât change the essence of our work.
The Truth
This leaves us back where we started: there arenât any qualities we can point to and say âthis is engineering, this is notâ. Itâs a standard Wittgenstein game problem: human constructs do not neatly fall into precise definitions. Rather something like âengineeringâ is a family of related concepts, where we judge whether something belongs based on how much it resembles other things in the family. In other words, âengineeringâ is âwhat engineers doâ. In other other words, something becomes engineering if enough engineers say itâs engineering.
Consider chemical engineering. Chemical engineering is unlike mechanical, civil, or electrical engineering. Chemical engineers create processes to manufacture products at scale, often using experimentation and iteration. But nobody would disagree that it is engineering. Chemical engineering started in the late 1800s, before states licensed engineers. If chemical engineering started now, people would refuse to call it engineering. And theyâd be wrong to refuse.
Once I realized this, my interviewing process changed a little. Instead of asking how they felt about certain engineering topics, I just asked them point blank. âDo you consider software engineering actually engineering?â
Of the 18 crossovers I talked to, 16 said yes.
Thatâs not the answer I expected going in. I assumed we werenât engineers, that weâre actually very far from being engineers. But then again, I was never a ârealâ engineer. I donât know what itâs like to be a ârealâ engineer, and so canât compare software engineering to other forms. I donât have the experience. These people did, and they considered software engineering real engineering.
Even the product owners and project managers are in a sense engineers [â¦] everyone is kind of an engineer, an engineer of sorts. -Kate (chemical)
It makes no sense to use ârealâ engineering in contrast to âsoftwareâ engineering. Going forward I will instead use the term âtradâ engineering.
Craft vs Engineering
Iâm gonna respond in a slightly different way. Not every software developer is a software engineer, just like not every single person who works in construction is an engineer. An engineer is a very specific set of skills. [â¦] Software [engineering] is skill plus understanding, all the processes and life cycle and the consequences and all the things that you should be aware of [and] avoid. -Dawn (Mechanical & Chemical)
That said, many of the crossovers also added an additional qualification: software engineering is real engineering, but a lot of people who write software arenât doing software engineering. This is not a problem with them, rather a problem with our field: we donât have a rich enough vocabulary to talk about what these developers do. Not everybody who works with electricity is going to be an electrical engineer; many will be electricians. And this is okay. Electrical engineering is a very narrow skill set in the broader field of electric professions and plenty of people have other important skills in that space. But we use things like âprogrammerâ, âsoftware engineerâ, and âsoftware developerâ interchangeably. What is the difference between a software engineer and a software developer?
Some people propose the word âsoftware craftsmanâ. This term comes from the book Software Craftsmanship: The New Imperative, by Pete McBreen. In it he argues that software is the kind of engineering, being much more free-form creative and flexible. Weâre not line workers but artisans, artists, people who take pride in the craft we do and the flexibility of our states. As he puts it:
Software development is all about the unknown. The production process for software is trivially easyâjust copy a disk or CD. The software engineering metaphor fails because we understand production, a mechanical task, much better than we understand design, an intellectual task. - Software Craftsmanship
Many people have asked me why I care so much about this project. Why does it matter whether or not software is âreallyâ engineering? Why canât we just say that âsoftware is softwareâ? Itâs because of these misconceptions. People have a stereotyped notion of what engineering looks like. Because software doesnât look like the stereotype, they assume that we are wholly unlike engineering. The engineering disciplines have nothing to teach us. We are breaking pristine ground and have no broader history to guide us.
In contrast, if we are doing engineering, then we have a landmark. We can meaningfully compare and contrast the work we do as software engineers from the work that others do as traditional engineers. We can adapt their insights and watch for their pitfalls. We can draw upon the extant knowledge of our society to make better software.
I believe everything McBreen said about software is fairly reasonable, about how hard it is to predict things and how itâs intensely personally created. What he gets wrong is the assumption that engineering is not this way. Engineering is much richer, more creative, and more artistic than he thought. But of course he would have an imperfect view: he is, after all, not a traditional engineer.
Hereâs my final take. This is the belief Iâve settled on in synthesizing all the interviews I did and does not necessarily reflect how the crossovers think. I went into this thinking that software wasnât really engineering. Maybe there were a few people who could count themselves as that but most of us were far below that threshold. I still believe that most of us are not engineers, because weâre working in domains that people donât see as engineering. Most people donât consider a website âengineeredâ. However, and this is a big however, thereâs a much smaller gap between âsoftware developmentâ and âsoftware engineeringâ than there is between âelectricianâ and âelectrical engineerâ, or between âtradeâ and âengineeringâ in all other fields. Most people can go between âsoftware craftâ and âsoftware engineeringâ without significant retraining. We are separated from engineering by circumstance, not by essence, and we can choose to bridge that gap at will.
In the next essay, weâll talk about the similarities and differences between traditional engineering and software engineering, and how theyâre actually not all that different in the end.
Part two, We are not Special, will be posted on Wednesday. You can check back here, subscribe to RSS, or join my newsletter to be notified. You can also follow me on twitter.
Thanks to
Glenn Vanderburg
,
Chelsea Troy
,
Will Craft
, and
Dan Luu
for feedback, and to all of the engineers whom I interviewed.
Appendix: Methodology
I only searched out people who work professionally for at least one year in each field. In practice, work experience in trad engineering ranged from a year and a half on the low end to over 15 years on the high end. Interviews ranged from half an hour to two hours, leading to a total of 24 hours of recorded interview. Two of the interviewees were not recorded, but I took notes. The breakdown of the specialties is:
- Civil engineers work on buildings. One person I talked to was a classic civil engineer, one specialized in designing mines, and two worked on oil rigs.
- Mechanical engineers design physical machines.
- Electrical engineers design circuits and electronics. Two worked on chipsets, while one was embedded in a submarine.
- Chemical engineers create processes to make chemicals at scale. They are generally not creating entirely new chemicals or chemical products, but work on how to produce extant ones. âChemicalsâ is a broad category here, covering everything from clean water to toothpaste.
- Industrial Engineers create holistic systems and procedures in an industry. One designed data center layouts and the other worked on integrating disparate air traffic systems.
These are very broad summaries of the specialties. Most engineers work in a subdomain of a specialty, such as automotive engineering or circuit design. There were also some fields of engineering I didnât cover in my interviews. This includes aerospace and nuclear engineering. I suspect (suspect) that aerospace would be roughly similar to mechanical engineering and nuclear engineering would be roughly similar to civil engineering. But it remains a threat to validity nonetheless.
The other two threats to validity are geographic location and crossover type. The majority of the interviewees were either in the US or the UK, with the rest being from the EU or Canada. I did not get a chance to interview anybody from Latin America, South America, Africa, or Asia. Everybody interviewed crossed from traditional engineering to software engineering. I was not able to find anyone who crossed the other way.
