Note: The first portion of this article is adapted from “Recoding America: Why Government Is Failing in the Digital Age and How We Can Do Better.”

Our elected leaders are frustrated. Yes, they’re working in a highly divisive environment which makes it hard to get anything done, or even to have a reasonable conversation across the aisle. But it’s not just that. They’re frustrated because they’re supposed to be steering the ship of our country through changing times, but the tiller they have their hands on doesn’t seem all that well connected to the rudder. When they can agree enough to get a bill passed or policy changed, their intention is to change speed or direction, but when they turn the wheel or try to speed up or slow down, it’s anyone’s guess what will actually happen.

Reform Medicare to improve the quality of care patients receive? You may end up driving medical practices out of the program and decreasing the quality of care. Increase the unemployment benefits workers receive during the pandemic? You may end up with millions of angry constituents who’ve waited months for their checks, while tens or maybe hundreds of billions of dollars are lost to fraud. The courts feel it too: They can reverse an order that separated kids from their parents at the border, averting further harm, but Customs and Border Protection’s systems no longer know which kids belong to which parents, and most of these families will remain torn apart.

There’s a convenient punching bag for many of these failures: outdated government technology, and outdated approaches to tech by the bureaucracy. But try to fix that through policy change and you’ll find it’s turtles all the way down. The levers leaders use to fix tech are the same ones they use to steer the economy, improve government-funded healthcare, manage immigration, and even strengthen our national defense. We increase budgets, cut budgets, make new rules, and hold hearings, but the tools we use to fix our tools aren’t working either.

Government tech policy backfires

Elected officials have been trying to use law and policy to improve government technology for a lot longer than most people realize. The Brooks Act of 1965, enacted when computing meant room-sized mainframes and automatic data processing, tried to ensure government would get the best prices for hardware and computing services by mandating that the General Services Administration do all the buying for agencies. In the 90s, when Amazon started selling books online and Google was indexing all the world’s knowledge, Senator William Cohen and Representative William Clinger decided the Brooks Act approach wasn’t sufficient and tried to get the White House Office of Management and Budget to take responsibility for digital strategy. The Clinger-Cohen Act was passed into law, but unfortunately some of what they got backfired.

To understand how things backfired, you’ll need to drop down from the bird’s eye view from which we normally see the bureaucracy all the way to the ground. You’ll need to see the problem of implementation of government policy through the eyes of Matthew Weaver of the U.S. Digital Service, who was literally reaching for the ground: He was trying to get data from satellites in space to ground stations, so that the Global Positioning System would continue to work. (See his account here.)

Weaver had been brought to Raytheon, the company the Air Force had hired to write the software for the next generation GPS satellites, because the Raytheon team was behind schedule and over budget. This issue of data transmission to the ground stations and back again was one of a few problems that was holding them back. There is an industry standard way of doing this, a simple, reliable protocol that is built into almost every operating system in the world.

But this team wasn’t using this simple protocol on its own. Instead, the team had written a piece of software to receive the message from that protocol, read the data, and then recode it into a different format, so they could feed it into a very complex piece of software called an Enterprise Service Bus, or ESB. The ESB eventually delivered the data to yet another piece of software, at which point the whole process ran in reverse order to deliver it back to the original, simple protocol. Because the data was taking such a roundabout route, it wasn’t arriving quickly enough for the ground stations to make the calculations needed. Using the simple protocol alone would have made the entire job a snap—as easy as nailing a couple of boards together. Instead, they had this massive Rube Goldberg contraption that was never going to work.

The people on this project knew quite well that using this ESB was a terrible idea. They’d have been relieved to just throw it out, plug in the simple protocol, and move on. But they couldn’t. It was a requirement in their contract. The contracting officers had required it because a policy document called the Air Force Enterprise Architecture had required it. The Air Force Enterprise Architecture required it because the Department of Defense Enterprise Architecture required it. And the DoD Enterprise Architecture required it because the Federal Enterprise Architecture, written by the Chief Information Officers Council, convened by the White House at the request of Congress, had required it. Was it really possible that this project was delayed indefinitely, racking up cost overruns in the billions, because Congress has ordered the executive branch to specify something as small and technical as an ESB?

The short answer is no. But it’s important to understand why pretty much everyone thought it was yes, and it ties back to Clinger and Cohen’s attempt to get the executive branch to take tech seriously. Among other provisions, the law as it was enacted required each federal agency to have an “Information Technology Architecture.” There was a sense that the technical architectures of the various agencies should be coordinated in some way, so the CIO Council got the job of coming up with that uber-architecture. The result seems to have been a classic example of design by committee. However the document came together, the 434-page Federal Enterprise Architecture, released in 1999, requires that federal technology solutions have a “service-oriented architecture.” And it defines that in terms of an Enterprise Service Bus. 

It’s true that many laws and policies fail because they are overly prescriptive and lock implementers into a narrow set of options. But that is not quite what happened here. Neither the Clinger-Cohen Act nor any other law explicitly required an ESB. Nowhere in the Federal Enterprise Architecture does it say “thou shalt always use an enterprise service bus.” There are five mentions of “enterprise service bus” in the document, but all of them are in charts or diagrams listing various application components that could support interoperability. ESBs became mandatory in practice within the Department of Defense through overzealous interpretations of law, policy, and guidance, combined with lack of technical understanding.

The accountability double-bind

How does this happen? When there are big, visible delivery failures, like HealthCare.gov or the unemployment insurance crisis, public servants are trapped between two distinct systems of accountability. In the first, politicians will hold the public servants accountable for outcomes: whether the website works to enroll people or whether benefits are actually getting to claimants. In this system there will be hearings. (Congressional committees held 10 separate hearings on HealthCare.gov in a single month in 2013, all as the people being called to testify were otherwise working night and day to get the site back up.)  Angry politicians will summon bureaucrats to the Hill, or the state capitol, or city council chambers, and their staffs will help them prepare incisive questions that show that they are paying close attention to the problem. These hearings will likely be televised or webcast, and a few sound bites might appear on the evening news or circulate on social media.

In the second system of accountability, various parts of the administrative state—the agency itself, the inspector general, the Government Accountability Office—will hold these same public servants accountable to process. Procurement and planning documents will be reviewed for any gaps, any skipped or partially skipped steps, any deviance from standard protocol, even if that deviance is legal, just nonstandard. If an ESB is thought to be “best practice”—even if it is not at all best practice—why wasn’t it used? If anti-fraud practices had been previously established, why weren’t all of them deployed? If the vendor has been hired under an Other Transaction Authority (a streamlined path for procurement that Congress approved to give agencies flexibility to exercise judgment), why was this exception allowed and a more thorough and standard process not followed? And who signed off on all these decisions? Angry politicians will sometimes ask such questions too, but they are less familiar with the many administrative rules and requirements. They tend to focus more on the issue their constituents care about: Why isn’t the thing working?

The accountability trap is a damned-if-you-do, damned-if-you-don’t situation. The first system is extremely uncomfortable for the public servants subjected to it. No one wants to be called up to testify in a televised hearing. No one wants to be in the video clip as a stone-faced bureaucrat with no good answers, being yelled at by a righteous—or self-righteous—politician fighting the good fight on behalf of the aggrieved public. In front of the cameras, you can’t say things like “it doesn’t work because we were forced to use an ESB.” You would look like you were trying to throw someone else under the bus, and the legislators wouldn’t understand what you were talking about anyway. Your job is simply to endure the hearing, produce as few viral sound bites as possible, and not incriminate others.

As painful and sometimes humiliating as these hearings are, if you’re a career civil servant, it is the second system of accountability that matters more to you. The legislature can’t fire or officially reprimand you, no matter how bad a job they think you did (although they can put political pressure on the administration to do so). They can’t make you ineligible for promotions and raises. On the other hand, violations of policy, process, and procedure—real or perceived—can do all of that, even if there is no hearing. It is the job of many other public servants to make sure that happens. The people under scrutiny want to avoid these repercussions for the obvious reason that being fired or demoted affects their livelihoods, but many also simply want to continue doing their jobs. They believe in what they do, and they believe that they are still the best shot the agency has at fulfilling its mission. They are often right. So they follow policy, process, and procedure even when it is at odds with achieving the desired outcomes. 

These dynamics kick into overdrive when technical issues are in question. Even the most competent tech team can hit resistance when trying to explain, for instance, why there should not be an ESB in the software they are building. A team could point out that it was only suggested, not required, and Weaver tried to do just that. But the reality is that the ESB became incrementally more codified as the Federal Enterprise Architecture gave rise to the Department of Defense Enterprise Architecture, which gave rise to the Air Force Enterprise Architecture, which gave rise to some large number of additional guidance documents within the Air Force and ultimately to the request for proposals and accompanying requirements documents that were issued when the contract was awarded. Arguments about what is or isn’t required happen all the time, but they are much less likely to lead to a suffocatingly risk-averse answer when the people involved in the argument understand the domain. Here, aside from the tech team, it is highly unlikely that anyone else in the debate—likely to be dozens of people in dozens of different roles—had any basis on which to judge whether an ESB was a good thing or a bad thing in this context. Thus, to nix the ESB, dozens of people in dozens of different roles would all have had to agree to jeopardize their jobs over a recommendation they didn’t understand. These discussions tend to function as a vetocracy, in which it takes all thumbs up in order to accept the risk, and only one thumbs down to stick with the less-risky option. The ESB stays.

In the business world, they say that culture eats strategy for breakfast—meaning that the people implementing the strategy, and the skills, attitudes, and assumptions they bring to it, will make more difference than even the most brilliant plan. In government, culture eats policy. Even when legislators and policymakers try to give implementers the flexibility to exercise judgment, the words they write take on an entirely different meaning, and have a very different effect, as they descend through the hierarchy, becoming more rigid with every step. 

The irony is how eagerly each rung in the hierarchy is trying to please its superiors. In the case of the obstructionist ESB, the CIO Council’s specificity was part of how they impressed Congress with their competence. Raytheon wanted to be very thorough. Everyone in between wanted to ensure full compliance with the directives they’d been issued. No one was trying to sabotage the project, and there was nothing like gross negligence in sight. In fact, if anything, the team was trying to do its job too diligently. Everyone was operating just as they were expected to, but as Weaver concluded from his work on the satellites, “almost all the outcomes generated by technical policy will be unintended consequences.” The tiller feels disconnected from the rudder.

Why hiring the right people is so hard

Culture eats policy governing technology in a special way, but risk-averse culture is an omnivore. One of its most destructive appetites is for civil service rules. Today’s civil service dates back to reforms in the mid- to late-19th century, when positions in government were filled through patronage. Starting with passage of the Pendleton Act in 1883, reformers have sought to ensure that federal employees are no longer hired or fired for arbitrary or political reasons, but on the basis of their skills and job performance. It’s the right objective, but implementation has wandered badly off course.

I first encountered how this system works in practice when my colleague Marina Nitze tried to hire a web designer. This was 2014, and Marina was the Chief Technology Officer of the Department of Veterans Affairs, charged with “redefining the art of the possible for how America honors and serves its veterans” but assigned no staff. By the time she finally secured approval for headcount, she’d already delivered two websites for the VA by just building them herself, and what she needed most was a web designer. She still had a desk in the White House office where I was working, and because I was trying to stand up a unit that would eventually also hire web designers and other technologists, I decided to follow along for her hiring journey.

It took countless hours to get her web designer job approved. There was no job classification for web designer, but she worked with the VA Human Resources team to adapt an existing IT job posting. Getting it posted seemed like an enormous accomplishment. Everyone in the Office of Science and Technology Policy – most of us with deep networks in consumer tech – told the best designers we knew about this amazing job, one that would literally save lives and let you work with the First Lady, and many of them applied. At that point, the HR team took over. The next steps were a black box to Marina.

A few months later, she got her first “cert.” Cert is short for certificate, and it’s just a list of candidates who, according to HR, meet the criteria for the job. This cert included no web designers. When I first heard this, I wondered if Marina was being picky. Marina had very high standards for herself; perhaps she had set too high a bar for this new role. But no, she meant it literally. It wasn’t just that none of the great web designers who had applied were on the list. There weren’t any great designers, or any good designers, or even any beginning designers. The people on the list simply weren’t designers at all. 

This result is not uncommon in federal hiring. It’s why almost half of all competitive, open-to-the-public job announcements result in no hire at all. If a hiring manager (like Marina) believes none of the candidates HR passes along to them can do the job, they can reject the cert. And they do, understandably, though at great cost. A hiring action can easily take months to get to a cert, so when you have to start that process over again, you’re burning shocking amounts of time and attention.

But what happens between the time a qualified candidate applies and the time the cert is issued without them on it? For one thing, a candidate might submit the kind of resume that is standard in business and nonprofits, usually one to two pages, with a high-level description of the duties of each relevant job, and a list of skills. But there’s a thing called a government resume. Government resumes are at least six pages, often longer. They have to have certain keywords. People who know how to write a government resume know what words to include that will get them through the HR review. 

One candidate who somewhat famously didn’t know how to write a government resume is Jack Cable. When the Pentagon sponsored a contest to see who could find the most security flaws in its software, they invited over 600 security researchers to compete. Jack beat them all, winning the contest and demonstrating not only his enormous skills in securing critical national security systems, but an incredible enthusiasm for serving his country. He was a dream candidate, and the Defense Digital Service (DDS), the team that had sponsored the Hack the Pentagon contest, encouraged Jack to apply for a job. But the resume Jack submitted described his experience developing “mobile applications in IonicJS, mobile applications using Angular, and APIs using Node.js, MongoDB, npm, Express gulp, and Babel”. This would have given a technical manager a good sense of the range of his skills, but no one technical reviewed his resume. DoD’s hiring protocols, like those of most agencies, required that it be reviewed by an HR staffer with a background in government hiring rules, not technology. The staffer saw what looked like a grab bag of gobbledygook and tried to match it to the job description, which required “experience that demonstrated accomplishment of computer-project assignments that required a wide range of knowledge of computer requirements and techniques pertinent to the position to be filled.” The fact that he’d just beat out 600 other security researchers meant nothing. His resume was deemed “not minimally qualified” and didn’t make the first cut.

Jack did end up working for the DoD, after DDS team members and others reached out and coached both him and various HR teams through the process, but it took many tries to get him in. They had to overcome several obstacles besides his non-compliant resume, including Jack’s age — he was 17 when he won the contest. In the course of the saga, an HR professional advised this highly sought-after security researcher to get a job selling computers at Best Buy for a few years and come back, because then he might be qualified for the job he was applying for. If the DDS team hadn’t persisted, he would have taken a much higher-paying coding job at a company that could immediately recognize his value, and he might have been lost to public service for years, if not the rest of his career. 

Jack’s tech skills meant nothing to the hiring process, but there is one skill that is always valued: the ability to cut and paste. Yadira Sanchez, a tech team leader at the Centers for Medicare and Medicaid Services (CMS), described to me her attempt to hire a product manager. Like Marina, she knew there were some extremely well-qualified candidates in her pool, some of whom were already doing a great job on the project as contractors. She avoided the mistake the DDS team had made, reminding applicants to get some help from someone who knew how to write a government resume. But none of those candidates made the cert, and in fact, the cert in fact contained no one with product-management experience. The top candidate had just “copied and pasted the exact same language in the exact same font from the bulleted list in the posting into their resume, and that qualified them,” Yadira told me. “They didn’t even put any other language around it. They didn’t even try to disguise what they’d done.” And yet, HR insisted this person was the most qualified for the job per their process.

But why do so many HR teams insist on a process that results in these unqualified candidates and the failure of half their hiring actions? Unhelpful and overly restrictive interpretations of the principle of equity collide with large candidate pools to create a decidedly inequitable and inefficient process. There can be hundreds, even thousands of applicants for a job. HR teams are supposed to consider every applicant with the same level of scrutiny, which makes assessment of large pools of candidates an enormous lift. The way to do that both quickly and “fairly” is to exercise as little judgment as possible. 

The first tool in the HR toolbox at this stage is the self-assessment. This is essentially a questionnaire asking each applicant to rate themselves on the skills listed in the position description. The HR team sends this to all applicants. Anyone who doesn’t return it is automatically out, which starts to narrow the pool. Of those who return it, you can take the candidates with the top scores (effectively those who know to and are willing to assess themselves at the “master” level for every competency), review their resumes for matches with the necessary keywords, and move them on to a pool of “minimally qualified” candidates.

Having only HR generalists with no knowledge of the domain perform screenings is justified by the lack of HR training by domain experts, or Subject Matter Experts (SMEs), as they are known. The sheer volume and finickiness of HR rules, which take many years to master, mean that hiring managers and SMEs are seen as insufficiently steeped in these rules to ensure that the process remains “fair.” If the qualifications include “Leading project management initiatives, methods, and practices to plan and carry out major IT projects,” you look for candidates who say they have “led project management initiatives, methods, and practices to plan and carry out major IT projects.” The closer the match, the fairer you have been. 

At this point, you’ve narrowed your field (and probably knocked out most of your qualified candidates), and you have one important tool left, one that will get you to your cert. There is a notable exception to the rules of equal treatment, found in Title 5 of the U.S. Code, Section 3309, which says, in part, “a preference eligible who receives a passing grade in an examination for entrance into the competitive service is entitled to additional points above his (sic) earned rating.” “Preference eligible” is used here as a noun, and it’s defined earlier in the code with a lot of legalese, but it essentially means a veteran. Once they’ve narrowed the pool through self-assessment and resume screening, HR managers call the remaining candidates “minimally qualified,” apply veterans preference according to a set of very specific rules, and voila, they have a cert they can pass on to the hiring manager.

Title 5 Section 3308 has become the source of much frustration by people trying to hire quickly and well in federal government. But closer examination reveals that the problem is not with the words lawmakers wrote. There is nothing wrong with giving qualified veterans preference over other applicants, and Title 5 states clearly that veterans preference should be applied after candidates are qualified through an examination. What’s wrong here is the way the law has been interpreted and codified into processes and procedures that HR staff in government now believe are the only way to hire. They believe that this process is consistent with the law, despite the fact that it is arguably illegal and certainly ineffective. The process as it has evolved has taken an entirely reasonable and, in my view, admirable attempt to support our nation’s veterans and made it harmful to the civil service and to anyone wanting to work in the public interest. Worst of all, because hiring managers have become conditioned to seeing certs with only veterans on them and assuming none of them is qualified, the process also harms the very people, veterans, that the law was designed to help. 

The good news is that the U.S. Digital Service and the Office of Personnel Management have now devised an alternative hiring process (called SME-QA for Subject Matter Expert Qualifying Assessments) that allows domain experts to work with HR to determine who is qualified and eligible – and it works dramatically better. The bad news is that using this process isn’t required and agencies aren’t exactly scrambling to use it. Changing the culture is possible, but it’s a long slog.

Aligning intent and outcomes

How do we push back against a culture that consistently prioritizes process over results? What can help is peeling back the layers of process that have accumulated on top of policy, and even layers of policy that have accumulated on top of the law, and using judgment to determine whether the outcome is in line with the intent of lawmakers. Did Congress want ESBs in all federal government software? Did the authors of Title 5 Section 3308 want to hire for copy and paste skills, or to engender bias against veterans? Of course not, but the words they wrote that became law were seeds planted in soil doomed to twist and disfigure what grows in it. 

Are our law and policymakers off the hook, then? In these cases, at least, they did their part. The fault lies, it seems, at the feet of the people who implement our laws and policies. But the vast majority of our civil servants are neither lazy, nor stupid, nor ill-intended. They are well-meaning and operating rationally within a set of incentives that are hard to see from the outside, trapped in a dichotomy of accountability. Those incentives include, but are not limited to, the criteria that determine their career trajectories. Public servants who play it safe tend to get promoted. Those who take risks sometimes do as well, but they become risky in and of themselves; promoting someone who operates outside of norms, even someone who operates legally and ethically, can tarnish reputations and make enemies. 

But the culture that enforces those norms doesn’t spring from nowhere. We crafted a system of hierarchy in which those at the top are supposed to make meaningful decisions and every step down the ladder should operate with greater constraints. We create systems designed to drain the jobs of bureaucrats, especially low-level bureaucrats, of any opportunity to exercise judgment. When things go wrong, we find new ways to constrain, and we make the hierarchy more and more rigid. Nicholas Bagley, a law professor at the University of Michigan, has written brilliantly about the “procedure fetish” responsible for so much government dysfunction (see here). In his view, much of the bureaucracy’s obsession with procedure stems from anxiety about the legitimacy of the administrative state. Is it any wonder, in our culture of fake accountability, that our civil servants are eager to legitimize their actions with ever-more specific scripts? 

Lawmakers appear to be in a bind as strangling as our civil servants’ accountability trap. The tools our Constitution grants them are those of rules, money, and oversight, but none of those levers seem to consistently get the results they intend. But ultimately lawmakers cannot blame their way out of their own conundrum of agency. In their oversight role, lawmakers pay attention to failures almost exclusively. They could choose to find and elevate public servants like the ones in my book who exercise judgment in the service of honoring lawmakers’ intent and get the outcomes lawmakers and the public want. They could show how a more flexible, adaptable hierarchy works by inviting those at the lower rungs to help write legislation that will be implementable with less distortion. We won’t lose our essentially hierarchical structure, but our leaders can operate within that structure in a way that can make it better serve our needs.

The key to a healthier culture in our bureaucracy is people. Our infrastructure for recruiting, hiring, promoting, and retaining them has been sorely neglected. It’s not just the ironies of fairness that need a closer look. Despite radical changes in the world around us, the Classification Act of 1949 is the last time we reformed how managerial jobs in the federal merit system are organized. There are vestigial requirements, like the one for industrial-organizational psychologists to review many routine actions, that function as crippling bottlenecks and slow our pipeline to a crawl. Somewhere in some obscure statute is language that ties many promotions to project management training that is sorely outdated. The criteria for ascending into the Senior Executive Service are similarly anachronistic. None of these things represents anything close to a silver bullet, but all of them desperately need attention. We have not been paying attention, and we are suffering the consequences, but we have failed to connect the dots between our neglect and our frustration. 

The policy debates that dominate our national discourse are all about which direction to steer the ship of state. Most of these are important and worthwhile discussions. But many of them would require significant implementation by a bureaucracy ill-equipped to deliver. By paying more attention to the unglamorous but critical machinery of government that connects steering to the rudder, we can begin to change the culture of government, and start getting more of the outcomes our laws and policies intend.