Private sector key to the success of Japan’s digital agency by MUKOYAMA Jun

“API Geoeconomic Briefing” is a weekly analysis of significant geopolitical and geoeconomic developments that precede the post-pandemic world. The briefing is written by experts at Asia Pacific Initiative (API) and includes an assessment of burgeoning trends in international politics and economics and the possible impact on Japan’s national interests and strategic response. (Editor-in-chief: Dr. HOSOYA Yuichi, Research Director, API & Professor, Faculty of Law, Keio University)

This article was posted to the Japan Times on August 22, 2021:

API Geoeconomic Briefing

August 22, 2021

Private sector key to the success of Japan’s digital agency

Fellow, Asia Pacific Initiative (API)





Following the 9/11 attacks in 2001, the FBI started to develop an innovative digital information sharing system based on a report by the National Commission on Terrorist Attacks Upon the United States, an independent investigation body, which pointed to its limited capacity to share information both internally and externally.

Although there were fragments of information that could have helped prevent the attacks — including on the movements of people who could have been watchlisted and suspected terrorists who obtained flight training — the FBI failed to piece them together, share them and take effective action as an organization, something that was hampered by its inadequate information system and paper-based case files.

Based on the lessons learned, the FBI developed Sentinel, a system to share case data electronically.

However, efforts to develop the system failed twice, despite some $400 million paid to the contractor, Lockheed Martin.

After wasting more than 10 years, the FBI took over the final development of Sentinel and, by employing Agile methodology and the Scrum process framework, finally managed to release the system in 2012.

The Japanese government is not alone in struggling with system development, and there are countless cases worldwide of governments facing similar difficulties. For example, the failed launch of the website — former U.S. President Barack Obama’s signature policy — as well as system procurement failures in the United Kingdom and Australia, which led to reforms of the government’s digital departments in those countries.

Examples of such failures in Japan are the patent system, which was not completed even after eight years and ¥5.5 billion spent since 2004, and more recently, an online system that was introduced amid the COVID-19 pandemic to accept applications for employment subsidies but was suspended immediately after launch.

Various administrative practices have often served as barriers to efficient system development: a single-year budget cycle; a procurement process based on the waterfall methodology, where each development phase must be completed before the next phase can begin; the lack of a competitive environment; vertically segmented ministries and agencies; decision-makers who lack expertise and leaving everything to developers; and the principle of infallibility.

Two decades have passed since the government launched the e-Japan strategy to seriously work on information technology reform.

The government has taken a number of measures to make step-by-step improvements, including splitting orders among multiple contractors to avoid entrusting everything to one entity, creating the post of chief information officer to manage IT-related procurement and hiring deputy CIOs who look into requirement definition documents and offer expert advice.

However, the priority of such policies was not high enough to secure sufficient authority or human resources to realize the planned programs.

And the failure to respond fully to such issues was brought to light amid the coronavirus pandemic.

Can the digital agency, to be established in September, overcome such failures of the past with a sense of urgency?


Private sector personnel

Upon establishment, the agency will be given the authority to manage the administrative systems-related budgets of ministries and agencies, and it looks like the agency has so far been successful in securing human resources.

But in order to breathe life into the agency, it is necessary to break away from longstanding bureaucratic practices in a true sense. The agency should make decisions and set goals driven by data and expertise, achieve results for the people — the main customers — by utilizing cutting-edge tools, and become an organization that evolves constantly.

Is it possible for the agency to become an innovative institution? The key will be whether professionals from the private sector can get involved in the process of making important decisions.

About 120 out of roughly 500 people set to work for the agency are coming from the private sector, and it has taken the ambitious approach of allowing such people to adopt new work styles such as working part time or remotely.

Recruitment has been conducted openly four times so far for project managers, engineers, senior management posts and experts in such fields as digital health and education. Detailed job descriptions and qualification requirements were provided, which is rare in job recruitment by ministries and agencies.

The move attracted a lot of attention, with 1,432 people applying in the first round of recruitment for 33 posts.

More ministries and agencies have been accepting people from the private sector lately, with the Financial Services Agency hiring specialists with the latest knowledge, for instance.

However, such appointments are mostly not found for higher positions, as those posts are maintained for career-track civil servants employed as new graduates on the premise of lifetime employment and nurtured within the bureaucratic system.

In the United Kingdom, which has a parliamentary Cabinet system like Japan, the government employs many people from the private sector in the fields where their expertise can be utilized, such as IT, real estate management, finance and medicine.

People from the private sector make up as much as 20% of staff members, even among senior civil servants such as directors and senior executive officers.

Mid-career hires occupy 15% of not only specialist and administrative support roles but also those with policy responsibilities.

Meanwhile, South Korea, although having labor practices similar to those of Japan, was ranked second in the E-Government Development Index of the United Nations’ Department of Economic and Social Affairs in 2020, thanks to two decades of efforts to utilize specialists in the bureaucracy.

Out of 670 workers at the National Information Society Agency — South Korea’s core agency for managing and supporting information systems — half of them came from the private sector and 90% hold a doctorate degree.

The agency hires specialists not only in the IT field but also in other areas necessary to construct an e-government, such as law, medicine, education and finance.

When South Korea was hit by the 1997-1998 financial crisis and was subsequently bailed out by the International Monetary Fund, inefficient government and bureaucracy were blamed, forcing the country to conduct reforms within the civil service.

The reforms focused on improving expertise, as there were criticisms over the frequent rotation of officials in government agencies within a short-term cycle of a year or so, which hampered the accumulation of professional knowledge.

In 2014, the South Korean government categorized all of its workers according to the need for long-term service and the level of expertise, making a major shift from the postwar personnel system centered on generalists to a dual system focusing also on specialists.


Japanese bureaucracy

Under Japan’s traditional lifetime employment and seniority system in the central government, a single mistake could ruin a bureaucratic career.

Such a system makes bureaucrats follow precedent and reluctant to take risks and make changes.

Moreover, about half of career bureaucrats are typical generalists with a background in law or social sciences, rotating posts every two years and starting again from scratch to build up expertise in a different field.

On the other hand, it is inevitable that the digital agency will have to nurture and utilize specialists with the latest knowledge to achieve its goals.

Considering that we are in an era where perspectives based on data and computer science are indispensable in making policies, not only the digital agency but all government ministries and agencies need to have highly specialized people.

The digital agency’s great experiment of hiring chief officers from the private sector — raising issues of how to create a new career path for specialists aside from the traditional career course for bureaucrats and how to treat people from the private sector in the personnel system — is expected to have an influence on the government’s overall personnel management.

The digital agency’s personnel system will serve as a role model that can also have an impact on businesses.

In Japan, which is facing a shortage of IT-related workers, it is of great significance that the digital agency will provide talented engineers with the chance to develop systems in what will become the nation’s largest data accumulation center and platform.

Not only can the agency become a source of such workers, it has the potential to bring about innovation in the entire IT industry by changing the way the government procures systems through its annual IT-related budget of some ¥700 billion.

About 70% of Japan’s IT-related workers are employed by vendor firms, which provide services and products, rather than user firms.

While U.S. firms make IT-related investments in a variety of areas — in-house development, outsourced development and the utilization of software packages, for example — 80% of such investments by Japanese firms are concentrated in outsourced development.

Because Japanese-style employment practices do not allow firms to be flexible in making adjustments to the number of IT-related workers depending on individual projects, many Japanese firms have been outsourcing development to third-party vendors.

Even in the private sector, firms are leaving everything to contractors just like ministries and agencies do. This is why digital technologies are treated in Japan less as a source of new business opportunities and more passively as a means to streamline or automate daily tasks.


Coping with COVID-19

As we enter an era where digital technologies directly add value to businesses and services, more companies will start nurturing specialists internally.

The government is being criticized for lacking ideas on how to utilize digital tools to cope with the COVID-19 pandemic.

If the digital agency effectively makes use of specialists to meet its strategic objectives, it can serve as a role model for the private sector to be more proactive regarding digitalization.

Jeff Sutherland, co-creator of the Scrum process framework adopted in the development of Sentinel and one of the 17 engineers who jointly released the Manifesto for Agile Software Development in 2001, has said: “Change or Die. Clinging to the old way of doing things, of command and control and rigid predictability, will bring only failure. In the meantime, the competition that is willing to change will leave you in the dust.”

The United States went through a major reform following its failure to prevent the 9/11 terrorist attacks, and so did South Korea after the 1997 bailout by the IMF. Now the question is this: Can Japan turn defeat into a chance for digital reform?


Disclaimer: The views expressed in this API Geoeconomic Briefing do not necessarily reflect those of the API, the API Institute of Geoeconomic Studies or any other organizations to which the author belongs.