The CFTC and other regulators should evaluate proposals to automate financial transactions with caution.
The crypto firm FTX recently filed an application with the Commodity Futures Trading Commission (CFTC) for authorization to clear margined products for retail investors in an “unbroken” manner. The proposal is complex and raises many concerns about investor protection and financial stability, but FTX’s proposal also raises a more fundamental question that is becoming increasingly relevant to regulators around the world.
How should regulators monitor the software that runs automated systems?
Software has long been part of the business models of many industries, but critical regulated activities can now be performed without human intervention. Just as courts are trying to figure out how to apportion liability for automated decisions with people increasingly “coming out of the loop,” regulators must grapple with a new reality about their role: Rather than just overseeing people, they are they are increasingly software supervisors.
To provide a little more context, the CFTC oversees the derivatives trading and clearing process. The more analogous version of derivatives clearing – the version the CFTC uses to oversee – manages risk by having layers of intermediaries each performing a risk management function. Brokers sit between investors and a clearinghouse, and both brokers and the clearinghouse regularly assess the collateral needed to support trading positions, asking for more margin if necessary.
The human relations involved allow for some discretion. For example, in March 2020, Citi is said to have experienced a technical glitch that prevented it from posting the margin it needed in time, but ICE clearinghouse granted a bit of a pardon and refrained from liquidating Citi’s position.
The recent FTX proposal would deviate from this model, eliminating brokers with their discretionary margin calls and replacing them with software. The software would assess margin requirements every second of every day based on the real-time interpretation of market events. Without discretion or grace, the software would quickly liquidate any investor who does not follow the rules, regardless of the consequences for the individual investor or for the financial markets in general.
If the proposal is approved, a lot will ride on FTX’s software. The software must perform the functions announced by FTX, and the software must also meet minimum reliability and cybersecurity standards.
But who is going to set the minimum standards and who will monitor their compliance? Who is going to check whether the software code as written corresponds to the proposal? Like many industry regulators, the CFTC does not employ a large number of software engineers. So what should an agency do?
Sometimes it’s appropriate for regulators to simply say “no” to automation. Due to the complexity of software code, an automated system can never be fail-safe. And if automation only makes an activity marginally more efficient than the non-automated alternative, then the benefits won’t be worth the risks and the regulator should insist on requiring a “human in the loop.”
However, if automation is deemed desirable, a multi-pronged approach is needed. If the automated system constitutes critical financial infrastructure, the software involved should be designed in accordance with best practice standards for software used in safety-critical environments such as aviation and nuclear power plants. Although the damage that financial companies can cause is sometimes minimized as ‘mere’ financial or economic damage, economic damage can be serious and even translate into physical damage. Just think of the hotline suicide numbers posted on crypto reddits because crypto assets have failed.
Therefore, software used to automate financial infrastructure should be considered security critical. Decisions made during the programming process, such as the choice of code libraries or diagnostic tests to be performed, must follow much stricter standards than equivalent decisions made in connection with the development of a less critical system such as a social media app.
Unfortunately, like many regulatory bodies, the CFTC currently lacks the capacity to assess compliance with these kinds of standards, or even check if regulated companies are misrepresenting what their software is doing. Regulators can and should try to build their own technology capabilities by hiring more software engineers, but competition for these staff can be fierce and government salaries are rarely competitive.
Ideally, Congress would increase agency budgets in proportion to the increased resources needed to oversee automated systems. But it might be more realistic to concentrate this expertise in ‘hub’ agencies. For example, the U.S. Treasury Department’s Office of Financial Research could serve as a center of interdisciplinary expertise for financial regulatory agencies. Alternatively, Congress could revive the US Office of Technology Assessment to serve as a more general government hub.
Until this software oversight expertise is developed within government, allowing a regulated entity to fully automate a critical activity will necessarily involve the regulatory body relinquishing any authority over that activity.
To be clear, even with the necessary expertise, there will be limits to what software standards can achieve. Stringent standards are needed to minimize programming errors, but the complexity of the software means that it will always be vulnerable to “normal accidents”.
Because something will inevitably go wrong with complex software, it’s critical that regulatory authorities also demand a combination of redundancies, frictions, inefficiencies, and backstops so that the public isn’t completely dependent on the automated system performing as expected. Just as pilots need to be able to disable the autopilot and take control of an airplane, financial regulators need circuit breakers and other tools to stop automated transactions.
The backstop FTX has proposed is a $250 million guarantee fund that will be available to absorb losses if needed. It may be impossible to determine with confidence whether this amount will be sufficient to protect the clearinghouse from insolvency, given the difficulties involved in valuing crypto assets and assessing the associated risks. But even assuming $250 million is enough, the guarantee fund will do nothing to protect individual investors who are wrongfully liquidated as a result of a software bug. It will also do nothing to address the systemic risks that could arise if asset prices across the market are affected by a technology glitch that forces a massive liquidation of FTX positions, such as in a “flash crash.”
As the CFTC evaluates FTX’s proposal, the agency should consider other measures that will compensate both for its limited ability to assess the quality of FTX’s technological plumbing, and for the failures that are inevitable even with the highest-quality software. So are other regulators thinking about how to oversee other software-automated systems.