What are the implications for having multiple implementations of the Bitcoin protocol? Do the positives outweigh the potential bugs and other issues?
This is an opinion editorial by Bill Scoresby, a bitcoin-based small business owner and writer of several guides for bitcoin self-custody.
The bugs that recently caused many LND nodes to fall out of sync with the Bitcoin blockchain were probably caused by an alternate implementation.
Perhaps you are wondering, “Who in the world is using anything other than Bitcoin Core?” You might not have known that other implementations of Bitcoin existed. Maybe you’re not sure what a different implementation even means.
Bitcoin Core began as the software that Satoshi Nakamoto wrote in C++ and released to the world. It has been updated with new versions leading to the present day. An alternate implementation is software that does the same thing as Bitcoin Core — enforces the same consensus rules — but is written differently, most often in a different coding language.
One of the major Lightning Network node versions (LND) relies on an alternate Bitcoin implementation called btcd. When a developer created a very large multisig transaction, btcd did not see it as valid because it contained too much witness data. Other Bitcoin implementations — most importantly Bitcoin Core — had no such limit on Taproot transaction witness data, and therefore accepted the transaction and the block that contained it as valid.
The result was miners kept adding new blocks on the chain because they weren’t using btcd and according to their rules nothing was wrong, but LND Lightning nodes couldn’t recognize any of these new blocks because they were built on top of the block containing that one transaction they saw as invalid.
When the bug happened again on November 1, it wasn’t just LND nodes that were affected. Some electrs instances (an implementation of the backend server for Electrum Wallet) also failed to reach consensus with the rest of the chain. While the LND nodes were put out of consensus because of a similar issue in btcd, it was an implementation of Bitcoin written in Rust that caused the electrs nodes to fall behind, including some very visible servers run by mempool.space.
The limit on the size of witness data exists to prevent DoS attacks, and is a part of Bitcoin Core as well ( although Core has a larger limit for Taproot transactions). It seems that the other two implementations that fell out of sync had code that maintained the smaller limit.
Very small differences in implementations can lead to a lack of consensus.
Satoshi didn’t like the idea of multiple implementations of Bitcoin. “I don’t believe a second, compatible implementation of Bitcoin will ever be a good idea.” The reason he gave was, “So much of the design depends on all nodes getting exactly identical results in lockstep that a second implementation would be a menace to the network.”
Menace? What’s the big deal?
You’ve probably heard that the chain with the most proof-of-work is the true chain. When two different miners find a block at the same time, the chain splits and other miners start building on whichever block they hear about first.
As soon as a new block is added to one side of the split, most nodes and miners accept that as the new true chain and abandon the other side of the split. These blocks are referred to as stale blocks, though some people call them orphaned blocks.
Since the average time between blocks in Bitcoin is 10 minutes, it is likely the whole network will learn about this new block before one is added to the losing side of the split, and the chain with the most work wins.
“Nodes will follow the valid chain with the most work…The key word here is valid. If the node receives a block that it determines invalid, it doesn’t matter how much work is done on top of that block, the node will not accept that chain.” — Andrew Chow
The key word is “valid.” The menace shows up when a miner finds a block that some other miners and nodes think isn’t valid. Miners who think it is valid will try to build new blocks on that chain. Miners who think it isn’t valid will try to build on the last valid block they know about. The result: Two chains and no way to know which is true.
How in the world would such a thing come to pass?
Well, as we saw in the case of the recent bug with LND nodes, if there is a bug in one implementation of Bitcoin that is not in other implementations, it can lead to a lack of consensus about whether a block is valid or not.
Bitcoin does not have a mechanism for fixing this. The community outside of the protocol has to decide what happens next. It sounds very unpleasant.
So much so that Bitcoin developer Peter Todd has said that other implementations need to match Bitcoin Core bug-for-bug.
There you go: Multiple implementations are dangerous!
First of all, most everyone runs Bitcoin Core.
Luke Dashjr sees about 43,000 nodes, 98% of which are running Bitcoin Core and something called Coin Dance sees close to 15,000 nodes, 96% of which are running Bitcoin Core. So, at the moment, it looks like very few people are using alternate implementations.
Nevertheless, there are active projects that are trying to build and maintain other codebases that implement the Bitcoin protocol. They include:
Bitcoin Knots.Libbitcoin.nix-bitcoin.btcd.
Jameson Lopp has an excellent page with a more exhaustive list and links to all of the other implementations.
All of these projects have extremely talented developers working on them, and each has existed for more than a few years. Why put so much effort into something that seems like such a problem?
Bitcoin is permissionless. Anyone can download the chain; anyone can interact with the network; and nobody can stop you from coding or running an alternate implementation.
Yet, clearly some people are in charge of making changes to the Bitcoin repository and the process for choosing them seems informal. While there is the Bitcoin Improvement Proposal (BIP) process for suggesting changes to Bitcoin Core, it is also pretty informal.
None of this is a direct problem. As Marty Bent points out, rough consensus can be a strength. If the process of changing Bitcoin is difficult and unclear, it means that changes will be more thoroughly vetted.
The next step of rough consensus is having more than one popular implementation.
There can be no doubt that it is already a very difficult job to be one of the people who has commit access to Bitcoin Core. In a world where Bitcoin plays a central role as a monetary instrument, this job will get much more difficult. A small group of developers could become a very worthwhile target. At the very least, their attention will be sought in order to lobby for various inclusions or exclusions in the next software release.
Think about the lobbying industry that currently exists in politics. Why wouldn’t such a thing develop around the people who have commit access to the only implementation of the Bitcoin protocol?
Like politicians now, they will be perceived to have access to power. As such, people will target them, except these developers won’t have the muscle of a state to defend them. What kind of life is that going to be? Who would voluntarily choose it?
At the end of the day, the global financial system is a pretty heavy weight to rest on the shoulders of the small group of people who have commit access to one GitHub repository. Maybe not so different from the global financial system we are trying to get away from where people’s monetary future hinges on the decisions of a few central bankers.
The presence and widespread use of multiple implementations on the Bitcoin network can mitigate these pressures by making it much more difficult for a malicious actor to change the Bitcoin protocol.
If participants in the Bitcoin network are more evenly distributed among different implementations, there is more room for good ideas to surface. Proposing changes to Bitcoin or rejecting them is a lot more decentralized if it isn’t all done in one camp.
Clearly, using different implementations of Bitcoin increases the risk of a chain split. A catastrophic chain split — where a significant portion of nodes and miners accidentally forked off — would not be good for Bitcoin, and certainly not its price. But it wouldn’t threaten Bitcoin’s permissionless nature.
A centralized development environment where everyone only builds on Bitcoin Core could threaten permissionless-ness. The conversation about the topic needs to address the risks of relying so heavily on Bitcoin Core rather than focusing solely on what problems might be caused by an alternate implementation.
There is a great, older article about this debate by Aaron van Wirdum. You can also read a more recent, informative thread about it.
This is a guest post by Bill Scoresby. Opinions expressed are entirely their own and do not necessarily reflect those of BTC Inc or Bitcoin Magazine.
This is an opinion editorial by Bill Scoresby, a bitcoin-based small business owner and writer of several guides for bitcoin self-custody.
The bugs that recently caused many LND nodes to fall out of sync with the Bitcoin blockchain were probably caused by an alternate implementation.
Perhaps you are wondering, “Who in the world is using anything other than Bitcoin Core?” You might not have known that other implementations of Bitcoin existed. Maybe you’re not sure what a different implementation even means.
Bitcoin Core began as the software that Satoshi Nakamoto wrote in C++ and released to the world. It has been updated with new versions leading to the present day. An alternate implementation is software that does the same thing as Bitcoin Core — enforces the same consensus rules — but is written differently, most often in a different coding language.
One of the major Lightning Network node versions (LND) relies on an alternate Bitcoin implementation called btcd. When a developer created a very large multisig transaction, btcd did not see it as valid because it contained too much witness data. Other Bitcoin implementations — most importantly Bitcoin Core — had no such limit on Taproot transaction witness data, and therefore accepted the transaction and the block that contained it as valid.
The result was miners kept adding new blocks on the chain because they weren’t using btcd and according to their rules nothing was wrong, but LND Lightning nodes couldn’t recognize any of these new blocks because they were built on top of the block containing that one transaction they saw as invalid.
When the bug happened again on November 1, it wasn’t just LND nodes that were affected. Some electrs instances (an implementation of the backend server for Electrum Wallet) also failed to reach consensus with the rest of the chain. While the LND nodes were put out of consensus because of a similar issue in btcd, it was animplementation of Bitcoin written in Rust that caused the electrs nodes to fall behind, including some very visible servers run by mempool.space.
The limit on the size of witness dataexists to prevent DoS attacks, and is a part of Bitcoin Core as well ( although Core has a larger limit for Taproot transactions). It seems that the other two implementations that fell out of sync had code that maintained the smaller limit.
Very small differences in implementations can lead to a lack of consensus.
Satoshi didn’t like the idea of multiple implementations of Bitcoin. “I don’t believe a second, compatible implementation of Bitcoin will ever be a good idea.” The reason he gave was, “So much of the design depends on all nodes getting exactly identical results in lockstep that a second implementation would be a menace to the network.”
Menace? What’s the big deal?
You’ve probably heard that the chain with the most proof-of-work is the true chain. When two different miners find a block at the same time, the chain splits and other miners start building on whichever block they hear about first.
As soon as a new block is added to one side of the split, most nodes and miners accept that as the new true chain and abandon the other side of the split. These blocks are referred to as stale blocks, though some people call them orphaned blocks.
Since the average time between blocks in Bitcoin is 10 minutes, it is likely the whole network will learn about this new block before one is added to the losing side of the split, and the chain with the most work wins.
“Nodes will follow the valid chain with the most work…The key word here is valid. If the node receives a block that it determines invalid, it doesn’t matter how much work is done on top of that block, the node will not accept that chain.” — Andrew Chow
The key word is “valid.” The menace shows up when a miner finds a block that some other miners and nodes think isn’t valid. Miners who think it is valid will try to build new blocks on that chain. Miners who think it isn’t valid will try to build on the last valid block they know about. The result: Two chains and no way to know which is true.
How in the world would such a thing come to pass?
Well, as we saw in the case of the recent bug with LND nodes, if there is a bug in one implementation of Bitcoin that is not in other implementations, it can lead to a lack of consensus about whether a block is valid or not.
Bitcoin does not have a mechanism for fixing this. The community outside of the protocol has to decide what happens next. It sounds very unpleasant.
So much so that Bitcoin developer Peter Todd has said that other implementations need to match Bitcoin Core bug-for-bug.
There you go: Multiple implementations are dangerous!
First of all, most everyone runs Bitcoin Core.
Luke Dashjr sees about 43,000 nodes, 98% of which are running Bitcoin Core and something called Coin Dance sees close to 15,000 nodes, 96% of which are running Bitcoin Core. So, at the moment, it looks like very few people are using alternate implementations.
Nevertheless, there are active projects that are trying to build and maintain other codebases that implement the Bitcoin protocol. They include:
Bitcoin Knots.Libbitcoin.nix-bitcoin.btcd.
Jameson Lopp has an excellent page with a more exhaustive list and links to all of the other implementations.
All of these projects have extremely talented developers working on them, and each has existed for more than a few years. Why put so much effort into something that seems like such a problem?
Bitcoin is permissionless. Anyone can download the chain; anyone can interact with the network; and nobody can stop you from coding or running an alternate implementation.
Yet, clearly some people are in charge of making changes to the Bitcoin repository and the process for choosing them seems informal. While there is the Bitcoin Improvement Proposal (BIP) process for suggesting changes to Bitcoin Core, it is also pretty informal.
None of this is a direct problem. As Marty Bent points out, rough consensus can be a strength. If the process of changing Bitcoin is difficult and unclear, it means that changes will be more thoroughly vetted.
The next step of rough consensus is having more than one popular implementation.
There can be no doubt that it is already a very difficult job to be one of the people who has commit access to Bitcoin Core. In a world where Bitcoin plays a central role as a monetary instrument, this job will get much more difficult. A small group of developers could become a very worthwhile target. At the very least, their attention will be sought in order to lobby for various inclusions or exclusions in the next software release.
Think about the lobbying industry that currently exists in politics. Why wouldn’t such a thing develop around the people who have commit access to the only implementation of the Bitcoin protocol?
Like politicians now, they will be perceived to have access to power. As such, people will target them, except these developers won’t have the muscle of a state to defend them. What kind of life is that going to be? Who would voluntarily choose it?
At the end of the day, the global financial system is a pretty heavy weight to rest on the shoulders of the small group of people who have commit access to one GitHub repository. Maybe not so different from the global financial system we are trying to get away from where people’s monetary future hinges on the decisions of a few central bankers.
The presence and widespread use of multiple implementations on the Bitcoin network can mitigate these pressures by making it much more difficult for a malicious actor to change the Bitcoin protocol.
If participants in the Bitcoin network are more evenly distributed among different implementations, there is more room for good ideas to surface. Proposing changes to Bitcoin or rejecting them is a lot more decentralized if it isn’t all done in one camp.
Clearly, using different implementations of Bitcoin increases the risk of a chain split. A catastrophic chain split — where a significant portion of nodes and miners accidentally forked off — would not be good for Bitcoin, and certainly not its price. But it wouldn’t threaten Bitcoin’s permissionless nature.
A centralized development environment where everyone only builds on Bitcoin Core could threaten permissionless-ness. The conversation about the topic needs to address the risks of relying so heavily on Bitcoin Core rather than focusing solely on what problems might be caused by an alternate implementation.
There is a great, older article about this debate by Aaron van Wirdum. You can also read a more recent, informative thread about it.
This is a guest post by Bill Scoresby. Opinions expressed are entirely their own and do not necessarily reflect those of BTC Inc or Bitcoin Magazine.
Bitcoin Magazine – Bitcoin News, Articles and Expert Insights
Bitcoin (BTC), which had traded in a tight range for most of Wednesday, suddenly surged…
Follow Nikolaus On 𝕏 Here For Daily Posts Today, the Chairman of the Federal Reserve,…
Having long lagged behind bitcoin (BTC) in terms of price action, ether (ETH) saw some…
A small, but significant development in Bitcoin’s much-debated decentralized consensus process is taking place, with…
Welcome to The Protocol, CoinDesk's weekly wrap-up of the most important stories in cryptocurrency tech…
After the election, as everyone except for muggles know, the price of BTC rallied to…