Bitcoin mining pools rely on centralization, but P2Pool and other protocols have sought to reduce the need for trust in third parties.
In a previous article, I wrote about the nature of decentralization versus centralization in Bitcoin mining and how to conceptualize that in a mostly qualitative sense. The article broke down the entire mining stack, from pool coordination all the way down to energy production to give a sense of the relationship between different layers of the mining stack and the potential to maximize decentralization, making the point that the further down the stack you go toward energy production, the more difficult and capital intensive it becomes to bring a meaningful level of decentralization to that layer.
In this article, I intend to go deeper into the topic of mining pools and miner coordination to facilitate independently-owned mining operations cooperating in an effort to mine blocks to append to the blockchain.
The Creation Of Mining Pools
Mining has come a long way since the days when you could simply click a button and reliably mine blocks all by yourself on a laptop CPU. Back then, it was effectively an amateur hobbyist endeavor that required no real capital investment or expertise, but nowadays it is a multi-billion dollar professional market with massive capital investment required at scale. It is a whole different ball game.
One of the natural consequences of this shift in the nature of the mining industry was the creation of mining pools very early on. When mining was effectively leaving a laptop running in the corner, the variance and unpredictability of when you would find a block was not really that big of a deal — eventually, you would and the power cost of keeping a laptop running was not really of economic importance.
Once things shifted to GPUs and ASICs, there was a material investment cost up front and a much more significant electricity cost to keep them operating. That unpredictability in when you would find a block became a much bigger issue to miners attempting to make a return on their capital investments and operate profitably. This is where mining pools come into the picture.
They allow miners to cooperate on finding a valid block header working on the same block together, sending the coinbase reward to the mining pool, which then distributes it proportionally among all of the participating miners relative to how much work they have done in helping to find the block. This is proven by turning in “shares” to the mining pool; blocks that do not meet the network difficulty target but are high enough to prove that a miner is not lying and is actually running hardware and trying to find a valid block.
Mining Pool Centralization
Centralized mining pools come with big implications for individual miners. They are a point of centralization in the process of selecting (or, more importantly, excluding) transactions for inclusion in a block. This gives each mining pool operator total control over the transactions that they choose to process on the blockchain, with no ability for the actual owners of the mining hardware to exert a say on that except by leaving the pool if they disagree with the criteria that the operator chooses to set.
They also custody individual miners’ bitcoin until those miners choose to withdraw them from the pool, leaving the pool operator acting as a custodian and central point that could defraud miners using the pool, or be pressured by governments to seize individual miners funds or apply KYC requirements to them.
So, what solutions exist to address this problem?
P2Pool: The Original Decentralized Mining Pool
P2Pool is the original decentralized mining pool protocol. It is a peer-to-peer protocol in which miners coordinate among themselves to split up the mining rewards as they work together to find a valid block that meets the difficulty target. This coordination is accomplished using what the protocol design refers to as a “sharechain.”
Miners in the P2Pool take blocks that do not meet the difficulty target of the network, and effectively mine their own blockchain composed of all of the copies of the single block that the pool is working on. When they meet the smaller difficulty target where the block would be turned into a pool to prove they are mining in a centralized model, they broadcast that block to the rest of the miners. P2Pool’s “share difficulty” was targeted so that miners would find a share roughly once every thirty seconds.
I’m sure readers are wondering how the payout to individual miners works. The coinbase transaction is structured so that an output is created for each individual miner in the P2Pool, splitting up funds directly from the coinbase transaction. Miners in the P2Pool verify that all of the payouts to themselves and everyone participating in the pool is correct, and that each miner who has contributed a share to the sharechain is correctly paid out for their work in each new share added. If some participating miner is not correctly structuring payouts to everyone in their latest share, then all of the other miners in the P2Pool stop including them in their own payouts and effectively “evict” that miner from the pool for not behaving fairly.
This design led to a few scaling problems which is why it is not in use anymore. As the participation in a P2Pool grows, so does the aggregate difficulty target for shares in the sharechain, which target roughly once every thirty seconds. This means that for smaller miners it becomes more difficult to reach the sharechain difficulty within any thirty-second period. This means that for smaller miners the variance, or unpredictability, in their income increases as the aggregate hash rate in a P2Pool increases. This also leads to a larger number of stale blocks as more miners are finding competing shares for the sharechain at roughly the same time as more miners join the P2Pool, leading to “wasted work” from the point of view of the individual miners who only get compensated if their share is included in the sharechain.
The other main scalability issue is in the payouts going directly to individual miners from the coinbase transaction itself. Given that each miner is paid out in proportion to the shares they’ve mined that are included in the sharechain, each miner in the P2Pool requires a new output added in the coinbase transaction.
This has two consequences. First: Miners are being given small, low-value UTXOs in every block the P2Pool finds, which comes with the cost of condensing those outputs later on and/or bearing the cost of much larger transactions when they go to spend their coins because of the numerous individual UTXOs they wind up with, instead of a single one when withdrawing after a period from a conventional pool. Second: Each new coinbase output is taking up blockspace that could be consumed by other people’s transactions and earning the P2Pool more in fee revenue. It’s a double whammy loss for miners participating in the protocol.
These two issues contributed to the protocol slowly dying off and eventually falling into a state of disuse. By all indications from my best effort to track down accurate and recent statistics (many old block explorers tracking mining pool share have shut down over the years), it seems like the last block mined by P2Pool was on February 12, 2019.
P2Pool With Payment Channel Payouts
In 2017, a month after SegWit activation locked in, Chris Belcher made a proposal to improve the scalability of P2Pool with the use of one-way payment channels and a group of hubs handling payouts to the miners.
The core purpose of the proposal is to address the issue of the larger coinbase transactions losing miners’ money in two different ways. At a high level, the idea is simply to pay the entire coinbase transaction out to a hub with payment channels open to the individual miners, and guarantee that the ability to claim the funds from the coinbase transaction is atomically linked to the miners being compensated for their shares over the payment channels.
To accomplish the goal of atomicity between the coinbase transaction and payment channels for payouts, the coinbase transaction output script has to be customized. In Belcher’s proposal, it is structured as a multi-branch script with three spending conditions:
A two-of-two multisig. Key one: the hub (Hc). Key two: miner who found the block (Mc).A single key and a hashlock. Key: the hub (H). Hashlock: a random value generated by the hub (X).A single key and a timelock. Key: the miner who found the block (M). Timelock: a CSV relative timelock of six months.
Any one of these spending conditions can be used to unlock the coinbase transaction output. Now, let’s look at the payment channel script to the miners so we can see how the two things interact:
A two-of-two multisig. Key one: the hub (Hc1). Key two: the miner (Mc1).A two-of-two multisig and a hashlock. Key one: the hub (Hu1). Key two: the miner (Mu1). Hashlock: the random value generated by the hub used in the coinbase (X).
Now, let’s walk through how these two things interact with each other.
As miners are producing shares to add to the sharechain, the hub monitors the progress. For each share, the hub updates the state of the channel with miners who turn in a share to pay them proportionally to the amount of work they are doing. However, they only give them a signature for the second script path that requires the hashlock preimage to spend — this guarantees that by default, without the hub giving them a signature for the first path, they cannot claim those funds unless the hub spends the coinbase output by itself using the script path with the hashlock, which requires them to publish the preimage.
Now, eventually one of the miners in the P2Pool will find a valid block and publish it to the network. At this point, the hub can update all of the payment channels with the miners and provide the signature to the first script path in the channel, allowing each miner to close their channel and collect the rewards they have earned whenever they want without needing the hashlock preimage.
At this point, the miner who found the block signs the first script path in the coinbase, allowing the hub to claim the funds from the coinbase. That miner is given a slight bonus from the mining rewards to incentivize them to sign cooperatively. But remember: if the miner refuses to cooperate, the hub can simply spend by itself using the hashlock path and reveal the preimage, allowing all of the miners to collect their fair share of the reward.
This just has the downside of forcing all of the channels to close on chain, needing to be reopened to continue mining. The last option exists in case the hub operator chooses to stop processing payouts, or disappears. After six months, the miner who found the block can claim the funds entirely for themselves if the hub hasn’t responded to cooperate or has spent the coins with the hashlock path.
This leaves two specific issues in terms of threat model with Belcher’s proposed improvements. Deciding which transactions to include in a block leaves room for variance in how much of the total block reward is based on what individual miners choose to include in the block templates they are mining.
When introducing payment channels, this creates a margin for error, i.e., the actual mined block reward doesn’t equal what a mining hub commits to in the payment channels to miners. In the case that actual fee estimates are smaller than what the block reward was, the hub can simply update the payment channels using the cooperative spend path with the lesser amount, and as long as they do not claim the coinbase output with the hashlock path, the miners have no choice but to accept the lesser payout that matches what the mining reward actually was.
In the case that the mining reward was slightly higher than the estimate, it is still in the hub’s best interest to update the channels to the miners to reflect this, as miners that the hub treats dishonestly can simply leave at any time. The only edge case where it might make sense for the hub to defect and keep the extra reward would be if someone paid an abnormally large miner fee, but aside from that situation, it’s in the hub’s and miners’ interests to adjust to any discrepancy between the reward estimate and the actual block reward.
The second issue is the fact that the hub is a central point that can be DDoS’d and forced to prevent the P2Pool from functioning. Belcher’s proposal involves using multiple hubs, and sending each coinbase transaction from different blocks to different hubs. However, this requires miners to have channels opened from all hubs they are using which, at Belcher’s estimate of a hub needing 50 times the block reward (about 650 BTC) to provide liquidity for miners, becomes incredibly capital inefficient.
Braidpool: Another Iteration
Enter Braidpool (warning: link is a direct PDF download from GitHub). Braidpool is a proposal from Bob McElrath and Kulpreet Singh building on Belcher’s proposal using payment channels. There are two major changes introduced that improve on outstanding issues left with Belcher’s proposal.
The first is an alteration in how the hubs and miners communicate with each other. They propose having miners attach a Tor v3 address to each of the shares they broadcast to the pool. This way, the hub can operate without exposing any network endpoint that is susceptible to DoS attacks.
The hub operator can then connect to miners in order to open and update channels with them, alleviating the need for miners to use multiple hubs in order to avoid a single point of attack. This allows a Braidpool to operate with a single hub, making the entire system more robust and capital efficient.
Source: Braidpool white paper
The second change is the use of a directed acyclic graph (DAG) instead of a sharechain. The problem with the sharechain was that, with the thirty second share-time target, the difficulty required for shares increased as the pool grew in size, making it more difficult for smaller miners. Using a DAG like Ethereum, where it is not a zero sum game of a single share making it into the sharechain and others being orphaned, allows miners to dynamically set a difficulty for shares that can be adjusted based on the hash rate they have and how frequently they can find shares with it.
The DAG structure includes everyone who has participated in it between actual found Bitcoin blocks, distributing the rewards proportionally between everyone based on the work that they have provided to the DAG. This solves the scaling issue of variance for individual miners as the pools grow larger.
Aside from these two changes, the rest of the structure is just like Belcher’s proposal, the coinbase and channel scripts are the same.
Final Thoughts
Some readers might wonder why Betterhash was not touched on in this article. While decentralizing the selection of transactions for inclusion in a block, it does not fully decentralize all of the functions of the pool — most importantly, the custodial nature of pools handling funds. This leaves miners open to coercion through refusals to pay out funds if the miner is selecting transactions that the pool does not approve of. Therefore, I would not consider it a decentralized mining pool, although it does marginally improve the situation in a hostile but not quite fully adversarial environment.
This article has been centered around P2Pool and proposed iterations to improve upon its scaling limitations. For the sake of not writing an entire book, I have not touched on other existing or potential designs. As soon as I am able to get to it, I plan on writing a follow up piece going into other mechanisms to decentralize mining pools.
This is a guest post by Shinobi. Opinions expressed are entirely their own and do not necessarily reflect those of BTC Inc or Bitcoin Magazine.
In a previous article, I wrote about the nature of decentralization versus centralization in Bitcoin mining and how to conceptualize that in a mostly qualitative sense. The article broke down the entire mining stack, from pool coordination all the way down to energy production to give a sense of the relationship between different layers of the mining stack and the potential to maximize decentralization, making the point that the further down the stack you go toward energy production, the more difficult and capital intensive it becomes to bring a meaningful level of decentralization to that layer.
In this article, I intend to go deeper into the topic of mining pools and miner coordination to facilitate independently-owned mining operations cooperating in an effort to mine blocks to append to the blockchain.
The Creation Of Mining Pools
Mining has come a long way since the days when you could simply click a button and reliably mine blocks all by yourself on a laptop CPU. Back then, it was effectively an amateur hobbyist endeavor that required no real capital investment or expertise, but nowadays it is a multi-billion dollar professional market with massive capital investment required at scale. It is a whole different ball game.
One of the natural consequences of this shift in the nature of the mining industry was the creation of mining pools very early on. When mining was effectively leaving a laptop running in the corner, the variance and unpredictability of when you would find a block was not really that big of a deal — eventually, you would and the power cost of keeping a laptop running was not really of economic importance.
Once things shifted to GPUs and ASICs, there was a material investment cost up front and a much more significant electricity cost to keep them operating. That unpredictability in when you would find a block became a much bigger issue to miners attempting to make a return on their capital investments and operate profitably. This is where mining pools come into the picture.
They allow miners to cooperate on finding a valid block header working on the same block together, sending the coinbase reward to the mining pool, which then distributes it proportionally among all of the participating miners relative to how much work they have done in helping to find the block. This is proven by turning in “shares” to the mining pool; blocks that do not meet the network difficulty target but are high enough to prove that a miner is not lying and is actually running hardware and trying to find a valid block.
Mining Pool Centralization
Centralized mining pools come with big implications for individual miners. They are a point of centralization in the process of selecting (or, more importantly, excluding) transactions for inclusion in a block. This gives each mining pool operator total control over the transactions that they choose to process on the blockchain, with no ability for the actual owners of the mining hardware to exert a say on that except by leaving the pool if they disagree with the criteria that the operator chooses to set.
They also custody individual miners’ bitcoin until those miners choose to withdraw them from the pool, leaving the pool operator acting as a custodian and central point that could defraud miners using the pool, or be pressured by governments to seize individual miners funds or apply KYC requirements to them.
So, what solutions exist to address this problem?
P2Pool: The Original Decentralized Mining Pool
P2Pool is the original decentralized mining pool protocol. It is a peer-to-peer protocol in which miners coordinate among themselves to split up the mining rewards as they work together to find a valid block that meets the difficulty target. This coordination is accomplished using what the protocol design refers to as a “sharechain.”
Miners in the P2Pool take blocks that do not meet the difficulty target of the network, and effectively mine their own blockchain composed of all of the copies of the single block that the pool is working on. When they meet the smaller difficulty target where the block would be turned into a pool to prove they are mining in a centralized model, they broadcast that block to the rest of the miners. P2Pool’s “share difficulty” was targeted so that miners would find a share roughly once every thirty seconds.
I’m sure readers are wondering how the payout to individual miners works. The coinbase transaction is structured so that an output is created for each individual miner in the P2Pool, splitting up funds directly from the coinbase transaction. Miners in the P2Pool verify that all of the payouts to themselves and everyone participating in the pool is correct, and that each miner who has contributed a share to the sharechain is correctly paid out for their work in each new share added. If some participating miner is not correctly structuring payouts to everyone in their latest share, then all of the other miners in the P2Pool stop including them in their own payouts and effectively “evict” that miner from the pool for not behaving fairly.
This design led to a few scaling problems which is why it is not in use anymore. As the participation in a P2Pool grows, so does the aggregate difficulty target for shares in the sharechain, which target roughly once every thirty seconds. This means that for smaller miners it becomes more difficult to reach the sharechain difficulty within any thirty-second period. This means that for smaller miners the variance, or unpredictability, in their income increases as the aggregate hash rate in a P2Pool increases. This also leads to a larger number of stale blocks as more miners are finding competing shares for the sharechain at roughly the same time as more miners join the P2Pool, leading to “wasted work” from the point of view of the individual miners who only get compensated if their share is included in the sharechain.
The other main scalability issue is in the payouts going directly to individual miners from the coinbase transaction itself. Given that each miner is paid out in proportion to the shares they’ve mined that are included in the sharechain, each miner in the P2Pool requires a new output added in the coinbase transaction.
This has two consequences. First: Miners are being given small, low-value UTXOs in every block the P2Pool finds, which comes with the cost of condensing those outputs later on and/or bearing the cost of much larger transactions when they go to spend their coins because of the numerous individual UTXOs they wind up with, instead of a single one when withdrawing after a period from a conventional pool. Second: Each new coinbase output is taking up blockspace that could be consumed by other people’s transactions and earning the P2Pool more in fee revenue. It’s a double whammy loss for miners participating in the protocol.
These two issues contributed to the protocol slowly dying off and eventually falling into a state of disuse. By all indications from my best effort to track down accurate and recent statistics (many old block explorers tracking mining pool share have shut down over the years), it seems like the last block mined by P2Pool was on February 12, 2019.
P2Pool With Payment Channel Payouts
In 2017, a month after SegWit activation locked in, Chris Belcher made a proposal to improve the scalability of P2Pool with the use of one-way payment channels and a group of hubs handling payouts to the miners.
The core purpose of the proposal is to address the issue of the larger coinbase transactions losing miners’ money in two different ways. At a high level, the idea is simply to pay the entire coinbase transaction out to a hub with payment channels open to the individual miners, and guarantee that the ability to claim the funds from the coinbase transaction is atomically linked to the miners being compensated for their shares over the payment channels.
To accomplish the goal of atomicity between the coinbase transaction and payment channels for payouts, the coinbase transaction output script has to be customized. In Belcher’s proposal, it is structured as a multi-branch script with three spending conditions:
A two-of-two multisig. Key one: the hub (Hc). Key two: miner who found the block (Mc).A single key and a hashlock. Key: the hub (H). Hashlock: a random value generated by the hub (X).A single key and a timelock. Key: the miner who found the block (M). Timelock: a CSV relative timelock of six months.
Any one of these spending conditions can be used to unlock the coinbase transaction output. Now, let’s look at the payment channel script to the miners so we can see how the two things interact:
A two-of-two multisig. Key one: the hub (Hc1). Key two: the miner (Mc1).A two-of-two multisig and a hashlock. Key one: the hub (Hu1). Key two: the miner (Mu1). Hashlock: the random value generated by the hub used in the coinbase (X).
Now, let’s walk through how these two things interact with each other.
As miners are producing shares to add to the sharechain, the hub monitors the progress. For each share, the hub updates the state of the channel with miners who turn in a share to pay them proportionally to the amount of work they are doing. However, they only give them a signature for the second script path that requires the hashlock preimage to spend — this guarantees that by default, without the hub giving them a signature for the first path, they cannot claim those funds unless the hub spends the coinbase output by itself using the script path with the hashlock, which requires them to publish the preimage.
Now, eventually one of the miners in the P2Pool will find a valid block and publish it to the network. At this point, the hub can update all of the payment channels with the miners and provide the signature to the first script path in the channel, allowing each miner to close their channel and collect the rewards they have earned whenever they want without needing the hashlock preimage.
At this point, the miner who found the block signs the first script path in the coinbase, allowing the hub to claim the funds from the coinbase. That miner is given a slight bonus from the mining rewards to incentivize them to sign cooperatively. But remember: if the miner refuses to cooperate, the hub can simply spend by itself using the hashlock path and reveal the preimage, allowing all of the miners to collect their fair share of the reward.
This just has the downside of forcing all of the channels to close on chain, needing to be reopened to continue mining. The last option exists in case the hub operator chooses to stop processing payouts, or disappears. After six months, the miner who found the block can claim the funds entirely for themselves if the hub hasn’t responded to cooperate or has spent the coins with the hashlock path.
This leaves two specific issues in terms of threat model with Belcher’s proposed improvements. Deciding which transactions to include in a block leaves room for variance in how much of the total block reward is based on what individual miners choose to include in the block templates they are mining.
When introducing payment channels, this creates a margin for error, i.e., the actual mined block reward doesn’t equal what a mining hub commits to in the payment channels to miners. In the case that actual fee estimates are smaller than what the block reward was, the hub can simply update the payment channels using the cooperative spend path with the lesser amount, and as long as they do not claim the coinbase output with the hashlock path, the miners have no choice but to accept the lesser payout that matches what the mining reward actually was.
In the case that the mining reward was slightly higher than the estimate, it is still in the hub’s best interest to update the channels to the miners to reflect this, as miners that the hub treats dishonestly can simply leave at any time. The only edge case where it might make sense for the hub to defect and keep the extra reward would be if someone paid an abnormally large miner fee, but aside from that situation, it’s in the hub’s and miners’ interests to adjust to any discrepancy between the reward estimate and the actual block reward.
The second issue is the fact that the hub is a central point that can be DDoS’d and forced to prevent the P2Pool from functioning. Belcher’s proposal involves using multiple hubs, and sending each coinbase transaction from different blocks to different hubs. However, this requires miners to have channels opened from all hubs they are using which, at Belcher’s estimate of a hub needing 50 times the block reward (about 650 BTC) to provide liquidity for miners, becomes incredibly capital inefficient.
Braidpool: Another Iteration
Enter Braidpool (warning: link is a direct PDF download from GitHub). Braidpool is a proposal from Bob McElrath and Kulpreet Singh building on Belcher’s proposal using payment channels. There are two major changes introduced that improve on outstanding issues left with Belcher’s proposal.
The first is an alteration in how the hubs and miners communicate with each other. They propose having miners attach a Tor v3 address to each of the shares they broadcast to the pool. This way, the hub can operate without exposing any network endpoint that is susceptible to DoS attacks.
The hub operator can then connect to miners in order to open and update channels with them, alleviating the need for miners to use multiple hubs in order to avoid a single point of attack. This allows a Braidpool to operate with a single hub, making the entire system more robust and capital efficient.
The second change is the use of a directed acyclic graph (DAG) instead of a sharechain. The problem with the sharechain was that, with the thirty second share-time target, the difficulty required for shares increased as the pool grew in size, making it more difficult for smaller miners. Using a DAG like Ethereum, where it is not a zero sum game of a single share making it into the sharechain and others being orphaned, allows miners to dynamically set a difficulty for shares that can be adjusted based on the hash rate they have and how frequently they can find shares with it.
The DAG structure includes everyone who has participated in it between actual found Bitcoin blocks, distributing the rewards proportionally between everyone based on the work that they have provided to the DAG. This solves the scaling issue of variance for individual miners as the pools grow larger.
Aside from these two changes, the rest of the structure is just like Belcher’s proposal, the coinbase and channel scripts are the same.
Final Thoughts
Some readers might wonder why Betterhash was not touched on in this article. While decentralizing the selection of transactions for inclusion in a block, it does not fully decentralize all of the functions of the pool — most importantly, the custodial nature of pools handling funds. This leaves miners open to coercion through refusals to pay out funds if the miner is selecting transactions that the pool does not approve of. Therefore, I would not consider it a decentralized mining pool, although it does marginally improve the situation in a hostile but not quite fully adversarial environment.
This article has been centered around P2Pool and proposed iterations to improve upon its scaling limitations. For the sake of not writing an entire book, I have not touched on other existing or potential designs. As soon as I am able to get to it, I plan on writing a follow up piece going into other mechanisms to decentralize mining pools.
This is a guest post by Shinobi. Opinions expressed are entirely their own and do not necessarily reflect those of BTC Inc or Bitcoin Magazine.
Feedzy
