The work of Rene Pickhardt shows that despite being mainly used for small, fast Bitcoin transactions, Lightning can also deliver large payments.
The Lightning Network is seen as the main solution in the Bitcoin ecosystem for making small BTC transactions feasible, as it enables the smallest of payments to be sent for nearly no fees.
Indeed, Lightning has come a long way since first being designed in 2015 and having gone live in 2018. There are now over 88,000 public channels holding over 4,000 BTC, according to data from Txstats.com.
The Bitcoin overlay network enjoyed substantial growth over the past year after it was put front and center in global media with the adoption of bitcoin as legal tender by El Salvador. The move conferred much legitimacy to Lightning as it became clear to the world that bitcoin could indeed be used to purchase daily goods.
Growth of Bitcoin’s Lightning Network over the past one year. Image source: Txstats.com.
While there is still much work to be done for a global adoption of the Lightning Network to become realistically possible, the protocol has solidified its position as the main scaling solution for Bitcoin payments.
However, due to the Lightning Network’s design of interconnected channels, payments sent across the network are not guaranteed to succeed and hence decisions must be made as to in what way the wallet should prioritize payment flow. One example of this is prioritizing by routing fees, which is the most popular prioritization; it seeks to pick a path in the network that entails minimized fee expenditure for the sender.
A path in the Lightning Network refers to the route taken by a Lightning payment from sender to recipient across different channels. Routing allows the payment to be made between a given sender and receiver without requiring the two parties to open a channel among themselves. Nodes between them route the payment through their channels in exchange for a routing fee.
Even though choosing the lowest-cost route may often work fine for small payments, as the payment amount increases the less likely it becomes for the payment to succeed. In other words, the likelihood that the lower-cost route — and the Lightning Network at large — can successfully deliver a given payment diminishes as the size of that payment grows.
Much of this problem revolves around the issue of liquidity, which is necessary for the movement of funds from both the sender and receiver standpoint — and every channel in-between.
Naturally, the larger the payment, the greater the need for liquidity in Lightning channels for the payment to successfully traverse the network until its destination. For small payments this doesn’t matter as much: the likelihood of a channel’s bitcoin capacity being enough for a small payment to go through is high, but that drops as the amount of BTC being sent increases. Therefore, senders of small Bitcoin payments sent through Lightning usually don’t need to worry much about the issue of liquidity as when sending larger payments.
Nevertheless, even small payments are susceptible to failure of delivery on Lightning. In that sense, increasing reliability for both small and large payments seems like a good idea.
Thinking of this problem, Lightning Network researcher and educator Rene Pickhardt began researching ways to optimize for the probability of a payment to be successful. Now, his research is bearing fruit.
Pickhardt Payments: Increasing Reliability For Lightning Payments
As explained above, the core of the reliability problem in Lightning payments is centered around liquidity. As a result, most of Pickhardt’s work tries to answer the question: Where is the liquidity?
There isn’t a single answer to that question. Worse, often there isn’t a way to be certain about it either, even when considering a single channel.
Whereas it can be trivial to determine the bitcoin capacity of a public Lightning channel, it isn’t easy to determine how that capacity is divided among its two peers — posing a challenge to correctly assess outgoing and incoming liquidity in that channel. Hence, the ability of a sender to accurately determine the needed liquidity for a successful routing of their payment is impaired.
Due to this uncertainty, Pickhardt found that the best answer is based on probability.
Pickhardt Payments quantifies the uncertainty around liquidity to a probability, mapping out all the possible paths for a given payment to take and selecting the one with the highest likelihood of succeeding. In other words, Pickhardt Payments seeks to select the path with the lowest level of uncertainty.
The probabilistic model used by Pickhardt estimates the amount of liquidity that there is in a channel. That model is run on the Lightning Network’s public payment channels to estimate how likely it is that a certain amount of liquidity exists in a given channel. The payment is then sent through the path that is more likely to contain the highest amount of liquidity — seeking to optimize for reliability. If the payment fails, the model learns from its mistake by updating its belief about the network.
Pickhardt’s approach applies decades-old computer science research to the burgeoning Lightning protocol. More specifically, the Lightning researcher draws from late father of information theory Claude Shannon’s work on communication channels, entropy and uncertainty to quantify communication uncertainty with probabilistic distributions.
Pickhardt Payments For Large Transactions
On Lightning, large payments are made more feasible by being broken down into smaller payments. These then get routed across the network to the destination and if they all are successful, then the original payment is successful.
Given that all divisions need to succeed in order for the original payment to succeed, the higher the number of payments an original payment is divided into, the lower the likelihood of success. This is because the probability of the original payment to succeed is found by multiplying the probabilities of each “child” payment succeeding. This creates a sort of a paradox because, in theory, the smaller the payment the higher the chance of success. Hence, an optimization challenge in finding the correct split emerges.
Pickhardt Payments seeks to increase the probability of the original payment succeeding by striving for this optimization, which is made by reducing the uncertainty costs. To achieve that, it leverages operations research (OR).
More specifically, Pickhardt Payments uses Minimum-cost flows, a well-studied tool from the field of OR, to model and conduct the optimal multi-part Lightning payment — improving the decision-making process of splitting and delivering payments on Lightning. In short, the premise is that there is an inherent cost attached to the uncertainty about the liquidity information for every channel.
Not to be confused with fee costs, uncertainty costs can be thought of as the opportunity cost in trying a given path. In theory, the higher the uncertainty cost, the higher the opportunity cost as it’s less likely that the payment will succeed. Quantifying that uncertainty allows Pickhardt Payments to minimize the opportunity cost and choose the path that maximizes the chance of success.
Ultimately, these optimizations allow Pickhardt Payments to send large Lightning payments that were previously impossible on the network.
Pickhardt’s design differs from current practices on Lightning because, just like with single payments, prioritization for the splitted payments is usually done based on fees. On the other hand, Pickhardt Payments prioritizes lowering the uncertainty costs as it seeks to increase the likelihood of payment delivery, meaning that fees can often get relatively expensive if compared to those commonly paid on Lightning.
However, Pickhardt told Bitcoin Magazine that his algorithm could also offer the choice to optimize for fees, though he argues doing so is at present computationally heavy given the existence of Lightning’s base fee. In that case, the challenge would be to find an optimal balance between the two options — which he calls “features” — as users dabble in what they ultimately wish to prioritize for.
How To Use Pickhardt Payments Today
Usage of Pickhardt Payments today can be done via LND manageJ by Carsten Otto. Alternatively, Pickhardt is developing a library which currently allows for simulations and production tests. Pickhardt told Bitcoin Magazine that he will eventually ship the library with bindings for implementations, but that is not currently as high a priority as technical correctness, security and runtime. Even though there isn’t yet a simple way a regular user can take advantage of Pickhardt Payments, availability should increase as developers keep working on the algorithm and library to boast its robustness and security.
Thanks to Rene Pickhardt for information and feedback.
For more details about Pickhardt Payments, navigate to the related research paper and mailing list post. More resources are also available on Pickhardt’s website.
The Lightning Network is seen as the main solution in the Bitcoin ecosystem for making small BTC transactions feasible, as it enables the smallest of payments to be sent for nearly no fees.
Indeed, Lightning has come a long way since first being designed in 2015 and having gone live in 2018. There are now over 88,000 public channels holding over 4,000 BTC, according to data from Txstats.com.
The Bitcoin overlay network enjoyed substantial growth over the past year after it was put front and center in global media with the adoption of bitcoin as legal tender by El Salvador. The move conferred much legitimacy to Lightning as it became clear to the world that bitcoin could indeed be used to purchase daily goods.
While there is still much work to be done for a global adoption of the Lightning Network to become realistically possible, the protocol has solidified its position as the main scaling solution for Bitcoin payments.
However, due to the Lightning Network’s design of interconnected channels, payments sent across the network are not guaranteed to succeed and hence decisions must be made as to in what way the wallet should prioritize payment flow. One example of this is prioritizing by routing fees, which is the most popular prioritization; it seeks to pick a path in the network that entails minimized fee expenditure for the sender.
A path in the Lightning Network refers to the route taken by a Lightning payment from sender to recipient across different channels. Routing allows the payment to be made between a given sender and receiver without requiring the two parties to open a channel among themselves. Nodes between them route the payment through their channels in exchange for a routing fee.
Even though choosing the lowest-cost route may often work fine for small payments, as the payment amount increases the less likely it becomes for the payment to succeed. In other words, the likelihood that the lower-cost route — and the Lightning Network at large — can successfully deliver a given payment diminishes as the size of that payment grows.
Much of this problem revolves around the issue of liquidity, which is necessary for the movement of funds from both the sender and receiver standpoint — and every channel in-between.
Naturally, the larger the payment, the greater the need for liquidity in Lightning channels for the payment to successfully traverse the network until its destination. For small payments this doesn’t matter as much: the likelihood of a channel’s bitcoin capacity being enough for a small payment to go through is high, but that drops as the amount of BTC being sent increases. Therefore, senders of small Bitcoin payments sent through Lightning usually don’t need to worry much about the issue of liquidity as when sending larger payments.
Nevertheless, even small payments are susceptible to failure of delivery on Lightning. In that sense, increasing reliability for both small and large payments seems like a good idea.
Thinking of this problem, Lightning Network researcher and educator Rene Pickhardt began researching ways to optimize for the probability of a payment to be successful. Now, his research is bearing fruit.
Pickhardt Payments: Increasing Reliability For Lightning Payments
As explained above, the core of the reliability problem in Lightning payments is centered around liquidity. As a result, most of Pickhardt’s work tries to answer the question: Where is the liquidity?
There isn’t a single answer to that question. Worse, often there isn’t a way to be certain about it either, even when considering a single channel.
Whereas it can be trivial to determine the bitcoin capacity of a public Lightning channel, it isn’t easy to determine how that capacity is divided among its two peers — posing a challenge to correctly assess outgoing and incoming liquidity in that channel. Hence, the ability of a sender to accurately determine the needed liquidity for a successful routing of their payment is impaired.
Due to this uncertainty, Pickhardt found that the best answer is based on probability.
Pickhardt Payments quantifies the uncertainty around liquidity to a probability, mapping out all the possible paths for a given payment to take and selecting the one with the highest likelihood of succeeding. In other words, Pickhardt Payments seeks to select the path with the lowest level of uncertainty.
The probabilistic model used by Pickhardt estimates the amount of liquidity that there is in a channel. That model is run on the Lightning Network’s public payment channels to estimate how likely it is that a certain amount of liquidity exists in a given channel. The payment is then sent through the path that is more likely to contain the highest amount of liquidity — seeking to optimize for reliability. If the payment fails, the model learns from its mistake by updating its belief about the network.
Pickhardt’s approach applies decades-old computer science research to the burgeoning Lightning protocol. More specifically, the Lightning researcher draws from late father of information theory Claude Shannon’s work on communication channels, entropy and uncertainty to quantify communication uncertainty with probabilistic distributions.
Pickhardt Payments For Large Transactions
On Lightning, large payments are made more feasible by being broken down into smaller payments. These then get routed across the network to the destination and if they all are successful, then the original payment is successful.
Given that all divisions need to succeed in order for the original payment to succeed, the higher the number of payments an original payment is divided into, the lower the likelihood of success. This is because the probability of the original payment to succeed is found by multiplying the probabilities of each “child” payment succeeding. This creates a sort of a paradox because, in theory, the smaller the payment the higher the chance of success. Hence, an optimization challenge in finding the correct split emerges.
Pickhardt Payments seeks to increase the probability of the original payment succeeding by striving for this optimization, which is made by reducing the uncertainty costs. To achieve that, it leverages operations research (OR).
More specifically, Pickhardt Payments uses Minimum-cost flows, a well-studied tool from the field of OR, to model and conduct the optimal multi-part Lightning payment — improving the decision-making process of splitting and delivering payments on Lightning. In short, the premise is that there is an inherent cost attached to the uncertainty about the liquidity information for every channel.
Not to be confused with fee costs, uncertainty costs can be thought of as the opportunity cost in trying a given path. In theory, the higher the uncertainty cost, the higher the opportunity cost as it’s less likely that the payment will succeed. Quantifying that uncertainty allows Pickhardt Payments to minimize the opportunity cost and choose the path that maximizes the chance of success.
Ultimately, these optimizations allow Pickhardt Payments to send large Lightning payments that were previously impossible on the network.
Pickhardt’s design differs from current practices on Lightning because, just like with single payments, prioritization for the splitted payments is usually done based on fees. On the other hand, Pickhardt Payments prioritizes lowering the uncertainty costs as it seeks to increase the likelihood of payment delivery, meaning that fees can often get relatively expensive if compared to those commonly paid on Lightning.
However, Pickhardt told Bitcoin Magazine that his algorithm could also offer the choice to optimize for fees, though he argues doing so is at present computationally heavy given the existence of Lightning’s base fee. In that case, the challenge would be to find an optimal balance between the two options — which he calls “features” — as users dabble in what they ultimately wish to prioritize for.
How To Use Pickhardt Payments Today
Usage of Pickhardt Payments today can be done via LND manageJ by Carsten Otto. Alternatively, Pickhardt is developing a library which currently allows for simulations and production tests. Pickhardt told Bitcoin Magazine that he will eventually ship the library with bindings for implementations, but that is not currently as high a priority as technical correctness, security and runtime. Even though there isn’t yet a simple way a regular user can take advantage of Pickhardt Payments, availability should increase as developers keep working on the algorithm and library to boast its robustness and security.
Thanks to Rene Pickhardt for information and feedback.
For more details about Pickhardt Payments, navigate to the related research paper and mailing list post. More resources are also available on Pickhardt’s website.
Feedzy
