The Lightning Network, often abbreviated as "LN", is changing the way people exchange value online, and it's one of the most exciting advancements to happen in Bitcoin's history.
Today, in 2021, the Lightning Network is still in its infancy. The Lightning Network is a protocol for using Bitcoin in a smart and non-obvious way. It is a second layer technology on top of Bitcoin.
The Lightning Network (often abbreviated as LN), is changing the way people exchange value online, and it's one of the most exciting advancements to happen in Bitcoin's history.
Today, in 2021, the Lightning Network is still in its infancy. The Lightning Network is a protocol for using Bitcoin in a smart and nonobvious way. It is a second layer technology on top of Bitcoin.
The concept of the Lightning Network was proposed in 2015, and the first implementation was launched in 2018. As of 2021, we're only beginning to see the opportunities the Lightning Network provides to Bitcoin including improved privacy, speed, and scale.
The concept of the Lightning Network was proposed in 2015, and the first implementation was launched in 2018. As of 2021, we're only beginning to see the opportunities the Lightning Network provides to Bitcoin, including improved privacy, speed, and scale.
With core knowledge of the Lightning Network, you can help shape the future of the network while also building opportunities for yourself.
We assume you already have some basic knowledge about Bitcoin, but if not, don't worry–we will explain the most important Bitcoin concepts, those you must know to understand the Lightning Network, in XREF HERE. If you want to learn more about Bitcoin, you can read _Mastering Bitcoin_, available for free online.
We assume you already have some basic knowledge about Bitcoin, but if not, don't worry—we will explain the most important Bitcoin concepts, those you must know to understand the Lightning Network, in <<bitcoin_fundamentals_review>>. If you want to learn more about Bitcoin, you can read _Mastering Bitcoin_, 2nd edition, by Andreas M. Antonopoulos (O'Reilly), available https://bitcoinbook.info/[for free online].
While the bulk of this book is written for programmers, the first few chapters are written to be approachable by anyone regardless of technical experience. In this chapter, we'll start with some terminology, then move to look at trust and its application in these systems, and finally we'll discuss the history and future of the Lightning Network. Let's get started.
@ -25,23 +25,23 @@ Digital Signature:: A digital signature is a mathematical scheme for verifying t
Hash Function:: A cryptographic hash function is a mathematical algorithm that maps data of arbitrary size to a bit string of a fixed size (a hash) and is designed to be a one-way function, that is, a function which is infeasible to invert.
Node:: A computer that participates in a network. A Lightning node is a computer that participates in the Lightning Network. A Bitcoin node is a computer that participates in the Bitcoin Network. Typically a Lightning Network user will run a Lightning node _and_ a Bitcoin node.
Node:: A computer that participates in a network. A Lightning node is a computer that participates in the Lightning Network. A Bitcoin node is a computer that participates in the Bitcoin network. Typically an LN user will run a Lightning node _and_ a Bitcoin node.
On-Chain vs. Off-Chain:: A payment is "on-chain" if it is recorded as a transaction on the Bitcoin (or other underlying) blockchain. Payments sent via payment channels between Lightning nodes, and which are not visible in the underlying blockchain, are called "off-chain" payments. Usually in the Lightning Network, the only on-chain transactions are those used to open and close a Lightning payment channel. A third type of channel modifying transaction exists, called splicing, which can be used to add/remove the amount of funds committed in a channel.
On-Chain versus Off-Chain:: A payment is _on-chain_ if it is recorded as a transaction on the Bitcoin (or other underlying) blockchain. Payments sent via payment channels between Lightning nodes, and which are not visible in the underlying blockchain, are called _off-chain_ payments. Usually in the Lightning Network, the only on-chain transactions are those used to open and close a Lightning payment channel. A third type of channel modifying transaction exists, called splicing, which can be used to add/remove the amount of funds committed in a channel.
Payment:: When value is exchanged on the Lightning Network, we call this a "payment" as compared to a "transaction" on the Bitcoin blockchain.
Payment Channel:: A _financial relationship_ between two nodes on the Lightning Network, typically implemented by multi-signature Bitcoin transactions that share control over bitcoin between the two Lightning nodes.
Payment Channel:: A _financial relationship_ between two nodes on the Lightning Network, typically implemented by multisignature Bitcoin transactions that share control over bitcoin between the two Lightning nodes.
Routing vs Sending:: Unlike Bitcoin where transactions are "sent" by broadcasting them to everyone, Lightning is a routed network where payments are "routed" across one or more payment channels following a _path_ from sender to recipient.
Transaction:: A data structure that records the transfer of control over some funds (e.g. some bitcoin). The Lightning Network relies on Bitcoin transactions (or those of another blockchain) to track control of funds.
Transaction:: A data structure that records the transfer of control over some funds (e.g., some bitcoin). The Lightning Network relies on Bitcoin transactions (or those of another blockchain) to track control of funds.
More detailed definitions of these and many other terms can be found in the <<glossary>>. Throughout this book, we will explain what these concepts mean and how these technologies actually work.
[TIP]
====
Throughout this book, you will see "Bitcoin" with the first letter capitalized, which refers to the _Bitcoin System_ and is a proper noun. You will also see "bitcoin", with a lower-case _b_, which refers to the currency unit. Each bitcoin is further subdivided into 100 million units each called a "satoshi" (singular) or "satoshis" (plural)
Throughout this book, you will see "Bitcoin" with the first letter capitalized, which refers to the _Bitcoin System_ and is a proper noun. You will also see "bitcoin," with a lowercase _b_, which refers to the currency unit. Each bitcoin is further subdivided into 100 million units each called a "satoshi" (singular) or "satoshis" (plural)
====
Now that you're familiar with these basic terms, let's move to a concept you are already comfortable with: trust.
@ -49,15 +49,15 @@ Now that you're familiar with these basic terms, let's move to a concept you are
=== Trust in Decentralized Networks
You will often hear people calling Bitcoin and the Lightning Network "trustless". At first glance this is confusing. After all, isn't trust a good thing? Banks even use it in their names! Isn't a "trustless" system, a system devoid of trust, a bad thing?
You will often hear people calling Bitcoin and the Lightning Network "trustless." At first glance this is confusing. After all, isn't trust a good thing? Banks even use it in their names! Isn't a "trustless" system, a system devoid of trust, a bad thing?
The use of the word "trustless" is intended to convey the ability to operate without _needing_ trust in the other participants in the system. In a decentralized system like Bitcoin, you can always choose to transact with someone you trust. However, the system ensures you can't be cheated even if you can't trust the other party in a transaction. Trust is a nice-to-have instead of a must-have property of the system.
Contrast that to traditional systems like banking where you _must_ place your trust in a third party, since it controls your money. If the bank violates your trust, you may be able to find some recourse from a regulator or court, but at an enormous cost of time, money, and effort.
Contrast that to traditional systems like banking where you must place your trust in a third party, since it controls your money. If the bank violates your trust, you may be able to find some recourse from a regulator or court, but at an enormous cost of time, money, and effort.
Trustless does not mean devoid of trust. It means that trust is not a necessary prerequisite to all transactions and that you can transact even with people you don't trust because the system prevents cheating.
Before we get into how the Lightning Network works, it's important to understand one basic concept that underlies Bitcoin, the Lightning Network and many other such systems: something we call a _fairness protocol_. A fairness protocol is a way to _achieve fair outcomes between participants, who do not need to trust each other, without the need for a central authority_ and it is the backbone of decentralized systems like Bitcoin.
Before we get into how the Lightning Network works, it's important to understand one basic concept that underlies Bitcoin, the Lightning Network, and many other such systems: something we call a _fairness protocol_. A fairness protocol is a way to achieve fair outcomes between participants, who do not need to trust each other, without the need for a central authority, and it is the backbone of decentralized systems like Bitcoin.
=== Fairness Without Central Authority
@ -66,13 +66,13 @@ When people have competing interests, how can they establish enough trust to eng
In broad terms, there are a handful of ways to ensure fair outcomes in interactions between individuals who may have competing interests:
* Require trust - you only interact with people whom you already trust, due to prior interactions, reputation, or familial relationships. This works well enough at small scale, especially within families and small groups, that it is the most common basis for cooperative behavior. Unfortunately, it doesn't scale and it suffers from tribalist (in-group) bias.
Require trust:: You only interact with people whom you already trust, due to prior interactions, reputation, or familial relationships. This works well enough at small scale, especially within families and small groups, that it is the most common basis for cooperative behavior. Unfortunately, it doesn't scale and it suffers from tribalist (in-group) bias.
* Rule of law - establish rules for interactions that are enforced by an institution. This scales better, but it cannot scale globally due to differences in customs and traditions, as well as the inability to scale the institutions of enforcement. One nasty side-effect of this solution is that the institutions become more and more powerful as they get bigger and that may lead to corruption.
Rule of law:: Establish rules for interactions that are enforced by an institution. This scales better, but it cannot scale globally due to differences in customs and traditions, as well as the inability to scale the institutions of enforcement. One nasty side-effect of this solution is that the institutions become more and more powerful as they get bigger and that may lead to corruption.
* Trusted third parties - put an intermediary in every interaction to enforce fairness. Combined with the "rule of law" to provide oversight of intermediaries, this scales better, but suffers from the same imbalance of power: the intermediaries get very powerful and may attract corruption. Concentration of power leads to systemic risk and systemic failure ("Too big to fail").
Trusted third parties:: Put an intermediary in every interaction to enforce fairness. Combined with the "rule of law" to provide oversight of intermediaries, this scales better, but suffers from the same imbalance of power: the intermediaries get very powerful and may attract corruption. Concentration of power leads to systemic risk and systemic failure ("Too big to fail").
* Game theoretical fairness protocols - this last category emerges from the combination of the internet and cryptography and is the subject of this section. Let's see how it works and what its advantages and disadvantages are.
Game theoretical fairness protocols:: This last category emerges from the combination of the internet and cryptography and is the subject of this section. Let's see how it works and what its advantages and disadvantages are.
==== Trusted Protocols Without Intermediaries
@ -80,80 +80,80 @@ Cryptographic systems like Bitcoin and the Lightning Network are systems that al
The big distinction between a cryptographic system like this and a traditional financial system is that in traditional finance you have a _trusted third party_, for example a bank, to ensure that outcomes are fair. A significant problem with such systems is that they give too much power to the third party, and they are also vulnerable to a _single point of failure_. If the trusted third party itself violates trust or attempts to cheat, the basis of trust breaks.
As you study cryptographic systems, you will notice a certain pattern: instead of relying on a trusted third party, these systems attempt to prevent unfair outcomes by using a system of incentives and disincentives. In cryptographic systems you place trust in the _protocol_, which is effectively a system with a set of rules that, if properly designed, will correctly apply the desired incentives and disincentives. The advantage of this approach is two-fold: Not only do you avoid trusting a third party, you also reduce the need to enforce fair outcomes. So long as the participants follow the agreed protocol and stay within the system, the incentive mechanism in that protocol achieves fair outcomes without enforcement.
As you study cryptographic systems, you will notice a certain pattern: instead of relying on a trusted third party, these systems attempt to prevent unfair outcomes by using a system of incentives and disincentives. In cryptographic systems you place trust in the _protocol_, which is effectively a system with a set of rules that, if properly designed, will correctly apply the desired incentives and disincentives. The advantage of this approach is twofold: not only do you avoid trusting a third party, you also reduce the need to enforce fair outcomes. So long as the participants follow the agreed protocol and stay within the system, the incentive mechanism in that protocol achieves fair outcomes without enforcement.
The use of incentives and disincentives to achieve fair outcomes is one aspect of a branch of mathematics called _game theory_, which studies "models of strategic interaction among rational decision makers"footnote:[Wikipedia "Game Theory": https://en.wikipedia.org/wiki/Game_theory]. Cryptographic systems that control financial interactions between participants, such as Bitcoin and the Lightning Network, rely heavily on game theory to prevent participants from cheating and allow participants who don't trust each other to achieve fair outcomes.
The use of incentives and disincentives to achieve fair outcomes is one aspect of a branch of mathematics called _game theory_, which studies "models of strategic interaction among rational decision makers."footnote:[The Wikipedia https://en.wikipedia.org/wiki/Game_theory[entry on game theory] provides more information.] Cryptographic systems that control financial interactions between participants, such as Bitcoin and the Lightning Network, rely heavily on game theory to prevent participants from cheating and allow participants who don't trust each other to achieve fair outcomes.
While game theory and its use in cryptographic systems may appear confounding and unfamiliar at first, chances are you're already familiar with these systems in your everyday life; you just don't recognize them yet. Below we'll use a simple example from childhood to help us identify the basic pattern. Once you understand the basic pattern, you will see it everywhere in the blockchain space and you will come to recognize it quickly and intuitively.
While game theory and its use in cryptographic systems may appear confounding and unfamiliar at first, chances are you're already familiar with these systems in your everyday life; you just don't recognize them yet. In the following we'll use a simple example from childhood to help us identify the basic pattern. Once you understand the basic pattern, you will see it everywhere in the blockchain space and you will come to recognize it quickly and intuitively.
In this book, we call this pattern a _Fairness Protocol_ defined as a process that uses a system of incentives and/or disincentives to ensure fair outcomes for participants who don't trust each other. Enforcement of a fairness protocol is only necessary to ensure that the participants can't escape the incentives or disincentives.
In this book, we call this pattern a _fairness protocol_, defined as a process that uses a system of incentives and/or disincentives to ensure fair outcomes for participants who don't trust each other. Enforcement of a fairness protocol is only necessary to ensure that the participants can't escape the incentives or disincentives.
==== A Fairness Protocol in Action
Let's look at an example of a fairness protocol, which you may already be familiar with.
Let's look at an example of a fairness protocol that you may already be familiar with.
Imagine a family lunch, with a parent and two children. The children are fussy eaters and the only thing they will agree to eat is fried potatoes. The parent has prepared a bowl of fried potatoes ("french fries" or "chips" depending on which English dialect you use). The two siblings must share the plate of chips. The parent must ensure a fair distribution of chips to each child, otherwise the parent will have to hear constant complaining (maybe all day) and there's always a possibility of an unfair situation escalating to violence. What is a parent to do?
Imagine a family lunch, with a parent and two children. The children are fussy eaters and the only thing they will agree to eat is fried potatoes. The parent has prepared a bowl of fried potatoes ("french fries" or "chips" depending on which English dialect you use). The two siblings must share the plate of chips. The parent must ensure a fair distribution of chips to each child; otherwise, the parent will have to hear constant complaining (maybe all day), and there's always a possibility of an unfair situation escalating to violence. What is a parent to do?
There are a few different ways that fairness can be achieved in this strategic interaction between two siblings that do not trust each other and have competing interests. The naive but commonly used method is for the parent to use their authority as a trusted third party: they split the bowl of chips into two servings. This is similar to traditional finance, where a bank, accountant or lawyer acts as a trusted third party to prevent any cheating between two parties who want to transact.
There are a few different ways that fairness can be achieved in this strategic interaction between two siblings that do not trust each other and have competing interests. The naive but commonly used method is for the parent to use their authority as a trusted third party: they split the bowl of chips into two servings. This is similar to traditional finance, where a bank, accountant, or lawyer acts as a trusted third party to prevent any cheating between two parties who want to transact.
The problem with this scenario is that it vests a lot of power and responsibility in the hands of the trusted third party. In this example, the parent is fully responsible for the equal allocation of chips, the parties merely wait, watch, and complain. The children accuse the parent of playing favorites and not allocating the chips fairly. The siblings fight over the chips, yelling "that chip is bigger!", and dragging the parent into their fight. It sounds awful, doesn't it? Should the parent yell louder? Take all of the chips away? Threaten to never make chips again and let those ungrateful children go hungry?
The problem with this scenario is that it vests a lot of power and responsibility in the hands of the trusted third party. In this example, the parent is fully responsible for the equal allocation of chips, and the parties merely wait, watch, and complain. The children accuse the parent of playing favorites and not allocating the chips fairly. The siblings fight over the chips, yelling "that chip is bigger!", and dragging the parent into their fight. It sounds awful, doesn't it? Should the parent yell louder? Take all of the chips away? Threaten to never make chips again and let those ungrateful children go hungry?
A much better solution exists: the siblings are taught to play a game called "split and choose". At each lunch one sibling splits the bowl of chips into two servings and the *other* sibling gets to choose which serving they want. Almost immediately, the siblings figure out the dynamic of this game. If the one splitting makes a mistake or tries to cheat, the other sibling can "punish" them by choosing the bigger bowl. It is in the best interest of both siblings, but especially the one splitting the bowl, to play fair. Only the cheater loses in this scenario. The parent doesn't even have to use their authority or enforce fairness. All the parent has to do is _enforce the protocol_; as long as the siblings cannot escape their assigned roles of "splitter" and "chooser", the protocol itself ensures a fair outcome without the need for any intervention. The parent can't play favorites or distort the outcome.
A much better solution exists: the siblings are taught to play a game called "split and choose." At each lunch one sibling splits the bowl of chips into two servings and the _other_ sibling gets to choose which serving they want. Almost immediately, the siblings figure out the dynamic of this game. If the one splitting makes a mistake or tries to cheat, the other sibling can "punish" them by choosing the bigger bowl. It is in the best interest of both siblings, but especially the one splitting the bowl, to play fair. Only the cheater loses in this scenario. The parent doesn't even have to use their authority or enforce fairness. All the parent has to do is _enforce the protocol_; as long as the siblings cannot escape their assigned roles of "splitter" and "chooser," the protocol itself ensures a fair outcome without the need for any intervention. The parent can't play favorites or distort the outcome.
[WARNING]
====
While the infamous chip battles of the 1980's neatly illustrate the point, any similarity between the scenario above and any of the authors' actual childhood experiences with their cousins is entirely coincidental...Or is it?
While the infamous chip battles of the 1980s neatly illustrate the point, any similarity between the preceding scenario and any of the authors' actual childhood experiences with their cousins is entirely coincidental...Or is it?
====
==== Security Primitives as Building Blocks
In order for a fairness protocol like this to work, there need to be certain guarantees, or _security primitives_, that can be combined to ensure enforcement. The first security primitive is _strict time ordering/sequencing_: the "splitting" action must happen before the "choosing" action. It's not immediately obvious, but unless you can guarantee that action A happens before action B, then the protocol falls apart. The second security primitive is _commitment with non-repudiation_. Each sibling must commit to their choice of role: either splitter or chooser. Also, once the splitting has been completed, the splitter is committed to the split they created - they cannot repudiate that choice and go try again.
In order for a fairness protocol like this to work, there need to be certain guarantees, or _security primitives_, that can be combined to ensure enforcement. The first security primitive is _strict time ordering/sequencing_: the "splitting" action must happen before the "choosing" action. It's not immediately obvious, but unless you can guarantee that action A happens before action B, then the protocol falls apart. The second security primitive is _commitment with nonrepudiation_. Each sibling must commit to their choice of role: either splitter or chooser. Also, once the splitting has been completed, the splitter is committed to the split they created—they cannot repudiate that choice and go try again.
Cryptographic systems offer a number of security primitives that can be combined in different ways to construct a fairness protocol. In addition to sequencing and commitment, we can also use many other tools:
- Hash functions to fingerprint data, as a form of commitment, or as the basis for a digital signature.
- Digital signatures for authentication, non-repudiation, and proof of ownership of a secret.
- Encryption/decryption to restrict access to information to authorized participants only.
- Hash functions to fingerprint data, as a form of commitment, or as the basis for a digital signature
- Digital signatures for authentication, nonrepudiation, and proof of ownership of a secret
- Encryption/decryption to restrict access to information to authorized participants only
This is only a small list of a whole "menagerie" of security and cryptographic primitives that are in use. More basic primitives and combinations are invented all the time.
In our real-life example, we saw one form of fairness protocol called "split and choose". This is just one of a myriad different fairness protocols that can be built by combining the building blocks of security primitives in different ways. But the basic pattern is always the same: two or more participants interact without trusting each other, by engaging in a series of steps that are part of an agreed protocol. The protocol's steps arrange incentives and disincentives to ensure that if the participants are rational, cheating is counter-productive and fairness is the automatic outcome. Enforcement is not necessary to get fair outcomes - it is only necessary to keep the participants from breaking out of the agreed protocol.
In our real-life example, we saw one form of fairness protocol called "split and choose." This is just one of a myriad different fairness protocols that can be built by combining the building blocks of security primitives in different ways. But the basic pattern is always the same: two or more participants interact without trusting each other by engaging in a series of steps that are part of an agreed protocol. The protocol's steps arrange incentives and disincentives to ensure that if the participants are rational, cheating is counterproductive and fairness is the automatic outcome. Enforcement is not necessary to get fair outcomes—it is only necessary to keep the participants from breaking out of the agreed protocol.
Now that you understand this basic pattern, you will start seeing it everywhere in Bitcoin, the Lightning Network and many other systems. Let's look at some specific examples next.
Now that you understand this basic pattern, you will start seeing it everywhere in Bitcoin, the Lightning Network, and many other systems. Let's look at some specific examples next.
==== Example of the Fairness Protocol
The most prominent example of a "fairness protocol" is Bitcoin's consensus algorithm, _Proof of Work_ (PoW). In Bitcoin, miners compete to verify transactions and aggregate them in blocks. To ensure that the miners do not cheat, without entrusting them with authority, Bitcoin uses a system of incentives and disincentives. Miners have to use electricity and dedicate hardware doing "work" that is embedded as a "proof" inside every block. This is achieved because of a property of hash functions where the output value is randomly distributed across the entire range of possible outputs. If miners succeed in producing a valid block fast enough, they are rewarded by earning the block reward for that block. Forcing miners to use a lot of electricity before the network considers their block means that they have an incentive to correctly validate the transactions in the block. If they cheat or make any kind of mistake, their block is rejected and the electricity they used to "prove" it is wasted. No one needs to force miners to produce valid blocks; the reward and punishment incentivize them to do so. All the protocol needs to do is ensure that only valid blocks with proof of work are accepted.
The most prominent example of a fairness protocol is Bitcoin's consensus algorithm, Proof of Work (PoW). In Bitcoin, miners compete to verify transactions and aggregate them in blocks. To ensure that the miners do not cheat, without entrusting them with authority, Bitcoin uses a system of incentives and disincentives. Miners have to use electricity and dedicate hardware doing "work" that is embedded as a "proof" inside every block. This is achieved because of a property of hash functions where the output value is randomly distributed across the entire range of possible outputs. If miners succeed in producing a valid block fast enough, they are rewarded by earning the block reward for that block. Forcing miners to use a lot of electricity before the network considers their block means that they have an incentive to correctly validate the transactions in the block. If they cheat or make any kind of mistake, their block is rejected and the electricity they used to "prove" it is wasted. No one needs to force miners to produce valid blocks; the reward and punishment incentivize them to do so. All the protocol needs to do is ensure that only valid blocks with proof of work are accepted.
The "fairness protocol" pattern can also be found in many different aspects of the Lightning Network:
The fairness protocol pattern can also be found in many different aspects of the Lightning Network:
* Those who fund channels make sure that they have a refund transaction signed before they publish the funding transaction.
* Whenever a channel is moved to a new state, the old state is "revoked" by ensuring that if anyone tries to broadcast it, they lose the entire balance and get punished.
* Those who forward payments know that if they commit funds forward, they can either get a refund or they get paid by the node preceding them.
* Those who forward payments know that if they commit funds forward, they can either get a refund or get paid by the node preceding them.
Again and again, we see this pattern. Fair outcomes are not enforced by any authority. They emerge as the natural consequence of a protocol that rewards fairness and punishes cheating. A fairness protocol that harnesses self-interest by directing it towards fair outcomes.
Again and again, we see this pattern. Fair outcomes are not enforced by any authority. They emerge as the natural consequence of a protocol that rewards fairness and punishes cheating, a fairness protocol that harnesses self-interest by directing it toward fair outcomes.
Bitcoin and the Lightning Network are both implementations of fairness protocols. So why do we need the Lightning Network? Isn't Bitcoin enough?
=== Motivation for the Lightning Network
Bitcoin is a system that records transactions on a globally replicated public ledger. Every transaction is seen, validated and stored by every participating computer. As you can imagine, this generates a lot of data and is difficult to scale.
Bitcoin is a system that records transactions on a globally replicated public ledger. Every transaction is seen, validated, and stored by every participating computer. As you can imagine, this generates a lot of data and is difficult to scale.
As Bitcoin and the demand for transactions grew, the number of transactions in each block increased until it eventually reached the block size limit.
Once blocks are "full", excess transactions are left to wait in a queue. Many users will increase the fees they're willing to pay in order to buy space for their transactions in the next block.
Once blocks are "full," excess transactions are left to wait in a queue. Many users will increase the fees they're willing to pay to buy space for their transactions in the next block.
If demand continues to outpace the capacity of the network, an increasing number of users' transactions are left waiting unconfirmed. Competition for fees also increases the cost of each transaction, making many smaller-value transactions (e.g. micro-transactions) completely uneconomical during periods of particularly high demand.
If demand continues to outpace the capacity of the network, an increasing number of users' transactions are left waiting unconfirmed. Competition for fees also increases the cost of each transaction, making many smaller-value transactions (e.g., microtransactions) completely uneconomical during periods of particularly high demand.
To solve this problem, we could increase the block size limit to create space for more transactions. An increase in the "supply" of block space will lead to a lower price equilibrium for transaction fees.
However, increasing block size shifts the cost to node operators and requires them to expend more resources to validate and store the blockchain. Because blockchains are gossip protocols, each node is required to know and validate every single transaction that occurs on the network. Furthermore, once validated, each transaction and block must be propagated to the node's "neighbors", multiplying the bandwidth requirements. As such, the greater the block size, the greater the bandwidth, processing, and storage requirements for each individual node. Increasing transaction capacity in this way has the undesirable effect of centralizing the system by reducing the number of nodes and node operators. Since node operators are not compensated for running nodes, if nodes are very expensive to run, only a few well-funded node operators will continue to run nodes.
However, increasing block size shifts the cost to node operators and requires them to expend more resources to validate and store the blockchain. Because blockchains are gossip protocols, each node is required to know and validate every single transaction that occurs on the network. Furthermore, once validated, each transaction and block must be propagated to the node's "neighbors," multiplying the bandwidth requirements. As such, the greater the block size, the greater the bandwidth, processing, and storage requirements for each individual node. Increasing transaction capacity in this way has the undesirable effect of centralizing the system by reducing the number of nodes and node operators. Since node operators are not compensated for running nodes, if nodes are very expensive to run, only a few well-funded node operators will continue to run nodes.
==== Scaling Blockchains
The side effects of increasing the block size or decreasing the block time with respect to centralization of the network are severe as a few calculations with the numbers show.
The side effects of increasing the block size or decreasing the block time with respect to centralization of the network are severe, as a few calculations with the numbers show.
Let us assume the usage of Bitcoin grows so that the network has to process 40,000 transactions per second, which is the approximate transaction processing level of the VISA network during peak usage.
@ -166,48 +166,47 @@ Users also have many other demands on their bandwidth and cannot be expected to
Furthermore, storing this information locally would result in 864 gigabytes per day. This is roughly one terabyte of data, or the size of a hard drive.
Verifying 40,000 ECDSA signatures per second is also barely feasible (c.f.: https://bitcoin.stackexchange.com/questions/95339/how-many-bitcoin-transactions-can-be-verified-per-second) making the _Initial Blockchain Download (IBD)_ of the Bitcoin blockchain (synchronizing and verifying everything starting from the genesis block) almost impossible without very expensive hardware.
Verifying 40,000 Elliptic Curve Digital Signature Algorithm (ECDSA) signatures per second is also barely feasible (see https://bitcoin.stackexchange.com/questions/95339/how-many-bitcoin-transactions-can-be-verified-per-second[this article on StackExchange]) making the _initial blockchain download (IBD)_ of the Bitcoin blockchain (synchronizing and verifying everything starting from the genesis block) almost impossible without very expensive hardware.
While 40,000 transactions per second seems like a lot, it only achieves parity with traditional financial payment networks at peak times. Innovations in machine-to-machine payments, micro-transactions and other applications are likely to push demand to many orders higher than that.
While 40,000 transactions per second seems like a lot, it only achieves parity with traditional financial payment networks at peak times. Innovations in machine-to-machine payments, microtransactions, and other applications are likely to push demand to many orders higher than that.
Simply put: You can't scale a blockchain to validate the entire world's transactions in a decentralized way.
_But what if each node wasn't required to know and validate every single transaction? What if there was a way to have scalable off-chain transactions, without losing the security of the Bitcoin network?_
In February 2015, Joseph Poon and Thaddeus Dryja proposed a possible solution to the Bitcoin Scalability Problem, with the publication of _"The Bitcoin Lightning Network: Scalable Off-Chain Instant Payments"_
In February 2015, Joseph Poon and Thaddeus Dryja proposed a possible solution to the Bitcoin Scalability Problem, with the publication of "The Bitcoin Lightning Network: Scalable Off-Chain Instant Payments."footnote:[Joseph Poon and Thaddeus Dryja. "The Bitcoin Lightning Network: Scalable Off-Chain Instant Payments." DRAFT Version 0.5.9.2. January 14, 2016. https://lightning.network/lightning-network-paper.pdf[].]
In the (now outdated) whitepaper, Poon and Dryja estimate that in order for Bitcoin to reach the 47,000 transactions per second processed at peak by Visa, it would require 8 GB blocks.
This would make running a node completely untenable for anyone but large scale enterprises and industrial grade operations.
This would make running a node completely untenable for anyone but large-scale enterprises and industrial-grade operations.
The result would be a network in which only a few users could actually validate the state of the ledger.
Bitcoin relies on users validating the ledger for themselves, without explicitly trusting third parties, in order to stay decentralized.
Pricing users out of running nodes would force the average user to trust third parties to discover the state of the ledger, ultimately breaking the trust model of Bitcoin.
The Lightning Network proposes a new network, a "second layer", where users can make payments to each other peer-to-peer, without the necessity of publishing a transaction to the Bitcoin blockchain for each payment.
The Lightning Network proposes a new network, a second layer, where users can make payments to each other peer-to-peer, without the necessity of publishing a transaction to the Bitcoin blockchain for each payment.
Users may pay each other on the Lightning Network as many times as they want, without creating additional Bitcoin transactions or incurring on-chain fees.
They only make use of the Bitcoin blockchain in order to load bitcoin onto the Lightning network initially and to "settle", that is, to remove bitcoin from the Lightning Network.
The result is that many more Bitcoin payments can take place "off-chain", with only the initial loading and final settlement transactions needing to be validated and stored by Bitcoin nodes.
Aside from reducing the burden on nodes, payments on the Lightning Network are cheaper for users as they do not need to pay blockchain fees, and more private for users as they are not published to all participants of the network and furthermore not stored permanently.
They only make use of the Bitcoin blockchain to load bitcoin onto the Lightning Network initially and to _settle_, that is, to remove bitcoin from the Lightning Network.
The result is that many more Bitcoin payments can take place off-chain, with only the initial loading and final settlement transactions needing to be validated and stored by Bitcoin nodes.
Aside from reducing the burden on nodes, payments on the Lightning Network are cheaper for users because they do not need to pay blockchain fees and more private for users because they are not published to all participants of the network and furthermore not stored permanently.
While the Lightning Network was initially conceived for Bitcoin, it can be implemented on any blockchain that meets some basic technical requirements. Other blockchains, such as Litecoin, already support the Lightning Network. Additionally, several other blockchains are developing similar "second layer" or "layer 2" solutions to help them scale.
While the Lightning Network was initially conceived for Bitcoin, it can be implemented on any blockchain that meets some basic technical requirements. Other blockchains, such as Litecoin, already support the Lightning Network. Additionally, several other blockchains are developing similar second layer or "layer 2" solutions to help them scale.
=== The Lightning Network's Defining Features
The Lightning Network is a network that operates as a "second layer" protocol on top of Bitcoin and other blockchains. The Lightning Network enables fast, secure, private, trustless, and permissionless payments. Here are some of the features of the Lightning Network:
The Lightning Network is a network that operates as a second layer protocol on top of Bitcoin and other blockchains. The Lightning Network enables fast, secure, private, trustless, and permissionless payments. Here are some of the features of the Lightning Network:
* Users of the Lightning Network can route payments to each other for low cost and in real time.
* Users who exchange value over the Lightning Network do not need to wait for block confirmations for payments.
* Once a payment on the Lightning Network has completed, usually within a few seconds, it is final and cannot be reversed. Like a Bitcoin transaction, a payment on the Lightning Network can only be refunded by the recipient.
* While "on-chain" Bitcoin transactions are broadcast and verified by all nodes in the network, payments routed on the Lightning Network are transmitted between pairs of nodes and are not visible to everyone, resulting in much greater privacy.
* Unlike transactions on the Bitcoin Network, payments routed on the Lightning Network do not need to be stored permanently. Lightning thus uses fewer resources and hence is cheaper. This property also has benefits for privacy.
* Whereas on-chain Bitcoin transactions are broadcast and verified by all nodes in the network, payments routed on the Lightning Network are transmitted between pairs of nodes and are not visible to everyone, resulting in much greater privacy.
* Unlike transactions on the Bitcoin network, payments routed on the Lightning Network do not need to be stored permanently. Lightning thus uses fewer resources and hence is cheaper. This property also has benefits for privacy.
* The Lightning Network uses onion routing, similar to the protocol used by The Onion Router (Tor) privacy network, so that even the nodes involved in routing a payment are only directly aware of their predecessor and successor in the payment route.
* When used on top of Bitcoin, the Lightning Network uses real bitcoin which is always in the possession (custody) and full control of the user. Lightning is not a separate token or coin, it _is_ Bitcoin.
* When used on top of Bitcoin, the Lightning Network uses real bitcoin, which is always in the possession (custody) and full control of the user. Lightning is not a separate token or coin, it _is_ Bitcoin.
[[user-stories]]
=== Lightning Network Use Cases, Users, and Their Stories
In order to better understand how the Lightning Network actually works, and why people use it, we'll be following a number of users and their stories.
To better understand how the Lightning Network actually works, and why people use it, we'll be following a number of users and their stories.
In our examples, some of the people have already used Bitcoin and others are completely new to the Bitcoin network. Each person and their story, as listed here, illustrate one or more specific use cases. We'll be revisiting them throughout this book:
@ -215,10 +214,10 @@ consumer::
Alice is a Bitcoin user who wants to make fast, secure, cheap, and private payments for small retail purchases. She buys coffee with bitcoin, using the Lightning Network.
merchant::
Bob owns a coffee shop, "Bob's Cafe". "On-chain" Bitcoin payments don't scale for small amounts like a cup of coffee, so he uses the Lightning Network to accept Bitcoin payments almost instantaneously and for low fees.
Bob owns a coffee shop, "Bob's Cafe." On-chain Bitcoin payments don't scale for small amounts like a cup of coffee, so he uses the Lightning Network to accept Bitcoin payments almost instantaneously and for low fees.
software service business::
Chan is a Chinese entrepreneur who sells information services related to the Lightning Network, as well as Bitcoin and other cryptocurrencies. Chan is selling these information services over the Internet by implementing micro-payments over the Lightning Network. Additionally, Chan has implemented a liquidity provider service that rents inbound channel capacity on the Lightning Network, charging a small bitcoin fee for each rental period.
Chan is a Chinese entrepreneur who sells information services related to the Lightning Network, as well as Bitcoin and other cryptocurrencies. Chan is selling these information services over the internet by implementing micropayments over the Lightning Network. Additionally, Chan has implemented a liquidity provider service that rents inbound channel capacity on the Lightning Network, charging a small bitcoin fee for each rental period.
gamer::
Dina is a teenage gamer from Russia. She plays many different computer games, but her favorite ones are those that have an "in-game economy" based on real money. As she plays games, she also earns money by acquiring and selling virtual in-game items. The Lightning Network allows her to transact in small amounts for in-game items as well as earn small amounts for completing quests.
@ -231,4 +230,4 @@ We looked at the history of the Lightning Network and the motivations behind sec
We learned basic terminology including node, payment channel, on-chain transactions, and off-chain payments.
Finally, we met Alice, Bob, Chan and Dina, who we'll be following throughout the rest of the book. In the next chapter, we'll meet Alice and walk through her thought process as she selects a Lightning wallet and prepares to make her first Lightning payment to buy a cup of coffee from Bob's Cafe.
Finally, we met Alice, Bob, Chan, and Dina, who we'll be following throughout the rest of the book. In the next chapter, we'll meet Alice and walk through her thought process as she selects a Lightning wallet and prepares to make her first Lightning payment to buy a cup of coffee from Bob's Cafe.
@ -96,11 +96,11 @@ Operating a node on a 32-bit operating system and/or 32-bit CPU is not recommend
At a minimum, the following will be required to run a Lightning node:
* **CPU**: Sufficient processing power will be required to run a Bitcoin node, which will continuously download and validate new blocks. The user also needs to consider the initial block download (IBD) when setting up a new Bitcoin node, which can take anywhere from several hours to several days. A 2-core or 4-core CPU is recommended.
CPU:: Sufficient processing power will be required to run a Bitcoin node, which will continuously download and validate new blocks. The user also needs to consider the initial blockchain download (IBD) when setting up a new Bitcoin node, which can take anywhere from several hours to several days. A 2-core or 4-core CPU is recommended.
* **RAM**: A system with 2 GB of RAM will _barely_ run both Bitcoin and Lightning nodes. It will perform much better with at least 4 GB of RAM. The IBD will be especially challenging with less than 4 GB of RAM. More than 8 GB of RAM is unnecessary because the CPU is the greater bottleneck for these types of services, due to cryptographic operations such as signature validation.
RAM:: A system with 2 GB of RAM will _barely_ run both Bitcoin and Lightning nodes. It will perform much better with at least 4 GB of RAM. The IBD will be especially challenging with less than 4 GB of RAM. More than 8 GB of RAM is unnecessary because the CPU is the greater bottleneck for these types of services, due to cryptographic operations such as signature validation.
* **Storage drive**: This can be a hard disk drive (HDD) or a solid state drive (SSD).
Storage drive:: This can be a hard disk drive (HDD) or a solid state drive (SSD).
An SSD will be significantly quicker (but more expensive) for running a node.
Most of the storage is used for the Bitcoin blockchain, which is hundreds of gigabytes in size.
A fair trade-off (cost for complexity) is to buy a small SSD to boot the OS from and a larger HDD to store large data objects (mostly databases).
@ -180,7 +180,7 @@ In addition to a Bitcoin and Lightning node, myNode can optionally install a var
==== Umbrel
Famous for their UX/UI (shown in <<umbrel>>), _Umbrel_ provides a very easy and accessible way to get your Bitcoin and Lightning node up and running in no time, especially for beginners. A very distinctive feature is that Umbrel utilizes Neutrino/SPV during the Initial Blockchain Download (IBD) so you can instantly start using your node. Once Bitcoin Core is fully synced in the background it automatically switches over and disables SPV mode. Umbrel OS supports the Raspberry Pi 4 and can also be installed on any Linux-based OS or on a virtual machine on macOS or Windows. You can also connect any wallet that supports Bitcoin Core P2P, Bitcoin Core RPC, the Electrum protocol, or lndconnect.
Famous for their UX/UI (shown in <<umbrel>>), _Umbrel_ provides a very easy and accessible way to get your Bitcoin and Lightning node up and running in no time, especially for beginners. A very distinctive feature is that Umbrel utilizes Neutrino/SPV during the initial blockchain download (IBD) so you can instantly start using your node. Once Bitcoin Core is fully synced in the background it automatically switches over and disables SPV mode. Umbrel OS supports the Raspberry Pi 4 and can also be installed on any Linux-based OS or on a virtual machine on macOS or Windows. You can also connect any wallet that supports Bitcoin Core P2P, Bitcoin Core RPC, the Electrum protocol, or lndconnect.
There's no need to wait for a rainy day—you can go right to https://getumbrel.com[Umbrel] to learn more.
@ -359,7 +359,7 @@ StateDirectoryMode=0710
WantedBy=multi-user.target
----
As the root user, install the script by copying it into the +systemd+ service folder +/lib/systemd/system/+ and then reload +systemd+:
As the root user, install the script by copying it into the +systemd+ service folder _/lib/systemd/system/_ and then reload +systemd+:
To configure your node, you need to create and reference a configuration file. By convention, this file is usually created in +/etc+, under a directory with the name of the program. For example, Bitcoin Core and LND configurations would usually be stored in:
+/etc/bitcoin/bitcoin.conf+
+/etc/lnd/lnd.conf+
To configure your node, you need to create and reference a configuration file. By convention, this file is usually created in _/etc_, under a directory with the name of the program. For example, Bitcoin Core and LND configurations would usually be stored in _/etc/bitcoin/bitcoin.conf_ and
_/etc/lnd/lnd.conf_, respectively.
These configuration files are text files with each line expressing one configuration option and its value. Default values are assumed for anything not defined in the configuration file. You can see what options can be set in the configuration in two ways. First, running the node application with a +help+ argument will show the options that can be defined on the command line. These same options can be defined in the configuration file. Second, you can usually find an example configuration file, with all the default options, in the code repository of the software.
You can find one example of a configuration file in each of the Docker images we used in <<set_up_a_lightning_node>>. For example, the file +code/docker/bitcoind/bitcoind/bitcoin.conf+:
You can find one example of a configuration file in each of the Docker images we used in <<set_up_a_lightning_node>>. For example, the file _code/docker/bitcoind/bitcoind/bitcoin.conf_:
.Configuration file for docker bitcoind (code/docker/bitcoind/bitcoind/bitcoin.conf)
That particular configuration file configures Bitcoin Core for operation as a +regtest+ node and provides a weak username and password for remote access, so you shouldn't use it for your node configuration. However, it serves to illustrate the syntax of a configuration file and you can make adjustments to it in the docker container to experiment with different options. See if you can use the +bitcoind -help+ command to understand what each of the options does in the context of the docker network we build in <<set_up_a_lightning_node>>.
That particular configuration file configures Bitcoin Core for operation as a +regtest+ node and provides a weak username and password for remote access, so you shouldn't use it for your node configuration. However, it serves to illustrate the syntax of a configuration file and you can make adjustments to it in the Docker container to experiment with different options. See if you can use the +bitcoind -help+ command to understand what each of the options does in the context of the docker network we build in <<set_up_a_lightning_node>>.
Often, the defaults suffice, and with a few modifications your node software can be configured quickly. To get a Bitcoin Core node running with minimal customization, you only need four lines of configuration:
[source, subs="quotes"]
----
server=1
daemon=1
txindex=1
rpcuser=USERNAME
rpcpassword=PASSWORD
rpcuser=_USERNAME_
rpcpassword=_PASSWORD_
----
Even the +txindex+ option is not strictly necessary, though it will ensure your Bitcoin node creates an index of all transactions, which is required for some applications. The +txindex+ option is not required to run a Lightning node.
A c-lightning Lightning node running on the same server also requires only a few lines in the configuration:
[source, subs="quotes"]
----
network=mainnet
bitcoin-rpcuser=USERNAME
bitcoin-rpcpassword=PASSWORD
bitcoin-rpcuser=_USERNAME_
bitcoin-rpcpassword=_PASSWORD_
----
In general, it is a good idea to minimize the amount of customization of these systems. The default configuration is carefully designed to support the most common deployments. If you modify a default value, it may cause problems later on or reduce the performance of your node. In short, modify only when necessary!
==== Network Configuration
Network configuration is normally not an issue when configuring a new application. However, peer-to-peer networks like Bitcoin and the Lightning network present some unique challenges for network configuration.
Network configuration is normally not an issue when configuring a new application. However, peer-to-peer networks like Bitcoin and the Lightning Network present some unique challenges for network configuration.
In a centralized service, your computer connects to the "big servers" of some corporation, and not vice-versa. Your home internet connection is actually configured on the assumption that you are simply a consumer of services provided by others. But in a peer-to-peer system, every peer both consumes from and provides services to other nodes. If you're running a Bitcoin or Lightning node at your home, you're providing a service to other computers on the internet. Your internet service by default is not configured to allow you to run servers and may need some additional configuration to enable others to reach your node.
In a centralized service, your computer connects to the "big servers" of some corporation, and not viceversa. Your home internet connection is actually configured on the assumption that you are simply a consumer of services provided by others. But in a peer-to-peer system, every peer both consumes from and provides services to other nodes. If you're running a Bitcoin or Lightning node at your home, you're providing a service to other computers on the internet. Your internet service by default is not configured to allow you to run servers and may need some additional configuration to enable others to reach your node.
If you want to run a Bitcoin or Lightning node, you need to make it possible for other nodes on the internet to connect to you. That means enabling incoming TCP connections to the Bitcoin port (port 8333 by default) or Lightning port (port 9735 by default). While you can run a Bitcoin node without incoming connectivity, you can't do that with a Lightning node. A Lightning node must be accessible to others from outside your network.
By default, your home internet router does not expect incoming connections from the outside, and in fact incoming connections are blocked. Your internet router IP address is the only externally accessible IP address, and all the computers you run inside your home network share that single IP address. This is achieved by a mechanism called _Network Address Translation (NAT)_ which allows your internet router to act as an intermediary for all outbound connections. If you want to allow an inbound connection you have to set up _Port Forwarding_, which tells your internet router that incoming connections on specific ports should be forwarded to specific computers inside the network. You can do this manually by changing your internet router configuration or, if your router supports it, through an automatic port forwarding mechanism called _Universal Plug and Play (UPNP)_.
By default, your home internet router does not expect incoming connections from the outside, and in fact incoming connections are blocked. Your internet router IP address is the only externally accessible IP address, and all the computers you run inside your home network share that single IP address. This is achieved by a mechanism called _Network Address Translation_ (_NAT_), which allows your internet router to act as an intermediary for all outbound connections. If you want to allow an inbound connection, you have to set up _port forwarding_, which tells your internet router that incoming connections on specific ports should be forwarded to specific computers inside the network. You can do this manually by changing your internet router configuration or, if your router supports it, through an automatic port forwarding mechanism called _Universal Plug and Play_ (_UPnP_).
An alternative mechanism to port forwarding is to enable The Onion Router (Tor), which provides a kind of virtual private network overlay that allows incoming connections to an _onion address_. If you run Tor, you don't need to do port forwarding nor enable incoming connections to Bitcoin or Lightning ports. If you run your nodes using Tor, all traffic goes through Tor and no other ports are used.
@ -434,48 +434,47 @@ Let's look at different ways you can make it possible for others to connect to y
===== It just works!
There is a possibility that your internet service provider or router is configured to support UPNP by default and everything just works automatically. Let's try this approach first, just in case we are lucky.
There is a possibility that your internet service provider or router is configured to support UPnP by default and everything just works automatically. Let's try this approach first, just in case we are lucky.
Assuming you already have a Bitcoin or Lightning node running, we will try and see if they are accessible from the outside.
[NOTE]
====
For this test to work, you have to have either a Bitcoin or Lightning node (or both) up and running on your home network. If your router supports UPNP, the incoming traffic will automatically be forwarded to the corresponding ports on the computer running the node.
For this test to work, you have to have either a Bitcoin or Lightning node (or both) up and running on your home network. If your router supports UPnP, the incoming traffic will automatically be forwarded to the corresponding ports on the computer running the node.
====
You can use some very popular and useful websites to find out what is your external IP address and whether it allows and forwards incoming connections to a known port. Here are two that are reliable:
https://canyouseeme.org/
https://www.whatismyip.com/port-scanner/
* https://canyouseeme.org/
* https://www.whatismyip.com/port-scanner/
By default, these services only allow you to check incoming connections to the IP address from which you are connecting. This is done to prevent you from using the service to scan other people's networks and computers. You will see your router's external IP address and a field for entering a port number. If you haven't changed the default ports in your node configuration, try port 8333 (Bitcoin) and/or 9735 (Lightning).
In <<ln_port_check>> you can see the result of checking port 9735 on a server running Lightning, using the +whatismyip.org+ port scanner tool. It shows that the server is accepting incoming connections to the Lightning port. If you see a result like this, you are all set!
[[ln_port_check]]
.Checking for incoming port 9735
image::images/mtln_0503.png[]
In <<ln_port_check>> you can see the result of checking port 9735 on a server running Lightning, using the +whatismyip.org+ port scanner tool. It shows that the server is accepting incoming connections to the Lightning port. If you see a result like this, you are all set!
===== Automatic port forwarding using upnp
===== Automatic port forwarding using UPnP
Sometimes, even if your internet router supports UPNP, it may be turned off by default. In that case you need to change your internet router configuration from its web administration interface:
Sometimes, even if your internet router supports UPnP, it may be turned off by default. In that case you need to change your internet router configuration from its web administration interface:
. Connect to your internet router's configuration website. Usually this can be done by connecting to the _gateway address_ of your home network using a web browser. You can find the gateway address by looking at the IP configuration of any computer on your home network. It is often the first address in one of the non-routable networks, like 192.168.0.1 or 10.0.0.1. Check all stickers on your router as well for the _gateway address_. Once found, open a browser and enter the IP address into the browser URL/Search box, e.g. "192.168.0.1" or "http://192.168.0.1".
. Connect to your internet router's configuration website. Usually this can be done by connecting to the _gateway address_ of your home network using a web browser. You can find the gateway address by looking at the IP configuration of any computer on your home network. It is often the first address in one of the non-routable networks, like 192.168.0.1 or 10.0.0.1. Check all stickers on your router as well for the _gateway address_. Once found, open a browser and enter the IP address into the browser URL/Search box, e.g., "192.168.0.1" or "http://192.168.0.1."
. Find the administrator username and password for the web configuration panel of the router. This is often written on a sticker on the router itself and may be as simple as "admin" and "password". A quick web search for your ISP and router model can also help you find this information.
. Find the administrator username and password for the web configuration panel of the router. This is often written on a sticker on the router itself and may be as simple as "admin" and "password." A quick web search for your ISP and router model can also help you find this information.
. Find a setting for UPNP and turn it on.
. Find a setting for UPnP and turn it on.
Restart your Bitcoin and/or Lightning node and repeat the open port test with one of the websites we used in the previous section.
===== Using tor for incoming connections
_The Onion Router (Tor)_ is a virtual private network with the special property that it encrypts communications between hops, such that any intermediary node cannot determine the origin or destination of a packet. Both Bitcoin and Lightning nodes support operation over Tor, which enables you to operate a node without revealing your IP address or location. Hence, it provides a high level of privacy to your network traffic. An added benefit of running Tor is that because it operates as a VPN, it resolves the problem of port forwarding from your internet router. Incoming connections are received over the Tor tunnel, and your node can be found through an ad-hoc generated _onion address_ instead of an IP address.
_The Onion Router_ (_Tor_) is a VPN with the special property that it encrypts communications between hops, such that any intermediary node cannot determine the origin or destination of a packet. Both Bitcoin and Lightning nodes support operation over Tor, which enables you to operate a node without revealing your IP address or location. Hence, it provides a high level of privacy to your network traffic. An added benefit of running Tor is that, because it operates as a VPN, it resolves the problem of port forwarding from your internet router. Incoming connections are received over the Tor tunnel, and your node can be found through an adhoc generated _onion address_ instead of an IP address.
Enabling Tor requires two steps: First you must install the Tor router and proxy on your computer. Second, you must enable the use of the Tor proxy in your Bitcoin or Lightning configuration.
To install Tor on a Ubuntu Linux system that uses the +apt+ package manager, run:
To install Tor on an Ubuntu Linux system that uses the +apt+ package manager, run:
----
sudo apt install tor
@ -491,7 +490,7 @@ tor.streamisolation=true
listen=localhost
----
This will enable Tor (+tor.active+), establish a v3 onion service (+tor.v3=true+), use a different onion stream for each connection (+tor.streamisolation+), and restrict listening for connections to the local host only, to avoid leaking your IP address (+listen=localhost+).
This will enable Tor (+tor.active+), establish a v3 onion service (+tor.v3=true+), use a different onion stream for each connection (+tor.streamisolation+), and restrict listening for connections to the local host only, to avoid leaking your IP address (pass:[<code>l⁠i⁠s⁠t⁠e⁠n​=⁠l⁠o⁠c⁠a⁠l⁠h⁠o⁠s⁠t</code>]).
You can check if Tor is correctly installed and working by running a simple one-line command. This command should work on most flavors of Linux:
@ -499,22 +498,21 @@ You can check if Tor is correctly installed and working by running a simple one-
If everything is working properly, the response of this command should be +"Congratulations. This browser is configured to use Tor."+.
If everything is working properly, the response of this command should be +"Congratulations. This browser is configured to use Tor."+
Due to the nature of Tor, you can't easily use an external service to check if your node is reachable via an onion address. Nonetheless, you should see your Tor onion address in the logs of your Lightning node. It is a long string of letters and numbers followed by the suffix +.onion+. Your node should now be reachable from the internet, with the added bonus of privacy!
===== Manual port forwarding
This is the most complex process and requires quite a bit of technical skill. The details depend on the type of internet router you have, your service provider settings and policies, and a lot of other context. Try UPNP or Tor first, before you try this much more difficult mechanism.
This is the most complex process and requires quite a bit of technical skill. The details depend on the type of internet router you have, your service provider settings and policies, and a lot of other context. Try UPnP or Tor first, before you try this much more difficult mechanism.
The basic steps are as follows:
. Find the IP address of the computer your node is on. This is usually dynamically allocated by the Dynamic Host Configuration Protocol (DHCP) and is often somewhere in the 192.168.x.x or 10.x.x.x range.
. Find the Media Access Control (MAC) address of your node's network interface. This can be found in the internet settings of that computer.
. Assign a static IP address for your node so that it is always the same one. You can use the IP address it currently has. On your internet router, look for "Static Leases" under the DHCP configuration. Map the MAC address to the IP address you selected. Now your node will always have that IP address allocated to it. Alternatively, you can look at your router's DHCP configuration and find out what its DHCP address range is. Select an unused address _outside_ of the DHCP address range. Then, on the server, configure the network to stop using DHCP and hard-code the selected non-DHCP IP address into the operating system network configuration.
. Find the media access control (MAC) address of your node's network interface. This can be found in the internet settings of that computer.
. Assign a static IP address for your node so that it is always the same one. You can use the IP address it currently has. On your internet router, look for "Static Leases" under the DHCP configuration. Map the MAC address to the IP address you selected. Now your node will always have that IP address allocated to it. Alternatively, you can look at your router's DHCP configuration and find out what its DHCP address range is. Select an unused address _outside_ of the DHCP address range. Then, on the server, configure the network to stop using DHCP and hardcode the selected non-DHCP IP address into the operating system network configuration.
. Finally, set up "Port Forwarding" on your internet router to route incoming traffic on specific ports to the selected IP address of your server.
@ -522,11 +520,11 @@ Once done reconfiguring, repeat the port check using one of the websites from th
=== Security of Your Node
A Lightning node is, by definition, a hot wallet. That means that the funds (both on-chain and off-chain) controlled by a Lightning node are directly controlled by keys that are loaded in the node's memory or stored on the node's hard disk. If a Lightning node is compromised, it is trivial to create on-chain or off-chain transactions to drain its funds. It is therefore critically important that you protect it from unauthorized access.
A Lightning node is, by definition, a _hot wallet_. That means that the funds (both on-chain and off-chain) controlled by a Lightning node are directly controlled by keys that are loaded in the node's memory or stored on the node's hard disk. If a Lightning node is compromised, it is trivial to create on-chain or off-chain transactions to drain its funds. It is therefore critically important that you protect it from unauthorized access.
Security is a holistic effort, meaning that you have to secure every layer of a system. As the saying goes: the chain is only as strong as the weakest link. This is an important concept in information security and we will apply it to our node.
Security is a holistic effort, meaning that you have to secure every layer of a system. As the saying goes: the chain is only as strong as the weakest link. This is an important concept in information security, and we will apply it to our node.
Despite all security measures you will take, remember that the Lightning Network is an early-stage experimental technology and there are likely to be exploitable bugs in the code of any project you use. *Do not put more money than you are willing to risk losing on the Lightning Network.*
Despite all the security measures you will take, remember that the Lightning Network is an early-stage experimental technology and there are likely to be exploitable bugs in the code of any project you use. _Do not put more money than you are willing to risk losing on the Lightning Network._
==== Operating System Security
@ -535,26 +533,25 @@ Securing an operating system is a vast topic that is beyond the scope of this bo
To secure your operating system, here are some of the top items to consider:
. Provenance: Start by ensuring that you are downloading the correct operating system image and verify any signatures or checksums before installing it. Extend this to any software that you install. Double-check any source or URL from where you download. Verify the integrity and correctness of the downloaded software via signature and checksum verification.
. Maintenance: Make sure that you keep your operating system up to date. Enable automated daily or weekly installation of security updates.
. Least privilege: Set up users for specific processes and give them the least access needed to run a service. Do not run processes with admin privileges (e.g. root).
. Process isolation: Use the operating system features to isolate processes from each other.
. File system permissions: Configure the file system carefully, on the least-privilege principle. Do not make files readable or writable by everyone.
. Strong authentication: Use strong randomly generated passwords or, whenever possible, public-key authentication. E.g. it is safer to use Secure Shell (SSH) with a cryptographic key pair instead of a password.
. Two-factor authentication (2FA): Use two-factor authentication wherever possible, including Universal 2-Factor (U2F) with hardware security keys. This applies to all external services you might be using such as your cloud service provider. You can apply this also to your own set-up such as your own SSH configuration. Use 2FA also for indirect services. For example, say you are using a cloud service. You gave your cloud service provider an email address, so you should also protect your email address with 2FA.
. Backup: Make backups of your system, and make sure you protect the backups with encryption too. Perform these backups periodically. At least once test if you can restore your backup and that your backup is complete and accessible. If possible, keep one copy of your backups on a different disk to avoid that a single hard disk failure destroys _both_ your active node as well as your backup copies.
. Vulnerability and exposure management: Use remote scanning to ensure you have minimized the attack surface of your system. Close any unnecessary services or ports.
- Minimize: Install only software and packages that you really need and use. Uninstall packages that you no longer use. It is recommended that you do _not_ use your node computer for non-node activities that you can perform on another of your computers. Especially, if you can, do _not_ use your node computer for browsing, surfing the internet, or reading your email.
Provenance:: Start by ensuring that you are downloading the correct operating system image and verify any signatures or checksums before installing it. Extend this to any software that you install. Double-check any source or URL from where you download. Verify the integrity and correctness of the downloaded software via signature and checksum verification.
Maintenance:: Make sure that you keep your operating system up to date. Enable automated daily or weekly installation of security updates.
Least privilege: Set up users for specific processes and give them the least access needed to run a service. Do not run processes with admin privileges (e.g., +root+).
Process isolation:: Use the operating system features to isolate processes from each other.
File system permissions:: Configure the file system carefully, on the least-privilege principle. Do not make files readable or writable by everyone.
Strong authentication:: Use strong randomly generated passwords or, whenever possible, public-key authentication. For example, it is safer to use Secure Shell (SSH) with a cryptographic key pair instead of a password.
Two-factor authentication (2FA):: Use two-factor authentication wherever possible, including Universal 2nd Factor (U2F) with hardware security keys. This applies to all external services you might be using such as your cloud service provider. You can apply this also to your own setup, such as your own SSH configuration. Use 2FA also for indirect services. For example, say you are using a cloud service. You gave your cloud service provider an email address, so you should also protect your email address with 2FA.
Backup:: Make backups of your system, and make sure you protect the backups with encryption too. Perform these backups periodically. At least once test if you can restore your backup and that your backup is complete and accessible. If possible, keep one copy of your backups on a different disk to avoid that a single hard disk failure destroys _both_ your active node as well as your backup copies.
Vulnerability and exposure management:: Use remote scanning to ensure you have minimized the attack surface of your system. Close any unnecessary services or ports. Install only software and packages that you really need and use. Uninstall packages that you no longer use. It is recommended that you do _not_ use your node computer for non-node activities that you can perform on another of your computers. Especially, if you can, do _not_ use your node computer for browsing, surfing the internet, or reading your email.
This is a list of the most basic security measures. It is by no means exhaustive.
==== Node Access
Your Lightning node will expose a Remote Procedure Call (RPC) Application Programming Interface (API). This means that your node can be controlled remotely by commands sent to a specific TCP port. Access control to that RPC API is achieved by some form of user authentication. Depending on the type of Lightning node you set up, this will either be done by username/password authentication or by a mechanism called an authentication _macaroon_. As the name implies, a macaroon is a more sophisticated type of cookie. Unlike a cookie, it is cryptographically signed and can express a set of access capabilities.
Your Lightning node will expose a remote procedure call (RPC) API. This means that your node can be controlled remotely by commands sent to a specific TCP port. Access control to that RPC API is achieved by some form of user authentication. Depending on the type of Lightning node you set up, this will either be done by username/password authentication or by a mechanism called an authentication _macaroon_. As the name implies, a macaroon is a more sophisticated type of cookie. Unlike a cookie, it is cryptographically signed and can express a set of access capabilities.
For example, LND uses macaroons to grant access to the RPC API. By default, the LND software creates three macaroons with different levels of access, called +admin+, +invoice+, and +readonly+. Depending on which macaroon you copy and use in your RPC client, you either have "readonly" access, "invoice" access (which includes the "readonly" capabilities), or "admin" access which gives you full control. There is also a macaroon "bakery" function in LND that can construct macaroons with any combination of capabilities with very fine-grained control.
For example, LND uses macaroons to grant access to the RPC API. By default, the LND software creates three macaroons with different levels of access, called +admin+, +invoice+, and +readonly+. Depending on which macaroon you copy and use in your RPC client, you either have _readonly_ access, _invoice_ access (which includes the readonly capabilities), or _admin_ access, which gives you full control. There is also a macaroon +bakery+ function in LND that can construct macaroons with any combination of capabilities with very fine-grained control.
If you use a username/password authentication model, make sure you select a long and random password. You will not have to type this password often as it will be stored in the configuration files. You should therefore pick one that cannot be guessed. Many of the examples you will see include poorly chosen passwords, and often people copy these into their own systems, providing easy access to anyone. Don't do that! Use a password manager to generate a long random alpha-numeric password. Since certain special characters such as +$?/!*\&%`"'+ can interfere with the command line, it is best to avoid these for passwords that will be used in a shell environment. To avoid problems stick with long random alpha-numeric passwords.
If you use a username/password authentication model, make sure you select a long and random password. You will not have to type this password often,because it will be stored in the configuration files. You should therefore pick one that cannot be guessed. Many of the examples you will see include poorly chosen passwords, and often people copy these into their own systems, providing easy access to anyone. Don't do that! Use a password manager to generate a long random alphanumeric password. Since certain special characters such as +$?/!*\&%`"'+ can interfere with the command line, it is best to avoid these for passwords that will be used in a shell environment. To avoid problems, stick with long random alphanumeric passwords.
A plain alphanumeric sequence that is longer than 12 characters and randomly generated is usually sufficient. If you plan to store large amounts of money on your Lightning node and are concerned about remote brute-force attacks, select a password length of more than 20 characters to make such attacks practically infeasible.
@ -562,32 +559,30 @@ A plain alphanumeric sequence that is longer than 12 characters and randomly gen
A very important consideration when running a Lightning node is the issue of backups. Unlike a Bitcoin wallet, where a BIP39 mnemonic phrase can recover all the state of the wallet, in Lightning this is _not_ the case.
Lightning wallets do use a BIP39 mnemonic phrase backup, but only for the on-chain wallet. However, due to the way channels are constructed, the mnemonic phrase is _not_ sufficient to restore a Lightning node. An additional layer of backups is needed, which is called the _Static Channel Backup (SCB)_. Without an SCB, a Lightning node operator may lose _all_ the funds that are in channels if they lose the Lightning node data store.
Lightning wallets do use a BIP39 mnemonic phrase backup, but only for the on-chain wallet. However, due to the way channels are constructed, the mnemonic phrase is _not_ sufficient to restore a Lightning node. An additional layer of backups is needed, which is called the _static channel backup_ (_SCB_). Without an SCB, a Lightning node operator may lose _all_ the funds that are in channels if they lose the Lightning node data store.
[WARNING]
====
Do _not_ fund channels until you have put a system in place to continuously backup your channel state. Your backups should be moved "offsite" to a different system and location from your node, so that they can survive a variety of system failures (power loss, data corruption, etc.) or natural disasters (flood, fire, etc.).
====
Static Channel Backups are not a panacea. First, the state of each channel needs to be backed up every time there is a new commitment transaction. Second, restoring from a channel backup is dangerous. If you do not have the _last_ commitment transaction and you accidentally broadcast an old (revoked) commitment, your channel peer will assume you are trying to cheat and claim the entire channel balance with a penalty transaction. To make sure you are closing the channel, you need to do a _cooperative close_. But a malicious peer could mislead your node into broadcasting an old, revoked commitment during that cooperative close, thereby cheating you by making your node inadvertently try to "cheat".
Furthermore, the backups of your channels need to be encrypted to maintain your privacy and your channel security. Otherwise, anyone who finds the backups can not only see all your channels, they could use the backups to close all your channels in a way that hands over the balance to your channel peers. In other words, a malicious person that gets access to your backups can cause you to lose all your channel funds.
SCBs are not a panacea. First, the state of each channel needs to be backed up every time there is a new commitment transaction. Second, restoring from a channel backup is dangerous. If you do not have the _last_ commitment transaction and you accidentally broadcast an old (revoked) commitment, your channel peer will assume you are trying to cheat and claim the entire channel balance with a penalty transaction. To make sure you are closing the channel, you need to do a _cooperative close_. But a malicious peer could mislead your node into broadcasting an old, revoked commitment during that cooperative close, thereby cheating you by making your node inadvertently try to cheat.
You can see that SCBs are not a fool-proof safeguard. They are a weak compromise because they swap one type of risk (data corruption or loss) for another type of risk (malicious peer). To restore from a static channel backup, you have to interact with your channel peers and hope they don't try to cheat you by either giving you an old commitment or by fooling your node into broadcasting a revoked commitment so they can penalize you. Despite the weaknesses of SCB, SCBs do make sense and you should perform them. If you do not perform SCBs and you lose your node data, you will lose your channel funds forever. Guaranteed! However, if you _do_ perform SCBs and you lose your node data, then you have a reasonable chance that some of your peers are honest and that you can recuperate some of your channel funds. If you are lucky, you might recover all your funds. In conclusion, it is best for you to perform continuous Static Channel Backups to a disk other than the primary node hard disk.
Furthermore, the backups of your channels need to be encrypted to maintain your privacy and your channel security. Otherwise, anyone who finds the backups can not only see all your channels but also could use the backups to close all your channels in a way that hands over the balance to your channel peers. In other words, a malicious person that gets access to your backups can cause you to lose all your channel funds.
Channel backup mechanisms are still a work-in-progress and a weakness in most Lightning implementations.
You can see that SCBs are not a fool-proof safeguard. They are a weak compromise because they swap one type of risk (data corruption or loss) for another type of risk (malicious peer). To restore from an SCB, you have to interact with your channel peers and hope they don't try to cheat you by either giving you an old commitment or by fooling your node into broadcasting a revoked commitment so they can penalize you. Despite the weaknesses of SCB, SCBs do make sense and you should perform them. If you do not perform SCBs and you lose your node data, you will lose your channel funds forever. Guaranteed! However, if you _do_ perform SCBs and you lose your node data, then you have a reasonable chance that some of your peers are honest and that you can recuperate some of your channel funds. If you are lucky, you might recover all your funds. In conclusion, it is best for you to perform continuous SCBs to a disk other than the primary node hard disk.
==== Static Channel Backups (SCB)
Channel backup mechanisms are still a work in progress and a weakness in most Lightning implementations.
At the time of writing this book, only LND offers a built-in mechanism for static channel backups. Eclair has a similar mechanism deployed for server-side deployments, although Eclair mobile does offer optional backup to a Google Drive. C-lightning recently merged the necessary interfaces for a plug-in to implement channel backups. Unfortunately, there is no consistent, agreed upon backup mechanism across different node implementations.
At the time of writing this book, only LND offers a built-in mechanism for SCBs. Eclair has a similar mechanism deployed for server-side deployments, although Eclair Mobile does offer optional backup to a Google Drive. c-lightning recently merged the necessary interfaces for a plug-in to implement channel backups. Unfortunately, there is no consistent, agreed upon backup mechanism across different node implementations.
File-based backups of the Lightning node databases are at best a partial solution because you run the risk of backing up an inconsistent database state. In addition, you may not reliably catch the latest state commitments. It is much better to have a backup mechanism that is triggered every time there is a state change in a channel, thereby ensuring data consistency.
To set up static channel backups in LND, set the +backupfilepath+ parameter either on the command line or in the configuration file. LND will then save an SCB file in that directory path. Of course, that's only the first step of the solution. Now, you have to setup a mechanism that monitors this file for changes. Each time the file changes the backup mechanism must copy this file to another, preferably "off-site" disk. Such backup mechanisms are beyond the scope of this book. Nonetheless, any sophisticated backup solution should be able to handle this scenario. Recall, the backup files should be encrypted too.
To set up SCBs in LND, set the +backupfilepath+ parameter either on the command line or in the configuration file. LND will then save an SCB file in that directory path. Of course, that's only the first step of the solution. Now, you have to setup a mechanism that monitors this file for changes. Each time the file changes, the backup mechanism must copy this file to another, preferably off-site disk. Such backup mechanisms are beyond the scope of this book. Nonetheless, any sophisticated backup solution should be able to handle this scenario. Recall, the backup files should be encrypted too.
==== Hot Wallet Risk
As we've discussed previously, the Lightning Network consists of a network of _hot wallets_. The funds you store in a Lightning wallet are *online all the time*. This makes them vulnerable. Hence, you should not store large amounts in a Lightning wallet. Large amounts should be kept in a _cold_ wallet that is _not_ online and which can transact only on-chain.
As we've discussed previously, the Lightning Network consists of a network of _hot wallets_. The funds you store in a Lightning wallet are online all the time. This makes them vulnerable. Hence, you should not store large amounts in a Lightning wallet. Large amounts should be kept in a _cold_ wallet that is _not_ online and which can transact only on-chain.
Even if you start small, as time passes you may still find you have a significant amount of money in a Lightning wallet. This is a typical scenario for store owners. If you use a Lightning node for an ecommerce operation, your wallet will likely receive funds often, but send funds rarely. You will therefore end up having two problems simultaneously: First, your channels will be imbalanced with large local balances outweighing small remote balances. Secondly, you will have too much money in the wallet. Fortunately, you can also solve both of these problems simultaneously.
@ -599,13 +594,13 @@ If your Lightning wallet balance becomes too large for your risk tolerance, you
===== On-chain sweep
Sweeping funds on-chain is accomplished by moving the funds from the Lightning wallet to a Bitcoin wallet. You do that by closing channels. When you close a channel, all funds from your local balance are "swept" to a Bitcoin address. The Bitcoin address for on-chain funds is usually generated by your Lightning wallet so it is still a hot-wallet. You may need to do an additional on-chain transaction to move the funds to a more secure address, such as one generated on your hardware wallet.
Sweeping funds on-chain is accomplished by moving the funds from the Lightning wallet to a Bitcoin wallet. You do that by closing channels. When you close a channel, all funds from your local balance are "swept" to a Bitcoin address. The Bitcoin address for on-chain funds is usually generated by your Lightning wallet, so it is still a hot wallet. You may need to do an additional on-chain transaction to move the funds to a more secure address, such as one generated on your hardware wallet.
Closing channels will incur an on-chain fee and will reduce your Lightning node's capacity and connectivity. However, if you run a popular ecommerce node, you will not lack incoming capacity and can strategically close channels with large local balances, essentially "bundling" your funds for movement on-chain. You may need to use some channel rebalancing techniques (see <<channel_rebalancing>>) before you close channels to maximize the benefits of this strategy.
===== Off-chain sweep
Another technique you can use involves running a second Lightning node that is not advertised on the network. You can establish large capacity channels from your public node (e.g. the one running your shop) to your unadvertised (hidden) node. On a regular basis, "sweep" funds by making a Lightning payment to your hidden node.
Another technique you can use involves running a second Lightning node that is not advertised on the network. You can establish large capacity channels from your public node (e.g., the one running your shop) to your unadvertised (hidden) node. On a regular basis, "sweep" funds by making a Lightning payment to your hidden node.
The advantage of this technique lies in the fact that the Lightning node that receives payments for your shop will be publicly known. This makes it a target for hackers, as any Lightning node associated with a shop would be assumed to have a large balance. A second node that is not associated with your shop will not easily be identified as a valuable target.
@ -617,28 +612,26 @@ Keep in mind that this technique does not move funds into cold storage. Both Lig
===== Submarine swap sweep
Another way to reduce your Lightning hot-wallet balance is to use a technique called a _submarine swap_. Submarine swaps, conceptualized by co-author Olaoluwa Osuntokun and Alex Bosworth, allow the exchange of on-chain bitcoin for Lightning payments and vice versa. Essentially, submarine swaps are atomic swaps between Lightning off-chain funds and Bitcoin on-chain funds.
Another way to reduce your Lightning hot-wallet balance is to use a technique called a _submarine swap_. Submarine swaps, conceptualized by coauthor Olaoluwa Osuntokun and Alex Bosworth, allow the exchange of on-chain bitcoin for Lightning payments and vice versa. Essentially, submarine swaps are atomic swaps between Lightning off-chain funds and Bitcoin on-chain funds.
A node operator can initiate a submarine swap and send all available channel balances to the other party who will send them as a result on-chain bitcoin in exchange.
A node operator can initiate a submarine swap and send all available channel balances to the other party, who will send them on-chain bitcoin in exchange.
In the future, this could be a paid service offered by nodes on the Lightning Network who advertise exchange rates or charge a flat fee for the conversion.
The advantage of a submarine swap for sweeping funds is that no channel needs to be closed. That means that we preserve our channels, only rebalancing our channels through this operation. As we send a Lightning payment out, we shift some balance from local to remote on one or more of our channels. Not only does that reduce the balance exposed in our node's hot wallet, it also increases the balance available for future incoming payments.
You could do this by trusting an intermediary to act as a gateway, but this risks your coins being stolen. But in the case of a submarine swap, the operation does not require trust. Submarine swaps are non-custodial _atomic_ operations. That means that the counterparty in your submarine swap cannot steal your funds because the on-chain payment depends on the completion of the off-chain payment and vice-versa.
You could do this by trusting an intermediary to act as a gateway, but this risks your coins being stolen. However, in the case of a submarine swap, the operation does not require trust. Submarine swaps are noncustodial _atomic_ operations. That means that the counterparty in your submarine swap cannot steal your funds because the on-chain payment depends on the completion of the off-chain payment and vice-versa.
===== Submarine swaps with loop
One example of a submarine swap service is _Loop_ by Lightning Labs, the same company that builds LND. Loop comes in two variations: _Loop In_ and _Loop Out_. _Loop In_ accepts a Bitcoin on-chain payment and converts it into a Lightning off-chain payment. _Loop Out_ converts a Lightning payment into a Bitcoin payment.
One example of a submarine swap service is _Loop_ by Lightning Labs, the same company that builds LND. Loop comes in two variations: Loop In and Loop Out. _Loop In_ accepts a Bitcoin on-chain payment and converts it into a Lightning off-chain payment. _Loop Out_ converts a Lightning payment into a Bitcoin payment.
[NOTE]
====
To use the _Loop_ service you must be running an LND Lightning node.
To use the Loop service you must be running an LND Lightning node.
====
For the purpose of reducing the balance of your Lightning hot wallet, you would use the Loop Out service. To use the Loop service, you need to install some additional software on your node. The Loop software runs alongside your LND node and provides some command line tools to execute submarine swaps. You can find the Loop software and installation instructions here:
https://github.com/lightninglabs/loop
For the purpose of reducing the balance of your Lightning hot wallet, you would use the Loop Out service. To use the Loop service, you need to install some additional software on your node. The Loop software runs alongside your LND node and provides some command-line tools to execute submarine swaps. You can find the Loop software and installation instructions on https://github.com/lightninglabs/loop[GitHub].
Once you have the software installed and running, a Loop Out operation is as simple as running a single command:
@ -664,23 +657,23 @@ Note that your maximum fee, which represents a worst-case scenario, will depend
=== Lightning Node Uptime and Availability
Unlike Bitcoin, Lightning nodes need to be online almost continuously. Your node needs to be online to receive payments, open channels, close channels (cooperatively), and monitor protocol violations. Node availability is such an important requirement in the Lightning Network that it is a metric used by various automatic channel management tools (e.g. +autopilot+) to decide which nodes to open channels with. You can also see "availability" as a node metric on popular node explorers (see <<ln_explorer>>) such as +1ml.com+.
Unlike Bitcoin, Lightning nodes need to be online almost continuously. Your node needs to be online to receive payments, open channels, close channels (cooperatively), and monitor protocol violations. Node availability is such an important requirement in the Lightning Network that it is a metric used by various automatic channel management tools (e.g., +autopilot+) to decide which nodes to open channels with. You can also see "availability" as a node metric on popular node explorers (see <<ln_explorer>>) such as https://1ml.com/[1ML].
Node availability is especially important to mitigate and resolve potential protocol violations (i.e. revoked commitments). While you can afford short interruptions from an hour up to one or two days, you cannot have your node offline for longer periods of time without risking loss of funds.
Node availability is especially important to mitigate and resolve potential protocol violations (i.e., revoked commitments). While you can afford short interruptions from an hour up to one or two days, you cannot have your node offline for longer periods of time without risking loss of funds.
Keeping a node online continuously is not easy, as various bugs and resource limitations can and will occasionally cause downtime. Especially if you run a busy and popular node, you will run into limitations of memory, swap space, number of open files, disk space, and so forth. A whole host of different problems will cause your node or your server to crash.
==== Tolerate Faults and Automate
If you have the time and skills you should test some basic fault scenarios on the Lightning testnet. On the testnet you will learn valuable lessons without risking any funds. Any step you perform to automate your system will improve your availability.
If you have the time and skills, you should test some basic fault scenarios on the Lightning testnet. On the testnet you will learn valuable lessons without risking any funds. Any step you perform to automate your system will improve your availability:
- Automatic computer server restart: What happens when your server or the operating system crashes? What happens when there is a power outage? Simulate this fault by pressing the "reset" button on your PC or by unplugging the power cable. After crash, reset, or power failure the computer should automatically restart itself. Some computers have a setting in their BIOS to specify how the computer should react on power failures. Test it to make sure the computer really restarts automatically on power failure without human intervention.
Automatic computer server restart:: What happens when your server or the operating system crashes? What happens when there is a power outage? Simulate this fault by pressing the "reset" button on your PC or by unplugging the power cable. After a crash, reset, or power failure the computer should automatically restart itself. Some computers have a setting in their BIOS to specify how the computer should react on power failures. Test it to make sure the computer really restarts automatically on power failure without human intervention.
- Automatic node restart: What happens when your node or one of your nodes crashes? Simulate this fault by killing the corresponding node processes. If a node crashes, it should automatically restart itself. Test it to make sure the node or nodes really restart automatically on failure without human intervention. If this is not the case, most likely your node is not set up correctly as an operating system service.
Automatic node restart:: What happens when your node or one of your nodes crashes? Simulate this fault by killing the corresponding node processes. If a node crashes, it should automatically restart itself. Test it to make sure the node or nodes really restart automatically on failure without human intervention. If this is not the case, most likely your node is not set up correctly as an operating system service.
- Automatic network reconnection: What happens if your network goes down? What happens when your ISP goes temporarily down? What happens when your ISP assigns a new IP address to your router or your computer? When the network comes back, do the nodes you are running automatically reconnect to the network? Simulate this fault by unplugging and later replugging the Ethernet cable from the device hosting your nodes. The nodes should automatically reconnect and continue operation without human intervention.
Automatic network reconnection:: What happens if your network goes down? What happens when your ISP goes down temporarily? What happens when your ISP assigns a new IP address to your router or your computer? When the network comes back, do the nodes you are running automatically reconnect to the network? Simulate this fault by unplugging and later replugging the Ethernet cable from the device hosting your nodes. The nodes should automatically reconnect and continue operation without human intervention.
- Configure your log files: All of the above failures should leave textual entries behind in the corresponding logfiles. Turn up the verbosity of logging if needed. Find these error entries in the logfiles and use them for monitoring.
Configure your logfiles:: All of the preceding failures should leave textual entries behind in the corresponding logfiles. Turn up the verbosity of logging if needed. Find these error entries in the logfiles and use them for monitoring.
==== Monitoring Node Availability
@ -688,11 +681,9 @@ Monitoring your node is an important part of keeping it running. You need to mon
There are a number of ways to do this, but most require some customization. You can use generic infrastructure monitoring or application monitoring tools, but you have to customize them specifically to query the Lightning node API to ensure the node is running, synchronized to the blockchain, and connected to channel peers.
There is a specialized service that offers Lightning node monitoring. It uses a Telegram bot to notify you of any interruptions in service. This is a free service, though you can pay (over Lightning of course) to get faster alerts. Find more information at:
https://lightning.watch
https://lightning.watch[Lightning.Watch] provides a specialized service that offers Lightning node monitoring. It uses a Telegram bot to notify you of any interruptions in service. This is a free service, though you can pay (over Lightning, of course) to get faster alerts.
Over time, we expect more third-party services to provide specialized Lightning node monitoring payable via micro-payments. Perhaps such services and their APIs will become standardized and will one day be directly supported by Lightning node software.
Over time, we expect more third-party services to provide specialized Lightning node monitoring payable via micropayments. Perhaps such services and their APIs will become standardized and will one day be directly supported by Lightning node software.
[[watchtowers]]
==== Watchtowers
@ -701,16 +692,17 @@ _Watchtowers_ are a mechanism for outsourcing the monitoring and penalty resolut
As we mentioned in previous chapters, the Lightning protocol maintains security through a penalty mechanism. If one of your channel partners broadcasts an old commitment transaction, your node will need to exercise the revocation clause and broadcast a penalty transaction to avoid losing money. But if your node is down during the protocol violation, you might lose money.
To solve this problem, we can use one or more _watchtowers_ to outsource the job of monitoring protocol violations and issuing penalty transactions. There are two parts to a watchtower setup: a watchtower server (or simply "watchtower") that monitors the blockchain, and a watchtower client that asks the watchtower server for this monitoring service.
To solve this problem, we can use one or more watchtowers to outsource the job of monitoring protocol violations and issuing penalty transactions. There are two parts to a watchtower setup: a watchtower server (or simply watchtower) that monitors the blockchain and a watchtower client that asks the watchtower server for this monitoring service.
Watchtower technology is still in the early stages of development and is not widely supported. However, below we list some experimental implementations that you can try.
Watchtower technology is still in the early stages of development and is not widely supported. However, in the following passage we list some experimental implementations that you can try.
LND software includes both a watchtower server and a watchtower client. You can activate the watchtower server by adding the following configuration options:
You can use LND's watchtower client by activating it in the configuration and then using the command line to connect it to a watchtower server. The configuration is:
@ -720,7 +712,7 @@ You can use LND's watchtower client by activating it in the configuration and th
wtclient.active=1
----
LND's commandline client +lncli+ shows the following options for managing the watchtower client:
LND's command-line client +lncli+ shows the following options for managing the watchtower client:
----
$ lncli wtclient
@ -743,11 +735,9 @@ OPTIONS:
--help, -h show help
----
C-lightning has the API hooks necessary for a watchtower client plug-in, though no such plug-in has been implemented yet.
c-lightning has the API hooks necessary for a watchtower client plug-in, though no such plug-in has been implemented yet.
Finally, a popular standalone watchtower server is _The Eye of Satoshi_ (TEOS). It can be found here:
https://github.com/talaia-labs/python-teos
Finally, a popular standalone watchtower server is _The Eye of Satoshi_ (TEOS). It can be found on https://github.com/talaia-labs/python-teos[GitHub].
=== Channel Management
@ -757,30 +747,30 @@ As a Lightning node operator, one of the recurring tasks you will need to perfor
As soon as you get your Lightning node up and running, you can fund its Bitcoin wallet and then start opening channels with those funds.
You must choose channel partners carefully as your node's ability to send payments depends on who your channel partners are and how well connected they are to the rest of the Lightning Network. You also want to have more than one channel to avoid being susceptible to a single point of failure. Since Lightning now supports multipath payments, you can split your initial funds into several channels and route bigger payments by combining their capacity. At the same time, avoid making your channels too small. Since you need to pay Bitcoin transaction fees to open and close a channel, the channel balance should not be so small that the on-chain fees consume a significant portion. It's all about balance!
You must choose channel partners carefully because your node's ability to send payments depends on who your channel partners are and how well connected they are to the rest of the Lightning Network. You also want to have more than one channel to avoid being susceptible to a single point of failure. Since Lightning now supports multipath payments, you can split your initial funds into several channels and route bigger payments by combining their capacity. At the same time, avoid making your channels too small. Since you need to pay Bitcoin transaction fees to open and close a channel, the channel balance should not be so small that the on-chain fees consume a significant portion. It's all about balance!
To summarize:
* Connect to a few well connected nodes.
* Open more than one channel.
* Don't open too many channels.
* Don't make the channels too small.
* Connect to a few well-connected nodes
* Open more than one channel
* Don't open too many channels
* Don't make the channels too small
One way to find well connected nodes is to open a channel to a popular merchant selling products on the Lightning Network. These nodes tend to be well funded and well connected. So, when you are ready to buy something online via Lightning, you can open a channel directly to the merchant's node. The merchant's node ID will be in the invoice you will receive when you try to buy something. That makes it easy.
Another way to find well-connected nodes is to use a Lightning Explorer (see <<ln_explorer>>) such as +1ml.com+ and browse the list of nodes sorted by channel capacity and number of channels. Don't go for the biggest nodes as that encourages centralization. Go for a node in the middle of the list so that you can help them grow. Another factor to consider might be the time span a node has been in operation. Nodes established for more than a year are likely to be more reputable and less risky than nodes that started operation a week ago.
Another way to find well-connected nodes is to use a Lightning Explorer (see <<ln_explorer>>) such as https://1ml.com/[1ML] and browse the list of nodes sorted by channel capacity and number of channels. Don't go for the biggest nodes,because that encourages centralization. Go for a node in the middle of the list so that you can help them grow. Another factor to consider might be the time span a node has been in operation. Nodes established for more than a year are likely to be more reputable and less risky than nodes that started operation a week ago.
[[autopilot]]
===== Autopilot
The task of opening channels can be partially automated with the use of an _autopilot_, which is software that opens channels automatically based on some heuristic rules. Autopilot software is still relatively new and it doesn't always select the best channel partners for you. Especially in the beginning, it might be better to open channels manually.
Autopilots currently exist in 3 forms.
The task of opening channels can be partially automated with the use of an _autopilot_, which is software that opens channels automatically based on some heuristic rules. Autopilot software is still relatively new, and it doesn't always select the best channel partners for you. Especially in the beginning, it might be better to open channels manually.
Autopilots currently exist in three forms:
- +lnd+ incorporates an autopilot that is fully integrated with +lnd+ and runs constantly in the background while turned on.
- +lib_autopilot.py+ can offer autopilot computations for any node implementation based on the gossip and channel data.
- A +c-lightning+ plug-in based on +lib_autopilot.py+ exists that provides an easy to use interface for +c-lightning+ users.
- A +c-lightning+ plug-in based on +lib_autopilot.py+ exists that provides an easy-to-use interface for +c-lightning+ users.
Be aware that the +lnd+ autopilot will start running in the background as soon as it is turned on via the config file. As a result it will start opening channels immediately if you have onchain outputs in your +lnd+ wallet.
Be aware that the +lnd+ autopilot will start running in the background as soon as it is turned on via the config file. As a result it will start opening channels immediately if you have on-chain outputs in your +lnd+ wallet.
If you want to have full control over the bitcoin transactions that you make and the channels that you open, make sure to turn the autopilot off _before_ you load your +lnd+ wallet with bitcoin funds.
If the autopilot was previously turned on, you might have to restart your +lnd+ before you top up your wallet with an on-chain transaction or before you close channels, which effectively gives you on-chain funds again.
It is crucial that you set key configuration values if you want to run the autopilot.
This configuration file would activate the autopilot.
It would open channels as long as the following two conditions are met:
1. Your node has currently less than 40 channels open.
1. Your node currently has less than 40 channels open.
2. Less than 70% of your total funds are off-chain in payment channels.
The numbers 40 and 0.7 are chosen completely arbitrarily here as we cannot make any recommendations that are valid for everyone about how many channels one should have open and what percentage of one's funds should be offchain.
The numbers 40 and 0.7 are chosen completely arbitrarily here because we cannot make any recommendations that are valid for everyone about how many channels one should have open and what percentage of one's funds should be off-chain.
The autopilot in +lnd+ will not take into account on-chain fees. In other words, it will not delay opening channels to a time period when fees are low.
To reduce fees, you can manually open channels during a time period when fees are low, e.g. during the weekend.
To reduce fees, you can manually open channels during a time period when fees are low, e.g., during the weekend.
The autopilot will make channel recommendations whenever the conditions are met and will immediately try to open a channel by using the appropriate current fees.
According to the above configuration file, the channels will be between 5 mBTC (minchansize=500000 satoshi) and 50 mBTC (maxchansize=5000000 satoshi) in size.
According to the preceding configuration file, the channels will be between 5 mBTC (minchansize=500,000 satoshi) and 50 mBTC (maxchansize=5,000,000 satoshi) in size.
As is common, the amounts in the configuration file are enumerated in satoshi.
Currently channels below 1 mBTC are not very useful and we do not recommend you open channels that are too small and below this amount.
Currently, channels below 1 mBTC are not very useful, and we do not recommend you open channels that are too small and below this amount.
With the wider adoption of multipath payments, smaller channels are less of a burden. But for the time being, this is our recommendation.
The +c-lightning+ plug-in which was originally written by René Pickhardt works very differently in comparison to the +lnd+ autopilot.
The +c-lightning+ plug-in, which was originally written by René Pickhardt (a coauthor of this book), works very differently in comparison with the +lnd+ autopilot.
First, it differs in the algorithms used to make the recommendations. We will not cover this here. Secondly, it differs in its user interface.
You will need to download the _autopilot plug-in_ from the +c-lightning+ plug-in repository at https://github.com/lightningd/plugins/tree/master/autopilot and activate it.
You will need to download the _autopilot plug-in_ from the +c-lightning+ plug-in https://github.com/lightningd/plugins/tree/master/autopilot[repository] and activate it.
[NOTE]
====
In order to activate a plug-in in +c-lightning+, place it into the `~/.lightning/plugins` directory, ensure that it's executable (e.g. `chmod +x ~/.lightning/plugins/autopilot.py`), then restart +lightningd+.
To activate a plug-in in +c-lightning+, place it into the _~/.lightning/plugins_ directory, ensure that it's executable (e.g., `chmod +x ~/.lightning/plugins/autopilot.py`), then restart +lightningd+.
Alternatively, if you don't want a plug-in to automatically activate when you start +lightningd+ you can place it in a different directory and manually activate it with the +plugin+ argument to +lightningd+:
Alternatively, if you don't want a plug-in to automatically activate when you start +lightningd+, you can place it in a different directory and manually activate it with the +plugin+ argument to +lightningd+:
@ -828,7 +818,7 @@ Alternatively, if you don't want a plug-in to automatically activate when you st
====
The autopilot in +c-lightning+ is controlled via 3 configuration values which can be set in the config file or as command line arguments when you start +lightningd+.
The autopilot in +c-lightning+ is controlled via three configuration values that can be set in the config file or as command-line arguments when you start +lightningd+.
----
[c-lightning-autopilot]
@ -837,7 +827,7 @@ autopilot-num-channels=10
autopilot-min-channel-size-msat=100000000msat
----
These values are the actual default config and you do not need to set them at all.
These values are the actual default config, and you do not need to set them at all.
The autopilot will not automatically run in the background like in +lnd+.
Instead, you have to start a run specifically with `lightning-cli autopilot-run-once` if you want the autopilot to open the recommended channels.
@ -867,22 +857,21 @@ There are three typical ways of getting inbound liquidity:
* Ask someone to open a channel to your node. Offer to reciprocate, so that both of your nodes become better connected and balanced.
* Use a _submarine swap_ (e.g. +Loop-In+) to exchange on-chain BTC for an inbound channel to your node.
* Use a submarine swap (e.g., +Loop-In+) to exchange on-chain BTC for an inbound channel to your node.
* Pay a third party service to open a channel with you. Several such services exist. Some charge a fee to provide liquidity, some are free.
Here is a list of currently available liquidity providers who will open a channel to your node for a fee:
* Bitrefill's _Thor_ service at https://www.bitrefill.com/thor-lightning-network-channels/
Creating inbound liquidity is challenging from both a practical and user experience perspective. Inbound liquidity does not happen automatically, so you have to find ways to build it for your node. This asymmetry of payment channels is also not intuitive. In most other payment systems you get paid first (inbound) before you pay others (outbound).
Creating inbound liquidity is challenging from both practical and user experience perspectives. Inbound liquidity does not happen automatically, so you have to find ways to build it for your node. This asymmetry of payment channels is also not intuitive. In most other payment systems you get paid first (inbound) before you pay others (outbound).
The challenge of creating inbound liquidity is most noticeable if you are a merchant or sell your services for Lightning payments. In that case, you need to be vigilant to ensure that you have enough inbound liquidity to be able to continue to receive payments. What if there is a surge of buyers on your store, but they can't actually pay you because there is no more inbound capacity?
@ -907,7 +896,7 @@ In the course of transacting and routing payments on Lightning, the combination
For example, if one of your channel partners is frequently routing payments through your node, you will exhaust the inbound capacity on that channel, while also exhausting the outbound capacity on the outgoing channels. Once that happens, you can no longer route payments through that route.
There are many ways to rebalance channels, each with different advantages and disadvantages. One way is to use a _submarine swap_ (e.g. +Loop-Out+) as described previously in this chapter. Another way to rebalance is to simply wait for routed payments that flow in the opposite direction. If your node is well connected, when a specific route becomes exhausted in one direction, the same route becomes available in the opposite direction. Other nodes may "discover" that route in the opposite direction and start using it as part of their payment path, thereby rebalancing the funds again.
There are many ways to rebalance channels, each with different advantages and disadvantages. One way is to use a submarine swap (e.g., +Loop-Out+) as described previously in this chapter. Another way to rebalance is to simply wait for routed payments that flow in the opposite direction. If your node is well connected, when a specific route becomes exhausted in one direction, the same route becomes available in the opposite direction. Other nodes may "discover" that route in the opposite direction and start using it as part of their payment path, thereby rebalancing the funds again.
A third way to rebalance channels is to purposefully create a _circular route_ that sends a payment from your node back to your node, via the Lightning Network. By sending a payment out on a channel with large local capacity and arranging the path so that it returns to your node on a channel with large remote capacity, both of those channels will become more balanced. An example of a circular route rebalancing strategy can be seen in <<circular_rebalancing>>.
@ -915,22 +904,22 @@ A third way to rebalance channels is to purposefully create a _circular route_ t
.Circular route rebalancing
image::images/mtln_0504.png[]
Circular rebalancing is supported by most Lightning node implementations and can be done on the command line or via one of the web management interfaces such as _Ride the Lightning (RTL)_ (see <<rtl>>).
Circular rebalancing is supported by most Lightning node implementations and can be done on the command line or via one of the web management interfaces such as Ride The Lightning (see <<rtl>>).
Channel rebalancing is a complex issue that is the subject of active research and covered in more detail in <<channel_rebalancing>>.
=== Routing Fees
Running a Lightning node allows you to earn fees by routing payments across your channels. Routing fees are generally not a significant source of income and dwarfed by the cost of operating a node. For example, on a relatively busy node that routes a dozen payments a day, the fees amount to no more than 2000 satoshis.
Running a Lightning node allows you to earn fees by routing payments across your channels. Routing fees are generally not a significant source of income and dwarfed by the cost of operating a node. For example, on a relatively busy node that routes a dozen payments a day, the fees amount to no more than 2,000 satoshis.
Nodes compete for routing fees by setting their desired fee rate on each channel. Routing fees are set by two parameters on each channel: a fixed _base fee_ that is charged for any payment and an additional variable _fee rate_ that is proportional to the payment amount.
When sending a Lightning payment, a node will select a path so as to minimize fees, minimize hops, or both. As a result, a routing fee market emerges from these interactions. There are currently many nodes that charge very low or no fees for routing, creating downward pressure on the routing fee "market".
When sending a Lightning payment, a node will select a path so as to minimize fees, minimize hops, or both. As a result, a routing fee market emerges from these interactions. There are currently many nodes that charge very low or no fees for routing, creating downward pressure on the routing fee market.
If you make no choices, your Lightning node will set a default base fee and fee rate for each new channel. The default values depend on the node implementation you use.
The _base fee_ is set in the unit of _millisatoshi_ (thousandths of a satoshi). The proportional _fee rate_ is set in the unit of _millionths_ and is applied to the payment amount.
The unit of _millionths_ is often abbreviated with _ppm_ (parts per million).
For example, a _base fee_ of 1,000 (millisatoshi) and a _fee rate_ of 1,000 ppm (millionths) would result in the following charges for a 100,000 satoshi payment:
The base fee is set in the unit of _millisatoshi_ (thousandths of a satoshi). The proportional fee rate is set in the unit of _millionths_ and is applied to the payment amount.
The unit of millionths is often abbreviated with _ppm_ (parts per million).
For example, a base fee of 1,000 (millisatoshi) and a fee rate of 1,000 ppm (millionths) would result in the following charges for a 100,000 satoshi payment:
[latexmath]
++++
@ -950,38 +939,32 @@ Broadly speaking, you can take one of two approaches to routing fees. You can ro
For most nodes, it is usually best to use the default routing fee values. This way, your node is competing on a mostly level playing field with other nodes who use the default values.
You can also use the routing fee settings to rebalance channels. If most of your channels have the default fees but you want to rebalance a particular channel, just decrease the fees on that specific channel to zero or to very low rates. Then sit back and wait for someone to route a payment over your "cheap" route and rebalance your channels for you as a side-effect.
You can also use the routing fee settings to rebalance channels. If most of your channels have the default fees but you want to rebalance a particular channel, just decrease the fees on that specific channel to zero or to very low rates. Then sit back and wait for someone to route a payment over your "cheap" route and rebalance your channels for you as a sideeffect.
=== Node Management
Managing your Lightning node on the command line is obviously not easy. It gives you the full flexibility of the node's API and the ability to write your own custom scripts to satisfy your personal requirements. But if you don't want to deal with the complexity of the command line and only need some basic node management capabilities, you should consider installing a web-based user interface that makes node management easier.
There are a number of competing projects that offer web-based Lightning node management. Some of the most popular ones are described below.
There are a number of competing projects that offer web-based Lightning node management. Some of the most popular ones are described in the following section.
[[rtl]]
==== Ride the Lightning (RTL)
==== Ride The Lightning
Ride The Lightning (RTL) is a graphical web user interface to help users manage Lightning node operations for the three main Lightning node implementations (LND, c-lightning, and Eclair). RTL is an open source project developed by Shahana Farooqui and many other contributors. You can find the RTL software on https://github.com/Ride-The-Lightning/RTL[GitHub].
_RTL_ is a graphical web user interface to help users manage Lightning node operations for the three main Lightning node implementations (LND, c-lightning, and Eclair). RTL is an open source project developed by Suheb, Shahana Farooqui, and many other contributors. You can find the RTL software here:
https://github.com/Ride-The-Lightning/RTL
Here is an example screenshot of RTL's web interface, as provided on the project repository:
<<rtl-web-interface>> shows an example screenshot of RTL's web interface, as provided on the project repository.
[[rtl-web-interface]]
.Example RTL web interface
image::images/mtln_0505.png[]
==== LNDMon
Lightning Labs, the makers of LND, provide a web-based graphical user interface called +lndmon+ to monitor the various metrics of an LND Lightning node. +lndmon+ only works with LND nodes. It is a read-only interface for monitoring and as such does not allow you to actively manage the node. It cannot open channels or make payments. Find +lndmon+ here:
https://github.com/lightninglabs/lndmon
Lightning Labs, the makers of LND, provide a web-based graphical user interface called +lndmon+ to monitor the various metrics of an LND Lightning node. +lndmon+ only works with LND nodes. It is a read-only interface for monitoring and as such does not allow you to actively manage the node. It cannot open channels or make payments. Find +lndmon+ on https://github.com/lightninglabs/lndmon[GitHub].
==== ThunderHub
A very aesthetically pleasing web-based graphical user interface similar to RTL but exclusive to LND. It can be used to make payments, rebalance channels and manage the node through a variety of features. Find +ThunderHub+ here:
https://thunderhub.io
https://thunderhub.io[ThunderHub] is a very aesthetically pleasing web-based graphical user interface similar to RTL but exclusive to LND. It can be used to make payments, rebalance channels and manage the node through a variety of features.
In the first part of this book we introduced the main concepts of the Lightning Network, worked through a comprehensive example of routing a payment and setting up the tools we can use to explore further. In the second part of the book we will explore the Lightning Network in a lot more technical detail, dissecting each of the building blocks.
In the first part of this book we introduced the main concepts of the Lightning Network and worked through a comprehensive example of routing a payment and setting up the tools we can use to explore further. In the second part of the book we will explore the Lightning Network in a lot more technical detail, dissecting each of the building blocks.
In this section we will outline the components of the Lightning Network in more detail and provide a "big picture" perspective to guide you through the following chapters.
=== The Lightning Network Protocol Suite
The Lightning Network is composed of a complex collection of protocols that run on top of the internet. We can broadly classify these protocols into five distinct layers that make up a _protocol stack_, where each layer build upon and uses the protocols in the layer below. Also, each protocol layer abstracts the underlying layers and "hides" some of the complexity.
The Lightning Network is composed of a complex collection of protocols that run on top of the internet. We can broadly classify these protocols into five distinct layers that make up a _protocol stack_, where each layer builds upon and uses the protocols in the layer below. Also, each protocol layer abstracts the underlying layers and "hides" some of the complexity.
The architecture diagram shown in <<lightning_network_protocol_suite>> provides an overview of these layers and their component protocols:
The architecture diagram shown in <<lightning_network_protocol_suite>> provides an overview of these layers and their component protocols.
[[lightning_network_protocol_suite]]
.The Lightning Network Protocol Suite
.The Lightning Network protocol suite
image::images/mtln_0601.png[]
The five layers of the Lightning Network, from the bottom up, are:
Network Connection Layer:: This contains the protocols that interact directly with the internet core protocols (TCP/IP), overlay protocols (Tor v2/v3), and internet services (DNS). This layer also contains the cryptographic transport protocols that protect Lightning messages.
Network connection layer:: This contains the protocols that interact directly with the internet core protocols (TCP/IP), overlay protocols (Tor v2/v3), and internet services (DNS). This layer also contains the cryptographic transport protocols that protect Lightning messages.
Messaging Layer:: This layer contains the protocols that nodes use to negotiate features, format messages, and encode message fields.
Messaging layer:: This layer contains the protocols that nodes use to negotiate features, format messages, and encode message fields.
Peer-2-Peer (P2P) Layer:: This layer is primary protocol layer for communication between Lightning nodes and contains all the different messages exchanged between nodes.
Peer-to-Peer (P2P) layer:: This layer is the primary protocol layer for communication between Lightning nodes and contains all the different messages exchanged between nodes.
Routing Layer:: This layer contains the protocols used to route payments between nodes, end-to-end and atomically. This layer contains the "core" functionality of the Lightning Network: routed payments.
Routing layer:: This layer contains the protocols used to route payments between nodes, end-to-end and atomically. This layer contains the core functionality of the Lightning Network: routed payments.
Payment Layer:: The highest layer of the network, which presents a reliable payment interface to applications.
Payment layer:: The highest layer of the network, which presents a reliable payment interface to applications.
=== Lightning in Detail
Over the next 10 chapters, we will diseect the protocol suite and examine each component of the Lightning Network in detail.
We spent quite some time trying to decide the best order of presenting this detail. It's not an easy choice as there is so much interdependence between different components: as you start explaining one, you find that it pulls in quite a few of the other componenents. Instead of a top-down or bottom-up approach, we ended up choosing a more meandering path that starts with the most fundamental building blocks that are unique to the Lightning Network-Payment Channels - and moves outwards from there. But since that path is not obvious, we will use the Lightning Protocol Suite shown in <<lightning_network_protocol_suite>> as a map. In each chapter will focus on one or more related components, and you will see them highlighted in the protocol suite. Kind of like a map marker saying "You are here!".
We spent quite some time trying to decide the best order of presenting this detail. It's not an easy choice because there is so much interdependence between different components: as you start explaining one, you find that it pulls in quite a few of the other componenents. Instead of a top-down or bottom-up approach, we ended up choosing a more meandering path that starts with the most fundamental building blocks that are unique to the Lightning Network-Payment Channels and moves outward from there. But since that path is not obvious, we will use the Lightning Protocol Suite shown in <<lightning_network_protocol_suite>> as a map. In each chapter will focus on one or more related components, and you will see them highlighted in the protocol suite. Kind of like a map marker saying "You are here!".
Here's what we will cover:
@ -38,17 +38,17 @@ Here's what we will cover:
<<routing>>:: Next, we will put together several payment channels in a network and route a payment from one end to the other. In that process we will dive into the Hash Time-Locked Contract smart contract and the Bitcoin Script that we use to construct it.
<<channel_operation>>:: Putting together the concepts of a simple payment channel and a routed payment using HTLCs, we will now look at how HTLCs are part of each channel's commitment transaction. We will also look at the protocol for adding, settling, failing and removing HTLCs from the commitments.
<<channel_operation>>:: Putting together the concepts of a simple payment channel and a routed payment using HTLCs, we will now look at how HTLCs are part of each channel's commitment transaction. We will also look at the protocol for adding, settling, failing, and removing HTLCs from the commitments.
<<onion_routing>>:: Next, we will look at how the HTLC information is propagated across the network inside the onion routing protocol. We will look at the mechanism for layered encryption and decryption that gives the Lightning Network some of its privacy characteristics.
<<gossip>>:: In this chapter we will look at how Lightning nodes find each other and learn about published channels, in order to construct a channel graph that they can use to find paths across the network.
<<gossip>>:: In this chapter we will look at how Lightning nodes find each other and learn about published channels to construct a channel graph that they can use to find paths across the network.
<<path_finding>>:: Next, we will see how the information from the gossip protocol is used by each node to build a "map" of the entire network, which it can use to find paths from one point to another to route payments. We'll also look at the exiting innovations in path finding such as multi-part payments.
<<path_finding>>:: Next, we will see how the information from the gossip protocol is used by each node to build a "map" of the entire network, which it can use to find paths from one point to another to route payments. We'll also look at the exiting innovations in pathfinding such as Multi-Part Payments.
<<invoices>>:: A key part of the Lightning Network are payment requests, also known as Lighting Invoices. In this chapter we dissect the structure and encoding of an invoice.
<<invoices>>:: A key part of the Lightning Network is payment requests, also known as Lightning Invoices. In this chapter we dissect the structure and encoding of an invoice.
<<wire_protocol>>:: Underpinning the Lightning Network is the Peer-to-Peer protocol that nodes use to exchange messages about the network and about their channels. In this chapter we look at how those messages are constructed and the extension capabilities built into messages with feature bits and TLV encoding.
<<wire_protocol>>:: Underpinning the Lightning Network is the peer-to-peer protocol that nodes use to exchange messages about the network and about their channels. In this chapter we look at how those messages are constructed and the extension capabilities built into messages with feature bits and Type-Length-Value (TLV) encoding.
<<encrypted_message_transport>>:: Moving down to the lower-level part of the network, we will look at the underlying encrypted transport system that ensures the secrecy and integrity of all communications between nodes.
In this chapter we will dive into payment channels and see how they are constructed. We will start with Alice's node opening a channel to Bob's node, building on the examples presented in the beginning of this book.
The messages exchanged by Alice and Bob's nodes are defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md["BOLT2-Peer Protocol"]. The transactions created by Alice and Bob's nodes are defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md["BOLT3-Transactions"]. In this chapter we are focusing on the "Channel Open & Close" and "Channel State Machine" parts of the Lightning protocol architecture, highlighted by a double outline in the center (Peer 2 Peer Layer) of <<LN_protocol_channel_highlight>>:
The messages exchanged by Alice and Bob's nodes are defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md["BOLT2-Peer Protocol."] The transactions created by Alice and Bob's nodes are defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md["BOLT3-Transactions."] In this chapter we are focusing on the "Channel Open & Close" and "Channel State Machine" parts of the Lightning protocol architecture, highlighted by a double outline in the center (Peer 2 Peer Layer) of <<LN_protocol_channel_highlight>>.
The Lightning Network is often described as a "Layer 2 Bitcoin Protocol", which makes it sound distinct from Bitcoin. Another way to describe Lightning is as a "smarter way to use Bitcoin" or just as an "Application on top of Bitcoin". Let's explore that.
The Lightning Network is often described as a "Layer 2 Bitcoin Protocol," which makes it sound distinct from Bitcoin. Another way to describe Lightning is as a "smarter way to use Bitcoin" or just as an "application on top of Bitcoin." Let's explore that.
Historically, Bitcoin transactions are broadcast to everyone and recorded on the Bitcoin blockchain in order to be considered "valid". As we will see however, if someone holds a pre-signed Bitcoin transaction that spends a 2-of-2 multisig output that gives them the exclusive ability to spend that Bitcoin, they effectively own that Bitcoin even if they don't broadcast the transaction.
Historically, Bitcoin transactions are broadcast to everyone and recorded on the Bitcoin blockchain to be considered valid. As we will see, however, if someone holds a pre-signed Bitcoin transaction that spends a 2-of-2 multisig output that gives them the exclusive ability to spend that Bitcoin, they effectively own that Bitcoin even if they don't broadcast the transaction.
You can think of the pre-signed Bitcoin transaction like a post-dated check (or cheque), one that can be "cashed" at anytime. Unlike the traditional banking system however, this transaction is not a "promise" of payment (also known as an IOU), but a verifiable bearer instrument that is equivalent to cash. So long as the bitcoin referenced in the transaction has not already been spent at the time of redemption (or at the time you try to "cash" the cheque), the Bitcoin system guarantees that this pre-signed transaction can be broadcast and recorded at any time. This is only true, of course, if this is the only pre-signed transaction. Within the Lightning Network two or more such pre-signed transactions exist at the same time, therefore we need a more sophisticated mechanism to still have the functionality of such a verifiable bearer instrument, as you will also learn in this chapter.
You can think of the pre-signed Bitcoin transaction like a post-dated check (or cheque), one that can be cashed at anytime. Unlike the traditional banking system, however, this transaction is not a "promise" of payment (also known as an IOU), but a verifiable bearer instrument that is equivalent to cash. So long as the bitcoin referenced in the transaction has not already been spent at the time of redemption (or at the time you try to "cash" the check), the Bitcoin system guarantees that this pre-signed transaction can be broadcast and recorded at any time. This is only true, of course, if this is the only pre-signed transaction. Within the Lightning Network two or more such pre-signed transactions exist at the same time; therefore, we need a more sophisticated mechanism to still have the functionality of such a verifiable bearer instrument, as you will also learn in this chapter.
The Lightning Network is simply a different and creative way of using Bitcoin. In the Lightning Network a combination of recorded (on-chain) and pre-signed but withheld (off-chain) transactions form a "layer" of payments that is a faster, cheaper and more private way to use Bitcoin. You can see this relationship between on-chain and off-chain Bitcoin transactions in <<on_off_chain>>:
The Lightning Network is simply a different and creative way of using Bitcoin. In the Lightning Network a combination of recorded (on-chain) and pre-signed but withheld (off-chain) transactions form a "layer" of payments that is a faster, cheaper, and more private way to use Bitcoin. You can see this relationship between on-chain and off-chain Bitcoin transactions in <<on_off_chain>>.
[[on_off_chain]]
.Lightning payment channel made of on-chain and off-chain transactions
@ -29,49 +29,49 @@ Lightning is Bitcoin. It's just a different way of using the Bitcoin system.
Before we understand payment channels, we have to take a small step back and understand how ownership and control work in Bitcoin.
When someone says they "own" bitcoin they typically mean that they know the private key of a Bitcoin address that has some unspent transaction outputs (see <<bitcoin_fundamentals_review>>). The private key allows them to sign a transaction to spend that bitcoin by transferring it to a different address. In Bitcoin "ownership" of bitcoin can be defined as the _ability to spend_ that bitcoin.
When someone says they "own" bitcoin, they typically mean that they know the private key of a Bitcoin address that has some unspent transaction outputs (see <<bitcoin_fundamentals_review>>). The private key allows them to sign a transaction to spend that bitcoin by transferring it to a different address. In Bitcoin "ownership" of bitcoin can be defined as the _ability to spend_ that bitcoin.
Keep in mind that the term "ownership" as used in Bitcoin is distinct from the term "ownership" used in a legal sense. A thief who has the private keys and can spend Bitcoin is a _de-facto owner_ of that Bitcoin even though they are not a lawful owner.
[TIP]
====
Bitcoin ownership is only about control of keys and the ability to spend the Bitcoin with those keys. As the popular Bitcoin saying by goes: "Your keys, your coins-not your keys, not your coins"
Bitcoin ownership is only about control of keys and the ability to spend the Bitcoin with those keys. As the popular Bitcoin saying goes: "Your keys, your coins-not your keys, not your coins."
====
==== Diversity of (Independent) Ownership and Multisig
Ownership and control of private keys is not always in the hands of one person. That's where things get interesting and complicated. We know that more than one person can come to know the same private key, either through theft or because the original holder of the key makes a copy and gives it to someone else. Are all these people owners? In a practical sense they are, because any one of the people who know the private key can spend the bitcoin without the approval of any other.
Ownership and control of private keys is not always in the hands of one person. That's where things get interesting and complicated. We know that more than one person can come to know the same private key, either through theft or because the original holder of the key makes a copy and gives it to someone else. Are all these people owners? In a practical sense, they are, because any one of the people who know the private key can spend the bitcoin without the approval of any other.
Bitcoin also has multisignature addresses where multiple private keys are needed to sign before spending (see <<multisig>>). From a practical perspective, ownership in a multisignature address depends on the quorum (K) and total (N) defined in the K-of-N scheme. A 1-of-10 multisignature scheme would allow any one (K) of 10 (N) signers to spend a bitcoin amount locked in that address. This is similar to the scenario where ten people have a copy of the same private key and any of them can independently spend it.
Bitcoin also has multisignature addresses where multiple private keys are needed to sign before spending (see <<multisig>>). From a practical perspective, ownership in a multisignature address depends on the quorum (`K`) and total (`N`) defined in the `K`-of-`N` scheme. A 1-of-10 multisignature scheme would allow any one (`K`) of 10 (`N`) signers to spend a bitcoin amount locked in that address. This is similar to the scenario where 10 people have a copy of the same private key and any of them can independently spend it.
==== Joint Ownership Without Independent Control
There is also the scenario where *no one* has quorum. In a 2-of-2 scheme like that used in the Lightning Network, neither signer can spend the bitcoin without obtaining a signature from the other party. Who owns the bitcoin in that case? No one really has ownership because no one has control. They each own the equivalent of a voting share in the decision, but both votes are needed. A key problem (pun intended) with a 2-of-2 scheme, in both Bitcoin and the law, is what happens if one of the parties is unavailable, or if there is a vote deadlock and any one party refuses to cooperate.
There is also the scenario where _no one_ has quorum. In a 2-of-2 scheme like that used in the Lightning Network, neither signer can spend the bitcoin without obtaining a signature from the other party. Who owns the bitcoin in that case? No one really has ownership because no one has control. They each own the equivalent of a voting share in the decision, but both votes are needed. A key problem (pun intended) with a 2-of-2 scheme, in both Bitcoin and the law, is what happens if one of the parties is unavailable, or if there is a vote deadlock and any one party refuses to cooperate.
==== Preventing "Locked" and Un-Spendable Bitcoin
If one of the two signers of a 2-of-2 multisig cannot or will not sign, the funds become un-spendable. Not only can this scenario occur accidentally (loss of keys), but it can be used as a form of blackmail by either party: "I won't sign unless you pay me a part of the funds".
If one of the two signers of a 2-of-2 multisig cannot or will not sign, the funds become un-spendable. Not only can this scenario occur accidentally (loss of keys), but it can be used as a form of blackmail by either party: "I won't sign unless you pay me a part of the funds."
Payment channels in Lightning are based on a 2-of-2 multisig address, with the two channel partners as signers in the multisig. At this time, channels are funded only by one of the two channel partners: When you choose to "open" a channel you deposit funds into the 2-of-2 multisig address with a transaction. Once that transaction is mined and the funds are in the multisig, you can't get them back without cooperation from your channel partner, because you need their signature (also) to spend the 2-of-2.
Payment channels in Lightning are based on a 2-of-2 multisig address, with the two channel partners as signers in the multisig. At this time, channels are funded only by one of the two channel partners: When you choose to "open" a channel you deposit funds into the 2-of-2 multisig address with a transaction. Once that transaction is mined and the funds are in the multisig, you can't get them back without cooperation from your channel partner, because you need their signature (also) to spend the bitcoin.
In the next section, as we look at how to open (create) a Lightning channel, we will see how we can prevent loss of funds or any blackmail scenario between the two partners by implementing a fairness protocol for the channel construction with the help of pre-signed transactions that spend the multisig output in a way that gives the peers in the channel exclusive ability to spend one of the outputs which encodes the amount of Bitcoin they own in the channel.
In the next section, as we look at how to open (create) a Lightning channel, we will see how we can prevent loss of funds or any blackmail scenario between the two partners by implementing a fairness protocol for the channel construction with the help of pre-signed transactions that spend the multisig output in a way that gives the peers in the channel exclusive ability to spend one of the outputs which encodes the amount of bitcoin they own in the channel.
=== Constructing a Payment Channel
In <<what_is_payment_channel>>, we described payment channels as a _financial relationship_ between two Lightning Nodes, which is established by funding a 2-of-2 multisignature address from the two channel partners.
In <<what_is_payment_channel>>, we described payment channels as a _financial relationship_ between two Lightning nodes, which is established by funding a 2-of-2 multisignature address from the two channel partners.
Let's assume that Alice wants to construct a payment channel allowing her to connect to Bob's store directly. First, the two nodes (Alice's and Bob's) have to establish an internet connection to each other, so that they can negotiate a payment channel.
==== Node Private and Public Keys
Every node on the Lightning Network is identified by a _node public key_. The public key uniquely identifies the specific node and is usually presented as a hexadecimal encoding. For example, Rene Pickhardt currently runs a Lightning Node (+ln.rene-pickhardt.de+) that is identified by the following node public key:
Every node on the Lightning Network is identified by a _node public key_. The public key uniquely identifies the specific node and is usually presented as a hexadecimal encoding. For example, René Pickhardt currently runs a Lightning Node (+ln.rene-pickhardt.de+) that is identified by the following node public key:
Each node generates a root private key when first initialized. The private key is kept private at all times (never shared) and securely stored in the node's wallet. From that private key, the node derives a public key which is the node identifier and shared with the network. Since the key space is enormous, as long as each node generates the private key randomly, it will have a unique public key, which can therefore uniquely identify it on the network.
Each node generates a root private key when first initialized. The private key is kept private at all times (never shared) and securely stored in the node's wallet. From that private key, the node derives a public key that is the node identifier and shared with the network. Since the key space is enormous, as long as each node generates the private key randomly, it will have a unique public key, which can therefore uniquely identify it on the network.
==== Node Network Address
@ -83,7 +83,7 @@ TCP/Tor:: A Tor "onion" address and TCP port number
The network address identifier is written as +Address:Port+, which is consistent with international standards for network identifiers, as used for example on the web.
For example, Rene's node with node public key +02a1ceb...45ea7b8+ currently advertises it's network address as the TCP/IP address:
For example, René's node with node public key +02a1ceb...45ea7b8+ currently advertises its network address as the TCP/IP address:
----
172.16.235.20:9735
@ -96,9 +96,9 @@ The default TCP port for the Lightning Network is 9735, but a node can choose to
==== Node Identifiers
Together the node public key and network address are written in the following format, separated by an +@+ sign, as +NodeID@Address:Port+
Together the node public key and network address are written in the following format, separated by an +@+ sign, as _+NodeID@Address:Port+_
@ -106,22 +106,22 @@ So the full identifier for Rene's node would be:
[TIP]
====
The alias of Rene's node is +ln.rene-pickhardt.de+ however this name exists just for better readability. Every node operator can announce whatever alias they want, and there is no mechanism that prevents node operators from selecting an alias that is already being used. Thus in order to refer to a node one must use the +NodeID@Address:Port+ schema.
The alias of René's node is +ln.rene-pickhardt.de+; however, this name exists just for better readability. Every node operator can announce whatever alias they want, and there is no mechanism that prevents node operators from selecting an alias that is already being used. Thus to refer to a node one must use the _+NodeID@Address:Port+_ schema.
====
The identifier above is often encoded in a QR code, making it easier for users to scan, if they want to connect their own node to the specific node identified by that address.
The preceding identifier above is often encoded in a QR code, making it easier for users to scan, if they want to connect their own node to the specific node identified by that address.
Much like Bitcoin Nodes, Lightning Nodes advertise their presence on the Lightning Network by "gossiping" their node public key and network address. That way, other nodes can find them and keep an inventory (database) of all the known nodes that they can connect to and exchange the messages that are defined in the Lightning P2P message protocol.
Much like Bitcoin nodes, Lightning nodes advertise their presence on the Lightning Network by "gossiping" their node public key and network address. That way, other nodes can find them and keep an inventory (database) of all the known nodes that they can connect to and exchange the messages that are defined in the Lightning P2P message protocol.
==== Connecting Nodes As Direct Peers
In order for Alice's node to connect to Bob's node, she will need Bob's node public key, or the full address containing the public key, IP or Tor address and port. Because Bob runs a store, Bob's node address can be retrieved from an invoice or a store payment page on the web. Alice can scan a QR code that contains the address and instruct her node to connect to Bob's node.
In order for Alice's node to connect to Bob's node, she will need Bob's node public key, or the full address containing the public key, IP or Tor address, and port. Because Bob runs a store, Bob's node address can be retrieved from an invoice or a store payment page on the web. Alice can scan a QR code that contains the address and instruct her node to connect to Bob's node.
Once Alice has connected to Bob's node, their nodes are now directly connected _peers_.
Once Alice has connected to Bob's node, their nodes are now directly connected peers.
[TIP]
====
To open a payment channel, two nodes must first be _connected_ as direct peers by opening a connection over the internet (or Tor).
To open a payment channel, two nodes must first be connected as direct peers by opening a connection over the internet (or Tor).
====
=== Constructing the Channel
@ -130,34 +130,34 @@ Now that Alice's and Bob's Lightning nodes are connected, they can begin the pro
[TIP]
====
We describe two different protocols in this scenario. First, there is a _message protocol_, which establishes how the Lightning Nodes communicate over the internet and what messages they exchange with each other. Second, there is the _cryptographic protocol_ which establishes how the two nodes construct and sign Bitcoin transactions.
We describe two different protocols in this scenario. First, there is a _message protocol_, which establishes how the Lightning nodes communicate over the internet and what messages they exchange with each other. Second, there is the _cryptographic protocol_, which establishes how the two nodes construct and sign Bitcoin transactions.
====
[[peer_protocol_channel_management]]
==== Peer Protocol for Channel Management
The Lightning Peer Protocol for Channel Management is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md[BOLT #2-Peer Protocol for Channel Management]. In this chapter we will be reviewing the "Channel Establishment" and "Channel Closing" sections of BOLT#2 in more detail.
The Lightning Peer Protocol for Channel Management is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md[BOLT #2-Peer Protocol for Channel Management]. In this chapter we will be reviewing the "Channel Establishment" and "Channel Closing" sections of BOLT#2 in more detail.
==== Channel Establishment Message Flow
Channel establishment is achieved by the exchange of six messages between Alice and Bob's nodes (three from each peer): +open_channel+, +accept_channel+, +funding_created+, +funding_signed+, +funding_locked+ and +funding_locked+. The six messages are shown as a time-sequence diagram in <<funding_message_flow>>:
Channel establishment is achieved by the exchange of six messages between Alice and Bob's nodes (three from each peer): +open_channel+, +accept_channel+, +funding_created+, +funding_signed+, +funding_locked+, and +funding_locked+. The six messages are shown as a time-sequence diagram in <<funding_message_flow>>.
In "<<funding_message_flow>>" Alice and Bob's nodes are represented by the vertical lines "A" and "B" on either side of the diagram. A time-sequence diagram like this shows time flowing downwards, and messages flowing from one side to the other between the two communication peers. The lines are sloped down to represent the elapsed time needed to transmit each message and the direction of the message is shown by an arrow at the end of each line.
In <<funding_message_flow>> Alice and Bob's nodes are represented by the vertical lines "A" and "B" on either side of the diagram. A time-sequence diagram like this shows time flowing downward, and messages flowing from one side to the other between the two communication peers. The lines are sloped down to represent the elapsed time needed to transmit each message, and the direction of the message is shown by an arrow at the end of each line.
The channel establishment involves three parts. First, the two peers communicate their capabilities and expectations, with Alice initiating a request through +open_channel+ and Bob accepting the channel request through +accept_channel+.
Second, Alice constructs the funding and refund transactions (as we will see later in this section) and sends +funding_created+ to Bob. Another name for the "refund" transaction is a "commitment" transaction, as it commits to the current distribution of balances in the channel. Bob responds by sending back the necessary signatures with +funding_signed+. This interaction is the basis for the _cryptographic protocol_ to secure the channel and prevent theft. Alice will now broadcast the funding transaction (on-chain), to establish and anchor the payment channel. The transaction will need to be confirmed on the Bitcoin blockchain.
Second, Alice constructs the funding and refund transactions (as we will see later in this section) and sends +funding_created+ to Bob. Another name for the "refund" transaction is a "commitment" transaction, as it commits to the current distribution of balances in the channel. Bob responds by sending back the necessary signatures with +funding_signed+. This interaction is the basis for the _cryptographic protocol_ to secure the channel and prevent theft. Alice will now broadcast the funding transaction (on-chain) to establish and anchor the payment channel. The transaction will need to be confirmed on the Bitcoin blockchain.
[TIP]
====
The name of the +funding_signed+ message can be a bit confusing. This message does not contain a signature for the funding transaction but it rather contains Bob's signature for the refund transaction that allows Alice to claim her bitcoin back from the multisig.
The name of the +funding_signed+ message can be a bit confusing. This message does not contain a signature for the funding transaction, but it rather contains Bob's signature for the refund transaction that allows Alice to claim her bitcoin back from the multisig.
====
Once the transaction has sufficient confirmations (as defined by the `minimum_depth` field in the `accept_channel` message), Alice and Bob exchange +funding_locked+ messages and the channel enters normal operating mode.
Once the transaction has sufficient confirmations (as defined by the `minimum_depth` field in the `accept_channel` message), Alice and Bob exchange +funding_locked+ messages, and the channel enters normal operating mode.
===== The open_channel message
@ -166,7 +166,7 @@ Alice's node requests a payment channel with Bob's node, by sending an +open_cha
The structure of the +open_channel+ message (taken from BOLT#2) is shown in <<open_channel_message>>.
[[open_channel_message]]
.The open_channel message
.The `open_channel` message
====
----
[chain_hash:chain_hash]
@ -193,9 +193,9 @@ The structure of the +open_channel+ message (taken from BOLT#2) is shown in <<op
The fields contained in this message specify the channel parameters that Alice wants as well as various configuration settings from Alice's nodes that reflect the security expectations for the operation of the channel.
Some of the channel construction parameters include:
Some of the channel construction parameters are listed here:
chain_hash:: This identifies which blockchain (e.g. Bitcoin mainnet) will be used for this channel. It is usually the hash of the genesis block of that blockchain.
chain_hash:: This identifies which blockchain (e.g., Bitcoin mainnet) will be used for this channel. It is usually the hash of the genesis block of that blockchain.
funding_satoshis:: The amount Alice will use to fund the channel, which is the total channel capacity.
@ -203,15 +203,15 @@ channel_reserve_satoshis:: The minimum balance in satoshis that is reserved on e
push_msat:: An optional amount that Alice will immediately "push" to Bob as a payment upon channel funding. **Setting this value to anything but 0 means effectively gifting money to your channel partner and should be used with caution.**
to_self_delay:: A very important security parameter for the protocol. The value in the `open_channel` message is used in the responder's commitment transaction, and the `accept_channel` the initiator's. This asymmetry exists to allow each side to express how long the other side needs to wait to unilaterally claim the funds in a commitment transaction. If Bob at any time unilaterally closes the channel against the will of Alice he commits to not accessing his own funds for the delay defined here. The higher this value the more security Alice has but the longer Bob might have to have his funds locked.
to_self_delay:: A very important security parameter for the protocol. The value in the `open_channel` message is used in the responder's commitment transaction, and the `accept_channel` the initiator's. This asymmetry exists to allow each side to express how long the other side needs to wait to unilaterally claim the funds in a commitment transaction. If Bob at any time unilaterally closes the channel against the will of Alice, he commits to not accessing his own funds for the delay defined here. The higher this value the more security Alice has but the longer Bob might have his funds locked.
funding_pubkey:: The public key Alice will contribute to the 2-of-2 multisig that anchors this channel.
_basepoint:: Master keys, used to derive child keys for various parts of the commitment, revocation, routed payment (HTLCs) and closing transactions. These will be used and explained in subsequent chapters.
X_basepoint:: Master keys, used to derive child keys for various parts of the commitment, revocation, routed payment (HTLCs), and closing transactions. These will be used and explained in subsequent chapters.
[TIP]
====
If you want to understand the other fields of this and Lightning peer protocol messages that we do not discuss in this book we suggest you look them up in the BOLT specifications. These messages and fields are important, but cannot be covered in enough detail in the scope of this book. We want you to understand the fundamental principles well enough that you can fill in the details by reading the actual protocol specification (BOLTs).
If you want to understand the other fields of this and Lightning peer protocol messages that we do not discuss in this book, we suggest you look them up in the BOLT specifications. These messages and fields are important, but cannot be covered in enough detail in the scope of this book. We want you to understand the fundamental principles well enough that you can fill in the details by reading the actual protocol specification (BOLTs).
====
===== The accept_channel message
@ -219,7 +219,7 @@ If you want to understand the other fields of this and Lightning peer protocol m
In response to Alice's +open_channel+ message, Bob sends back the +accept_channel+ message shown in <<accept_channel_message>>.
[[accept_channel_message]]
.The accept_channel message
.The `accept_channel` message
====
----
[32*byte:temporary_channel_id]
@ -250,7 +250,7 @@ minimum_depth:: This is the number of confirmations that Bob's node expects for
==== The Funding Transaction
Once Alice's node receives Bob's +accept_channel+ message, it has the information necessary to construct the _funding transaction_ that anchors the channel to the Bitcoin blockchain. As we discussed in earlier chapters, a lightning payment channel is anchored by a 2-of-2 multisignature address. First, we need to generate that multisignature address in order to allow us to construct the funding transaction (and the refund transaction as described below).
Once Alice's node receives Bob's +accept_channel+ message, it has the information necessary to construct the _funding transaction_ that anchors the channel to the Bitcoin blockchain. As we discussed in earlier chapters, a lightning payment channel is anchored by a 2-of-2 multisignature address. First, we need to generate that multisignature address to allow us to construct the funding transaction (and the refund transaction as described subsequently).
==== Generating a Multisignature Address
@ -262,49 +262,49 @@ Alice's node constructs a multisignature script as shown here:
[[A_B_multisig]]
.A 2-of-2 multisig script with Alice and Bob's funding_pubkey values
Note that in practice, the funding keys are deterministically _sorted_ (using lexicographical order of the serialized compressed form of the public keys) before being placed in the witness script. By agreeing to this sorted order ahead of time, we ensure both parties will construct an identical funding transaction output, which is signed by the commitment transaction signature exchanged.
Note that, in practice, the funding keys are deterministically _sorted_ (using lexicographical order of the serialized compressed form of the public keys) before being placed in the witness script. By agreeing to this sorted order ahead of time, we ensure both parties will construct an identical funding transaction output, which is signed by the commitment transaction signature exchanged.
This script is encoded as a Pay-to-Witness-Script-Hash Bitcoin address, that looks something like this:
This script is encoded as a Pay-to-Witness-Script-Hash (P2WSH) Bitcoin address, which looks something like this:
Alice's node can now construct a funding transaction, sending the amount agreed with Bob (funding_satoshis) to the 2-of-2 multisig address. Let's assume that funding_satoshis was 140,000 and Alice is spending a 200,000 satoshi output and creating 60,000 satoshi change. The transaction will look something like this:
Alice's node can now construct a funding transaction, sending the amount agreed with Bob (`funding_satoshis`) to the 2-of-2 multisig address. Let's assume that funding_satoshis was 140,000 and Alice is spending a 200,000 satoshi output and creating 60,000 satoshi change. The transaction will look something like Figure 7-4.
[[A_B_funding_Tx]]
.Alice constructs the funding transaction
image::images/mtln_0704.png["Alice constructs the funding transaction"]
Alice *does not broadcast* this transaction, because doing so would put her 140,000 satoshi at risk. Once spent to the 2-of-2 multisig, there is no way for Alice to recover her money without Bob's signature.
Alice *does not broadcast* this transaction because doing so would put her 140,000 satoshi at risk. Once spent to the 2-of-2 multisig, there is no way for Alice to recover her money without Bob's signature.
.Dual-funded payment channels
****
In the current implementation of Lightning, channels are funded only by the node initiating the channel (Alice in our example). Dual-funded channels have been proposed, but not yet implemented. In a dual-funded channel, both Alice and Bob would contribute inputs to the funding transaction. Dual-funded channels require a slightly more complicated message flow and cryptographic protocol, so they have not been implemented yet but are planned for a future update to the Lightning BOLTs. The C-Lightning implementation includes an experimental version of a variant on dual funded channels.
In the current implementation of Lightning, channels are funded only by the node initiating the channel (Alice in our example). Dual-funded channels have been proposed, but not yet implemented. In a dual-funded channel, both Alice and Bob would contribute inputs to the funding transaction. Dual-funded channels require a slightly more complicated message flow and cryptographic protocol, so they have not been implemented yet but are planned for a future update to the Lightning BOLTs. The c-lightning implementation includes an experimental version of a variant on dual funded channels.
****
==== Holding Signed Transactions Without Broadcasting
An important Bitcoin feature that makes Lightning possible is the ability to construct and sign transactions, but not broadcast them. The transaction is *valid* in every way, but until it is broadcast and confirmed on the Bitcoin blockchain it is not recognized and its outputs are not spendable as they have not been created on the blockchain. We will use this capability many times in the Lightning Network and Alice's node uses the capability when constructing the funding transaction: holding it and not broadcasting it yet.
An important Bitcoin feature that makes Lightning possible is the ability to construct and sign transactions, but not broadcast them. The transaction is _valid_ in every way, but until it is broadcast and confirmed on the Bitcoin blockchain it is not recognized and its outputs are not spendable because they have not been created on the blockchain. We will use this capability many times in the Lightning Network, and Alice's node uses the capability when constructing the funding transaction: holding it and not broadcasting it yet.
==== Refund Before Funding
To prevent loss of funds, Alice cannot put her bitcoin into a 2-of-2 until she has a way to get a refund if things go wrong. Essentially, she must plan the "exit" from the channel before she enters into this arrangement.
Consider the legal construct of a prenuptial agreement, also known as a "prenup". When two people enter into a marriage their money is bound together by law (depending on jurisdiction). Prior to entering into the marriage, they can sign an agreement that specifies how to separate their assets if they dissolve their marriage through divorce.
Consider the legal construct of a prenuptial agreement, also known as a "prenup." When two people enter into a marriage their money is bound together by law (depending on jurisdiction). Prior to entering into the marriage, they can sign an agreement that specifies how to separate their assets if they dissolve their marriage through divorce.
We can create a similar agreement in Bitcoin. For example, we can create a refund transaction, which functions like a prenup, allowing the parties decide how the funds in their channel will be divided before their funds are actually locked into the multisignature funding address.
==== Constructing the Pre-Signed Refund Transaction
Alice will construct the "refund transaction" immediately after constructing (but not broadcasting) the funding transaction. The refund transaction spends the 2-of-2 multisig back to Alice's wallet. We call this refund transaction a _commitment transaction_ as it commits both channel partners to distributing the channel balance fairly. Since Alice funded the channel on her own, she gets the entire balance and both Alice and Bob commit to refunding Alice with this transaction.
Alice will construct the refund transaction immediately after constructing (but not broadcasting) the funding transaction. The refund transaction spends the 2-of-2 multisig back to Alice's wallet. We call this refund transaction a _commitment transaction_ because it commits both channel partners to distributing the channel balance fairly. Since Alice funded the channel on her own, she gets the entire balance and both Alice and Bob commit to refunding Alice with this transaction.
In practice, it is a bit more complicated as we will see in subsequent chapters, but for now let's keep things simple and assume it looks like this:
In practice, it is a bit more complicated as we will see in subsequent chapters, but for now let's keep things simple and assume it looks like Figure 7-5.
[[A_B_fund_refund_Tx]]
.Alice also constructs the refund transaction
@ -314,26 +314,26 @@ Later in this chapter we will see how more commitment transactions can be made t
==== Chaining Transactions Without Broadcasting
So now, Alice has constructed the two transactions shown in <<A_B_fund_refund_Tx>>. But you might be wondering how is this possible? Alice hasn't broadcast the funding transaction to the Bitcoin blockchain. As far as everyone on the network is concerned, that transaction doesn't exist. The refund transaction is constructed so as to *spend* one of the outputs of the funding transaction, even though that output doesn't exist yet either. How can you spend an output that hasn't been confirmed on the Bitcoin blockchain?
So now, Alice has constructed the two transactions shown in <<A_B_fund_refund_Tx>>. But you might be wondering how this is possible. Alice hasn't broadcast the funding transaction to the Bitcoin blockchain. As far as everyone on the network is concerned, that transaction doesn't exist. The refund transaction is constructed so as to _spend_ one of the outputs of the funding transaction, even though that output doesn't exist yet either. How can you spend an output that hasn't been confirmed on the Bitcoin blockchain?
The refund transaction is not yet a valid transaction. In order for it to become a valid transaction two things must happen:
The refund transaction is not yet a valid transaction. For it to become a valid transaction two things must happen:
* The funding transaction must be broadcast to the Bitcoin Network. (To ensure the security of the Lightning Network we will also require it to be confirmed by the Bitcoin blockchain though this is not strictly necessary to chain transactions.)
* The refund transaction's input needs Alice and Bob's signature
* The funding transaction must be broadcast to the Bitcoin network. (To ensure the security of the Lightning Network we will also require it to be confirmed by the Bitcoin blockchain though this is not strictly necessary to chain transactions.)
* The refund transaction's input needs Alice's and Bob's signatures.
But even though these two things haven't happened and even though Alice's node hasn't broadcast the funding transaction, she can still construct the refund transaction. She can do so because she can calculate the funding transaction's hash and reference it as an input in the refund transaction.
But even though these two things haven't happened, and even though Alice's node hasn't broadcast the funding transaction, she can still construct the refund transaction. She can do so because she can calculate the funding transaction's hash and reference it as an input in the refund transaction.
Notice how Alice has calculated +6da3c2...387710+ as the funding transaction hash? If and when the funding transaction is broadcast, that hash will be recorded as the transaction ID of the funding transaction. Therefore, the 0 output of the funding transaction (the 2-of-2 address output) will then be referenced as output ID +6da3c2...387710:0+. The refund transaction can be constructed to spend that funding transaction output even though it doesn't exist yet because Alice knows what its identifier will be once confirmed.
Notice how Alice has calculated +6da3c2...387710+ as the funding transaction hash? If and when the funding transaction is broadcast, that hash will be recorded as the transaction ID of the funding transaction. Therefore, the `0` output of the funding transaction (the 2-of-2 address output) will then be referenced as output ID +6da3c2...387710:0+. The refund transaction can be constructed to spend that funding transaction output even though it doesn't exist yet because Alice knows what its identifier will be once confirmed.
This means that Alice can create a chained transaction by referencing an output that doesn't yet exist, knowing that the reference will be valid if the funding transaction is confirmed, making the refund transaction valid too. As we will see in the next section, this "trick" of chaining transactions before they are broadcast requires a very important feature of Bitcoin that was introduced in August of 2017: _Segregated Witness_.
==== Solving Malleability (Segregated Witness)
Alice has to depend on the transaction ID of the funding transaction being known before confirmation. But before the introduction of Segregated Witness (a.k.a SegWit) in August 2017, this was not sufficient to protect Alice. Because of the way transactions were constructed with the signatures (witnesses) included in the transaction ID, it was possible for a third party (e.g. Bob) to broadcast an alternative version of a transaction with a malleated (modified) transaction ID. This is known as _Transaction Malleability_ and made it difficult to implement indefinite lifetime payment channels securely.
Alice has to depend on the transaction ID of the funding transaction being known before confirmation. But before the introduction of Segregated Witness (SegWit) in August 2017, this was not sufficient to protect Alice. Because of the way transactions were constructed with the signatures (witnesses) included in the transaction ID, it was possible for a third party (e.g., Bob) to broadcast an alternative version of a transaction with a _malleated_ (modified) transaction ID. This is known as _Transaction Malleability_, and prior to SegWit, this problem made it difficult to implement indefinite lifetime payment channels securely.
If Bob could modify Alice's funding transaction before it was confirmed and produce a replica that had a different transaction ID, Bob could make Alice's refund transaction invalid and hijack her bitcoin. Alice would be at Bob's mercy to get a signature to release her funds and could easily be blackmailed. Bob couldn't steal the funds, but he could prevent Alice from getting them back.
The introduction of SegWit made unconfirmed transaction IDs immutable from the PoV of 3rd parties, meaning that Alice could be sure that the transaction ID of the funding transaction would not change. As a result, Alice can be confident that if she gets Bob's signature on the refund transaction, she has a way to recover her money. She now has a way to implement the Bitcoin equivalent of a "prenup" before locking her funds into the multisig.
The introduction of SegWit made unconfirmed transaction IDs immutable from the point of view of third parties, meaning that Alice could be sure that the transaction ID of the funding transaction would not change. As a result, Alice can be confident that if she gets Bob's signature on the refund transaction, she has a way to recover her money. She now has a way to implement the Bitcoin equivalent of a "prenup" before locking her funds into the multisig.
[TIP]
====
@ -355,11 +355,11 @@ Now that Alice has constructed the necessary transactions, the channel construct
With this message, Alice gives Bob the important information about the funding transaction that anchors the payment channel:
funding_txid:: This is the transaction ID of the funding transaction, and is used to create the channel ID once the channel is established.
funding_txid:: This is the transaction ID (TxID) of the funding transaction, and is used to create the channel ID once the channel is established.
funding_output_index:: This is the output index, so Bob knows which output of the transaction (e.g. output 0) is the 2-of-2 multisig output funded by Alice. This is also used to form the channel ID.
funding_output_index:: This is the output index, so Bob knows which output of the transaction (e.g., output `0`) is the 2-of-2 multisig output funded by Alice. This is also used to form the channel ID.
Finally, Alice also sends the +signature+ corresponding to Alice's funding_pubkey and used to spend from the 2-of-2 multisig. This is needed by Bob as he will also need to create his own version of a commitment transaction. That commitment transaction needs a signature from Alice which she provides to him. Note that the commitment transactions of Alice and Bob look slightly different thus the signatures will be different. Knowing how the commitment transaction of the other party looks like is crucial and part of the protocol to provide the valid signature.
Finally, Alice also sends the +signature+ corresponding to Alice's `funding_pubkey` and used to spend from the 2-of-2 multisig. This is needed by Bob because he will also need to create his own version of a commitment transaction. That commitment transaction needs a signature from Alice, which she provides to him. Note that the commitment transactions of Alice and Bob look slightly different, thus the signatures will be different. Knowing what the commitment transaction of the other party looks like is crucial and part of the protocol to provide the valid signature.
[TIP]
====
@ -375,9 +375,9 @@ After receiving the +funding_created+ message from Alice, Bob now knows the fun
More precisely, a `channel_id`, which is the 32 byte representation of a funding UTXO, is generated by XOR'ing the lower 2-bytes of the funding TXID with the index of the funding output.
More precisely, a `channel_id`, which is the 32 byte representation of a funding UTXO, is generated by XORing the lower 2-bytes of the funding TxID with the index of the funding output.
Bob will also need to send Alice his signature for the refund transaction, based on Bob's funding_pubkey which formed the 2-of-2 multisig. While Bob already has his local refund transaction this will allow Alice to complete the refund transaction with all necessary signatures and be sure her money is refundable in case something goes wrong.
Bob will also need to send Alice his signature for the refund transaction, based on Bob's `funding_pubkey` that formed the 2-of-2 multisig. Although Bob already has his local refund transaction, this will allow Alice to complete the refund transaction with all necessary signatures and be sure her money is refundable in case something goes wrong.
Bob constructs a +funding_signed+ message and sends it to Alice. Here we see the contents of this message:
@ -392,13 +392,13 @@ Bob constructs a +funding_signed+ message and sends it to Alice. Here we see the
==== Broadcasting the Funding Transaction
Upon receiving the +funding_signed+ message from Bob, Alice now has both signatures needed to sign the refund transaction. Her "exit plan" is now secure and therefore she can broadcast the funding transaction without fear of having her funds locked. If anything goes wrong, Alice can simply broadcast the refund transaction and get her money back, without any further help from Bob.
Upon receiving the +funding_signed+ message from Bob, Alice now has both signatures needed to sign the refund transaction. Her "exit plan" is now secure, and therefore she can broadcast the funding transaction without fear of having her funds locked. If anything goes wrong, Alice can simply broadcast the refund transaction and get her money back, without any further help from Bob.
Alice now sends the funding transaction to the Bitcoin network, so that it can be mined into the blockchain. Both Alice and Bob will be watching for this transaction and waiting for +minimum_depth+ confirmations (e.g. 6 confirmations) on the Bitcoin blockchain.
Alice now sends the funding transaction to the Bitcoin network so that it can be mined into the blockchain. Both Alice and Bob will be watching for this transaction and waiting for +minimum_depth+ confirmations (e.g., six confirmations) on the Bitcoin blockchain.
[TIP]
====
Of course Alice will use the Bitcoin Protocol to verify that the signature that Bob sent her is indeed valid. This step is very crucial. If for some reason Bob was sending wrongful data to Alice her "exit plan" would be sabotaged.
Of course Alice will use the Bitcoin Protocol to verify that the signature that Bob sent her is indeed valid. This step is very crucial. If for some reason Bob were sending wrongful data to Alice, her "exit plan" would be sabotaged.
====
===== The funding_locked message
@ -417,17 +417,17 @@ Let's assume that Alice wants to send 70,000 satoshis to Bob to pay her bill at
In principle, sending a payment from Alice to Bob is simply a matter of redistributing the balance of the channel. Before the payment is sent, Alice has 140,000 satoshis and Bob has none. After the 70,000 satoshi payment is sent, Alice has 70,000 satoshis and Bob has 70,000 satoshis.
Therefore, all Alice and Bob have to do is create and sign a transaction that spends the 2-of-2 multisig to two outputs paying Alice and Bob their corresponding balance. We call this updated transaction a _commitment transaction_.
Therefore, all Alice and Bob have to do is create and sign a transaction that spends the 2-of-2 multisig to two outputs paying Alice and Bob their corresponding balances. We call this updated transaction a _commitment transaction_.
Alice and Bob operate the payment channel by _advancing the channel state_ through a series of commitments. Each commitment updates the balances to reflect payments that have flowed across the channel. Both Alice and Bob can initiate a new commitment to update the channel.
In <<competing_commitments_1>> we see several commitment transactions:
In <<competing_commitments_1>> we see several commitment transactions.
The first commitment transaction shown in <<competing_commitments_1>> is the "refund transaction" that Alice constructed before funding the channel. In the diagram, this is "Commitment #0". After Alice pays Bob 70,000 satoshis, the new commitment transaction ("Commitment #1") has two outputs paying Alice and Bob their respective balance. We have included two subsequent commitment transactions (Commitment #2 and Commitment #3) which represent Alice paying Bob an additional 10,000 satoshis and then 20,000 satoshis respectively.
The first commitment transaction shown in <<competing_commitments_1>> is the refund transaction that Alice constructed before funding the channel. In the diagram, this is Commitment #0. After Alice pays Bob 70,000 satoshis, the new commitment transaction (Commitment #1) has two outputs paying Alice and Bob their respective balances. We have included two subsequent commitment transactions (Commitment #2 and Commitment #3) which represent Alice paying Bob an additional 10,000 satoshis and then 20,000 satoshis, respectively.
Each signed and valid commitment transaction can be used by either channel partner at any time to close the channel by broadcasting it to the Bitcoin network. Since they both have the most recent commitment transaction and can use it at any time, they can also just hold it and not broadcast it. It's their guarantee of a fair exit from the channel.
@ -435,34 +435,34 @@ Each signed and valid commitment transaction can be used by either channel partn
You may be wondering how it is possible for Alice and Bob to have multiple commitment transactions, all of them attempting to spend the same 2-of-2 output from the funding transaction. Aren't these commitment transactions conflicting? Isn't this a "double-spend" that the Bitcoin system is meant to prevent?
It is indeed! In fact, we rely on Bitcoin's ability to *prevent* a double spend to make Lightning work. No matter how many commitment transactions Alice and Bob construct and sign, only one of them can actually get confirmed.
It is indeed! In fact, we rely on Bitcoin's ability to _prevent_ a double spend to make Lightning work. No matter how many commitment transactions Alice and Bob construct and sign, only one of them can actually get confirmed.
As long as Alice and Bob hold these transactions and don't broadcast them, the funding output is unspent. But if a commitment transaction is broadcast and confirmed, it will spend the funding output. If Alice or Bob attempt to broadcast more than one commitment transaction, only one of them will be confirmed and the others will be rejected as attempted (and failed) double-spends.
As long as Alice and Bob hold these transactions and don't broadcast them, the funding output is unspent. But if a commitment transaction is broadcast and confirmed, it will spend the funding output. If Alice or Bob attempts to broadcast more than one commitment transaction, only one of them will be confirmed and the others will be rejected as attempted (and failed) double-spends.
If more than one commitment transaction are broadcast, there are many factors that will determine which one gets confirmed first: the amount of fees included, the speed of propagation of these competing transactions, network topology, etc. Essentially it becomes a race without a predictable outcome. That doesn't sound very secure. It sounds like someone could cheat.
If more than one commitment transactions are broadcast, there are many factors that will determine which one gets confirmed first: the amount of fees included, the speed of propagation of these competing transactions, network topology, etc. Essentially it becomes a race without a predictable outcome. That doesn't sound very secure. It sounds like someone could cheat.
==== Cheating with Old Commitment Transactions
Let's look more carefully at the commitment transactions in <<competing_commitments_1>>. All four commitment transactions are signed and valid. But only the last one accurately reflects the most recent channel balances. In this particular scenario, Alice has an opportunity to cheat, by broadcasting an older commitment and getting it confirmed on the Bitcoin blockchain. Let's say Alice transmits Commitment #0 and gets it confirmed: she will effectively close the channel and take all 140,000 satoshis herself. In fact, in this particular example any commitment but #3 improves Alice's position and allows her to "cancel" at least part of the payments reflected in the channel.
Let's look more carefully at the commitment transactions in <<competing_commitments_1>>. All four commitment transactions are signed and valid. But only the last one accurately reflects the most recent channel balances. In this particular scenario, Alice has an opportunity to cheat by broadcasting an older commitment and getting it confirmed on the Bitcoin blockchain. Let's say Alice transmits Commitment #0 and gets it confirmed: she will effectively close the channel and take all 140,000 satoshis herself. In fact, in this particular example any commitment but Commitment #3 improves Alice's position and allows her to "cancel" at least part of the payments reflected in the channel.
In the next section we will see how the Lightning Network resolves this problem - preventing older commitment transactions from being used by the channel partners by a mechanism of revocation and penalties. There are other ways to prevent the transmission of older commitment transactions, such as _eltoo channels_ but they require an upgrade to Bitcoin called _input rebinding_.
In the next section we will see how the Lightning Network resolves this problem—preventing older commitment transactions from being used by the channel partners by a mechanism of revocation and penalties. There are other ways to prevent the transmission of older commitment transactions, such as eltoo channels, but they require an upgrade to Bitcoin called input rebinding.
==== Revoking Old Commitment Transactions
Bitcoin transactions do not expire and cannot be "canceled". Neither can they be stopped or censored once they have been broadcast. So how do we "revoke" a transaction that another person holds that has already been signed?
Bitcoin transactions do not expire and cannot be "canceled." Neither can they be stopped or censored once they have been broadcast. So how do we "revoke" a transaction that another person holds that has already been signed?
The solution used in Lightning is another example of a fairness protocol. Instead of trying to control the ability to broadcast a transaction, there is a built-in _penalty mechanism_ that ensures it is not in the best interest of a would be cheater to transmit an old commitment transaction. They can always broadcast it, but they will most likely lose money if they do so.
[TIP]
====
The word "revoke" is a misnomer because it implies that older commitments are somehow made invalid and cannot be broadcast and confirmed. But this is not the case, since valid Bitcoin transactions cannot be "revoked". Instead, the Lightning protocol uses a penalty mechanism to punish the channel partner who broadcasts an old commitment.
The word "revoke" is a misnomer because it implies that older commitments are somehow made invalid and cannot be broadcast and confirmed. But this is not the case, since valid Bitcoin transactions cannot be revoked. Instead, the Lightning protocol uses a penalty mechanism to punish the channel partner who broadcasts an old commitment.
====
There are three elements that make up the Lightning protocol's revocation and penalty mechanism:
* Asymmetric commitment transactions - Alice's commitment transactions are slightly different from those held by Bob.
* Asymmetric commitment transactions: Alice's commitment transactions are slightly different from those held by Bob.
* Delayed spending - The payment to the party holding the commitment transaction is delayed (timelocked), whereas the payment to the other party can be claimed immediately.
* Delayed spending: The payment to the party holding the commitment transaction is delayed (timelocked), whereas the payment to the other party can be claimed immediately.
* Revocation keys to unlock a penalty option for old commitments.
@ -471,19 +471,19 @@ Let's look at these three elements in turn.
==== Asymmetric Commitment Transactions
Alice and Bob hold slightly different commitment transactions. Let's look specifically at Commitment #2 from <<competing_commitments_1>>, in more detail:
Alice and Bob hold slightly different commitment transactions. Let's look specifically at Commitment #2 from <<competing_commitments_1>>, in more detail in Figure 7-7.
By convention, within the Lightning protocol, we refer to the two channel partners as _self_ (also known as _local_) and _remote_, depending on which side we're looking at. The outputs that pay each channel partner are called _to_local_ and _to_remote_, respectively.
By convention, within the Lightning protocol, we refer to the two channel partners as _self_ (also known as _local_) and _remote_, depending on which side we're looking at. The outputs that pay each channel partner are called `to_local` and `to_remote`, respectively.
In <<asymmetric_1>> we see that Alice holds a transaction that pays 60,000 satoshis _to_self_ (can be spent by Alice's keys), and 80,000 satoshis _to_remote_ (can be spent by Bob's keys).
@ -491,9 +491,9 @@ Bob holds the mirror-image of that transaction, where the first output is 80,000
==== Delayed (Timelocked) Spending to_self
Using asymmetric transactions allows the protocol to easily ascribe _blame_ to the cheating party. An invariant that the _broadcasting_ party must always wait ensures that the "honest" party has time to refute the claim, and revoke their funds. This asymmetry is manifested in the form of differing outputs for each side: the _to_local_ output is always timelocked and can't be spent immediately, whereas the _to_remote_ output is not timelocked and can be spent immediately.
Using asymmetric transactions allows the protocol to easily ascribe _blame_ to the cheating party. An invariant that the _broadcasting_ party must always wait ensures that the "honest" party has time to refute the claim and revoke their funds. This asymmetry is manifested in the form of differing outputs for each side: the _to_local_ output is always timelocked and can't be spent immediately, whereas the _to_remote_ output is not timelocked and can be spent immediately.
In the commitment transaction held by Alice, for example, the _to_local_ output that pays her is timelocked for 432 blocks, whereas the _to_remote_ output that pays Bob can be spent immediately. Bob's commitment transaction for Commitment #2 is the mirror image: his own (_to_local_) output is timelocked and Alice's _to_remote_ output can be spent immediately.
In the commitment transaction held by Alice, for example, the _to_local_ output that pays her is timelocked for 432 blocks, whereas the _to_remote_ output that pays Bob can be spent immediately (see Figure 7-9). Bob's commitment transaction for Commitment #2 is the mirror image: his own (_to_local_) output is timelocked and Alice's _to_remote_ output can be spent immediately.
[[asymmetric_delayed_1]]
.Asymmetric and delayed commitment transactions
@ -501,22 +501,22 @@ image::images/mtln_0709.png[Asymmetric and delayed commitment transactions]
That means that if Alice closes the channel by broadcasting and confirming the commitment transaction she holds, she cannot spend her balance for 432 blocks, but Bob can claim his balance immediately. If Bob closes the channel using the commitment transaction he holds, he cannot spend his output for 432 blocks while Alice can immediately spend hers.
The delay is there for one reason: to allow the *remote* party to exercise a penalty option if an old (revoked) commitment should be broadcast by the other channel partner. Let's look at the revocation keys and penalty option next.
The delay is there for one reason: to allow the _remote_ party to exercise a penalty option if an old (revoked) commitment should be broadcast by the other channel partner. Let's look at the revocation keys and penalty option next.
The delay is negotiated by Alice and Bob, during the initial channel construction message flow, as a field called +to_self_delay+. To ensure the security of the channel, the delay is scaled to the capacity of the channel - meaning a channel with more funds has longer delays in the +to_self+ outputs in commitments. Alice's node includes a desired +to_self_delay+ in the +open_channel+ message. If Bob find this acceptable, his node includes the same value for +to_self_delay+ in the +accept_channel+ message. If they do not agree, then the channel is rejected (see +shutdown+ message).
The delay is negotiated by Alice and Bob, during the initial channel construction message flow, as a field called +to_self_delay+. To ensure the security of the channel, the delay is scaled to the capacity of the channel—meaning a channel with more funds has longer delays in the +to_self+ outputs in commitments. Alice's node includes a desired +to_self_delay+ in the +open_channel+ message. If Bob finds this acceptable, his node includes the same value for +to_self_delay+ in the +accept_channel+ message. If they do not agree, then the channel is rejected (see +shutdown+ message).
==== Revocation Keys
As we discussed previously, the word "revocation" is a bit misleading because it implies that the "revoked" transaction cannot be used.
In fact, the "revoked" transaction can be used but if it is used, and it has been "revoked", then one of the channel partners can take all of the channel funds by creating a penalty transaction.
In fact, the revoked transaction can be used, but if it is used, and it has been revoked, then one of the channel partners can take all of the channel funds by creating a penalty transaction.
The way this works is that the _to_local_ output is not only timelocked, but it has two spending conditions in the script: It can be spent by _self_ after the timelock delay *or* it can be spent by _remote_ immediately with a revocation key for this commitment.
The way this works is that the _to_local_ output is not only timelocked, but it also has two spending conditions in the script: It can be spent by _self_ after the timelock delay *or* it can be spent by _remote_ immediately with a revocation key for this commitment.
So, in our example, each side holds a commitment transaction that includes a revocation option in the _to_local_ output, as shown in <<asymmetric_delayed_revocable_1>>:
So, in our example, each side holds a commitment transaction that includes a revocation option in the _to_local_ output, as shown in <<asymmetric_delayed_revocable_1>>.
[[asymmetric_delayed_revocable_1]]
.Asymmetric, delayed and revocable commitments
.Asymmetric, delayed, and revocable commitments
image::images/mtln_0710.png["Asymmetric, delayed and revocable commitments"]
[[commitment_transaction]]
@ -541,16 +541,16 @@ OP_CHECKSIG
This is a conditional script (see <<conditional_scripts>>), which means the output can be spent if _either_ of the two conditions is met. The first clause allows the output to be spent by anyone who can sign for +<revocationpubkey>+. The second clause is timelocked by +<to_self_delay>+ blocks and can only be spent after that many blocks by anyone who can sign for +<local_delayedpubkey>+. In our example, we had set the +<to_self_delay>+ timelock to 432 blocks, but this is a configurable delay that is negotiated by the two channel partners. The +to_self_delay+ timelock duration is usually chosen in proportion to the channel capacity, meaning that larger capacity channels (more funds), have longer +to_self_delay+ timelocks to protect the parties.
The first clause allows the output to be spent by anyone who can sign for +<revocationpubkey>+. A critical requirement to the security of this script is that the remote party *cannot* unilaterally sign with the `revocationpubkey`. To see why this is important, consider the scenario where the remote party breaches a previously revoked commitment, if they can sign with this key, then they can simply take the revocation clause _themselves_ and steal all the funds in the channel. Instead, we derive the `revocationpubkey` for _each_ state based on information from _both_ the self (local) and remote party. A clever use of symmetric and asymmetric cryptography is used to allow both sides to compute the `revocationpubkey` public key, but only the honest self party to compute the private key given their secret information.
The first clause allows the output to be spent by anyone who can sign for +<revocationpubkey>+. A critical requirement to the security of this script is that the remote party _cannot_ unilaterally sign with the `revocationpubkey`. To see why this is important, consider the scenario in which the remote party breaches a previously revoked commitment. If they can sign with this key, then they can simply take the revocation clause _themselves_ and steal all the funds in the channel. Instead, we derive the `revocationpubkey` for _each_ state based on information from _both_ the self (local) and remote party. A clever use of symmetric and asymmetric cryptography is used to allow both sides to compute the `revocationpubkey` public key, but only allow the honest self party to compute the private key given their secret information.
[TIP]
====
As shown above, each side sends a `revocation_basepoint` during the initial channel negotiation messages as well as a `first_per_commitment_point`. The `revocation_basepoint` is static for the lifetime of the channel, while each new channel state will be based off a new `first_per_commitment_point`.
As shown previously, each side sends a `revocation_basepoint` during the initial channel negotiation messages as well as a `first_per_commitment_point`. The `revocation_basepoint` is static for the lifetime of the channel, while each new channel state will be based off a new `first_per_commitment_point`.
Given this information, the `revocationpubkey` for each channel state is derived via the following series of Elliptic Curve and hashing operations: `revocationpubkey = revocation_basepoint * sha256(revocation_basepoint || per_commitment_point) + per_commitment_point * sha256(per_commitment_point || revocation_basepoint)`.
Given this information, the `revocationpubkey` for each channel state is derived via the following series of elliptic curve and hashing operations: `revocationpubkey = revocation_basepoint * sha256(revocation_basepoint || per_commitment_point) + per_commitment_point * sha256(per_commitment_point || revocation_basepoint)`.
Due to the commutative property of the Abelian groups that Elliptic Curves are defined over, once the `per_commitment_secret` (the private key for the `per_commitment_point`) is revealed by the remote party, self can derive the private key for the `revocationpubkey` with the following operation: `revocation_priv = (revocationbase_priv * sha256(revocation_basepoint || per_commitment_point)) + (per_commitment_secret * sha256(per_commitment_point || revocation_basepoint)) mod N`.
Due to the commutative property of the abelian groups that elliptic curves are defined over, once the `per_commitment_secret` (the private key for the `per_commitment_point`) is revealed by the remote party, self can derive the private key for the `revocationpubkey` with the following operation: `revocation_priv = (revocationbase_priv * sha256(revocation_basepoint || per_commitment_point)) + (per_commitment_secret * sha256(per_commitment_point || revocation_basepoint)) mod N`.
To see why this works in practice, notice that we can _reorder_ (commute) and expand the public key computation of the original formula for `revocationpubkey`:
To see why this works in practice, notice that we can _reorder_ (commute) and expand the public key computation of the original formula for `revocationpubkey`:
@ -567,11 +567,11 @@ The timelock used in the commitment transaction with +CHECKSEQUENCEVERIFY+ is a
The second output (to_remote) output of the commitment transaction, is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#to_remote-output[BOLT #3 Commitment Transaction - to_remote Output] and in the simplest form is a Pay-to-Witness-Public-Key-Hash (P2WPKH) for +<remote_pubkey>+, meaning that it simply pays the owner who can sign for +<remote_pubkey>+.
Now that we've defined the commitment transactions in detail, let's see how Alice and Bob advance the state of the channel, create and sign new commitment transactions and revoke old commitment transactions.
Now that we've defined the commitment transactions in detail, let's see how Alice and Bob advance the state of the channel, create and sign new commitment transactions, and revoke old commitment transactions.
=== Advancing the Channel State
To advance the state of the channel, Alice and Bob exchange two messages: +commitment_signed+ and +revoke_and_ack+. The +commitment_signed+ message can be sent by either channel partner when they have an update to the channel state. The other channel partner then may respond with +revoke_and_ack+ to _revoke_ the old commitment and _acknowledge_ the new commitment.
To advance the state of the channel, Alice and Bob exchange two messages: +commitment_signed+ and +revoke_and_ack+ messages. The +commitment_signed+ message can be sent by either channel partner when they have an update to the channel state. The other channel partner then may respond with +revoke_and_ack+ to _revoke_ the old commitment and _acknowledge_ the new commitment.
In <<commitment_message_flow>> we see the Alice and Bob exchanging two pairs of +commitment_signed+ and +revoke_and_ack+. The first flow shows a state update initiated by Alice (left to right +commitment_signed+), to which Bob responds (right to left +revoke_and_ack+). The second flow shows a state update initiated by Bob and responded to by Alice.
@ -581,7 +581,7 @@ image::images/mtln_0711.png[Commitment and revocation message flow]
==== The Commitment_Signed Message
The structure of the +commitment_signed+ message is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#committing-updates-so-far-commitment_signed[BOLT #2-Peer Protocol -commitment_signed] and shown here:
The structure of the +commitment_signed+ message is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#committing-updates-so-far-commitment_signed[BOLT #2-Peer Protocol -commitment_signed] and shown here:
[[commitment_signed_message]]
.The commitment_signed message
@ -599,7 +599,7 @@ The structure of the +commitment_signed+ message is defined in https://github.co
[NOTE]
====
The use of HTLCs to commit updates will be explained in detail in <<htlcs>> and <<channel_operation>>
The use of HTLCs to commit updates will be explained in detail in <<htlcs>> and <<channel_operation>>.
====
Alice's +commitment_signed+ message gives Bob the signature needed (Alice's part of the 2-of-2) for a new commitment transaction.
@ -608,7 +608,7 @@ Alice's +commitment_signed+ message gives Bob the signature needed (Alice's part
Now that Bob has a new commitment transaction, he can revoke the previous commitment by giving Alice a revocation key, and construct the new commitment with Alice's signature.
The +revoke_and_ack+ message is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#completing-the-transition-to-the-updated-state-revoke_and_ack[BOLT #2-Peer Protocol-revoke_and_ack] and shown here:
The +revoke_and_ack+ message is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#completing-the-transition-to-the-updated-state-revoke_and_ack[BOLT #2-Peer Protocol-revoke_and_ack] and shown here:
[[revoke_and_ack_message]]
.The revoke_and_ack message
@ -620,31 +620,31 @@ The +revoke_and_ack+ message is defined in https://github.com/lightningnetwork/l
----
+channel_id+:: This is the identifier of the channel
+channel_id+:: This is the identifier of the channel.
+per_commitment_secret+:: Used to generate a revocation key for the previous (old) commitment, effectively revoking it.
+next_per_commitment_point+:: Used to build a revocation_pubkey for the new commitment, so that it can later be revoked.
+next_per_commitment_point+:: Used to build a `revocation_pubkey` for the new commitment, so that it can later be revoked.
[[revocation]]
==== Revoking and Re-Committing
==== Revoking and Recommitting
Let's look at this interaction between Alice and Bob more closely.
Alice is giving Bob the means to create a new commitment. In return, Bob is revoking the old commitment to assure Alice that he won't use it. Alice can only trust the new commitment if she has the revocation key to punish Bob for publishing the old commitment. From Bob's perspective, he can safely "revoke" the old commitment by giving Alice the keys to penalize him, because he has a signature for a new commitment.
Alice is giving Bob the means to create a new commitment. In return, Bob is revoking the old commitment to assure Alice that he won't use it. Alice can only trust the new commitment if she has the revocation key to punish Bob for publishing the old commitment. From Bob's perspective, he can safely revoke the old commitment by giving Alice the keys to penalize him, because he has a signature for a new commitment.
When Bob responds with +revoke_and_ack+, he gives Alice a +per_commitment_secret+. This secret can be used to construct the revocation signing key for the old commitment, which allows Alice to seize all channel funds by exercising a penalty.
As soon as Bob has given this secret to Alice, he *must not* ever broadcast that old commitment. If he does, he will give Alice the opportunity to penalize him by taking the funds. Essentially, Bob is giving Alice the ability to hold him accountable for broadcasting an old commitment and in effect he has "revoked" his ability to use that old commitment.
As soon as Bob has given this secret to Alice, he _must not_ ever broadcast that old commitment. If he does, he will give Alice the opportunity to penalize him by taking the funds. Essentially, Bob is giving Alice the ability to hold him accountable for broadcasting an old commitment and in effect he has revoked his ability to use that old commitment.
Once Alice has received the +revoke_and_ack+ from Bob she can be sure that Bob cannot broadcast the old commitment without being penalized. She now has the keys necessary to create a penalty transaction if Bob broadcasts an old commitment.
Once Alice has received the +revoke_and_ack+ from Bob, she can be sure that Bob cannot broadcast the old commitment without being penalized. She now has the keys necessary to create a penalty transaction if Bob broadcasts an old commitment.
[[revocation_secret_derivation]]
==== Cheating and Penalty in Practice
In practice, both Alice and Bob have to monitor for "cheating". They are monitoring the Bitcoin blockchain for any commitment transactions related to any of the channels they are operating. If they see a commitment transaction confirmed on-chain they will check to see if it is the most recent commitment. If it is an "old" commitment, they must immediately construct and broadcast a penalty transaction. The penalty transaction spends *both* the +to_local+ and +to_remote+ outputs, closing the channel and sending both balances to the "cheated" channel partner.
In practice, both Alice and Bob have to monitor for cheating. They are monitoring the Bitcoin blockchain for any commitment transactions related to any of the channels they are operating. If they see a commitment transaction confirmed on-chain, they will check to see if it is the most recent commitment. If it is an "old" commitment, they must immediately construct and broadcast a penalty transaction. The penalty transaction spends _both_ the +to_local+ and +to_remote+ outputs, closing the channel and sending both balances to the cheated channel partner.
In order to more easily allow both sides to keep track of the commitment numbers of the passed revoke commitments, each commitment actually _encodes_ the number of the commitment within the lock time and sequence fields in a transition. Within the protocol, this special encoding is referred to as "state hints". Assuming a party knows the current commitment number, they're able to use the state hints to easily recognize if a broadcasted commitment was a revoked one, and if so, which commitment number was breached, as that number is used to easily look up which revocation secret should be used in the revocation secret tree (shachain).
To more easily allow both sides to keep track of the commitment numbers of the passed revoke commitments, each commitment actually _encodes_ the number of the commitment within the lock time and sequence fields in a transition. Within the protocol, this special encoding is referred to as _state hints_. Assuming a party knows the current commitment number, they're able to use the state hints to easily recognize if a broadcasted commitment was a revoked one, and if so, which commitment number was breached, as that number is used to easily look up which revocation secret should be used in the revocation secret tree (shachain).
Rather than encode the state hint in plain sight, an _obfuscated_ state hint is used in its place. This obfuscation is achieved by first XOR'ing the current commitment number with a set of random bytes generated deterministically using the funding public keys of both sides of the channel. A total of 6 bytes across the lock time and sequence (24 bits of the locktime and 24 bits of the sequence) are used to encode the state hint within the commitment transaction, so 6 random bytes are needed to use for XOR'ing. To obtain these 6 bytes, both sides obtain the SHA-256 hash of the initiator's funding key concatenated to the responder's funding key. Before encoding the current commitment height, the integer is XOR'd with this state hint obfuscater, and then encoded in the lower 24 bits of the locktime, and the upper 64 bits of the sequence.
Rather than encode the state hint in plain sight, an _obfuscated_ state hint is used in its place. This obfuscation is achieved by first XORing the current commitment number with a set of random bytes generated deterministically using the funding public keys of both sides of the channel. A total of 6 bytes across the lock time and sequence (24 bits of the locktime and 24 bits of the sequence) are used to encode the state hint within the commitment transaction, so 6 random bytes are needed to use for XORing. To obtain these 6 bytes, both sides obtain the SHA-256 hash of the initiator's funding key concatenated to the responder's funding key. Before encoding the current commitment height, the integer is XORed with this state hint obfuscater, and then encoded in the lower 24 bits of the locktime, and the upper 64 bits of the sequence.
Let's review our channel between Alice and Bob and show a specific example of a penalty transaction. In <<competing_commitments_2>> we see the four commitments on Alice and Bob's channel. Alice has made three payments to Bob:
@ -656,11 +656,11 @@ Let's review our channel between Alice and Bob and show a specific example of a
.Revoked and current commitments
image::images/mtln_0712.png[Revoked and current commitments]
With each commitment, Alice has revoked the previous (older) commitment. The current state of the channel and the correct balance is represented by Commitment #3. All previous commitments have been revoked and Bob has the keys necessary to issue penalty transactions against them, in case Alice tries to broadcast one of them.
With each commitment, Alice has revoked the previous (older) commitment. The current state of the channel and the correct balance is represented by Commitment #3. All previous commitments have been revoked, and Bob has the keys necessary to issue penalty transactions against them, in case Alice tries to broadcast one of them.
Alice might have an incentive to cheat, because all the previous commitment transactions would give her a higher proportion of the channel balance than she is entitled. Let's say for example that Alice tried to broadcast Commitment #1. That commitment transaction would pay Alice 70,000 satoshis and Bob 70,000 satoshis. If Alice was able to broadcast and spend her to_local output she would effectively be stealing 30,000 satoshis from Bob by rolling back her last two payments to Bob.
Alice might have an incentive to cheat because all the previous commitment transactions would give her a higher proportion of the channel balance than she is entitled to. Let's say for example that Alice tried to broadcast Commitment #1. That commitment transaction would pay Alice 70,000 satoshis and Bob 70,000 satoshis. If Alice was able to broadcast and spend her to_local output she would effectively be stealing 30,000 satoshis from Bob by rolling back her last two payments to Bob.
Alice decides to take a huge risk and broadcast the revoked Commitment #1, to steal 30,000 satoshis from Bob. In <<cheating_commitment>> we see Alice's old commitment that she broadcasts to the Bitcoin blockchain:
Alice decides to take a huge risk and broadcast the revoked Commitment #1, to steal 30,000 satoshis from Bob. In <<cheating_commitment>> we see Alice's old commitment that she broadcasts to the Bitcoin blockchain.
[[cheating_commitment]]
.Alice cheating
@ -670,7 +670,7 @@ As you can see, Alice's old commitment has two outputs, one paying herself 70,00
Unfortunately for Alice, Bob's node is diligently monitoring the Bitcoin blockchain and sees an old commitment transaction broadcast and (eventually) confirmed on-chain.
Bob's node will immediately broadcast a penalty transaction. Since this old commitment was revoked by Alice, Bob has the +per_commitment_secret+ that Alice sent him. He uses that secret to construct a signature for the +revocation_pubkey+. While Alice has to wait for 432 blocks, Bob can spend *both* outputs immediately. He can spend the +to_remote+ output with his private keys, because it was meant to pay him anyway. He can also spend the output meant for Alice with a signature from the revocation key. His node broadcasts the penalty transaction shown in <<penalty_transaction>>, below:
Bob's node will immediately broadcast a penalty transaction. Since this old commitment was revoked by Alice, Bob has the +per_commitment_secret+ that Alice sent him. He uses that secret to construct a signature for the +revocation_pubkey+. While Alice has to wait for 432 blocks, Bob can spend _both_ outputs immediately. He can spend the +to_remote+ output with his private keys because it was meant to pay him anyway. He can also spend the output meant for Alice with a signature from the revocation key. His node broadcasts the penalty transaction shown in <<penalty_transaction>>.
[[penalty_transaction]]
.Cheating and penalty
@ -684,11 +684,11 @@ You may have noticed there is a special situation that needs to be dealt with: i
=== Closing the Channel (Cooperative Close)
So far we've looked at the commitment transactions as one possible way to close a channel, unilaterally. This type of channel closure is not ideal, since it forces a timelock on the channel partner that uses it.
So far we've looked at the commitment transactions as one possible way to close a channel, unilaterally. This type of channel closure is not ideal because it forces a timelock on the channel partner that uses it.
A better way to close a channel is a cooperative close. In a cooperative close, the two channel partners negotiate a final commitment transaction called the _closing transaction_ that pays each party their balance immediately to the destination wallet of their choice. Then, the partner that initiated the channel closing flow will broadcast the closing transaction.
The closing message flow is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#channel-close[BOLT #2-Peer Protocol-Channel Close] and is shown in <<closing_message_flow>> below:
The closing message flow is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#channel-close[BOLT #2-Peer Protocol-Channel Close] and is shown in <<closing_message_flow>>.
[[closing_message_flow]]
.The channel close message flow
@ -710,19 +710,19 @@ Channel closing starts with one of the two channel partners sending the +shutdow
+channel_id+:: The channel identifier for the channel we want to close
+len+:: The length of the script of the destination wallet that this channel partner wants to receive their balance.
+scriptpubkey+:: A Bitcoin script of the destination wallet, in one of the "standard" Bitcoin address formats (P2PKH, P2SH, P2WPKH, P2WSH etc.)
+scriptpubkey+:: A Bitcoin script of the destination wallet, in one of the "standard" Bitcoin address formats (P2PKH, P2SH, P2WPKH, P2WSH, etc.)
Let's say Alice sends the +shutdown+ message to Bob to close their channel. Alice will specify a Bitcoin script that corresponds to the Bitcoin address of her wallet. She's telling Bob - let's make a closing transaction that pays my balance to this wallet.
Let's say Alice sends the +shutdown+ message to Bob to close their channel. Alice will specify a Bitcoin script that corresponds to the Bitcoin address of her wallet. She's telling Bob: let's make a closing transaction that pays my balance to this wallet.
Bob will respond with his own +shutdown+ message indicating that he agrees to cooperatively close the channel. His +shutdown+ message includes the script for his wallet address.
Now both Alice and Bob have each other's preferred wallet address, they can construct identical closing transactions to settle the channel balance.
Now both Alice and Bob have each other's preferred wallet address, and they can construct identical closing transactions to settle the channel balance.
==== The closing_signed Message
Assuming the channel has no outstanding commitments or updates and the channel partners have exchanged the +shutdown+ messages shown in the previous section, they can now finish this cooperative close.
The *funder* of the channel (Alice in our example) starts by sending a +closing_signed+ message to Bob. This message proposes a transaction fee for the on-chain transaction, and Alice's signature (the 2-of-2 multisig) for the closing transaction. The +closing_signed+ message is shown here:
The _funder_ of the channel (Alice in our example) starts by sending a +closing_signed+ message to Bob. This message proposes a transaction fee for the on-chain transaction, and Alice's signature (the 2-of-2 multisig) for the closing transaction. The +closing_signed+ message is shown here:
[[closing_signed_message]]
.The closing_signed message
@ -744,11 +744,11 @@ Once Alice receives a +closing_signed+ message with the same fee as the one she
==== The Cooperative Close Transaction
The cooperative close transaction looks similar to the last commitment transaction that Alice and Bob had agreed on. However, unlike the last commitment transaction, it does not have timelocks or penalty revocation keys in the outputs. Since both parties cooperate to produce this transaction and they won't be making any further commitments, there is no need for the asymmetric, delayed and revocable elements in this transaction.
The cooperative close transaction looks similar to the last commitment transaction that Alice and Bob had agreed on. However, unlike the last commitment transaction, it does not have timelocks or penalty revocation keys in the outputs. Since both parties cooperate to produce this transaction and they won't be making any further commitments, there is no need for the asymmetric, delayed, and revocable elements in this transaction.
Typically the addresses used in this cooperative close transaction are generated freshly for each channel being closed. However, it's also possible for both sides to _lock in_ a "delivery" address to be used to send their cooperatively settled funds to. Within the TLV namespace of both the `open_channel` and `accept_channel` messages, both sides are free to specify an "upfront shutdown script". Commonly, this address is derived from keys that reside in cold storage. This practice serves to increase the security of channels: if a channel partner is somehow hacked, then the hacker isn't able to cooperatively close the channel using an address they control. Instead, the uncompromised honest channel partner will refuse to cooperate on a channel closure if the specified upfront shutdown address isn't used. This feature effectively creates a "closed loop", restricting the flow of funds out of a given channel.
Typically the addresses used in this cooperative close transaction are generated freshly for each channel being closed. However, it's also possible for both sides to _lock in_ a "delivery" address to be used to send their cooperatively settled funds to. Within the TLV namespace of both the `open_channel` and `accept_channel` messages, both sides are free to specify an "upfront shutdown script." Commonly, this address is derived from keys that reside in cold storage. This practice serves to increase the security of channels: if a channel partner is somehow hacked, then the hacker isn't able to cooperatively close the channel using an address they control. Instead, the uncompromised honest channel partner will refuse to cooperate on a channel closure if the specified upfront shutdown address isn't used. This feature effectively creates a "closed loop," restricting the flow of funds out of a given channel.
Alice broadcasts a transaction shown in <<closing_transaction>> below to close the channel:
Alice broadcasts a transaction shown in <<closing_transaction>> to close the channel.
[[closing_transaction]]
.The cooperative close transaction
@ -758,10 +758,10 @@ As soon as this closing transaction is confirmed on the Bitcoin blockchain, the
=== Conclusion
In this section we looked at payment channels in much more detail. We examined three message flows used by Alice and Bob to negotiate funding, commitments and closing of the channel. We also showed the structure of the funding, commitment and closing transactions and looked at the revocation and penalty mechanisms.
In this section we looked at payment channels in much more detail. We examined three message flows used by Alice and Bob to negotiate funding, commitments, and closing of the channel. We also showed the structure of the funding, commitment, and closing transactions and looked at the revocation and penalty mechanisms.
As we will see in the next few chapters, HTLCs are used even for local payments between channel partners. They are not necessary, but the protocol is much simpler if local (one channel) and routed (many channels) payments are done in the same way.
In a single payment channel, the number of payments per second is only bound by the network capacity between Alice and Bob. As long as the channel partners are able to send a few bytes of data back and forth to agree to a new channel balance they have effectively made a payment. This is why we can achieve a much greater throughput of payments on the Lighting Network (off-chain) than the transaction throughput that can be handled by the Bitcoin blockchain (on-chain).
In a single payment channel, the number of payments per second is only bound by the network capacity between Alice and Bob. As long as the channel partners are able to send a few bytes of data back and forth to agree to a new channel balance, they have effectively made a payment. This is why we can achieve a much greater throughput of payments on the LN (off-chain) than the transaction throughput that can be handled by the Bitcoin blockchain (on-chain).
In the next few sections we will discuss routing, HTLCs and their use in channel operations.
In the next few chapters we will discuss routing, HTLCs, and their use in channel operations.
In this chapter we will finally unpack how payment channels can be connected to form a _network of payment channels_ via a process called _routing_. Specifically, we will look at the first part of the routing layer, the Atomic & Trustless Multihop Payment protocol. This is highlighted by a double outline in the protocol suite, shown in <<LN_protocol_routing_highlight>>:
In this chapter we will finally unpack how payment channels can be connected to form a network of payment channels via a process called _routing_. Specifically, we will look at the first part of the routing layer, the Atomic and Trustless Multihop Contracts protocol. This is highlighted by a double outline in the protocol suite, shown in <<LN_protocol_routing_highlight>>.
[[LN_protocol_routing_highlight]]
.The Lightning Network Protocol Suite
@ -14,13 +14,13 @@ In this section we will examine routing from the perspective of Dina, a gamer wh
The innovation of routed payment channels allows Dina to receive tips without maintaining a separate channel with every one of her fans who want to tip her.
As long as there exists a path of well-funded channels from that viewer to Dina, she will be able to receive payment from that fan.
In <<dina_routing_diagram>> we see a possible network layout created by various payment channels between Lightning nodes. Everyone in this diagram can send Dina a payment by constructing a path. Imagine "Fan 4" wants to send Dina a payment, do you see the path that could allow that to happen? Fan 4 could route a payment to Dina via Fan 3, Bob, and Chan. Similarly, Alice could route a payment to Dina via Bob and Chan.
In <<dina_routing_diagram>> we see a possible network layout created by various payment channels between Lightning nodes. Everyone in this diagram can send Dina a payment by constructing a path. Imagine Fan 4 wants to send Dina a payment. Do you see the path that could allow that to happen? Fan 4 could route a payment to Dina via Fan 3, Bob, and Chan. Similarly, Alice could route a payment to Dina via Bob and Chan.
[[dina_routing_diagram]]
.Fans connected indirectly to Dina on the Lightning Network
image::images/mtln_0802.png["Fans connected indirectly to Dina on the Lightning Network"]
The nodes along the path from the fan to Dina are intermediaries and called "routing nodes" in the context of routing a payment. There is no functional difference between the "routing nodes" and the nodes operated by Dina's fans. Any Lightning node is capable of routing payments across its payment channels.
The nodes along the path from the fan to Dina are intermediaries called _routing nodes_ in the context of routing a payment. There is no functional difference between the routing nodes and the nodes operated by Dina's fans. Any Lightning node is capable of routing payments across its payment channels.
Importantly, the routing nodes are unable to steal the funds while routing a payment from a fan to Dina.
Furthermore, routing nodes cannot lose money while participating in the routing process.
@ -32,17 +32,17 @@ This enables fans to use intermediary nodes to pay Dina, without leaking private
This process of connecting a series of payment channels with end-to-end security, and the incentive structure for nodes to _forward_ payments, is one of the key innovations of the Lightning Network.
In this chapter, we'll dive into the mechanism of routing in the Lightning Network, detailing the precise manner in-which payments flow through the network. First, we will clarify the concept of "routing" and compare it to that of "path finding", as these are often confused and used interchangeably. Next, we will construct the fairness protocol: An atomic, trustless, multihop protocol used to route payments. To demonstrate how this fairness protocol works, we will be using a physical equivalent of transferring gold coins between 4 people. Finally, we will look at the atomic, trustless, multihop protocol implementation currently used in the Lightning Network, which is called a Hash Time-Locked Contract (HTLC).
In this chapter, we'll dive into the mechanism of routing in the Lightning Network, detailing the precise manner inwhich payments flow through the network. First, we will clarify the concept of routing and compare it to that of path finding because these are often confused and used interchangeably. Next, we will construct the fairness protocol: an atomic, trustless, multihop protocol used to route payments. To demonstrate how this fairness protocol works, we will be using a physical equivalent of transferring gold coins between four people. Finally, we will look at the atomic, trustless, multihop protocol implementation currently used in the Lightning Network, which is called a Hash Time-Locked Contract (HTLC).
=== Routing vs. Path Finding
=== Routing Versus Path Finding
It's important to note that we separate the concept of _routing_ from the concept of _path finding_. These two concepts are often confused and the term "routing" is often used to describe both concepts. Let's remove the ambiguity, before we proceed any further.
It's important to note that we separate the concept of _routing_ from the concept of _path finding_. These two concepts are often confused, and the term _routing_ is often used to describe both concepts. Let's remove the ambiguity, before we proceed any further.
Path Finding, which is covered in <<path_finding>> is the process of finding and choosing a contiguous path made of payment channels which connects the sender A to the recipient B. The sender of a payment does the path finding, by examining the _channel graph_ which they have assembled from channel announcements gossiped by other nodes.
Path finding, which is covered in <<path_finding>>, is the process of finding and choosing a contiguous path made of payment channels that connects sender A to recipient B. The sender of a payment does the path finding by examining the _channel graph_ that they have assembled from channel announcements gossiped by other nodes.
Routing refers to the series of interactions across the network that attempt to forward a payment from some point A to another point B, across the path previously selected by path finding. Routing is the active process of sending a payment on a path, which involves the cooperation of all the intermediary nodes along that path.
An important rule of thumb is that it's possible for a _path_ to exist between Alice and Bob (perhaps even more than one), yet there may not be an active _route_ on which to send the payment. One example is the scenario where all the nodes connecting Alice and Bob are currently off-line. In this example, one can examine the _channel graph_ and connect a series of payment channels from Alice to Bob, hence a _path_ exists. However, as the intermediary nodes are offline, the payment cannot be sent and so no _route_ exists.
An important rule of thumb is that it's possible for a _path_ to exist between Alice and Bob (perhaps even more than one), yet there may not be an active _route_ on which to send the payment. One example is the scenario in which all the nodes connecting Alice and Bob are currently offline. In this example, one can examine the channel graph and connect a series of payment channels from Alice to Bob, hence a _path_ exists. However, because the intermediary nodes are offline, the payment cannot be sent and so no _route_ exists.
=== Creating a Network of Payment Channels
@ -50,7 +50,7 @@ Before we dive into the concept of an atomic trustless multihop payment, let's w
Let's return to Alice who, in previous chapters, purchased a coffee from Bob with whom she has an open channel.
Now Alice is watching a live stream from Dina, the gamer, and wants to send Dina a tip of 50,000 satoshis via the Lightning Network. But Alice has no direct channel with Dina. What can Alice do?
Alice could open a direct channel with Dina, however that would require liquidity and on-chain fees which could be more than the value of the tip itself. Instead, Alice can use her existing open channels to send a tip to Dina _without_ the need to open a channel directly with Dina. This is possible, as long as there exists some path of channels from Alice to Dina with sufficient capacity to route the tip.
Alice could open a direct channel with Dina; however, that would require liquidity and on-chain fees which could be more than the value of the tip itself. Instead, Alice can use her existing open channels to send a tip to Dina _without_ the need to open a channel directly with Dina. This is possible, as long as there exists some path of channels from Alice to Dina with sufficient capacity to route the tip.
As you can see in <<routing_network>>, Alice has an open channel with Bob, the coffee shop owner. Bob, in turn, has an open channel with the software developer Chan who helps him with the point of sale system he uses in his coffee shop. Chan is also the owner of a large software company which develops the game that Dina plays, and they already have an open channel which Dina uses to pay for the game's license and in-game items.
@ -59,7 +59,7 @@ As you can see in <<routing_network>>, Alice has an open channel with Bob, the c
image::images/mtln_0803.png["A network of payment channels between Alice and Dina"]
It's possible to trace a _path_ from Alice to Dina that uses Bob and Chan as intermediary routing nodes.
Alice can then craft a _route_ from this outlined path, and use it to send a tip of a few thousand satoshis to Dina, with the payment being _forwarded_ by Bob and Chan.
Alice can then craft a _route_ from this outlined path and use it to send a tip of a few thousand satoshis to Dina, with the payment being _forwarded_ by Bob and Chan.
Essentially, Alice will pay Bob, who will pay Chan, who will pay Dina. No direct channel from Alice to Dina is required.
The main challenge is to do this in a way that prevents Bob and Chan from stealing the money that Alice wants delivered to Dina.
@ -68,7 +68,7 @@ The main challenge is to do this in a way that prevents Bob and Chan from steali
To understand how the Lightning Network protects the payment while being routed, we can compare it to an example of routing physical payments with gold coins in the real world.
Assume Alice wants to give 10 gold coins to Dina, but does not have direct access to Dina. However, Alice knows Bob, who knows Chan, who knows Dina and so she decides to ask Bob and Chan for help. This is shown in <<alice_dina_routing_1>>.
Assume Alice wants to give 10 gold coins to Dina, but does not have direct access to Dina. However, Alice knows Bob, who knows Chan, who knows Dina, so she decides to ask Bob and Chan for help. This is shown in <<alice_dina_routing_1>>.
[[alice_dina_routing_1]]
.Alice wants to pay Dina 10 gold coins
@ -77,7 +77,7 @@ image::images/mtln_0804.png[]
Alice can pay Bob to pay Chan to pay Dina, but how does she make sure that Bob or Chan don't run off with the coins after receiving them?
In the physical world contracts could be used for safely carrying out a series of payments.
Alice could negotiate a contract with Bob which reads:
Alice could negotiate a contract with Bob which reads as shown in <<alice_bob_contract_1>>.
[[alice_bob_contract_1]]
====
@ -88,7 +88,7 @@ While this contract is nice in the abstract, in the real world, Alice runs the r
Even if Bob is caught and prosecuted, Alice faces the risk that he might be bankrupt and be unable to return her 10 gold coins.
Assuming these issues are magically solved, it's still unclear how to leverage such a contract to achieve our desired outcome: getting the coins delivered to Dina.
Let's improve our contract, to incorporate these considerations, as follows:
Let's improve our contract to incorporate these considerations, as given in <<alice_bob_contract_2>>.
[[alice_bob_contract_2]]
====
@ -98,13 +98,13 @@ _I Alice will reimburse you Bob with 10 gold coins if you can prove to me (for e
You might ask yourself why should Bob sign such a contract.
He has to pay Chan but ultimately gets nothing out of the exchange, and he runs the risk that Alice might not reimburse him. Bob could offer Chan a similar contract to pay Dina, but similarly Chan has no reason to accept it either.
Even putting aside the risk, Bob and Chan must _already_ have 10 gold coins to send, otherwise they wouldn't be able to participate in the contract.
Even putting aside the risk, Bob and Chan must _already_ have 10 gold coins to send; otherwise, they wouldn't be able to participate in the contract.
Thus Bob and Chan face both risk and opportunity cost for agreeing to this contract, and they would need to be compensated to accept it.
Alice can then make this attractive to both Bob and Chan, by offering them fees of 1 gold coin each, if they transmit her payment to Dina.
Alice can then make this attractive to both Bob and Chan by offering them fees of 1 gold coin each, if they transmit her payment to Dina.
The contract would then read:
The contract would then read as given in <<alice_bob_contract_3>>.
[[alice_bob_contract_3]]
====
@ -118,13 +118,13 @@ The difference of 1 gold coin is the fee that Bob will earn for helping out with
.Alice pays Bob, Bob pays Chan, Chan pays Dina
image::images/mtln_0805.png[]
As there is still the issue of trust and the risk that either Alice or Bob won't honor the contract, all parties decide to use an escrow service.
Because there is still the issue of trust and the risk that either Alice or Bob won't honor the contract, all parties decide to use an escrow service.
At the start of the exchange, Alice could "lock up" these 12 gold coins in escrow that will only be paid to Bob once he proves that he's paid 11 gold coins to Chan.
This escrow service is an idealized one, which does not introduce other risks (e.g. counterparty risk). Later we will see how we can replace the escrow with a Bitcoin smart contract. Let's assume for now that everyone trusts this escrow service.
This escrow service is an idealized one, which does not introduce other risks (e.g., counterparty risk). Later we will see how we can replace the escrow with a Bitcoin smart contract. Let's assume for now that everyone trusts this escrow service.
In the Lightning Network, the receipt (proof of payment) could take the form of a secret that only Dina knows.
In practice, this secret would be a large random number that is large enough to prevent others from guessing it (typically _very, very_ large number, encoded using 256 bits!).
In practice, this secret would be a random number that is large enough to prevent others from guessing it (typically a _very, very_ large number, encoded using 256 bits!).
Dina generates this secret value +R+ from a random number generator.
@ -132,12 +132,12 @@ The secret could then be committed to the contract by including the SHA256 hash
latexmath:[\(H = SHA256(R)\)]
We call this hash of the payment's secret the payment hash.
The secret which "unlocks" the payment is called the payment secret.
We call this hash of the payment's secret the _payment hash_.
The secret that "unlocks" the payment is called the _payment secret_.
For now, we keep things simple and assume that Dina's secret is simply the text line: `+Dinas secret+`. This secret message is called the _payment secret_ or _payment pre-image_.
For now, we keep things simple and assume that Dina's secret is simply the text line: `Dinas secret`. This secret message is called the _payment secret_ or _payment pre-image_.
In order to "commit" to this secret, Dina computes the SHA256 hash which when encoded in hex, can be displayed as: `+0575965b3b44be51e8057d551c4016d83cb1fba9ea8d6e986447ba33fe69f6b3+`.
To "commit" to this secret, Dina computes the SHA256 hash, which when encoded in hexadecimal, can be displayed as follows: `0575965b3b44be51e8057d551c4016d83cb1fba9ea8d6e986447ba33fe69f6b3`.
To facilitate Alice's payment, Dina will create the payment secret and the payment hash and send the payment hash to Alice. In <<alice_dina_routing_3>> we see that Dina sends the payment hash to Alice via some external channel (dashed line), such as an email or text message:
@ -145,13 +145,13 @@ To facilitate Alice's payment, Dina will create the payment secret and the payme
.Dina sends the hashed secret to Alice
image::images/mtln_0806.png["Dina sends the hashed secret to Alice"]
Alice doesn't know the secret but she can rewrite her contract to use the hash of the secret as a proof of payment:
Alice doesn't know the secret, but she can rewrite her contract to use the hash of the secret as a proof of payment, as shown in <<alice_bob_contract_4>>.
[[alice_bob_contract_4]]
====
_I Alice will reimburse you Bob with 12 gold coins if you can show me a valid message that hashes to:`+057596...+`.
_I Alice will reimburse you Bob with 12 gold coins if you can show me a valid message that hashes to:`057596...`.
You can acquire this message by setting up a similar contract with Chan who has to set up a similar contract with Dina.
In order to assure you that you will be reimbursed I will provide the 12 gold coins to a trusted escrow before you set up your next contract._
To assure you that you will be reimbursed, I will provide the 12 gold coins to a trusted escrow before you set up your next contract._
====
This new contract now protects Alice from Bob not forwarding to Chan, protects Bob from not being reimbursed by Alice, and ensures that there will be proof that Dina was ultimately paid via the hash of Dina's secret.
@ -161,21 +161,21 @@ After Bob and Alice agree to the contract, and Bob receives the message from the
Note that since Bob is taking a service fee of 1 coin, he will only forward 11 gold coins to Chan once Chan shows proof that he has paid Dina.
Similarly, Chan will also demand a fee and will expect to receive 11 gold coins once he has proved that he has paid Dina the promised 10 gold coins.
Bob's contract with Chan will read:
Bob's contract with Chan will read as shown in <<alice_bob_contract_5>>.
[[alice_bob_contract_5]]
====
_I Bob will reimburse you Chan with 11 gold coins if you can show me a valid message that hashes to:`+057596...+`.
_I Bob will reimburse you Chan with 11 gold coins if you can show me a valid message that hashes to:`057596...`.
You can acquire this message by setting up a similar contract with Dina.
In order to assure you that you will be reimbursed I will provide the 11 gold coins to an trusted escrow before you set up your next contract._
To assure you that you will be reimbursed, I will provide the 11 gold coins to a trusted escrow before you set up your next contract._
====
Once Chan gets the message from the escrow that Bob has deposited the 11 gold coins, Chan sets up a similar contract with Dina:
Once Chan gets the message from the escrow that Bob has deposited the 11 gold coins, Chan sets up a similar contract with Dina, as given in <<alice_bob_contract_6>>.
[[alice_bob_contract_6]]
====
_I Chan will reimburse you Dina with 10 gold coins if you can show me a valid message that hashes to:`+057596...+`.
In order to assure you that you will be reimbursed after revealing the secret I will provide the 10 gold coins to an trusted escrow._
_I Chan will reimburse you Dina with 10 gold coins if you can show me a valid message that hashes to:`057596...`.
To assure you that you will be reimbursed after revealing the secret, I will provide the 10 gold coins to a trusted escrow._
====
Everything is now in place.
@ -184,9 +184,9 @@ Bob has a contract with Chan and has placed 11 gold coins in escrow.
Chan has a contract with Dina and has placed 10 gold coins in escrow.
It is now up to Dina to reveal the secret, which is the pre-image to the hash she has established as proof of payment.
Dina now sends +"Dinas secret"+ to Chan.
Dina now sends +Dinas secret+ to Chan.
Chan checks that +"Dinas secret" hashes to +057596...+. Chan now has proof of payment and so instructs the escrow service to release the 10 gold coins to Dina.
Chan checks that +Dinas secret+ hashes to +057596...+. Chan now has proof of payment and so instructs the escrow service to release the 10 gold coins to Dina.
Chan now provides the secret to Bob. Bob checks it and instructs the escrow service to release the 11 gold coins to Chan.
@ -205,11 +205,11 @@ So while no one can steal money from Alice, everyone would still have their mone
Luckily, this can be resolved by adding a deadline to the contract.
We could amend the contract so that if it is not fulfilled by a certain deadline, then the contract expires and the escrow service returns the money to the person who made the original deposit.
We call this deadline a "time lock".
We call this deadline a _time lock_.
The deposit is locked with the escrow service for a certain amount of time, and is eventually released even if no proof of payment was provided.
The deposit is locked with the escrow service for a certain amount of time and is eventually released even if no proof of payment was provided.
In order to factor this in, the contract between Alice and Bob is once again amended with a new clause:
To factor this in, the contract between Alice and Bob is once again amended with a new clause, as given in <<alice_bob_contract_7>>.
[[alice_bob_contract_7]]
====
@ -220,7 +220,7 @@ If Bob does not provide the secret by this time, Alice's deposit will be refunde
Bob, of course, now has to make sure he receives the proof of payment within 24 hours.
Even if he successfully pays Chan, if he receives the proof of payment later than 24 hours he will not be reimbursed. To remove that risk, Bob must give Chan an even shorter deadline.
In turn, Bob will alter his contract with Chan in the following way:
In turn, Bob will alter his contract with Chan as shown in <<alice_bob_contract_8>>.
[[alice_bob_contract_8]]
====
@ -228,7 +228,7 @@ _Chan has 22 hours to show the secret after the contract was signed.
If he does not provide the secret by this time, Bob's deposit will be refunded by the escrow service and the contract becomes invalid._
====
As you might have guessed, Chan will also alter his contract with Dina:
As you might have guessed, Chan will also alter his contract with Dina, as we see in <<alice_bob_contract_9>>.
[[alice_bob_contract_9]]
====
@ -241,15 +241,15 @@ Either the contract fails or succeeds, there's no middle ground.
In the context of the Lightning Network, we call this "all or nothing" property _atomicity_.
As long as the escrow is trustworthy and faithfully performs its duty, then no party will have their coins stolen in the process.
As long as the escrow is trustworthy and faithfully performs its duty, no party will have their coins stolen in the process.
The pre-condition to this _route_ working at all, is that all parties in the path have enough money to satisfy the required series of deposits.
The precondition to this _route_ working at all is that all parties in the path have enough money to satisfy the required series of deposits.
While this seems like a minor detail we will see later in this chapter that this requirement is actually one of the more difficult issues for Lightning Network nodes.
While this seems like a minor detail, we will see later in this chapter that this requirement is actually one of the more difficult issues for LN nodes.
It becomes progressively more difficult as the size of the payment increases.
Furthermore, the parties cannot use their money while it is locked in escrow.
Thus users forwarding payments face an opportunity cost for locking the money, which is ultimately reimbursed through routing fees, as we saw in the example above.
Thus users forwarding payments face an opportunity cost for locking the money, which is ultimately reimbursed through routing fees, as we saw in the preceding example.
Now that we've seen a physical payment routing example, we will see how this can be implemented on the Bitcoin blockchain, without any need for third-party escrow. To do this we will be setting up the contracts between the participants using Bitcoin Script. We replace the third-party escrow with _smart contracts_ that implement a fairness protocol. Let's break that concept down and implement it!
@ -257,67 +257,67 @@ Now that we've seen a physical payment routing example, we will see how this can
As we saw in the first chapter of this book, the innovation of Bitcoin is the ability to use cryptographic primitives to implement a fairness protocol that substitutes trust in third parties (intermediaries), with a trusted protocol.
In our gold coin example, we needed an "escrow" service in order to prevent any one of the parties from reneging on their obligations. The innovation of cryptographic fairness protocols allows us to replace the escrow service with a protocol.
In our gold coin example, we needed an escrow service to prevent any one of the parties from reneging on their obligations. The innovation of cryptographic fairness protocols allows us to replace the escrow service with a protocol.
The properties of the fairness protocol we want to create are:
Trustless Operation:: The participants in a routed payment do not need to trust each other, or any intermediary or third party. Instead, they trust the protocol to protect them from cheating.
Trustless operation:: The participants in a routed payment do not need to trust each other, or any intermediary or third party. Instead, they trust the protocol to protect them from cheating.
Atomicity:: The payment is fully executed, or it fails and everyone is refunded. There is no possibility of an intermediary collecting a routed payment and not forwarding it to the next hop. Thus, the intermediaries can't cheat or steal.
Atomicity:: Either the payment is fully executed, or it fails and everyone is refunded. There is no possibility of an intermediary collecting a routed payment and not forwarding it to the next hop. Thus, the intermediaries can't cheat or steal.
Multihop:: The security of the system extends end-to-end for payments routed through multiple payment channels, just as it is for a payment between the two ends of a single payment channel.
An optional, additional property, is the ability to split payments into multiple parts while maintaining atomicity for the entire payment. These are called _Multi-Part Payments (MPP)_ and are explored further in <<mpp>>.
An optional, additional property is the ability to split payments into multiple parts while maintaining atomicity for the entire payment. These are called _Multi-Part Payments_ (_MPP_) and are explored further in <<mpp>>.
Bitcoin Script is flexible enough that there are dozens of ways to implement a fairness protocol that has the properties of atomicity, trustless operation and multihop security. Choosing a specific implementation is dependent on certain tradeoffs between privacy, efficiency and complexity.
Bitcoin Script is flexible enough that there are dozens of ways to implement a fairness protocol that has the properties of atomicity, trustless operation, and multihop security. Choosing a specific implementation is dependent on certain trade-offs between privacy, efficiency, and complexity.
The fairness protocol for routing used in the Lightning Network today is called a Hash Time-Locked Contract (HTLC). HTLCs use a hash pre-image as the secret that unlocks a payment, as we saw in the gold coin example in this chapter. The recipient of a payment generates a random secret number and calculates its hash. The hash becomes the condition of payment and once the secret is revealed, all the participants can redeem their incoming payments. HTLCs offer atomicity, trustless operation and multihop security.
The fairness protocol for routing used in the Lightning Network today is called a Hash Time-Locked Contract (HTLC). HTLCs use a hash pre-image as the secret that unlocks a payment, as we saw in the gold coin example in this chapter. The recipient of a payment generates a random secret number and calculates its hash. The hash becomes the condition of payment, and once the secret is revealed, all the participants can redeem their incoming payments. HTLCs offer atomicity, trustless operation, and multihop security.
Another proposed mechanism for implementing routing is a _Point Time-Locked Contract (PTLC)_. PTLCs also achieve atomicity, trustless operation and multihop security, but do so with increased efficiency and better privacy. Efficient implementation of PTLCs depends on a new digital signature algorithm called _Schnorr signatures_, which is expected to be activated in Bitcoin in 2021.
Another proposed mechanism for implementing routing is a _Point Time-Locked Contract_ (_PTLC_). PTLCs also achieve atomicity, trustless operation, and multihop security, but do so with increased efficiency and better privacy. Efficient implementation of PTLCs depends on a new digital signature algorithm called _Schnorr signatures_, which is expected to be activated in Bitcoin in 2021.
=== Revisiting Our Example
Let's revisit our example from the first part of this chapter. Alice wants to "tip" Dina, with a Lightning payment. Let's say Alice wants to send Dina 50,000 satoshis as a tip.
Let's revisit our example from the first part of this chapter. Alice wants to tip Dina with a Lightning payment. Let's say Alice wants to send Dina 50,000 satoshis as a tip.
For Alice to pay Dina, Alice will need Dina's node to generate a Lightning invoice. We will discuss this in more detail in <<invoices>>. For now, let's assume that Dina has a website that can produce a Lightning invoice for tips.
[TIP]
====
Lightning payments can be sent without an invoice, using a feature called _keysend_, which we will discuss in more detail in <<keysend>>. For now, we will explain the simpler payment flow using an invoice.
Lightning payments can be sent without an invoice using a feature called _keysend_, which we will discuss in more detail in <<keysend>>. For now, we will explain the simpler payment flow using an invoice.
====
Alice visits Dina's site, enters the amount of 50,000 satoshis in a form and in response Dina's Lightning node generates a payment request for 50,000 satoshis in the form of a Lightning invoice. This interaction takes place over the web and outside the Lightning network, as shown in <<alice_dina_invoice_1>>:
Alice visits Dina's site, enters the amount of 50,000 satoshis in a form, and in response Dina's Lightning node generates a payment request for 50,000 satoshis in the form of a Lightning invoice. This interaction takes place over the web and outside the Lightning Network, as shown in <<alice_dina_invoice_1>>.
[[alice_dina_invoice_1]]
.Alice requests an invoice from Dina's website
image::images/mtln_0807.png["Alice requests an invoice from Dina's website"]
As we saw in previous examples, we assume that Alice does not have a direct payment channel to Dina. Instead, Alice has a channel to Bob, Bob has a channel to Chan and Chan has a channel to Dina. To pay Dina, Alice must find a path that connects her to Dina. We will discuss that step in more detail in <<path_finding>>. For now, let's assume that Alice is able to gather information about available channels and sees that there is a path from her to Dina, via Bob and Chan.
As we saw in previous examples, we assume that Alice does not have a direct payment channel to Dina. Instead, Alice has a channel to Bob, Bob has a channel to Chan, and Chan has a channel to Dina. To pay Dina, Alice must find a path that connects her to Dina. We will discuss that step in more detail in <<path_finding>>. For now, let's assume that Alice is able to gather information about available channels and sees that there is a path from her to Dina, via Bob and Chan.
[NOTE]
====
Remember how Bob and Chan might expect a small compensation for routing the payment through their nodes? Alice wants to pay Dina 50,000 satoshis, but as you will see in the following sections she will send Bob 50,200 satoshis. The extra 200 satoshis will pay Bob and Chan 100 satohis each, as a routing fee.
Remember how Bob and Chan might expect a small compensation for routing the payment through their nodes? Alice wants to pay Dina 50,000 satoshis, but as you will see in the following sections she will send Bob 50,200 satoshis. The extra 200 satoshis will pay Bob and Chan 100 satoshis each, as a routing fee.
====
Now, Alice's node can construct a Lightning payment. In the next few sections we will see how Alice's node constructs a Hash Time-Locked Contract (HTLCs) to pay Dina and how that HTLC is forwarded along the path from Alice to Dina.
Now, Alice's node can construct a Lightning payment. In the next few sections, we will see how Alice's node constructs a Hash Time-Locked Contract (HTLC) to pay Dina and how that HTLC is forwarded along the path from Alice to Dina.
==== On-Chain vs. Off-Chain Settlement of HTLCs
==== On-Chain Versus Off-Chain Settlement of HTLCs
The purpose of the Lightning Network is to enable _off-chain_ transactions that are trusted just the same as on-chain transactions, because no one can cheat. The reason no one can cheat is because at any time, any of the participants can take their off-chain transactions on-chain. Each off-chain transaction is ready to be submitted to the Bitcoin blockchain at any time. Thus, the Bitcoin blockchain acts as a dispute-resolution and final settlement mechanism if necessary.
The purpose of the Lightning Network is to enable _off-chain_ transactions that are trusted just the same as on-chain transactions because no one can cheat. The reason no one can cheat is because at any time, any of the participants can take their off-chain transactions on-chain. Each off-chain transaction is ready to be submitted to the Bitcoin blockchain at any time. Thus, the Bitcoin blockchain acts as a dispute-resolution and final settlement mechanism if necessary.
The mere fact that any transaction can be taken on-chain at any time is precisely the reason that all those transactions can be kept off-chain. If you know you have recourse, you can continue to cooperate with the other participants and avoid the need for on-chain settlement and extra fees.
In all the examples that follow, we will assume that any of these transactions can be made on-chain at any time. The participants will choose to keep them off-chain, but there is no difference in the functionality of the system other than the higher fees and delay imposed by on-chain mining of the transactions. The example works the same if all the transactions are on-chain or off-chain.
[[htlcs]]
=== Hash Time Locked Contracts (HTLCs)
=== Hash Time-Locked Contracts
In this section we explain how Hash Time Locked Contracts (HTLCs) work.
In this section we explain how HTLCs work.
The first part of a Hash Time-Locked Contract is the "Hash". This refers to the use of a cryptographic hash algorithm to commit to a randomly generated secret. Knowledge of the secret allows redemption of the payment. The cryptographic hash function, guarantees that while it's infeasible for anyone to guess the secret pre-image, it's easy for anyone to verify the hash, and there's only one possible pre-image that resolves the payment condition.
The first part of an HTLC is the _hash_. This refers to the use of a cryptographic hash algorithm to commit to a randomly generated secret. Knowledge of the secret allows redemption of the payment. The cryptographic hash function, guarantees that while it's infeasible for anyone to guess the secret pre-image, it's easy for anyone to verify the hash, and there's only one possible pre-image that resolves the payment condition.
In <<alice_dina_invoice_2>> we see Alice getting a Lightning invoice from Dina. Inside that invoice Dina has encoded a _payment hash_, which is the cryptographic hash of a secret that Dina's node produced. Dina's secret is called the _payment pre-image_. The payment hash acts as an identifier that can be used to route the payment to Dina. The payment pre-image acts as a receipt and proof of payment once the payment is complete.
@ -325,7 +325,7 @@ In <<alice_dina_invoice_2>> we see Alice getting a Lightning invoice from Dina.
.Alice gets a payment hash from Dina
image::images/mtln_0808.png["Alice gets a payment hash from Dina"]
In the Lightning Network, Dina's payment pre-image won't be a phrase like "Dina's secret", but a random number generated by Dina's node. Let's call that random number +R+.
In the Lightning Network, Dina's payment pre-image won't be a phrase like +Dinas secret+ but a random number generated by Dina's node. Let's call that random number +R+.
Dina's node will calculate a cryptographic hash of +R+, such that:
@ -341,14 +341,14 @@ The use of a cryptographic hash function is one element that guarantees _trustle
==== HTLCs in Bitcoin Script
In our gold coin example, Alice had a contract enforced by escrow like this:
In our gold coin example, Alice had a contract enforced by escrow like the one in <<alice_bob_contract_10>>.
[[alice_bob_contract_10]]
====
_Alice will reimburse Bob with 12 gold coins if you can show a valid message that hashes to:_ +0575...f6b3+. _Bob has 24 hours to show the secret after the contract was signed. If Bob does not provide the secret by this time, Alice's deposit will be refunded by the escrow service and the contract becomes invalid._
====
Let's see how we would implement this as an HTLC in Bitcoin Script. In <<received_htlc>> we see an HTLC Bitcoin Script as currently used in the Lightning Network. You can find this definition in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#offered-htlc-outputs[BOLT3 - Transactions]:
Let's see how we would implement this as an HTLC in Bitcoin Script. In <<received_htlc>> we see an HTLC Bitcoin Script as currently used in the Lightning Network. You can find this definition in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#offered-htlc-outputs[BOLT3: Transactions]:
[[received_htlc]]
.HTLC implemented in Bitcoin Script (BOLT3)
@ -374,9 +374,9 @@ OP_ENDIF
----
====
Wow that looks complicated! Don't worry though, we will take it one step at a time and simplify it.
Wow, that looks complicated! Don't worry though, we will take it one step at a time and simplify it.
The Bitcoin Script currently used in the Lightning Network is quite complex because it is optimized for on-chain space efficiency, which makes very compact but difficult to read.
The Bitcoin Script currently used in the Lightning Network is quite complex because it is optimized for on-chain space efficiency, which makes it very compact but difficult to read.
In the following sections, we will focus on the main elements of the script and present simplified scripts that are slightly different from what is actually used in Lightning.
@ -384,7 +384,7 @@ The main part of the HTLC is in line 10 of <<received_htlc>>. Let's build it up
==== Payment Pre-Image and Hash Verification
The core of an HTLC is the "hash", where payment can be made if the recipient knows the payment pre-image. Alice locks the payment to a specific payment hash and Bob has to present a payment pre-image to claim the funds. The Bitcoin system can verify that Bob's payment pre-image is correct by hashing it and comparing the result to the payment hash that Alice used to lock the funds.
The core of an HTLC is the hash, where payment can be made if the recipient knows the payment pre-image. Alice locks the payment to a specific payment hash, and Bob has to present a payment pre-image to claim the funds. The Bitcoin system can verify that Bob's payment pre-image is correct by hashing it and comparing the result to the payment hash that Alice used to lock the funds.
This part of an HTLC can be implemented in Bitcoin Script as follows:
@ -412,23 +412,23 @@ The unlocking script combined with the locking script would produce:
The Bitcoin Script engine would evaluate this script as follows:
1. +R+ is pushed to the stack
2. The OP_SHA256 operator takes the value +R+ off the stack and hashes it, pushing the result H~R~ to the stack
2. The OP_SHA256 operator takes the value +R+ off the stack and hashes it, pushing the result +H~R~+ to the stack
3. +H+ is pushed to the stack
4. The OP_EQUAL operator compares +H+ and H~R~. If they are equal, the result is +TRUE+, the script is complete and the payment is verified.
4. The OP_EQUAL operator compares +H+ and +H~R~+. If they are equal, the result is +TRUE+, the script is complete, and the payment is verified.
==== Extending HTLCs from Alice to Dina
Alice will now extend the HTLC across the network so that it reaches Dina.
In <<alice_dina_htlc_1>>, we see the HTLC propagated across the network from Alice to Dina. Alice has given Bob an HTLC for 50,200 satoshi. Bob can now create an HTLC for 50,100 satoshi and give it to Chan.
[[alice_dina_htlc_1]]
.Propagating the HTLC across the network
image::images/mtln_0809.png["Propagating the HTLC across the network"]
In <<alice_dina_htlc_1>> we see the HTLC propagated across the network from Alice to Dina. Alice has given Bob an HTLC for 50,200 satoshi. Bob can now create an HTLC for 50,100 satoshi and give it to Chan.
Bob knows that Chan can't redeem Bob's HTLC without broadcasting the secret, at which point Bob can also use the secret to redeem Alice's HTLC. This is a really important point, because it ensures end-to-end _atomicity_ of the HTLC. To spend the HTLC, one needs to reveal the secret, which then makes it possible for others to spend their HTLC also. Either all the HTLCs are spendable, or none of the HTLCs are spendable: atomicity!
Bob knows that Chan can't redeem Bob's HTLC without broadcasting the secret, at which point Bob can also use the secret to redeem Alice's HTLC. This is a really important point because it ensures end-to-end _atomicity_ of the HTLC. To spend the HTLC, one needs to reveal the secret, which then makes it possible for others to spend their HTLC also. Either all the HTLCs are spendable, or none of the HTLCs are spendable: atomicity!
Since Alice's HTLC is 100 satoshi more that the HTLC Bob gave to Chan, Bob will earn 100 satoshi as a routing fee if this payment completes.
Because Alice's HTLC is 100 satoshi more than the HTLC Bob gave to Chan, Bob will earn 100 satoshi as a routing fee if this payment completes.
Bob isn't taking a risk and isn't trusting Alice or Chan. Instead, Bob is trusting that a signed transaction together with the secret will be redeemable on the Bitcoin blockchain.
Notice Dina's channel balance goes from 50,000 satoshi to 100,000 satoshi. Chan's channel balance is reduced from 200,000 satoshi to 150,000 satoshi. The channel capacity hasn't changed, but 50,000 has moved from Chan's side of the channel to Dina's side of the channel.
Chan now has the secret and has paid Dina 50,000 satoshi. He can do this without any risk, because the secret allows Chan to redeem the 50,100 HTLC from Bob. Chan has the option to commit that HTLC onchain and spend it by revealing the secret on the Bitcoin blockchain. But, like Dina, he'd rather avoid transaction fees. So instead, he sends the secret to Bob so they can update their channel balances to reflect a 50,100 satoshi Lightning payment from Bob to Chan. In <<alice_dina_htlc_redeem_2>> we see Chan sending the secret to Bob and receiving a payment in return.
Chan now has the secret and has paid Dina 50,000 satoshi. He can do this without any risk, because the secret allows Chan to redeem the 50,100 HTLC from Bob. Chan has the option to commit that HTLC on-chain and spend it by revealing the secret on the Bitcoin blockchain. But, like Dina, he'd rather avoid transaction fees. So instead, he sends the secret to Bob so they can update their channel balances to reflect a 50,100 satoshi Lightning payment from Bob to Chan. In <<alice_dina_htlc_redeem_2>> we see Chan sending the secret to Bob and receiving a payment in return.
[[alice_dina_htlc_redeem_2]]
.Chan settles Bob's HTLC off-chain
@ -462,24 +462,24 @@ Bob has recieved 50,200 satoshi from Alice and paid 50,100 satoshi to Chan, so h
Alice receives the secret and has settled the 50,200 satoshi HTLC. The secret can be used as a _receipt_ to prove that Dina got paid for that specific payment hash.
The final channel balances reflect Alice's payment to Dina and the routing fees paid at each hop, as shown in <<alice_dina_htlc_redeem_4>>
The final channel balances reflect Alice's payment to Dina and the routing fees paid at each hop, as shown in <<alice_dina_htlc_redeem_4>>.
[[alice_dina_htlc_redeem_4]]
.Channel balances after the payment
image::images/mtln_0813.png["Channel balances after the payment"]
[[preventing_theft]]
==== Signature Binding - Preventing Theft of HTLCs
==== Signature Binding: Preventing Theft of HTLCs
There's a catch. Did you notice it?
If Alice, Bob and Chan create the HTLCs as shown above, they face a small but not insignificant risk of loss. Any of those HTLCs can be redeemed (spent) by anyone who knows the secret. At first only Dina knows the secret. Dina is supposed to only spend the HTLC from Chan. But Dina could spend all three HTLCs at the same time, or even in a single spending transaction! After all Dina knows the secret before anyone else. Similarly, once Chan knows the secret he is only supposed to spend the HTLC offered by Bob. But what if Chan also spends Alice's offered HTLC too?
If Alice, Bob, and Chan create the HTLCs as shown in <<alice_dina_htlc_redeem_4>>, they face a small but not insignificant risk of loss. Any of those HTLCs can be redeemed (spent) by anyone who knows the secret. At first only Dina knows the secret. Dina is supposed to only spend the HTLC from Chan. But Dina could spend all three HTLCs at the same time, or even in a single spending transaction! After all, Dina knows the secret before anyone else. Similarly, once Chan knows the secret he is only supposed to spend the HTLC offered by Bob. But what if Chan also spends Alice's offered HTLC too?
This is not _trustless_! It fails the most important security feature. We need to fix this.
The HTLC script must have an additional condition that binds each HTLC to a specific recipient. We do this by requiring a digital signature that matches the public key of each recipient, thereby preventing anyone else from spending that HTLC. Since only the designated recipient has the ability to produce a digital signature matching that public key, only the designated recipient can spend that HTLC.
Let's look at the scripts again with this modification in mind. Alice's HTLC for Bob is modified to include Bob's Public Key and the +OP_CHECKSIG+ operator.
Let's look at the scripts again with this modification in mind. Alice's HTLC for Bob is modified to include Bob's public key and the +OP_CHECKSIG+ operator.
Notice that we also changed +OP_EQUAL+ to +OP_EQUALVERIFY+. When an operator has the suffix +VERIFY+ it does not return +TRUE+ or +FALSE+ on the stack. Instead it *halts* execution and fails the script if the result is false and continues without any stack output if it is true.
Notice that we also changed +OP_EQUAL+ to +OP_EQUALVERIFY+. When an operator has the suffix +VERIFY+, it does not return +TRUE+ or +FALSE+ on the stack. Instead, it _halts_ execution and fails the script if the result is false and continues without any stack output if it is true.
====
To redeem this HTLC, Bob has to present an unlocking script that includes a signature from Bob's private key as well as the secret payment pre-image, like this:
@ -498,19 +498,19 @@ To redeem this HTLC, Bob has to present an unlocking script that includes a sign
<Bob's Signature> <R>
----
The unlocking and locking script are combined and evaluated by the scripting engine, as follows:
The unlocking and locking scripts are combined and evaluated by the scripting engine, as follows:
3. OP_SHA256 pops and hashes +R+ from the top of the stack and pushes H~R~ on to the stack
4. +H+ is pushed on to the stack
5. OP_EQUALVERIFY pops H and H~R~ and compares them. If they are not the same, execution halts. Otherwise, we continue without output to the stack
6. +<Bob's Pub>+ key is pushed to the stack
7. OP_CHECKSIG pops +<Bob's Sig>+ and +<Bob's Pub>+ and verifies the signature. The result (TRUE/FALSE) is pushed to the stack.
1. +<Bob's Sig>+ is pushed on to the stack.
2. +R+ is pushed on to the stack.
3. +OP_SHA256+ pops and hashes +R+ from the top of the stack and pushes +H~R~+ on to the stack.
4. +H+ is pushed on to the stack.
5. +OP_EQUALVERIFY+ pops +H+ and +H~R~+ and compares them. If they are not the same, execution halts. Otherwise, we continue without output to the stack.
6. +<Bob's Pub>+ key is pushed to the stack.
7. +OP_CHECKSIG+ pops +<Bob's Sig>+ and +<Bob's Pub>+ and verifies the signature. The result (TRUE/FALSE) is pushed to the stack.
As you can see, this is slightly more complicated, but now we have fixed the HTLC and made sure only the intended recipient can spend it.
@ -522,7 +522,7 @@ Let's look at the first part of the HTLC script so far:
OP_SHA256 <H> OP_EQUALVERIFY
----
If we look at this in a symbolic representation above, it looks like the +OP_+ operators take up the most space. But that's not the case. Bitcoin Script is encoded in binary, with each operator representing one byte. Meanwhile, the +<H>+ value we use as a placeholder for the payment hash is a 32-byte (256-bit) value. You can find a listing of all the Bitcoin Script operators and their binary and hex encoding in https://en.bitcoin.it/wiki/Script[Bitcoin Wiki: Script], or in https://github.com/bitcoinbook/bitcoinbook/blob/develop/appdx-scriptops.asciidoc[Mastering Bitcoin Appendix: Transaction Script Language Operators, Constants, and Symbols].
If we look at this in the preceding symbolic representation, it looks like the +OP_+ operators take up the most space. But that's not the case. Bitcoin Script is encoded in binary, with each operator representing one byte. Meanwhile, the +<H>+ value we use as a placeholder for the payment hash is a 32-byte (256-bit) value. You can find a listing of all the Bitcoin Script operators and their binary and hex encoding in https://en.bitcoin.it/wiki/Script[Bitcoin Wiki: Script], or in https://github.com/bitcoinbook/bitcoinbook/blob/develop/appdx-scriptops.asciidoc[_Mastering Bitcoin_ Appendix: Transaction Script Language Operators, Constants, and Symbols].
Represented in hexadecimal, our HTLC script would look like this:
In hexadecimal encoding, +OP_SHA256+ is +a8+ and +OP_EQUALVERIFY+ is +88+. The total length of this script is 34 bytes, of which 32 bytes are the hash.
As we've mentioned previously, any participant in the Lightning network should be able to take an off-chain transaction they hold and put it on-chain if they need to enforce their claim to funds. To take a transaction on-chain, they'd have to pay transaction fees to the miners and these fees are proportional to the *size in bytes* of the transaction.
As we've mentioned previously, any participant in the Lightning Network should be able to take an off-chain transaction they hold and put it on-chain if they need to enforce their claim to funds. To take a transaction on-chain, they'd have to pay transaction fees to the miners, and these fees are proportional to the size in bytes of the transaction.
Therefore, we want to find ways to minimize the on-chain "weight" of transactions by optimizing the script as much as possible. One way to do that is to add another hash function on top of the SHA256 algorithm, one that produces smaller hashes. The Bitcoin Script language provides the +OP_HASH160+ operator that "double hashes" a pre-image: first the pre-image is hashed with SHA256 and then the resulting hash is hashed again with the RIPEMD160 hash algorithm. The hash resulting from RIPEMD160 is 160 bits or 20 bytes, much more compact. In Bitcoin Script this is a very common optimization that is used in many of the common address formats.
Therefore, we want to find ways to minimize the on-chain "weight" of transactions by optimizing the script as much as possible. One way to do that is to add another hash function on top of the SHA256 algorithm, one that produces smaller hashes. The Bitcoin Script language provides the +OP_HASH160+ operator that "double hashes" a pre-image: first the pre-image is hashed with SHA256, and then the resulting hash is hashed again with the RIPEMD160 hash algorithm. The hash resulting from RIPEMD160 is 160 bits or 20 bytes--much more compact. In Bitcoin Script this is a very common optimization that is used in many of the common address formats.
So, let's use that optimization instead. Our SHA256 hash is +057596...69f6b3+. Putting that through another round of hasing with RIPEMD160 gives us the result:
So, let's use that optimization instead. Our SHA256 hash is +057596...69f6b3+. Putting that through another round of hashing with RIPEMD160 gives us the result:
----
R = "Dinas secret"
@ -560,7 +560,7 @@ Encoded in hex this is:
a9 9e017f6767971ed7cea17f98528d5f5c0ccb2c71 88
----
Where OP_HASH160 is +a9+ and OP_EQUALVERIFY is +88+. This script is only 22 bytes long! We've saved 12 bytes from every transaction that redeems an HTLC on-chain.
Where +OP_HASH160+ is +a9+ and +OP_EQUALVERIFY+ is +88+. This script is only 22 bytes long! We've saved 12 bytes from every transaction that redeems an HTLC on-chain.
With that optimization, you now see how we arrive at the HTLC script shown in line 10 of <<received_htlc>>:
@ -572,11 +572,11 @@ With that optimization, you now see how we arrive at the HTLC script shown in li
==== HTLC Cooperative and Timeout Failure
So far we looked at the "Hash" part of HTLC, and how it would work if everyone cooperated and was online at the time of payment.
So far we looked at the "hash" part of HTLC and how it would work if everyone cooperated and was online at the time of payment.
What happens if someone goes offline or fails to cooperate? What happens if the payment cannot succeed?
We need to ensure a way to "fail gracefully", because occasional routing failures are inevitable. There are two ways to fail: cooperatively and with a time-locked refund.
We need to ensure a way to "fail gracefully," because occasional routing failures are inevitable. There are two ways to fail: cooperatively and with a time-locked refund.
Cooperative failure is relatively simple: the HTLC is unwound by every participant in the route removing the HTLC output from their commitment transactions without changing the balance. We'll look at how that works in detail in <<channel_operation>>.
@ -584,9 +584,9 @@ Let's look at how we can reverse an HTLC without the cooperation of one or more
To prevent this, every HTLC script includes a refund clause that is connected to a time-lock. Remember our original escrow contract? "Bob has 24 hours to show the secret after the contract is signed. If Bob does not provide the secret by this time, Alice's deposit will be refunded"
The time-locked refund is an important part of the script that ensures _atomicity_, so that the entire end-to-end payment either succeeds or fails gracefully. There is not "half paid" state to worry about. If there is a failure, every participant can either unwind the HTLC cooperatively with their channel partner, or put the time-locked refund transaction on-chain unilaterally to get their money back.
The time-locked refund is an important part of the script that ensures _atomicity_, so that the entire end-to-end payment either succeeds or fails gracefully. There is no "half paid" state to worry about. If there is a failure, every participant can either unwind the HTLC cooperatively with their channel partner or put the time-locked refund transaction on-chain unilaterally to get their money back.
To implement this refund in Bitcoin Script, we use a special operator +OP_CHECKLOCKTIMEVERIFY+ also known +OP_CLTV+ for short. Here's the script, as seen previously in line 13 of <<received_htlc>>:
To implement this refund in Bitcoin Script, we use a special operator pass:[<code>O⁠P⁠_⁠C⁠H⁠E⁠C⁠K⁠L⁠O⁠C⁠K⁠T⁠I⁠M⁠E​V⁠E⁠R⁠I⁠F⁠Y</code>] also known +OP_CLTV+ for short. Here's the script, as seen previously in line 13 of <<received_htlc>>:
----
...
@ -595,26 +595,26 @@ To implement this refund in Bitcoin Script, we use a special operator +OP_CHECKL
...
----
The +OP_CLTV+ operator takes an expiry time defined as the block height after which this transaction is valid. If the transaction timelock is not set the same as +<cltv_expiry>+, the evaluation of the script fails and the transaction is invalid. Otherwise, the script continues without any output to the stack. Remember, the +VERIFY+ suffix means this operator does not output +TRUE+ or +FALSE+, but instead either halts/fails, or continues without stack output.
The +OP_CLTV+ operator takes an expiry time defined as the block height after which this transaction is valid. If the transaction time-lock is not set the same as +<cltv_expiry>+, the evaluation of the script fails and the transaction is invalid. Otherwise, the script continues without any output to the stack. Remember, the +VERIFY+ suffix means this operator does not output +TRUE+ or +FALSE+ but instead either halts/fails or continues without stack output.
Essentially, the +OP_CLTV+ acts as a "gatekeeper" preventing the script from proceeding any further if the <cltv_expiry> block height has not been reached on the Bitcoin blockchain.
Essentially, the +OP_CLTV+ acts as a "gatekeeper" preventing the script from proceeding any further if the +<cltv_expiry>+ block height has not been reached on the Bitcoin blockchain.
The +OP_DROP+ operator simply drops the topmost item on the script stack. This is necessary in the beginning, because there is a "leftover" item from the previous script lines. It is necessary *after* +OP_CLTV+ in order to remove the +<cltv_expiry>+ item from the top of the stack as it is no longer necessary.
The +OP_DROP+ operator simply drops the topmost item on the script stack. This is necessary in the beginning because there is a "leftover" item from the previous script lines. It is necessary _after_ +OP_CLTV+ to remove the +<cltv_expiry>+ item from the top of the stack because it is no longer necessary.
Finally, once the stack has been cleaned up, there should be a public key and signature left behind, that +OP_CHECKSIG+ can verify. As we saw in <<preventing_theft>>, this is necessary to ensure that only the rightful owner of the funds can claim them, by binding this output to their public key and requiring a signature.
Finally, once the stack has been cleaned up, there should be a public key and signature left behind that +OP_CHECKSIG+ can verify. As we saw in <<preventing_theft>>, this is necessary to ensure that only the rightful owner of the funds can claim them, by binding this output to their public key and requiring a signature.
==== Decrementing Time-Locks
As the HTLCs are extended from Alice to Dina, the time-lock refund clause in each HTLC has a _different_ cltv_expiry value. We will see this in more detail as we talk about <<onion_routing>>. But suffice it to say that to ensure an orderly unwinding of a payment that fails, each hop needs to wait a bit less for their refund. The difference between time-locks for each hop is called the cltv_expiry_delta and is set by each node and advertized to the network as we will see in <<gossip>>.
As the HTLCs are extended from Alice to Dina, the time-lock refund clause in each HTLC has a _different_ +cltv_expiry+ value. We will see this in more detail in <<onion_routing>>. But suffice it to say that to ensure an orderly unwinding of a payment that fails, each hop needs to wait a bit less for their refund. The difference between time-locks for each hop is called the +cltv_expiry_delta+ and is set by each node and advertised to the network as we will see in <<gossip>>.
For example, Alice sets the refund time-lock on the first HTLC to a block height of current+500 blocks ("current" being the current block height). Bob would then set the time-lock cltv_expiry on the HTLC to Chan to current+450 blocks. Chan would set the time-lock to current+400 blocks from the current block height. This way, Chan can get a refund on the HTLC he offered to Dina _before_ Bob gets a refund on the HTLC he offered to Chan. Bob can get a refund of the HTLC he offered to Chan before Alice can get a refund for the HTLC she offered to Bob. The decrementing time-lock prevents race conditions and ensures the HTLC chain is unwound backwards, from the destination towards the origin.
For example, Alice sets the refund time-lock on the first HTLC to a block height of current+500 blocks ("current" being the current block height). Bob would then set the time-lock +cltv_expiry+ on the HTLC to Chan to current+450 blocks. Chan would set the time-lock to current+400 blocks from the current block height. This way, Chan can get a refund on the HTLC he offered to Dina _before_ Bob gets a refund on the HTLC he offered to Chan. Bob can get a refund of the HTLC he offered to Chan before Alice can get a refund for the HTLC she offered to Bob. The decrementing time-lock prevents race conditions and ensures the HTLC chain is unwound backward, from the destination toward the origin.
=== Conclusion
In this chapter we saw how Alice can pay Dina even if she doesn't have a direct payment channel. Alice can find a path that connects her to Dina and route a payment across several payment channels so that it reaches Dina.
To ensure that the payment is _atomic_ and _trustless_ across _multiple hops_, Alice must implement a fairness protocol in cooperation with all the intermediary nodes in the path. The fairness protocol is currently implemented as a _Hash Time-Locked Contract(HTLC)_, which commits funds to a payment hash derived from a secret payment pre-image.
To ensure that the payment is _atomic_ and _trustless_ across _multiple hops_, Alice must implement a fairness protocol in cooperation with all the intermediary nodes in the path. The fairness protocol is currently implemented as an HTLC, which commits funds to a payment hash derived from a secret payment pre-image.
Each of the participants in the payment route can extend an HTLC to the next participant, without worrying about theft or stuck funds. The HTLC can be redeemed by revealing the secret payment pre-image. Once an HTLC reaches Dina, she reveals the pre-image which flows backwards resolving all the HTLCs offered.
Each of the participants in the payment route can extend an HTLC to the next participant, without worrying about theft or stuck funds. The HTLC can be redeemed by revealing the secret payment pre-image. Once an HTLC reaches Dina, she reveals the pre-image, which flows backward resolving all the HTLCs offered.
Finally, we saw how a time-locked refund clause completes the HTLC, ensuring that every participant can get a refund if the payment fails but for whatever reason one of the participants doesn't cooperate in unwinding the HTLCs. By always having the option to go on-chain for a refund, the HTLC achieves the fairness goal of atomicity and trustless operation.
@ -800,6 +800,7 @@ Each hop receiving an error, will generate an +ammag+ key and obfuscate the retu
Eventually, the sender (origin node) receives a return packet. It will then generate +ammag+ and +um+ keys for each hop and XOR de-obfuscate the return error iteratively until it reveals the return packet.
[[failure_messages]]
==== Failure Messages
The +failuremsg+ is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#failure-messages[BOLT #4 - Onion Routing - Failure Messages].
@ -877,7 +878,7 @@ pre-image (which _must_ match the payment hash of the HTLC) to settle the
payment. As a result, `keysend` payments can be carried out without first
obtaining an invoice from the receiver, as the pre-image is "pushed" over to
the receiver. A `keysend` payment uses a TLV custom record type of `5482373484`
to encode a 32-byte pre-image value.
to encode a 32-byte pre-image value.
==== Keysend and Custom Records in Lightning Applications
Payment delivery on the Lightning Network depends on finding a path from the sender to the recipient, a process called _path finding_. Since the routing is done by the sender, the sender must find a suitable path to reach the destination. This path is then encoded in an onion, as we saw in <<onion_routing>>.
In this chapter we will examine the problem of path finding, understand how uncertainty about channel balances complicates this problem and look at how a typical path finding implementation attempts to solve it.
In this chapter we will examine the problem of path finding, understand how uncertainty about channel balances complicates this problem, and look at how a typical path finding implementation attempts to solve it.
=== Path Finding in the Lightning Protocol Suite
Path finding, path selection, multi-path payments (MPP) and the payment attempt trial & error loop occupy the majority of the "Payment Layer" at the top of the protocol suite.
Path finding, path selection, multipath payments (MPP), and the payment attempt trial and error loop occupy the majority of the "Payment Layer" at the top of the protocol suite.
These components are highlighted by a double outline in the protocol suite, shown in <<LN_protocol_pathfinding_highlight>>:
These components are highlighted by a double outline in the protocol suite, shown in <<LN_protocol_pathfinding_highlight>>.
[[LN_protocol_pathfinding_highlight]]
.The Lightning Network Protocol Suite
@ -23,9 +23,9 @@ That's because path finding isn't an activity that requires any form of coordina
=== Path Finding: What Problem Are We Solving?
The term path finding may be somewhat misleading, because it implies a search for _a single path_ connecting two nodes. In the beginning, when the Lightning Network was small and not well interconnected, the problem was indeed about finding a way to join payment channels to reach the recipient.
The term path finding may be somewhat misleading because it implies a search for _a single path_ connecting two nodes. In the beginning, when the Lightning Network was small and not well interconnected, the problem was indeed about finding a way to join payment channels to reach the recipient.
But, as the Lightning Network has grown explosively, the path finding problem's nature has shifted. In mid-2021, as we finish this book, the Lightning Network consists of 20,000 nodes connected by at least 55,000 public channels with an aggregate capacity of almost 2,000 BTC. A node has on average 8.8 channels, while the top 10 most connected nodes have between 400 and 2000 channels _each_. A visualization of just a small subset of the Lightning Network channel graph is shown in <<lngraph>>:
But, as the Lightning Network has grown explosively, the path finding problem's nature has shifted. In mid-2021, as we finish this book, the Lightning Network consists of 20,000 nodes connected by at least 55,000 public channels with an aggregate capacity of almost 2,000 BTC. A node has on average 8.8 channels, while the top 10 most connected nodes have between 400 and 2,000 channels _each_. A visualization of just a small subset of the LN channel graph is shown in <<lngraph>>.
[[lngraph]]
.A visualization of part of the Lightning Network as of July 2021
@ -33,14 +33,14 @@ image::images/mtln_1202.png[]
[NOTE]
====
The network visualization above was produced with a simple python script you can find in code/lngraph in the book's repository
The network visualization in <<lngraph>> was produced with a simple python script you can find in code/lngraph in the book's repository.
====
If the sender and recipient are connected to other well-connected nodes and have at least one channel with adequate capacity - there will be thousands of paths. The problem becomes: selecting the _best_ path that will succeed in payment delivery, out of a list of thousands of possible paths.
If the sender and recipient are connected to other well-connected nodes and have at least one channel with adequate capacity, there will be thousands of paths. The problem becomes selecting the _best_ path that will succeed in payment delivery, out of a list of thousands of possible paths.
==== Selecting the Best Path
To select the "best" path, we have to first define what we mean by "best". There may be many different criteria such as:
To select the best path, we have to first define what we mean by "best." There may be many different criteria such as:
* Paths with enough liquidity. Obviously if a path doesn't have enough liquidity to route our payment, then it is not a suitable path.
@ -48,28 +48,28 @@ To select the "best" path, we have to first define what we mean by "best". There
* Paths with short timelocks. We may want to avoid locking our funds for too long and therefore select paths with shorter timelocks.
All of these criteria may be desirable to some extent and selecting paths that are favorable across many dimensions is not an easy task. Optimization problems like this may be too complex to solve for the "best" solution, but often can be solved for some approximation of the optimal. Which is good news, because otherwise path finding would be an intractable problem.
All of these criteria may be desirable to some extent, and selecting paths that are favorable across many dimensions is not an easy task. Optimization problems like this may be too complex to solve for the "best" solution, but often can be solved for some approximation of the optimal, which is good news because otherwise path finding would be an intractable problem.
==== Path Finding in Math and Computer Science
Path finding in the Lightning Network falls under a general category of _graph theory_ in mathematics and the more specific category of _graph traversal_ in computer science.
A network such as the Lightning Network can be represented as a mathematical construct called a _graph_, where _nodes_ are connected to each other by _edges_ (equivalent to the payment channels). The Lightning Network forms a _directed graph_ because the nodes are linked _asymmetrically_, since the channel balance is split between the two channel partners and the payment liquidity is different in each direction. A directed graph with numerical capacity constrains on its edges is called a _flow network_, a mathematical construct used to optimize transportation and other similar networks. Flow networks can be used as a framework when solutions need to achieve a specific flow while minimizing cost, known as the Minimum Cost Flow Problem (MCFP).
A network such as the Lightning Network can be represented as a mathematical construct called a _graph_, where _nodes_ are connected to each other by _edges_ (equivalent to the payment channels). The Lightning Network forms a _directed graph_ because the nodes are linked _asymmetrically_, since the channel balance is split between the two channel partners and the payment liquidity is different in each direction. A directed graph with numerical capacity constraints on its edges is called a _flow network_, a mathematical construct used to optimize transportation and other similar networks. Flow networks can be used as a framework when solutions need to achieve a specific flow while minimizing cost, known as the minimum cost flow problem (MCFP).
==== Capacity, Balance, Liquidity
In order to better understand the problem of transporting satoshis from point A to point B, we need to better define three important terms: capacity, balance, and liquidity. We use these terms to describe a payment channel's ability to route a payment.
To better understand the problem of transporting satoshis from point A to point B, we need to better define three important terms: capacity, balance, and liquidity. We use these terms to describe a payment channel's ability to route a payment.
In a payment channel connecting A<-->B:
Capacity:: This is the aggregate amount of satoshis that were funded into the 2-of-2 multisig with the funding transaction. It represents the maximum amount of value is held in the channel. The channel capacity is announced by the gossip protocol and is known to nodes.
Capacity:: This is the aggregate amount of satoshis that were funded into the 2-of-2 multisig with the funding transaction. It represents the maximum amount of value held in the channel. The channel capacity is announced by the gossip protocol and is known to nodes.
Balance:: This is the amount of satoshis held by each channel partner that can be sent to the other channel partner. A subset of the balance of A, can be sent in the direction (A--->B) towards node B. A subset of the balance of B can be sent in the opposite direction (A<---B).
Balance:: This is the amount of satoshis held by each channel partner that can be sent to the other channel partner. A subset of the balance of A can be sent in the direction (A->B) toward node B. A subset of the balance of B can be sent in the opposite direction (A<-B).
Liquidity:: The available (subset) balance that can actually be sent across the channel in one direction. Liquidity of A is equal to the balance of A minus the channel reserve and any pending HTLCs committed by A.
The only value known to the network (via gossip announcements) is the aggregate capacity of the channel. Some unknown portion of that capacity is distributed as each partners balance. Some subset of that balance is available to send across the channel in one direction:
The only value known to the network (via gossip announcements) is the aggregate capacity of the channel. Some unknown portion of that capacity is distributed as each partner's balance. Some subset of that balance is available to send across the channel in one direction:
If we knew the exact channel balances of every channel, we could compute one or more payment paths using any of the standard path finding algorithms taught in good computer science programs. But we don't know the channel balances, we only know the aggregate channel capacity, which is advertised by nodes in channel announcements. In order for a payment to succeed, there must be adequate balance on the sending side of the channel. If we don't know how the capacity is distributed between the channel partners, we don't know if there is enough balance in the direction we are trying to send the payment.
If we knew the exact channel balances of every channel, we could compute one or more payment paths using any of the standard path finding algorithms taught in good computer science programs. But we don't know the channel balances; we only know the aggregate channel capacity, which is advertised by nodes in channel announcements. In order for a payment to succeed, there must be adequate balance on the sending side of the channel. If we don't know how the capacity is distributed between the channel partners, we don't know if there is enough balance in the direction we are trying to send the payment.
Balances are not announced in channel updates for two reasons: privacy and scalability. First, announcing balances would reduce the privacy of the Lightning Network as it would allow surveillance of payment by statistical analysis of the changes in balances. Second, if nodes announced balances (globally) with every payment, the Lightning Network's scaling would be as bad as that of on-chain Bitcoin transactions which are broadcast to all participants. Therefore, balances are not announced. To solve the path finding problem in the face of uncertainty of balances, we need innovative path finding strategies. These strategies must relate closely to the routing algorithm that is used, which is source-based onion-routing where it is the responsibility of the sender to find a path through the network.
Balances are not announced in channel updates for two reasons: privacy and scalability. First, announcing balances would reduce the privacy of the Lightning Network because it would allow surveillance of payment by statistical analysis of the changes in balances. Second, if nodes announced balances (globally) with every payment, the Lightning Network's scaling would be as bad as that of on-chain Bitcoin transactions which are broadcast to all participants. Therefore, balances are not announced. To solve the path finding problem in the face of uncertainty of balances, we need innovative path finding strategies. These strategies must relate closely to the routing algorithm that is used, which is source-based onion routing in which it is the responsibility of the sender to find a path through the network.
The uncertainty problem can be described mathematically as a _range of liquidity_, indicating the lower and upper bounds of liquidity based on the information that is known. Since we know the capacity of the channel and we know the channel reserve balance (the minimum allowed balance on each end), the liquidity can be defined as:
Our channel liquidity uncertainty range is the range between the minimum and maximum possible liquidity. This is unknown to the network, except the two channel partners. However, as we will see we can use failed HTLCs returned from our payment attempts to update our liquidity estimate and reduce uncertainty. If for example we get an HTLC failure code that tells us that a channel cannot fulfill an HTLC that is smaller than our estimate for maximum liquidity, that means the maximum liquidity can be updated to the amount of the failed HTLC. In simpler terms, if we think the liquidity can handle an HTLC of N satoshis and we find out it fails to deliver M satoshis (where M is smaller), then we can update our estimate to M-1 as the upper bound. We tried to find the ceiling and bumped against it, so it's lower than we thought!
Our channel liquidity uncertainty range is the range between the minimum and maximum possible liquidity. This is unknown to the network, except the two channel partners. However, as we will see, we can use failed HTLCs returned from our payment attempts to update our liquidity estimate and reduce uncertainty. If for example we get an HTLC failure code that tells us that a channel cannot fulfill an HTLC that is smaller than our estimate for maximum liquidity, that means the maximum liquidity can be updated to the amount of the failed HTLC. In simpler terms, if we think the liquidity can handle an HTLC of _N_ satoshis and we find out it fails to deliver _M_ satoshis (where _M_ is smaller), then we can update our estimate to __M__–1 as the upper bound. We tried to find the ceiling and bumped against it, so it's lower than we thought!
==== Path Finding Complexity
Finding a path through a graph is a problem modern computers can solve rather efficiently.
Developers mainly choose breadth-first search if the edges are all of equal weight.
In cases where the edges are not of equal weight, an algorithm based on Dijkstra Algorithm is used, such as https://en.wikipedia.org/wiki/A*_search_algorithm[A* ("A-star")].
In cases where the edges are not of equal weight, an algorithm based on Dijkstra's algorithm is used, such as https://en.wikipedia.org/wiki/A*_search_algorithm[A* (pronounced "A-star")].
In our case the weights of the edges can represent the routing fees.
Only edges with a capacity larger than the amount to be sent will be included in the search.
In this basic form, path finding in the Lightning network is very simple and straight forward.
In this basic form, path finding in the Lightning Network is very simple and straightforward.
However, channel liquidity is unknown to the sender. This turns our easy theoretical computer science problem into a rather complex real-world problem.
We now have to solve a path finding problem with only partial knowledge.
For example, we suspect which edges might be able to forward a payment because their capacity seems big enough.
But we can't be certain unless we try it out or ask the channel owners directly.
Even if we were able to ask the channel owners directly, their balance might change by the time we have asked others, computed a path, constructed an onion and send it along.
Even if we were able to ask the channel owners directly, their balance might change by the time we have asked others, computed a path, constructed an onion, and sent it along.
Not only do we have limited information but the information we have is highly dynamic and might change at any point in time without our knowledge.
==== Keeping It Simple
@ -123,43 +123,43 @@ This probing is done by the Lightning node or wallet and is not directly observe
However, the user might suspect that probing is taking place if the payment is not completed instantly.
====
While "blind probing" is not optimal and leaves ample room for improvement, it should be noted that even this simplistic strategy works surprisingly well for smaller payments and well-connected nodes.
While blind probing is not optimal and leaves ample room for improvement, it should be noted that even this simplistic strategy works surprisingly well for smaller payments and well-connected nodes.
Most Lightning node and wallet implementations improve on this approach, by ordering/weighting the list of candidate paths. Some implementations order the candidate paths by cost (fees), or some combination of cost/capacity.
Most Lightning node and wallet implementations improve on this approach by ordering/weighting the list of candidate paths. Some implementations order the candidate paths by cost (fees) or some combination of cost and capacity.
=== Path Finding and Payment Delivery Process
Path finding and payment delivery involves several steps, which we list below. Different implementations may use different algorithms and strategies, but the basic steps are likely to be very similar:
Path finding and payment delivery involves several steps, which we list here. Different implementations may use different algorithms and strategies, but the basic steps are likely to be very similar:
* Create a _channel graph_ from announcements and updates, containing the capacity of each channel and filter the graph ignoring any channels with insufficient capacity for the amount we want to send.
. Create a _channel graph_ from announcements and updates containing the capacity of each channel and filter the graph ignoring any channels with insufficient capacity for the amount we want to send.
* Find paths connecting the sender to the recipient.
. Find paths connecting the sender to the recipient.
* Order the paths by some weight (this may be part of the previous step's algorithm).
. Order the paths by some weight (this may be part of the previous step's algorithm).
* Try each path in order until payment succeeds. (the trial-and-error loop)
. Try each path in order until payment succeeds (the trial-and-error loop).
* Optionally use the HTLC failure returns to update our graph, reducing uncertainty
. Optionally use the HTLC failure returns to update our graph, reducing uncertainty.
We can group these steps into three primary activities:
1. Channel graph construction
2. Path finding (filtered and ordered by some heuristics)
3. Payment attempt(s)
* Channel graph construction
* Path finding (filtered and ordered by some heuristics)
* Payment attempt(s)
These three activities can be repeated in a _payment round_ if we use the failure returns to update the graph, or if we are doing multi-path payments (see <<mpp>>).
These three activities can be repeated in a _payment round_ if we use the failure returns to update the graph, or if we are doing multipath payments (see <<mpp>>).
In the next sections we will look at each of these steps in more detail, as well as more advanced payment strategies.
=== Channel Graph Construction
In <<gossip>> we covered the three main messages that nodes "gossip": +node_announcement+, +channel_announcement+, and +channel_update+. These three messages allow any node to gradually construct a "map" of the Lightning Network in the form of a _channel graph_. Each of these messages provides a critical piece of information for the channel graph:
In <<gossip>> we covered the three main messages that nodes use in their gossip: +node_announcement+, +channel_announcement+, and +channel_update+. These three messages allow any node to gradually construct a "map" of the Lightning Network in the form of a _channel graph_. Each of these messages provides a critical piece of information for the channel graph:
node_announcement:: This contains the information about a node on the Lightning Network, such as its node ID (public key), network address (e.g. IPv4/6 or Tor), capabilities/features etc.
+node_announcement+:: This contains the information about a node on the Lightning Network, such as its node ID (public key), network address (e.g., IPv4/6 or Tor), capabilities/features, etc.
channel_announcement:: This contains the capacity and channel ID of a public (announced) channel between two nodes and proof of the channel's existence and ownership.
+channel_announcement+:: This contains the capacity and channel ID of a public (announced) channel between two nodes and proof of the channel's existence and ownership.
channel_update:: This contains a node's fee and timelock (CLTV) expectations for routing an outgoing (from that node's perspective) payment over a specified channel.
+channel_update+:: This contains a node's fee and timelock (CLTV) expectations for routing an outgoing (from that node's perspective) payment over a specified channel.
In terms of a mathematical graph, the +node_announcement+ is the information needed to create the nodes or _vertices_ of the graph. The +channel_announcement+ allows us to create the _edges_ of the graph representing the payment channels. Since each direction of the payment channel has its own balance, we create a directed graph. The +channel_update+ allows us to incorporate fees and timelocks to set the _cost_ or _weight_ of the graph edges.
@ -167,36 +167,36 @@ Depending on the algorithm we will use for path finding, we may establish a numb
For now, let's ignore the cost function and simply establish a channel graph showing nodes and channels, using the +node_announcement+ and +channel_announcement+ messages.
In this chapter we will see how Selena attempts to find a path to pay Rashid 1,000,000 (1m) satoshis. To start, Selena is constructing a channel graph using the information from the Lightning Network gossip to discover nodes and channels. Selena will then explore her channel graph to find a path to send a payment to Rashid.
In this chapter we will see how Selena attempts to find a path to pay Rashid 1,000,000 (1M) satoshis. To start, Selena is constructing a channel graph using the information from Lightning Network gossip to discover nodes and channels. Selena will then explore her channel graph to find a path to send a payment to Rashid.
This is _Selena's_ channel graph. There is no such thing as *the* channel graph, there is only ever *a channel graph* and it is always from the perspective of the node that has constructed it (see <<map_territory_relation>>).
This is _Selena's_ channel graph. There is no such thing as _the_ channel graph, there is only ever _a channel graph_, and it is always from the perspective of the node that has constructed it (see <<map_territory_relation>>).
[TIP]
====
Selena does not contruct a channel graph only when sending a payment. Rather, Selena's node is _continuously_ building and updating a channel graph. From the moment Selena's node starts and connects to any peer on the network it will participate in the "gossip" and use every message to learn as much as possible about the network.
Selena does not contruct a channel graph only when sending a payment. Rather, Selena's node is _continuously_ building and updating a channel graph. From the moment Selena's node starts and connects to any peer on the network it will participate in the gossip and use every message to learn as much as possible about the network.
====
[[map_territory_relation]]
.The map-territory relation
.The Map-Territory Relation
****
From Wikipedia's https://en.wikipedia.org/wiki/Map%E2%80%93territory_relation[_Map Territory Relation_], "The map–territory relation describes the relationship between an object and a representation of that object, as in the relation between a geographical territory and a map of it."
From Wikipedia's https://en.wikipedia.org/wiki/Map%E2%80%93territory_relation[page on the Map–Territory Relation], "The map–territory relation describes the relationship between an object and a representation of that object, as in the relation between a geographical territory and a map of it."
The mapterritory relation is best illustrated in "Sylvie and Bruno Concluded", a short story by Lewis Carroll which describes a fictional map that is 1:1 scale of the territory it maps, therefore having perfect accuracy but becoming completely useless as it would cover the entire territory if unfolded.
The map–territory relation is best illustrated in "Sylvie and Bruno Concluded," a short story by Lewis Carroll which describes a fictional map that is 1:1 scale of the territory it maps, therefore having perfect accuracy but becoming completely useless as it would cover the entire territory if unfolded.
What does this mean for the Lightning Network? LN is the territory, and a channel graph is a map of that territory:
What does this mean for the Lightning Network? The Lightning Network is the territory, and a channel graph is a map of that territory.
While we could imagine a theoretical (Platonic ideal) channel graph that represents the complete, up-to-date map of the Lightning Network, such a map is simply the Lightning Network itself. Each node has its own channel graph which is constructed from announcements and is necessarily incomplete, incorrect, and out-of-date!
The map can never completely and accurately describe the territory.
****
Selena listens to node_announcement messages and discovers 4 other nodes (in addition to Rashid, the intended recipient). The resulting graph represents a network of six nodes: (S)elena and (R)ashid are the (S)ender and (R)ecipient respectively; (A)lice, (B)ob, (X)avier and (Y)an are intermediary nodes. Selena's initial graph is just a list of nodes, shown in <<channel_graph_nodes>>:
Selena listens to +node_announcement+ messages and discovers four other nodes (in addition to Rashid, the intended recipient). The resulting graph represents a network of six nodes: Selena and Rashid are the Sender and Recipient respectively; Alice, Bob, Xavier and Yan are intermediary nodes. Selena's initial graph is just a list of nodes, shown in <<channel_graph_nodes>>.
[[channel_graph_nodes]]
.Node announcements
image::images/mtln_1203.png[]
Selena also receives seven channel_announcement messages with the corresponding channel capacities, allowing her to construct a basic "map" of the network, shown in <<channel_graph_1>>, below:
Selena also receives seven +channel_announcement+ messages with the corresponding channel capacities, allowing her to construct a basic "map" of the network, shown in <<channel_graph_1>>. (The names Alice, Bob, Selena, Xavier, Yan, and Rashid have been replaced by their initials: A, B, S, X, and R, respectively.)
[[channel_graph_1]]
.The channel graph
@ -206,7 +206,7 @@ image::images/mtln_1204.png[]
As you can see from <<channel_graph_1>>, Selena does not know any of the balances of the channels. Her initial channel graph contains the highest level of uncertainty.
But wait: Selena does know *some* channel balances! She knows the balances of the channels that her own node has connected with other nodes. While this does not seem like much, it in fact very important information for constructing a path - Selena knows the actual liquidity of her own channels. Let's update the channel graph to show this information. We will use a "?" symbol to represent the unknown balances, as shown in <<channel_graph_2>>:
But wait: Selena does know _some_ channel balances! She knows the balances of the channels that her own node has connected with other nodes. While this does not seem like much, it is in fact very important information for constructing a path—Selena knows the actual liquidity of her own channels. Let's update the channel graph to show this information. We will use a "?" symbol to represent the unknown balances, as shown in <<channel_graph_2>>.
[[channel_graph_2]]
.Channel graph with known and unknown balances
@ -216,11 +216,11 @@ While the "?" symbol seems ominous, a lack of certainty is not the same as compl
Uncertainty can be quantified, because we know the maximum and minimum possible liquidity and can calculate probabilities for smaller (more precise) ranges.
Once we attempt to send an HTLC we can learn more about channel balances: if we succeed, then the balance was *at least* sufficient to transport the specific amount. Meanwhile if we get a "temporary channel failure" error, the most likely reason is a lack of liquidity for the specific amount.
Once we attempt to send an HTLC we can learn more about channel balances: if we succeed, then the balance was _at least_ sufficient to transport the specific amount. Meanwhile if we get a "temporary channel failure" error, the most likely reason is a lack of liquidity for the specific amount.
[TIP]
====
You may be thinking "What's the point of learning from a successful HTLC?" After all, if it succeeded we're "done". But consider that we may be sending one part of a multi-part payment. We also may be sending other single-part payments within a short time. Anything we learn about liquidity is useful for the next attempt!
You may be thinking "What's the point of learning from a successful HTLC?" After all, if it succeeded we're "done." But consider that we may be sending one part of a multipart payment. We also may be sending other single-part payments within a short time. Anything we learn about liquidity is useful for the next attempt!
====
==== Liquidity Uncertainty and Probability
@ -233,21 +233,21 @@ To quantify the uncertainty of a channel's liquidity, we can apply probability t
* The more channels (hops), the lower the chance of success.
While these may be obvious, they have important implications, especially for the use of Multi-Path Payments (see <<mpp>>). The math is not difficult to follow.
While these may be obvious, they have important implications, especially for the use of multipath payments (see <<mpp>>). The math is not difficult to follow.
Let's use probability theory to see how we arrived at these conclusions.
First, let's posit that a channel with capacity +c+ has liquidity on one side with an unknown value in the range of +(0, c)+ or "range between 0 and c". For example, if the capacity is +5+, then the liquidity will be in the range +(0, 5)+. Now, from this we see that if we want to send 5 satoshis, our chance of success is only 1 in 6 (16.66%), because we will only succeed if the liquidity is exactly 5.
First, let's posit that a channel with capacity +c+ has liquidity on one side with an unknown value in the range of +(0, c)+ or "range between 0 and c." For example, if the capacity is +5+, then the liquidity will be in the range +(0, 5)+. Now, from this we see that if we want to send 5 satoshis, our chance of success is only 1 in 6 (16.66%), because we will only succeed if the liquidity is exactly 5.
More simply, if the possible values for the liquidity are 0,1,2,3,4,5 only one of those six possible values will be sufficient to send our payment. To continue this example, if our payment amount was 3, then we would succeed if the liquidity was 3, 4, or 5. So our chances of success are 3 in 6 (50%). Expressed in math, the success probability function for a single channel is:
More simply, if the possible values for the liquidity are 0,1,2,3,4, and 5, only one of those six possible values will be sufficient to send our payment. To continue this example, if our payment amount was 3, then we would succeed if the liquidity was 3, 4, or 5. So our chances of success are 3 in 6 (50%). Expressed in math, the success probability function for a single channel is:
latexmath:[$P_c(a) = (c + 1 - a) / (c + 1)$]
where +a+ is the amount and +c+ is the capacity
From the equation we see that if the amount is close to 0, the probability is close to 1 whereas if the amount exceeds the capacity, the probability is zero.
From the equation we see that if the amount is close to 0, the probability is close to 1, whereas if the amount exceeds the capacity, the probability is zero.
In other words: "Smaller payments have a better chance of successful delivery" or "Larger capacity channels give us better chances of delivery for a specific amount" and "You can't send a payment on a channel with insufficient capacity".
In other words: "Smaller payments have a better chance of successful delivery" or "Larger capacity channels give us better chances of delivery for a specific amount" and "You can't send a payment on a channel with insufficient capacity."
Now let's think about the probability of success across a path made of several channels. Let's say our first channel has 50% chance of success (P = 0.5). Then if our second channel has 50% chance of success (P = 0.5), it is intuitive that our overall chance is 25% (P = 0.25).
@ -255,38 +255,38 @@ We can express this as an equation that calculates the probability of a payment'
latexmath:[$P_{payment} = \prod_{i=1}^n P_i$]
Where P_i_ is the probability of success over one path or channel, and P_payment_ is the overall probability of a successful payment over all the paths/channels.
Where P~__i__~ is the probability of success over one path or channel, and P~__payment__~ is the overall probability of a successful payment over all the paths/channels.
From the equation we see that since the probability of success over a single channel is always less than or equal to 1, the probability across many channels will _drop exponentially_.
In other words, "The more channels (hops) you use, the lower the chance of success".
In other words, "The more channels (hops) you use, the lower the chance of success."
[NOTE]
====
There is a lot of mathmatical theory and modelling behind the uncertainty of the liquidity in the channels. Fundamental work about modeling the uncertainty intervals of the channel liquidity can be found in the paper https://arxiv.org/abs/2103.08576[Security and Privacy of Lightning Network Payments with Uncertain Channel Balances] by (co-author of this book) Pickhardt et. al.
There is a lot of mathmatical theory and modeling behind the uncertainty of the liquidity in the channels. Fundamental work about modeling the uncertainty intervals of the channel liquidity can be found in the paper https://arxiv.org/abs/2103.08576["Security and Privacy of Lightning Network Payments with Uncertain Channel Balances"] by (coauthor of this book) Pickhardt et al.
====
==== Fees and Other Channel Metrics
Next, our sender will add information to the graph from +channel_update+ messages received from the intermediary nodes. As a reminder, the +channel_update+ contains a wealth of information about a channel and the expectations of one of the channel partners.
In <<channel_graph_3>> below we see how Selena can update the channel graph based on +channel_update+ messages from A, B, X and Y. Note that the channel ID and channel direction (included in +channel_flags+) tells Selena which channel and which direction this update refers to. Each channel partner gossips one or more +channel_update+ messages to announce their fee expectations and other information about the channel. For example, in the top left we see the +channel_update+ sent by (A)lice for the channel A--B and the direction A-to-B. With this update, Alice tells the network how much she will charge in fees to route an HTLC to Bob over that specific channel. Bob may announce a channel update (not shown in this diagram) for the opposite direction with completely different fee expectations. Any node may send a new +channel_update+ to change the fees or timelock expectations at any time.
In <<channel_graph_3>> we see how Selena can update the channel graph based on +channel_update+ messages from A, B, X, and Y. Note that the channel ID and channel direction (included in +channel_flags+) tell Selena which channel and which direction this update refers to. Each channel partner gossips one or more +channel_update+ messages to announce their fee expectations and other information about the channel. For example, in the top left we see the +channel_update+ sent by Alice for the channel A--B and the direction A-to-B. With this update, Alice tells the network how much she will charge in fees to route an HTLC to Bob over that specific channel. Bob may announce a channel update (not shown in this diagram) for the opposite direction with completely different fee expectations. Any node may send a new +channel_update+ to change the fees or timelock expectations at any time.
[[channel_graph_3]]
.Channel graph fees and other channel metrics
image::images/mtln_1206.png[]
The fee and timelock information are very important not just as path selection metrics. As we saw in <<onion_routing>>, the sender needs to add up fees and timelocks (cltv_expiry_delta) at each hop to make the onion. The process of calculating fees happens from the recipient to the sender *backwards* along the path, because each intermediary hop expects an incoming HTLC with higher amount and expiry timelock than the outgoing HTLC they will send to the next hop. So, for example, if Bob wants 1000 satoshis in fees and 30 blocks of expiry timelock delta, to send a payment to Rashid, then that amount and expiry delta must be added to the HTLC _from Alice_.
The fee and timelock information are very important not just as path selection metrics. As we saw in <<onion_routing>>, the sender needs to add up fees and timelocks (+cltv_expiry_delta+) at each hop to make the onion. The process of calculating fees happens from the recipient to the sender _backward_ along the path because each intermediary hop expects an incoming HTLC with higher amount and expiry timelock than the outgoing HTLC they will send to the next hop. So, for example, if Bob wants 1,000 satoshis in fees and 30 blocks of expiry timelock delta to send a payment to Rashid, then that amount and expiry delta must be added to the HTLC _from Alice_.
It is also important to note that a channel must have liquidity that is sufficient not only for the payment amount but also for the cumulative fees of all the subsequent hops. Even though Selena's channel to Xavier (S-->X) has enough liquidity for a 1m satoshi payment, it *does not* have enough liquidity once we consider fees. We need to know fees because only paths that have sufficient liquidity for *both payment and all fees* will be considered.
It is also important to note that a channel must have liquidity that is sufficient not only for the payment amount but also for the cumulative fees of all the subsequent hops. Even though Selena's channel to Xavier (S->X) has enough liquidity for a 1M satoshi payment, it _does not_ have enough liquidity once we consider fees. We need to know fees because only paths that have sufficient liquidity for _both payment and all fees_ will be considered.
=== Finding Candidate Paths
Finding a suitable path through a directed graph like this is a well-studied computer science problem (known broadly as the "Shortest Path problem"), which can be solved by a variety of algorithms depending on the desired optimization and resource constraints.
Finding a suitable path through a directed graph like this is a well-studied computer science problem (known broadly as the _shortest path problem_), which can be solved by a variety of algorithms depending on the desired optimization and resource constraints.
The most famous algorithm solving this problem was invented by Dutch mathematician E. W. Dijkstra in 1956, known simply as https://en.wikipedia.org/wiki/Dijkstra's_algorithm[_Dijkstra's Algorithm_]. In addition to the original Dijkstra algorithm, there are many variations and optimizations, such as https://en.wikipedia.org/wiki/A*_search_algorithm[A* ("A-star")], which is a heuristic-based algorithm.
The most famous algorithm solving this problem was invented by Dutch mathematician E. W. Dijkstra in 1956, known simply as https://en.wikipedia.org/wiki/Dijkstra's_algorithm[_Dijkstra's algorithm_]. In addition to the original Dijkstra's algorithm, there are many variations and optimizations, such as https://en.wikipedia.org/wiki/A*_search_algorithm[A* ("A-star")], which is a heuristic-based algorithm.
As mentioned previously, the "search" must be applied _backwards_ to account for fees that are accumulated from recipient to sender. Thus, Dijkstra, A* or some other algorithm would search for a path from the recipient to the sender, using fees, estimated liquidity, timelock delta (or some combination) as a cost function for each hop.
As mentioned previously, the "search" must be applied _backward_ to account for fees that are accumulated from recipient to sender. Thus, Dijkstra, A*, or some other algorithm would search for a path from the recipient to the sender, using fees, estimated liquidity, and timelock delta (or some combination) as a cost function for each hop.
Using one such algorithm, Selena calculates several possible paths to Rashid, sorted by shortest path:
@ -299,25 +299,25 @@ Using one such algorithm, Selena calculates several possible paths to Rashid, so
4. S->A->B->X->Y->R
But, as we saw previously, the channel +S->X+ does not have enough liquidity for a 1m satoshi payment once fees are considered. So paths 2 and 3 are not viable. That leaves paths 1 and 4 as possible paths for the payment.
But, as we saw previously, the channel +S->X+ does not have enough liquidity for a 1M satoshi payment once fees are considered. So Paths 2 and 3 are not viable. That leaves Paths 1 and 4 as possible paths for the payment.
With two possible paths, Selena is ready to attempt delivery!
=== Payment Delivery (Trial-And-Error Loop)
Selena's node starts the trial-and-error loop, by constructing the HTLCs, building the onion and attempting delivery of the payment. For each attempt, there are three possible outcomes:
Selena's node starts the trial-and-error loop by constructing the HTLCs, building the onion, and attempting delivery of the payment. For each attempt, there are three possible outcomes:
- A successful result (+update_fulfill_htlc+)
- An error (+update_fail_htlc+)
- A "stuck" payment with no response (neither success, nor failure)
- A "stuck" payment with no response (neither success nor failure)
If the payment fails, then it can be re-tried via a different path by updating the graph (changing a channel's metrics) and recalculating an alternative path.
If the payment fails, then it can be retried via a different path by updating the graph (changing a channel's metrics) and recalculating an alternative path.
We'll look at what happens if the payment is "stuck" in <<stuck_payments>>. The important detail is that a stuck payment is the worst outcome because we cannot retry with another HTLC as both (the stuck one and the retry one) might go through eventually and cause a double payment.
We looked at what happens if the payment is "stuck" in <<stuck_payments>>. The important detail is that a stuck payment is the worst outcome because we cannot retry with another HTLC because both (the stuck one and the retry one) might go through eventually and cause a double payment.
==== First Attempt (Path #1)
Selena attempts the first path (S->A->B->R). She constructs the onion and sends it, but receives a failure code from Bob's node. Bob reports back a +temporary channel failure+ with a +channel_update+ identifying the channel B->R as the one that can't deliver. This attempt is shown in <<path_1_fail>>:
Selena attempts the first path (S->A->B->R). She constructs the onion and sends it, but receives a failure code from Bob's node. Bob reports back a +temporary channel failure+ with a +channel_update+ identifying the channel B->R as the one that can't deliver. This attempt is shown in <<path_1_fail>>.
From this failure code, Selena will deduce that Bob doesn't have enough liquidity to deliver the payment to Rashid on that channel. Importantly, this failure narrows the uncertainty of the liquidity of that channel! Previously, Selena's node assumed that the liquidity on Bob's side of the channel was somewhere in the range (0, 4m). Now, she can assume that the liquidity is in the range (0, 999999). Similarly, Selena can now assume that the liquidity of that channel on Rashid's side is in the range (1m, 4m), instead of (0, 4m). Selena has learned a lot from this failure.
From this failure code, Selena will deduce that Bob doesn't have enough liquidity to deliver the payment to Rashid on that channel. Importantly, this failure narrows the uncertainty of the liquidity of that channel! Previously, Selena's node assumed that the liquidity on Bob's side of the channel was somewhere in the range (0, 4M). Now, she can assume that the liquidity is in the range (0, 999999). Similarly, Selena can now assume that the liquidity of that channel on Rashid's side is in the range (1M, 4M), instead of (0, 4M). Selena has learned a lot from this failure.
==== Second Attempt (Path #4)
Now Selena attempts the fourth candidate path (S->A->B->X->Y->R). This is a longer path and will incur more fees, but it's now the best option for delivery of the payment.
Fortunately, Selena receives an +update_fulfill_htlc+ message from Alice, indicating that the payment was successful, as shown in <<path_4_success>>:
Fortunately, Selena receives an +update_fulfill_htlc+ message from Alice, indicating that the payment was successful, as shown in <<path_4_success>>.
Selena has also learnt a lot from this successful payment. She now knows that all the channels on the path S->A->B->X->Y->R had enough liquidity to deliver the payment. Furthermore, she now knows that each of these channels has moved the HTLC amount (1m + fees) to the other end of the channel. This allows Selena to recalculate the range of liquidity on the receiving side of all the channels in that path, replacing the minimum liquidity with 1m+fees.
Selena has also learnt a lot from this successful payment. She now knows that all the channels on the path S->A->B->X->Y->R had enough liquidity to deliver the payment. Furthermore, she now knows that each of these channels has moved the HTLC amount (1M + fees) to the other end of the channel. This allows Selena to recalculate the range of liquidity on the receiving side of all the channels in that path, replacing the minimum liquidity with 1M + fees.
===== Stale knowledge?
Selena now has a much better "map" of the Lightning Network (at least as far as these 7 channels go). This knowledge will be useful for any subsequent payments that Selena attempts to make.
Selena now has a much better "map" of the Lightning Network (at least as far as these seven channels go). This knowledge will be useful for any subsequent payments that Selena attempts to make.
However, this knowledge becomes somewhat "stale" as the other nodes send or route payments. Selena will never see any of these payments (unless she is the sender). Even if she is involved in routing payments, the onion routing mechanism means she can only see the changes for one hop (her own channels).
Therefore, Selena's node must consider how long to keep this knowledge before assuming that it is stale and no longer useful.
[[mpp]]
=== Multi-Path Payments (MPP)
=== Multipath Payments (MPP)
_Multi-Path Payments (MPP)_ are a feature that was introduced in the Lightning Network in 2020 and is already very widely available. Multi-Path Payments allow a payment to be split into multiple _parts_ which are sent as HTLCs over several different paths to the intended recipient, preserving the _atomicity_ of the overall payment. In this context, atomicity means that either all the HTLC parts of a payment are eventually fulfilled or the entire payment fails and all the HTLC parts fail. There is no possibility of a partially successful payment.
_Multipath payments (MPP)_ are a feature that was introduced in the Lightning Network in 2020 and is already very widely available. Multipath payments allow a payment to be split into multiple _parts_ which are sent as HTLCs over several different paths to the intended recipient, preserving the _atomicity_ of the overall payment. In this context, atomicity means that either all the HTLC parts of a payment are eventually fulfilled or the entire payment fails and all the HTLC parts fail. There is no possibility of a partially successful payment.
Multi-Path Payments are a significant improvement in the Lightning Network as they make it possible to send amounts that won't "fit" in any single channel by splitting them into smaller amounts for which there is sufficient liquidity. Furthermore, Multi-Path Payments have been shown to increase the probability of a successful payment, as compared to a single-path payment.
Multipath payments are a significant improvement in the Lightning Network because they make it possible to send amounts that won't "fit" in any single channel by splitting them into smaller amounts for which there is sufficient liquidity. Furthermore, multipath payments have been shown to increase the probability of a successful payment, as compared to a single-path payment.
[TIP]
====
Now that MPP is available it is best to think of a single-path payment as a subcategory of a MPP. Essentially, a single-path is just a multi-path of size one. All payments can be considered as Multi-Path Payments unless the size of the payment and liquidity available make it possible to deliver with a single part.
Now that MPP is available it is best to think of a single-path payment as a subcategory of an MPP. Essentially, a single-path is just a multipath of size one. All payments can be considered as multipath payments unless the size of the payment and liquidity available make it possible to deliver with a single part.
====
==== Using MPP
MPP is not something that a user will select, but rather it is a node path-finding and payment delivery strategy. The same basic steps are implemented: create a graph, select paths and the trial-and-error loop. The difference is that during path selection we must also consider how to split the payment in order to optimize delivery.
MPP is not something that a user will select, but rather it is a node pathfinding and payment delivery strategy. The same basic steps are implemented: create a graph, select paths, and the trial-and-error loop. The difference is that during path selection we must also consider how to split the payment to optimize delivery.
In our example we can see some immediate improvements to our path finding problem that become possible with MPP. First, we can utilize the S->X channel that has known insufficient liquidity to transport 1m satoshis plus fees. By sending a smaller part along that channel, we can use paths that were previously unavailable. Second, we have the unknown liquidity of the B->R channel, which is insufficient to transport the 1m amount, but might be sufficient to transport a smaller amount.
In our example we can see some immediate improvements to our path finding problem that become possible with MPP. First, we can utilize the S->X channel that has known insufficient liquidity to transport 1M satoshis plus fees. By sending a smaller part along that channel, we can use paths that were previously unavailable. Second, we have the unknown liquidity of the B->R channel, which is insufficient to transport the 1M amount, but might be sufficient to transport a smaller amount.
===== Splitting payments
The fundamental question is how to split the payments. More specifically, what is the optimal number of splits and the optimal amounts for each split?
The fundamental question is how to split the payments. More specifically, what are the optimal number of splits and the optimal amounts for each split?
This is an area of ongoing research, where novel strategies are emerging. Multi-path payments lead to a different algorithmic approach than single path payments, even though single-path solutions can emerge from a multi-path optimization (i.e. a single-path may be the optimal solution suggested by a multi-path path finding algorithm).
This is an area of ongoing research where novel strategies are emerging. Multipath payments lead to a different algorithmic approach than single-path payments, even though single-path solutions can emerge from a multipath optimization (i.e., a single path may be the optimal solution suggested by a multipath path finding algorithm).
If you recall, we found that the uncertainty of liquidity/balances leads to some (somewhat obvious) conclusions that we can apply in MPP path finding, namely:
* Smaller payments have a higher chance of succeeding
* Smaller payments have a higher chance of succeeding.
* The more channels you use, the chance of success becomes (exponentially) lower.
From the first of these insights, we might conclude that splitting a large payment (e.g. 1 million satoshis) into tiny payments increases the chance that each of those smaller payments will succeed. The number of possible paths with sufficient liquidity will be greater if we send smaller amounts.
From the first of these insights, we might conclude that splitting a large payment (e.g., 1 million satoshis) into tiny payments increases the chance that each of those smaller payments will succeed. The number of possible paths with sufficient liquidity will be greater if we send smaller amounts.
To take this idea to an extreme, why not split the 1m satoshi payment into one million separate 1-satoshi parts? Well, the answer lies in our second insight: since we would be using more channels/paths to send our million single-satoshi HTLCs, our chance of success would drop exponentially.
To take this idea to an extreme, why not split the 1M satoshi payment into one million separate 1-satoshi parts? Well, the answer lies in our second insight: since we would be using more channels/paths to send our million single-satoshi HTLCs, our chance of success would drop exponentially.
If it's not obvious, the two insights above create a "sweet spot" where we can maximize our chances of success: splitting into smaller payments but not too many splits!
If it's not obvious, the two preceding insights create a "sweet spot" where we can maximize our chances of success: splitting into smaller payments but not too many splits!
Quantifying this optimal balance of size/number-of-splits for a given channel graph is out of the scope of this book but it is an active area of research. Some current implementations use a very simple strategy of splitting the payment in two halves, four quarters etc.
Quantifying this optimal balance of size/number-of-splits for a given channel graph is out of the scope of this book, but it is an active area of research. Some current implementations use a very simple strategy of splitting the payment in two halves, four quarters, etc.
[NOTE]
====
To read more about the optimization problem known as Minimum-Cost Flows involved when splitting payments into different sizes and allocating them to paths see the paper: https://arxiv.org/abs/2107.05322[Optimally Reliable & Cheap Payment Flows on the Lightning Network] by (co-author of this book) René Pickhardt & Stefan Richter.
To read more about the optimization problem known as minimum-cost flows involved when splitting payments into different sizes and allocating them to paths see the paper https://arxiv.org/abs/2107.05322["Optimally Reliable & Cheap Payment Flows on the Lightning Network"] by (coauthor of this book) René Pickhardt and Stefan Richter.
====
In our example, Selena's node will attempt to split the 1m satoshi payment into two parts with 600k and 400k satoshi respectively and send them on two different paths. This is shown in <<mpp_paths>>:
In our example, Selena's node will attempt to split the 1M satoshi payment into two parts with 600k and 400k satoshi, respectively, and send them on two different paths. This is shown in <<mpp_paths>>.
[[mpp_paths]]
.Sending two parts of a multi-path payment
.Sending two parts of a multipath payment
image::images/mtln_1209.png[]
Because the S->X channel can now be utilized, and (luckily for Selena), the B->R channel has sufficient liquidity for 600k satoshis, the two parts are successful along paths that were previously not possible.
==== Trial-And-Error Over Multiple "Rounds"
Multi-Path Payments lead to a somewhat modified "trial-and-error" loop for payment delivery. Because we are attempting multiple paths in each attempt, we have four possible outcomes:
Multipath payments lead to a somewhat modified trial-and-error loop for payment delivery. Because we are attempting multiple paths in each attempt, we have four possible outcomes:
* All parts succeed, the payment is successful
* Some parts succeed, some fail with errors returned
* All parts fail with errors returned
* Some parts are "stuck", no errors are returned
* Some parts are "stuck," no errors are returned
In the second case, where some parts fail with errors returned and some parts succeed, we can now _repeat_ the trial-and-error loop, but *only for the residual amount*.
In the second case, where some parts fail with errors returned and some parts succeed, we can now _repeat_ the trial-and-error loop, but _only for the residual amount_.
Let's assume for example that Selena had a much larger channel graph with hundreds of possible paths to reach Rashid. Her path finding algorithm might find an optimal payment split consisting of 26 parts of varying sizes. After attempting to send all 26 parts in the first round, 3 of those parts failed with errors.
Let's assume for example that Selena had a much larger channel graph with hundreds of possible paths to reach Rashid. Her path finding algorithm might find an optimal payment split consisting of 26 parts of varying sizes. After attempting to send all 26 parts in the first round, three of those parts failed with errors.
If those 3 parts consisted of, say 155k satoshis, then Selena would restart the path finding effort, only for 155k satoshis. The next round could find completely different paths (optimized for the residual amount of 155k), and split the 155k amount into completely different splits!
If those three parts consisted of, say 155k satoshis, then Selena would restart the path finding effort, only for 155k satoshis. The next round could find completely different paths (optimized for the residual amount of 155k), and split the 155k amount into completely different splits!
[TIP]
====
While it seems like 26 split parts are a lot, tests on the Lightning Network have successfully delivered a payment of 0.3679 BTC by splitting it into 345 parts.
====
Furthermore, Selena's node would update the channel graph using the information gleaned from the successes and errors of the first round, to find the most optimal paths and splits for the second round.
Furthermore, Selena's node would update the channel graph using the information gleaned from the successes and errors of the first round to find the most optimal paths and splits for the second round.
Let's say that Selena's node calculates that the best way to send the 155k residual is 6 parts split as 80k, 42k, 15k, 11k, 6.5k and 500 satoshis. In the next round, Selena gets only one error, indicating that the 11k satoshi part failed. Again, Selena updates the channel graph based on the information gleaned and runs the path finding again, to send the 11k residual. This time, she succeeds with 2 parts of 6k and 5k satoshis respectively.
Let's say that Selena's node calculates that the best way to send the 155k residual is six parts split as 80k, 42k, 15k, 11k, 6.5k, and 500 satoshis. In the next round, Selena gets only one error, indicating that the 11k satoshi part failed. Again, Selena updates the channel graph based on the information gleaned and runs the path finding again to send the 11k residual. This time, she succeeds with 2 parts of 6k and 5k satoshis, respectively.
This multi-round example of sending a payment using MPP is shown in <<mpp_rounds>>:
This multiround example of sending a payment using MPP is shown in <<mpp_rounds>>.
[[mpp_rounds]]
.Sending a payment in multiple rounds with MPP
image::images/mtln_1210.png[]
In the end, Selena's node used three rounds of path finding to send the 1m satoshis in 30 parts.
In the end, Selena's node used three rounds of path finding to send the 1M satoshis in 30 parts.
=== Conclusion
@ -438,4 +438,4 @@ In this chapter we looked at path finding and payment delivery. We saw how to us
We also examined the uncertainty of channel liquidity (from the perspective of the sender) and the implications that has for path finding. We saw how we can quantify the uncertainty and use probability theory to draw some useful conclusions. We also saw how we can reduce uncertainty by learning from both successful and failed payments.
Finally, we saw how the newly deployed Multi-Path Payments feature allows us to split payments into parts, increasing the probability of success even for larger payments.
Finally, we saw how the newly deployed multipath payments feature allows us to split payments into parts, increasing the probability of success even for larger payments.
In this chapter, we dive into the wire protocol of the Lightning network,
In this chapter, we dive into the wire protocol of the Lightning Network
and also cover all the various extensibility levers that have been built into
the protocol. By the end of this chapter, and aspiring reader should be able to
write their very own wire protocol parser for the Lighting Network. In addition
the protocol. By the end of this chapter, an ambitious reader should be able to
write their very own wire protocol parser for the Lightning Network. In addition
to being able to write a custom wire protocol parser, a reader of this chapter
will gain a deep understanding with respect of the various upgrade mechanisms
will gain a deep understanding of the various upgrade mechanisms
that have been built into the protocol.
=== Messaging Layer in the Lightning Protocol Suite
The messaging layer, which is detailed in this chapter, consists of _Message Framing and Format_, _Type Length Value (TLV)_ encoding, and _Feature Bits_. These components are highlighted by a double outline in the protocol suite, shown in <<LN_protocol_wire_message_highlight>>:
The messaging layer, which is detailed in this chapter, consists of "Message Framing and Format," "Type Length Value (TLV)" encoding, and "Feature Bits." These components are highlighted by a double outline in the protocol suite, shown in <<LN_protocol_wire_message_highlight>>.
We begin by describing the highlevel structure of the wire _framing_
We begin by describing the high-level structure of the wire _framing_
within the protocol. When we say framing, we mean the way that the bytes are
packed on the wire to _encode_ a particular protocol message. Without knowledge
of the framing system used in the protocol, a string of bytes on the wire would
resemble a series of random bytes as no structure has been imposed. By applying
resemble a series of random bytes because no structure has been imposed. By applying
proper framing to decode these bytes on the wire, we'll be able to extract
structure and finally parse this structure into protocol messages within our
higher-level language.
It's important to note that as the Lightning Network is an _end to end
It's important to note that the Lightning Network is an _end-to-end
encrypted_ protocol, and the wire framing is itself encapsulated within an
_encrypted_ message transport layer. As we see in <<encrypted_message_transport>> the Lighting
Network uses a custom variant of the Noise protocol to handle
_encrypted_ message transport layer. As we see in <<encrypted_message_transport>>, the Lightning
Network uses a custom variant of the Noise Protocol to handle
transport encryption. Within this chapter, whenever we give an example of wire
framing, we assume the encryption layer has already been stripped away (when
decoding), or that we haven't yet encrypted the set of bytes before we send
@ -42,9 +42,9 @@ them on the wire (encoding).
With that said, we're ready to describe the high-level schema used to
encode messages on the wire:
* Messages on the wire begin with a _2byte_ type field, followed by a
* Messages on the wire begin with a _2-byte_ type field, followed by a
message payload.
* The message payload itself, can be up to 65 KB in size.
* The message payload itself can be up to 65 KB in size.
* All integers are encoded in big-endian (network order).
* Any bytes that follow after a defined message can be safely ignored.
@ -53,29 +53,28 @@ encryption layer, we don't need an explicit length for each message type. This
is due to the fact that transport encryption works at the _message_ level, so
by the time we're ready to decode the next message, we already know the total
number of bytes of the message itself. Using 2 bytes for the message type
(encoded in big-endian) means that the protocol can have up to `2^16 - 1` or
`65535` distinct messages. Continuing, as we know all messages _MUST_ be below
65KB, this simplifies our parsing as we can use a _fixed_ sized buffer, and
(encoded in big-endian) means that the protocol can have up to `2^16 – 1` or
`65,535` distinct messages. Continuing, because we know all messages must be below
65KB, this simplifies our parsing as we can use a _fixed_ size buffer and
maintain strong bounds on the total amount of memory required to parse an
incoming wire message.
The final bullet point allows for a degree of _backwards_ compatibility, as new
nodes are able to provide information in the wire messages that older nodes
(which may not understand them can safely ignore). As we see below, this
The final bullet point allows for a degree of _backward_ compatibility because new nodes are able to provide information in the wire messages that older nodes
(which may not understand them) can safely ignore. As we see subsequently, this
feature combined with a very flexible wire message extensibility format also
allows the protocol to achieve _forwards_ compatibility as well.
allows the protocol to achieve _forward_ compatibility as well.
==== Type Encoding
With this highlevel background provided, we now start at the most primitive
With this high-level background provided, we now start at the most primitive
layer: parsing primitive types. In addition to encoding integers, the Lightning
Protocol also allows for encoding of a vast array of types including: variable
Protocol also allows for encoding of a vast array of types including variable
length byte slices, elliptic curve public keys, Bitcoin addresses, and
signatures. When we describe the _structure_ of wire messages later in this
chapter, we refer to the high-level type (the abstract type) rather than the
lowerlevel representation of said type. In this section, we peel back this
lower-level representation of said type. In this section, we peel back this
abstraction layer to ensure our future wire parser is able to properly
encoding/decode any of the higherlevel types.
encoding/decode any of the higher-level types.
In <<message_types>>, we map the name of a given message type to the
high-level routine used to encode/decode the type.
@ -84,20 +83,20 @@ high-level routine used to encode/decode the type.
.High-level message types
[options="header"]
|===
| High Level Type | Framing | Comment
| `node_alias` | A 32-byte fixed-length byte slice. | When decoding, reject if contents are not a valid UTF-8 string.
| `channel_id` | A 32-byte fixed-length byte slice that maps an outpoint to a 32 byte value. | Given an outpoint, one can convert it to a `channel_id` by taking the txid of the outpoint and XOR'ing it with the index (interpreted as the lower 2 bytes).
| `short_chan_id` | An unsigned 64-bit integer (`uint64`) | Composed of the block height (24 bits), transaction index (24 bits), and output index (16 bits) packed into 8 bytes.
| `milli_satoshi` | An unsigned 64-bit integer (`uint64`) | Represents 1000th of a satoshi.
| `satoshi` | An unsigned 64-bit integer (`uint64`) | The based unit of bitcoin.
| `pubkey` | An secp256k1 public key encoded in _compressed_ format, occupying 33 bytes. | Occupies a fixed 33-byte length on the wire.
| High-level type | Framing | Comment
| `node_alias` | A 32-byte fixed-length byte slice | When decoding, reject if contents are not a valid UTF-8 string
| `channel_id` | A 32-byte fixed-length byte slice that maps an outpoint to a 32-byte value | Given an outpoint, one can convert it to a `channel_id` by taking the TxID of the outpoint and XORing it with the index (interpreted as the lower 2 bytes)
| `short_chan_id` | An unsigned 64-bit integer (`uint64`) | Composed of the block height (24 bits), transaction index (24 bits), and output index (16 bits) packed into 8 bytes
| `milli_satoshi` | An unsigned 64-bit integer (`uint64`) | Represents 1000th of a satoshi
| `satoshi` | An unsigned 64-bit integer (`uint64`) | The base unit of bitcoin
| `pubkey` | An secp256k1 public key encoded in _compressed_ format, occupying 33 bytes | Occupies a fixed 33-byte length on the wire
| `sig` | An ECDSA signature of the secp256k1 Elliptic Curve. | Encoded as a _fixed_ 64-byte byte slice, packed as `R \|\| S`
| `uint8` | An 8-bit integer. |
| `uint16` | A 16-bit integer. |
| `uint64` | A 64-bit integer. |
| `[]byte` | A variable length byte slice. | Prefixed with a 16-bit integer denoting the length of the bytes.
| `color_rgb` | RGB color encoding. | Encoded as a series if 8-bit integers.
| `net_addr` | The encoding of a network address. | Encoded with a 1 byte prefix that denotes the type of address, followed by the address body.
| `uint8` | An 8-bit integer |
| `uint16` | A 16-bit integer |
| `uint64` | A 64-bit integer |
| `[]byte` | A variable length byte slice | Prefixed with a 16-bit integer denoting the length of the bytes
| `color_rgb` | RGB color encoding | Encoded as a series if 8-bit integers
| `net_addr` | The encoding of a network address | Encoded with a 1-byte prefix that denotes the type of address, followed by the address body
|===
In the next section, we describe the structure of each of the wire messages
@ -108,23 +107,23 @@ body.
=== Type Length Value (TLV) Message Extensions
Earlier in this chapter we mentioned that messages can be up to 65 KB in size,
and if while parsing a messages, extra bytes are left over, then those bytes
and if while parsing a message, extra bytes are left over, then those bytes
are to be ignored. At an initial glance, this requirement may appear to be
somewhat arbitrary, however this requirement allows for de-coupled de-synchronized evolution of the Lighting
Protocol itself. We discuss this more towards the end of the chapter. But first, we turn our attention to exactly what those "extra bytes" at
somewhat arbitrary; however, this requirement allows for decoupled desynchronized evolution of the Lightning
Protocol itself. We discuss this more toward the end of the chapter. But first, we turn our attention to exactly what those "extra bytes" at
the end of a message can be used for.
==== The Protocol Buffer Message Format
==== The Protocol Buffers Message Format
The Protocol Buffer (Protobuf) message serialization format started out as an
internal format used at Google, and has blossomed into one of the most popular
The Protocol Buffers (Protobuf) message serialization format started out as an
internal format used at Google and has blossomed into one of the most popular
message serialization formats used by developers globally. The Protobuf format
describes how a message (usually some sort of data structure related to an API)
is encoded on the wire and decoded on the other end. Several "Protobuf
compilers" exists in dozens of languages which act as a bridge that allows any
language to encode a Protobuf that will be able to decode by a compliant decode
in another language. Such crosslanguage data structure compatibility allows
for a wide range of innovation, because it's possible to transmit structure and even
in another language. Such cross-language data structure compatibility allows
for a wide range of innovation because it's possible to transmit structure and even
typed data structures across language and abstraction boundaries.
Protobufs are also known for their flexibility with respect to how they
@ -132,41 +131,38 @@ handle changes in the underlying messages structure. As long as the field
numbering schema is adhered to, then it's possible for a newer write of
Protobufs to include information within a Protobuf that may be unknown to any
older readers. When the old reader encounters the new serialized format, if
there're types/fields that it doesn't understand, then it simply _ignores_
them. This allows old clients and new clients to co-exist, as all clients can
there are types/fields that it doesn't understand, then it simply _ignores_
them. This allows old clients and new clients to coexist because all clients can
parse some portion of the newer message format.
==== Forwards & Backwards Compatibility
Protobufs are extremely popular amongst developers as they have built in
support for both forwards and backwards compatibility. Most developers are
likely familiar with the concept of backwards computability. In simple terms,
the principles states that any changes to a message format or API should be
done in a manner that doesn't break support for older clients. Within our
Protobuf extensibility examples above, backwards computability is achieved by
ensuring that new additions to the proto format don't break the known portions
of older readers. Forwards computability on the other hand is just as important
for de-synchronized updates however it's less commonly known. For a change to
be forwards compatible, then clients are to simply ignore any information
they don't understand. The soft for mechanism of upgrading the Bitcoin
consensus system can be said to be both forwards and backwards compatible: any
clients that don't update can still use Bitcoin, and if they encounters any
==== Forward and Backward Compatibility
Protobufs are extremely popular amongst developers because they have built in
support for both forward and backward compatibility. Most developers are
likely familiar with the concept of backward compatibility. In simple terms,
the principle states that any changes to a message format or API should be
done in a manner that doesn't break support for older clients. Within our preceding Protobuf extensibility examples, backward compatibility is achieved by
ensuring that new additions to the Protobuf format don't break the known portions
of older readers. Forward compatibility, on the other hand, is just as important
for desynchronized updates; however, it's less commonly known. For a change to
be forward compatible, clients are to simply ignore any information
they don't understand. The soft fork mechanism of upgrading the Bitcoin
consensus system can be said to be both forward and backward compatible: any
clients that don't update can still use Bitcoin, and if they encounter any
transactions they don't understand, then they simply ignore them as their funds
aren't using those new features.
[[tlv]]
=== Type-Length-Value (TLV) Format
In order to be able to upgrade messages in both a forwards and backwards
compatible manner, in addition to feature bits (more on that later), the LN
utilizes a custom message serialization format plainly called: Type Length
Value, or TLV for short. The format was inspired by the widely used Protobuf
To be able to upgrade messages in a manner that is both forward and backward
compatible, in addition to feature bits (more on that later), the Lightning Network utilizes a custom message serialization format plainly called Type-Length-Value, or TLV for short. The format was inspired by the widely used Protobuf
format and borrows many concepts by significantly simplifying the
implementation as well as the software that interacts with message parsing. A
curious reader might ask "why not just use Protobufs"? In response, the
Lighting developers would respond that we're able to have the best of the
curious reader might ask "why not just use Protobufs?" In response, the
Lightning developers would respond that we're able to have the best of the
extensibility of Protobufs while also having the benefit of a smaller
implementation and thus smaller attack. As of version v3.15.6, the Protobuf
implementation and thus smaller attack. As of version 3.15.6, the Protobuf
compiler weighs in at over 656,671 lines of code. In comparison LND's
implementation of the TLV message format weighs in at only 2.3k lines of code
(including tests).
@ -177,31 +173,31 @@ individual TLV records. A single TLV record has three components: the type of
the record, the length of the record, and finally the opaque value of the
record:
* `type`: An integer representing the name of the record being encoded.
* `length`: The length of the record.
* `value`: The opaque value of the record.
`type`:: An integer representing the name of the record being encoded.
`length`:: The length of the record.
`value`:: The opaque value of the record.
Both the `type` and `length` are encoded using a variable sized integer that's inspired by the variable sized integer (varint) used in Bitcoin's p2p protocol, called `BigSize` for short.
Both the `type` and `length` are encoded using a variable sized integer that's inspired by the variable sized integer (varint) used in Bitcoin's P2P protocol, called `BigSize` for short.
==== BigSize Integer Encoding
In its fullest form, a `BigSize`
integer can represent value up to 64-bits. In contrast to Bitcoin's varint
integer can represent value up to 64bits. In contrast to Bitcoin's varint
format, the `BigSize` format instead encodes integers using a big-endian byte
ordering.
The `BigSize` varint has the components: the discriminant and the body. In the
The `BigSize` varint has two components: the discriminant and the body. In the
context of the `BigSize` integer, the discriminant communicates to the decoder
the size of the variable size integer that follows. Remember that the unique thing about
the size of the variable sized integer that follows. Remember that the unique thing about
variable sized integers is that they allow a parser to use fewer bytes to encode
smaller integers than larger ones, saving space. Encoding of a `BigSize`
integer follows one of the four following options:
1. If the value is less than `0xfd` (`253`): Then the discriminant isn't really used, and the encoding is simply the integer itself. This allows us to encode very small integers with no additional overhead.
2. If the value is less than or equal to `0xffff` (`65535`):The discriminant is encoded as `0xfd`, which indicates that the value that follows is larger than `0xfd`, but smaller than `0xffff`). The number is then encoded as a 16-bit integer. Including the discriminant, then we can encode a value that is greater than 253, but less than 65535 using 3 bytes.
2. If the value is less than or equal to `0xffff` (`65535`):The discriminant is encoded as `0xfd`, which indicates that the value that follows is larger than `0xfd`, but smaller than `0xffff`). The number is then encoded as a 16-bit integer. Including the discriminant, then we can encode a value that is greater than 253, but less than 65,535 using 3 bytes.
3. If the value is less than `0xffffffff` (`4294967295`): The discriminant is encoded as `0xfe`. The body is encoded using 32-bit integer, Including the discriminant, then we can encode a value that's less than `4,294,967,295` using 5 bytes.
3. If the value is less than `0xffffffff` (`4294967295`): The discriminant is encoded as `0xfe`. The body is encoded using 32-bit integer, including the discriminant, then we can encode a value that's less than `4,294,967,295` using 5 bytes.
4. Otherwise, we just encode the value as a full-size 64-bit integer.
@ -211,45 +207,45 @@ integer follows one of the four following options:
Within the context of a TLV message, record types below `2^16` are said to be _reserved_ for future use. Types beyond this
range are to be used for "custom" message extensions used by higher-level application protocols.
The `value` of a record depends on the `type`. In other words, it can take any form as parsers will attempt to interpret it depending on the context of the type itself.
The `value` of a record depends on the `type`. In other words, it can take any form because parsers will attempt to interpret it depending on the context of the type itself.
==== TLV Canonical Encoding
One issue with the Protobuf format is that encodings of the same message may
output an entirely different set of bytes when encoded by two different
versions of the compiler. Such instances of a non-canonical encoding are not
acceptable within the context of Lighting, as many messages contain a
versions of the compiler. Such instances of a noncanonical encoding are not
acceptable within the context of Lightning, as many messages contain a
signature of the message digest. If it's possible for a message to be encoded
in two different ways, then it would be possible to break the authentication of
a signature inadvertently by re-encoding a message using a slightly different
set of bytes on the wire.
In order to ensure that all encoded messages are canonical, the following
To ensure that all encoded messages are canonical, the following
constraints are defined when encoding:
* All records within a TLV stream MUST be encoded in order of strictly
* All records within a TLV stream must be encoded in order of strictly
increasing type.
* All records must minimally encode the `type` and `length` fields. In other words, the smallest `BigSize` representation for an integer MUST be used at all times.
* All records must minimally encode the `type` and `length` fields. In other words, the smallest `BigSize` representation for an integer must be used at all times.
* Each `type` may only appear once within a given TLV stream.
In addition to these encoding constraints, a series of higher-level
interpretation requirements are also defined based on the _arity_ of a given `type` integer. we dive further into these details towards the end of the
chapter once we describe how the Lighting Protocol is upgraded in practice and
interpretation requirements is also defined based on the _arity_ of a given `type` integer. We dive further into these details toward the end of the
chapter once we describe how the Lightning Protocol is upgraded in practice and
in theory.
[[feature_bits]]
=== Feature Bits & Protocol Extensibility
As the Lighting Network is a decentralized system, no single entity can enforce a
Because the Lightning Network is a decentralized system, no single entity can enforce a
protocol change or modification upon all the users of the system. This
characteristic is also seen in other decentralized networks such as Bitcoin.
However, unlike Bitcoin overwhelming consensus *is not* required to change a
subset of the Lightning Network. Lighting is able to evolve at will without a
strong requirement of coordination, as unlike Bitcoin, there is no global consensus required in the Lightning Network. Due to this fact and the several
upgrade mechanisms embedded in the Lighting Network, only the
participants that wish to use these new Lighting Network features need to
However, unlike Bitcoin, overwhelming consensus _is not_ required to change a
subset of the Lightning Network. Lightning is able to evolve at will without a
strong requirement of coordination because, unlike Bitcoin, there is no global consensus required in the Lightning Network. Due to this fact and the several
upgrade mechanisms embedded in the Lightning Network, only the
participants that wish to use these new Lightning Network features need to
upgrade, and then they are able to interact with each other.
In this section, we explore the various ways that developers and users are
@ -260,43 +256,44 @@ manner.
==== Feature Bits As an Upgrade Discoverability Mechanism
An astute reader may have noticed the various locations that "feature bits" are
included within the Lightning Protocol. A "feature bit" is a bitfield that can
An astute reader may have noticed the various locations that feature bits are
included within the Lightning Protocol. A _feature bit_ is a bitfield that can
be used to advertise understanding or adherence to a possible network protocol
update. Feature bits are commonly assigned in pairs, meaning that each
potential new feature/upgrade always defines two bits within the bitfield.
One bit signals that the advertised feature is _optional_, meaning that the
node knows about the feature and can use it, but doesn't
consider it required for normal operation. The other bit signals that the
feature is instead *required*, meaning that the node will not continue
feature is instead _required_, meaning that the node will not continue
operation if a prospective peer doesn't understand that feature.
Using these two bits (optional and required), we can construct a simple
compatibility matrix that nodes/users can consult in order to determine if a peer is compatible with a desired feature:
compatibility matrix that nodes/users can consult to determine if a peer is compatible with a desired feature, as shown in <<table1302>>.
From this simplified compatibility matrix, we can see that as long as the other
party knows about our feature bit, then we can interact with them using the
protocol. If the party doesn't even know about what bit we're referring to
*and* they require the feature, then we are incompatible with them. Within the
_and_ they require the feature, then we are incompatible with them. Within the
network, optional features are signaled using an _odd bit number_ while
required feature are signaled using an _even bit number_. As an example, if a peer signals that they known of a feature that uses bit +15+, then we know that
required features are signaled using an _even bit number_. As an example, if a peer signals that they known of a feature that uses bit +15+, then we know that
this is an optional feature, and we can interact with them or respond to
their messages even if we don't know about the feature. If
they instead signaled the feature using bit +16+, then we know this is a
required feature, and we can't interact with them unless our node also
understands that feature.
The Lighting developers have come up with an easy to remember phrase that
encodes this matrix: "it's OK to be odd". This simple rule allows for a
The Lightning developers have come up with an easy to remember phrase that
encodes this matrix: "it's OK to be odd." This simple rule allows for a
rich set of interactions within the protocol, as a simple bitmask operation
between two feature bit vectors allows peers to determine if certain
interactions are compatible with each other or not. In other words, feature
@ -304,12 +301,12 @@ bits are used as an upgrade discoverability mechanism: they easily allow to
peers to understand if they are compatible or not based on the concepts of
optional, required, and unknown feature bits.
Feature bits are found in the: `node_announcement`, `channel_announcement`, and
Feature bits are found in the `node_announcement`, `channel_announcement`, and
`init` messages within the protocol. As a result, these three messages can be
used to signal the knowledge and/or understanding of in-flight protocol
updates within the network. The feature bits found in the `node_announcement`
message can allow a peer to determine if their _connections_ are compatible or
not. The feature bits within the `channel_announcement` messages allows a peer
not. The feature bits within the `channel_announcement` messages allow a peer
to determine if a given payment type or HTLC can transit through a given peer or
not. The feature bits within the `init` message allow peers to understand if
they can maintain a connection, and also which features are negotiated for the
@ -317,51 +314,51 @@ lifetime of a given connection.
==== TLV for Forward and Backward Compatibility
As we learned earlier in the chapter, Type-Length-Value, or TLV records can be
used to extend messages in a forwards and backwards compatible manner.
Overtime, these records have been used to extend existing messages without
As we learned earlier in the chapter, TLV records can be
used to extend messages in a forward and backward compatible manner.
Overtime, these records have been used to extend existing messages without
breaking the protocol by utilizing the "undefined" area within a message beyond
that set of known bytes.
As an example, the original Lighting Protocol didn't have a concept of the
As an example, the original Lightning Protocol didn't have a concept of the
"largest amount HTLC" that could traverse through a channel as dictated by a routing
policy. Later on, the `max_htlc` field was added to the `channel_update`
message to phase-in this concept over time. Peers that received a
`channel_update` that set such a field but didn't even know the upgrade existed
where unaffected by the change, but have their HTLCs rejected if they are
message to phase in this concept over time. Peers that receive a
`channel_update` that sets such a field but don't even know the upgrade existed
are unaffected by the change, but have their HTLCs rejected if they are
beyond the limit. Newer peers, on the other hand, are able to parse, verify,
and utilize the new field.
Those familiar with the concept of soft-forks in Bitcoin may now see some
Those familiar with the concept of softforks in Bitcoin may now see some
similarities between the two mechanisms. Unlike Bitcoin consensus-level
soft-forks, upgrades to the Lighting Network don't require overwhelming
consensus in order to adopt. Instead, at minimum, only two peers within the
network need to understand a new upgrade in order to start using it. Commonly these two peers may be the recipient and sender of a
soft forks, upgrades to the Lightning Network don't require overwhelming
consensus to be adopted. Instead, at minimum, only two peers within the
network need to understand a new upgrade to start using it. Commonly these two peers may be the recipient and sender of a
payment, or may be the channel partners of a new payment channel.
==== A Taxonomy of Upgrade Mechanisms
Rather than there being a single widely utilized upgrade mechanism within the
network (such as soft-forks for Bitcoin), there exist several possible upgrade mechanisms within the Lighting Network. In this
section, we enumerate these upgrade mechanisms, and
network (such as softforks for Bitcoin), there exist several possible upgrade mechanisms within the Lightning Network. In this
section, we enumerate these upgrade mechanisms and
provide a real-world example of their use in the past.
===== Internal network upgrades
We start with the upgrade type that requires the most protocol-level
coordination: internal network upgrades. An internal network upgrade is
characterized by one that requires *every single node* within a prospective payment path to understand the new feature. Such an upgrade is similar to any
characterized by one that requires _every single node_ within a prospective payment path to understand the new feature. Such an upgrade is similar to any
upgrade within the internet that requires hardware-level upgrades within
the core-relay portion of the upgrade. In the context of LN however, we deal
the core-relay portion of the upgrade. In the context of the Lightning Network, however, we deal
with pure software, so such upgrades are easier to deploy, yet they still
require much more coordination than any other upgrade mechanism in the
network.
One example of such an upgrade within the network was the introduction of a TLV
encoding for the routing information encoded within the onion
packets. The prior format used a hard-coded fixed-length message
packets. The prior format used a hardcoded fixed-length message
format to communicate information such as the next hop.
As this format was fixed it meant that new protocol-level upgrades weren't possible. The move to the more flexible TLV
Because this format was fixed, it meant that new protocol-level upgrades weren't possible. The move to the more flexible TLV
format meant that after this upgrade, any sort of feature that
modified the type of information communicated at each hop could be rolled out at will.
@ -373,9 +370,9 @@ using a mixed set of nodes.
===== End-to-end upgrades
To contrast the internal network upgrade, in this section we describe the
_end to end_ network upgrade. This upgrade mechanism differs from the internal
_end-to-end_ network upgrade. This upgrade mechanism differs from the internal
network upgrade in that it only requires the "ends" of the payment, the sender
and recipient to upgrade.
and recipient, to upgrade.
This type of upgrade allows
for a wide array of unrestricted innovation within the network. Because of the
@ -383,19 +380,19 @@ onion encrypted nature of payments within the network, those forwarding HTLCs
within the center of the network may not even know that new features are being
utilized.
One example of an end-to-end upgrade within the network was the roll-out of multi-part payments (MPP). MPP is a protocol-level feature that enables a
One example of an end-to-end upgrade within the network was the rollout of multipart payments (MPP). MPP is a protocol-level feature that enables a
single payment to be split into multiple parts or paths, to be assembled at the
recipient for settlement. The roll out our MPP was coupled with a new
recipient for settlement. The rollout of MPP was coupled with a new
`node_announcement` level feature bit that indicates that the recipient knows
how to handle partial payments. Assuming a sender and recipient know about each
other (possibly via a BOLT 11 invoice), then they're able to use the new
feature without any further negotiation.
Another example of an end-to-end upgrade are the various types of
"spontaneous" payments deployed within the network. One early type of
spontaneous payments called _keysend_ worked by simply placing the pre-image of a payment within the encrypted onion. Upon receipt, the destination would decrypt the
pre-image, then use that to settle the payment. As the entire packet is end-to-end encrypted, this payment type was safe, since none of the intermediate nodes
are able to fully unwrap the onion to uncover the payment pre-image.
_spontaneous_ payments deployed within the network. One early type of
spontaneous payments called _keysend_ worked by simply placing the preimage of a payment within the encrypted onion. Upon receipt, the destination would decrypt the
preimage, then use that to settle the payment. Because the entire packet is end-to-end encrypted, this payment type was safe, since none of the intermediate nodes
are able to fully unwrap the onion to uncover the payment preimage.
==== Channel Construction Level Updates
@ -404,11 +401,11 @@ the channel construction level, but which don't modify the structure of the HTLC
how the channel is funded or created. As an example, the eltoo channel type
can be rolled out within the network using a new `node_announcement` level
feature bit as well as a `channel_announcement` level feature bit. Only the two
peers on the sides of the channels needs to understand and advertise these new
peers on the sides of the channels need to understand and advertise these new
features. This channel pair can then be used to forward any payment type
granted the channel supports it.
Another is the "anchor outputs" channel format which allows the commitment fee to be
Another is the _anchor outputs_ channel format which allows the commitment fee to be
bumped via Bitcoin's Child-Pays-For-Parent (CPFP) fee management mechanism.
In this chapter we will review the Lightning Network's _Encrypted Message
Transport_, sometimes referred to as the _Brontide Protocol_, which allows peers to
In this chapter we will review the Lightning Network's _encrypted message
transport_, sometimes referred to as the _Brontide Protocol_, which allows peers to
establish end-to-end encrypted communication, authentication, and integrity
checking.
[NOTE]
====
Part of this chapter includes some highly technical detail about the encryption protocol and encryption algorithms used in the Lightning Encrypted Transport. You may decide to skip that section, if you are not interested in those details.
Part of this chapter includes some highly technical detail about the encryption protocol and encryption algorithms used in the Lightning encrypted transport. You may decide to skip that section, if you are not interested in those details.
====
=== Encrypted Transport in the Lightning Protocol Suite
The _Encrypted Transport_ component of the Lightning Network and its several components are shown in the left most part of the _Network Connection Layer_ in <<LN_protocol_encrypted_transport_highlight>>:
The _encrypted transport_ component of the Lightning Network and its several components are shown in the left most part of the _Network Connection Layer_ in <<LN_protocol_encrypted_transport_highlight>>.
[[LN_protocol_encrypted_transport_highlight]]
.The Lightning Network Protocol Suite
@ -25,44 +25,42 @@ Unlike the vanilla Bitcoin P2P network, every node in the Lightning Network is
identified by a unique public key which serves as it identity. By default, this
public key is used to end-to-end encrypt _all_ communication within the
network. Encryption by default at the lowest level of the protocol ensures that
all messages are authenticated, are immune to man-in-the-middle attacks,
snooping by 3rd parties, and ensures privacy at the fundamental transport
all messages are authenticated, are immune to man-in-the-middle (MITM) attacks and snooping by third parties, and ensures privacy at the fundamental transport
level. In this chapter, we'll learn about the encryption protocol used by the
Lightning network in detail. Upon completion of this chapter, the reader will
Lightning Network in detail. Upon completion of this chapter, the reader will
be familiar with the state of the art in encrypted messaging protocols, as well
as the various properties such a protocol provides to the network. It's worth
mentioning that the core of the encrypted message transport is _agnostic_ to
its usage within the context of the Lightning Network. As a result, the
custom encrypted message transport Lightning uses, can be dropped into any context
custom encrypted message transport Lightning uses can be dropped into any context
that requires encrypted communication between two parties.
=== The Channel Graph as Decentralized Public Key Infrastructure
As we learned in the chapter on multi-hop forwarding, very node has a long-term
As we learned in the chapter on multihop forwarding, every node has a long-term
identity that is used as the identifier for a vertex during path finding and
also used in the asymmetric cryptographic operations related to the creation of
onion encrypted routing packets. This public key, which serves as a node's
long-term identity is included in the DNS bootstrapping response, as well as
embedded within the Channel Graph. As a result, before a node attempts to
long-term identity, is included in the DNS bootstrapping response, as well as
embedded within the channel graph. As a result, before a node attempts to
connect out to another node on the P2P network, it already knows the public key
of the node it wishes to connect to.
Additionally, if the node being connected to already h a series of public
channels within the graph, then the connecting node is able to further verify
the sanctity of the identity of the node. As the entire channel graph is fully
Additionally, if the node being connected to already has a series of public
channels within the graph, then the connecting node is able to further verify the identity of the node. Because the entire channel graph is fully
authenticated, one can view it as a sort of decentralized public key
infrastructure: in order to register a key, a public channel in the Bitcoin
blockchain must be opened, once a node no longer has any public channels, then
infrastructure (PKI): to register a key, a public channel in the Bitcoin
blockchain must be opened, and once a node no longer has any public channels, then
they've effectively been removed from the PKI.
As Lightning is a decentralized network, it's imperative that no one central
Because Lightning is a decentralized network, it's imperative that no one central
party is designated the power to provision a public key identity within the
network. In place of a central party, the Lightning Network uses the Bitcoin
blockchain as a sybil mitigation mechanism, as gaining an identity on the
blockchain as a Sybil mitigation mechanism because gaining an identity on the
network has a tangible cost: the fee needed to create a channel in the
blockchain, as well as the opportunity cost of the capital allocated to their
channels. In the process of essentially rolling a domain specific PKI, the
Lightning network is able to significantly simply its encrypted transport
Lightning Network is able to significantly simplify its encrypted transport
protocol as it doesn't need to deal with all the complexities that come along
with TLS, the Transport Layer Security protocol.
@ -73,102 +71,100 @@ TLS used in spite of the drawbacks of the existing PKI system? It is indeed a
fact that "self signed certificates" can be used to effectively sidestep the
existing global PKI system by simply asserting to the identity of a given
public key amongst a set of peers. However, even with the existing PKI system
out of the way, TLS has several drawbacks that prompted the creators of the LN
out of the way, TLS has several drawbacks that prompted the creators of the Lightning Network
to instead opt for a more compact custom encryption protocol.
To start with, TLS is a protocol that has been around for several decades and
as a result has evolved over time as new advances have been made in the space
of transport encryption. However, overtime this evolution has caused the
of transport encryption. However, overtime this evolution has caused the
protocol to balloon in size and complexity. Over the past few decades several
vulnerabilities in TLS has been discovered, and patched with each evolution
vulnerabilities in TLS have been discovered and patched, with each evolution
further increasing the complexity of the protocol. As a result of the age of
the protocol several versions and iterations exist, meaning a client needs to
understand many of the prior iterations of the protocol in order to communicate
with a large portion of the public internet further increasing implementation
the protocol, several versions and iterations exist, meaning a client needs to
understand many of the prior iterations of the protocol to communicate
with a large portion of the public internet, further increasing implementation
complexity.
In the past several memory safety vulnerabilities have been discovered in
widely used implementations of SSL/TLS. Packaging such a protocol within every
Lightning node would serve to increase the attack surface of nodes exposed to
to the public peer to peer network. In order to increase the security of the
network as a whole, and minimize exploitable attack surface, the creators of
the LN instead opted to adopt the Noise Protocol Framework. Noise as a protocol
Lightning node would serve to increase the attack surface of nodes exposed to the public peer-to-peer network. To increase the security of the
network as a whole and minimize exploitable attack surface, the creators of
the Lightning Network instead opted to adopt the Noise Protocol Framework. Noise as a protocol
internalizes several of the security and privacy lessons learned over time due
to continual scrutiny of the TLS protocol over decades. In a way, the existence
of Noise allows the community to effective "start over", with a more compact,
of Noise allows the community to effective "start over," with a more compact,
simplified protocol that retains all the added benefits of TLS.
=== The Noise Protocol Framework
The Noise Protocol Framework is a modern, extensible, and flexible message
encryption protocol designed by the creators of the Signal protocol. The Signal
protocol is one of the most widely used message encryption protocols in the
encryption protocol designed by the creators of the Signal Protocol. The Signal Protocol is one of the most widely used message encryption protocols in the
world. It's used by both Signal and Whatsapp, which cumulatively are used by
over a billion people around the world. The Noise framework is the result of
decades of evolution both within academia as well as industry of message
encryption protocols. Lightning uses the Noise protocol framework to implement
a _messageoriented_ encryption protocol used by all nodes to communicate with
encryption protocols. Lightning uses the Noise Protocol Framework to implement
a _message-oriented_ encryption protocol used by all nodes to communicate with
each other.
A communication session using Noise has two distinct phases: the handshake
phase, and the messaging phase. Before two parties can communicate with each
phase and the messaging phase. Before two parties can communicate with each
other, they first need to arrive at a shared secret known only to them which
will be used to encrypt and authenticate messages sent to each other. A flavor
of an authenticated key agreement is used to arrive at a final shared key
between the tow parties. In the context of the Noise protocol, this
authenticated key agreement is referred to as a "handshake". Once that
between the two parties. In the context of the Noise protocol, this
authenticated key agreement is referred to as a _handshake_. Once that
handshake has been completed, both nodes can now being to send each other
encrypted messages. Each time peers need to connect, or reconnect to each
other, a fresh iteration of the handshake protocol is executed ensuring that
forward secrecy (leaking the key of a prior transcript doesn't compromise any
future transcripts) is achieved.
encrypted messages. Each time peers need to connect or reconnect to each
other, a fresh iteration of the handshake protocol is executed, ensuring that
forward secrecy is achieved (leaking the key of a prior transcript doesn't compromise any
future transcripts).
As the Noise protocol allows a protocol designer to drop choose from several
Because the Noise Protocol allows a protocol designer to choose from several
cryptographic primitives such as symmetric encryption and public key
cryptography, its customary that each flavor of the Noise protocol is referred
to by a unique name. In the spirit of "Noise", each flavor of the protocol
selects a name derived from some sort of "noise". In the context of the
cryptography, it's customary that each flavor of the Noise Protocol is referred
to by a unique name. In the spirit of "Noise," each flavor of the protocol
selects a name derived from some sort of "noise." In the context of the
Lightning Network, the flavor of the Noise protocol used is sometimes referred to
as "Brontide". A brontide is a low billowing noise, similar to what one would
as Brontide. A _brontide_ is a low billowing noise, similar to what one would
hear during a thunderstorm when very far away.
=== Lightning Encrypted Transport in Detail
In this section we will break down the Lightning Encrypted Transport protocol and delve into the details of the cryptographic algorithms and protocol used to establish encrypted, authenticated and integrity-assured communications between peers. Feel free to skip this section if you find this level of detail daunting.
In this section we will break down the Lightning encrypted transport protocol and delve into the details of the cryptographic algorithms and protocol used to establish encrypted, authenticated, and integrity-assured communications between peers. Feel free to skip this section if you find this level of detail daunting.
After the initial handshake initialization, we can begin the actual execution
of the handshake process. The handshake is compromised of a series of
three messages sent between the initiator and responder, hence referred to as
"acts". As each act is a single message sent between the parties, a handshake
of the handshake process. The handshake is composed of a series of
three messages sent between the initiator and responder, henceforth referred to as
"acts." Because each act is a single message sent between the parties, a handshake
is completed in a total of 1.5 round trips (0.5 for each act).
The first act completes the initial portion of the incremental Triple Diffie
Hellman key exchange (using a new ephemeral key generated by the initiator),
The first act completes the initial portion of the incremental triple Diffie–Hellman (DH) key exchange (using a new ephemeral key generated by the initiator)
and also ensures that the initiator actually knows the long-term public key of
the responder. During the second act, the responder transmits the thermal key
they wish to use for the session to the initiator, and one again incrementally
mixes this new key into the Triple DH handshake. During the third and final
the responder. During the second act, the responder transmits the ephemeral key
they wish to use for the session to the initiator, and once again incrementally
mixes this new key into the triple DH handshake. During the third and final
act, the initiator transmits their long-term static public key to the
responder, and executes the final DH operation to mix that into the final
responder and executes the final DH operation to mix that into the final
resulting shared secret.
====== Act One
@ -346,7 +344,7 @@ The handshake message is _exactly_ 50 bytes: 1 byte for the handshake
version, 33 bytes for the compressed ephemeral public key of the initiator,
The prefixed message length is encoded as a 2-byte big-endian integer, for a
total maximum packet length of `2 + 16 + 65535 + 16` = `65569` bytes.
total maximum packet length of `2 + 16 + 65,535 + 16` = `65,569` bytes.
====== Encrypting and Sending Messages
====== Encrypting and sending messages
In order to encrypt and send a Lightning message (`m`) to the network stream,
To encrypt and send a Lightning message (`m`) to the network stream,
given a sending key (`sk`) and a nonce (`sn`), the following steps are
completed:
@ -562,42 +560,42 @@ completed:
3. Encrypt `l` (using `ChaChaPoly-1305`, `sn`, and `sk`), to obtain `lc`
(18 bytes)
* The nonce `sn` is encoded as a 96-bit little-endian number. As the
decoded nonce is 64 bits, the 96-bit nonce is encoded as: 32 bits
of leading 0s followed by a 64-bit value.
* The nonce `sn` MUST be incremented after this step.
decoded nonce is 64 bits, the 96-bit nonce is encoded as 32 bits
of leading zeros followed by a 64-bit value.
* The nonce `sn` must be incremented after this step.
* A zero-length byte slice is to be passed as the AD (associated data).
4. Finally, encrypt the message itself (`m`) using the same procedure used to
encrypt the length prefix. Let encrypted ciphertext be known as `c`.
* The nonce `sn` MUST be incremented after this step.
encrypt the length prefix. Let this encrypted ciphertext be known as `c`.
* The nonce `sn` must be incremented after this step.
5. Send `lc || c` over the network buffer.
====== Receiving and Decrypting Messages
====== Receiving and decrypting messages
In order to decrypt the _next_ message in the network stream, the following
To decrypt the _next_ message in the network stream, the following
steps are completed:
1. Read _exactly_ 18 bytes from the network buffer.
2. Let the encrypted length prefix be known as `lc`.
3. Decrypt `lc` (using `ChaCha20-Poly1305`, `rn`, and `rk`), to obtain the size of
3. Decrypt `lc` (using `ChaCha20-Poly1305`, `rn`, and `rk`) to obtain the size of
the encrypted packet `l`.
* A zero-length byte slice is to be passed as the AD (associated data).
* The nonce `rn` MUST be incremented after this step.
4. Read _exactly_ `l+16` bytes from the network buffer, and let the bytes be
* The nonce `rn` must be incremented after this step.
4. Read _exactly_ `l+16` bytes from the network buffer, and let the bytes be
known as `c`.
5. Decrypt `c` (using `ChaCha20-Poly1305`, `rn`, and `rk`), to obtain decrypted
plaintext packet `p`.
* The nonce `rn` MUST be incremented after this step.
5. Decrypt `c` (using `ChaCha20-Poly1305`, `rn`, and `rk`) to obtain decrypted
plain-text packet `p`.
* The nonce `rn` must be incremented after this step.
===== Lightning message key rotation
Changing keys regularly and forgetting previous keys is useful to prevent the
decryption of old messages, in the case of later key leakage (i.e. backwards
decryption of old messages, in the case of later key leakage (i.e., backward
secrecy).
Key rotation is performed for _each_ key (`sk` and `rk`) _individually_. A key
is to be rotated after a party encrypts or decrypts 1000 times with it (i.e.
is to be rotated after a party encrypts or decrypts 1,000 times with it (i.e.,
every 500 messages). This can be properly accounted for by rotating the key
once the nonce dedicated to it exceeds 1000.
once the nonce dedicated to it exceeds 1,000.
Key rotation for a key `k` is performed according to the following steps:
@ -609,8 +607,8 @@ Key rotation for a key `k` is performed according to the following steps:
=== Conclusion
Lightning's underlying transport encryption, is based on the Noise protocol and offers strong security guarantees of privacy, authenticity and integrity for all communications between Lightning peers.
Lightning's underlying transport encryption is based on the Noise Protocol and offers strong security guarantees of privacy, authenticity, and integrity for all communications between Lightning peers.
Unlike Bitcoin where peers often communicate "in the clear" (without encryption), all Lightning communications are encrypted peer-to-peer. In addition to transport encryption (peer-to-peer), in the Lightning Network payments are *also* encrypted into onion packets (hop-to-hop) and payment details are sent out-of-band between the sender and recipient (end-to-end). The combination of all these security mechanisms is cumulative and provides layered defense against de-anonymization, man-in-the-middle attacks and network surveillance.
Unlike Bitcoin where peers often communicate "in the clear" (without encryption), all Lightning communications are encrypted peer-to-peer. In addition to transport encryption (peer-to-peer), in the LN payments are _also_ encrypted into onion packets (hop-to-hop) and payment details are sent out-of-band between the sender and recipient (end-to-end). The combination of all these security mechanisms is cumulative and provides layered defense against deanonymization, man-in-the-middle attacks, and network surveillance.
Of course, no security is perfect and we will see in <<security_and_privacy>> that these properties can be degraded and attacked. However, the Lightning Network significantly improves upon the privacy of Bitcoin.
As we've learned throughout the book, minimally two pieces of data are required
to complete a Lightning payment: a payment hash, and a destination. As
to complete a Lightning payment: a payment hash and a destination. As
`SHA-256` is used in the Lightning Network to implement HTLCs, this information
requires 32-bytes in order to communicate. Destinations on the other hand are
requires 32 bytes to communicate. Destinations, on the other hand, are
simply the `secp256k1` public key of the node that wishes to receive a payment.
The purpose of a payment request in the context of the Lightning Network is to
communicate these two pieces of information from sender to receiver. The QR-codefriendly format for communicating the information required
to complete a payment from receiver to sender, is described in https://github.com/lightningnetwork/lightning-rfc/blob/master/11-payment-encoding.md[BOLT #11 - Invoice Protocol for Lightning Payments]. In practice, more than just the
payment hash and destination are communicated in a payment request in order to
make the encoding more fully feature.
communicate these two pieces of information from sender to receiver. The QR-code–friendly format for communicating the information required
to complete a payment from receiver to sender is described in https://github.com/lightningnetwork/lightning-rfc/blob/master/11-payment-encoding.md[BOLT #11: Invoice Protocol for Lightning Payments]. In practice, more than just the
payment hash and destination are communicated in a payment request to
make the encoding more fully featured.
=== Lightning Payment Requests vs. Bitcoin Addresses
=== Lightning Payment Requests versus Bitcoin Addresses
A commonly asked question when people first encounter a Lightning Payment
request is: why can't a normal static address format be used instead?
In order to answer this question, you must first internalize how Lightning
To answer this question, you must first internalize how Lightning
differs from base layer Bitcoin as a payment method. Compared to a Bitcoin
address which may be used to make a potentially unbounded number of payments
(though re-using a Bitcoin address may degrade one's privacy), a Lightning
payment request should only ever be used *once*. This is due to the fact that
(though reusing a Bitcoin address may degrade one's privacy), a Lightning
payment request should only ever be used _once_. This is due to the fact that
sending a payment to a Bitcoin address essentially uses a public key
cryptosystem to "encode" the payment in a manner that only the true "owner" of
that Bitcoin address can redeem it.
In contrast, in order to complete a Lightning payment, the recipient must
In contrast, to complete a Lightning payment, the recipient must
reveal a "secret" to the entire payment route including the sender. This can be
interpreted as usage of a kind of domain specific symmetric cryptography, as
the payment pre-image is for practical purposes a nonce (number only used
interpreted as usage of a kind of domain-specific symmetric cryptography because
the payment preimage is for practical purposes a nonce (number only used
once). If the sender attempts to make another payment using that identical
payment hash, then they risk losing funds, as the payment may not actually be
delivered to the destination. It's safe to assume that after a pre-image has
been reveled, all nodes in the path will keep it around forever, then rather
than forward the HTLC in order to collect a routing fee if the payment is
payment hash, then they risk losing funds because the payment may not actually be
delivered to the destination. It's safe to assume that after a preimage has
been revealed, all nodes in the path will keep it around forever, then rather
than forward the HTLC to collect a routing fee if the payment is
completed, they can simply settle the payment at that instance and gain the
entire payment amount in return. As a result, it's unsafe to ever use a payment
request more than once.
New variants of the original Lightning Payment request exist that allow the sender to reuse them as many times as they want. These variants flip the normal payment flow as the sender transmits a pre-image within the encrypted onion payload to the receiver, who is the only
New variants of the original Lightning payment request exist that allow the sender to reuse them as many times as they want. These variants flip the normal payment flow as the sender transmits a preimage within the encrypted onion payload to the receiver, who is the only
one that is able to decrypt it and settle the payment. Alternatively, assuming
a mechanism that allows a sender to typically request a new payment request
from the receiver, then an interactive protocol can be used in order to allow a
degree of payment request re-use.
from the receiver, then an interactive protocol can be used to allow a
In this section, we'll describe the mechanism used to encode the set of
information required to complete a payment on the Lightning Network. As
mentioned earlier, the payment hash and destination is the minimum amount of
information required to complete a payment. However in practice, more
information required to complete a payment. However, in practice, more
information such as time-lock information, payment request expiration, and
possibly an on-chain fallback address are also communicated. The full specification document is https://github.com/lightningnetwork/lightning-rfc/blob/master/11-payment-encoding.md[BOLT #11 - Invoice Protocol for Lightning Payments]
possibly an on-chain fallback address are also communicated. The full specification document is https://github.com/lightningnetwork/lightning-rfc/blob/master/11-payment-encoding.md[BOLT #11: Invoice Protocol for Lightning Payments].
==== Payment Request Encoding in Practice
First, let's examine what a real payment request looks like in practice. The
following is a valid payment request that could have been used to complete a
payment on the mainnet Lightning Network at time it was created:
payment on the mainnet Lightning Network at the time it was created:
Looking at the string, we can tease out a portion that we can parse with our
eyes, while the rest of it just looks like a random set of strings. The part
that is somewhat parse able by a human is referred to as the "human readable
prefix". It allows a human to quickly extract some relevant information from a
that is somewhat parsable by a human is referred to as the _human-readable prefix_. It allows a human to quickly extract some relevant information from a
payment request at a glance. In this case, we can see that this payment is for
the mainnet instance of the Lightning network (`lnbc`), and is requesting 2500
uBTC (micro-bitcoin), or `25,0000,000` satoshis. The latter potion is referred
to as the "data" portion and uses an extensible format to encode the
the mainnet instance of the Lightning Network (`lnbc`), and is requesting 2,500
uBTC (microbitcoin), or `25,000,000` satoshis. The latter potion is referred
to as the data portion and uses an extensible format to encode the
information required to complete a payment.
Each version of instance of the Lightning Network (mainnet, testnet, etc) has
its own human readable prefix. This allows client software and also humans to
Each version of instance of the Lightning Network (mainnet, testnet, etc.) has
its own human-readable prefix (see <<table1501>>). This allows client software and also humans to
quickly determine if a payment request can be satisfied by their node or not.
.BOLT 11 Network Prefixes
[[table1501]]
.BOLT 11 network prefixes
[options="header"]
|=============================
|Network |BOLT 11 Prefix
|Network |BOLT 11 prefix
|mainnet |`lnbc`
|testnet |`lntb`
|simnet/regtest|`lnbcrt`
|=============================
The first portion of the human readable prefix is a "compact" expression of the
The first portion of the human-readable prefix is a _compact_ expression of the
amount of the payment request. The compact amount is encoded in two parts:
first, an integer is used as the "base" amt. This is then followed by a
first, an integer is used as the _base_ amount. This is then followed by a
`multiplier` that allows us to specify distinct order of magnitude increases
offset by the base amount. If we return to our initial example, then we can
take the `2500u` portion and decrease it by a factor of 1000 to instead use
`2500m` or (2500 `mBTC`). As a rule of thumb in order to ascertain the amount
take the `2500u` portion and decrease it by a factor of 1,000 to instead use
`2500m` or (2,500 `mBTC`). As a rule of thumb, to ascertain the amount
of an invoice at a glance, take the base factor and multiply it by the
`multiplier`.
A full list of the currently defined multipliers is a follows:
A full list of the currently defined multipliers is given in <<table1502>>.
.BOLT 11 Amount Multipliers
[[table1502]]
.BOLT 11 amount multipliers
[options="header"]
|==============================================
|Multiplier|Bitcoin Unit|Multiplication Factor
|Multiplier|Bitcoin unit|Multiplication factor
|`m`|milli|0.001
|`u`|micro|0.000001
|`n`|nano|0.000000001
@ -127,92 +127,92 @@ A full list of the currently defined multipliers is a follows:
==== Bech32 & the Data Segment
If the "unreadable" portion of looks familiar, then that's because it uses the
very same encoding scheme as Segwit compatible Bitcoin addresses use today,
If the "unreadable" portion looks familiar, then that's because it uses the
very same encoding scheme as SegWit compatible Bitcoin addresses use today,
namely `bech32`. Describing the `bech32` encoding scheme is outside the scope
of this chapter. In brief, it's a sophisticated way to encode short strings
that has very good error correction as well as detection properties.
The data portion can be separated into 3 sections:
The data portion can be separated into three sections:
* The timestamp.
* Zero or more tagged key-value pairs.
* The signature of the entire invoice.
* The timestamp
* Zero or more tagged key-value pairs
* The signature of the entire invoice
The timestamp is expressed in seconds since the 1970, or the Unix Epoch. This
The timestamp is expressed in seconds since the year 1970, or the Unix Epoch. This
timestamp allows the sender to gauge how old the invoice is, and as we'll see
later, allows the receiver to force an invoice to only be valid for a period of
time if they wish.
Similar to the TLV format we learned about in <<tlv>>, the BOLT 11 invoice
format uses a series of extensible key-value pairs to encode information
needed to satisfy a payment. As key-value pairs are used, it's easy for add
needed to satisfy a payment. Because key-value pairs are used, it's easy to add
new values in the future if a new payment type or additional
requirement/functionality is introduced.
Finally a signature is included that covers the entire invoice signed by the
Finally, a signature is included that covers the entire invoice signed by the
destination of the payment. This signature allows the sender to verify that the
payment request was indeed created by the destination of the payment. Unlike
Bitcoin payment request's which aren't signed, this allows us to ensure that a
Bitcoin payment requests which aren't signed, this allows us to ensure that a
particular entity signed the payment request. The signature itself is encoded
using a recovery ID, which allows a more compact signature to be used that
allows public key extraction. When verifying the signature, the verifies
allows public key extraction. When verifying the signature, the recovery ID
extracts the public key, then verifies that against the public key included in
the invoice.
===== Tagged invoice fields
The tagged invoice fields are encoded in the main "body" of the invoice. These
fields represent different key=value pairs that express either additional
information that may help complete the payment, or information which is
_required_ to complete the payment. As a slight variant of `bech32` is
The tagged invoice fields are encoded in the main body of the invoice. These
fields represent different key-value pairs that express either additional
information that may help complete the payment or information which is
_required_ to complete the payment. Because a slight variant of `bech32` is
utilized, each of these fields are actually in the "base 5" domain.
A given tag field is comprised of 3 components:
A given tag field is comprised of three components:
* The `type` of the field (5 bits).
* The `length` of the data of the field (10 bits)
* The `data` itself, which is `length* 5 bytes` in size.
* The `type` of the field (5 bits)
* The `length` of the data of the field (10 bits
* The `data` itself, which is `length* 5 bytes` in size
A full list of all the currently defined tagged fields is as follows:
A full list of all the currently defined tagged fields is given in <<table1503>>.
.BOLT 11 Tagged Invoice Fields
[[table1503]]
.BOLT 11 tagged invoice fields
[options="header"]
|===
|Field Tag|Data Length|Usage
|Field tag|Data length|Usage
|`p`|`52`|The `SHA-256` payment hash.
|`s`|`52`|A `256-bit` secret that increase the end to end privacy of a payment by mitigating probing by intermediate nodes.
|`s`|`52`|A `256-bit` secret that increases the end-to-end privacy of a payment by mitigating probing by intermediate nodes.
|`d`|Variable|The description, a short UTF-8 string of the purpose of the payment.
|`n`|`53`|The public key of the destination node.
|`h`|`52`|A hash that represents a description of the payment itself. This can be used to commit to a description that's over 639 bytes in length.
|`x`|Variable|The expiry time in seconds of the payment. The default is 1 hour (3600) if not specified.
|`x`|Variable|The expiry time in seconds of the payment. The default is 1 hour (3,600) if not specified.
|`c`|Variable|The `min_cltv_expiry` to use for the final hop in the route. The default is 9 if not specified.
|`f`|Variable|A fallback on-chain address to be used to complete the payment if the payment cannot be completed over LN.
|`f`|Variable|A fallback on-chain address to be used to complete the payment if the payment cannot be completed over the Lightning Network.
|`r`|Variable|One or more entries that allow a receiver to give the sender additional ephemeral edges to complete the payment.
`9`|Variable|A set of 5-bit values that contain the feature bits that are required in order to complete the payment.
|`9`|Variable|A set of 5-bit values that contain the feature bits that are required in order to complete the payment.
|===
The elements contained in the field `r` are commonly referred to as "routing
hints". They allow the receiver to communicate an extra set of edges that may
help the sender complete their payment. The "hints" are usually used when the
The elements contained in the field `r` are commonly referred to as _routing hints_. They allow the receiver to communicate an extra set of edges that may
help the sender complete their payment. The hints are usually used when the
receiver has some/all private channels, and they wish to guide the sender into
this "unmapped" portion of the channel graph. A routing hints encodes
this "unmapped" portion of the channel graph. A routing hint encodes
effectively the same information that a normal `channel_update` message does.
The update is itself packed into a single value with the following fields:
* The `pubkey` of the outgoing node in the edge (264 bits).
* The `short_channel_id` of the "virtual" edge (64 bits).
* The base fee (`fee_base_msat`) of the edge (32 bits).
* The proportional fee (`fee_proportional_millionths`) (32 bits).
* The CLTV expiry delta (`cltv_expiry_delta`) (16 bits).
* The `pubkey` of the outgoing node in the edge (264 bits)
* The `short_channel_id` of the "virtual" edge (64 bits)
* The base fee (`fee_base_msat`) of the edge (32 bits)
* The proportional fee (`fee_proportional_millionths`) (32 bits)
* The CLTV expiry delta (`cltv_expiry_delta`) (16 bits)
The final portion of the data segment is the set of feature bits that
communicate to eh sender the functionality needed in order to complete a
communicate to the sender the functionality needed to complete a
payment. As an example, if a new payment type is added in the future that isn't
backwards compatible with the original payment type, then the receiver can set
a _required_ feature bit in order to communicate that the payer needs to
underhand that feature in order to complete the payment.
backward compatible with the original payment type, then the receiver can set
a _required_ feature bit to communicate that the payer needs to
understand that feature to complete the payment.
=== Conclusion
As we have seen, invoices are a lot more than just a request for an amount. They contain critical information about _how_ to make the payment, such as "routing hints", the destination node's public key, ephemeral keys to increase security and much more.
As we have seen, invoices are a lot more than just a request for an amount. They contain critical information about _how_ to make the payment, such as routing hints, the destination node's public key, ephemeral keys to increase security, and much more.
In this chapter, we look at some of the most important issues related to the security and privacy of the Lightning Network. First, we'll consider privacy, what it means, how to evaluate it, and some things you can do to protect your own privacy while using LN. Then we'll explore some common attacks and mitigation techniques.
In this chapter, we look at some of the most important issues related to the security and privacy of the Lightning Network. First, we'll consider privacy, what it means, how to evaluate it, and some things you can do to protect your own privacy while using the Lightning Network. Then we'll explore some common attacks and mitigation techniques.
=== Why Is Privacy Important?
The key value proposition of cryptocurrency is censorship resistant money. Bitcoin offers participants the possibility of storing and transferring their wealth without interference by governments, banks, or corporations.The Lightning Network continues this mission.
The key value proposition of cryptocurrency is censorship resistant money. Bitcoin offers participants the possibility of storing and transferring their wealth without interference by governments, banks, or corporations.The Lightning Network continues this mission.
Unlike trivial scaling solutions like custodial Bitcoin banks, LN aims to scale Bitcoin without compromising on self-custody, which should lead to greater censorship resistance in the Bitcoin ecosystem. However, the LN operates under a different security model, which introduces novel security and privacy challenges.
Unlike trivial scaling solutions like custodial Bitcoin banks, the Lightning Network aims to scale Bitcoin without compromising on self-custody, which should lead to greater censorship resistance in the Bitcoin ecosystem. However, the Lightning Network operates under a different security model, which introduces novel security and privacy challenges.
=== Definitions of Privacy
The question: "Is Lightning private?" has no direct answer. Privacy is a complex topic; it is often difficult to precisely define what we mean by privacy, particularly if you are not a privacy researcher. Fortunately, privacy researchers use processes to analyze and evaluate the privacy characteristics of systems and we can use them too! Let's look at how a security researcher might seek to answer the question, "Is Lightning private?" in two general steps.
The question: "Is Lightning private?" has no direct answer. Privacy is a complex topic; it is often difficult to precisely define what we mean by privacy, particularly if you are not a privacy researcher. Fortunately, privacy researchers use processes to analyze and evaluate the privacy characteristics of systems, and we can use them too! Let's look at how a security researcher might seek to answer the question, "Is Lightning private?" in two general steps.
First, a privacy researcher would define a _security model_ that specifies what an adversary is capable of and aims to achieve.
Then, they would describe the relevant properties of the system and check whether it conforms to the requirements.
@ -21,13 +21,13 @@ A security model is based on a set of underlying _security assumptions_.
In cryptographic systems, these assumptions are often centered around the mathematical properties of the cryptographic primitives such as ciphers, signatures, and hash functions.
The security assumptions of the Lightning Network are that the ECDSA signatures, SHA-256 hash function, and other cryptographic functions used in the protocol behave within their security definitions.
For example, we assume that it is practically impossible to find a preimage (and second preimage) of a hash function.
This allows the LN to rely on the HTLC mechanism (which uses the preimage of a hash function) for the atomicity of multi-hop payments: nobody except the final recipient can reveal the payment secret and resolve the HTLC.
We also assume a degree of connectivity in the network, namely, that Lightning channels form a connected graph. Therefore, it is possible to find a path from any sender to any receiver. Finally, we assume network messages are propagated within certain timeouts.
This allows the LN to rely on the HTLC mechanism (which uses the preimage of a hash function) for the atomicity of multihop payments: nobody except the final recipient can reveal the payment secret and resolve the HTLC.
We also assume a degree of connectivity in the network, namely that Lightning channels form a connected graph. Therefore, it is possible to find a path from any sender to any receiver. Finally, we assume network messages are propagated within certain timeouts.
Now that we've identified some of our underlying assumptions, let's consider some possible adversaries.
Here are some possible models of adversaries in the Lightning Network.
An "honest-but-curious" forwarding node can observe payment amounts, the immediately preceding and following nodes and the graph of announced channels with their capacities.
An "honest-but-curious" forwarding node can observe payment amounts, the immediately preceding and following nodes, and the graph of announced channels with their capacities.
A very well-connected node can do the same but to a larger extent.
For example, consider the developers of a popular wallet who maintain a node that their users connect to by default.
This node would be responsible for routing a large share of payments to and from the users of that wallet.
@ -36,7 +36,7 @@ If two colluding nodes happen to be on the same payment path, they would underst
[NOTE]
====
Multi-Path Payments (see <<mpp>>), enable users to obfuscate their payment amounts given their non-uniform split sizes.
Multipath payments (see <<mpp>>) enable users to obfuscate their payment amounts given their nonuniform split sizes.
====
What may be the goals of a Lightning attacker?
@ -48,31 +48,31 @@ Availability:: The system is functioning most of the time.
The important properties of the Lightning Network are mostly centered around confidentiality and availability. Some of the most important properties to protect include:
* only the sender and the receiver know the payment amount
* no one can link senders and receivers
* an honest user cannot be blocked from sending and receiving payments
* Only the sender and the receiver know the payment amount.
* No one can link senders and receivers.
* An honest user cannot be blocked from sending and receiving payments.
For each privacy goal and security model, there is a certain probability that an attacker succeeds.
This probability depends on various factors, such as the size and the structure of the network.
This probability depends on various factors, such as the size and structure of the network.
Other things being equal, it is generally easier to successfully attack a small network rather than a large one.
Similarly, the more centralized the network is, the more capable an attacker can be if "central" nodes are under their control.
Of course, the term centralization must be defined precisely to build security models around it, and there are many possible definitions of how centralized a network is.
Finally, as a payment network, the LN depends on economic stimuli.
Finally, as a payment network, the Lightning Network depends on economic stimuli.
The size and structure of fees affect the routing algorithm and, therefore, can either aid the attacker by forwarding most payments through their nodes or prevent this from happening.
=== Anonymity Set
What does it mean to deanonymize someone?
In simple terms, deanonymization implies linking some action with a person's "real world" identity, such as their name or physical address.
In simple terms, deanonymization implies linking some action with a person's real-world identity, such as their name or physical address.
In privacy research, the notion of deanonymization is more nuanced.
First, we are not necessarily talking about names and addresses.
Discovering someone's IP address or telephone number may also be considered deanonymization.
A piece of information that allows linking a user's action to their previous actions is referred to as _identity_.
Second, deanonymization is not binary; a user is neither fully anonymous nor completely deanonymized.
Instead, privacy research looks at anonymity compared to the _anonymity set_.
Instead, privacy research looks at anonymity compared to the anonymity set.
The anonymity set is a central notion in privacy research.
The _anonymity set_ is a central notion in privacy research.
It refers to the set of identities such that, from an attacker's viewpoint, a given action could correspond to anyone in the set.
Consider a real-life example.
Imagine you meet a person on a city street.
@ -87,7 +87,7 @@ However, if you are a city official and have access to the database that matches
This example illustrates a few important points.
First, every bit of information may bring the adversary closer to their goal.
It may not be necessary to shrink the anonymity set to the size of one.
For instance, if the adversary plans a targeted DoS attack and can take down 100 servers, the anonymity set of 100 suffices.
For instance, if the adversary plans a targeted denial-of-service (DoS) attack and can take down 100 servers, the anonymity set of 100 suffices.
Second, the adversary can cross-correlate information from different sources.
Even if a privacy leak looks relatively benign, we never know what it can achieve in combination with other data sources.
Finally, especially in cryptographic settings, the attacker always has the "last resort" of brute force search.
@ -102,9 +102,9 @@ However, the real network leaks information that allows an attacker to narrow do
The smaller the anonymity set, the higher the chance of successful deanonymization.
=== Differences Between LN and Bitcoin in Terms of Privacy
=== Differences Between the Lightning Network and Bitcoin in Terms of Privacy
While it's true that transactions on the Bitcoin network do not associate realworld identities with Bitcoin addresses, all transactions are broadcast in cleartext and can be analyzed.
While it's true that transactions on the Bitcoin network do not associate real-world identities with Bitcoin addresses, all transactions are broadcast in cleartext and can be analyzed.
Multiple companies have been established to deanonymize users of Bitcoin and other cryptocurrencies.
At first glance, Lightning provides better privacy than Bitcoin because Lightning payments are not broadcast to the whole network.
@ -122,41 +122,41 @@ It is possible to use Lightning over Tor, but many nodes do not use this functio
A Lightning user, when sending a payment, has its neighbors in its anonymity set.
Specifically, a routing node only knows the immediately preceding and following nodes.
The routing node does not know whether its immediate neighbors in the payment route are the ultimate sender or receiver.
Therefore, the anonymity set of a node in Lightning roughly equals its neighbors, see <<anonymity_set>>.
Therefore, the anonymity set of a node in Lightning roughly equals its neighbors (see <<anonymity_set>>).
[[anonymity_set]]
.The anonymity set of Alice and Bob constitutes their neighbors.
image::images/mtln_1601.png["The anonymity set of Alice and Bob constitutes their neighbors"]
Similar logic applies to payment receivers.
Many users open only a handful of payment channels, therefore, limiting their anonymity sets.
Many users open only a handful of payment channels, therefore limiting their anonymity sets.
Moreover, in Lightning, the anonymity set is static or at least slowly changing.
In contrast, one can achieve significantly larger anonymity sets in on-chain CoinJoin transactions.
CoinJoin transactions with anonymity sets larger than 50 are quite frequent.
Typically, the anonymity sets in a CoinJoin transaction correspond to a dynamically changing set of users.
Finally, Lightning users can also be denied service, having their channels blocked or depleted by an attacker.
Forwarding payments requires capital - a scarce resource! - to be temporarily blocked in HTLCs along the route.
An attacker may send many payments but fail to finalize them, occupying honest user's capital for long periods.
Forwarding payments requires capital—a scarce resource!—to be temporarily blocked in HTLCs along the route.
An attacker may send many payments but fail to finalize them, occupying honest users' capital for long periods.
This attack vector is not present (or at least not as obvious) in Bitcoin.
In summary, while some aspects of the Lightning Network's architecture suggest that it is a step forward in terms of privacy compared to Bitcoin, other properties of the protocol may make attacks on privacy easier. Thorough research is needed to evaluate what privacy guarantees the LN provides and improve the state of affairs.
In summary, while some aspects of the Lightning Network's architecture suggest that it is a step forward in terms of privacy compared to Bitcoin, other properties of the protocol may make attacks on privacy easier. Thorough research is needed to evaluate what privacy guarantees the Lightning Network provides and improve the state of affairs.
The issues discussed in this part of the chapter summarize research available in mid2021. However, this area of research and development is growing quickly. We are happy to report that the authors are aware of multiple research teams currently working on Lightning privacy.
The issues discussed in this part of the chapter summarize research available in mid-2021. However, this area of research and development is growing quickly. We are happy to report that the authors are aware of multiple research teams currently working on Lightning privacy.
Now let's review some of the attacks on the Lightning Network privacy that have been described in academic literature.
Now let's review some of the attacks on LN privacy that have been described in academic literature.
=== Attacks On Lightning
Recent research describes various ways in which the security and privacy of the LN may be compromised.
Recent research describes various ways in which the security and privacy of the Lightning Network may be compromised.
==== Observing Payment Amounts
One of the goals for a privacy-preserving payment system is to hide the payment amount from uninvolved parties.
The Lightning Network is an improvement over layer-one in this regard.
The Lightning Network is an improvement over Layer 1 in this regard.
While Bitcoin transactions are broadcast in cleartext and can be observed by anyone, Lightning payments only travel through a few nodes along the payment path.
However, intermediary nodes do see the payment amount; although this payment amount might not correspond to the actual total payment amount, see <<mpp>>.
However, intermediary nodes do see the payment amount, although this payment amount might not correspond to the actual total payment amount (see <<mpp>>).
This is necessary to create a new HTLC at every hop.
The availability of payment amounts to intermediary nodes do not present an immediate threat.
However, an _honest-but-curious_ intermediary node may use it as a part of a larger attack.
@ -168,28 +168,28 @@ An attacker might be interested in learning the sender and/or the receiver of a
This breach of privacy could harm censorship resistance, as an intermediary node could censor payments to or from certain receivers or senders.
Ideally, linking senders to receivers should not be possible to anyone other than the sender and the receiver.
In the following sections, we will consider two types of adversaries: the off-path and the on-path adversary.
In the following sections, we will consider two types of adversaries: the off-path adversary and the on-path adversary.
An off-path adversary tries to assess the sender and the receiver of a payment without participating in the payment routing process.
An on-path adversary can leverage any information it might gain by routing the payment of interest.
First, consider the *off-path adversary*.
In the first step of this attack scenario, a potent off-path adversary deduces the individual balances in each payment channel via probing (described in a subsequent section) and forms a network snapshot at time _t1_. For simplicity's sake, let's make _t1_ equal 12:05.
It then probes the network again at sometime later at time _t2_, which we'll make 12:10. The attacker would then compare the snapshots at 12:10 and 12:05 and use the differences between the two snapshots to infer information about payments that took place by looking at paths that have changed.
First, consider the _off-path adversary_.
In the first step of this attack scenario, a potent off-path adversary deduces the individual balances in each payment channel via probing (described in a subsequent section) and forms a network snapshot at time __t~1~__. For simplicity's sake, let's make __t~1~__ equal 12:05.
It then probes the network again at sometime later at time __t~2~__, which we'll make 12:10. The attacker would then compare the snapshots at 12:10 and 12:05 and use the differences between the two snapshots to infer information about payments that took place by looking at paths that have changed.
In the simplest case, if only one payment occurred between 12:10 and 12:05, the adversary would observe a single path where the balances have changed by the same amounts.
Thus, the adversary learns almost everything about this payment: the sender, the recipient, and the amount.
If multiple payment paths overlap, the adversary needs to apply heuristics to identify such overlap and separate the payments.
Now, we turn our attention to an *on-path adversary*.
Now, we turn our attention to an _on-path adversary_.
Such an adversary might seem convoluted.
However, in June 2020, researchers noted that the single most central node https://arxiv.org/pdf/2006.12143.pdf[observed] close to 50% of all LN payments, while the four most central nodes https://arxiv.org/pdf/1909.06890.pdf[observe] an average of 72% payments.
However, in June 2020, researchers noted that the single most central node https://arxiv.org/pdf/2006.12143.pdf[observed close to 50% of all LN payments], while the four most central nodes https://arxiv.org/pdf/1909.06890.pdf[observed an average of 72% payments].
These findings emphasize the relevance of the on-path attacker model.
Even though intermediaries in a payment path only learn their successor and predecessor, there are several leakages that a malicious or honest-but-curious intermediary might use to infer the sender and the receiver.
The on-path adversary can observe the amount of any routed payment as well as time-lock deltas (see <<onion_routing>>).
Hence, the adversary can exclude any nodes from the sender's or the receiver's anonymity set with capacities lower than the routed amount.
Therefore, we observe a tradeoff between privacy and payment amounts.
Therefore, we observe a trade-off between privacy and payment amounts.
Typically, the larger the payment amount is, the smaller the anonymity sets are.
We note that this leakage could be minimized with multi-part payments or with large capacity payment channels.
We note that this leakage could be minimized with multipart payments or with large capacity payment channels.
Similarly, payment channels with small time-lock deltas could be excluded from a payment path.
More precisely, a payment channel cannot pertain to a payment if the remaining time the payment might be locked for is larger than what the forwarding node would be willing to accept.
This leakage could be evicted by adhering to the so-called shadow routes.
@ -214,7 +214,7 @@ In information security, probing refers to the technique of sending requests to
Lightning channels are prone to probing.
Recall that a standard Lightning payment starts with the receiver creating a random payment secret and sending its hash to the sender.
Note for the intermediary nodes, all hashes look random.
Note that for the intermediary nodes, all hashes look random.
There is no way to tell whether a hash corresponds to a real secret or was generated randomly.
The probing attack proceeds as follows.
@ -229,7 +229,7 @@ The question is: how exactly will it fail?
There are two scenarios.
If Alice has more than 500 thousand satoshis on her side of the channel to Bob, she forwards the payment.
Bob decrypts the payment onion and realizes that the payment is intended for him.
He looks up his local store of payment secrets, searches for the preimage that corresponds to the payment hash but does not find one.
He looks up his local store of payment secrets and searches for the preimage that corresponds to the payment hash, but does not find one.
Following the protocol, Bob returns the "unknown payment hash" error to Alice, who relays it back to Mallory.
As a result, Mallory knows that the payment _could have succeeded_ if the payment hash was real.
Therefore, Mallory can update her estimation of Alice's balance from "between zero and 1 million" to "between 500 thousand and one million."
@ -241,14 +241,14 @@ Note that in any case, Mallory's estimation becomes twice as precise after just
She can continue probing, choosing the next probing amount such that it divides the current estimation interval in half.
This well-known search technique is called _binary search_.
With binary search, the number of probes is _logarithmic_ in the desired precision.
For example, to obtain Alice's balance in a channel of 1 million satoshis up to a single satoshi, Mallory would only have to perform latexmath:[$\log_2(1000000) \approx 20$] probings.
For example, to obtain Alice's balance in a channel of 1 million satoshis up to a single satoshi, Mallory would only have to perform log~2~ (1,000,000) ≈ 20 probings.
If one probing takes 3 seconds, one channel can be precisely probed in only about a minute!
Channel probing can be made even more efficient.
In its simplest variant, Mallory directly connects to the channel she wants to probe.
Is it possible to probe a channel without opening a channel to one of its endpoints?
Imagine Mallory now wants to probe a channel between Bob and Charlie but doesn't want to open another channel, which requires paying on-chain fees and waiting for confirmations of the funding transactions.
Instead, Mallory re-uses her existing channel to Alice and sends a probe along the route Mallory - Alice - Bob - Charlie.
Instead, Mallory reuses her existing channel to Alice and sends a probe along the route Mallory -> Alice -> Bob -> Charlie.
Mallory can interpret the "unknown payment hash" error in the same way as before: the probe has reached the destination; therefore, all channels along the route have sufficient balances to forward it.
But what if Mallory receives the "insufficient balance" error?
Does it mean that the balance is insufficient between Alice and Bob or between Bob and Charlie?
@ -271,18 +271,18 @@ One popular Lightning wallet even performs probing internally to check whether a
There are other potential countermeasures against channel probing.
First, it is hard for an attacker to target unannounced channels.
Second, nodes that implement just-in-time or JIT routing may be less prone to the attack.
Finally, as multi-part payments make the problem of insufficient capacity less severe, the protocol developers may consider hiding some of the error details without harming efficiency.
Second, nodes that implement just-in-time (JIT) routing may be less prone to the attack.
Finally, as multipart payments make the problem of insufficient capacity less severe, the protocol developers may consider hiding some of the error details without harming efficiency.
[[denial_of_service]]
==== Denial of Service
When resources are made publicly available, there is a risk that attackers may attempt to make that resource unavailable by executing a denial-of-service attack.
When resources are made publicly available, there is a risk that attackers may attempt to make that resource unavailable by executing a denial-of-service (DoS) attack.
Generally, this is achieved by the attacker bombarding a resource with requests, which are indistinguishable from legitimate queries.
The attacks seldom result in the target suffering financial loss aside from the opportunity cost of their service being down and are merely intended to grief the target.
The attacks seldom result in the target suffering financial loss aside from the opportunity cost of their service being down and are merely intended to aggrieve the target.
Typical mitigations for denial-of-service attacks require authentication for requests to separate legitimate users from malicious ones or to incur a trivial cost to regular users but will act as a sufficient deterrent to an attacker launching requests at scale.
Anti-denial-of-service measures can be seen everywhere on the internet - websites apply rate limits to ensure that no one user can consume all of their server's attention, film review sites require login authentication to keep angry r/prequelmemes (Reddit group) members at bay, and data services sell API keys to limit the number of queries.
Typical mitigations for DoS attacks require authentication for requests to separate legitimate users from malicious ones. These mitigations incur a trivial cost to regular users but will act as a sufficient deterrent to an attacker launching requests at scale.
Anti-denial-of-service measures can be seen everywhere on the internet—websites apply rate limits to ensure that no one user can consume all of their server's attention, film review sites require login authentication to keep angry r/prequelmemes (Reddit group) members at bay, and data services sell API keys to limit the number of queries.
===== Dos in bitcoin
@ -290,38 +290,38 @@ In Bitcoin, the bandwidth that nodes use to relay transactions and the space tha
Any node on the network can consume bandwidth and mempool space by sending a valid transaction.
If this transaction is mined in a valid block, they will pay transaction fees, which adds a cost to using these shared network resources.
In the past, the Bitcoin network faced an attempted denial-of-service attack where attackers spammed the network with low-fee transactions.
In the past, the Bitcoin network faced an attempted DoS attack where attackers spammed the network with low-fee transactions.
Many of these transactions were not selected by miners due to their low transaction fees, so the attackers could consume network resources without paying the fees.
To address this issue, a minimum transaction relay fee that set a threshold fee that nodes require to propagate transactions was set.
This measure largely ensured that the transactions that consume network resources will eventually pay their chain fees.
The minimum relay fee is acceptable to regular users but would hurt attackers financially if they tried to spam the network.
While some transactions may not make it into valid blocks during high fee environments, these measures have largely been effective at disincentivizing this type of spam.
While some transactions may not make it into valid blocks within high-fee environments, these measures have largely been effective at disincentivizing this type of spam.
===== Dos in lightning
===== DoS in Lightning
Similarly to Bitcoin, the Lightning Network charges fees for the use of its public resources, but in this case, the resources are public channels, and the fees come in the form of routing fees.
The ability to route payments through nodes in exchange for fees provides the network with a large scalability benefit - nodes that are not directly connected can still transact - but it comes at the cost of exposing a public resource that must be protected against DoS attacks.
The ability to route payments through nodes in exchange for fees provides the network with a large scalability benefit—nodes that are not directly connected can still transact—but it comes at the cost of exposing a public resource that must be protected against DoS attacks.
When a Lightning node forwards a payment on your behalf, it uses data and payment bandwidth to update its commitment transaction, and the amount of the payment is reserved in their channel balance until it is settled or failed.
In successful payments, this is acceptable because the node is eventually paid out its fees.
Failed payments do not incur fees in the current protocol.
This allows nodes to costlessly route failed payments through any channels.
This is great for legitimate users, who wouldn't like to pay for failed attempts, but also allows attackers to costlessly consume nodes' resources - much like the low-fee transactions on Bitcoin that never end up paying miner fees.
This is great for legitimate users, who wouldn't like to pay for failed attempts, but also allows attackers to costlessly consume nodes' resources—much like the low-fee transactions on Bitcoin that never end up paying miner fees.
At the time of writing, a discussion is https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-June/002734.html[ongoing] on the lightning-dev mailing list as to how best address this issue.
===== Known dos attacks
There are two known DoS attacks on public Lightning channels which render a target channel, or a set of target channels, unusable.
There are two known DoS attacks on public LN channels which render a target channel, or a set of target channels, unusable.
Both attacks involve routing payments through a public channel, then holding them until their timeout, thus maximizing the attack's duration.
The requirement to fail payments to not pay fees is fairly simple to meet because malicious nodes can simply reroute payments to themselves.
In the absence of fees for failed payments, the only cost to the attacker is the on-chain cost of opening a channel to dispatch these payments through, which can be trivial in lowfee environments.
In the absence of fees for failed payments, the only cost to the attacker is the on-chain cost of opening a channel to dispatch these payments through, which can be trivial in low-fee environments.
==== Commitment Jamming
Lightning nodes update their shared state using asymmetric commitment transactions, on which HTLCs are added and removed to facilitate payments.
Each party is limited to a total of https://github.com/lightningnetwork/lightning-rfc/blob/c053ce7afb4cbf88615877a0d5fc7b8dbe2b9ba0/02-peer-protocol.md#the-open_channel-message[483] HTLCs in the commitment transaction at a time.
A channel jamming attack allows an attacker to render a channel unusable by routing 483 payments through the target channel and holding them until they timeout.
A channel jamming attack allows an attacker to render a channel unusable by routing 483 payments through the target channel and holding them until they timeout.
It should be noted that this limit was chosen in the specification to ensure that all the HTLCs can be swept in a https://github.com/lightningnetwork/lightning-rfc/blob/master/05-onchain.md#penalty-transaction-weight-calculation[single justice transaction].
While this limit _may_ be increased, transactions are still limited by the block size, so the number of slots available is likely to remain limited.
@ -329,7 +329,7 @@ While this limit _may_ be increased, transactions are still limited by the block
==== Channel Liquidity Lockup
A channel liquidity lockup attack is comparable to a channel jamming attack in that it routes payments through a channel and holds them so that the channel is unusable.
Rather than locking up slots on the channel commitment, this attack routes large HTLCs through a target channel that consumes all the channel's available bandwidth.
Rather than locking up slots on the channel commitment, this attack routes large HTLCs through a target channel, consuming all the channel's available bandwidth.
This attack's capital commitment is higher than the commitment jamming attack because the attacking node needs more funds to route failed payments through the target.
=== Cross-Layer Deanonymization
@ -342,17 +342,17 @@ Ideally, the upper layer should perceive a lower layer as a black box.
In reality, though, implementations are not ideal, and the details _leak_ into the upper layer.
This is the problem of leaky abstractions.
In the context of Lightning, the LN protocol relies on the Bitcoin protocol and the Lightning P2P network.
Up to this point, we only considered the privacy guarantees offered by the LN in isolation.
In the context of Lightning, the LN protocol relies on the Bitcoin protocol and the LN P2P network.
Up to this point, we only considered the privacy guarantees offered by the Lightning Network in isolation.
However, creating and closing payment channels are inherently performed on the Bitcoin blockchain.
Consequently, for a complete analysis of LN's privacy provisions, one needs to consider every layer of the technological stack users might interact with.
Consequently, for a complete analysis of the LIghtning Network's privacy provisions, one needs to consider every layer of the technological stack users might interact with.
Specifically, a deanonymizing adversary can and will use off-chain and on-chain data to cluster or link LN nodes to corresponding Bitcoin addresses.
Some of the goals of attackers attemptin to deanonymize LN users, in a cross-layer context:
Attackers attempting to deanonymize LN users may have various goals, in a cross-layer context:
* Cluster Bitcoin addresses owned by the same user (layer-1). We call these Bitcoin entities.
* Cluster LN nodes owned by the same user (layer-2).
* Unambiguously link sets of Lightning nodes to the sets of Bitcoin entities that control them.
* Cluster Bitcoin addresses owned by the same user (Layer 1). We call these Bitcoin entities.
* Cluster LN nodes owned by the same user (Layer 2).
* Unambiguously link sets of LN nodes to the sets of Bitcoin entities that control them.
There are several heuristics and usage patterns that allow an adversary to cluster Bitcoin addresses and LN nodes owned by the same LN users.
Moreover, these clusters can be linked across layers using other powerful cross-layer linking heuristics.
@ -368,10 +368,10 @@ Opening a channel causes a monetary flow from a _source entity_ to a _funding en
In early 2021, https://arxiv.org/pdf/2007.00764.pdf[Romiti et al.] identified four heuristics that allow the clustering of these entities.
Two of them capture certain leaky funding behavior and two describe leaky settlement behaviors.
* Star Heuristic (Funding): if a component contains one source entity that forwards funds to one or more funding entities, these funding entities are likely controlled by the same user.
* Snake Heuristic (Funding): if a component contains one source entity that forwards funds to one or more entities, which themselves are used as source and funding entities, then all these entities are likely controlled by the same user.
* Collector Heuristic (Settlement): if a component contains one destination entity that receives funds from one or more settlement entities, these settlement entities are likely controlled by the same user.
* Proxy Heuristic (Settlement): if a component contains one destination entity that receives funds from one or more entities, which themselves are used as settlement and destination entities, then these entities are likely controlled by the same user.
Star heuristic (funding):: If a component contains one source entity that forwards funds to one or more funding entities, these funding entities are likely controlled by the same user.
Snake heuristic (funding):: If a component contains one source entity that forwards funds to one or more entities, which themselves are used as source and funding entities, then all these entities are likely controlled by the same user.
Collector heuristic (settlement):: If a component contains one destination entity that receives funds from one or more settlement entities, these settlement entities are likely controlled by the same user.
Proxy heuristic (settlement):: If a component contains one destination entity that receives funds from one or more entities, which themselves are used as settlement and destination entities, then these entities are likely controlled by the same user.
It is worthwhile pointing out that these heuristics might produce false positives.
For instance, if transactions of several unrelated users are combined in a CoinJoin transaction, then the Star or the Proxy heuristic can produce false positives.
However, users tend to use similar aliases for their own different nodes.
For example, _LNBig.com Billing_ is likely owned by the same user as the node with alias _LNBig.com_.
Given this observation, one can cluster LN nodes applying their node aliases.
Given this observation, one can cluster LN nodes by applying their node aliases.
Specifically, one clusters LN nodes into a single address if their aliases are similar with respect to some string similarity metric.
Another method to cluster LN nodes is applying their IP or Tor addresses.
@ -396,21 +396,21 @@ If the same IP or Tor addresses correspond to different LN nodes, these nodes ar
===== Countermeasures
For more privacy, aliases should be sufficiently different from one another.
While the public announcement of IP addresses may be unavoidable for those nodes that wish to have incoming channels in the LN, linkability across nodes of the same user can be mitigated if the clients for each node are hosted with different service providers and thus IP addresses.
While the public announcement of IP addresses may be unavoidable for those nodes that wish to have incoming channels in the Lightning Network, linkability across nodes of the same user can be mitigated if the clients for each node are hosted with different service providers and thus IP addresses.
==== Cross-Layer Linking: Lightning Nodes and Bitcoin Entities
Associating LN nodes to Bitcoin entities is a serious breach of privacy that is exacerbated by the fact that most LN nodes publicly expose their IP addresses.
Typically, an IP address can be considered as a unique identifier of a user.
Two widely observed behavior patterns reveal links between LN nodes and Bitcoin entities.
* Coin reuse: whenever users close payment channels, they get back their corresponding coins. However, many users reuse those coins in opening a new channel.
Coin reuse:: Whenever users close payment channels, they get back their corresponding coins. However, many users reuse those coins in opening a new channel.
Those coins can effectively be linked to a common LN node.
* Entity reuse: typically, users fund their payment channels from Bitcoin addresses corresponding to the same Bitcoin entity.
Entity reuse:: Typically, users fund their payment channels from Bitcoin addresses corresponding to the same Bitcoin entity.
These cross-layer linking algorithms could be foiled if users possess multiple unclustered addresses or use multiple wallets to interact with the LN.
These cross-layer linking algorithms could be foiled if users possess multiple unclustered addresses or use multiple wallets to interact with the Lightning Network.
The possible deanonymization of Bitcoin entities illustrates how important it is to consider the privacy of both layers simultaneously instead of one of them at a time.
The possible deanonymization of Bitcoin entities illustrates how important it is to consider the privacy of both layers simultaneously instead of one at a time.
//TODO from author: maybe here we should/could include the corresponding figures from the Romiti et al. paper. it would greatly improve and help the understanding of the section
@ -419,30 +419,30 @@ The possible deanonymization of Bitcoin entities illustrates how important it is
The Lightning Network, as the name suggests, is a peer-to-peer network of payment channels.
Therefore, many of its properties (privacy, robustness, connectivity, routing efficiency) are influenced and characterized by its network nature.
In this section, we discuss and analyze the LN from the point of view of network science.
In this section, we discuss and analyze the Lightning Network from the point of view of network science.
We are particularly interested in understanding the LN channel graph, its robustness, connectivity, and other important characteristics.
==== How Does the Lightning Graph Look in Reality?
One could have expected that the LN is a random graph, where edges are randomly formed between nodes.
If this was the case, then the LN's degree distribution would follow a Gaussian normal distribution.
One could have expected that the Lightning Network is a random graph, where edges are randomly formed between nodes.
If this was the case, then the Lightning Network's degree distribution would follow a Gaussian normal distribution.
In particular, most of the nodes would have approximately the same degree, and we would not expect nodes with extraordinarily large degrees.
This is because the normal distribution exponentially decreases for values outside of the interval around the average value of the distribution.
The depiction of a random graph looks like a mesh network topology.
It looks decentralized and non-hierarchical: every node seems to have equal importance.
It looks decentralized and nonhierarchical: every node seems to have equal importance.
Additionally, random graphs have a large diameter.
In particular, routing in such graphs is challenging as the shortest path between any two nodes is moderately long.
In particular, routing in such graphs is challenging because the shortest path between any two nodes is moderately long.
However, in stark contrast, the LN graph is completely different.
===== Lightning graph today
Lightning is a financial network.
Thus, the growth and formation of the network are also influenced by economic incentives.
Whenever a node joins the LN, it may want to maximize its connectivity to other nodes in order to increase its routing efficiency. This phenomenon is called preferential attachment.
Whenever a node joins the Lightning Network, it may want to maximize its connectivity to other nodes in order to increase its routing efficiency. This phenomenon is called preferential attachment.
These economic incentives result in a fundamentally different network than a random graph.
Based on snapshots of publicly announced channels, the degree distribution of the LN follows a power-law function.
Based on snapshots of publicly announced channels, the degree distribution of the Lightning Network follows a power-law function.
In such a graph, the vast majority of nodes have very few connections to other nodes, while only a handful of nodes have numerous connections.
At a high-level, this graph topology resembles a star: the network has a well-connected core and a loosely connected periphery.
At a highlevel, this graph topology resembles a star: the network has a well-connected core and a loosely connected periphery.
Networks with power-law degree distribution are also called scale-free networks.
This topology is advantageous for routing payments efficiently but prone to certain topology-based attacks.
@ -450,14 +450,14 @@ This topology is advantageous for routing payments efficiently but prone to cert
An adversary might want to disrupt the Lightning Network and may decide its goal is to dismantle the whole network into many smaller components, making payment routing practically impossible in the whole network.
A less ambitious, but still malicious and severe goal might be to only take down certain network nodes.
Such a disruption might occur on the node-level or on the edge-level.
Such a disruption might occur on the node level or on the edge level.
Let's suppose an adversary can take down any node in the LN.
For instance, it can DDoS them or make them non-operational by any means.
It turns out that if the adversary chooses nodes randomly, then scale-free networks like the LN are robust against node-removal attacks.
Let's suppose an adversary can take down any node in the Lightning Network.
For instance, it can attack them with a distributed denial of service (DDoS) attack or make them nonoperational by any means.
It turns out that if the adversary chooses nodes randomly, then scale-free networks like the Lightning Network are robust against node-removal attacks.
This is because a random node lies on the periphery with a small number of connections, therefore playing a negligible role in the network's connectivity.
However, if the adversary is more prudent, it can target the most well-connected nodes.
Not surprisingly, the LN and other scale-free networks are _not_ robust against targeted node-removal attacks.
Not surprisingly, the Lightning Network and other scale-free networks are _not_ robust against targeted node-removal attacks.
On the other hand, the adversary could be more stealthy.
Several topology-based attacks target a single node or a single payment channel.
@ -466,48 +466,48 @@ More generally, an adversary can deplete all the outgoing capacity of a node to
This could be easily obtained by routing payments through the victim node with amounts equalling to the outgoing capacity of each payment channel.
After completing this so-called node isolation attack, the victim cannot send or route payments anymore unless it receives a payment or rebalances its channels.
To conclude, even by design, it is possible to remove edges and nodes from the routable LN.
To conclude, even by design, it is possible to remove edges and nodes from the routable Lightning Network.
However, depending on the utilized attack vector, the adversary may have to provide more or fewer resources to carry out the attack.
===== Temporality of the ln
===== Temporality of the Lightning Network
The LN is a dynamically changing, permissionless network.
The Lightning Network is a dynamically changing, permissionless network.
Nodes can freely join or leave the network, they can open and create payment channels anytime they want.
Therefore, a single static snapshot of the LN graph is misleading. We need to consider the temporality and ever-changing nature of the network. For now, the LN graph is growing in terms of the number of nodes and payment channels.
Its effective diameter is also shrinking; that is, nodes become closer to each other, as we can see in <<temporal_ln>>:
Its effective diameter is also shrinking; that is, nodes become closer to each other, as we can see in <<temporal_ln>>.
[[temporal_ln]]
.The steady growth of the LN in nodes, channels, and locked capacity (as of September 2021)
image::images/mtln_1602.png["The steady growth of the LN in terms of nodes, channels and locked capacity (as of September 2021)"]
.The steady growth of the Lightning Network in nodes, channels, and locked capacity (as of September 2021)
image::images/mtln_1602.png["The steady growth of the Lightning Network in terms of nodes, channels, and locked capacity (as of September 2021)"]
In social networks, triangle closing behavior is common.
Specifically, in a graph where nodes represent people and friendships are represented as edges, it is somewhat expected that triangles will emerge in the graph.
A triangle, in this case, represents pairwise friendships between three people.
For instance, if Alice knows Bob and Bob knows Charlie, then it is likely that at some point, Bob will introduce Alice to Charlie.
However, this behavior would be strange in the LN.
Nodes are simply not incentivized to close triangles as they could have just routed payments instead of opening a new payment channel.
Surprisingly, triangle closing is a common practice in the LN.
The number of triangles was steadily growing before the implementation of multi-part payments.
For instance, if Alice knows Bob and Bob knows Charlie, then it is likely that at some point Bob will introduce Alice to Charlie.
However, this behavior would be strange in the Lightning Network.
Nodes are simply not incentivized to close triangles because they could have just routed payments instead of opening a new payment channel.
Surprisingly, triangle closing is a common practice in the Lightning Network.
The number of triangles was steadily growing before the implementation of multipart payments.
This is counterintuitive and surprising given that nodes could have just routed payments through the two sides of the triangle instead of opening the third channel.
This may mean that routing inefficiencies incentivized users to close triangles and not fall back on routing.
Hopefully, multi-part payments will help increase the effectiveness of payment routing.
Hopefully, multipart payments will help increase the effectiveness of payment routing.
=== Centralization in the LN
=== Centralization in the Lightning Network
A common metric to assess the centrality of a node in a graph is its _betweenness centrality_. Central point dominance is a metric derived from betweenness centrality, used to assess the centrality of a network.
For a precise definition of central point dominance, the reader is referred to https://doi.org/10.2307/3033543[Freeman's work].
The larger the central point dominance of a network is, the more centralized the network is.
We can observe that the LN has a greater central point dominance, ie. it is more centralized, than a random graph (Erdős-Rényi graph) or a scale-free graph (Barabási-Albert graph) of equal size.
We can observe that the Lightning Network has a greater central point dominance (i.e., it is more centralized) than a random graph (Erdős–Rényi graph) or a scale-free graph (Barabási-Albert graph) of equal size.
In general, our understanding of the dynamic nature of the LN channel graph is rather limited.
It is fruitful to analyze how protocol changes like multi-part payments can affect the dynamics of the LN.
It is fruitful to analyze how protocol changes like multipart payments can affect the dynamics of the Lightning Network.
It would be beneficial to explore the temporal nature of the LN graph in more depth.
=== Economic Incentives and Graph Structure
The Lightning graph forms spontaneously and nodes connect to each other based on mutual interest.
The LN graph forms spontaneously and nodes connect to each other based on mutual interest.
As a result, incentives drive graph development.
Let's look at some of the relevant incentives:
@ -519,8 +519,8 @@ Let's look at some of the relevant incentives:
- While we should not base our security assumptions on altruism, to a certain extent, altruistic behavior drives Bitcoin (accepting incoming connections, serving blocks).
- What role does it play in Lightning?
In the early stages of the Lightning Network, many node operators have claimed that the earned routing fees do not compensate for the opportunity costs stemming from liquidity lock-up. This would indicate that operating a node may be driven mostly by altruistic incentives "for the good of the network".
This might change in the future if LN has significantly larger traffic or if a market for routing fees emerge.
In the early stages of the Lightning Network, many node operators have claimed that the earned routing fees do not compensate for the opportunity costs stemming from liquidity lock-up. This would indicate that operating a node may be driven mostly by altruistic incentives "for the good of the network."
This might change in the future if the Lightning Network has significantly larger traffic or if a market for routing fees emerges.
On the other hand, if a node wishes to optimize its routing fees, it would minimize the average shortest path lengths to every other node.
Put differently, a profit-seeker node will try to locate itself in the _center_ of the channel graph or close to it.
@ -533,11 +533,11 @@ In the meantime, there are some measures that you can take to guard your node ag
=== Unannounced Channels
If you intend to use the Lightning Network to send and receive funds between nodes and wallets you control, and have no interest in routing other users' payments, there is little need to announce your channels to the rest of the network.
You could open a channel between, say, your desktop PC running a full node and your mobile phone running a Lightning Wallet and simply forgo the channel announcement discussed in <<ch03_How_Lightning_Works>>.
These are sometimes called "private" channels, however it is more correct to refer to them as "unannounced" channels as they are not strictly private.
You could open a channel between, say, your desktop PC running a full node and your mobile phone running a Lightning wallet and simply forgo the channel announcement discussed in <<ch03_How_Lightning_Works>>.
These are sometimes called "private" channels, however it is more correct to refer to them as "unannounced" channels because they are not strictly private.
Unannounced channels will not be known to the rest of the network and won't normally be used to route other users' payments.
They can still be used to route payments if other nodes are made aware of them, for example an invoice could contain routing hints which suggests a path with an unannounced channel.
They can still be used to route payments if other nodes are made aware of them; for example, an invoice could contain routing hints which suggests a path with an unannounced channel.
However, assuming that you've only opened an unannounced channel with yourself, you do gain some measure of privacy.
Since you are not exposing your channel to the network, you lower the risk of a denial-of-service attack on your node.
You can also more easily manage the capacity of this channel, since it will only be used to receive or send directly to your node.
@ -546,29 +546,29 @@ There are also advantages to opening an unannounced channel with a known party t
For example, if Alice and Bob frequently play poker for bitcoin, they could open a channel to send their winnings back and forth.
Under normal conditions, this channel will not be used to route payments from other users or collect fees.
And since the channel will not be known to the rest of the network, any payments between Alice and Bob cannot be inferred by tracking changes in the channel's routing capacity.
This confers some privacy to Alice and Bob, however if one of them decides to make other users aware of the channel, such as by including it in the routing hints of an invoice, then this privacy is lost.
This confers some privacy to Alice and Bob; however, if one of them decides to make other users aware of the channel, such as by including it in the routing hints of an invoice, then this privacy is lost.
It should also be noted that to open an unannounced channel, a public transaction must be made on the Bitcoin blockchain.
Hence it is possible to infer the existence and size of the channel if a malicious party is monitoring the blockchain for channel opening transactions and attempting to match them to channels on the network.
Furthermore, when the channel is closed, the final balance of the channel will be made public once it's committed to the Bitcoin blockchain.
However, since the opening and commitment transactions are pseudonymous it will not be a simple matter to connect it back to Alice or Bob.
However, since the opening and commitment transactions are pseudonymous, it will not be a simple matter to connect it back to Alice or Bob.
In addition, the Taproot update of 2021 makes it difficult to distinguish between channel opening and closing transactions and other specific kinds of Bitcoin transactions.
Hence, while unannouned channels are not completely private, they do provide some privacy benefits when used carefully.
[[routing_considerations]]
=== Routing Considerations
As covered in the <<denial_of_service>> section, nodes that open public channels expose themselves to the risk of a series of attacks on their channels.
As covered in <<denial_of_service>>, nodes that open public channels expose themselves to the risk of a series of attacks on their channels.
While mitigations are being developed on the protocol level, there are many steps that a node can take to protect against denial of service attacks on their public channels.
* Minimum HTLC size: on channel open, your node can set the minimum HTLC size that it will accept.
Minimum HTLC size:: On channel open, your node can set the minimum HTLC size that it will accept.
Setting a higher value ensures that each of your available channel slots cannot be occupied by a very small payment.
* Rate Limiting: many node implementations allow nodes to dynamically accept or reject HTLCs that are forwarded through your node.
Rate limiting:: Many node implementations allow nodes to dynamically accept or reject HTLCs that are forwarded through your node.
Some useful guidelines for a custom rate limiter are as follows:
+
** Limit the number of commitment slots a single peer may consume
** Monitor failure rates from a single peer, and rate limit if their failures spike suddenly
* Shadow channels: nodes that wish to open large channels to a single target can instead open a single public channel to the target and support it with further private channels called [shadow channels](joost on TFTC).
These channels can still be used for routing but are not announced to potential attackers.
Shadow channels:: Nodes that wish to open large channels to a single target can instead open a single public channel to the target and support it with further private channels called pass:[<a href='https://anchor.fm/tales-from-the-crypt/episodes/197-Joost-Jager-ekghn6'>shadow channels</a>]. These channels can still be used for routing but are not announced to potential attackers.
==== Accepting Channels
At present, Lightning nodes struggle with bootstrapping inbound liquidity. While there are some paid
@ -576,7 +576,7 @@ solutions to acquiring inbound liquidity, like swap services, channel markets, a
Stepping back to the context of Bitcoin, this can be compared to the way that Bitcoin Core treats its incoming and outgoing connections differently out of concern that the node may be eclipsed.
If a node opens an incoming connection to your Bitcoin node, you have no way of knowing whether the initiator randomly selected you or is specifically targeting your node with malicious intent.
Your outgoing connections do not need to be treated with such suspicion because the node was either selected randomly from a pool of many potential peers, or you intentionally connected to the peer manually.
Your outgoing connections do not need to be treated with such suspicion because either the node was selected randomly from a pool of many potential peers or you intentionally connected to the peer manually.
The same can be said in Lightning.
When you open a channel, it is done with intention, but when a remote party opens a channel to your node, you have no way of knowing whether this channel will be used to attack your node or not.
@ -590,30 +590,32 @@ Our suggestion is not to set an exclusive list of "mega-hubs" from which you wil
Some potential strategies are:
* No risk: do not accept any incoming channels.
* Low risk: accept channels from a known set of nodes that you have previously had successful channels open with
* Medium risk: only accept channels from nodes that have been present in the graph for a longer period, and have some long-lived channels.
* Higher risk: accept any incoming channels, and implement the mitigations described in <<routing_considerations>>.
No risk:: Do not accept any incoming channels.
Low risk:: Accept channels from a known set of nodes that you have previously had successful channels open with
Medium risk:: Only accept channels from nodes that have been present in the graph for a longer period and have some long-lived channels.
Higher risk:: Accept any incoming channels, and implement the mitigations described in <<routing_considerations>>.
=== Conclusion
In summary, privacy and security are nuanced, complex topics and while many researchers and developers are looking for network-wide improvements, it's important for everyone participating in the network to understand what they can do to protect their own privacy and increase security on an individual node level.
In summary, privacy and security are nuanced, complex topics, and while many researchers and developers are looking for network-wide improvements, it's important for everyone participating in the network to understand what they can do to protect their own privacy and increase security on an individual node level.
=== References and Further Reading
In this chapter, we used many references from ongoing research on Lightning security. You may find these useful articles and papers listed by topic below:
In this chapter, we used many references from ongoing research on Lightning security. You may find these useful articles and papers listed by topic in the following lists.
===== Privacy and probing attacks
* Jordi Herrera-Joancomartí et al. https://eprint.iacr.org/2019/328[On the Difficulty of Hiding the Balance of Lightning Network Channels]
* Nisslmueller et al. https://arxiv.org/abs/2003.00003[Toward Active and Passive Confidentiality Attacks On Cryptocurrency Off-Chain Networks]
* Tikhomirov et al. https://arxiv.org/abs/2004.00333[Probing Channel Balances in the Lightning Network]
* Kappos et al. https://arxiv.org/abs/2003.12470[An Empirical Analysis of Privacy in the Lightning Network]
* Jordi Herrera-Joancomartí et al. https://eprint.iacr.org/2019/328["On the Difficulty of Hiding the Balance of Lightning Network Channels"], _Asia CCS '19: Proceedings of the 2019 ACM Asia Conference on Computer and Communications Security_ (July 2019): 602–612.
* Utz Nisslmueller et al. "Toward Active and Passive Confidentiality Attacks on Cryptocurrency Off-Chain Networks." arXiv preprint, https://arxiv.org/abs/2003.00003[] (2020).
* Sergei Tikhomirov et al. "Probing Channel Balances in the Lightning Network." arXiv preprint, https://arxiv.org/abs/2004.00333[] (2020).
* George Kappos et al. "An Empirical Analysis of Privacy in the Lightning Network." arXiv preprint, https://arxiv.org/abs/2003.12470[] (2021).
* https://github.com/LN-Zap/zap-desktop/blob/v0.7.2-beta/services/grpc/router.methods.js[Zap source code with the probing function]
===== Congestion attacks
* Mizrahi, A., Zohar, A. https://arxiv.org/abs/2002.06564[Congestion Attacks in Payment Channel Networks]
* Ayelet Mizrahi and Aviv Zohar. "Congestion Attacks in Payment Channel Networks." arXiv preprint, https://arxiv.org/abs/2002.06564[] (2020).
===== Routing considerations
* Jager, J. https://anchor.fm/tales-from-the-crypt/episodes/197-Joost-Jager-ekghn6[Tales from the Crypt Episode 197]
* Marty Bent, interview with Joost Jager, _Tales from the Crypt_, podcast audio, October 2, 2020, https://anchor.fm/tales-from-the-crypt/episodes/197-Joost-Jager-ekghn6[].
@ -42,7 +42,7 @@ The Lightning P2P protocol is highly extensible and has undergone a lot of chang
===== TLV extensibility
The Type-Length-Value (see <<tlv>>) mechanism for extending the messaging protocol is extremely powerful and has already enabled the introduction of several new capabilities in Lightning while maintaining both forward and backward compatibility.
A prominent example that is currently being developed and makes use of this is path blinding and trampoline payments which allows a recipient to hide itself from the sender but also allows mobile clients to send payments without the necessity to store the full channel graph on their devices by using a third party to which they don't need to reveal the final recipient.
A prominent example, which is currently being developed and makes use of this, is path blinding and trampoline payments. This allows a recipient to hide itself from the sender, but also allows mobile clients to send payments without the necessity of storing the full channel graph on their devices by using a third party to which they don't need to reveal the final recipient.
==== Payment Channel Construction
@ -66,13 +66,13 @@ HODL invoices:: Payments where the final HTLC is not collected, committing the s
Onion routed message services:: The onion routing mechanism, and the underlying public key databse of nodes can be used to send data that is unrelated to payments, such as text messages or forum posts. The use of Lightning to enable paid messaging as a solution to spam posts and sybil attacks (spam) is another innovation that was implemented independently of the core protocol.
Offers:: Currently Proposed as BOLT #12 but already implemented by some nodes this is a communication protocol to request (recurring) invoices from remote nodes via Onion messages.
Offers:: Currently proposed as BOLT #12 but already implemented by some nodes, this is a communication protocol to request (recurring) invoices from remote nodes via Onion messages.
[[lapps]]
=== Lightning Applications (LApps)
While still in their infancy, we are already seeing the emergence of interesting Lightning Applications. Broadly defined as an application that uses the Lightning Protocol or a Lightning client as a component, LApps are the application layer of Lightning.
A prominent example are LNURL that provides a similar functionality as BOLT #12 Offers but just over http and Lightning address that works on top of offers to provide users with an email style address to which others can send funds while the software in the background requests an invoice against the LNURL endpoint of the node.
A prominent example is LNURL, which provides a similar functionality as BOLT #12 Offers, but just over http and Lightning addresses. It works on top of Offers to provide users with an email-style address to which others can send funds while the software in the background requests an invoice against the LNURL endpoint of the node.
Further LApps are being built for simple games, messaging applications, micro-services, payable-APIs, paid dispensers (eg. fuel pumps), derivative trading systems, and much more.
This appendix contains attribution and license notices for material included by permission granted via open licenses. The source files of most graphics can be found on this link: https://docs.google.com/presentation/d/1zTcrvJYkQrXrA_cYYuj7hqmxCMrncBqix9AX71wRtB4/edit?usp=sharing
This appendix contains attribution and license notices for material included by permission granted via open licenses.
This quick glossary contains many of the terms used in relation to Bitcoin. These terms are used throughout the book, so bookmark this for a quick reference.
This quick glossary contains many of the terms used in relation to Bitcoin and the Lightning Network. These terms are used throughout the book, so bookmark this for a quick reference.
address::
Bitcoin invoice addresses compactly encode the information necessary to pay a receiver. A modern address consists of a string of letters and numbers that starts with bc1 and looks like +bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4+. An address is shorthand for a receiver's locking script which can be used by a sender to sign over funds to the receiver. Most addresses either represent the receiver's public key or some form of redeemscript that defines more complex spending conditions. The above example is a bech32 address encoding a witness program locking funds to the hash of a public key (Pay to Witness Public Key Hash). There are also older address formats that start with 1 or 3 that use the base58check address encoding to represent public key hashes or script hashes.
Bitcoin addresses compactly encode the information necessary to pay a receiver. A modern address consists of a string of letters and numbers that starts with bc1 and looks like +bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4+. An address is shorthand for a receiver's locking script which can be used by a sender to sign over funds to the receiver. Most addresses either represent the receiver's public key or some form of script that defines more complex spending conditions. The above example is a bech32 address encoding a witness program locking funds to the hash of a public key (See _Pay to Witness Public Key Hash_). There are also older address formats that start with 1 or 3 that use the base58check address encoding to represent public key hashes or script hashes.
Asymmetric Cryptographic System::
Asymmetric cryptography, or public-key cryptography, is a cryptographic system that uses pairs of keys: public keys which may be disseminated widely, and private keys which are known only to the owner.
The generation of such keys depends on cryptographic algorithms based on mathematical problems to produce one-way functions.
The generation of such keys depends on cryptographic algorithms based on mathematical problems to produce functions that are easy to solve one way, but very difficult to solve in reverse.
Effective security only requires keeping the private key private; the public key can be openly distributed without compromising security.
Autopilot is a recommendation engine for Lightning Network nodes that uses statistics of the known topology to suggest which nodes they should open channels with.
autopilot::
An autopilot is a recommendation engine for Lightning nodes that uses statistics of the Lightning Network topology to suggest which nodes they should open channels with.
Depending on the implementation of the autopilot, the channel capacity may also be recommended.
Autopilots are not part of the Lightning Network Protocol.
@ -23,20 +22,20 @@ balance::
The balance of a payment channel is the amount of bitcoin that belongs to each channel partner.
For example, Alice could open a channel with Bob for the value of 1 BTC.
The channel balance is then 1 BTC to Alice and 0 BTC to Bob.
As the users transact the channel balance will update.
As the users transact, the channel balance will update.
For example if Alice sends 0.2 BTC to Bob, then the balance is now 0.8 BTC to Alice and 0.2 to Bob.
When the channel is closed the bitcoin in the channel will be divided between the two channel partners according to the latest balance encoded in the commitment transaction.
In the Lightning Network, the ability to send and receive payments is limited by channel balances.
See _capacity_.
bech32::
Bech32 refers to a generic checksummed base32 encoded format featuring strong error detection guarantees. While bech32 was originally developed to be used as the address format for native segwit outputs (BIP173), it is also used to encode lightning invoices (BOLT11). While native segwit version 0 outputs (P2WPKH and P2WSH) use bech32, higher native segwit output versions (e.g. P2TR) use the improved variant bech32m (BIP350). Bech32(m) addresses are sometimes referred to as "bc1" for the prefix of such addresses. Native segwit outputs are more blockspace efficient than older output types and therefore may require lower fees to be spent.
Bech32 refers to a generic check-summed base32-encoded format featuring strong error-detection guarantees. While bech32 was originally developed to be used as the address format for native Segwit outputs (BIP-173), it is also used to encode lightning invoices (BOLT #11). While native Segwit version 0 outputs (P2WPKH and P2WSH) use bech32, higher native Segwit output versions (e.g. Pay-to-Taproot or P2TR) use the improved variant bech32m (BIP-350). Bech32(m) addresses are sometimes referred to as "bc1" addresses, reflecting the prefix of such addresses. Native segwit outputs are more blockspace-efficient than older addresses and therefore may reduce transaction fees for the owner of such an address.
bip, BIP::
Bitcoin Improvement Proposals. A set of proposals that members of the Bitcoin community have submitted to improve Bitcoin. For example, BIP-21 is a proposal to improve the Bitcoin uniform resource identifier (URI) scheme. BIPs can be found at https://github.com/bitcoin/bips.
Bitcoin Improvement Proposal (BIP)::
A proposal that members of the Bitcoin community have submitted to improve Bitcoin. For example, BIP-21 is a proposal to improve the Bitcoin uniform resource identifier (URI) scheme. BIPs can be found at https://github.com/bitcoin/bips.
bitcoin, Bitcoin::
Depending on the context, could refer to the name of the currency unit (the coin), the network or the underlying enabling protocol. Written as bitcoin with a lowercase "b" usually refers to the currency unit. Bitcoin with an uppercase "B" usually refers to the protocol. See https://www.bitcoin.org for general information. The source code can be found at https://github.com/bitcoin/bitcoin.
Depending on the context, could refer to the name of the currency unit (the coin), the network or the underlying enabling protocol. Written as bitcoin with a lowercase "b" usually refers to the currency unit. Bitcoin with an uppercase "B" usually refers to the protocol, or system.
Bitcoin mining::
Bitcoin mining is the process of constructing a block from recent Bitcoin transactions and then solving a computational problem required as proof of work.
@ -45,78 +44,71 @@ Bitcoin mining::
Every time a new block is created, the mining node will receive new bitcoin created within the coinbase transaction of that block.
block::
A block consist of a header and a body of valid transactions.
The block is marked with a timestamp and commits to a specific predecessor block.
When hashed, the block header provides the proof of work which makes the blockchain probabilistically immutable.
Blocks must adhere to the rules enforced by network consensus to extend the main blockchain.
A block is a data structure in the Bitcoin blockchain that consists of a header and a body of Bitcoin transactions.
The block is marked with a timestamp and commits to a specific predecessor (parent) block.
When hashed, the block header provides the proof of work that makes the blockchain probabilistically immutable.
Blocks must adhere to the rules enforced by network consensus to extend the blockchain.
When a block is appended to the blockchain, the included transactions are considered to have their first confirmation.
blockchain::
The blockchain is a distributed log of all Bitcoin transactions.
Transactions are grouped in discrete updates called blocks limited to up to 4 million weight units.
The blockchain is a distributed log, or database, of all Bitcoin transactions.
Transactions are grouped in discrete updates called blocks, limited up to 4 million weight units.
Blocks are produced approximately every ten minutes via a stochastic process called mining.
Each block includes a computationally intensive "proof of work".
Each block includes a computationally intensive "proof of work."
The proof of work requirement is used to regulate the block intervals and protect the blockchain against attacks to rewrite history:
an attacker would need to outdo existing proof of work to replace already published blocks, making each block probabilistically immutable as it is buried under subsequent blocks.
BOLT::
BOLT, or Basis Of Lightning Technology, is the formal specification of the Lightning Network protocol. Unlike Bitcoin, which has a reference implementation that also serves as the protocol's specification, the various Lightning Network implementations follow BOLT so they can work with one another to form the same network. It is available at https://github.com/lightningnetwork/lightning-rfc.
BOLT, or Basis Of Lightning Technology, is the formal specification of the Lightning Network. Unlike Bitcoin, which has a reference implementation that also serves as the protocol's specification, the various Lightning Network implementations follow BOLT so they can work with one another to form the same network. It is available at https://github.com/lightningnetwork/lightning-rfc.
capacity::
The capacity of a payment channel is equivalent to the amount of bitcoin provided by the funding transaction.
As the funding transaction is publicly visible on the blockchain, and the channel is announced via the gossip protocol, the capacity is public information.
It does not reveal any information about how much bitcoin each of the channel partners owns in the channel.
It does not reveal any information about how much bitcoin each of the channel partners owns in the channel, i.e. the balance.
A high capacity does not guarantee that the channel can be used for routing in both directions.
The capacity does not change if some of the capacity is locked up in HTLCs.
This means that even a high capacity channel could at any time be unable to route a payment of a certain amount in both directions.
Not to be confused with the balance.
c-lightning::
Implementation of the Lightning Network Protocol by the Victoriabased company https://blockstream.com[Blockstream]. It is written in C. Source code is at https://github.com/ElementsProject/lightning.
Implementation of the Lightning Network Protocol by the Victoria-based company https://blockstream.com[Blockstream]. It is written in C. Source code is at https://github.com/ElementsProject/lightning.
closing transaction::
If both channel partners agree to close a channel they will create an exercise settlement transaction that reflects the most recent commitment transaction.
It does not include any Hashed Time Lock Contracts or Revocable Sequence Maturity Contracts.
After exchanging signatures for a closing transaction, no further channel updates should be made as this one allows one side to enforce the closing transaction on the blockchain.
If both channel partners agree to close a channel they will create a settlement transaction that reflects the most recent commitment transaction.
After exchanging signatures for a closing transaction, no further channel updates should be made.
Mutually closing a channel with the help of a closing transaction has the advantage that fewer blockchain transactions are required to claim all funds, in comparison to unilaterally forcing a channel close by publishing a commitment transaction. Additionally, funds for both parties are immediately spendable from a closing transaction.
CLTV::
CLTV is a shortform for the bitcoin OP code OP_CHECKLOCKTIMEVERIFY. This defines an absolute blockheight before an output can be spend. The atomicity of the routing process heavily depends on CLTV values in HTLCs. Routing nodes announce their CLTV expiry deltas that they wish for an incoming and outgoing htlc to have at least via the gossip protocol.
CLTV is an acronym/abbreviation for the Bitcoin Script operator OP_CHECKLOCKTIMEVERIFY. This defines an absolute blockheight before an output can be spend. The atomicity of the routing process heavily depends on CLTV values in HTLCs. Routing nodes announce, via the gossip protocol, their expected CLTV expiry deltas that they wish for any incoming and outgoing HTLCs.
coinbase::
The coinbase is a special field only permitted in the sole input of coinbase transactions.
The coinbase allows up 100 bytes of arbitrary data, but since BIP34 must first feature the current block height to ensure that coinbase transactions are unique.
The coinbase allows up 100 bytes of arbitrary data, but since BIP-34 must first feature the current block height to ensure that coinbase transactions are unique.
Not to be confused with coinbase transaction.
coinbase transaction::
The first transaction in a block which is always created by a miner and which includes a single coinbase.
The coinbase transaction may claim the block reward and assign it to one or more outputs.
The block reward consists of two parts.
First, newly generated coins. The amount of allowed coins to be generated is part of the consensus rules and decreases over time based on the current block height.
Second, the miner may also claim all the fees of transactions included in the current block.
The block reward consists of the block subsidy (newly created bitcoin) and the sum of all transaction fees from transactions included in the block.
Coinbase outputs can only be spent after maturing for 100 blocks.
If the block includes any segwit transactions, the coinbase transaction must include a commitment to the witness transaction identifiers in an additional nulldata output.
Not to be confused with coinbase.
If the block includes any Segwit transactions, the coinbase transaction must include a commitment to the witness transaction identifiers in an additional output.
cold storage::
Refers to keeping an amount of bitcoin offline. Cold storage is achieved when Bitcoin private keys are created and stored in a secure offline environment. Cold storage is important to protect bitcoin holdings. Online computers are vulnerable to hackers and should not be used to store a significant amount of bitcoin.
Commitment Transaction::
commitment transaction::
A commitment transaction is a Bitcoin transaction, signed by both channel partners, that encodes the latest balance of a channel.
Every time a new transaction is made or forwarded using the channel, the channel balance will update, and a new commitment transaction will be signed by both parties.
Importantly, for a channel between Alice and Bob, both Alice and Bob keep their own version of the commitment transaction, which is also signed by the other party.
Every time a new payment is made or forwarded using the channel, the channel balance will update, and a new commitment transaction will be signed by both parties.
Importantly, in a channel between Alice and Bob, both Alice and Bob keep their own version of the commitment transaction, which is also signed by the other party.
At any point, the channel can be closed by either Alice or Bob if they submit their commitment transaction to the Bitcoin blockchain.
Submitting an older (outdated) commitment transaction is considered "cheating" (i.e. a protocol breach) in the Lightning network and can be penalized by the other party, claiming all the funds in the channel for themselves.
Submitting an older (outdated) commitment transaction is considered "cheating" (i.e. a protocol breach) in the Lightning network and can be penalized by the other party, claiming all the funds in the channel for themselves, via a penalty transaction.
confirmations::
Once a transaction is included in a block, it has one confirmation. As soon as _another_ block is mined on the same blockchain, the transaction has two confirmations, and so on. Six or more confirmations are considered sufficient proof that a transaction cannot be reversed.
Once a transaction is included in a block, it has one confirmation. As soon as _another_ block is mined on the blockchain, the transaction has two confirmations, and so on. Six or more confirmations are considered sufficient proof that a transaction cannot be reversed.
contract::
A contract is a set of Bitcoin transactions which together result in a certain desired behavior.
Examples are RSMCs to create a trustless, bi-directional payment channel or HTLCs to create a mechanism which allows trustless forwarding of payments through third parties.
Diffie Hellman (DH) Key Exchange::
On the Lightning Network, the Elliptic Curve Diffie-Hellman method is used.
Diffie Hellman Key Exchange (DHKE)::
On the Lightning Network, the Elliptic Curve Diffie-Hellman (ECDH) method is used.
It is an anonymous key agreement protocol that allows two parties, each having an elliptic-curve public-private key pair, to establish a shared secret over an insecure communication channel.
This shared secret may be directly used as a key, or to derive another key.
The key, or the derived key, can then be used to encrypt subsequent communications using a symmetric-key cipher.
Via https://en.wikipedia.org/w/index.php?title=Elliptic-curve_Diffie%E2%80%93Hellman&oldid=836070673
digital signature::
A digital signature is a mathematical scheme for verifying the authenticity of digital messages or documents.
A valid digital signature gives a recipient reason to believe that the message was created by a known sender, that the sender cannot deny having sent the message, and that the message was not altered in transit.
A digital signature is a mathematical scheme for verifying the authenticity and integrity of digital messages or documents.
They can be seen as cryptographic commitments in which the message is not hidden.
Double-spending is the result of successfully spending some money more than once.
Bitcoin protects against double-spending by verifying that each transaction added to the blockchain plays by the rules; this means checking that the inputs for the transaction have not previously already been spent.
Bitcoin protects against double-spending by verifying that each transaction added to the blockchain adheres to the rules of consensus; this means checking that the inputs for the transaction have not been previously spent.
ECDSA::
Elliptic Curve Digital Signature Algorithm or ECDSA is a cryptographic algorithm used by Bitcoin to ensure that funds can only be spent by their rightful owners.
Elliptic Curve Digital Signature Algorithm (ECDSA)::
Elliptic Curve Digital Signature Algorithm or ECDSA is a cryptographic algorithm used by Bitcoin to ensure that funds can only be spent by the holder of the correct private key.
Eclair::
Implementation of the Lightning Network Protocol by the Paris based company https://acinq.co[ACINQ]. It is written in Scala. Source code is at https://github.com/ACINQ/eclair.
encoding::
Encoding is the process of converting a message into a different form.
For example, converting a human-readable form to a digitally space-efficient form.
Encoding is the process of converting a message into a different form. For example, converting a number from decimal to a hexadecimal.
Electrum server::
An Electrum server is a Bitcoin node with an additional interface (API) is often required by bitcoin wallets that do not run a full node. For example, these wallets check the status of specific transactions or broadcast transactions to the mempool using Electrum server APIs. Some Lightning wallets also use Electrum servers, so even if they are non-custodial, they may compromise user sovereignty in that users trust the Electrum server to provide accurate information and privacy in that calls made to the Electrum server may reveal private information.
An Electrum server is a Bitcoin node with an additional interface (API). It is often required by bitcoin wallets that do not run a full node. For example, these wallets check the status of specific transactions or broadcast transactions to the mempool using Electrum server APIs. Some Lightning wallets also use Electrum servers.
ephemeral key::
Ephemeral keys are mainly within the SPHINX Mix Format and Onion Routing on the Lightning Network.
They are generated for each execution of the routing process.
Ephemeral keys are keys that are only used for a short time and not retained after use. They are often derived for use in one session from another key that is held long term. Ephemeral keys are mainly used within the SPHINX Mix Format and Onion Routing on the Lightning Network.
This increases the security of transported messages or payments.
Even if an ephemeral key leaks, only information about a single payment becomes public.
Even if an ephemeral key leaks, only information about a single session becomes public.
feature bits::
A binary string that Lightning nodes use to communicate to each other which features they support.
Feature bits are included in many Lightning Messages as well as BOLT #11.
They can be decoded using BOLT #9, and will tell nodes which features the node has enabled, and whether these are backward-compatible.
Feature bits are included in many Lightning Messages as well as BOLT 11.
They can be decoded using BOLT 9, and will tell nodes which features the node has enabled, and whether these are backwardcompatible.
Also known as feature flags.
fees::
In the context of the Lightning Network, nodes will charge routing fees for forwarding other users' payments.
Individual nodes can set their own fee policies which will be calculated as the sum of a fixed _base_fee_ and a _fee_rate_ which depends on the payment amount.
In the context of Bitcoin, the sender of a transaction pays a fee to miners for including the transaction in a block.
Bitcoin transaction fees do not include a base fee and only depende linearly on the size of the transaction in Bytes but not on the amount.
Individual nodes can set their own fee policies which will be calculated as the sum of a fixed +base_fee+ and a +fee_rate+ which depends on the payment amount.
In the context of Bitcoin, the sender of a transaction pays a transaction fee to miners for including the transaction in a block.
Bitcoin transaction fees do not include a base fee and depend linearly on the weight of the transaction, but not on the amount.
funding transaction::
The funding transaction is used to open a payment channel.
From the perspective of the Bitcoin network, the process of opening a channel by creating a RSMC is started by creating the funding transaction and finished by broadcasting it to the Bitcoin network and have it included in the blockchain.
The value of the funding transaction is exactly the capacity of the payment channel.
The output of the funding transaction is a 2-of-2 multisignature script (multisig) where each channel partner controls one key.
The funding transaction is used to open a payment channel. The value (in bitcoin) of the funding transaction is exactly the capacity of the payment channel.
The output of the funding transaction is a 2-of-2 multisignature script (multisig) where each channel partner controls one key. Due to its multisig nature, it can only be spent by mutual agreement between the channel partners.
It will eventually be spent by one of the commitment transactions or by the closing transaction.
Due to its multisig nature, it can only be spent mutually.
It is part of the RSMC to ensure that either side of the channel can withdraw their funds without the necessity to trust the channel partner.
globalfeatures::
Globalfeatures of a Lightning Network node are the features of interest for all other nodes.
global features (field: +globalfeatures+)::
Globalfeatures of a Lightning node are the features of interest for all other nodes.
Most commonly they are related to supported routing formats.
They are announced in the `_init_` message of the peer protocol as well as the `_channel_announcement_` and `_node_announcement_` messages of the gossip protocol.
Gossip Protocol::
gossip protocol::
Lightning Network nodes send and receive information about the topology of the Lightning Network through gossip messages which are exchanged with their peers.
The gossip protocol is mainly defined in BOLT 7 and defines the format of the _node_announcement_, _channel_announcement_ and _channel_update messages_.
In order to prevent SPAM, node announcement messages will only be forwarded if the node already has a channel and channel announcement messages will only be forwarded if the funding transaction of the channel has been confirmed by the Bitcoin network.
The gossip protocol is mainly defined in BOLT #7 and defines the format of the _node_announcement_, _channel_announcement_ and _channel_update messages_.
In order to prevent spam, node announcement messages will only be forwarded if the node already has a channel and channel announcement messages will only be forwarded if the funding transaction of the channel has been confirmed by the Bitcoin network.
Usually, Lightning nodes connect with their channel partners, but it is fine to connect with any other Lightning node in order to process gossip messages.
hardware wallet::
A hardware wallet is a special type of Bitcoin wallet which stores the user's private keys in a secure hardware device.
As of writing the book, hardware wallets for consumers are not available for Lightning Network nodes as they need to be online to participate in the protocol.
Several groups are currently working on solutions.
As of writing the book, hardware wallets are not available for Lightning Network nodes as the keys used by Lightning need to be online to participate in the protocol.
hash::
A digital fingerprint of some binary input.
A fixed-size digital fingerprint of some arbitrary-length binary input. Also known as a _digest_.
hash-based message authentication code (HMAC)::
An HMAC is a message authentication code method for verifying the integrity and authenticity of a message based on a hash function and a cryptographic key.
It is used in onion routing to ensure the integrity of a packet at each hop, as well as within the Norse protocol variant used for message encryption.
Hash-based Message Authentication Code (HMAC)::
An HMAC is an algorithm for verifying the integrity and authenticity of a message based on a hash function and a cryptographic key.
It is used in onion routing to ensure the integrity of a packet at each hop, as well as within the Noise protocol variant used for message encryption.
hash function::
A cryptographic hash function is a mathematical algorithm that maps data of arbitrary size to a bit string of a fixed size (a hash) and is designed to be a one-way function, that is, a function which is infeasible to invert.
The only way to recreate the input data from an ideal cryptographic hash function's output is to attempt a brute-force search of possible inputs to see if they produce a match, or use a rainbow table of matched hashes.
The ideal cryptographic hash function has five main properties: It is deterministic, so the same message always results in the same hash.
It is quick to compute the hash value for any given message.
It is infeasible to generate a message from its hash value except by trying all possible messages.
A small change to a message should change the hash value so extensively that the new hash value appears uncorrelated with the old hash value.
It is infeasible to find two different messages with the same hash value.
The only way to recreate the input data from an ideal cryptographic hash function's output is to attempt a brute-force search of possible inputs to see if they produce a match.
From Wikipedia: https://en.wikipedia.org/w/index.php?title=Cryptographic_hash_function&oldid=868055371
hashlocks::
A hashlock is a type of encumbrance that restricts the spending of an output until a specified piece of data is publicly revealed. Hashlocks have the useful property that once any hashlock is opened publicly, any other hashlock secured using the same key can also be opened. This makes it possible to create multiple outputs that are all encumbered by the same hashlock and which all become spendable at the same time.
A hashlock is a Bitcoin Script spending condition that restricts the spending of an output until a specified piece of data is revealed. Hashlocks have the useful property that once any hashlock is revealed through spending, any other hashlock secured using the same key can also be spent. This makes it possible to create multiple outputs that are all encumbered by the same hashlock and which all become spendable at the same time.
HTLC::
A Hashed TimeLock Contract or HTLC is a class of payments that use hashlocks and timelocks to require that the receiver of a payment either acknowledges receiving the payment prior to a deadline by generating cryptographic proof of payment (usually called the preimage of the payment hash) or forfeits the ability to claim the payment, returning it to the payer.
Hash Time-Locked Contract (HTLC)::
A Hash Time-Locked Contract or HTLC is a Bitcoin Script that consists of hashlocks and timelocks to require that the recipient of a payment either spends the payment prior to a deadline by presenting the has pre-image or the sender can claim a refund after the timelock expires.
On the Lightning Network HTLCs are outputs in the commitment transaction of a payment channel and are used to enable the trustless routing of payments.
invoice::
The payment process on the Lightning Network is initiated by the payee who issues an invoice.
Invoices include the payment hash, the amount, a description and the expiry time.
The payment process on the Lightning Network is initiated by the recipient (payee) who issues an invoice, also known as a _payment request_.
Invoices include the payment hash, the amount, a description and the expiry time. Lightning invoices are defined in BOLT #11.
Invoices can also include a fallback Bitcoin address to which the payment can be made in case no route can be found, as well as hints for routing a payment through a private channel.
JIT Routing::
"Just in Time" Routing.
An alternative to source-based routing was first proposed by co-author René Pickhardt.
With JIT routing, intermediary nodes along a path can pause an in-flight payment to rebalance their channels.
JIT-Routing::
"Just in Time" Routing is an alternative to source-based routing that was first proposed by co-author René Pickhardt.
With JIT-Routing, intermediary nodes along a path can pause an in-flight payment, to re-balance their channels before proceeding with the payment.
This might allow them to successfully forward payments that might otherwise have failed due to a lack of outgoing capacity.
Lightning message::
A Lightning message is an encrypted data string that can be sent between two peers on the Lightning Network. Similar to other communication protocols, Lightning messages consist of a header and a body. The header and the body have their own HMAC. This ensures that the headers of fixed length will also be encrypted and adversaries won't be able to deduce what messages are being sent by inspecting the length. Lightning Messages are the core building block that is defined in the Messaging layer.
A Lightning message is an encrypted data string that can be sent between two peers on the Lightning Network. Similar to other communication protocols, Lightning messages consist of a header and a body. The header and the body have their own HMAC. Lightning messages are the main building block of the Messaging layer.
Other components of the Lightning Network are the gossip protocol, the transport layer, and payment requests.
Lightning Network Protocol Suite::
The Lightning Protocol Suite consists of 5 Layers which are responsible for various parts of the protocol.
From bottom (the first layer) to the top (the fifth layer) these layers are called: Network Communication Layer, Messaging Layer, Peer 2 Peer Layer, Routing Layer, Payment Layer.
Various BOLTs define tasks on one or several layers.
The Lightning Protocol Suite consists of five Layers that are responsible for various parts of the protocol.
From bottom (the first layer) to the top (the fifth layer), these layers are called the Network Communication Layer, the Messaging Layer, the Peer 2 Peer Layer, the Routing Layer, and the Payment Layer.
Various BOLTs define parts of one or several layers.
Lightning Network Node, Lightning Node::
A participant on the Lightning Network.
A Lightning user will run Lightning node software in order to interact with other Lightning nodes.
Lightning Network node, Lightning node::
A computer participating in the Lightning Network, via the Lightning Peer-to-Peer protocol.
Lightning nodes have the ability to open channels with other nodes, send and receive payments, and route payments from other users.
Typically a Lightning node user will also run a Bitcoin node.
@ -249,30 +224,30 @@ lnd::
Implementation of the Lightning Network Protocol by the San Francisco based company https://lightning.engineering[Lightning Labs].
It is written in Go. Source code is at https://github.com/lightningnetwork/lnd.
localfeatures::
Localfeatures of a Lightning Network node are the configurable features of direct interest of the peer.
Local features (field: +localfeatures+)::
Localfeatures of a Lightning Network node are the configurable features of direct interest the its peers.
They are announced in the `_init_` message of the peer protocol as well as the `_channel_announcement_` and `_node_announcement_` messages of the gossip protocol.
Locktime::
Locktime, or more technically nLockTime, is the part of a transaction which indicates the earliest time or earliest block when that transaction may be added to the blockchain.
Locktime, or more technically nLockTime, is the part of a Bitcoin transaction that indicates the earliest time or earliest block when that transaction may be added to the blockchain.
Messaging Layer::
The Layer builds on top of the Network Connection Layer of the Lightning Network Protocol Suite.
It is responsible to ensure an encrypted and secure communication and exchange of information via the chosen Network Connection Layer protocol.
The messaging layer defines the framing and format of Lightning Messages as defined in BOLT 01.
The feature bits defined in BOLT 09 are also part of this layer.
It is responsible for ensuring an encrypted and secure communication and exchange of information via the chosen Network Connection Layer protocol.
The messaging layer defines the framing and format of Lightning Messages as defined in BOLT #1.
The feature bits defined in BOLT #9 are also part of this layer.
millisatoshi::
The smallest unit of account on the Lightning Network. A millisatoshi is one hundred billionth of a single bitcoin. A millisatoshi is one thousandth of one Satoshi. Millisatoshis do not exist, nor can they be settled on the Bitcoin network.
The smallest unit of account on the Lightning Network. A millisatoshi is one hundred billionth of a single bitcoin. A millisatoshi is one thousandth of one satoshi. Millisatoshis do not exist on, nor can they be settled on, the Bitcoin network.
MPP::
A multipart payment (which is often also referred to as multipath payment) is a method for payments where the sender can split the payment amount into multiple smaller parts and deliver them potentially along multiple potentially disjoint paths. As the MPP strategies do not require one to send the smaller splits along different paths the term multipart payment is more accurate than multipath payment. In computer science multipart paryments are modelled as network flows.
Multi-Part Payment (MPP))::
A Multi-Part Payment (MPP), often also referred to as a Multi-Path Payment) is a method for splitting the payment amount into multiple smaller parts and delivering them along one or more paths. Since MPP can send many or all parts over a single path the term Multi-Part Payment is more accurate than Multi-Path Payment. In computer science Multi-Part Payments are modelled as network flows.
multi-signature::
Mult-isignature (multisig) refers to requiring more than one key to authorize a Bitcoin transaction.
Payment channels are always encoded as multisignature addresses requiring one signature from each peer of the payment channel.
In the standard case of a two-party payment channel, a 2-of-2 multisignature address is used.
multisignature::
Multi-signature (multisig) refers to a script that requires more than one signature to authorize spending.
Payment channels are always encoded as mutlisig addresses requiring one signature from each partner of the payment channel.
In the standard case of a two-party payment channel, a 2-of-2 multisig address is used.
node::
See _Lightning Network Node_
@ -280,22 +255,18 @@ node::
network capacity::
Lightning network capacity is the total amount of bitcoin locked and circulated inside the Lightning Network.
It is the sum of capacities of each public channel.
It reflects the usage of the Lightning Network to some extent because we expect that people lock bitcoin into Lightning channels in order to spend it or forward other users' payments.
Hence the higher the amount of bitcoin locked in channels, the higher the expected usage of the Lightning Network.
Note that since only public channel capacity can be observed, the true network capacity is unknown.
See _private channel_.
It reflects the usage of the Lightning Network to some extent because we expect that people put bitcoin into Lightning channels in order to spend it or forward other users' payments.
Hence the higher the amount of bitcoin in Lightning channels, the higher the expected usage of the Lightning Network.
Note that since only public channel capacity can be observed, the true network capacity is unknown. Also, since a channel's capacity can enable an unlimited number of payments back-and-forth, network capacity does not imply a limit of value transferred on the Lightning Network.
Network Connection Layer::
The lowest layer of the Lightning Network Protocol Suite.
Its responsability is to support internet protocols like IPv4, IPv6, TOR2 and TOR3 and use them to establise a secure cryptographic communication channel on top of them as defined in BOLT8 or to speak DNS for the Bootstrapping of the Network as defined BOLT 10.
Its responsibility is to support internet protocols like IPv4, IPv6, TOR2, and TOR3, and use them to establish a secure cryptographic communication channel as defined in BOLT 8, or to speak DNS for the bootstrapping of the network as defined in BOLT 10.
Noise_XK::
The template of the Noise protocol framework to establish an authenticated and encrypted communication channel between two peers of the Lightning Network.
X means that no public key needs to be known from the initiator of the connection.
K means that the public key of the receiver needs to be known.
More particular (from: http://www.noiseprotocol.org/noise.html) the protocol enables encryption to a known recipient and strong forward secrecy. This payload is encrypted based on an ephemeral-ephemeral DH as well as an ephemeral-static DH with the recipient's static key pair. Assuming the ephemeral private keys are secure, and the recipient is not being actively impersonated by an attacker that has stolen its static private key, this payload cannot be decrypted. Sender authentication is resistant to key-compromise impersonation (KCI). The sender authentication is based on an ephemeral-static DH ("es" or "se") between the sender's static key pair and the recipient's ephemeral key pair. Assuming the corresponding private keys are secure, this authentication cannot be forged.
// the noise protocol documentation is according to their IPR section public domain. The author is Trevor Perrin (noise@trevp.net)
onion routing::
Onion routing is a technique for anonymous communication over a computer network.
@ -303,161 +274,150 @@ onion routing::
The encrypted data is transmitted through a series of network nodes called onion routers, each of which peels away a single layer, uncovering the data's next destination.
When the final layer is decrypted, the message arrives at its destination.
The sender remains anonymous because each intermediary knows only the location of the immediately preceding and following nodes.
From Wikipedia: https://en.wikipedia.org/w/index.php?title=Onion_routing&oldid=870849217
output::
The output of a bitcoin transaction, also called an Unspent Transaction Output or UTXO.
An output is essentially an amount of bitcoin that can be spent, as well as a script that defines what conditions need to be fulfilled for that bitcoin to be spent.
Every bitcoin transaction consumes the outputs of the sender and creates new outputs that can be spent later by the receiver.
A typical bitcoin output will require the signature of the owner to be spent, but outputs can require the fulfillment of more complex scripts.
For example, a multisignature script requires that two or more users to sign before an output can be spent, which is a fundamental building block of the Lightning Network.
The output of a Bitcoin transaction, also called an Unspent Transaction Output or UTXO.
An output is an indivisible amount of bitcoin that can be spent, as well as a script that defines what conditions need to be fulfilled for that bitcoin to be spent.
Every bitcoin transaction consumes some outputs of previously recorded transactions and creates new outputs that can be spent later by subsequent transactions.
A typical bitcoin output will require a signature to be spent, but outputs can require the fulfillment of more complex scripts.
For example, a multisignature script requires that two or more key holders sign before the output can be spent, which is a fundamental building block of the Lightning Network.
Pay To Public Key Hash (P2PKH)::
P2PKH or Pay-to-PubKey-Hash is a type of transaction that pays a Bitcoin address that contains P2PKH scripts.
An output locked by a P2PKH script can be unlocked (spent) by presenting a public key and a digital signature created by the corresponding private key.
P2PKH is a type of output that locks bitcoin to the hash of a public key. An output locked by a P2PKH script can be unlocked (spent) by presenting the public key matching the hash and a digital signature created by the corresponding private key.
Pay To Script Hash (P2SH)::
P2SH or Pay-to-Script-Hash is a powerful type of transaction that greatly simplifies the use of complex transaction scripts. With P2SH the complex script that details the conditions for spending the output (redeem script) is not presented in the locking script. Instead, only a hash of it is in the locking script.
P2SH is a versatile type of output that allows the use of complex Bitcoin Scripts. With P2SH the complex script that details the conditions for spending the output (redeem script) is not presented in the locking script. Instead, value is locked to the hash of a script, which must be presented and fulfilled to spend the output.
P2SH address::
P2SH addresses are Base58Check encodings of the 20-byte hash of a script, P2SH addresses use the version prefix "5", which results in Base58Check-encoded addresses that start with a "3". P2SH addresses hide all of the complexity, so that the person making a payment does not see the script.
P2SH addresses are Base58Check encodings of the 20-byte hash of a script. P2SH addresses start with a "3". P2SH addresses hide all of the complexity, so that the sender of a payment does not see the script.
Pay To Witness Public Key Hash (P2WPKH)::
The signature of a Pay-to-Witness-Public-Key-Hash (P2WPKH) contains the same information as a P2PKH spending, but is located in the witness field instead of the scriptSig field. The scriptPubKey is also modified.
P2WPKH is the Segwit equivalent of P2PKH, using a segregated witness. The signature to spend a P2WPKH output is put in the witness tree instead of the scriptSig field. See _Segwit_
P2WPKH address::
The "native Segwit v0" address format, P2WPKH addresses are bech32 encoded and start with "bc1q"
Pay To Witness Script Hash (P2WSH)::
The difference between Pay-to-Script-Hash (P2SH) and Pay-to-Witness-Script-Hash (P2WSH) is the location change of the cryptographic proof from the scriptSig field to the witness field and the modification of the scriptPubKey.
P2WSH is the Segwit equivalent of P2SH, using a segregated witness. The signature and script to spend a P2WSH output is put in the witness tree instead of the scriptSig field. See _Segwit_
P2WSH address::
The "native Segwi v0" script address format, P2WSH addresses are bech32 encoded and start with "bc1q"
Pay To Taproot (P2TR)::
Activating in November 2021, Taproot is a new output type that locks bitcoin to a tree of spending conditions, or a single root condition.
P2TR address::
The Taproot address format, representing Segwit v1, is a bech32m encoded address and starts with "bc1p"
payment::
A payment occurs if bitcoin is transferred within the Lightning Network.
Payments are generally not seen on the blockchain.
The recipient initiates a payment by creating an invoice.
The invoice includes a payment hash which is the hash of a secret preimage.
This payment hash is used by the Hashed Time Lock Contracts during the routing process.
A Lightning payment occurs when bitcoin is transferred within the Lightning Network. Payments are generally not seen on the Bitcoin blockchain.
payment channel::
A payment channel is a financial relationship between two nodes on the Lightning Network, created using a multisignature bitcoin transaction.
A payment channel is a financial relationship between two nodes on the Lightning Network, created using a bitcoin transaction paying a multisignature address.
The channel partners can use the channel to send bitcoin back and forth between each other without committing all of the transactions to the Bitcoin blockchain.
In a typical payment channel only two transactions, the funding transaction and the commitment transaction, are added to the blockchain.
The other transactions are not included in the blockchain and are said to occur "off-chain".
There are various methods of constructing a payment channel discussed further in the chapter on channel construction.
Payments sent across the channel are not recorded in the blockchain and are said to occur "off-chain".
Payment Layer::
The top and fifth Layer of the Lightning Network Protocol Suite operates on top of the Routing Layer.
Its responsability is to enable the Payment process via BOLT #11 invoices.
While it heaviliy uses the Channel Graph from the Gossip Protcol as defined in BOLT #7 the actual strategies to deliver a payment are not part of the specification of the Protocol and left to the implementations.
As this topic is very import to ensure reliability of the Payment process and Payment deliver we included it in this book.
Its responsibility is to enable the Payment process via BOLT #11 invoices.
While it heavily uses the Channel Graph from the Gossip Protocol as defined in BOLT #7 the actual strategies to deliver a payment are not part of the specification of the Protocol and left to the implementations.
As this topic is very important to ensure reliability of the payment delivery process, we included it in this book.
peer::
Two parties which form a payment channel are called peers.
In particular, they are connected via an encrypted, authenticated communication over a TCP Socket.
The participants in a Peer-to-Peer network. In the Lightning Network, peers connect to each other via encrypted, authenticated communication through a TCP socket, over IP or Tor.
Peer-to-Peer Layer::
The Peer to Peer layer is the third layer of the Lightning Network Protocol Suite and works on top of the Messaging Layer.
It is responsible to define the syntax and semantics of Information exchanged between peers via Lightning Messages.
This consists of Control messages as defined in BOLT #9, Channel esablishment, operation and closing messages as defined in BOLT #2 as well as Gossip and Routing Messages as defined in BOLT #7.
The Peer-to-Peer layer is the third layer of the Lightning Network Protocol Suite and works on top of the Messaging Layer.
It is responsible for defining the syntax and semantics of information exchanged between peers via Lightning Messages.
This consists of control messages as defined in BOLT #9, channel establishment, operation and closing messages as defined in BOLT #2 as well as gossip and routing messages as defined in BOLT #7.
private channel::
A channel not announced to the rest of the network.
Technically "private" is a misnomer as these channels can still be identified through routing hints and commitment transactions.
They are better described as "unannounced" channels.
With an unannounced channel, the channel partners can send and receive payments between each other as normal.
However, the rest of the network will not be aware of it and so cannot typically use it to route payments.
With an unannounced channel, the channel partners can send and receive payments between each other as normal.
However, the rest of the network will not be aware of the channel and so cannot typically use it to route payments.
As the number of and capacity of unannounced channels is unknown, the total public channel count and capacity only accounts for a portion of the total Lightning Network.
preimage::
In mathematics, given a function $f$ and a value $h$ the preimage of $h$ with respect to $f$ is the set of values $R = \{r_1,r_2,...\}$ such that $f(r_i) = h$ for all $\r_i \in R$.
In layman's terms, it is the set of values which is mapped to $h$ by the function $f$.
This preimage set can be empty, finite or infinite.
In cryptography, the function $f$ is usually taken to be a hash function.
Cryptographers use the term preimage for an arbitrary element of $R$.
In particular, when using SHA-256, it should be stated that each element has an infinite number of preimages.
However, it is still believed to be computationally hard to find such a preimage.
In the context of cryptography and specifically in the Lightning Network, the preimage refers to the input of a hash function that produces a specific hash. It is not feasible to compute the preimage from the hash (hash functions only go one way). By selecting a secret random value as a preimage and calculating its hash, we can commit to that preimage and later reveal it. Anyone can confirm that the revealed preimage correctly produces the hash.
Proof of Work (PoW)::
A piece of data that requires significant computation to find.
In Bitcoin, miners must find a numeric solution to the SHA256 algorithm that meets a network-wide target, called the difficulty target.
Data that requires significant computation to find, and can be verified by anyone easily to prove the amount of work that was required to produce it.
In Bitcoin, miners must find a numeric solution to the SHA-256 algorithm that meets a network-wide target, called the difficulty target.
See _Bitcoin mining_ for more information.
PTLC::
A PTLC is the short form for Point Timelock contract. Similar to an HTLC it is a Bitcoin script that allows a conditional spend either on the presentation of a secret or after a certain blockheight has passed. The secret in PTLCs does not depend on a preimage of a hash function but rather on the private key from an elliptic curve point. The security assumption is thus based on the discret logarithm. PTLCs are not yet used on the Lightning Network.
Point Time-Locked Contract (PTLC)::
A Point Time-Locked Contract (PTLC) is a Bitcoin script that allows a conditional spend either on the presentation of a secret or after a certain blockheight has passed, similar to an HTLC. Unlike HTLCs, PTLCs do not depend on a preimage of a hash function but rather on the private key from an elliptic curve point. The security assumption is thus based on the discrete logarithm. PTLCs are not yet implemented on the Lightning Network.
Relative Timelock::
Relative Timelock is a type of timelock which allows an input to specify the earliest time the input can be added to a block. The time is relative and is based on when the output referenced by that input was included in a block. Such a feature is jointly achieved by nSequence field and CheckSequenceVerify opcode, which was introduced by BIP68/112/113.
relative timelock::
A relative timelock is a type of timelock which allows an input to specify the earliest time the input can be added to a block. The time is relative and is based on when the output referenced by that input was recorded in a block. Relative timelocks are set by the nSequence transaction field and CHECKSEQUENCEVERIFY (CSV) Bitcoin Script opcode, which was introduced by BIP-68/112/113.
Revocable Sequence Maturity Contract::
Revocable Sequence Maturity Contract (RSMC)::
This contract is used to construct a payment channel between two Bitcoin or Lightning Network users who do not need to trust each other.
The name comes from a sequence of states which are encoded as commitment transactions and can be revoked if wrongfully published and mined by the Bitcoin network.
These contracts are commonly referred to as RSMCs.
Unlike an HTLC, whose timeout is to make an HTLC temporary, and therefore should be absolute; a RSMC timeout is meant to only start when a commitment transaction is mined, and therefore should be using a Relative Timelock.
revocation key::
Each Revocable Sequence Maturity Contract contains two revocation keys.
Each channel partner knows one revocation key.
Knowing both revocation keys, the output of the Revocable Sequence Maturity Contract can be spent within the predefined timelock.
Revocation keys are used to disincentivize channel partners from broadcasting an old channel state.
While negotiating a new channel state, the old revocation keys are being shared.
Revocation keys are used instead of signatures since they can be derived with an HD key derivation scheme.
This makes it less cumbersome to store all revocation keys of old states.
Every RSMC contains two revocation keys. Each channel partner knows one revocation key. Knowing both revocation keys, the output of the RSMC can be spent within the predefined timelock. While negotiating a new channel state, the old revocation keys are shared, thereby "revoking" the old state. Revocation keys are used to disincentivize channel partners from broadcasting an old channel state.
RIPEMD-160::
RIPEMD-160 is a 160-bit cryptographic hash function. RIPEMD-160 is a strengthened version of RIPEMD with a 160-bit hash result, and is expected to be secure for the next ten years or more.
RIPEMD-160 is a cryptographic hash function that produces a 160-bit (20-byte) hash.
Routing Layer::
The fourth Layer of the Lightning Network Protocol Suite operats on top of the Peer 2 Peer Layer.
Its responsability is to define the cryptographic primitives and neccessary communication protocol to allow the secure and atomic transport of bitcoin from a sending node to a recipient node.
While BOLT #4 defines the onion format that is used to communicate transport information to remote peers with whom no direct connections exist the actual transport of the Onions and cryptographic primitives are defined in BOLT #2.
The fourth Layer of the Lightning Network Protocol Suite operates on top of the Peer-to-Peer layer.
Its responsibility is to define the cryptographic primitives and necessary communication protocol to allow the secure and atomic transport of bitcoin from a sending node to a recipient node.
While BOLT 4 defines the onion format that is used to communicate transport information to remote peers with whom no direct connections exist, the actual transport of the Onions and cryptographic primitives are defined in BOLT 2.
topology::
The topology of the Lightning Network describes the shape of the Lightning Network as a mathematical graph.
Nodes of the graph are the Lightning Network nodes or participants.
The edges of the graph are the payment channels.
The topology of the Lightning Network is publicly broadcast with the help of the gossip protocol unless nodes decide to act privately.
This means that the Lightning Network may be significantly larger than the announced number of nodes.
The topology of the Lightning Network describes the shape of the Lightning Network as a mathematical graph. Nodes of the graph are the Lightning nodes (network participants/peers). The edges of the graph are the payment channels.
The topology of the Lightning Network is publicly broadcast with the help of the gossip protocol, with the exception of unannounced channels.
This means that the Lightning Network may be significantly larger than the announced number of channels and nodes.
Knowing the topology is of particular interest in the source-based routing process of payments in which the sender discovers a route.
Also, the topology is important for features such as the autopilot.
satoshi::
A satoshi is the smallest denomination of bitcoin that can be recorded on the blockchain. It is the equivalent of 0.00000001 bitcoin and is named after the creator of Bitcoin, Satoshi Nakamoto.
A satoshi is the smallest unit (denomination) of bitcoin that can be recorded on the blockchain. One satoshi is 1/100 millionth (0.00000001) bitcoin and is named after the creator of Bitcoin, Satoshi Nakamoto.
Satoshi Nakamoto::
Satoshi Nakamoto is the name used by the person or group of people who designed Bitcoin and created its original reference implementation, Bitcoin Core. As a part of the implementation, they also devised the first blockchain database. In the process, they were the first to solve the double-spending problem for digital currency. Their real identity remains unknown.
Satoshi Nakamoto is the name used by the person or group of people who designed Bitcoin and created its original reference implementation. As part of the implementation, they also devised the first blockchain database. In the process, they were the first to solve the double-spending problem for digital currency. Their real identity remains unknown.
Schnorr Signature::
A linear sceme for digital signatures that is supposed to be activated in Bitcoin during November 2021. It enables many improvements on the Lightning Network like PTLCs which are supposed to superceed HTLCs.
A new digital signature scheme that will be activated in Bitcoin in November 2021. It enables innovations on the Lightning Network, such as efficient PTLCs (an improvement on HTLCs).
Script::
Bitcoin uses a scripting system for transactions called Script. Forth-like, it is simple, stack-based, and processed from left to right. It is purposefully not Turing-complete, with no loops.
Script, Bitcoin Script::
Bitcoin uses a scripting system for transactions called Script. Resembling the Forth programming language, it is simple, stack-based, and processed from left to right. It is purposefully Turing-incomplete, without loops or recursion.
ScriptPubKey (aka pubkey script)::
ScriptPubKey or pubkey script, is a script included in outputs which sets the conditions that must be fulfilled for those satoshis to be spent. Data for fulfilling the conditions can be provided in a signature script.
ScriptPubKey or pubkey script, is a script included in outputs which sets the conditions that must be fulfilled for those outputs to be spent. Data for fulfilling the conditions can be provided in a signature script. See also _ScriptSig_.
ScriptSig (aka signature script)::
ScriptSig or signature script is the data generated by a spender, which are almost always used as variables to satisfy a pubkey script.
secret key (aka private key)::
The secret number that unlocks bitcoin sent to the corresponding address. pass:[<span class="keep-together">A secret</span>] key looks like the following:
Segregated Witness is an upgrade to the Bitcoin protocol that adds a new witness for signatures and other transaction authorization proofs. This new witness field is exempt from the calculation of a new transaction ID, which solves most classes of 3rd party transaction malleability. Segregated Witness was deployed as a soft fork and is a change that technically makes Bitcoin’s protocol rules more restrictive.
Segregated Witness, Segwit::
Segregated Witness, or Segwit, is an upgrade to the Bitcoin protocol introduced in 2017 that adds a new witness for signatures and other transaction authorization proofs. This new witness field is exempt from the calculation of the transaction ID, which solves most classes of 3rd-party transaction malleability. Segregated Witness was deployed as a soft fork and is a change that technically makes Bitcoin’s protocol rules more restrictive.
SHA::
The Secure Hash Algorithm or SHA is a family of cryptographic hash functions published by the National Institute of Standards and Technology (NIST). The Bitcoin protocol currently uses SHA256.
The Secure Hash Algorithm or SHA is a family of cryptographic hash functions published by the National Institute of Standards and Technology (NIST). The Bitcoin protocol currently uses SHA-256, which produces a 256-bit hash.
short channel id (scid)::
Once a channel is established, the index of the funding transaction on the blockchain is used as the short channel id to uniquely identify the channel.
The short channel id consists of 8 bytes referring to 3 numbers.
In its serialized form it depicts these 3 numbers as decimal values separated by the letter **x**.
In its serialized form it depicts these 3 numbers as decimal values separated by the letter **x** (eg. +600123x01x00+)
The first number (4 bytes) is the block height.
The second number (2 bytes) is the index of the funding transaction with the blocks.
The last number (2 bytes) is the transaction output.
simplified payment verification (SPV)::
SPV or simplified payment verification is a method for verifying particular transactions were included in a block without downloading the entire block. The method is used by some lightweight Bitcoin clients.
Simplified Payment Verification (SPV)::
SPV or simplified payment verification is a method for verifying particular transactions were included in a block without downloading the entire block. The method is used by some lightweight Bitcoin and Lightning wallets.
source-based routing::
On the Lightning Network, the sender of a payment decides the route of the payment.
@ -466,7 +426,7 @@ source-based routing::
Source-based routing is fundamentally different to how routing works on the Internet Protocol.
soft fork::
Soft fork, or Soft-Forking Change, is a protocol upgrade that's forwards+backwards compatible so it allows old nodes and new nodes to both continue using the same chain.
Soft fork, or Soft-Forking Change, is a protocol upgrade that's forwards & backwards compatible so it allows both old nodes and new nodes to continue using the same chain.
SPHINX Mix Format::
A particular technique for Onion Routing used in the Lightning Network and invented by George Danezis and Ian Goldberg in 2009.
@ -474,41 +434,37 @@ SPHINX Mix Format::
While the privacy of the sender and receiver of the payment is protected, each node is still able to return an error message along the path to the originator of the message.
The paper can be found at https://cypherpunks.ca/~iang/pubs/Sphinx_Oakland09.pdf
Submarine Swap::
A Submarine Swaps enables transfers between on-chain Bitcoin addresses and off-chain locations, like the Lightning Network. Just as standard LN transfers chain payments by means of HTLCs that make the final claim on funds conditional on the recipient revealing a secret to all links in the chain, Submarine Swaps use the same logic and procedure to transfer funds across the on-chain/off-chain barrier with minimal trust. Reverse Submarine Swaps allow bitcoin transfers in the opposite direction, from an off-chain LN node to an on-chain address.
submarine swap::
A submarine swap is a trustless atomic swap between on-chain Bitcoin addresses and off-chain Lightning Network payments. Just as LN payments use HTLCs that make the final claim on funds conditional on the recipient revealing a secret (hash preimage), submarine swaps use the same mechanism to transfer funds across the on-chain/off-chain barrier with minimal trust. Reverse submarine swaps allow swaps in the opposite direction, from an off-chain LN payment to an on-chain Bitcoin address.
timelock::
A timelock is a type of encumbrance that restricts the spending of some bitcoin until a specified future time or block height. Timelocks feature prominently in many Bitcoin contracts, including payment channels and Hashed Timelock Contracts.
A timelock is a type of encumbrance that restricts the spending of some bitcoin until a specified future time or block height. Timelocks feature prominently in many Bitcoin contracts, including payment channels and Hash Time-Locked Contracts.
transaction::
Transactions are a binary format used by the Bitcoin protocol to transfer bitcoin from one address to another.
Several transactions are built into a block which has to be confirmed by the Bitcoin network through the process of mining.
Transactions can only be included in a block if they contain a valid signature (more precisely a valid input script) matching the output script defined by the previous owner.
The first transaction in each block is called the coinbase and generates new bitcoin.
Transactions can also contain contracts and should not be confused with payments.
Transactions are data structures used by Bitcoin to transfer bitcoin from one address to another.
Several thousand transactions are aggregated in a block, which is then recorded (mined) on the blockchain.
The first transaction in each block is called the coinbase transaction and generates new bitcoin.
transaction malleability::
Transaction malleability is a property that the hash of a transaction can change without changing the semantic of the transaction (the UTXOs it is spending, the destinations and the corresponding amounts).
For example, altering the signature can change the hash of a transaction, because of the non-deterministism of ECDSA signing.
Transaction malleability is a property that the hash of a transaction can change without changing the semantics of the transaction.
For example, altering the signature can change the hash of a transaction.
A commitment transaction needs the hash of a funding transaction and if the hash of the funding transaction changes, transactions depending on it will become invalid. This will make users unable to claim the refunds if there are any.
The Segregated Witness soft fork addresses this issue and is therefore an important upgrade to support Lightning Network.
The Segregated Witness soft fork addresses this issue and was therefore an important upgrade to support the Lightning Network.
transport layer::
In computer networking, the transport layer is a conceptual division of the methods used by computers (and ultimately applications) to talk to each other.
The transport layer provides communication services between computers such as flow control, verification, and multiplexing (to allow multiple applications to work on a computer at the same time).
unspent transaction output (UTXO)::
Unspent Transaction Output (UTXO)::
See _output_
wallet::
A wallet is a piece of software that holds Bitcoin addresses and secret keys. It is used to send, receive, and store bitcoin. In the context of the Lightning Network it should also hold revocation secrets of old channels state and the latest presigned commitment transactions.
A wallet is a piece of software that holds Bitcoin private keys. It is used to create and sign Bitcoin transactions. In the context of the Lightning Network it also holds revocation secrets of old channel state and the latest pre-signed commitment transactions.
watchtower::
Watchtowers are a security service on the Lightning network that monitor channels.
Watchtowers are a security service on the Lightning network that monitors payment channels for potential protocol breaches.
In the case that one of the channel partners goes offline or loses their backup, a watchtower keeps their own backups and can restore their channel information.
They also monitor the Bitcoin blockchain and can submit a penalty transaction in the case that one of the partners tries to "cheat" by broadcasting an outdated state.
Watchtowers can be run by the channel partners themselves, or as a paid service offered by a third party.
Watchtowers have no control over the funds in the channels themselves.
Some contributed definitions have been sourced under a CC-BY license from the https://en.bitcoin.it/wiki/Main_Page[Bitcoin Wiki], https://en.wikipedia.org[Wikipedia], https://github.com/bitcoinbook/bitconbook[Mastering Bitcoin] or from other open source documentation sources.
+
Watchtowers can be run by the channel partners themselves, or as a paid service offered by a third party. Watchtowers have no control over the funds in the channels themselves.
Some contributed definitions have been sourced under a CC-BY license from the https://en.bitcoin.it/wiki/Main_Page[Bitcoin Wiki], https://en.wikipedia.org[Wikipedia], https://github.com/bitcoinbook/bitconbook[Mastering Bitcoin] or from other open source publications.
The Lightning Network (LN) is a second-layer peer-to-peer network that allows us to make Bitcoin payments "off-chain", meaning without committing them as transactions to the Bitcoin blockchain.
The Lightning Network (LN) is a second-layer peer-to-peer network that allows us to make Bitcoin payments "off-chain," meaning without committing them as transactions to the Bitcoin blockchain.
LN gives us Bitcoin payments that are secure, cheap, fast, and much more private, even for very small payments.
LN gives us Bitcoin payments that are secure, cheap, fast, and much more private, even for very small payments.
Building on the idea of payment channels, first proposed by Bitcoin's inventor Satoshi Nakamoto, the Lightning Network is a routed network of payment channels where payments "hop" across a path of payment channels from the sender to the recipient.
The initial idea of the Lightning Network was proposed in 2015 in the groundbreaking paper _""The Bitcoin Lightning Network: Scalable Off-Chain Instant Payments"_ by Joseph Poon and Thaddeus Dryja. By 2017 there was a "test" Lightning Network running on the internet, as different groups build compatible implementations and coordinated to set some interoperability standards. In 2018 the Lightning Network went "live" and payments started flowing.
The initial idea of the Lightning Network was proposed in 2015 in the groundbreaking paper "The Bitcoin Lightning Network: Scalable Off-Chain Instant Payments", by Joseph Poon and Thaddeus Dryja. By 2017, there was a "test" Lightning Network running on the internet, as different groups built compatible implementations and coordinated to set some interoperability standards. In 2018, the Lightning Network went "live" and payments started flowing.
In 2019 Andreas M. Antonopoulos, Olaoluwa Osuntokun, and René Pickhardt agreed to collaborate to write this book. It appears we have been successful!
In 2019, Andreas M. Antonopoulos, Olaoluwa Osuntokun, and René Pickhardt agreed to collaborate to write this book. It appears we have been successful!
=== How to Use This Book
@ -164,8 +164,8 @@ Or you can support his work directly with Bitcoin (also via the Lightning Networ
==== Contacting Olaoluwa Osuntokun
You can contact Olaoluwa Osuntokun at his proffesional email address:
link:$$laolu@lightning.engineering$$[]
You can contact Olaoluwa Osuntokun at his professional email address:
__laolu@lightning.engineering__
Follow Olaoluwa on Twitter:
link:$$https://twitter.com/roasbeef$$[]
@ -212,4 +212,6 @@ Thank you.
=== Sources
Some of the material in this book has been sourced from a variety of public-domain sources, open-licensed sources, or by permission. See <<sources_licenses>> for source, license and attribution details.
Some of the material in this book has been sourced from a variety of public domain sources, open license sources, or with permission. See <<sources_licenses>> for source, license, and attribution details.
Andreas M. Antonopoulos
3 years ago