@ -10,7 +10,7 @@ Today, in 2021, the Lightning Network is still in its infancy. The Lightning Net
The concept of the Lightning Network was proposed in 2015, and the first implementation was launched in 2018. As of 2021, we're only beginning to see the opportunities the Lightning Network provides to Bitcoin including improved privacy, speed, and scale.
With core knowledge of the Lightning Network, you can help shape the future of the network while also building opportunities for yourself.
We assume you already have some basic knowledge about Bitcoin, but if not, don't worry – we will explain the most important Bitcoin concepts, those you must know to understand the Lightning Network, in <<bitcoin_fundamentals>>. If you want to learn more about Bitcoin, you can read _Mastering Bitcoin_, available for free online.
We assume you already have some basic knowledge about Bitcoin, but if not, don't worry – we will explain the most important Bitcoin concepts, those you must know to understand the Lightning Network, in <<bitcoin_fundamentals_review>>. If you want to learn more about Bitcoin, you can read _Mastering Bitcoin_, available for free online.
While the bulk of this book is written for programmers, the first few chapters are written to be approachable by anyone regardless of technical experience. In this chapter, we'll start with some terminology, then move to looking at trust and its application in these systems, and finally we'll discuss the history and future of the Lightning Network. Let's get started.
@ -51,7 +51,7 @@ Now that you're familiar with these basic terms, let's move to a concept you are
You will often hear people calling Bitcoin and the Lightning Network "trustless". At first glance this is confusing. After all, isn't trust a good thing? Banks even use it in their names! Isn't a "trustless" system, a system devoid of trust, a bad thing?
The use of the the word "trustless" is intended to convey the ability to operate without _needing_ trust in the other participants in the system. In a decentralized system like Bitcoin, you can always choose to transact with someone you trust. However, the system ensures you can't be cheated even if you can't trust the other party in a transaction. Trust is a nice-to-have instead of a must-have property of the system.
The use of the word "trustless" is intended to convey the ability to operate without _needing_ trust in the other participants in the system. In a decentralized system like Bitcoin, you can always choose to transact with someone you trust. However, the system ensures you can't be cheated even if you can't trust the other party in a transaction. Trust is a nice-to-have instead of a must-have property of the system.
Contrast that to traditional systems like banking where you _must_ place your trust in a third party, since it controls your money. If the bank violates your trust, you may be able to find some recourse from a regulator or court, but at an enormous cost of time, money, and effort.
@ -80,7 +80,7 @@ Cryptographic systems like Bitcoin and the Lightning Network are systems that al
The big distinction between a cryptographic system like this and a traditional financial system is that in traditional finance you have a _trusted third party_, for example a bank, to ensure that outcomes are fair. A significant problem with such systems is that they give too much power to the third party, and they are also vulnerable to a _single point of failure_. If the trusted third party itself violates trust or attempts to cheat, the basis of trust breaks.
As you study cryptographic systems, you will notice a certain pattern: instead of relying on a trusted third party, these systems attempt to prevent unfair outcomes by using a system of incentives and disincentives. In cryptographic systems you place trust in the _protocol_, which is effectively a system with a set of rules that, if properly designed, will correctly apply the desired incentives and disincentives. The advantage of this approach is two-fold. Not only do you avoid trusting a third party, you also reduce the need to enforce fair outcomes. So long as the participants follow the agreed protocol and stay within the system, the incentive mechanism in that protocol achieves fair outcomes without enforcement.
As you study cryptographic systems, you will notice a certain pattern: instead of relying on a trusted third party, these systems attempt to prevent unfair outcomes by using a system of incentives and disincentives. In cryptographic systems you place trust in the _protocol_, which is effectively a system with a set of rules that, if properly designed, will correctly apply the desired incentives and disincentives. The advantage of this approach is two-fold: Not only do you avoid trusting a third party, you also reduce the need to enforce fair outcomes. So long as the participants follow the agreed protocol and stay within the system, the incentive mechanism in that protocol achieves fair outcomes without enforcement.
The use of incentives and disincentives to achieve fair outcomes is one aspect of a branch of mathematics called _game theory_, which studies "models of strategic interaction among rational decision makers" footnote:[Wikipedia "Game Theory": https://en.wikipedia.org/wiki/Game_theory]. Cryptographic systems that control financial interactions between participants, such as Bitcoin and the Lightning Network, rely heavily on game theory to prevent participants from cheating and allow participants who don't trust each other to achieve fair outcomes.
@ -55,7 +55,7 @@ While your funds on the Lightning Network are called to be "off-chain" they are
One thing about the Lightning Network is however messing with this understanding of ownership.
On the Lightning Network there exist several presigned Transactions which allocate some of the bitcoin to you and some to your channel partner.
Without the the ability to sequence transaction and invalidate old ones via the penalty based revocation mechanism (or other techniques) we could not define clear ownership as the funds would be distributed among its two owners according to which ever transaction would first be broadcasted and successfully mined.
Without the ability to sequence transaction and invalidate old ones via the penalty based revocation mechanism (or other techniques) we could not define clear ownership as the funds would be distributed among its two owners according to which ever transaction would first be broadcasted and successfully mined.
If the last paragraph of this summary was confusing: No worries! We are getting there now!
@ -129,7 +129,7 @@ The `revoke_and_ack` Lightning message contains three data fields.
* [`point`:`next_per_commitment_point`]
While it is really simple and straight forward it is very crucial.
Bob shares the the `per_commitment_secret` of the old commitment transaction which serves as the revocation key and would allow Alice in future to penalize Bob if he publishes the old commitment transaction without the HTLC output.
Bob shares the `per_commitment_secret` of the old commitment transaction which serves as the revocation key and would allow Alice in future to penalize Bob if he publishes the old commitment transaction without the HTLC output.
As in a future Alice and Bob might want to negotiate additional commitment transactions he already shares back the `next_per_commitment_point` that he will use in his next commitment transaction.
Alice checks that the `per_commitment_secret` produces the last `per_commitment_point` and constructs her new commitment transaction with the HTLC output.
@ -17,7 +17,7 @@ A key requirement for a second layer protocol such as Lightning (which will be d
The `nSequence` field was intended to allow users to transmit updated versions of a transaction to the network, changing the outputs of a transaction, effectively creating a payment channel.
Such a payment channel would then be valid as long as the transaction was not mined.
According to a mailing list post in 2013, by early Bitcoin developer Mike Hearn, Satoshi Nakamoto envisioned this construct for high frequency trading.footnote:HearnBitcoinDev[Mike Hearn on Bitcoin-dev - April 16th 2013 - Anti DoS for tx replacement http://web.archive.org/web/20190501234813/https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2013-April/002417.html.]
According to a mailing list post in 2013 by early Bitcoin developer Mike Hearn, Satoshi Nakamoto envisioned this construct for high frequency trading.footnote:HearnBitcoinDev[Mike Hearn on Bitcoin-dev - April 16th 2013 - Anti DoS for tx replacement http://web.archive.org/web/20190501234813/https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2013-April/002417.html.]
However, there were some weaknesses in this initial formulation that limited its potential. Firstly, the payment channel would only be open until the transaction was mined in a block, either limiting the duration of the payment channel or handing control of the payment channel to the miners. Secondly, there was no economic incentive for miners to respect the `nSequence` number (to replace the state of the channel in the mempool), reducing the utility of this mechanism.
@ -25,9 +25,9 @@ The Revocable Sequence Maturity Contracts (RSMC), which formed payment channels
Using this new feature of the `nSequence` number, the RSMC was able to enforce a novel penalty mechanism: each time a party goes to claim a state on chain (which may or may not be the latest), the other party is given a period of time to challenge that claim (enforced by the `nSequence` number of `OP_CHECKSEQUENCEVERIFY`).
If a participant of the channel is able to present evidence to the blockchain of an intent to claim a revoked state, then the honest party s rewarded with all the funds of the cheating party.
If a participant of the channel is able to present evidence to the blockchain of an intent to claim a revoked state, then the honest party is rewarded with all the funds of the cheating party.
This enforcement of prior revoked states (only the latest channel state is valid) created a very strong economic incentive via the penalty mechanism: if a party attempts to renegade a signed contract, they lose all their funds!
This enforcement of prior revoked states (only the latest channel state is valid) created a very strong economic incentive via the penalty mechanism: if a party attempts to renege a signed contract, they lose all their funds!
This "challenge period" enforced by the `nSequence` was one of the novel contributions of the RSMC to the payment channel design space: the blockchain was used as a court wherein disputes could be handled objectively.
@ -50,13 +50,13 @@ Alice cannot effectively broadcast the refund transaction to be mined in a block
Therefore, Bob can securely receive more updates to the channel balance as long as it remains `open`.
This mechanism would allow two users to engage in several smaller transactions which all happened outside of the Bitcoin network.
While this construction of the unidirectional payment channel would have solved the custody problem of exchanges it has never been widely implemented.
While this construction of the unidirectional payment channel would have solved the custody problem of exchanges, it has never been widely implemented.
A fundamental blocker to deploying payment channels in production was the issue of transaction malleability.
In the worst case, this could be exploited by an attacker to prevent a party from closing the channel on-chain as their settlement transaction had a chance to become invalidated if the funding transaction has it's transaction ID inadvertently changed.
Also as a payment channel this system was not too useful as the channel could only at total send the total amount of provided bitcoin in the funding transaction.
Also, as a payment channel this system was not too useful because the channel could only at total send the total amount of provided bitcoin in the funding transaction.
Once the timelock was over or all bitcoin were sent to B the channel would have to be closed.
The obvious idea of opening two channels one from A to B and one from B to A would not have helped as each of those channels would have to be closed and reestablished once it ran dry.
The core breakthrough for the Lightning Network to become a reality was the ability to create payment channels which technically can live forever and can send money back and forth as often as the peers wish to in combination with routing payments among several channels.
@ -66,8 +66,8 @@ The core breakthrough for the Lightning Network to become a reality was the abil
You can watch a video explaining the construction and operation of unidirectional payment channels online at: https://youtu.be/AcP3czefanM
====
Surprisingly both properties took quite some while until the community figured them out.
Technically speaking the unidirectional payment channel has all the important ingredients (funding transaction to a 2-2 multisignature wallet, a transaction spending from the wallet encoding the balance, a timelock to allow refunding if the other side becomes unresponsive, off chain communication and the fact that no additional trust other than the one in the bitcoin network) of modern payment channels which are used in the Lightning Network.
Surprisingly, both properties took quite some while until the community figured them out.
Technically speaking, the unidirectional payment channel has all the important ingredients (funding transaction to a 2-2 multisignature wallet, a transaction spending from the wallet encoding the balance, a timelock to allow refunding if the other side becomes unresponsive, off chain communication and the fact that no additional trust, other than trust in the bitcoin network, is required) of modern payment channels which are used in the Lightning Network.
Despite being widely used in today's world, we will study the unidirectional payment channel in more depth in this book as it is an easy to understand educational example to approach the construction of today's payment channels.
This setup has one safety issue as transactions have been malleable without the segwit upgrade.
A problem that needed to be solved for any payment channel construction that we know up till today and which has been fixed in August 2017.
@ -85,13 +85,13 @@ The key difference was that a trusted party would have co-signed the spend of th
The Ultra Server was not able to steal bitcoin.
The next day, probably in response to Gavin's blogpost, Meni Rosenfeld started a discussion related to how these ideas could be combined.footnote:[Meni Rosenfeld on Bitcointalk - July 5th 2012 - Trustless, instant, off-the-chain Bitcoin payments http://web.archive.org/web/20190419103457/https://bitcointalk.org/index.php?topic=91732.0]
As Hashed Timelocked Contracts have neither been invented nor seen to solve the issue of trustless routing, Rosenfeld imagined trusted routing nodes.
Without mentioning the term network or routing of payments the idea of connecting payment channels and being able to send funds from anyone to anyone else even if there was no direct channel was born.
In Rosenfelds solution payment providers would be the ultraservers and they would settle the transactions among themselves based on trust.
It took us another 3 years until the Lightning Network whitepaper emerged which had solved all the bits and bolts necessary to get rid of the trust in Rosenfelds solution.
As Hashed Timelocked Contracts had neither been invented nor seen to solve the issue of trustless routing, Rosenfeld imagined trusted routing nodes.
Without mentioning the term network or routing of payments, the idea of connecting payment channels and being able to send funds from anyone to anyone else, even if there was no direct channel between them, was born.
In Rosenfeld's solution, payment providers would be the ultraservers and they would settle the transactions among themselves based on trust.
It took us another 3 years until the Lightning Network whitepaper emerged which had solved all the bits and bolts necessary to get rid of the trust in Rosenfeld's solution.
It was 2013 that Bitcoin developer Mike Hearn referred to Meni Rosenfelds proposal and suggesting to reactivate the `nSequence` field which Satoshi previously had deactivated.footnote:HearnBitcoinDev[]
Also Hearn referred to a section on the contracts article talking about the case of micropayment channels with the help of `nSequence`.
It was 2013 that Bitcoin developer Mike Hearn referred to Meni Rosenfeld's proposal and suggesting to reactivate the `nSequence` field which Satoshi previously had deactivated.footnote:HearnBitcoinDev[]
Also, Hearn referred to a section on the contracts article talking about the case of micropayment channels with the help of `nSequence`.
Submitting an older (outdated) commitment transaction is considered "cheating" (i.e. protocol breach) in the Lightning network and can be penalized by the other party claiming all the funds in the channel for themselves.
computationally easy::
A problem is considered to be computationally easy if there exists an algorithm that is able to compute the solution to the problem rather quickly.
A problem is considered to be computationally easy if there exists an algorithm that is able to compute the solution to the problem rather quickly (in polynomial time complexity).
computationally hard::
A problem is considered to be computationally hard if no algorithm exists or is known that is able to compute the solution to the problem rather quickly.
A problem is considered to be computationally hard if no algorithm exists or is known that is able to compute the solution to the problem rather quickly (in polynomial time complexity).
confirmations::
Once a transaction is included in a block, it has one confirmation. As soon as _another_ block is mined on the same blockchain, the transaction has two confirmations, and so on. Six or more confirmations are considered sufficient proof that a transaction cannot be reversed.
@ -146,7 +146,7 @@ Diffie Hellman Key Exchange::
It is an anonymous key agreement protocol that allows two parties, each having an elliptic-curve public-private key pair, to establish a shared secret over an insecure communication channel.
This shared secret may be directly used as a key, or to derive another key.
The key, or the derived key, can then be used to encrypt subsequent communications using a symmetric-key cipher.
An example of the derived key would be the shared secrete between the ephemeral session key of a sender of an onion with the nodes public key of a hop of the onion as described and used by the SPHINX Mix Format.
An example of the derived key would be the shared secret between the ephemeral session key of a sender of an onion with the nodes public key of a hop of the onion as described and used by the SPHINX Mix Format.
Via https://en.wikipedia.org/w/index.php?title=Elliptic-curve_Diffie%E2%80%93Hellman&oldid=836070673
digital signature::
@ -157,7 +157,7 @@ digital signature::
double-spending::
Double-spending is the result of successfully spending some money more than once.
Bitcoin protects against double-spending by verifying each transaction added to the blockchain plays by the rules that the inputs for the transaction have not previously already been spent.
Bitcoin protects against double-spending by verifying that each transaction added to the blockchain plays by the rules; this means checking that the inputs for the transaction have not previously already been spent.
The Revocable Sequence Maturity Contracts used to construct payment channels heavily attempt to double-spend bitcoin.
@ -8,11 +8,11 @@ Thus the sender has to be aware of the Channel Graph.
The channel graph is the interconnected set of publicly advertised channels (more on that later), and the nodes that these channels interlink.
As channels are backed by a funding transaction that is happening on chain one might falsely believe that Lightning Nodes could just extract the existing channels from the Bitcoin Blockchain.
However this only possible to a certain extend.
The Funding transactions are Pay to Whitness Script addresses and the nature of the script will only be revealed once the funding transaction output is spent.
The Funding transactions are Pay to Witness Script addresses and the nature of the script will only be revealed once the funding transaction output is spent.
Even if the nature of the script was known we emphasize that not all 2 - of - 2 multisignature scripts correspond to payment channels.
Since not even every Pay to Whitness Script address that we see in the Bitcoin Blockchain corresponds to a payment channel this approach is not helpful.
Since not even every Pay to Witness Script address that we see in the Bitcoin Blockchain corresponds to a payment channel this approach is not helpful.
There are actually more reasons why looking at the Bitcoin Blockchain might not be helpful.
For example on the Lightnign Network the Bitcoin keys that are used for signing are rotated by the nodes for every channel and update.
For example on the Lightning Network the Bitcoin keys that are used for signing are rotated by the nodes for every channel and update.
Thus even if we could reliably detect funding transactions on the Bitcoin Blockchain we would not know which two nodes on the Lightning Network own that particular channel.
Thus we could node create the Channel Graph that would be used to create candidate paths during the payment delivery via onion routing.
@ -20,11 +20,11 @@ The Lightning Network solves the afore mentioned problem by implementing a gossi
Gossip protocols are typical for peer 2 peer networks and are being used so that nodes can share information with other nodes without talking to those other nodes directly.
For that Lightning Nodes open encrypted peer 2 peer connections to each other and share novel information that they have received from other peers.
As soon as a node wants to share some information - for example about a newly created channel - it sends a message to all its peers.
Uppon receiving a message a node decides if the received message was novel and if so it will forward the information to its peers.
In this way if the peer 2 peer network is connected all new information that is necessary for the operation of the network will eventually be propagated to all other peers.
Upon receiving a message a node decides if the received message was novel and if so it will forward the information to its peers.
In this way if the peer 2 peer network is connected all new information that is necessary for the operation of the network will eventually be propagated to all other peers.
Obviously if a new peer initially wants to join the network it needs to know some other peers on the Network initially.
Only then the new peer could connect to them in order to learn about the existance of other peers.
Only then the new peer could connect to them in order to learn about the existence of other peers.
In this chapter, we'll explore exactly _how_ Lightning nodes discover each
other, and also the channel graph itself.
When most refer to the _network_ part
@ -43,8 +43,8 @@ It might be usefull to get familiar with the different terminology that we have
@ -22,12 +22,12 @@ In theory, one can examine the _channel graph_ and connect a series of payment c
In this section we will examine routing from the perspective of Dina, a gamer who receives donations from her fans while she livestreams her game sessions.
The innovation of routed payment channels allows Dina to receive tips without maintaining a separate channel with every one of her fans who want to tip her.
As long as there exists a path of well-funded channels from that viewer to Glori, she will be able to receive payment from that fan.
As long as there exists a path of well-funded channels from that viewer to Dina, she will be able to receive payment from that fan.
The nodes along the path from the fan to Dina are intermediaries and called "routing nodes" in the context of routing a payment.
[[dina-routing-diagram]]
.Any one of Dina's fans in the diagram can pay her by routing via the nodes in between them and Dina
image:images/dina-routing-diagram.PNG["Any one of Dina's fans in the diagram can pay her by routing via the nodes in between them and Dina"]
image:images/dina-routing-diagram.png["Any one of Dina's fans in the diagram can pay her by routing via the nodes in between them and Dina"]
Importantly, the routing nodes are unable to steal the funds while routing a payment from a fan to Dina.
Furthermore, routing nodes cannot lose money while participating in the routing process.
@ -129,7 +129,7 @@ We call this hash of the payment's secret the payment hash.
The secret which "unlocks" the payment is called the payment secret.
For now, we keep things simple and assume that Dina's secret is simply the text line: `+Dinas secret+`.
In order to "commit" to this secret, she computes the SHA256 hash which when encoded in hex, can be displayed as: `+f23c83babfb0e5f001c5030cf2a06626f8a940af939c1c35bd4526e90f9759f5+`.
In order to "commit" to this secret, she computes the SHA256 hash which when encoded in hex, can be displayed as: `+0575965b3b44be51e8057d551c4016d83cb1fba9ea8d6e986447ba33fe69f6b3+`.
footnote:[You can verify this by typing `echo -n "Dinas secret" | sha256sum` to your Linux command line shell.]
To facilitate Alice's payment, Dina will create the secret and the payment hash and send the payment hash to Alice. Alice doesn't know the secret but she can rewrite her contract to use the hash of the secret as a proof of payment:
@ -320,7 +320,7 @@ OP_CHECKSIGVERIFY
Alice can present this script to Bob in order to kick off the conditional payment.
For the chained aspect, Alice needs to be able to communicate the proper payment details to each hop in the route.
Recall that each hop will specify a forwarding fee rate, as well as other parameters that express their forwarding policy.
In addition to this forwarding rate, Alice also needs to be conceded about what time locks to use.
In addition to this forwarding rate, Alice also needs to be concerned about what time locks to use.
Each node in the hop needs some time to be able to settle the outgoing, then incoming payment on-chain in the worst case.
As a result, when constructing the final route, we need to give each node some buffer time, we call this before time, the "time lock delta".
Factoring in this time-lock delta, the time-lock of the outgoing HTLC will decrease as the route progresses, as the outgoing HTLC will expire before the incoming HTLC.
@ -343,7 +343,7 @@ With this bare foundation laid, the following questions may have come across you
In the network today, the sender is the one that selects the route and decides nearly all the details of the resulting route.
As for how path finding is done, there is no single approach that all nodes in the network use.
Instead, answer to the second question has a very large solution space, meaning there are several algorithms and neuritics used in the network today.
Instead, the answer to the second question has a very large solution space, meaning there are several algorithms and heuristics used in the network today.
Most commonly, a variation of Dijkstra's algorithm is used which takes into account additional Lightning Network details such as fees and time-locks.
Remember from earlier that a path turns into a route which is used to trigger a payment attempt.
As several conditions need to be satisfied for the HTLC to be completely extended, the sender may need to try several routes until one succeeds.
@ -390,7 +390,7 @@ Last but not least the Lightning Network can actually run on Tor to use it as a
This means that all connections of a node with its peers and the resulting communication will by obfuscated once more through the TOR network.
====
Lets stick to our example in which Alice still wants to tip Dina and has decided to use the path via Bob and Chan.
Let's stick to our example in which Alice still wants to tip Dina and has decided to use the path via Bob and Chan.
We note that there might have been alternative paths from Alice to Dina but for now we will just assume it is this path that Alice has decided to use.
In order to kick off the transfer, Alice needs to send a special message to Bob to kick off the multi-hop transfer.
You'll learn about the specific structure of this message in later chapters, but for now we'll call it an "HTLC Add" message.
@ -481,7 +481,7 @@ If this wasn't the case, and the onion packet shrank as it was being processed,
Since Dina is the final recipient of the payment, we only have 65 bytes worth of data to fill with actual content.
The remaining bytes are filled with random bytes to pad out the packet in an unpredictable manner.
Taking a step back, before Alice is able to prepare the remainder of the packet, we needs to generate an ephemeral key (a key only used once).
Taking a step back, before Alice is able to prepare the remainder of the packet, we need to generate an ephemeral key (a key only used once).
This ephemeral key is then used to generate a series of additional keys, which are themselves used for encryption, authentication, and also as input to a CSPRNG to deterministically generate the set of random filler bytes.
In the spirit of onion encryption, Alice will begin encrypting the payload from the last hop, adding a new layer of encryption with each new hop.
During processing, each node will authenticate the contents of the payload, then process the packet (decrypting it and shifting around some bytes) to prepare it for processing by the next node in the route.
Andreas M. Antonopoulos
3 years ago
