Payment channels are the core and most fundamental building block of the Lightning Network.
Of course, every detail of a technology is exists for a reason but the Lightning Network is literally built around the idea and concept of payment channels.
In the previous chapters you have already learned about payment channels, what properties they have, and on a high level how they work and how they can be constructed.
Of course, every detail of a technology exists for a reason and is important.
However the Lightning Network is literally built around the idea and concept of payment channels.
In the previous chapters you have already learned that payment channels
* allow two peers who created it to send and receive Bitcoin up to the amount specified by the capacity of the channel as often as they want to.
* split the capacity of the channel into a balance between the two peers which - as long as the channel is open is only known by the owners of the channel and increases privacy.
* do not require peers to do any additional onchain transactions other than the one needed to open and - potentially at a later state - to close the channel.
* can stay open for an arbitrary time. Potentially in the best case forever.
* do not require peers to trust each other as any attempt by a peer to cheat would enable the other peer to receive all the funds in the channel as a panality.
* can be connected to a network and allow peers to send money a long a path of connected channels without the necessity to trust the intermediary nodes as they have no ability to steal the Bitcoin that are being forwarded.
*
In this chapter we will dig deeper into the protocol details that are needed to open and close payment channels.
Working through this rather technical chapter you will be able to understand how the protocol design achieves the properties of payment channels.
Where necessary some information from the first chapters of this book will be repeated.
If you are new to the topic we highly encourage you to start there first.
If you however already know a fair share about bitcoin script, OP_CODES and protocol design it might be sufficient to skip the previous chapter and start here with us.
This books follows the construction of payment channels as described in BOLT 02 which is titled `peer protocol` and describes how two peers communicate to open, maintain and close a channel.
In this section we will only talk about maintaining and closing a channel.
The operation of a channel which means either making or forwarding a payment is discussed in our chapter about routing.
Also other constructions of payment channels are known and being discussed by the developers but as of writing only the channels as described in BOLT 2 are supported by the protocol and the implementations.
There will be one big difference though.
We will only discuss opening and closing a channel.
The operation and maintainance of a channel which means either making or forwarding a payment is discussed in our chapter about routing.
Also other constructions of payment channels are known and being discussed by the developers.
Historically speaking these are the Duplex Micropayment channels introduced by Christian Decker during his time as a PhD student at ETH Zuric and the eltoo channels which where also introduced by Christian Decker.
The eltoo channels are certainly a more elgant and cleaner why of achieving payment channels with the afore mentioned properties.
However they require the activation of BIP 118 and a softfork and are - at the time of writing - a potential future protocol change.
Thus this chapter will only focus on the pentalty based channels as described in the Lightning Network Whitepaper and specified in BOLT 02 which are currently supported by the protocol and the implementations.
[NOTE]
====
The Lightning Network does not need consensus of features across it's participants.
If the Bitcoin Softfork related to BIP 118 activates and people implement eltoo channels nodes that support eltoo can create payment channels and the onion routing of payments a long a path of channels would work just fine even if some of the channels are the modern eltoo channels or some channels are the legacy channels.
Actually when Lightning network connections are established nodes signal feature bits of global and local features that they support.
Thus havning the ability to create eltoo channels would just be an additional feature bit.
In this sense upgrading the Lightnign Network is much easier than upgrading Bitcoin where consensus among all stakeholders is needed.
====
To repeat what you should already know a payment channel is encoded as an unspent 2 out of 2 multisignature transaction output.
The capacity of the channel relates to the amount that is bound to the unspent 2 out of 2 multisignature transaction output.
It is opened wht the help of a funding transaction that sends bitcoin to a 2 out of 2 multisignature output together with a communication protocol that helps to initialize its state.
Let's quickly summarize what you should already know about payment channels on a technical level and for what you will learn the details in this chapter.
A payment channel is encoded as an unspent 2 - of - 2 multisignature transaction output.
The capacity of the channel relates to the amount that is bound to the unspent 2 - of - 2 multisignature transaction output.
It is opened with the help of a funding transaction that sends bitcoin to a 2 - of - 2 multisignature output together with a communication protocol that helps to initialize and maintain its state.
The balance of the channel encodes how the capacity is split between the two peers who maintain the channel.
Technically it is encoded by a the most recent pair of a sequence of pairs of similar (but not equal) presigned commitment transactions.
Each channel partner has both signatures for on of the commitment transactions from the sequence of pairs.
The split of the capacity is realized by a `to_local` and a `to_remote` output that is part of every commitment transaction
Technically the balance is encoded by a the most recent pair of a sequence of pairs of similar (but not equal) presigned commitment transactions.
These commitment transactions should never hit the blockchain and serve as a safty net for the participants in case the channel partner becomes unresponsive of disappears.
They are also the reason why the Lightning Network is called an offchain scaling solution.
Each channel partner has both signatures for one of the commitment transactions from the sequence of pairs.
The split of the capacity is realized by a `to_local` and a `to_remote` output that is part of every commitment transaction.
The `to_local` output goes to an address that is controlled by the peer that holds this fully signed commitment transaction.
`to_local` outputs, which also exist in the second stage HTLC transactions, have two spending conditions.
`to_local` outputs, which also exist in the second stage HTLC transactions - which we discuss in the routing chapter - have two spending conditions.
The `to_local` output can be spent either at any time with the help of a revocation secrete or after a timelock with the secret key that is controlled by the peer holding this commitment transaction.
The revocation secrete is necessary to economically disincentivice peers to publish previous commitment transactions.
Addresses and revokation secretes change with every new pair of commitment transactions that are being negotiated.
The Lightning Network as a protocol defines the communication protocols that are necessary to achieve this.
### Security of a Payment channel
While the BOLTs introduce payment channels directly with the opening protocol we have decided to talk about the security model first.
The security of payment channels come through a penalty based revocation system which help two parties to split the capacity of the payment channel into a balance sheet without the necessity to trust each other.
In this chapter we start from an insecure approach of creating a payment channel and explain why it is insecure.
We will then explain how time locks are being used to create revokable sequence maturity contracts that create the penality based revokation system which economically incentivizes people maintain the most recent state.
After you understood these concepts we will quickly walk you through the technical details of opening and closing a channel.
Any known payment channel constuction uses a 2 - of - 2 multisgnature output as the basis of the payment channel.
We call the amount that is attached to this output the capacity of the channel.
In every case Both channel partners hold 1 secret key of the multisignature address which means that they can only collaboratively controll the funds.
#### An example for a highly insecure payment channel construction
Let us assume Alice does not know the details about the Lightning Network and tries to naivly open a payment channel in a way that leads to her most likely loosing her funds.
Alice has heard that Payment channel are 2 - of - 2 multisnature outputs.
As she wants to have a channel with Bob and since she knows a public key from Bob she decides to open a channel by sending money to a 2 - of - 2 multisignature address that comes from Bob's and her key.
We call the Transaction that Alices used a **funding transaction** as it is supposed to fund the payment channel.
However signing and broadcasting this funding transaction would be a huge mistake.
As we have discussed the Bitcoins from the resulting UTXO can only be spent if Alice and Bob work together and both provide a signature for a transaction spending those coins.
If Bob would not respond to Alice in future Alice would have lost her Bitcoins forever.
That is because the coins would be stuck in the 2 - of - 2 multisignature address to which she has just sent them.
Luckily Alice has previously read Mastering Bitcoin and she knows all the properties of Bitcoin script and is aware of the risks that are involved to sending Bitcoins to a 2 - of - 2 multisignature address to which she does not controll both keys.
She is also aware of the "Don't trust. Verify" principle that Bitcoiners follow and doesn't want to trust Bob to help her moving or accessing her coins.
She would much more prefere to keep control over her coins even though they shall be stored in this 2 - of - 2 multisnature address.
While this seems like an impossible problem Alice has an idea:
What if she could already prepare a refund transaction (which we call commitment transaction in future) that sends all the bitcoin back to an address that she controlls?
Before broadcasting her funding transaction she already prepares and finnishes it so that she knows the transaction id.
She can now create the commitment transaction that spends the output of the funding transaction and ask Bob to provide a signature.
At that time Bob has nothing to loose by signing the commitment transaction.
He did not have Coins at the multisig address anyway.
Even if he did Alice intends to spend from an output which Bob never was involved in.
Thus at that point for Bob it is perfectly reasonable to sign the commitment transaction that spends the funding transaction.
On the other side you as a reader might think:
Why would Alice send money to a multisignature address just to prepare a transaction that sends the money back to her?
We really hope you have wondered about this because this is really the point where the innovation begins.
Just because in general people are expected to broadcast a transaction to the bitcoin network as soon as they have signed it noone forcses you to do that.
As Alice would loose access of her Bitcoins once she sends it to a 2 - of - 2 multisignature output for which she only controlls one key she needs to make sure that she will be able to regain access of her coins in case Bob becomes unresponisive.
Thus before Alice publishes the funding transaction she will create another transaction that sends all the bitcoin from the 2 - of - 2 multisignature output back to an address which she controlls.
.The situation can be seen in the following picture
image:channel-construction-opening-1.png[]
Of course for the commitment transaction Alice would need to get a Signature from Bob before she can safely broadcast the funding transaction.
After publishing the funding transaction instead of braodcasting the commitment transaction she will keep it at a save place.
For this to work Alice needs to be sure that the funding transaction could not be published with a different transaction id.
This malleability was possible before the Segwit upgrade of Bitcoin.
We will discuss the details later but didn't want to leave them out here.
[NOTE]
====
This entire process might be surprising (... comparison with HTTP server push and AJAX...)
====
Having Segwit and this first commitment transaction is actually secure for Alice.
We have seen the first of three main properties that commit transactions fulfill:
Commitment Transactions refund channel participants in case the other side becomes irresponsive.
The second purpose was implicitely defined by the first purpose:
Commitment Transactions split the capacity of the channel into a balance which is owned by each partner.
Initially this split means that all the capacity is naturally on the side of the partner who funded the channel.
Of course during the lifetime of the channel the balance could change.
For example Alice might want to send some funds to Bob.
This could happen because she wants to pay Bob or because she wants Bob to forward the funds through a path of channels to another merchant that she wants to pay.
Let us assume as an example that Alice wants to send 30k Satoshi to Bob.
For now we can assume that through some communication protocol Alice and Bob would negotiate a double spent of the funding transaction output of 100k satoshi.
The new commitment transaction for which Alice and Bob would exchange signatures would send 70k satoshi to Alice and 30k Satoshi to Bob.
.The situation can be seen in the following picture
image:channel-construction-opening-2.png[]
Whenever Alice and Bob want to change the balance of the payment channel they will negotiate a new commitment transaction.
Effectively they double spend the funding transaction output.
But as the commitment transactions are not broadcasted - as long as the channel stays open - they will be able to do that.
At this point we want to emphasize that the section was labeled in a way that suggests that this construction is insecure.
So the main question reads:
What can go wrong with the insecure payment channel?
The thing that goes and makes this construction insecure lies within the mechanics of Bitcoin.
The key innovation of Bitcoin was to prevent the double spending problem of electronic coins.
After Alice and Bob have exchanged signatures for the second commitment transaction Bob cannot rely on the fact that he really owns 30k satoshi.
Of course he could close the channel by publishing the second commitment transaction assigning 30k satoshi to an address that he controlls.
But similarly Alice could broadcast the first commitment transaction and transfer the entire capacity of the channel back to an address that she controlls.
As Bitcoin prevents double spending of the funding transaction miners will include only one of the two commitment transactions.
Thus we need to adapt the idea with the commitment transactions to create the ability to revoke an old commitment transaction.
Regarding the fact that Bob and Alice Both have a copy of the transaction and that Bob cannot controll the Data that Alice has stored on her hardware it seems pretty hopeless.
Luckily the scripting language in Bitcoin allows at least for changing commitment transactions in a way that econimically disencentivices channel partners to publish and outdated balances after they have negotated a new balance.
#### Secure Payment channels via Revokable Commitment transactions
[NOTE]
====
In summary we can conclude that commitment transactions fulfill three purposes:
1. refund channel participants in case the other side becomes irresponsive
2. Splite the capacity of the channel into the current balance that peers have agreed upon.
3. Allow revocation of old state through the means of a penality via a revocable sequence maturity contract.
====
We have not yet explained how channel partners actually communicate to negotiate a new balance.
Because it seems pretty amazing that we can make this swap revocation secret for signature atomic.
In order to understand this we first need to understand the general communication of how a channel is opened.
The actual negotiation of the new state is also done with HTLCs.
That is why we only explain this in the routing chapter and ask you to stay patient.
[NOTE]
====
*TODO: Move this note to routing chapter?*
HTLCS fullfill the following purposes:
1. Make a conditional payment.
2. Help to update a new balance in a channel
3. Make payments through a path of channel atomic, meaning that peers along the path cannot steal funds.
====
### Opening a payment channel
Currently payment channels can only be opened by one side.
We call the process of creating a new payment channel "opening a payment channel".
Currently a payment channel can only exists between exactly two peers.
Therefore you might be surprised to learn that even though two users are owning and maintaining the channel the current construction requires only one user to open the channel.
This does not mean that only one peer is needed to open a channel.
It means that only one peer - namingly the one who opens the channel - provides the funds and capacity for the channel.
Let us assume for the remainder of the section that Alice wants to open a channel with Bob.
Opening a payment channel is not as easy as sending bitcoins to a 2 out of 2 multisignature output.
In a fully functional payment channel the bitcoins are being sent to a 2 out of 2 multisignature address to which each owner controlls a key.
Thus Alice needs to know the public key of Bob which will be part of the 2 out of 2 multisignature address.
She will do that by sending Bob and `open_channel` message signaling her interest to open a channel.
It does however mean that the user who opens the channel also has to provide the bitcoins to fund the channel.
Let us stick to our example where Alice opens a channel with Bob with a capacity of 100k satoshi.
This means that Alice provides 100k satoshi.
Alice will do that by creating a so called funding transaction.
This transaction sends 100k satoshi from an address that she - or her lightning node software controlls - to a 2 - of - 2 multisig address for which she and Bob know 1 secret key each.
The amount of Bitcoin that is sent to the multisig output by Alice is called the capacity of the payment channel.
Thus for the reminder of the chapter in all examples we assume the payment channels that we use as examples already magically exist and the two peers Alice and Bob already have all the necessary data at hand.
[NOTE]
====
Even though Alice and Bob both have a public node key to which they own the private secret opening a payment channel is not as easy as sending bitcoins to the 2 out of 2 multisignature output that belongs to the public keys of Alice and Bob.
Let us assume for a moment that Alice would send 100k Satoshi to the Multisig address resulting from hers and Bob's public node id.
In that case Alice will never be able to maintain her funds back without the help of Bob.
Of course we want our payment channels to work in a way that Alice does not need to trust Bob.
Bob could however refuse to sign a transaction that sends all those outputs back to an address that is controlled by Alice.
He would be able to blackmail Alice to assign a significant amount of those Bitcoin to an output address that is controlled by him.
Thus Bob can't steel the coins from Alice directly but he can threten Alice to have her coins lost forever.
This example shows that unfortunatelly opening a channel will be a little bit more complex than just sending Bitcoins to a multisignature address.
====
[NOTE]
====
@ -43,10 +210,45 @@ The importance of the segwit upgrade.
In order to avoid the reuse of addresses Alice and Bob will generate a new set of keys for the multisig address that they use to open the channel.
Alice needs to inform Bob which key she intends to use for their channel and ask him which key he intends to use.
She will do that by sending Bob and `open_channel` message signaling her interest to open a channel.
This message contains a lot of additional data fields.
Most of them specify meta data which is necessary for the channel operation and can be be safely ignored for now.
With the `chain_hash` Alice signals that she intends to open the channel on the Bitcoin blockchain.
While the Lightning Network was certainly invented to scale the amount of payments that can be conducted on the Bitcoin Network it is interesting to note that the Network is designed in a way that allows to build channels over various currencies.
If a node has channels with more than one currency it is even possible to route payments through multi asset channels.
However this turns out to be a little bit tricky in reality as the exchange rate between currencies might change which might lead the forwarding node to wait for a better ex hange rate to settle or to abort the payment process.
For the opening process the final channel id cannot be determined yet thus Alice needs to select a random channel id which Bob and she can use to identify the messages for this channel during the opening phase.
This design descision allows multiple channels to exist between two nodes - though currently only LND supports this feature.
Alice tells Bob for how many satoshis she wishes to open the channel.
This information is necessary to construct the commitment transaction ...
Once the channel is open Alice will be able to send 99k satoshi along this channel.
Bob on the other side will be able to receive 99k satoshi along that channel.
This means that initially Alice will not be able to recieve Bitcoins on this channel and that Bob initially will not be able to send Bitcoin along that channel.
[NOTE]
====
The current construction could be generalized to multiparty channels and channel factories.
However the communication protocol would suffer from increased complexity.
====
Chapter overview:
* describes how channels are put together at the script+transaction level
* details how a channel if funded in the protocol
* details how a channel is updated in the protocol
** including Key derrivation!
* details how a channel is updated in the protocol (moved to routing!)
* describes what needs to happen when a channel is force closed
@ -19,24 +19,24 @@ In this chapter you will learn how to set up each of the software packages for t
==== Using the command-line
The examples in this chapter, and more broadly in most of this book, use a command-line terminal. That means that you type commands into a terminal and receive text responses. Furthermore, the examples are demonstrated on an operating system based on the Linux kernel and GNU software system, specifically the latest long-term stable release of Ubuntu (Ubuntu 18.04 LTS). The majority of the examples can be replicated on other operating systems such as Windows or Mac OS, with small modifications to the commands. The biggest difference between operating systems is the _package manager_ which installs the various software libraries and their pre-requisites. In the given examples, we will use +apt+, which is the package manager for Ubuntu. On Mac OS, a common package manager used for open source development is Homebrew (command +brew+) found at https://brew.sh.
The examples in this chapter, and more broadly in most of this book, use a command-line terminal. That means that you type commands into a terminal and receive text responses. Furthermore, the examples are demonstrated on an operating system based on the Linux kernel and GNU software system, specifically the latest long-term stable release of Ubuntu (Ubuntu 20.04 LTS). The majority of the examples can be replicated on other operating systems such as Windows or Mac OS, with small modifications to the commands. The biggest difference between operating systems is the _package manager_ which installs the various software libraries and their pre-requisites. In the given examples, we will use +apt+, which is the package manager for Ubuntu. On Mac OS, a common package manager used for open source development is Homebrew (command +brew+) found at https://brew.sh.
In most of the examples here, we will be building the software directly from the source code. While this can be quite challenging, it gives us the most power and control. You may choose to use docker containers, pre-compiled packages or other installation mechanisms instead if you get stuck!
[TIP]
====
((("$ symbol")))((("shell commands")))((("terminal applications")))In many of the examples in this chapter we will be using the operating system's command-line interface (also known as a "shell"), accessed via a "terminal" application. The shell will display a prompt; you type a command; and the shell responds with some text and a new prompt for your next command. The prompt may look different on your system, but in the following examples it is denoted by a +$+ symbol. In the examples, when you see text after a +$+ symbol, don't type the +$+ symbol but type the command immediately following it, then press Enter to execute the command. In the examples, the lines below each command are the operating system's responses to that command. When you see the next +$+ prefix, you'll know it's a new command and you should repeat the process.
((("$ symbol")))((("shell commands")))((("terminal applications")))In many of the examples in this chapter we will be using the operating system's command-line interface (also known as a "shell"), accessed via a "terminal" application. The shell will first display a prompt as an indicator that it is ready for your command. Then you type a command and press "Enter" to which the shell responds with some text and a new prompt for your next command. The prompt may look different on your system, but in the following examples it is denoted by a +$+ symbol. In the examples, when you see text after a +$+ symbol, don't type the +$+ symbol but type the command immediately following it. Then press the Enter key to execute the command. In the examples, the lines below each command are the operating system's responses to that command. When you see the next +$+ prefix, you'll know it is a new command and you should repeat the process.
====
To keep things consistent, we use the +bash+ shell in all command-line examples. While other shells will behave in a similar way, and you will be able to run all the examples without it, some of the shell scripts are written specifically for the +bash+ shell and may require some changes or customization to run in another shell. For consistency, you can install the +bash+ shell on Windows and Mac OS, and it comes installed by default on most Linux systems.
To keep things consistent, we use the +bash+ shell in all command-line examples. While other shells will behave in a similar way, and you will be able to run all the examples without it, some of the shell scripts are written specifically for the +bash+ shell and may require some changes or customizations to run in another shell. For consistency, you can install the +bash+ shell on Windows and Mac OS, and it comes installed by default on most Linux systems.
==== Donwloading the book repository
All the code examples are available in the book's repository. The repository will be kept up-to-date, as much as possible, so you should always look for the latest version in the repository, instead of copying it from the printed book or ebook version of this test.
All the code examples are available in the book's online repository. Because the repository will be kept up-to-date as much as possible, you should always look for the latest version in the online repository, instead of copying it from the printed book or the ebook.
You can download the repository as a ZIP bundle by visiting +github.com/lnbook/lnbook+ and selecting the "Clone or Download" green button on the right.
You can download the repository as a ZIP bundle by visiting +github.com/lnbook/lnbook+ and selecting the green "Clone or Download" button on the right.
Alternatively, you can use the +git+ command, to create a version-controlled clone of the repository on your local computer. Git is a distributed version control system that is used by most developers to collaborate on software development and track changes to software repositories. Donwload and install +git+ by following the instructions on https://git-scm.com/.
Alternatively, you can use the +git+ command to create a version-controlled clone of the repository on your local computer. Git is a distributed version control system that is used by most developers to collaborate on software development and track changes to software repositories. Donwload and install +git+ by following the instructions on https://git-scm.com/.
To make a local copy of the repository on your computer, run the git command as follows:
@ -50,13 +50,13 @@ You now have a complete copy of the book repository in a folder called +lnbook+.
=== Docker Containers
Many developers use a _container_, which is a type of virtual machine, to install a pre-configured operating system and application with all the necessary dependencies. Much of the Lightning software can also be installed using a container system such as _Docker_ (command +docker+) found at https://docker.com. Container installations are a lot easier, especially for those who are not used to a command-line environment.
Many developers use a _container_, which is a type of virtual machine, to install a pre-configured operating system and applications with all the necessary dependencies. Much of the Lightning software can also be installed using a container system such as _Docker_ (command +docker+) found at https://docker.com. Container installations are a lot easier, especially for those who are not used to a command-line environment.
The book's repository contains a collection of docker containers that can be used to set up a consistent development environment to practice and replicate the examples on any system. Because the container is a complete operating system that runs with a consistent configuration, you can be sure that the examples will work on your computer and not worry about dependencies, library versions or differences in configuration.
The book's repository contains a collection of docker containers that can be used to set up a consistent development environment to practice and replicate the examples on any system. Because the container is a complete operating system that runs with a consistent configuration, you can be sure that the examples will work on your computer and need not worry about dependencies, library versions or differences in configuration.
Docker containers are often optimized to be small (less disk space). However, in this book we are using containers to _standardize_ the environment and make it consistent for all readers. Furthermore, these containers are not meant to be used to run services in the background. Instead, they are meant to be used to test the examples and learn by interacting with the software. For these reasons, the containers are quite large and come with a lot of development tools and utilities. Also, the containers are built on Ubuntu, instead of the Alpine distribution (more commonly used for Linux containers), as we want to work with a distribution that is familiar to many developers rather than one that is lightweight.
Docker containers are often optimized to be small, i.e. occupy the minimum disk space. However, in this book we are using containers to _standardize_ the environment and make it consistent for all readers. Furthermore, these containers are not meant to be used to run services in the background. Instead, they are meant to be used to test the examples and learn by interacting with the software. For these reasons, the containers are quite large and come with a lot of development tools and utilities. Commonly the Alpine distribution is used for Linux containers due to their reduced size. Nonetheless, we provide containers built on Ubuntu because more developers are familiar with Ubuntu, and this familiarity is more important to us than size.
You can find the latest container definitions and build configurations in the book's repository under the +code/docker+ folder. Each container is in a separate folder beneath:
You can find the latest container definitions and build configurations in the book's repository under the +code/docker+ folder. Each container is in a separate folder as can be seen below:
//// $ tree -F --charset=asciii code
[docker-dir-list]
@ -107,20 +107,20 @@ code
==== Installing Docker
Before we begin, you should install the docker container system on your computer. Docker is an open system that is distributed for free as a _Community Edition_, for many different operating systems including Windows, Mac OS and Linux. The Windows and Mac versions are called _Docker Desktop_, which is GUI desktop application and command-line tools, and the Linux version is called _Docker Engine_, which is a server daemon and command-line tools. We will be using the command-line tools, which are identical across all platforms.
Before we begin, you should install the docker container system on your computer. Docker is an open system that is distributed for free as a _Community Edition_ for many different operating systems including Windows, Mac OS and Linux. The Windows and Mac versions are called _Docker Desktop_ and consist of a GUI desktop application and command-line tools. The Linux version is called _Docker Engine_ and is comprised of a server daemon and command-line tools. We will be using the command-line tools, which are identical across all platforms.
Go ahead and install Docker for your operating system by following the instructions to _"Get Docker"_ from the Docker website found here:
https://docs.docker.com/get-docker/
Select your operating system from the list, and follow the instructions to install.
Select your operating system from the list and follow the installation instructions.
[TIP]
====
If you install on Linux, follow the post-installation instructions to ensure you can run Docker as a regular user instead of root. Otherwise, you will need to prefix the +docker+ command with +sudo+, running it as root like: +sudo docker+.
If you install on Linux, follow the post-installation instructions to ensure you can run Docker as a regular user instead of user _root_. Otherwise, you will need to prefix all +docker+ commands with +sudo+, running them as root like: +sudo docker+.
====
Once you have Docker installed, you can test your installation by running the demo container +hello-world+, like this:
Once you have Docker installed, you can test your installation by running the demo container +hello-world+ like this:
[docker-hello-world]
----
@ -134,7 +134,7 @@ This message shows that your installation appears to be working correctly.
==== Basic docker commands
In this chapter we use docker quite extensively. We will be using the following docker commands and arguments:
In this chapter we use +docker+ quite extensively. We will be using the following +docker+ commands and arguments:
*Building a container*
@ -158,11 +158,11 @@ docker run -it [--network netname] [--name cname] tag
docker exec cname command
----
...where +cname+ is the name we gave the container in the run command, and +command+ is an executable or script that we want to run inside the container.
...where +cname+ is the name we gave the container in the +run+ command, and +command+ is an executable or script that we want to run inside the container.
*Stopping a container*
In most cases, if we are running a container in an _interactive_ and _terminal_ mode, with the +i+ and +t+ flags (combined as +-it+), the container can be stopped by simply pressing +CTRL-C+, or exiting the shell with +exit+ or +CTRL-D+. If the container does not exit, you can stop it from another terminal, like this:
In most cases, if we are running a container in an _interactive_ as well as _terminal_ mode, i.e. with the +i+ and +t+ flags (combined as +-it+) set, the container can be stopped by simply pressing +CTRL-C+ or by exiting the shell with +exit+ or +CTRL-D+. If a container does not terminate, you can stop it from another terminal like this:
----
docker stop cname
@ -172,7 +172,7 @@ docker stop cname
*Deleting a container by name*
If you name a container, instead of letting docker name it randomly, you cannot use that name again until the container is deleted. Docker will return an error like this:
If you name a container instead of letting docker name it randomly, you cannot reuse that name until the container is deleted. Docker will return an error like this:
----
docker: Error response from daemon: Conflict. The container name "/bitcoind" is already in use...
----
@ -183,7 +183,7 @@ To fix this, delete the existing instance of the container:
docker rm cname
----
...where +cname+ is the name we have the container (+bitcoind+ in the example error message)
...where +cname+ is the name assigned to the container (+bitcoind+ in the example error message)
*List running containers*
@ -191,7 +191,7 @@ docker rm cname
docker ps
----
These basic docker commands will be enough to get you started and will allow you to run all the examples in this chapter. Let's see them in action, in the following sections.
These basic docker commands will be enough to get you started and will allow you to run all the examples in this chapter. Let's see them in action in the following sections.
=== Bitcoin Core and Regtest
@ -199,19 +199,19 @@ Most of the Lightning node implementations need access to a full Bitcoin node in
Installing a full Bitcoin node and synching the Bitcoin blockchain is outside the scope of this book and is a relatively complex endeavor in itself. If you want to try it, refer to _Mastering Bitcoin_ (https://github.com/bitcoinbook/bitcoinbook), "Chapter 3: Bitcoin Core: The Reference Implementation" which discusses the installation and operation of a Bitcoin node.
A Bitcoin node can also be operated in _regtest_ mode, where the node creates a local simulated Bitcoin blockchain for testing purposes. In the following examples, we will be using regtest mode to allow us to demonstrate lightning without having to synchronize a Bitcoin node, or risk any funds.
A Bitcoin node can be operated in _regtest_ mode, where the node creates a local simulated Bitcoin blockchain for testing purposes. In the following examples we will be using the +regtest+ mode to allow us to demonstrate Lightning without having to synchronize a Bitcoin node or risk any funds.
The container for Bitcoin Core is bitcoind that runs Bitcoin Core in regtest mode and mines a new block every 10 seconds. It's RPC port is exposed on port 18443 and accessible for RPC calls with the username regtest and the password regtest. You can also access it with an interactive shell and run +bitcoin-cli+ commands locally.
The container for Bitcoin Core is +bitcoind+. It is configured to run Bitcoin Core in +regtest+ mode and to mine a new block every 10 seconds. Its RPC port is exposed on port 18443 and accessible for RPC calls with the username +regtest+ and the password +regtest+. You can also access it with an interactive shell and run +bitcoin-cli+ commands locally.
===== Building the Bitcoin Core Container
Let's start by building and running the bitcoind container. First, we use the +docker build+ command to build it:
Let's start by building and running the +bitcoind+ container. First, we use the +docker build+ command to build it:
Next, let's run the bitcoind container and have it mine some blocks. We use the +docker run+ command, with the flags for _interactive (i)_ and _terminal (t)_, and the +name+ argument to give the running container a custom name:
Next, let's run the +bitcoind+ container and have it mine some blocks. We use the +docker run+ command, with the flags for _interactive (i)_ and _terminal (t)_, and the +name+ argument to give the running container a custom name:
----
$ docker run -it --name bitcoind lnbook/bitcoind
@ -252,13 +252,13 @@ Mining 1 block every 10 seconds
Balance: 100.00000000
----
As you can see, bitcoind starts up and mines 101 blocks to get the chain started. This is because under the bitcoin consensus rules, newly mined bitcoin is not spendable until 100 blocks have elapsed. By mining 101 blocks, we make the 1st block's coinbase spendable. After that initial mining activity, we mine a new block every 10 seconds, to keep the chain moving forward.
As you can see, bitcoind starts up and mines 101 simulated blocks to get the chain started. This is because under the bitcoin consensus rules, newly mined bitcoin is not spendable until 100 blocks have elapsed. By mining 101 blocks, we make the first block's coinbase spendable. After that initial mining activity, a new block is mined every 10 seconds to keep the chain moving forward.
For now, there are no transactions. But we now have some test bitcoin that has been mined in the wallet and is available to spend. When we connect some Lightning nodes to this chain, we will send some bitcoin to their wallets so that we can open some Lightning channels between the Lightning nodes.
For now, there are no transactions. But we have some test bitcoin that has been mined in the wallet and is available to spend. When we connect some Lightning nodes to this chain, we will send some bitcoin to their wallets so that we can open some Lightning channels between the Lightning nodes.
===== Interacting with the Bitcoin Core Container
In the mean time, we can also interact with the bitcoind container by sending it shell commands. The container is sending a log file to the terminal, displaying the mining process of the bitcoind process. To interact with the shell we can issue commands in another terminal, using the +docker exec+ command. Since we previously named the running container with the +name+ argument, we can refer to it with that name when we run the +docker exec+ command. First, let's run an interactive +bash+ shell:
In the mean time, we can also interact with the +bitcoind+ container by sending it shell commands. The container is sending a log file to the terminal, displaying the mining process of the bitcoind process. To interact with the shell we can issue commands in another terminal, using the +docker exec+ command. Since we previously named the running container with the +name+ argument, we can refer to it by that name when we run the +docker exec+ command. First, let's run an interactive +bash+ shell:
----
$ docker exec -it bitcoind /bin/bash
@ -272,9 +272,9 @@ root@e027fd56e31a:/bitcoind# ps x
root@e027fd56e31a:/bitcoind#
----
Running the interactive shell puts us "inside" the container and logged in as the +root+ user, as we can see from the new shell prompt +root@e027fd56e31a:/bitcoind#+. If we issue the +ps x+ command to see what processes are running, we see both bitcoind and the script +mine.sh+ are running in the background. To exit this shell, type +CTRL-D+ or +exit+ and you will be returned to your operating system prompt.
Running the interactive shell puts us "inside" the container. It logs in as user +root+, as we can see from the prefix +root@+ in the new shell prompt +root@e027fd56e31a:/bitcoind#+. If we issue the +ps x+ command to see what processes are running, we see both bitcoind and the script +mine.sh+ are running in the background. To exit this shell, type +CTRL-D+ or +exit+ and you will be returned to your operating system prompt.
Instead of running an interactive shell, we can also issue a single command that is executed inside the container, for example to run the +bitcoin-cli+ command, like this:
Instead of running an interactive shell, we can also issue a single command that is executed inside the container. In the following example we run the +bitcoin-cli+ command to obtain information about the current blockchain state:
As you can see, we need to tell +bitcoin-cli+ where the bitcoind data directory is, with the +datadir+ argument. We can then issue RPC commands to the Bitcoin Core node and get JSON encoded results.
As you can see, we need to tell +bitcoin-cli+ where the bitcoind data directory is by using the +datadir+ argument. We can then issue RPC commands to the Bitcoin Core node and get JSON encoded results.
All the docker containers also have +jq+ installed, which is a command-line JSON encoder/decoder, to help us process JSON on the command-line or from inside scripts. You can send the JSON output of any command to +jq+ using the +|+ character ("pipe" notation). For example, if we pipe the +getblockchaininfo+ JSON result we got above, we can extract the specific field +blocks+ like this:
All our docker containers have +jq+, which is a command-line JSON encoder/decoder, preinstalled. +jq+ helps us to process JSON-formatted data via the command-line or from inside scripts. You can send the JSON output of any command to +jq+ using the +|+ character. This character as well as this operation is called a "pipe". Let's apply a +pipe+ and +jq+ to the previous command as follows:
The +jq+ JSON decoder extract the result "189" from the +getblockchaininfo+, which we could use in a subsequent command.
+jq .blocks+ instructs the +jq+ JSON decoder to extract the field +blocks" from the +getblockchaininfo+ result. In our case, it extracts and prints the value of 189 which we could use in a subsequent command.
As you will see in the following sections, we can run several containers and then interact with them individually, issuing commands to extract information (such as the Lightning node public key), or to take an action (open a Lightning channel to another node). The +docker run+ and +docker exec+, together with +jq+ for JSON decoding are all we need to build a working Lightning Network that mixes many different node implementations and allows us to try out various experiments, all on our own computer.
As you will see in the following sections, we can run several containers at the same time and then interact with them individually. We can issue commands to extract information such as the Lightning node public key, or to take actions such as opening a Lightning channel to another node. The +docker run+ and +docker exec+ commands together with +jq+ for JSON decoding are all we need to build a working Lightning Network that mixes many different node implementations. This enables us to try out diverse experiments on our own computer.
First, we will install the golang package and associated libraries. We need, _at minimum_ Go version 1.13 or later. The official Go language packages are distributed as binaries from https://golang.org/dl. For convenience they are also packaged as debian packages distributed through the +apt+ command. You can follow the instructions on https://golang.org/dl or use the apt commands below on a Debian/Ubuntu Linux system:
First, we will install the golang package and associated libraries. We need, _at minimum_ Go version 1.13 or later. The official Go language packages are distributed as binaries from https://golang.org/dl. For convenience they are also packaged as debian packages distributed through the +apt+ command. You can follow the instructions on https://golang.org/dl or use the apt commands below on a Debian/Ubuntu Linux system as described on https://github.com/golang/go/wiki/Ubuntu:
== Running and Maintaining a Lightning Network Node
[[operating_ln_node]]
== Operating a Lightning Network Node
=== Implications of Configurations
After reading this far, you've probably set up a Lightning wallet. In this chapter we will take things one step further and set up a full Lightning node, learning how to operate and maintain it over time.
=== Backups
There are many reasons why you might want to set up your own Lightning node:
* To be a full participant of the Lightning Network, not just an end-user.
* To run an e-commerce store, or earn income with Lightning payments.
* To develop new services, applications or plugins on the Lightning Network
* To increase your privacy while using Lightning
Of course, there are costs to running a Lightning Network node. You need a computer (of course), a permanent Internet connection, lots of disk space, and lots of time!
But the skills you will learn from this experience are valuable and can be applied to a variety of other tasks too.
Let's get started!
=== Choosing your platform
There are many ways you can run a Lightning node, from a small mini-PC hosted in your home, to a dedicated server, to a hosted server in the cloud. The method you choose will depend on the resources you have and how much money you want to spend.
==== Running in the "cloud"
Virtual Private Server (VPS) and "cloud computing" services such as Microsoft Azure, Google Cloud, or Amazon Web Services (AWS) are quite affordable and can be set up very quickly. A Lightning node can be hosted for between $20 and $40 per month on such a service.
However, as the saying goes, "'Cloud' is just other people's computers". Using these services means running your node on other people's computers, with all the privacy and security implications that has. A Lightning node running in the "cloud", will always be less secure, less private that one running on your own computer. Additionally, these cloud computing services are very centralized. The vast majority of Bitcoin and Lightning nodes running on such services are located in a handful of data centers in Virginia, Sunnyvale, Seattle, London and Frankfurt. When the networks or data centers of these providers have service problems, it affects thousands of nodes on so-called "decentralized" networks.
Running a Lightning node in the cloud is better than not running a node at all, and worse than running it on your own computer at home or in your office.
==== Running a node at home
If you have a reasonable capacity internet link at home, or in your office, you can certainly run a Lightning node there. Any "broadband" connection is sufficient for this purpose (running a lightweight node), and a fast connection will allow you to run a Bitcoin full node too.
While you can run a Lightning node (and even a Bitcoin node) on your laptop, it will become annoying quite fast. These programs consume your computer's resources and need to run 24/7, and you will find yourself competing against the background services for your computer's attention (meaning your browser and other desktop workloads will be slowed down).
Instead, most users will choose to run a node on a dedicated computer. Fortunately, you don't need a "server" class computer to do this. You can run a Lightning node on a mini-PC, such as a Raspberry Pi or an Atom-based fanless PC, computers which are commonly used as a media server or home automation hub. These are relatively inexpensive, costing between $50 and $150 USD at that time of this writing. To run on a mini-PC, you will need an external USB hard drive, which again is relatively inexpensive, costing approximately $50 USD. The advantage of a dedicated mini-PC as a platform for Lightning and Bitcoin nodes is that it can run continuously, silently and unobtrusively on your home WiFi network, tucked behind your router or TV. No one will even know that this little box is actually a global banking system!
==== Using an installer/helper
Installing a Lightning node (or also a Bitcoin node), may be daunting if you are not familiar with a command-line environment. However, there are a number of projects that make "helpers", software that installs and configures the various components for you. You will still need to learn some command-line incantations to interact with your node, but most of the initial work is done for you.
One of the most popular and complete such "helpers", is _RaspibBlitz_, a project built by Christian Rootzoll, which is intended to be installed on a Raspberry Pi 4. RaspiBlitz comes with a recommended hardware "kit" that you can build in a matter of hours, or at most a weekend. If you attent a Lightning "hackathon" in your city, you are likely to see many people working on their RaspiBlitz set up, swapping tips and helping each other. You can find the RaspiBlitz project here:
https://github.com/rootzoll/raspiblitz
image::[images/raspiblitz.jpg]
MyNode is another popular open source project including a lot of Bitcoin related software. Is is super easy to install, you just have to flash it on an SD card, put it into your nano-computer an plug it in. You do not need any screen to use myNode as the administrative tools are accessible from a browser. You can manage it from a computer or even from your smartphone. Once installed, go to http://mynode.local/ and create a lightning wallet and node in two clicks.
==== Choosing your hardware
Independently of the type of computer you will use, from a server in the cloud to a mini-PC, you will have to choose the hardware configuration that suits your needs and wallet.
If you use a mini-PC or rent a server, the storage can be the costliest part. Let's have a look at the different options available. First there are two main types of drives, HDDs and SSDs. HDDs are a lot cheaper and SSDs are a lot faster, but both would do the job. The fastest SSDs up to date use the NVMe protocol, but depending on your hardware they may not be better than the cheaper version, the SATA ones. As an example, the Raspberry Pi 4 cannot benefit from them because of the limited bandwidth of it's USB port. To choose the size, let's look at the blockchain. As of July 2020 it's size is ~330GB including the transaction index. If you want to have some margin available for the future or to install other stuff on your node, choose at least 1TB.
For the CPU and RAM, low-end equipment should be enough to run a lightning node. The 10-year-old computer in your garage or your old student laptop should do the job. Even the $40 Raspberry Pi 4 has a powerful enough CPU to manage it. For the RAM 2GB is enough but 4GB is probably a better option.
* Bandwidth
* Full node backend (bitcoind, btcd) or lightweight backend (neutrino)
* Where will you host it (cloud hosting, VPS, own server, your laptop, RPi, MiniPC)
* What OS will you use (Linux variant, Windows, other)
* Docker or direct installation
=== Choose your node implementation
* Which project (LND, eclair, c-lightning, other) will you use
* Language and build-system familiarity
* Integration needs
* Features
* Performance
=== Node startup and configuration
* Startup and headless operation
* Process isolation
* Auto-unlocking
* IBD configuration
* Fine tuning performance
* RPC security and authentication
* Logging
* Process monitoring
=== Node and channel backups
* Difference between Bitcoin wallet (BIP39) backups and LN backups
* Need for backup of channel state
==== Static channel backups
@ -11,25 +95,54 @@
==== Hot wallet risk
==== Sweeping funds
===== On-chain sweep
===== Off-chain sweep
==== Watchtowers
=== Channel management
In order to participate on the Lightning Network you need to have payment channels on your Lighting networknode.
It might be confusing to decide with whom to open channels and which parameters to set for the specific channel policies.
An obvious strategy might be to open channels with shops where you want to buy products.
But what happens if those shops do only accept large channels which are beyond your funds or do not accept public channels at all?
Also if you are not a consumer but a vendor you want to receive payments on the Lightning Network.
Therefore you need others to open channels with you or you need to open channels and use submarine swaps and rebalancing techniques to provide yourself with the ability to receive payments.
Both techniques will be discussed later in this chapter.
==== Autopilots
==== Private vs public channels
==== Rebalancing
==== Manually choosing nodes for outbound channels
==== submarine swaps
==== Autopilot
==== Fees and Centrality
==== Getting inbound liquidity
==== On-chain fees for channel management
==== Submarine swaps
==== Splice-in/Splice-out
=== Routing fees
* Earning fees from routing
* Setting routing fees
* High volume/low cost vs. High cost/low volume
* Zero fee routing
=== Node monitoring
* RTL
* lndash
* External node monitors (1ml etc.)
=== Channel maintenance
==== Inactive channels and nodes
==== When to force-close
==== Re-balancing channels
=== Running multiple Lightning Network nodes
==== Private channels
==== Topology
=== Hardware Requirements for a Lightning Nodes
@ -82,3 +195,4 @@ At a minimum, the following will be required to run a Lightning Node:
In addition, the user will also want to consider some kind of data backup solution.
This could be a cloud-based automated backup to a server or web service the user controls.
Or it could be a hardware backup, such as a second hard drive that can be used if the first hard drive fails.
@ -40,7 +40,7 @@ _I (Alice) will give you (Bob) 10 golden coins if you pass them on to Wei_
While this contract is nice in the real world Alice yields the issue that Bob might just breach the contract and hope not to get caught by law enforcement.
Even if Bob got caught by law enforcement Alice faces the risk that he might be bankrupt and her 10 golden coins would be gone anyway.
Assuming these issues are magically solved it would still be unclear to from a contract point of view that Wei also has to have a contract with Gloria to deliver the coins.
Thus we improove our contract:
Thus we improve our contract:
_I (Alice) will reimburse you (Bob) with 10 golden coins if you can proof to me (for example via a receipt) that you already have delivered 10 golden coins to Wei_
@ -64,23 +64,23 @@ We call this hash the payment hash.
In reality Gloria would come up with a large random number as a secret.
This is to be really secure and prevent others from guessing it.
But let us assume that in our case Glorias secret take reads `*Glorias secret*`.
She would commit to the secret by computing the sha256 hash which reads `*70c87220dd901a004804b49e9ec2fd73283fad127cf112fefa67e6b79b8739b7*`.
You can verify this by typing `echo "Glorias secret" | sha256sum` to your linux command line.
She would commit to the secret by computing the sha256 hash which reads `*f23c83babfb0e5f001c5030cf2a06626f8a940af939c1c35bd4526e90f9759f5*`.
You can verify this by typing `echo -n "Glorias secret" | sha256sum` to your linux command line.
As Alice wants to send 10 golden coins to Gloria she is told by Gloria to use this payment hash to receive a proof of payment.
Alice now sets up a contract that reads:
_I (Alice) will reimburse you (Bob) with 12 golden coins if you can show me a valid message - we call it preimage - that hashes to `*70c87220dd901a004804b49e9ec2fd73283fad127cf112fefa67e6b79b8739b7*`. You can acquire this message by setting up a similar Contract with Wei who has to set up a similar contract with Gloria. In order to assure you that you will get reimbursed I will provide the 12 Golden coins to an trusted escrow before you set up your next contract._
_I (Alice) will reimburse you (Bob) with 12 golden coins if you can show me a valid message - we call it preimage - that hashes to `*f23c83babfb0e5f001c5030cf2a06626f8a940af939c1c35bd4526e90f9759f5*`. You can acquire this message by setting up a similar Contract with Wei who has to set up a similar contract with Gloria. In order to assure you that you will get reimbursed I will provide the 12 Golden coins to an trusted escrow before you set up your next contract._
After Bob and Alice agree to the contract and Bob receives the message from the escrow that Alice has deposited the 12 golden coins Bob negotiates a very similar contract with Wei.
Note that due to the service fees he will only forward 11 golden coins to Wei and demand from Wei who also wants to earn a fee of 1 golden coin to show proof that 10 golden coins have been delivered to Gloria.
_I (Bob) will reimburse you (Wei) with 11 golden coins if you can show me a valid message - we call it preimage - that hashes to `*70c87220dd901a004804b49e9ec2fd73283fad127cf112fefa67e6b79b8739b7*`. You can acquire this message by setting up a similar contract with Gloria. In order to assure you that you will get reimbursed I will provide the 11 Golden coins to an trusted escrow before you set up your next contract._
_I (Bob) will reimburse you (Wei) with 11 golden coins if you can show me a valid message - we call it preimage - that hashes to `*f23c83babfb0e5f001c5030cf2a06626f8a940af939c1c35bd4526e90f9759f5*`. You can acquire this message by setting up a similar contract with Gloria. In order to assure you that you will get reimbursed I will provide the 11 Golden coins to an trusted escrow before you set up your next contract._
As Wei gets message from the escrow that Bob has deposited the 10 golden coins Wei sets up a similar contract with Gloria:
_I (Wei) will reimburse you (Gloria) with 10 golden coins if you can show me a valid message - we call it preimage - that hashes to `*70c87220dd901a004804b49e9ec2fd73283fad127cf112fefa67e6b79b8739b7*`. In order to assure you that you will get reimbursed after revealing the secret I will provide the 10 Golden coins to an trusted escrow._
_I (Wei) will reimburse you (Gloria) with 10 golden coins if you can show me a valid message - we call it preimage - that hashes to `*f23c83babfb0e5f001c5030cf2a06626f8a940af939c1c35bd4526e90f9759f5*`. In order to assure you that you will get reimbursed after revealing the secret I will provide the 10 Golden coins to an trusted escrow._
As Gloria learns from the escrow that the coins where deposited she reveals the secret preimage to Wei.
Since she initially came up with the secret and committed to it in form of the payment hash she obviously is able to provide the secrete to Wei and their escrow service.
@ -167,8 +167,8 @@ cltv stands for OP_CHECKTIMELOCKVERIFY and is the OP_CODE that will be used in t
Finally in the last data field there are 1336 Bytes of data included which is an `onion routing packet`.
The format of this packet will be discussed in the last section of this chapter.
For now it is important to note that it includes encrypted routing hints and information of the payment path that can only be partially decrypted by the recipient of the onion routing packet to extract information to whom to forward the payment or to learn that one as the final recipient.
In any case the onion roting packet is always of the same size preventing the possability to guess the position of an intermediary node within a path.
In our particular case Bob will be able to decrypt the first couple bytes of the onion routing packet and learn that the payment is not to be forwored but intendet to be for him.
In any case the onion roting packet is always of the same size preventing the possibility to guess the position of an intermediary node within a path.
In our particular case Bob will be able to decrypt the first couple bytes of the onion routing packet and learn that the payment is not to be forwarded but intended to be for him.
The received information is enough for Bob to create a new commitment transaction.
This commitment transaction now has not only 2 outputs encoding the balance between Alice and Bob but a third output which encodes the hashed time locked contract.
@ -178,10 +178,10 @@ This commitment transaction now has not only 2 outputs encoding the balance betw
image:images/routing-setup-htlc-1.png[]
We can see that Bob Assumes that Alice will agree to lock 15 mBTC of her previous balance and assign it to the HTLC output.
Creating this HTLC output can be compared to giving Alices golden coins to the escrow service.
Creating this HTLC output can be compared to giving Alice golden coins to the escrow service.
In our situation the bitcoin network can enforce the HTLC as Bob and Alice have agreed upon.
Bob's Balance has not changed yet.
In Bitcoin outpus are mainly described by scripts.
In Bitcoin outputs are mainly described by scripts.
The received HTLC in Bob's commitment transaction will use the following bitcoin script to define the output:
@ -205,11 +205,11 @@ The received HTLC in Bob's commitment transaction will use the following bitcoin
We can see that there are basically three conditions to claim the output.
1. Directly if a revocation key is known. This would happen if at a later state Bob fraudulently publishes this particular commitment transaction. As a newer state could only be agreed upon if Alice has learnt Bob's half of the revocation secret she could directly claim the funds and keep them even if Bob was later able to provide a proof of payment. This is mainly described in this line `OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL` and can be down by using `<revocation_sig> <revocationpubkey> as a witness script.
2. If Bob has successfully delivered the payment and learnt the preimage he can spend the HTLC output with the help of the preimage and his `local_HTLC_secret`. This is to make sure that only Bob can spend this output if the commitment transaction hits the chain and not any other third party who might know the preimage because they had been included in the routing process. Claiming this output requires an HTLC-success transaction whih we describe later.
3. Finally Alice can use her `remote_HTLC_secret` to spend the HTLC output after the timeoput of `cltv_expiry` was passed by using the following witness script `<remoteHTLCsig> 0`
2. If Bob has successfully delivered the payment and learnt the preimage he can spend the HTLC output with the help of the preimage and his `local_HTLC_secret`. This is to make sure that only Bob can spend this output if the commitment transaction hits the chain and not any other third party who might know the preimage because they had been included in the routing process. Claiming this output requires an HTLC-success transaction which we describe later.
3. Finally Alice can use her `remote_HTLC_secret` to spend the HTLC output after the timeout of `cltv_expiry` was passed by using the following witness script `<remoteHTLCsig> 0`
As the commitment transaction spends the 2 out of 2 multisig fundin transaction Bob needs two signatures after he constructed this commitment transaction.
He can obviosly compute his own signature but he needs also the signature from Alice.
As the commitment transaction spends the 2 out of 2 multisig funding transaction Bob needs two signatures after he constructed this commitment transaction.
He can obviously compute his own signature but he needs also the signature from Alice.
As Alice initiated the payment and wanted the HTLC to be set up she will be reluctant to provide a signature.
@ -234,10 +234,10 @@ At this time he would be able to publish either the old one or the new one witho
However this is save for Alice as Bob has less money in this old state and is economically not incentivised to publish the old commitment transaction.
Alice on the other side has no problem if Bob publishes the new commitment transaction as she wanted to send him money.
If Bob can provide the preimage he is by their agreement and expectation entitled to claim the HTLC output.
Should Bob decide to sabotatge to future steps of the protocol Alice can either publish her commitment transaction without Bob being able to punish her.
Should Bob decide to sabotage to future steps of the protocol Alice can either publish her commitment transaction without Bob being able to punish her.
He will just not have received the funds from Alice.
This is important!
Despitethe fact that Bob has a new commitment transaction with two valid signatures and an HTLC output inside he cannot seen his HTLC as being set up successfully.
Despitethe fact that Bob has a new commitment transaction with two valid signatures and an HTLC output inside he cannot seen his HTLC as being set up successfully.
He first needs to have Alice invalidate her old state.
That is why - in the case that he is not the final recipient of the funds - he should not forward the HTLC yet by setting up a new HTLC on the next channel with Wei.
Alice will not invalidate her commitment transaction yet as she has to first get her new commitment transaction and she wants Bob to invalidate his old commitment transaction which he can safely do at this time.
@ -252,13 +252,13 @@ The `revoke_and_ack` Lightning message contains three data fields.
* [`point`:`next_per_commitment_point`]
While it is really simple and straight forward it is very crucial.
Bob shares the the `per_commitment_secret` of the old commitment transaction which serves as the revocation key and would allow Alice in future to penalize Bob if he publishes the old commitment transactio without the HTLC output.
Bob shares the the `per_commitment_secret` of the old commitment transaction which serves as the revocation key and would allow Alice in future to penalize Bob if he publishes the old commitment transaction without the HTLC output.
As in a future Alice and Bob might want to negotiate additional commitment transactions he already shares back the `next_per_commitment_point` that he will use in his next commitment transaction.
Alice checks that the `per_commitment_secret` produces the last `per_commitment_point` and constructs her new commitment transaction with the HTLC output.
Alice's version of the HTLC output is slightly different to the one that Bob had.
The reason is the asymmetrie of the pentalty based payment channel construction protocol.
Alice is offering in her commitment transaction an HTLC to the `remote` partner of the channl while Bob as accepting and offered HTLC to himself the `local` partner of the channel.
The reason is the asymmetries of the penalty based payment channel construction protocol.
Alice is offering in her commitment transaction an HTLC to the `remote` partner of the channel while Bob as accepting and offered HTLC to himself the `local` partner of the channel.
Thus the Bitcoin script is adopted slightly.
It is a very good exercise to go through both scripts and see where they differ.
You could also try to use Bob's HTLC output script to come up with Alice's and vice versa and check your result with the following script.
@ -279,7 +279,7 @@ You could also try to use Bob's HTLC output script to come up with Alice's and v
OP_ENDIF
OP_ENDIF
Bob can redeem the HTLC with `<remoteHTLCsig> <payment_preimage>` as the whitness script and in case the commitment tranaction is revoked but published by alice Bob can trigger the penality by spending this output immediately with the following witness script `<revocation_sig> <revocationpubkey>`.
Bob can redeem the HTLC with `<remoteHTLCsig> <payment_preimage>` as the witness script and in case the commitment transaction is revoked but published by Alice, Bob can trigger the penality by spending this output immediately with the following witness script `<revocation_sig> <revocationpubkey>`.
[[routing-setup-htlc-4]]
.Bob knows how Alice's commitment transaction will look like and sends over the necessary signatures.
This process is completely symmetrical to the one where Alice sent her signatures for Bob's new commitment transaction.
Now Alice is the one having two valid commitment transactions.
Technically she can still abort the payment by publishing her old commitment transaction to the bitcon network.
Noone would lose anything as Bob knows that the contract is still being set up and not fully set up yet.
Technically she can still abort the payment by publishing her old commitment transaction to the bitcoin network.
Noone would lose anything as Bob knows that the contract is still being set up and not fully set up yet.
This is a little bit different than how the situation would look like in a real world scenario.
Recall Alice and Bob both have set up a new commitment transaction and have exchanged signatures.
In the real world one would argue that this contract is now valid.
@ -298,7 +298,7 @@ In the real world one would argue that this contract is now valid.
image:images/routing-setup-htlc-5.png[]
Now Bob and Alice both have a new commitment transaction with and additional HTLC output and we have achieved a major step towards updating a payment channel.
The new Balance of Alice and Bob does not reflect yet that Alice has succesfully send 15 mBTC to Bob.
The new Balance of Alice and Bob does not reflect yet that Alice has successfully send 15 mBTC to Bob.
However the hashed time locked contracts are now set up in a way that secure settlement in exchange for the proof of payment will be possible.
This yields another round of communication with Lightning messages and setting up additional commitment transactions which in case of good cooperation remove the outstanding HTLCs.
Interestingly enough the `commitment_signed` and `revoke_and_ack` mechanism that we described to add an HTLC can be reused to update the commitment transaction.
@ -313,13 +313,13 @@ This message has the type 130 and only 3 data fields:
As other messages Bob uses the `channel_id` field to indicates for which channel he returns the preimage.
The htlc that is being removed is identified by the same `id` that was used to set up the HTLC in the commitment transaction initially.
You might argue that Alice would not need to know the id of the HTLC for which Bob releases the preimage as the preimage and payment hash could be unique.
However with this design the protocoll supports that a payment channel has several htlcs with the same preimage but only settles one.
One could argue that this does not make too much sense and it is good to be criticle but this is how the protcol is designed and what it supports.
However with this design the protocol supports that a payment channel has several htlcs with the same preimage but only settles one.
One could argue that this does not make too much sense and it is good to be critical but this is how the protocol is designed and what it supports.
Finally in the last field Bob provides the `payment_preimage` which Alice can check hashes to the payment hash.
[WARNING]
====
When designing, implementing or studying a protocol one should ask: Is it safe to this or that in this moment of the protocol and can this be abused. We discussed for example the messages that where necessary for an HTLC to become valid. We pointed out that Bob should not see the received HTLC as valid even though he already has a new commitment transaction with signatures and invalidated his old commitment transaction before Alice also revoked her old commitment transaction. We also saw that noone is able to mess with the protocol of setting up a commitment transaction as in the worst case the protocol could be aborted and any dispute could be resolved by the Bitcoin Netwok. In the same way we should ask ourselves is it safe for Bob to just send out and release the preimage even though neither he nor alice have created the new pair of commitment transactions in which the HTLCs are removed. It is important to take a short break and ask yourself if Bob will in any case be able to claim the funds from the HTLC if the preimage is correct?
When designing, implementing or studying a protocol one should ask: Is it safe to this or that in this moment of the protocol and can this be abused. We discussed for example the messages that where necessary for an HTLC to become valid. We pointed out that Bob should not see the received HTLC as valid even though he already has a new commitment transaction with signatures and invalidated his old commitment transaction before Alice also revoked her old commitment transaction. We also saw that noone is able to mess with the protocol of setting up a commitment transaction as in the worst case the protocol could be aborted and any dispute could be resolved by the Bitcoin Network. In the same way we should ask ourselves is it safe for Bob to just send out and release the preimage even though neither he nor alice have created the new pair of commitment transactions in which the HTLCs are removed. It is important to take a short break and ask yourself if Bob will in any case be able to claim the funds from the HTLC if the preimage is correct?
====
It is safe for Bob to tell Alice the preimage.
@ -337,7 +337,7 @@ Isn't it remarkable that even though the process of exchanging funds for an prei
=== Source based Onion Routing
So far you have learnt that payment channels can be connected to a network which can be utilized to send payment from one participant to another one through a path of payment channels.
You have seen that with the use of HTLCs the intemediary nodes along the path are not able to steal any funds that they are supposed to forward and you have also learnt how a node can set up and settle an HTLC.
You have seen that with the use of HTLCs the intermediary nodes along the path are not able to steal any funds that they are supposed to forward and you have also learnt how a node can set up and settle an HTLC.
While this is all great it leaves a couple of questions open:
- Who chooses the path?
@ -347,58 +347,58 @@ While this is all great it leaves a couple of questions open:
The short answer to the first questions is that only the sender decides which path to choose.
Despite the fact that the Lightning Network is currently running the second question is still not answered in an optimal way and became a serious research topic.
For now we will only say that in the standard case the sender more or less randomly selects and tries paths of channels until it is possible to send the amount along that selected path.
With multi path payments the sender can split the amount and use the same strategy with multiple pahts.
More deails will be discuss in the advanced section about path finding.
With multi path payments the sender can split the amount and use the same strategy with multiple paths.
More details will be discuss in the advanced section about path finding.
There we explore and explain the current approach which seems to work good enough most of the time.
You will also learn about potential improvements that are currently being researched in that chapter.
The short answer to the third question is that no other node in the network learns about this path.
Nodes along the path only learn on which channel they received a payment and on which channel they are supposed to forward it.
Neither do they know whether the peer on the receiving channel initiated the payment nor do they know whether the peer on the outgoing channel is the final recipient of the payment.
We exepect you to be surprised that it is actually possible to create such an algorithm with modern cryptography.
We expect you to be surprised that it is actually possible to create such an algorithm with modern cryptography.
This is why we will now devote quite some space to write and discuss about source based onion routing.
This technique is fundamentally different to the best effort routing approach that is implemented on the internet protocol.
Best effort routing is know to have poor privacy protection of the transfered data and needs end to end encryption on the upper layers to be secure.
As many upper layer protocols did not include end to end encryption we learnt from the Snoweden revelations that spying agencies have been massivily collecting data that was transfered over the internet together with the meta data like IP addresses of senders and recipients.
As many upper layer protocols did not include end to end encryption we learnt from the Snoweden revelations that spying agencies have been massively collecting data that was transfered over the internet together with the meta data like IP addresses of senders and recipients.
To get rid of these problems the Lightning Network utilizes a sourced base onion routing based on the SPHINX Mix format.
The SHINX mix format was originally designed to allow email remails to offer the possability to send an answer without creating a security threat of the remailer service being able to know who was communicating with whom.
The SHINX mix format was originally designed to allow email remails to offer the possibility to send an answer without creating a security threat of the remailer service being able to know who was communicating with whom.
In that sense and very roughly speaking the SPHINX Mix format can be compared with the onion routing that is well known from the TOR network.
[NOTE]
====
While the Lightning Network also uses an onion routing scheme it is actually very differnt to the onion routing scheme that is used in the TOR network.
The biggest difference is that TOR is being used for arbitrary data to be exchanged between two participants where on the Lightning Network the main usecase is to pay people and transfer data that encodes monitary value.
On the Lightning Network there is no analogy to the exit nodes of the Tor Network which on the TOR network produce a security risk. Lightning user should still not get theimpression that their data and information is perfectly secure. Knowing the announced fee rates and CLTV deltas a node might be able to guess the destination of an onion.
While the Lightning Network also uses an onion routing scheme it is actually very different to the onion routing scheme that is used in the TOR network.
The biggest difference is that TOR is being used for arbitrary data to be exchanged between two participants where on the Lightning Network the main usecase is to pay people and transfer data that encodes monetary value.
On the Lightning Network there is no analogy to the exit nodes of the Tor Network which on the TOR network produce a security risk. Lightning user should still not get theimpression that their data and information is perfectly secure. Knowing the announced fee rates and CLTV deltas a node might be able to guess the destination of an onion.
In TOR the security can be compromised if all randomly chosen TOR hops are acting together. In Lightning the payment hash identifies a payment and thus not every node along the path needs to be compromised in order to attack the security.
On the TOR network nodes are basically connected via a full graph as every node could create an encrypted connection with every other node on top of the Internet Protocol almost instantaneously and at no cost. On the Lightning Network payments can only flow along existing payment channels. Removing and adding of those channels is a slow and expensive process as it requires onchain bitcoin transactions.
On the Lightnign Network nodes might not be able to forwad a payment package because they do not own enough funds on their side of the payment channel. On the other hand there are hardly any plausible reasons other then its wish to act malliciously why a TOR node might not be able to forward an onion.
Last but not least the Lightning Network can actuly run on TOR.
On the Lightning Network nodes might not be able to forward a payment package because they do not own enough funds on their side of the payment channel. On the other hand there are hardly any plausible reasons other then its wish to act maliciously why a TOR node might not be able to forward an onion.
Last but not least the Lightning Network can actually run on TOR.
This means that all connections of a node with its peers and the resulting communication will by obfuscated once more through the TOR network.
====
Lets stick to our example in which Alice still wants to tip Gloria and has decided to use the path via Bob and Wei.
We note that there might have been alternative paths from Alice to Gloria but for now we will just assume it is this path that Alice has decided to use.
You have already learnt that Alice needs to set up an HTLC with Bob via and `update_add_htlc` message.
As discussed the `update_add_htlc` message containes a data field of 1366 Bytes in length that is the onion package.
This onion cointains all the information about the path that Alice intends to use to send the payment to Gloria.
As discussed the `update_add_htlc` message contains a data field of 1366 Bytes in length that is the onion package.
This onion contains all the information about the path that Alice intends to use to send the payment to Gloria.
However Bob who receives the onion cannot read all the information about the path as most of the onion is hidden from him through a sequence of encryptions.
The name onion comes from the analogy to an onion that consits of several layers. In our case every layer corresponds to one round of encryption.
The name onion comes from the analogy to an onion that consists of several layers. In our case every layer corresponds to one round of encryption.
Each round of encryption uses different encryption keys.
They are chosen by Alice in a way that only the rightful recipient of an onion can peel of (decrypt) the top layer of the onion.
For example after Bob received the onion from Alice he will be able to decrypt the first layer and he will only see the information that he is supposed to forward the onion to Wai by setting up an HTLC with Wei.
For example after Bob received the onion from Alice he will be able to decrypt the first layer and he will only see the information that he is supposed to forward the onion to Wei by setting up an HTLC with Wei.
The HTLC with Wei should use the same Payment Hash as the receiving HTLC from Alice.
The amount of the forwarded HTLC was specified in Bob decrypted layer of the onion.
It will be slightly smaller than the imount of his incoming HTLC from Alice.
It will be slightly smaller than the amount of his incoming HTLC from Alice.
The difference of these two amounts has to be at least as big as to cover the routing fees that Bob's node announced earlier on the gossip protocol.
In order to set up the HTLC Bob will modify the onion a little bit.
He removes the information that he could read from it and passes it along to Wei.
Wei in turn is only able to see that he is supposed to forwad the package to Gloria.
Wai knows he recieved the onion from Bob but has no clue that it was actually Alice who initiated the onion in the first place.
Wei in turn is only able to see that he is supposed to forward the package to Gloria.
Wei knows he received the onion from Bob but has no clue that it was actually Alice who initiated the onion in the first place.
In this way every participant is only able to peel of one layer of the onion by decrypting it.
Each participant will only learn the information it has to learn to fullfil the routing request.
Each participant will only learn the information it has to learn to fulfill the routing request.
For example Bob will only know that Alice offered him an HTLC and sent him an onion and that he is supposed to offer an HTLC to Wei and forward a slightly modified onion.
Bob does not know if Alice is the originator of this payment as she could also just have forwarded the payment to him.
Due to the layered encryption he cannot see the inside of Wei's, and Gloria's layer.
@ -412,17 +412,17 @@ Let us now look at the construction of the Onion that Alice has to follow and at
The onions are a data structure that at every hop consists of four parts:
1. The version byte
2. The header consisting of a public key that can be used by the recipient to produce the shared secret for decrypting the outer layer and to derieve the public key that has to be put in the header of the modified onion for the next recipient.
2. The header consisting of a public key that can be used by the recipient to produce the shared secret for decrypting the outer layer and to derive the public key that has to be put in the header of the modified onion for the next recipient.
3. The payload
4. an authentication via an HMAC.
For now we will ignore how the public keys are derived and exchanged and focus on the payload of the onion.
Only the payload is actually encrypted and will be peeled of layer by layer.
The payload consits of a sequence of a sequence of per hop data.
The payload consists of a sequence of a sequence of per hop data.
This data can come in two formats the legacy one and the Type Length Value (TLV) Format.
While the TLV format offers more flexability in both cases the routing information that is encoded into the onion is the same for every but the last hop.
While the TLV format offers more flexibility in both cases the routing information that is encoded into the onion is the same for every but the last hop.
On the last hop the TLV information departs from the legacy information as it allows to include a preimage.
This is nice as it allow a payer to initiate a payment without the neccessity to ask the payer for an invoice and payment hash first.
This is nice as it allow a payer to initiate a payment without the necessity to ask the payer for an invoice and payment hash first.
We will this feature called key send in a different chapter.
A node needs three pieces of information to forward the package:
@ -444,12 +444,12 @@ On the incoming HTLCs David should have seen that exact amount.
Usually this amount is intended to say how many satoshis should be forwarded.
Since the short channel id was set to zero in this particular case it is interpreted as the payment amount.
Finally the CLTV delta which David should use to forward the payment is also set to zero as David is the final hop.
These data fields consit of 20 Bytes.
These data fields consist of 20 Bytes.
The Lightning Network protocol actually allows to store 65 Bytes of data the Onion for every hope.
- 1 Byte Realm which signals nodes how to decode the following 32 Bytes.
- 32 Byte for routing hints (20 of which we have already used).
- 32 Byte of a Hasched Message Authentication code.
- 32 Byte of a Hashed Message Authentication code.
Since the additional 12 Byte of data for the routing hints were not needed at this time they are set to zero.
In the next diagram we can see how the per hop payload for David looks like.
On important feature to protect the privacy is to make sure that onions are always of equal length independ of their position along the payment path.
Thus onions are always expected to contain 20 entries of 65 Bytes with per hop data.
As David is the final recipient there is only reasonable data for 65 Bytes ofth per hop data.
As David is the final recipient there is only reasonable data for 65 Bytes ofthe per hop data.
This is not a problem as the other 19 fields are filled with junk data.
You could also see this in the previous diagram.
@ -481,13 +481,13 @@ However others cannot derive the same shared secrete as they neither know Alice'
[NOTE]
====
Let `(d,D)` be the secret and Public key of David and let G be the generator point of the elliptic curve so that `D = d*G`.
similarily let `(ek_d, EPK_D)` the ephemeral keys that Alice has generated for David such that the Publikc ephemeral Key `EPK_D = ek_d*G`.
Similarly let `(ek_d, EPK_D)` the ephemeral keys that Alice has generated for David such that the Public ephemeral Key `EPK_D = ek_d*G`.
Alice computed the shared secret as ss_`d = ek_d*D`.
Using the definition of public keys this is the same as `ek_d*(d*G)=(ek_d*d)*G`.
Since multiplication with the generator point is a group homomorphism we can apply the law of associativity.
And because the secretes are just numbers modulo some prime we can change the order of the multiplication resulting in `ss_d = (d*ek_d)*G`.
With the same argument as before we apply the law of associativity and apply the definition of public keys resulting in `(d*ek_d)*G = d*(ek_d*G) = d*EPK_D`.
We just saw why `ek_d*D = d*EPK_D = ss_d` and why Alice and Davide will be able to derive the same shared secrete if Alice puts the ephmeral pubilic key inside the onion.
We just saw why `ek_d*D = d*EPK_D = ss_d` and why Alice and Davide will be able to derive the same shared secrete if Alice puts the ephemeral public key inside the onion.
====
After the encrypted Onion for David is created Alice will create the next outer layer by creating the onion for Wei.
We emphasize that Wei has no chance to decrypt the inner part of the onion.
However the information for Wei should also be protected from others.
Thus Alice conducts another ECDH.
This time with Wei's public key and and emphemeral keypair that she has generated particularly for Wei.
This time with Wei's public key and and ephemeral keypair that she has generated particularly for Wei.
She uses the shared secret to encrypt the onion payload.
She would be able to construct the entire onion for Wei - which actually Bob does while he forwards the onion.
The Onion that Wei would receive can be seen in the following diagram:
@ -520,29 +520,29 @@ The Onion that Wei would receive can be seen in the following diagram:
.`per_hop` payload of Glorias onion and the encrypted
image:images/routing-onion-5.png[]
Note that in the entire onion there will be Wei's empheral public key.
Note that in the entire onion there will be Wei's ephemeral public key.
David ephemeral public key is not stored anywhere in the onion.
Neither in the header, nor in the payload data.
However we have seen that David needed to have this key in the header of the Onion that he received.
Luckily the ephemeral keys that Alice used for the ECDH with david can be derived from the ephmeral key that she used for Wei.
Thus after Wei decrypts his layer he can use the shared secrete and his emphermal public key to derrive the Empheral public key that David is supposed to use and store it in the header of the Onion that he forwards to David.
The exact progress to generate the empheral keys for every hope will be explained at the very end of the chapter.
Luckily the ephemeral keys that Alice used for the ECDH with David can be derived from the ephemeral key that she used for Wei.
Thus after Wei decrypts his layer he can use the shared secrete and his ephemeral public key to derive the ephemeral public key that David is supposed to use and store it in the header of the Onion that he forwards to David.
The exact progress to generate the ephemeral keys for every hope will be explained at the very end of the chapter.
Similarly it is important to recognize that Alice removed data from the end of Davids onion payload to create space for the per hop data in Wei's onion.
Thus when Wei has received his onion and removed his routing hints and per hopd data the onion would be to short and he somehow needs to be able to append the 65 Bytes of filled junk data in a way that the HMACs will still be valid.
This process is of filler generation as well as the process of derriving the emphemeral keys is described in the end of this chapter.
Thus when Wei has received his onion and removed his routing hints and per hop data the onion would be to short and he somehow needs to be able to append the 65 Bytes of filled junk data in a way that the HMACs will still be valid.
This process is of filler generation as well as the process of deriving the ephemeral keys is described in the end of this chapter.
What is important to know is that every hope can derive the Ephemeral Public key that is necessary for the next hop and that the onions save space by always storing only one ephemeral key instead of all the keys for all the hops.
Finally after Alice has computed the encrypted version for Wei she will use the exact same process to compute the encrypted version for Bob.
For Bobs onion she actually computes the header and provides the emphemeral public key herself.
For Bobs onion she actually computes the header and provides the ephemeral public key herself.
Note how Wei was still supposed to forward 3000 satoshis but How Bob was supposed to forward a different amount.
The difference is the routing fee for Wai.
Bob on the other hand will only forward the onion if the difference between the mount to forward and the HTLC that Alice sets up while transfering the Onion to him is large enough to cover for the fees that he would like to earn.
The difference is the routing fee for Wei.
Bob on the other hand will only forward the onion if the difference between the mount to forward and the HTLC that Alice sets up while transferring the Onion to him is large enough to cover for the fees that he would like to earn.
[NOTE]
====
We have not discussed the exact cryptographic algorithms and schemes that are being used to compute the cypthertext from the plain text.
Also we have not discussed how the HMACs are being computed at every step and how everything fits together while the Onions are always being trucated and modified on the outer layer.
We have not discussed the exact cryptographic algorithms and schemes that are being used to compute the ciphertext from the plain text.
Also we have not discussed how the HMACs are being computed at every step and how everything fits together while the Onions are always being truncated and modified on the outer layer.
If everything until here made perfect sense to you and you want to learn about those details we believe that you have all the necessary tools at hand to read BOLT 04 which is why we decided not to include all those technical details here in the book.
BOLT 04 is the open source specification of the onion routing scheme that is being used on the Lightning Network and a perfect resource for the missing details.
====
@ -551,15 +551,15 @@ TODO: everything from here on will most likely change and could even be redundan
Onions are being constructed from the inside to the outside.
As the inside of the onion is decrypted last it has to correspond to the recipient which in our case is Gloria.
As every layer of the Onion is encrypted by Alice in such a way that only the respective recpient can decrypt their layer Alice needs to come up with a sequence of encryption keys that she will use for each and every hop.
The main concept that is being used is the shared secret computation via an elliptic Courve Diffie Helmann Key exchange (ECDH) between Alice and each of the hops.
As every layer of the Onion is encrypted by Alice in such a way that only the respective recipient can decrypt their layer Alice needs to come up with a sequence of encryption keys that she will use for each and every hop.
The main concept that is being used is the shared secret computation via an elliptic Curve Diffie Hellmann Key exchange (ECDH) between Alice and each of the hops.
However for the recipients to be able to to compute their shared secrete they have to know a public key which they can use.
If Alice used the same private key for the computation of each of the shared secrets Alice would have to send the same public key with the onion.
the different payments could be linked together by an attacker that is why
Every layer of the onion has 32 Bytes of `per_hop` data.
This data is split into 4 data filds
This data is split into 4 data fields
- The 8 Byte `short_channel_id` indicates on which channel the onion should be forwarded next
- The 8 Bytes `amt_to_forward` is a 64 Bit unsigned integer that encodes an amount in millisatoshi and indicates the amount that is supposed to be forwarded
@ -576,20 +576,20 @@ This data is split into 4 data filds
image:images/routing-onion-6.png[]
Interestingly enough Alice can construct the onion with different encryption keys for Bob, Wei and Gloria without the necessity to estable a peer connection with them.
Interestingly enough Alice can construct the onion with different encryption keys for Bob, Wei and Gloria without the necessity to establish a peer connection with them.
She only needs a public key from each participant which is the public `node_id` of the lightning node and known to Alice.
As other nodes she has learnt about the existance of public payment cannels and the public `node_id` of other participants via the gossip protocol which we described in its own chapter.
In order to have a different encryption key for every layer Alice produces a shared secrete with each hop using the public `node_id` of each node and conduct an Elliptic Curve Diffi Hellmann Key exchange (ECDH).
As other nodes she has learnt about the existence of public payment channels and the public `node_id` of other participants via the gossip protocol which we described in its own chapter.
In order to have a different encryption key for every layer Alice produces a shared secrete with each hop using the public `node_id` of each node and conduct an Elliptic Curve Diffie Hellmann Key exchange (ECDH).
She starts by generating a temporary session key.
This key will also be called the ephemeral key.
This private key multiplied with the generator Point of the Elliptic curve that is being used in Bitcoin produces a public key.
This happens in the same way how the nodes public key is generated from the secrete private key of the node.
Alice could use this session keys to conduct the diffi hellmann key exchange if she would send the public key with the onion.
However she wishes to use a different session key to conduct the diffie Hellmann key exchange with each of the nodes along the path.
Alice could use this session keys to conduct the Diffie Hellmann key exchange if she would send the public key with the onion.
However she wishes to use a different session key to conduct the Diffie Hellmann key exchange with each of the nodes along the path.
**TODO**: WHY?!
Yet she does not want to add a public key (which consumes quite some space) into every layer of the onion.
Luckily there is a nice deterministic way in which she can derive different sessions keys for every hop and execute the Diffi Hellmann and allow the hops to use their shared secrete to derieve the next session public key.
Luckily there is a nice deterministic way in which she can derive different sessions keys for every hop and execute the Diffie Hellmann and allow the hops to use their shared secrete to derive the next session public key.
Lets explore this in detail with the following example:
@ -608,13 +608,13 @@ However with that information nodes would know that Alice was the originator of
In the first part of the routing chapter you have learnt that payments securely flow through the network via a path of HTLCs.
You saw how a single HTLC is negotiated between two peer and added to the commitment transaction of each peer.
In the second part you have seen how the necessary information for setting up HTLCs along a path of hops are being transfered via onions from the source to the sender.
A mechanism that protocets the privacy of payer and payee.
A mechanism that protects the privacy of payer and payee.
However there are quite some challenges and things that can go not as expected.
This is why we we want to discuss how errors are being handled and what users and developers should take into consideration.
Most importantnly it is absolutely necessary that you anderstand that once your node sent out an onion on your behalf (most likely because you wanted to pay someone) Everything that happens to the onion is now out of your control.
Most importantly it is absolutely necessary that you understand that once your node sent out an onion on your behalf (most likely because you wanted to pay someone) Everything that happens to the onion is now out of your control.
* You cannot force nodes to forward the onion immediatly.
* You cannot force nodes to forward the onion immediately.
* You cannot force nodes to send back an error if they cannot forward the onion because of missing liquidity or other reasons.
* You cannot be sure that the recipient has the preimage to the payment hash or releases it as soon as the HTLCs of the correct amount arrived.
@ -627,7 +627,7 @@ While our user experience is that most payments find a path and settle in far le
[NOTE]
====
There are ideas out that might solve this issue to some degree by allowing the payer to abort a payment. You can find more about that under the terms `cancable payments` or `stuckless payments`. However the proposals that exist only reverse the problem as now the sender can misbehave and the recipient looses control. Another solution is to use many paths in a multipath payment and include some reduncancy and ignore the problem that a path takes longer to complete.
There are ideas out that might solve this issue to some degree by allowing the payer to abort a payment. You can find more about that under the terms `cancelable payments` or `stuckless payments`. However the proposals that exist only reverse the problem as now the sender can misbehave and the recipient looses control. Another solution is to use many paths in a multipath payment and include some redundancy and ignore the problem that a path takes longer to complete.
====
Despite these principle problems there are plausible situations in which the routing process fails and in which honest nodes can and should react.
@ -641,35 +641,35 @@ Some - but not all - of the reasons for errors could be:
* The recipient might not have issued an invoice and does not know the payment details.
* The amount of the final HTLC is too low and the recipient does not want to release the preimage.
If errors like those occure a node should send back a reply onion.
If errors like those occur a node should send back a reply onion.
The reply onion will be encrypted at each hop with the same shared secrets that have been used to construct the onion or decrypt a layer.
These shared keys are all known to the originator of the payment.
The onion innermost onion contains the error message and an HMAC for the error message.
The process makes sure that the sender of the onion and recipient of the reply can be sure that the error really originated from the node that the error messages says.
Another important step in the process of handeling errors is to abort the routing process.
Another important step in the process of handling errors is to abort the routing process.
We discussed that the sender of a payment cannot just remove the HTLC on the channel along which the sender sent the payment.
Recall for example the situation in which Alice sent and onion to Bob who set up an HTLC with Wei.
If Alices wanted to remove the HTLC with Bob this would put a financiel risk on Bob.
He fears that his HTLC with Wei still might be fullfilled meaning that he could not claim the reimbursement from Alice.
If Alice wanted to remove the HTLC with Bob this would put a financial risk on Bob.
He fears that his HTLC with Wei still might be fulfilled meaning that he could not claim the reimbursement from Alice.
Thus Bob would never agree to remove the HTLC with Alice unless he already has removed his HTLC with Wei.
If however the HTLC between Alice and Bob are set up and the HTLC between Bob and Wei are set up but Wei encounters problems with forwarding the onion it is perfectly Wei has more options than Alice.
While sending back the error Onion to Bob Wei could ask him to remove the HTLC.
Bob has no risk in removing the HTLC with Wei and Wei also has no risk as there is no downstream HTLC.
Removing an HTLC is happening very similar to adding HTLCs.
Due to the just presented argument only peers who have accepted an offered HTLC can initate the removal of HTLCs.
Due to the just presented argument only peers who have accepted an offered HTLC can initiate the removal of HTLCs.
In the case of errors peers signals that they wish to remove the HTLC by sending an `update_fail_htlc` or `update_fail_malformed_htlc` message.
These messages contain the id of an HTLC that should be removed in the next version of the commit transaction.
In the same handshake like process that was used to exchange `commitment_signed` and `revoke_and_ack` messages the new state and thus pair of commitment signatures has to be negotiated and agreed upon.
This also means while the balance of a channel thas was involved in a failed routing process will not have changed at the end it will have negotiated two new commitment transactions.
This also means while the balance of a channel that was involved in a failed routing process will not have changed at the end it will have negotiated two new commitment transactions.
Despite having the same balance it must not got back to the previous commitment transaction which did not include the HTLC as this commitment transaction was revoked.
If it was used to force close the channel the channel partner would have the ability to create a pentality transaction and get all the funds.
If it was used to force close the channel the channel partner would have the ability to create a penalty transaction and get all the funds.
==== Setteling HTLCs
==== Settling HTLCs
In the last section you you understood the error cases that can happen with onion routing via the chain of HTLCs.
You have learnt how HTLCs are removed if there is an error.
Of course HTLCs also need to be removed and the balance needs to be updated if the chain of HTLCs was successfully set up to the destination and the preimage is being released.
Not surprisingly this process is initiated with anther lightning message called `update_fulfill_htlc`.
You will remember that HTLCs are set up and supposed to be removed with a new balance for the recipient in exachange for a secrete `preimage`.
You will remember that HTLCs are set up and supposed to be removed with a new balance for the recipient in exchange for a secrete `preimage`.
Recalling the complex protocol with `commitment_signed` and `revoke_and_ack` messages you might wonder how to make this exchange `preimage` for new state atomic.
The cool thing is it doesn't have to be.
Once a channel partner with an accepted incoming HTLC knows the preimage can savely just pass it to the channel partner.
@ -677,28 +677,28 @@ That is why the `update_fulfill_htlc` message contains only the `channel_id` the
You might wonder that channel partner could now refuse to sign a new channel state by sending `commitment_siged` and `revoke_and_ack` messages.
This is not a problem though.
In that case the recipient of the offered HTLC can just go on chain by force closing the channel.
Once that has happend the preimage can be used to claim the HTLC output.
Once that has happened the preimage can be used to claim the HTLC output.
==== Some Considerations for routing nodes
Accepting and HTLC removes funds from a peer that the peer cannot utilize unless the HTLC is removed due to success or failure.
Similarly forwarding an HTLC binds some funds from your nodes payment channel until the HTLC is being removed again.
As we explained in the very beginning of the chapter engaging into the forwarding process of HTLCs does neither yield a direct risk to loose funds nor does it gain the chance to gain funds.
However the funds in jepordy could be locked for some time.
However the funds in jeopardy could be locked for some time.
In the worst case the routing process needs to be resolved on chain as the payment channel was forced close due to some other circumstances.
In that case outstanding HTLCs produce additional onchain food print and costs.
Thus there are two small economic risks envolved with the participation in the routing process.
Thus there are two small economic risks involved with the participation in the routing process.
. Higher onchain fees in case of forced channel closes due to the higher footprint of HTLCs
. Opportunity costs of locked funds. While the HTLC is active the funds cannot be used otherwise.
In economics and financial mathmatics the idea to pay another person that takes a risk is widly spread and seems reasonable.
Owners of routing nodes might want to monitor the routing behavior and opportinuties and compare them to the onchain costs and the opportunity costs in order to compute their own routing fees that they wish to charge to accept and forward HTLCs.
In economics and financial mathematics the idea to pay another person that takes a risk is widely spread and seems reasonable.
Owners of routing nodes might want to monitor the routing behavior and opportunities and compare them to the onchain costs and the opportunity costs in order to compute their own routing fees that they wish to charge to accept and forward HTLCs.
Also one should notice that HTLCs are outputs in the commitment transaction.
Lightning network protocol allows users to pay a single satoshi.
However it is impossible to set up HTLCs for this amount.
The reason is that the corresponding outputs in the commitment transaction would be below the dust limit.
Such cases are solved in practise with the following trick:
Such cases are solved in practice with the following trick:
Instead of setting up an HTLC the amount is taken from the output of the sender but not added to the output of the recipient.
Thus the HTLCs which are below the dust limit can understood as additional fees in the commitment transaction.
Most Lightning Nodes support the configuration of minimum accepted HTLC values.
Andreas M. Antonopoulos
4 years ago
committed by
GitHub
