Archives/lnbook
2
0
Fork 0
mirror of https://github.com/lnbook/lnbook synced 2024-11-04 18:00:26 +00:00
Code Issues Projects Releases Wiki Activity

Appendices and README update

Browse Source
This commit is contained in:
Andreas M. Antonopoulos 2021-08-04 09:34:13 +02:00
parent 78da6f5b50
commit e21944239c
4 changed files with 122 additions and 105 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
README.md
View File

@ -13,7 +13,9 @@ The book is suitable for technical readers with an understanding of the fundamen
## Status
The current status of the book is "RELEASE PREP". See below for status of specific chapters and read the contribution guide to learn how and where to contribute.
The current status of the book is "IN PRODUCTION": ONLY COPY-EDIT CONTRIBUTIONS ARE ACCEPTED.
See below for status of specific chapters and read the contribution guide to learn how and where to contribute.
### Legend
@ -45,10 +47,13 @@ The current status of the book is "RELEASE PREP". See below for status of specif
| [LN Security and Privacy](13_security_privacy_ln.asciidoc) | ################ | :heavy_check_mark: |
| APPENDICES | APPENDICES | APPENDICES |
| [A1 - Bitcoin Fundamentals Review](appendix-bitcoin-fundamentals-review.asciidoc) | ########### | :heavy_check_mark: |
| [An - License Notices](appendix_license_notices.asciidoc) | # | :heavy_check_mark: |
| [A2 - Payment Requests (Invoices)](appendix_payment_requests.asciidoc) | #### | :heavy_check_mark: |
| [A3 - Wire Protocol](appendix_wire_protocol.asciidoc) | ################# | :heavy_check_mark: |
| [A4 - Encrypted Message Transport (Brontide)](appendix_encrypted_transport.asciidoc) | ########## | :heavy_check_mark: |
| [A5 - License Notices](appendix_license_notices.asciidoc) | # | :heavy_check_mark: |
Total Word Count: 110718
Total Word Count: 125288
Target Word Count: 100,000-120,000

49
contrib/brontide.asciidoc → appendix_encrypted_transport.asciidoc
View File

@ -1,6 +1,11 @@
= Brontide: Lightning's Encrypted Message Transport
[appendix]
[[brontide]]
[[encrypted_message_transport]]
== Lightning's Encrypted Message Transport (Brontide)
== Intro
In this chapter we will review the Lightning Network's _Encrypted Message Transport_, also known as the _Brontide Protocol_, which allows peers to establish end-to-end encrypted communication, authentication and integrity checking.
=== Introduction
Unlike the vanilla Bitcoin P2P network, every node in the Lightning Network is
identified by a unique public key which serves as it identity. By default, this
@ -18,7 +23,7 @@ custom encrypted message transport Lightning uses, commonly referred to as
"Brontide" (more on that later) can be dropped into any context that requires
encrypted communication between two parties.
== The Channel Graph as Decentralized Public Key Infrastructure
=== The Channel Graph as Decentralized Public Key Infrastructure
As we learned in the chapter on multi-hop forwarding, very node has a long-term
identity that is used as the identifier for a vertex during path finding and
@ -48,7 +53,7 @@ Lightning network is able to significantly simply its encrypted transport
protocol as it doesn't need to deal with all the complexities that come along
with TLS, the Transport Layer Security protocol.
== Why Not TLS?
=== Why Not TLS?
Readers familiar with the TLS system may be wondering at this point: why wasn't
TLS used in spite of the drawbacks of the existing PKI system? It is indeed a
@ -80,7 +85,7 @@ to continual scrutiny of the TLS protocol over decades. In a way, the existence
of Noise allows the community to effective "start over", with a more compact,
simplified protocol that retains all the added benefits of TLS.
== The Noise Protocol Framework
=== The Noise Protocol Framework
The Noise Protocol Framework is a modern, extensible, and flexible message
encryption protocol designed by the creators of the Signal protocol. The Signal
@ -114,7 +119,7 @@ Lightning Network, the flavor of Noise use will be referred to from here on as
"Brontide". A brontide is a low billowing noise, similar to what one would hear
during a thunderstorm when very far away.
=== Noise Protocol Handshakes
==== Noise Protocol Handshakes
The Noise protocol is extremely flexible in that it advertises several
handshakes, each with different security and privacy properties for a would be
@ -129,7 +134,7 @@ handshake. Instead, a clever series of Elliptic-Curve Diffie-Hellman (ECDH) and
Message Authentication Code (MAC) checks are used to authenticate the
responder.
=== Handshake Notation & Protocol Flow
==== Handshake Notation & Protocol Flow
Each handshakes typically consist of several steps. At each step some
(possibly) encrypted material is sent to the opposite party, an ECDH (or
@ -148,12 +153,12 @@ refer to a new ephemeral key. ECDH operations between two keys are notated as
the concatenation of two keys. As an example, `ee` represents an ECDH operation
between two ephemeral keys.
== Brontide: Lightning's P2P Encryption
=== Brontide: Lightning's P2P Encryption
=== High-Level Overview
==== High-Level Overview
Using the notation laid out earlier, we can succinctly describe the `Noise_XK`
as follows:
as follows:
```
Noise_XK(s, rs):
<- rs
@ -196,7 +201,7 @@ variant, the ASCII string is hashed into a digest, which is used to initialize
the starting handshake state. In the context of Brontide, the ASCII string
describing the protocol is: `Noise_XK_secp256k1_ChaChaPoly_SHA256`.
=== Brontide: A Handshake in Three Acts
==== Brontide: A Handshake in Three Acts
The handshake portion of Brontide can be see prated into three distinct "acts".
The entire handshake takes 1.5 round trips between the initiator and responder.
@ -229,7 +234,7 @@ functions that will operate on the handshake and messaging state. When
describing the handshake protocol, we'll use these variables in a manner
similar to pseudo-code in order to reduce the verbosity of the explanation of
each step in the protocol. We'll define the _functional_ primitives of the
handshake as:
handshake as:
* `ECDH(k, rk)`: performs an Elliptic-Curve Diffie-Hellman operation using
`k`, which is a valid `secp256k1` private key, and `rk`, which is a valid public key
@ -263,7 +268,7 @@ handshake as:
* `a || b` denotes the concatenation of two byte strings `a` and `b`
==== Handshake Session State Initialization
===== Handshake Session State Initialization
Before starting the handshake process, both sides need to initialize the
starting state that they'll use to advance the handshake process. To start,
@ -295,13 +300,13 @@ does indeed know the public key of the responder.
Bitcoin's compressed format:
* `h = SHA-256(h || ls.pub.serializeCompressed())`
==== Handshake Acts
===== Handshake Acts
After the initial handshake initialization, we can begin the actual execution
of the handshake process. The Brontide handshake is compromised of a series of
three messages sent between the initiator and responder, hence referred to as
"acts". As each act is a single message sent between the parties, a handshake
is completed in a total of 1.5 round trips (0.5 for each act).
is completed in a total of 1.5 round trips (0.5 for each act).
The first act completes the initial portion of the incremental Triple Diffie
Hellman key exchange (using a new ephemeral key generated by the initiator),
@ -313,7 +318,7 @@ act, the initiator transmits their long-term static public key to the
responder, and executes the final DH operation to mix that into the final
resulting shared secret.
===== Act One
====== Act One
```
-> e, es
@ -374,7 +379,7 @@ and 16 bytes for the `poly1305` tag.
* The received ciphertext is mixed into the handshake digest. This step serves
to ensure the payload wasn't modified by a MITM.
===== Act Two
====== Act Two
```
<- e, ee
@ -432,7 +437,7 @@ for the `poly1305` tag.
* The received ciphertext is mixed into the handshake digest. This step serves
to ensure the payload wasn't modified by a MITM.
===== Act Three
====== Act Three
```
-> s, se
@ -502,7 +507,7 @@ construction, and 16 bytes for a final authenticating tag.
10. `rn = 0, sn = 0`
* The sending and receiving nonces are initialized to 0.
==== Transport Message Encryption
===== Transport Message Encryption
At the conclusion of Act Three, both sides have derived the encryption keys, which
will be used to encrypt and decrypt messages for the remainder of the
@ -545,7 +550,7 @@ The structure of packets on the wire resembles the following:
The prefixed message length is encoded as a 2-byte big-endian integer, for a
total maximum packet length of `2 + 16 + 65535 + 16` = `65569` bytes.
===== Encrypting and Sending Messages
====== Encrypting and Sending Messages
In order to encrypt and send a Lightning message (`m`) to the network stream,
given a sending key (`sk`) and a nonce (`sn`), the following steps are
@ -566,7 +571,7 @@ completed:
* The nonce `sn` MUST be incremented after this step.
5. Send `lc || c` over the network buffer.
===== Receiving and Decrypting Messages
====== Receiving and Decrypting Messages
In order to decrypt the _next_ message in the network stream, the following
steps are completed:
@ -583,7 +588,7 @@ steps are completed:
plaintext packet `p`.
* The nonce `rn` MUST be incremented after this step.
==== Lightning Message Key Rotation
===== Lightning Message Key Rotation
Changing keys regularly and forgetting previous keys is useful to prevent the
decryption of old messages, in the case of later key leakage (i.e. backwards

37
contrib/payment-requests.asciidoc → appendix_payment_requests.asciidoc
View File

@ -1,8 +1,13 @@
= Lightning Payment Requests
[appendix]
[[invoices]]
[[lightning_payment_requests]]
== Lightning Payment Requests
== Intro
In this chapter we will look at _Lightning Payment Requests_ or as they are more commonly known _Lightning Invoices_.
As we've learned in prior chapters, minimally two prides of data are required
=== Introduction
As we've learned in prior chapters, minimally two pieces of data are required
to complete a Lightning payment: a payment hash, and a destination. As
`SHA-256` is used in the Lightning Network to implement HTLCs, this information
requires 32-bytes in order to communicate. Destinations on the other hand are
@ -15,10 +20,10 @@ to complete a payment from receiver to sender. In practice, more than just the
payment hash and destination are communicated in a payment request in order to
make the encoding more fully feature.
== Lightning Payment Requests vs Bitcoin Addresses
=== Lightning Payment Requests vs Bitcoin Addresses
A commonly asked question when people first encounter a Lightning Payment
request is: why can't a normal static address format be used instead?
request is: why can't a normal static address format be used instead?
In order to answer this question, one must first internalize how Lightning
differs from base layer Bitcoin as a payment method. Compared to a Bitcoin
@ -27,7 +32,7 @@ address which may be used to make a potentially unbounded number of payments
payment request should only ever be used *once*. This is due to the fact that
sending a payment to a Bitcoin address essentially uses a public key
cryptosystem to "encode" the payment in a manner that only the true "owner" of
that Bitcoin address can redeem it.
that Bitcoin address can redeem it.
In contrast, in order to complete a Lightning payment, the recipient must
reveal a "secret" to the entire payment route including the sender. This can be
@ -40,7 +45,7 @@ been reveled, all nodes in the path will keep it around _forever_, then rather
than forward the HTLC in order to collect a routing fee if the payment is
completed, they can simply _settle_ the payment at that instance and gain the
entire payment amount in return. As a result, it's unsafe to ever use a payment
request more than once.
request more than once.
As we'll see later in the book, there exist new variants of the original
Lightning Payment request that allow the sender to -reuse them as many times as
@ -51,7 +56,7 @@ a mechanism that allows a sender to typically request a new payment request
from the receiver, then an interactive protocol can be used in order to allow a
degree of payment request re-use.
== BOLT 11: Lightning Payment Request Serialization & Interpretation
=== BOLT 11: Lightning Payment Request Serialization & Interpretation
In this section, we'll describe the mechanism used to encode the set of
information required to complete a payment on the Lightning Network. As
@ -60,16 +65,16 @@ information required to complete a payment. However in practice, more
information such as time-lock information, payment request expiration, and
possibly an on-chain fallback address are also communicated.
=== Payment Request Encoding in Practice
==== Payment Request Encoding in Practice
First, let's examine what a real payment request looks like in practice. The
following is a valid payment request that could have been used to complete a
payment on the mainnet Lightning Network at time it was created:
payment on the mainnet Lightning Network at time it was created:
```
lnbc2500u1pvjluezpp5qqqsyqcyq5rqwzqfqqqsyqcyq5rqwzqfqqqsyqcyq5rqwzqfqypqdq5xysxxatsyp3k7enxv4jsxqzpuaztrnwngzn3kdzw5hydlzf03qdgm2hdq27cqv3agm2awhz5se903vruatfhq77w3ls4evs3ch9zw97j25emudupq63nyw24cg27h2rspfj9srp
```
=== The Human Readable Prefix
==== The Human Readable Prefix
Looking at the string, we can tease out a portion that we can parse with our
eyes, while the rest of it just looks like a random set of strings. The part
@ -120,7 +125,7 @@ A full list of the currently defined multipliers is a follows:
=== Bech32 & the Data Segment
==== Bech32 & the Data Segment
If the "unreadable" portion of looks familiar, then that's because it uses the
very same encoding scheme as segwit compatible Bitcoin addresses use today,
@ -128,7 +133,7 @@ namely `bech32`. Describing the `bech32` encoding scheme is outside the scope
of this chapter. In brief, it's a sophisticated way to encode short strings
that has very good error correction as well as detection properties.
The data portion can be separated into 3 sections:
The data portion can be separated into 3 sections:
* The timestamp.
* Zero or more tagged key-value pairs.
@ -143,7 +148,7 @@ Similar to the TLV format we learned about in Chapter XXX, the BOLT 11 invoice
format uses a series of extensible key-value pairs to encode information
needed to satisfy a payment. As key-value pairs are used, it's easy for add
new values in the future if a new payment type or additional
requirement/functionality is introduced.
requirement/functionality is introduced.
Finally a signature is included ed that covers the entire invoice signed by the
destination of the payment. This signature allows the sender to verify that the
@ -155,7 +160,7 @@ allows public key extraction. When verifying the signature, the verifies
extracts the public key, then verifies that against the public key included in
the invoice.
==== Tagged Invoice Fields
===== Tagged Invoice Fields
The tagged invoice fields are encoded in the main "body" of the invoice. These
fields represent different key=value pairs that express either additional
@ -163,7 +168,7 @@ information that may help complete the payment, or information which is
_required_ to complete the payment. As a slight variant of `bech32` is
utilized, each of these fields are actually in the "base 5" domain.
A given tag field is comprised of 3 components:
A given tag field is comprised of 3 components:
* The `type` of the field (5 bits).
* The `length` of the data of the field (10 bits)

130
contrib/wire_protocol.asciidoc → appendix_wire_protocol.asciidoc
View File

@ -1,6 +1,8 @@
= Wire Protocol: Framing & Extensibility
[appendix]
[[wire_protocol]]
== Wire Protocol: Framing & Extensibility
== Intro
=== Introduction
In this chapter, we'll dive into the wire protocol of the Lightning network,
and also cover all the various extensibility levers that have been built into
@ -10,7 +12,7 @@ to being able to write a custom wire protocol parser, a reader of this chapter
will gain a deep understanding with respect of the various upgrade mechanisms
that have been built into the protocol.
== Wire Framing
=== Wire Framing
First, we being by describing the high level structure of the wire _framing_
within the protocol. When we say framing, we mean the way that the bytes are
@ -30,7 +32,7 @@ framing, we assume the encryption layer has already been stripped away (when
decoding), or that we haven't yet encrypted the set of bytes before we send
them on the wire (encoding).
=== High-Level Wire Framing
==== High-Level Wire Framing
With that said, we're ready to being describe the high-level schema used to
encode messages on the wire:
@ -58,7 +60,7 @@ nodes are able to provide information in the wire messages that older nodes
feature combined with a very flexible wire message extensibility format also
allows the protocol to achieve _forwards_ compatibility as well.
=== Type Encoding
==== Type Encoding
With this high level background provided, we'll now start at the most primitive
layer: parsing primitive types. In addition to encoding integers, the Lightning
@ -76,39 +78,39 @@ high-level routine used to encode/decode the type.
.High-level message types
[options="header"]
|================================================================================
| High Level Type | Framing | Comment
| `node_alias` | A 32-byte fixed-length byte slice. | When decoding, reject if contents are not a valid UTF-8 string.
| `channel_id` | A 32-byte fixed-length byte slice that maps an outpoint to a 32 byte value. | Given an outpoint, one can convert it to a `channel_id` by taking the txid of the outpoint and XOR'ing it with the index (interpreted as the lower 2 bytes).
| `short_chan_id` | An unsigned 64-bit integer (`uint64`) | Composed of the block height (24 bits), transaction index (24 bits), and output index (16 bits) packed into 8 bytes.
| `milli_satoshi` | An unsigned 64-bit integer (`uint64`) | Represents 1000th of a satoshi.
| `satoshi` | An unsigned 64-bit integer (`uint64`) | The based unit of bitcoin.
| `satoshi` | An unsigned 64-bit integer (`uint64`) | The based unit of bitcoin.
| `pubkey` | An secp256k1 public key encoded in _compressed_ format, occupying 33 bytes. | Occupies a fixed 33-byte length on the wire.
| High Level Type | Framing | Comment
| `node_alias` | A 32-byte fixed-length byte slice. | When decoding, reject if contents are not a valid UTF-8 string.
| `channel_id` | A 32-byte fixed-length byte slice that maps an outpoint to a 32 byte value. | Given an outpoint, one can convert it to a `channel_id` by taking the txid of the outpoint and XOR'ing it with the index (interpreted as the lower 2 bytes).
| `short_chan_id` | An unsigned 64-bit integer (`uint64`) | Composed of the block height (24 bits), transaction index (24 bits), and output index (16 bits) packed into 8 bytes.
| `milli_satoshi` | An unsigned 64-bit integer (`uint64`) | Represents 1000th of a satoshi.
| `satoshi` | An unsigned 64-bit integer (`uint64`) | The based unit of bitcoin.
| `satoshi` | An unsigned 64-bit integer (`uint64`) | The based unit of bitcoin.
| `pubkey` | An secp256k1 public key encoded in _compressed_ format, occupying 33 bytes. | Occupies a fixed 33-byte length on the wire.
| `sig` | An ECDSA signature of the secp256k1 Elliptic Curve. | Encoded as a _fixed_ 64-byte byte slice, packed as `R \|\| S`
| `uint8` | An 8-bit integer. |
| `uint16` | A 16-bit integer. |
| `uint64` | A 64-bit integer. |
| `[]byte` | A variable length byte slice. | Prefixed with a 16-bit integer denoting the length of the bytes.
| `color_rgb` | RGB color encoding. | Encoded as a series if 8-bit integers.
| `net_addr` | The encoding of a network address. | Encoded with a 1 byte prefix that denotes the type of address, followed by the address body.
| `[]byte` | A variable length byte slice. | Prefixed with a 16-bit integer denoting the length of the bytes.
| `color_rgb` | RGB color encoding. | Encoded as a series if 8-bit integers.
| `net_addr` | The encoding of a network address. | Encoded with a 1 byte prefix that denotes the type of address, followed by the address body.
|================================================================================
In the next section, we'll describe the structure of each of the wire messages
including the prefix type of the message along with the contents of its message
body.
=== Type Length Value (TLV) Message Extensions
==== Type Length Value (TLV) Message Extensions
Earlier in this chapter we mentioned that messages can be up to 65 KB in size,
and if while parsing a messages, extra bytes are left over, then those bytes
are to be _ignored_. At an initial glance, this requirement may appear to be
somewhat arbitrary, however upon close inspection it's actually the case that
this requirement allows for de-coupled de-synchronized evolution of the Lighting
this requirement allows for de-coupled de-synchronized evolution of the Lighting
Protocol itself. We'll opine further upon this notion towards the end of the
chapter. First, we'll turn our attention to exactly what those "extra bytes" at
the end of a message can be used for.
==== The Protcol Buffer Message Format
===== The Protcol Buffer Message Format
The Protocol Buffer (protobuf) message serialization format started out as an
internal format used at Google, and has blossomed into one of the most popular
@ -130,7 +132,7 @@ there're types/fields that it doesn't understand, then it simply _ignores_
them. This allows old clients and new clients to _co-exist_, as all clients can
parse _some_ portion of the newer message format.
==== Forwards & Backwards Compatibility
===== Forwards & Backwards Compatibility
Protobufs are extremely popular amongst developers as they have built in
support for both _forwards_ and _backwards_ compatibility. Most developers are
@ -148,7 +150,7 @@ clients that don't update can still use Bitcoin, and if they encounters any
transactions they don't understand, then they simply ignore them as their funds
aren't using those new features.
==== Lighting's Protobuf Inspired Message Extension Format: `TLV`
===== Lighting's Protobuf Inspired Message Extension Format: `TLV`
In order to be able to upgrade messages in both a forwards and backwards
compatible manner, in addition to feature bits (more on that later), the LN
@ -167,7 +169,7 @@ In comparison lnd's implementation of the TLV message format weighs in at only
With the necessary background presented, we're now ready to describe the TLV
format in detail. A TLV message extension is said to be a _stream_ of
individual TLV records. A single TLV record has three components: the type of
the record, the length of the record, and finally the opaque value of the
the record, the length of the record, and finally the opaque value of the
record:
* `type`: An integer representing the name of the record being encoded.
@ -192,7 +194,7 @@ size of the integer to be encoded.
* If the value is _less than_ `0xfd` (`253`):
* Then the discriminant isn't really used, and the encoding is simply the
integer itself.
integer itself.
* This value allows us to encode very small integers with no additional
overhead
@ -230,7 +232,7 @@ acceptable within teh context of Lighting, was many messages contain a
signature of the message digest. If it's possible for a message to be encoded
in two different ways, then it would be possible to break the authentication of
a signature inadvertently by re-encoding a message using a slightly different
set of bytes on the wire.
set of bytes on the wire.
In order to ensure that all encoded messages are canonical, the following
constraints are defined when encoding:
@ -240,7 +242,7 @@ constraints are defined when encoding:
* All records must _minimally encode_ the `type` and `length` fields. In
orther woards, the smallest BigSIze representation for an integer MUST be
used at all times.
used at all times.
* Each `type` may only appear _once_ within a given TLV stream.
@ -251,7 +253,7 @@ chapter cone we talked about how the Lighting Protocol is upgraded in practice
and in theory.
=== Wire Messages
==== Wire Messages
In this section, well outline the precise structure of each of the wire
messages within the protocol. We'll do so in two parts: first we'll enumerate
@ -264,18 +266,18 @@ First, we'll lead with an enumeration of all the currently defined types:
.Message Types
[options="header"]
|==============================================================================
| Type Integer | Message Name | Category
| 16 | `init` | Connection Establishment
| 17 | `error` | Error Communication
| 18 | `ping` | Connection Liveness
| Type Integer | Message Name | Category
| 16 | `init` | Connection Establishment
| 17 | `error` | Error Communication
| 18 | `ping` | Connection Liveness
| 19 | `pong` | Connection Liveness
| 32 | `open_channel` | Channel Funding
| 33 | `accept_channel` | Channel Funding
| 34 | `funding_created` | Channel Funding
| 35 | `funding_signed` | Channel Funding
| 36 | `funding_locked` | Channel Funding + Channel Operation
| 38 | `shutdown` | Channel Closing
| 39 | `closing_signed` | Channel Closing
| 38 | `shutdown` | Channel Closing
| 39 | `closing_signed` | Channel Closing
| 128 | `update_add_htlc` | Channel Operation
| 130 | `update_fulfill_hltc` | Channel Operation
| 131 | `update_fail_htlc` | Channel Operation
@ -283,7 +285,7 @@ First, we'll lead with an enumeration of all the currently defined types:
| 133 | `revoke_and_ack` | Channel Operation
| 134 | `update_fee` | Channel Operation
| 135 | `update_fail_malformed_htlc` | Channel Operation
| 136 | `channel_reestablish` | Channel Operation
| 136 | `channel_reestablish` | Channel Operation
| 256 | `channel_announcement` | Channel Announcement
| 257 | `node_announcement` | Channel Announcement
| 258 | `channel_update` | Channel Announcement
@ -297,7 +299,7 @@ First, we'll lead with an enumeration of all the currently defined types:
In the above table, the `Category` field allows us to quickly categonize a
message based on its functionality within the protocol itself. At a high level,
we place a message into one of 8 (non exhaustive) buckets including:
we place a message into one of 8 (non exhaustive) buckets including:
* *Connection Establishment*: Sent when a peer to peer connection is first
established. Also used in order to negotiate the set of _feature_ supported
@ -307,7 +309,7 @@ we place a message into one of 8 (non exhaustive) buckets including:
protocol level errors to each other.
* *Connection Liveness*: Used by peers to check that a given transport
connection is still live.
connection is still live.
* *Channel Funding*: Used by peers to create a new payment channel. This
process is also known as the channel funding process.
@ -330,7 +332,7 @@ this roadmap laid out, we'll now visit each message category in order to define
the precise structure and semantics of all defined messages within the LN
protocol.
==== Connection Establishment Messages
===== Connection Establishment Messages
Messages in this category are the very first message sent between peers once
they establish a transport connection. At the time of writing of this chapter,
@ -339,12 +341,12 @@ The `init` message is sent by _both_ sides of the connection once it has been
first established. No other messages are to be sent before the `init` message
has been sent by both parties.
The structure of the `init` message is defined as follows:
The structure of the `init` message is defined as follows:
`init` message:
* type: `16`
* fields:
* fields:
* `uint16`: `global_features_len`
* `global_features_len*byte`: `global_features`
* `uint16`: `features_len`
@ -368,12 +370,12 @@ they're used later in the chapter.
An `init` message is then examined by a peer in order to determine if the
connection is well defined based on the set of optional and required feature
bits advertised by both sides.
bits advertised by both sides.
An optional feature means that a peer knows about a feature, but they don't
consider it critical to the operation of a new connection. An example of one
would be something like the ability to understand the semantics of a newly
added field to an existing message.
added field to an existing message.
On the other hand, required feature indicate that if the other peer doesn't
know about the feature, then the connection isn't well defined. An example of
@ -381,7 +383,7 @@ such a feature would be a theoretical new channel type within the protocol: if
your peer doesn't know of this feature, they you don't want to keep the
connection as they're unable to open your new preferred channel type.
==== Error Communication Messages
===== Error Communication Messages
Messages in this category are used to send connection level errors between two
peers. As we'll see later, another type of error exists in the protocol: an
@ -392,7 +394,7 @@ broadcast the latest signed commitment)
The sole message in this category is the `error` message:
* type: `17`
* fields:
* fields:
* `channel_id`: `chan_id`
* `uint16`: `data_len`
* `data_len*byte`: `data`
@ -408,14 +410,14 @@ have a channel with may indicate that the channel cannot continue without
manual intervention, so the only option at that point is to force close the
channel by broadcasting the latest commitment state of the channel.
==== Connection Liveness
===== Connection Liveness
Messages in this section are used to probe to determine if a connection is
still live or not. As the LN protocol somewhat abstracts over the underlying
transport being used to transmit the messages, a set of protocol level `ping`
and `pong` messages are defined.
First, the `ping` message:
First, the `ping` message:
* type: `18`
* fields:
@ -430,7 +432,7 @@ Next it's companion, the `pong` message:
* `uint16`: `pong_body_len`
* `ping_body_len*bytes`: `pong_body`
A `ping` message can be sent by either party at any time.
A `ping` message can be sent by either party at any time.
The `ping` message includes a `num_pong_bytes` field that is used to instruct
the receiving node with respect to how large the payload it sends in its `pong`
@ -449,7 +451,7 @@ default the LN uses an _encrypted_ transport, so a passive network monitor
cannot read the plaintext bytes, thus only has timing and packet sizes to go
off of.
==== Channel Funding
===== Channel Funding
As we go on, we enter into the territory of the core messages that govern the
functionality and semantics of the Lightning Protocol. In this section, we'll
@ -494,7 +496,7 @@ commitment transaction.
At the time of writing of this chapter, a single TLV record is defined within
the set of optional TLV records that may be appended to the end of a defined
message:
message:
* type: 0
* data: `upfront_shutdown_script`
@ -543,7 +545,7 @@ channel.
In response, the initiator will send the `funding_created` message:
* type: `34`
* fields:
* fields:
* `32*byte`: `temp_chan_id`
* `32*byte`: `funding_txid`
* `uint16`: `funding_output_index`
@ -588,7 +590,7 @@ Once the funding transaction obtains a `minimum_depth` number of confirmations,
then the `funding_locked` message is to be sent by both sides. Only after this
message has been received, and sent can the channel being to be used.
==== Channel Closing
===== Channel Closing
* type: `38`
* fields:
@ -701,7 +703,7 @@ transactions:
* `uint32`: `feerate_per_kw`
This message can only be sent by the initiator of the channel they're the ones
that will pay for the commitment fee of the channel as along as it's open.
that will pay for the commitment fee of the channel as along as it's open.
The `update_fail_malformed_htlc` is sent to remove a corrupted HTLC:
@ -720,7 +722,7 @@ then it won't be able to decrypt the packet. As a result it also can't properly
forward the HTLC, therefore it'll send this message to signify that the HTLC
has been corrupted somewhere along the route back to the sender.
==== Channel Announcement
===== Channel Announcement
Messages in this category are used to announce components of the Channel Graph
authenticated data structure to the wider network. The Channel Graph has a
@ -793,7 +795,7 @@ an active channel edge within the Channel graph:
* `uint64`: `htlc_minimum_msat`
* `uint32`: `fee_base_msat`
* `uint32`: `fee_proportional_millionths`
* `uint16`: `htlc_maximum_msat`
* `uint16`: `htlc_maximum_msat`
In addition to being able to enable/disable a channel this message allows a
node to update it's routing fees as well as other fields that shape the type of
@ -815,7 +817,7 @@ advertise their channel to the network, then they'll each send the
`announce_signatures` message which allows both sides to emplace the 4
signatures required to generate a `announce_signatures` message.
==== Channel Graph Syncing
===== Channel Graph Syncing
The `query_short_chan_ids` allows a peer to obtain the channel information
related to a series of short channel IDs:
@ -857,7 +859,7 @@ of a channel in the chain, a node on the network can use a block height as a
sort of _cursor_ to seek through the chain in order to discover a set of newly
opened channels. In Chapter XXX, we'll go through the protocol peers use to
sync the channel graph in more detail.
The `reply_channel_range` message is the response to `query_channel_range` and
includes the set of short channel IDs for known channels within that range:
@ -875,7 +877,7 @@ As a response to `query_channel_range`, this message sends back the set of
channels that were opened within that range. This process can be repeated with
the requester advancing their cursor further down the chain in order to
continue syncing the Channel Graph.
The `gossip_timestamp_range` message allows a peer to start receiving new
incoming gossip messages on the network:
* type: `265:
@ -890,7 +892,7 @@ also set the `first_timestamp` and `timestamp_range` fields if they wish to
receive a backlog of updates they may have missed while they were down.
== Feature Bits & Protocol Extensibility
=== Feature Bits & Protocol Extensibility
As the Lighting Network is a decentralized system, no one entity can enforce a
protocol change or modification upon all the users of the system. This
@ -912,7 +914,7 @@ extensibility mechanisms within the network which can be used to upgrade the
network partially or fully in a decoupled, desynchronized, decentralized
manner.
=== Feature Bits as an Upgrade Discoverability Mechanism
==== Feature Bits as an Upgrade Discoverability Mechanism
An astute reader may have noticed the various locations that "feature bits" are
included within the Lightning Protocol. A "feature bit" is a bitfield that can
@ -971,13 +973,13 @@ not. The feature bits within the `init` message all peers to understand kif
they can maintain a connection, and also which features are negotiated for the
lifetime of a given connection.
=== Utilizing TLV Records for Forwards+Backwards Compatibility
==== Utilizing TLV Records for Forwards+Backwards Compatibility
As we learned earlier in the chapter, Type Length Value, or TLV records can be
used to extend messages in a forwards and backwards compatible manner.
Overtime, these records have been used to _extend_ existing messages without
breaking the protocol by utilizing the "undefined" area within a message beyond
that set of known bytes.
that set of known bytes.
As an example, the original Lighting Protocol didn't have a concept of the
_largest_ HTLC that could traverse through a channel as dictated by a routing
@ -986,7 +988,7 @@ message to phase in such a concept over time. Peers that held a
`channel_update` that set such a field but didn't even know the upgrade existed
where unaffected by the change, but may see their HTLCs rejected if they are
beyond the said limit. Newer peers on the other hand are able to parse, verify
and utilize the new field at will.
and utilize the new field at will.
Those familiar with the concept of soft-forks in Bitcoin may now see some
similarities between the two mechanism. Unlike Bitcoin consensus-level
@ -996,7 +998,7 @@ network need to understand new upgrade in order to start utilizing it without
any permission. Commonly these tow peers may be the receiver and sender of a
payment, or it may the initiator and responder of a new payment channel.
=== A Taxonomy of Upgrade Mechanisms
==== A Taxonomy of Upgrade Mechanisms
Rather than there being a single widely utilized upgrade mechanism within the
network (such as soft forks for base layer Bitcoin), there exist a wide
@ -1004,7 +1006,7 @@ gradient of possible upgrade mechanisms within the Lighting Network. In this
section, we'll enumerate the various upgrade mechanism within the network, and
provide a real-world example of their usage in the past.
==== Internal Network Upgrades
===== Internal Network Upgrades
We'll start with the upgrade type that requires the most extra protocol-level
coordination: internal network upgrades. An internal network upgrade is
@ -1035,7 +1037,7 @@ are required to forward the payment. However, if a new upgrade type instead
changed the _HTLC_ format, then the entire path would need to be upgraded,
otherwise the payment wouldn't be able to be fulfilled.
==== End to End Upgrades
===== End to End Upgrades
To contrast the internal network upgrade, in this section we'll describe the
_end to end_ network upgrade. This upgrade type differs from the internal
@ -1065,7 +1067,7 @@ end encrypted, this payment type was safe, since none of the intermediate nodes
are able to fully unwrap the onion to uncover the payment pre-image that
corresponded to that payment hash.
==== Channel Construction Level Updates
===== Channel Construction Level Updates
The final broad category of updates within the network are those that happen at
the channel construction level, but which don't modify the structure of the