((("onion routing", id="ix_10_onion_routing-asciidoc0", range="startofrange")))In this chapter we will describe the Lightning Network's onion routing mechanism. The invention of _onion routing_ precedes the Lightning Network by 25 years! Onion routing was invented by U.S. Navy researchers as a communications security protocol. Onion routing is most famously used by Tor, the onion-routed internet overlay that allows researchers, activists, intelligence agents, and everyone else to use the internet privately and anonymously.
In this chapter we are focusing on the "Source-based onion routing (SPHINX)" part of the Lightning protocol architecture, highlighted by an outline in the center (routing layer) of <<LN_protocol_onion_highlight>>.
In this chapter we are focusing on the "Source-based onion routing (SPHINX)" part of the Lightning protocol architecture, highlighted by an outline in the center (routing layer) of <<LN_protocol_onion_highlight>>.
[[LN_protocol_onion_highlight]]
.Onion routing in the Lightning protocol suite
@ -15,7 +15,7 @@ Each of the intermediary nodes can only "peel" one layer and see who is next in
The Lightning Network uses an implementation of onion routing protocol based on pass:[<em>Sphinx</em>],footnote:[George Danezis and Ian Goldberg, "Sphinx: A Compact and Provably Secure Mix Format," in _IEEE Symposium on Security and Privacy_ (New York: IEEE, 2009), 269–282.] developed in 2009 by George Danezis and Ian Goldberg.
The implementation of onion routing in the Lightning Network is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md[BOLT #4: Onion Routing Protocol].
The implementation of onion routing in the Lightning Network is defined in https://github.com/lightning/bolts/blob/master/04-onion-routing.md[BOLT #4: Onion Routing Protocol].
=== A Physical Example Illustrating Onion Routing
@ -33,7 +33,7 @@ There may be many paths that make it possible for Alice to reach Dina. We will e
====
[role="pagebreak-before"]
As a reminder, the path selected by Alice is shown in <<alice_dina_path>>.
As a reminder, the path selected by Alice is shown in <<alice_dina_path>>.
((("onion routing","peeling the layers")))Alice now has an envelope that says "To Bob" on the outside. It represents an encrypted message that only Bob can open (decrypt). Alice will now begin the process by sending this to Bob. The entire process is shown in <<sending_nested_envelopes>>.
((("onion routing","peeling the layers")))Alice now has an envelope that says "To Bob" on the outside. It represents an encrypted message that only Bob can open (decrypt). Alice will now begin the process by sending this to Bob. The entire process is shown in <<sending_nested_envelopes>>.
[[sending_nested_envelopes]]
.Sending the envelopes
@ -106,7 +106,7 @@ Now that we know the terminology used in Lightning onion routing, let's restate
==== Alice Selects the Path
((("onion routing","selecting a path")))From <<routing>> we know that Alice will send a 50,000 satoshi payment to Dina via Bob and Chan. This payment is transmitted via a series of HTLCs, as shown in <<alice_dina_htlc_path>>.
((("onion routing","selecting a path")))From <<routing>> we know that Alice will send a 50,000 satoshi payment to Dina via Bob and Chan. This payment is transmitted via a series of HTLCs, as shown in <<alice_dina_htlc_path>>.
[[alice_dina_htlc_path]]
.Payment path with HTLCs from Alice to Dina
@ -124,7 +124,7 @@ From the announcements and updates, Alice knows the following information about
In practice, other information is also exchanged, such as the largest (`htlc_maximum_msat`) and smallest (`htlc_minimum_msat`) HTLCs a channel will carry, but these aren't used as directly during onion route construction as the preceding fields are.
This information is used by Alice to identify the nodes, channels, fees, and timelocks for the following detailed path, shown in <<alice_dina_path_detail>>.
This information is used by Alice to identify the nodes, channels, fees, and timelocks for the following detailed path, shown in <<alice_dina_path_detail>>.
[[alice_dina_path_detail]]
.A detailed path constructed from gossiped channel and node information
@ -138,7 +138,7 @@ Alice already knows her own channel to Bob and therefore doesn't need this info
[NOTE]
====
Both formats are specified in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-structure[BOLT #4: Onion Routing Protocol, Packet Structure].
Both formats are specified in https://github.com/lightning/bolts/blob/master/04-onion-routing.md#packet-structure[BOLT #4: Onion Routing Protocol, Packet Structure].
====
Alice will start building the hop data from the end of the path backwards: Dina, Chan, then Bob.
Alice serializes it in TLV format, as shown (simplified) in <<dina_onion_payload>>.
Alice serializes it in TLV format, as shown (simplified) in <<dina_onion_payload>>.
[[dina_onion_payload]]
.Dina's payload is constructed by Alice
@ -186,7 +186,7 @@ amt_to_forward: 50,000,000
outgoing_cltv_value: 700,018
----
Alice serializes this payload in TLV format, as shown (simplified) in <<chan_onion_payload>>.
Alice serializes this payload in TLV format, as shown (simplified) in <<chan_onion_payload>>.
[[chan_onion_payload]]
.Chan's payload is constructed by Alice
@ -211,7 +211,7 @@ Similarly, Chan expects a timelock delta of 20 blocks. So Chan's incoming HTLC m
Fees and timelock delta expectations for a channel are set by the difference between incoming and outgoing HTLCs. Since the incoming HTLC is created by the _preceding node_, the fee and timelock delta is set in the onion payload to that preceding node. Bob is told how to make an HTLC that meets Chan's fee and timelock expectations.
====
Alice serializes this payload in TLV format, as shown (simplified) in <<bob_onion_payload>>.
Alice serializes this payload in TLV format, as shown (simplified) in <<bob_onion_payload>>.
[[bob_onion_payload]]
.Bob's payload is constructed by Alice
@ -240,7 +240,7 @@ With these keys, Alice can achieve a high degree of privacy and integrity:
[WARNING]
====
Like a chopped onion, the following technical details may bring tears to your eyes. Feel free to skip to the next section if you get confused. Come back to this and read https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-construction[BOLT #4: Onion Routing, Packet Construction], if you want to learn more.
Like a chopped onion, the following technical details may bring tears to your eyes. Feel free to skip to the next section if you get confused. Come back to this and read https://github.com/lightning/bolts/blob/master/04-onion-routing.md#packet-construction[BOLT #4: Onion Routing, Packet Construction], if you want to learn more.
====
@ -256,7 +256,7 @@ filler zero bytes during Sphinx packet processing.
++pad++:: Used to generate filler bytes for padding the onion to a fixed length.
The relationship between the various keys and how they are generated is diagrammed in <<onion_keygen>>.
The relationship between the various keys and how they are generated is diagrammed in <<onion_keygen>>.
For simplicity and to avoid getting too technical, we have not included these details in the book. See the preceding links if you want to see the inner workings.
[[shared_secret]]
===== Shared secret generation
((("shared secret (ss)")))One important detail that seems almost magical is the ability for Alice to create a _shared secret_ with another node simply by knowing their public keys. ((("Diffie-Hellman Key Exchange (DHKE)", id="ix_10_onion_routing-asciidoc10", range="startofrange")))This is based on the invention of Diffie–Hellman key exchange (DH) in the 1970s that revolutionized cryptography. Lightning onion routing uses Elliptic Curve Diffie–Hellman (ECDH) on Bitcoin's +secp256k1+ curve. It's such a cool trick that we try to explain it in simple terms in <<ecdh_explained>>.
((("shared secret (ss)")))One important detail that seems almost magical is the ability for Alice to create a _shared secret_ with another node simply by knowing their public keys. ((("Diffie-Hellman Key Exchange (DHKE)", id="ix_10_onion_routing-asciidoc10", range="startofrange")))This is based on the invention of Diffie–Hellman key exchange (DH) in the 1970s that revolutionized cryptography. Lightning onion routing uses Elliptic Curve Diffie–Hellman (ECDH) on Bitcoin's +secp256k1+ curve. It's such a cool trick that we try to explain it in simple terms in <<ecdh_explained>>.
// To editor: Maybe put this in an appendix instead of a sidebar?
@ -398,7 +398,7 @@ Note that the very first hop uses the original session key generated because thi
[[wrapping_the_onion]]
=== Wrapping the Onion Layers
((("onion routing","wrapping the onion layers", id="ix_10_onion_routing-asciidoc11", range="startofrange")))The process of wrapping the onion is detailed in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-construction[BOLT #4: Onion Routing, Packet Construction].
((("onion routing","wrapping the onion layers", id="ix_10_onion_routing-asciidoc11", range="startofrange")))The process of wrapping the onion is detailed in https://github.com/lightning/bolts/blob/master/04-onion-routing.md#packet-construction[BOLT #4: Onion Routing, Packet Construction].
In this section we will describe this process at a high and somewhat simplified level, omitting certain details.
@ -421,7 +421,7 @@ The onion payload is 1,300 bytes. Each hop payload is 65 bytes or less (padded t
As each layer is "peeled off," more filler data (essentially junk) is added at the end of the onion payload so the next hop gets an onion of the same size and is once again the "first hop" in the onion.
The onion size is 1,366 bytes, structured as shown in <<onion_packet>>:
The onion size is 1,366 bytes, structured as shown in <<onion_packet>>:
1 byte:: A version byte
33 bytes:: A compressed public session key (<<session_key>>) from which the per-hop shared secret (<<shared_secret>>) can be generated without revealing Alice's identity
@ -482,18 +482,18 @@ The inner HMAC is then revealed during the inverse of the "shift and encrypt" ro
Alice starts with an empty 1,300-byte field, the fixed-length _onion payload_. Then, she fills the onion payload with a pseudorandom byte stream "filler" that is generated from the ++pad++ key.
This is shown in <<onion_payload_filler>>.
This is shown in <<onion_payload_filler>>.
[NOTE]
====
Random byte stream generation uses the ChaCha20 algorithm, as a cryptographic secure pseudorandom number generator (CSPRNG). Such an algorithm will generate a deterministic, long, nonrepeating stream of seemingly random bytes from an initial seed. The details are specified in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#pseudo-random-byte-stream[BOLT #4: Onion Routing, Pseudo Random Byte Stream].
Random byte stream generation uses the ChaCha20 algorithm, as a cryptographic secure pseudorandom number generator (CSPRNG). Such an algorithm will generate a deterministic, long, nonrepeating stream of seemingly random bytes from an initial seed. The details are specified in https://github.com/lightning/bolts/blob/master/04-onion-routing.md#pseudo-random-byte-stream[BOLT #4: Onion Routing, Pseudo Random Byte Stream].
====
[[onion_payload_filler]]
.Filling the onion payload with a random byte stream
image::images/mtln_1016.png[]
Alice will now insert Dina's hop payload into the left side of the 1,300-byte array, shifting the filler to the right and discarding anything that overflows. This is visualized in <<onion_add_dina>>.
Alice will now insert Dina's hop payload into the left side of the 1,300-byte array, shifting the filler to the right and discarding anything that overflows. This is visualized in <<onion_add_dina>>.
[[onion_add_dina]]
.Adding Dina's hop payload
@ -505,7 +505,7 @@ Next row down we see the result: the 1,300 byte onion payload contains Dina's ho
Next, Alice obfuscates the entire onion payload so that _only Dina_ can read it.
To do this, Alice generates a byte stream using the ++rho++ key (which Dina also knows). Alice uses a bitwise exclusive or (XOR) between the bits of the onion payload and the byte stream created from ++rho++. The result appears like a random (or encrypted) byte stream of 1,300 bytes length. This step is shown in <<onion_obfuscate_dina>>.
To do this, Alice generates a byte stream using the ++rho++ key (which Dina also knows). Alice uses a bitwise exclusive or (XOR) between the bits of the onion payload and the byte stream created from ++rho++. The result appears like a random (or encrypted) byte stream of 1,300 bytes length. This step is shown in <<onion_obfuscate_dina>>.
[[onion_obfuscate_dina]]
.Obfuscating the onion payload
@ -520,7 +520,7 @@ XOR is an _involutory_ function, which means that if it is applied twice, it und
Because only Alice and Dina have the ++rho++ key (derived from Alice and Dina's shared secret), only they can do this. Effectively, this encrypts the onion payload for Dina's eyes only.
Finally, Alice calculates a hash-based message authentication code (HMAC) for Dina's payload, which uses the ++mu++ key as its initialization key. This is shown in <<dina_hop_payload_hmac>>.
Finally, Alice calculates a hash-based message authentication code (HMAC) for Dina's payload, which uses the ++mu++ key as its initialization key. This is shown in <<dina_hop_payload_hmac>>.
[[dina_hop_payload_hmac]]
.Adding an HMAC integrity checksum to Dina's hop payload
@ -672,7 +672,7 @@ If Bob extracts and removes his hop payload, the remaining onion will not be 1,3
Bob generates filler in a slightly different way than Alice, but following the same general principle.
First, Bob _extends_ the onion payload by 1,300 bytes and fills them with +0+ values. Now the onion packet is 2,600 bytes long, with the first half containing the data Alice sent and the next half containing zeroes. This operation is shown in <<bob_extends>>.
First, Bob _extends_ the onion payload by 1,300 bytes and fills them with +0+ values. Now the onion packet is 2,600 bytes long, with the first half containing the data Alice sent and the next half containing zeroes. This operation is shown in <<bob_extends>>.
[[bob_extends]]
.Bob extends the onion payload by 1,300 (zero-filled) bytes
@ -707,7 +707,7 @@ In this case, Bob extracts the inner HMAC (he's already verified the integrity o
==== Bob Removes His Payload and Left-Shifts the Onion
Now Bob can remove his hop payload from the front of the onion and left-shift the remaining data. An amount of data equal to Bob's hop payload from the second-half 1,300 bytes of filler will now shift into the onion payload space. This is shown in <<bob_removes_shifts>>.
Now Bob can remove his hop payload from the front of the onion and left-shift the remaining data. An amount of data equal to Bob's hop payload from the second-half 1,300 bytes of filler will now shift into the onion payload space. This is shown in <<bob_removes_shifts>>.
Now Bob can keep the first half 1,300 bytes, and discard the extended (filler) 1,300 bytes.
@ -760,7 +760,7 @@ Bob now constructs and HTLC to send to Chan, as follows:
+payment_hash+:: The RIPEMD160(SHA-256) payment hash:
This is the same as the payment hash from Alice's HTLC.
@ -782,7 +782,7 @@ Chan repeats the exact same process as Bob:
5. Chan uses the ++rho++ key to produce 2,600 bytes.
6. Chan uses the generated byte stream to XOR and de-obfuscate the onion payload. Simultaneously, the XOR operation obfuscates the extra 1,300 zeroes, turning them into filler.
6. Chan uses the generated byte stream to XOR and de-obfuscate the onion payload. Simultaneously, the XOR operation obfuscates the extra 1,300 zeroes, turning them into filler.
7. Chan extracts the inner HMAC in the payload, which will become the outer HMAC for Dina.
@ -812,10 +812,10 @@ Errors generally fall into three categories: onion failures, node failures, and
[NOTE]
====
Unlike messages in the peer-to-peer (P2P) protocol (defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md[BOLT #2: Peer Protocol for Channel Management]), errors are not sent as P2P messages but are wrapped inside onion return packets and follow the reverse of the onion path (back-propagating).
Unlike messages in the peer-to-peer (P2P) protocol (defined in https://github.com/lightning/bolts/blob/master/02-peer-protocol.md[BOLT #2: Peer Protocol for Channel Management]), errors are not sent as P2P messages but are wrapped inside onion return packets and follow the reverse of the onion path (back-propagating).
====
Error return is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#returning-errors[BOLT #4: Onion Routing, Returning Errors].
Error return is defined in https://github.com/lightning/bolts/blob/master/04-onion-routing.md#returning-errors[BOLT #4: Onion Routing, Returning Errors].
Errors are encoded by the returning node (the one that discovered an error) in a _return packet_ as follows:
@ -831,7 +831,7 @@ The return packet HMAC verification checksum is calculated with the ++um++ key,
[TIP]
====
The ++um++ key name is the reverse of the ++mu++ name, indicating the same use but in the opposite direction (back-propagation).
The ++um++ key name is the reverse of the ++mu++ name, indicating the same use but in the opposite direction (back-propagation).
====
Next, the returning node generates an +ammag+ (inverse of the word "gamma") key and obfuscates the return packet using an XOR operation with a byte stream generated from +ammag+.
@ -845,7 +845,7 @@ Eventually, the sender (origin node) receives a return packet. It will then gene
[[failure_messages]]
==== Failure Messages
((("error return","failure messages", id="ix_10_onion_routing-asciidoc16", range="startofrange")))((("failure messages, onion routing and", id="ix_10_onion_routing-asciidoc17", range="startofrange")))The +failuremsg+ is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#failure-messages[BOLT #4: Onion Routing, Failure Messages].
((("error return","failure messages", id="ix_10_onion_routing-asciidoc16", range="startofrange")))((("failure messages, onion routing and", id="ix_10_onion_routing-asciidoc17", range="startofrange")))The +failuremsg+ is defined in https://github.com/lightning/bolts/blob/master/04-onion-routing.md#failure-messages[BOLT #4: Onion Routing, Failure Messages].
A failure message consists of a two-byte +failure code+ followed by the data applicable to that failure type.
Francisco Calderón
2 years ago