In this chapter we will describe the Lightning Network's _Onion Routing_ mechanism. The invention of _onion routing_ precedes the Lightning Network by 25 years! Onion routing was invented by U.S. Navy researchers as a communications security protocol. Onion routing is most famously used by _Tor_, the onion routed internet overlay that allows researchers, activists, intelligence agents and everyone else to use the internet privately and anonymously.
In this chapter we are focusing on the "Source based Onion Routing (SPHINX)" part of the Lightning protocol architecture, highlighted by a double outline in the center (Routing Layer) of <<LN_protocol_onion_highlight>>:
Onion routing describes a method of encrypted communication where a message sender builds successive _nested layers of encryption_ that are "peeled" off by each intermediary node, until the innermost layer is delivered to the intended recipient. The name "onion routing" describes this use of layered encryption that is peeled off one layer at a time, like the skin of an onion.
Each of the intermediary nodes can only "peel" one layer and see who is next in the communications path. Onion routing ensures that no one except the sender knows the destination or length of the communication path. Each intermediary only knows the previous and next hop.
The Lightning Network uses an implementation of onion routing protocol based on _Sphinx_footnote:[http://www0.cs.ucl.ac.uk/staff/G.Danezis/papers/sphinx-eprint.pdf[George Danezis and Ian Goldberg. Sphinx: A compact and provably secure mix format. In IEEE Symposium on Security and Privacy, pp 269–282. IEEE, 2009.]] developed in 2009 by George Danezis and Ian Goldberg.
The implementation of onion routing in the Lightning Network is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md[BOLT #4 - Onion Routing Protocol]
There are many ways to describe onion routing, but one of the easiest is to use the physical equivalent of sealed envelopes. An envelope represents a layer of encryption, allowing only the named recipient to open it and read the contents.
Let's say Alice wants to send a secret letter to Dina, indirectly via some intermediaries.
==== Selecting a path
The Lightning Network uses _source routing_, which means that the payment path is selected and specified by the sender, and only the sender. In this example, our Alice's secret letter to Dina will be the equivalent of a payment. To make sure the letter reaches Dina, Alice will create a path from her to Dina, using Bob and Chan as intermediaries.
[TIP]
====
There may be many paths that make it possible for Alice to reach Dina. We will explain the process of selecting the _optimum_ path in <<path_finding>>. For now, we'll assume that the path selected by Alice uses Bob and Chan as intermediaries to get to Dina.
====
As a reminder, the path selected by Alice is shown in <<alice_dina_path>>, below:
[[alice_dina_path]]
image::images/alice_dina_path.png["Alice to Bob to Chan to Dina"]
Let's see how Alice can use this path without revealing information to intermediaries Bob and Chan.
.Source-based routing
****
Source-based routing is not how packets are typically routed on the internet today, though source routing was possible in the early days.
Internet routing is based on _packet switching_ at each intermediary routing node. An IPv4 packet, for example, includes the sender and recipient's IP address and every other IP routing node decides how to forward each packet towards the destination.
However, the lack of privacy in such a routing mechanism, where every intermediary node sees the sender and recipient, make this a poor choice for use in a payment network.
****
==== Building the layers
Alice starts by writing a secret letter to Dina. She then seals the letter inside an envelope and writes "To Dina" on the outside (see <<dina_envelope>>). The envelope represents encryption with Dina's public key, so that only Dina can open the envelope and read the letter.
[[dina_envelope]]
.Dina's secret letter, sealed in an envelope
image::images/dina_envelope.png[Dina's secret letter, sealed in an envelope]
Dina's letter will be delivered to Dina by Chan, who is immediately before Dina in the "path". So, Alice puts Dina's envelope inside an envelope addressed to Chan (see <<chan_envelope>>). The only part that Chan can read is the destination (routing instructions): "To Dina". Sealing this inside an envelope addressed to Chan represents encrypting it with Chan's public key so that only Chan can read the envelope address. Chan still can't open Dina's envelope. All he sees is the instructions on the outside (the address).
Now, this letter will be delivered to Chan by Bob. So Alice puts it inside an envelope addressed to Bob (see <<bob_envelope>>). As before, the envelope represents a message encrypted to Bob that only Bob can read. Bob can only read the outside of Chan's envelope (the address), so he knows to send it to Chan.
Now, if we could look through the envelopes (with X-rays!) we would see the envelopes nested one inside the other, as shown in <<nested_envelopes>>, below:
Alice now has an envelope that says "To Bob" on the outside. It represents an encrypted message that only Bob can open (decrypt). Alice will now begin the process by sending this to Bob. The entire process is shown in <<sending_nested_envelopes>> below:
[[sending_nested_envelopes]]
.Sending the envelopes
image::images/sending_nested_envelopes.png[Sending the envelopes]
As you can see, Bob receives the envelope from Alice. He knows it came from Alice, but doesn't know if Alice is the original sender or just someone forwarding envelopes. He opens it to find an envelope inside that says "To Chan". Since this is addressed to Chan, Bob can't open it. He doesn't know what's inside it and doesn't know if Chan is getting a letter or another envelope to forward. Bob doesn't know if Chan is the ultimate recipient or not. Bob forwards the envelope to Chan.
Chan receives the envelope from Bob. He doesn't know that it came from Alice. He doesn't know if Bob is an intermediary or the sender of a letter. Chan opens the envelope and finds another envelope inside addressed "To Dina", which he can't open. Chan forwards it to Dina, not knowing if Dina is the final recipient.
Dina receives an envelope from Chan. Opening it she finds a letter inside, so now she knows she's the intended recipient of this message. She reads the letter, knowing that none of the intermediaries know where it came from and no one else has read her secret letter!
This is the essence of onion routing. The sender wraps a message in layers, specifying exactly how it will be routed and preventing any of the intermediaries from gaining any information about the path or payload. Each intermediary peels one layer, sees only a forwarding address and doesn't know anything other than the previous and next hop in the path.
Now, let's look at the details of the onion routing implementation in the Lightning Network.
The first node, which is the payment sender or Alice in our example, is called the _-_origin node_. The last node, which is the payment recipient or Dina in our example, is called the _final node_.
Each intermediary node, or Bob and Chan in our example, is called a _hop_. Every hop must set up an _outgoing HTLC_ to the next hop. The information communicated to each hop by Alice is called the _hop payload_ or _hop data_. The message that is routed from Alice to Dina is called an _onion_ and consists of encrypted _hop payload_ or _hop data_ messages encrypted to each hop.
Now that we know the terminology used in Lightning Onion Routing, let's restate Alice's task: Alice must construct an _onion_ with _hop data_, telling each _hop_ how to construct an _outgoing HTLC_ in order to send a payment to the _final node_ (Dina).
From <<routing>> we know that Alice will send a 50,000 satoshi payment to Dina via Bob and Chan. This payment is transmitted via a series of HTLCs, as shown in <<alice_dina_htlc_path>>, below:
[[alice_dina_htlc_path]]
.Payment path with HTLCs from Alice to Dina
image::images/alice-dina-htlc.png[Payment path with HTLCs from Alice to Dina]
As we see in <<gossip>>, Alice is able to construct this path to Dina because Lightning nodes announce their channels to the entire Lightning Network using the _Lightning Gossip Protocol_. After the initial channel announcement, Bob and Chan each sent out an additional "channel update" message with their routing fee and timelock expectations for payment routing.
* A +fee_base_msat+ and +fee_proportional_millionths+ which Alice can use to calculate the total routing fee expected by that node for relay on that channel.
This information is used by Alice to identify the nodes, channels, fees, and timelocks for the following detailed path, shown in <<alice_dina_path_detail>>:
.A detailed path constructed from gossiped channel and node information
image::images/alice_dina_path_detail.png[A path constructed from gossiped channel and node information]
Alice already knows her own channel to Bob, and therefore doesn't need this info to construct the path. Note also that Alice didn't need a channel update from Dina, because she has the update from Chan for that last channel in the path.
==== Alice constructs the payloads
There are two possible formats that Alice can use for the information communicated to each hop: A legacy fixed-length format called the _hop data_ and a more flexible Type-Length-Value (TLV) based format called the _hop payload_. The TLV message format is explained in more detail in <<tlv>>. It offers flexibility by allowing fields to be added to the protocol at will. Both formats are specified in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-structure[BOLT #4 - Onion Routing - Packet Structure]
Alice will start building the hop data from the end of the path backwards: Dina, Chan, Bob.
==== Final node payload for Dina
Alice first build the payload that will be delivered to Dina. Dina will not be constructing an "outgoing HTLC", because Dina is the final node and payment recipient. For this reason, the payload for Dina is different that all the others, but only Dina will know this since it will be encrypted in the innermost layer of the onion. Essentially, this is the "secret letter to Dina" we saw in our physical envelope example.
The hop payload for Dina must match the information in the invoice generated by Dina for Alice and will contain (at least) the following fields in Type-Lenght-Value (TLV) format:
amt_to_forward:: The amount of this payment in milli-satoshis. If this is only one part of a multi-part payment, the amount is less than the total. Otherwise, this is a single full payment and it is equal to the invoice amount and +total_msat+ value.
outgoing_cltv_value:: The payment expiry timelock set to the value +min_final_cltv_expiry+ in the invoice.
payment_secret:: A special 256-bit secret value from the invoice, allowing Dina to recognize this incoming payment.
total_msat:: The total amount matching the invoice. This may be omitted if there is only one part, in which case it is assume to match +amt_to_forward+ and must equal the invoice amount.
The invoice Alice received from Dina specified the amount as 50,000 satoshis, which is 50,000,000 milli-satoshis. Dina specified the minimum expiry for the payment +min_final_cltv_expiry+ as 18 blocks (3 hours, given 10-minute on average Bitcoin blocks). At the time Alice is attempting to make the payment, let's say the Bitcoin blockchain has recorded 700,000 blocks. So Alice must set the +outgoing_cltv_value+ to a *minimum* block height of 700,018.
As you can see, the +amt_to_forward+ field is 50,100,000 milli-satoshis, or 50,100 satoshis. That's because Chan expects a fee of 100 satoshis to route a payment to Dina. In order for Chan to "earn" that routing fee, Chan's incoming HTLC must be 100 satoshis more than Chan's outgoing HTLC. Since Chan's incoming HTLC is Bob's outgoing HTLC, the instructions to Bob reflect the fee Chan earns. In simple terms, Bob needs to be told to send 50,100 satoshi to Chan, so that Chan can send 50,000 satoshi and keep 100 satoshi.
Similarly, Chan expects a timelock delta of 20 blocks. SO Chan's incoming HTLC must expire 20 blocks *later* than Chan's outgoing HTLC. To achieve this, Alice tells Bob to make his outgoing HTLC to Chan expire at block height 700,038 - 20 blocks later than Chan's HTLC to Dina.
Fees and timelock delta expectations for a channel are set by the difference between incoming and outgoing HTLCs. Since the incoming HTLC is created by the _preceding node_, the fee and timelock delta is set in the onion payload to that preceding node. Bob is told how to make an HTLC that meets Chan's fee and timelock expectations.
Alice serializes this payload in TLV format, as shown (simplified) in <<bob_onion_payload>> below:
[[bob_onion_payload]]
.Bob's payload is constructed by Alice
image::images/bob_onion_payload.png[Bob's payload is constructed by Alice]
==== Finished hop payloads
Alice has now built the three hop payloads that will be wrapped in an onion. A simplified view of the payloads is shown in <<onion_hop_payloads>>, below:
[[onion_hop_payloads]]
.Hop payloads for all the hops
image::images/onion_hop_payloads.png[Hop payloads for all the hops]
==== Key generation
Alice must now generate several keys that will be used to encrypt the various layers in the onion.
Remember the goals of onion routing, that Alice can achieve with these keys:
* Alice can encrypt each layer of the onion so that only the intended recipient can read it.
* Every intermediary can check that the message is not modified.
* No one in the path will know who sent this onion or where it is going. Alice doesn't reveal her identity as the sender or Dina's identity as the recipient of the payment.
* Each hop only learns about the previous and next hop.
* No one can know how long the path is, or where in the path they are.
[WARNING]
====
Like a chopped onion, the following technical details may bring tears to your eyes. Feel free to skip to the next section if you get confused. Come back to this and read BOLT #4 if you want to learn more.
The basis for all the keys used in the onion, is a _shared secret_ that Alice and Bob can both generate independently using the Elliptic Curve Diffie-Hellman (ECDH) algorithm. From the shared secret (ss), they can independently generate four additional keys named rho, mu, um and pad:
rho:: Used to obfuscate the per-hop data between Alice and Bob.
mu:: Used in the Hash-Based Message Authentication Code (HMAC) for integrity/authenticity verification.
um:: Used in error reporting.
pad:: Used to generate filler bytes for padding the onion to a fixed length.
The relationship between the various keys and how they are generated is shown in the diagram <<onion_keygen>>, below:
To avoid revealing her identity, Alice does not use her own node's public key in building the onion. Instead, Alice creates a temporary 32-byte (256-bit) key called the _session private key_ and corresponding _session public key_. This serves as a temporary "identity" and key *for this onion only*. From this session key, Alice will build all the other keys that will be used in this onion.
For simplicity and to avoid getting too technical, we have not included these details in the book. See the links above if you want to see the inner workings.
One important detail that seems almost magical is the ability for Alice to create a _shared secret_ with another node simply by knowing their public keys. This is based on the invention of Diffie-Hellman key exchange (DH) in the 1970s that revolutionized cryptography. Lightning Onion Routing uses Elliptic Curve Diffie-Hellman (ECDH) on Bitcoin's +secp256k1+ curve. It's such a cool trick that we try to explain it in simple terms in <<ecdh_explained>>
// To editor: Maybe put this in an appendix instead of a sidebar?
Assume Alice's private key is +a+ and Bob's private key is +b+. Using the Elliptic Curve, they multiply each private key by the generator point +G+ to produce their public keys +A+ and +B+ respectively:
+A = aG+
+B = bG+
Now Alice and Bob can create a shared secret +ss+, a value that they can both calculate independently without exchanging any information, such that
+ss = aB = bA+
Follow along, as we demonstrate the math that proves this is possible:
ss
= aB, calculated by Alice who knows +a+ and +B+
= a(bG), because we know B = bG
= (ab)G, because of associativity
= (ba)G, because the curve is an abelian group
= b(aG), because of associativity
= bA, can be calculated by Bob who knows +b+ and +A+
We have therefore shown that
ss = aB = bA
Alice can multiple her private key with Bob's public key to calculate +ss+
Bob can multiply his private key with Alice's public key to calculate +ss+
Thus, they will both get the same result which they can use as a shared key to symmetrically encrypt secrets between the two of them without communicating the shared secret.
We've mentioned the fact that none of the "hop" nodes know how long the path is, or where they are in the path. How is this possible?
If you have a set of directions, even if encrypted, can't you tell how far you are from the beginning or end simply by looking at *where* in the list of directions you are?
The "trick" used in onion routing is to always make the path (the list of directions), the same length for every node. This is achieved by keeping the onion packet the same length at every step. As each layer is peeled, it is replaced with filler data (essentially junk) and the next hop gets an onion of the same size.
The onion size is 1366 bytes made of:
* 1 byte: A version byte
* 33 bytes: A compression public session key (<<session_key>>) from which the per-hop shared secret (<<shared_secret>>) can be generated without revealing Alice's identity
* 1300 bytes: The actual _onion payload_ containing the instructions for each hop
The process of wrapping the onion is detailed in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-construction[BOLT #4 - Onion Routing - Packet Construction].
In this section we will describe this process at a high-level and somewhat simplified - omitting certain details.
For each hop Alice essentially repeats the same process for each hop:
1. Alice generates the per-hop shared secret and the rho, mu, and pad keys
1. Alice generates 1300 bytes of filler and fills the 1300-byte onion payload field with this filler.
1. Alice calculates the HMAC for the hop payload
1. Alice calculates the length of the hop payload + HMAC + space to store the length itself
1. Alice _right shifts_ the onion payload by the calculated space needed to fit the hop payload. The rightmost "filler" data is discarded, making enough space on the left for the payload.
1. Alice inserts the length + hop payload + HMAC at the front of the payload field in the space made from shifting the filler.
1. Alice uses the _rho_ key to generate a 1300 byte one-time-pad.
1. Alice obfuscates the entire onion payload by XOR-ing with the bytes generated from rho.
1. Alice calculates the HMAC of the onion payload
1. Alice adds the session public key (so that the hop can calculate the shared secret)
1. Alice adds the version number.
Next, Alice repeats the process. The new keys are calculated, the onion payload is shifted (dropping more junk), the new hop payload is added to the front and the whole onion payload encrypted with the rho byte-stream for the next hop.
