lnbook/10_onion_routing.asciidoc

[[onion_routing]]
== Onion Routing

In this chapter we will describe the Lightning Network's _Onion Routing_ mechanism. The invention of _onion routing_ precedes the Lightning Network by 25 years! Onion routing was invented by U.S. Navy researchers as a communications security protocol. Onion routing is most famously used by _Tor_, the onion routed internet overlay that allows researchers, activists, intelligence agents and everyone else to use the internet privately and anonymously.

In this chapter we are focusing on the "Source based Onion Routing (SPHINX)" part of the Lightning protocol architecture, highlighted by a double outline in the center (Routing Layer) of <<LN_protocol_onion_highlight>>:

[[LN_protocol_onion_highlight]]
.The Lightning Network Protocol Suite
image::images/LN-protocol-onion-highlight.png["The Lightning Network Protocol Suite"]

Onion routing describes a method of encrypted communication where a message sender builds successive _nested layers of encryption_ that are "peeled" off by each intermediary node, until the innermost layer is delivered to the intended recipient. The name "onion routing" describes this use of layered encryption that is peeled off one layer at a time, like the skin of an onion.

Each of the intermediary nodes can only "peel" one layer and see who is next in the communications path. Onion routing ensures that no one except the sender knows the destination or length of the communication path. Each intermediary only knows the previous and next hop.

The Lightning Network uses an implementation of onion routing protocol based on _Sphinx_footnote:[http://www0.cs.ucl.ac.uk/staff/G.Danezis/papers/sphinx-eprint.pdf[George Danezis and Ian Goldberg. Sphinx: A compact and provably secure mix format. In IEEE Symposium on Security and Privacy, pp 269–282. IEEE, 2009.]] developed in 2009 by George Danezis and Ian Goldberg.

The implementation of onion routing in the Lightning Network is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md[BOLT #4 - Onion Routing Protocol]

=== A physical example illustrating onion routing

There are many ways to describe onion routing, but one of the easiest is to use the physical equivalent of sealed envelopes. An envelope represents a layer of encryption, allowing only the named recipient to open it and read the contents.

Let's say Alice wants to send a secret letter to Dina, indirectly via some intermediaries.

==== Selecting a path

The Lightning Network uses _source routing_, which means that the payment path is selected and specified by the sender, and only the sender. In this example, our Alice's secret letter to Dina will be the equivalent of a payment. To make sure the letter reaches Dina, Alice will create a path from her to Dina, using Bob and Chan as intermediaries.

[TIP]
====
There may be many paths that make it possible for Alice to reach Dina. We will explain the process of selecting the _optimum_ path in <<path_finding>>. For now, we'll assume that the path selected by Alice uses Bob and Chan as intermediaries to get to Dina.
====

As a reminder, the path selected by Alice is shown in <<alice_dina_path>>, below:

[[alice_dina_path]]
image::images/alice_dina_path.png["Alice to Bob to Chan to Dina"]

Let's see how Alice can use this path without revealing information to intermediaries Bob and Chan.

.Source-based routing
****
Source-based routing is not how packets are typically routed on the internet today, though source routing was possible in the early days.
Internet routing is based on _packet switching_ at each intermediary routing node. An IPv4 packet, for example, includes the sender and recipient's IP address and every other IP routing node decides how to forward each packet towards the destination.
However, the lack of privacy in such a routing mechanism, where every intermediary node sees the sender and recipient, make this a poor choice for use in a payment network.
****

==== Building the layers

Alice starts by writing a secret letter to Dina.  She then seals the letter inside an envelope and writes "To Dina" on the outside (see <<dina_envelope>>). The envelope represents encryption with Dina's public key, so that only Dina can open the envelope and read the letter.

[[dina_envelope]]
.Dina's secret letter, sealed in an envelope
image::images/dina_envelope.png[Dina's secret letter, sealed in an envelope]

Dina's letter will be delivered to Dina by Chan, who is immediately before Dina in the "path". So, Alice puts Dina's envelope inside an envelope addressed to Chan (see <<chan_envelope>>). The only part that Chan can read is the destination (routing instructions): "To Dina". Sealing this inside an envelope addressed to Chan represents encrypting it with Chan's public key so that only Chan can read the envelope address. Chan still can't open Dina's envelope. All he sees is the instructions on the outside (the address).

[[chan_envelope]]
.Chan's envelope, containing Dina's sealed envelope
image::images/chan_envelope.png[Chan's envelope, containing Dina's sealed envelope]

Now, this letter will be delivered to Chan by Bob. So Alice puts it inside an envelope addressed to Bob (see <<bob_envelope>>). As before, the envelope represents a message encrypted to Bob that only Bob can read. Bob can only read the outside of Chan's envelope (the address), so he knows to send it to Chan.

[[bob_envelope]]
.Bob's envelope, containing Chan's sealed envelope
image::images/bob_envelope.png[Bob's envelope, containing Chan's sealed envelope]

Now, if we could look through the envelopes (with X-rays!) we would see the envelopes nested one inside the other, as shown in <<nested_envelopes>>, below:

[[nested_envelopes]]
.Nested envelopes
image::images/nested_envelopes.png[Nested envelopes]

==== Peeling the layers

Alice now has an envelope that says "To Bob" on the outside. It represents an encrypted message that only Bob can open (decrypt). Alice will now begin the process by sending this to Bob. The entire process is shown in <<sending_nested_envelopes>> below:

[[sending_nested_envelopes]]
.Sending the envelopes
image::images/sending_nested_envelopes.png[Sending the envelopes]

As you can see, Bob receives the envelope from Alice. He knows it came from Alice, but doesn't know if Alice is the original sender or just someone forwarding envelopes. He opens it to find an envelope inside that says "To Chan". Since this is addressed to Chan, Bob can't open it. He doesn't know what's inside it and doesn't know if Chan is getting a letter or another envelope to forward. Bob doesn't know if Chan is the ultimate recipient or not. Bob forwards the envelope to Chan.

Chan receives the envelope from Bob. He doesn't know that it came from Alice. He doesn't know if Bob is an intermediary or the sender of a letter. Chan opens the envelope and finds another envelope inside addressed "To Dina", which he can't open. Chan forwards it to Dina, not knowing if Dina is the final recipient.

Dina receives an envelope from Chan. Opening it she finds a letter inside, so now she knows she's the intended recipient of this message. She reads the letter, knowing that none of the intermediaries know where it came from and no one else has read her secret letter!

This is the essence of onion routing. The sender wraps a message in layers, specifying exactly how it will be routed and preventing any of the intermediaries from gaining any information about the path or payload. Each intermediary peels one layer, sees only a forwarding address and doesn't know anything other than the previous and next hop in the path.

Now, let's look at the details of the onion routing implementation in the Lightning Network.

=== Onion routing HTLCs

Onion routing in the Lightning Network appears complex at first glance, but once you understand the basic concept is really quite simple.

From a practical perspective, Alice is telling every intermediary node what HTLC to set up with the next node in the path.

The first node, which is the payment sender or Alice in our example, is called the _-_origin node_.  The last node, which is the payment recipient or Dina in our example, is called the _final node_.

Each intermediary node, or Bob and Chan in our example, is called a _hop_. Every hop must set up an _outgoing HTLC_ to the next hop. The information communicated to each hop by Alice is called the _hop payload_ or _hop data_. The message that is routed from Alice to Dina is called an _onion_ and consists of encrypted _hop payload_ or _hop data_ messages encrypted to each hop.

Now that we know the terminology used in Lightning Onion Routing, let's restate Alice's task: Alice must construct an _onion_ with _hop data_, telling each _hop_ how to construct an _outgoing HTLC_ in order to send a payment to the _final node_ (Dina).

==== Alice selects the path

From <<routing>> we know that Alice will send a 50,000 satoshi payment to Dina via Bob and Chan. This payment is transmitted via a series of HTLCs, as shown in <<alice_dina_htlc_path>>, below:

[[alice_dina_htlc_path]]
.Payment path with HTLCs from Alice to Dina
image::images/alice-dina-htlc.png[Payment path with HTLCs from Alice to Dina]

As we see in <<gossip>>, Alice is able to construct this path to Dina because Lightning nodes announce their channels to the entire Lightning Network using the _Lightning Gossip Protocol_. After the initial channel announcement, Bob and Chan each sent out an additional "channel update" message with their routing fee and timelock expectations for payment routing.

From the announcements and updates, Alice knows the following information about the channels between Bob, Chan and Dina:

* A +short_channel_id+ (short channel ID) for each channel, that Alice can use to reference the channel, when constructing the path

* An +cltv_expiry_delta+ (timelock delta) which Alice can add to the expiry time for each HTLC

* A +fee_base_msat+ and +fee_proportional_millionths+ which Alice can use to calculate the total routing fee expected by that node for relay on that channel.

This information is used by Alice to identify the nodes, channels, fees, and timelocks for the following detailed path, shown in <<alice_dina_path_detail>>:

[[alice_dina_path_detail]]
.A detailed path constructed from gossiped channel and node information
image::images/alice_dina_path_detail.png[A path constructed from gossiped channel and node information]

Alice already knows her own channel to Bob, and therefore doesn't need this info to construct the path. Note also that Alice didn't need a channel update from Dina, because she has the update from Chan for that last channel in the path.

==== Alice constructs the payloads

There are two possible formats that Alice can use for the information communicated to each hop: A legacy fixed-length format called the _hop data_ and a more flexible Type-Length-Value (TLV) based format called the _hop payload_. The TLV message format is explained in more detail in <<tlv>>. It offers flexibility by allowing fields to be added to the protocol at will. Both formats are specified in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-structure[BOLT #4 - Onion Routing - Packet Structure]

Alice will start building the hop data from the end of the path backwards: Dina, Chan, Bob.

==== Final node payload for Dina

Alice first build the payload that will be delivered to Dina. Dina will not be constructing an "outgoing HTLC", because Dina is the final node and payment recipient. For this reason, the payload for Dina is different that all the others, but only Dina will know this since it will be encrypted in the innermost layer of the onion. Essentially, this is the "secret letter to Dina" we saw in our physical envelope example.

The hop payload for Dina must match the information in the invoice generated by Dina for Alice and will contain (at least) the following fields in Type-Lenght-Value (TLV) format:

amt_to_forward:: The amount of this payment in milli-satoshis. If this is only one part of a multi-part payment, the amount is less than the total. Otherwise, this is a single full payment and it is equal to the invoice amount and +total_msat+ value.

outgoing_cltv_value:: The payment expiry timelock set to the value +min_final_cltv_expiry+ in the invoice.

payment_secret:: A special 256-bit secret value from the invoice, allowing Dina to recognize this incoming payment.

total_msat:: The total amount matching the invoice. This may be omitted if there is only one part, in which case it is assume to match +amt_to_forward+ and must equal the invoice amount.

The invoice Alice received from Dina specified the amount as 50,000 satoshis, which is 50,000,000 milli-satoshis. Dina specified the minimum expiry for the payment +min_final_cltv_expiry+ as 18 blocks (3 hours, given 10-minute on average Bitcoin blocks). At the time Alice is attempting to make the payment, let's say the Bitcoin blockchain has recorded 700,000 blocks. So Alice must set the +outgoing_cltv_value+ to a *minimum* block height of 700,018.

Alice constructs the hop payload for Dina as follows:

----
amt_to_forward : 50,000,000
outgoing_cltv_value: 700,018
payment_secret: fb53d94b7b65580f75b98f10...03521bdab6d519143cd521d1b3826
total_msat: 50,000,000
----

Alice serializes it in TLV format as shown (simplified) <<dina_onion_payload>> below:

[[dina_onion_payload]]
.Dina's payload is constructed by Alice
image::images/dina_onion_payload.png[Dina's payload is constructed by Alice]

==== Hop payload for Chan

Next, Alice constructs the hop payload for Chan. This will tell Chan how to setup an outgoing HTLC to Dina.

The hop payload for Chan includes three fields: +short_channel_id+, +amt_to_forward+ and +outgoing_cltv_value+:

----
short_channel_id: 010002010a42be
amt_to_forward: 50,000,000
outgoing_cltv_value: 700,018
----

Alice serializes this payload in TLV format, as shown (simplified) in <<chan_onion_payload>> below:

[[chan_onion_payload]]
.Chan's payload is constructed by Alice
image::images/chan_onion_payload.png[Chan's payload is constructed by Alice]

==== Hop payload for Bob

Finally, Alice constructs the hop payload for Bob, which also contains the same three fields as the hop payload for Chan, but with different values:

----
short_channel_id: 000004040a61f0
amt_to_forward: 50,100,000
outgoing_cltv_value: 700,038
----

As you can see, the +amt_to_forward+ field is 50,100,000 milli-satoshis, or 50,100 satoshis. That's because Chan expects a fee of 100 satoshis to route a payment to Dina. In order for Chan to "earn" that routing fee, Chan's incoming HTLC must be 100 satoshis more than Chan's outgoing HTLC. Since Chan's incoming HTLC is Bob's outgoing HTLC, the instructions to Bob reflect the fee Chan earns. In simple terms, Bob needs to be told to send 50,100 satoshi to Chan, so that Chan can send 50,000 satoshi and keep 100 satoshi.

Similarly, Chan expects a timelock delta of 20 blocks. SO Chan's incoming HTLC must expire 20 blocks *later* than Chan's outgoing HTLC. To achieve this, Alice tells Bob to make his outgoing HTLC to Chan expire at block height 700,038 - 20 blocks later than Chan's HTLC to Dina.

[TIP]
====
Fees and timelock delta expectations for a channel are set by the difference between incoming and outgoing HTLCs. Since the incoming HTLC is created by the _preceding node_, the fee and timelock delta is set in the onion payload to that preceding node. Bob is told how to make an HTLC that meets Chan's fee and timelock expectations.
====

Alice serializes this payload in TLV format, as shown (simplified) in <<bob_onion_payload>> below:

[[bob_onion_payload]]
.Bob's payload is constructed by Alice
image::images/bob_onion_payload.png[Bob's payload is constructed by Alice]

==== Finished hop payloads

Alice has now built the three hop payloads that will be wrapped in an onion. A simplified view of the payloads is shown in <<onion_hop_payloads>>, below:

[[onion_hop_payloads]]
.Hop payloads for all the hops
image::images/onion_hop_payloads.png[Hop payloads for all the hops]

==== Key generation

Alice must now generate several keys that will be used to encrypt the various layers in the onion.

Remember the goals of onion routing, that Alice can achieve with these keys:

* Alice can encrypt each layer of the onion so that only the intended recipient can read it.
* Every intermediary can check that the message is not modified.
* No one in the path will know who sent this onion or where it is going. Alice doesn't reveal her identity as the sender or Dina's identity as the recipient of the payment.
* Each hop only learns about the previous and next hop.
* No one can know how long the path is, or where  in the path they are.


[WARNING]
====
Like a chopped onion, the following technical details may bring tears to your eyes. Feel free to skip to the next section if you get confused. Come back to this and read BOLT #4 if you want to learn more.
====

To avoid revealing her identity, Alice does not use her own node's public key in building the onion. Instead, Alice creates a temporary 32-byte (256-bit) key called the _session key_. This serves as a temporary "identity" *for this onion only*. From this session key, Alice will build all the other keys that will be used in this onion.

In addition, Alice generates several ephemeral keys and shared secret keys for each hop. Finally, Alice also generates keys to produce a random sequence that will be used as "filler" to pad the onion and obfuscate its length.

The key generation, random byte generation, ephemeral keys and how they are used in packet construction are specified in three sections of BOLT #4:

* https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#key-generation[Key Generation]
* https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#pseudo-random-byte-stream[Random Byte Stream]
* https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-construction[Packet Construction]

For simplicity and to avoid getting too technical, we have not included these details in the book. See the links above if you want to see the inner workings.

One important detail that seems almost magical is the ability for Alice to create a _shared secret_ with another node simply by knowing their public keys. This is based on the invention of Diffie-Hellman key exchange (DH) in the 1970s that revolutionized cryptography. Lightning Onion Routing uses Elliptic Curve Diffie-Hellman (ECDH) on Bitcoin's +secp256k1+ curve. It's such a cool trick that we try to explain it in simple terms in <<ecdh_explained>>


// To editor: Maybe put this in an appendix instead of a sidebar?

.Elliptic Curve Diffie-Hellman (ECDH) explained
****
Assume Alice's private key is +a+ and Bob's private key is +b+. Using the Elliptic Curve, they multiply each private key by the generator point +G+ to produce their public keys +A+ and +B+ respectively:

+A = aG+

+B = bG+

Now Alice and Bob can create a shared secret +ss+, a value that they can both calculate independently without exchanging any information, such that

+ss = aB = bA+

Follow along, as we demonstrate the math that proves this is possible:

ss

= aB, calculated by Alice who knows +a+ and +B+

= a(bG), because we know B = bG

= (ab)G, because of associativity

= (ba)G, because the curve is an abelian group

= b(aG), because of associativity

= bA, can be calculated by Bob who knows +b+ and +A+

We have therefore shown that

ss = aB = bA

Alice can multiple her private key with Bob's public key to calculate +ss+

Bob can multiply his private key with Alice's public key to calculate +ss+

Thus, they will both get the same result which they can use as a shared key to symmetrically encrypt secrets between the two of them without communicating the shared secret.

****

==== Wrapping the onion layers