lnbook/09_channel_operation.asciidoc

[[channel_operation]]
== Channel Operation and Payment Forwarding

In this chapter we will bring together payment channels and Hash Time-Locked Contracts (HTLCs). In <<payment_channels>> we explained the way Alice and Bob construct a payment channel between their two nodes. We also looked at the commitment and penalty mechanisms that secure the payment channel. In <<routing>> we looked at Hash Time-Locked Contracts (HTLCs) and how these can be used to route a payment across a path made of multiple payment channels. In this chapter we bring the two concepts together by looking at how HTLCs are managed on each payment channel, how the HTLCs are committed to the channel state and how they are settled to update the channel balances.

Specifically, we will be discussing "Adding, Settling, Failing HTLCs" and the "Channel State Machine" that form the overlap between the Peer 2 Peer layer and the Routing layer, as highlighted by a double outline in <<LN_protocol_channelops_highlight>>:

[[LN_protocol_channelops_highlight]]
.The Lightning Network Protocol Suite
image::images/LN_protocol_channelops_highlight.png["The Lightning Network Protocol Suite"]


=== Local (single channel) vs Routed (multiple channels)

Even though it is possible to send payments across a payment channel simply by updating the channel balances and creating new commitment transactions, the Lightning protocol uses HTLCs even for "local" payments across a payment channel. The reason for this is to maintain the same protocol design regardless of whether a payment is only one hop (across a single payment channel), or several hops (routed across multiple payment channels).

By maintaining the same abstraction for both local and remote, we not only simplify the protocol design but we also improve privacy. For the recipient of a payment there is no discernible difference between a payment made directly by their channel partner and a payment forwarded by their channel partner on behalf of someone else.

=== Forwarding payments and updating commitments with HTLCs

We will revisit our example from <<routing>>, to demonstrate how the HTLCs from Alice to Dina gets committed to each payment channel. As you recall in our example, Alice is paying Dina 50,000 satoshis, by routing an HTLC via Bob and Chan. The network is shown in <<alice_dina_htlc_2>> below:

[[alice_dina_htlc_2]]
.Alice pays Dina with an HTLC routed via Bob and Chan
image::images/alice-dina-htlc-1.png["Alice pays Dina with an HTLC routed via Bob and Chan"]

We will focus on the payment channel between Alice and Bob and review the messages and transactions that they use to process this HTLC.

==== HTLC and commitment message flow

The message flow between Alice and Bob (and also between any pair of channel partners) is shown in <<HTLC_commitment_message_flow>>:

[[HTLC_commitment_message_flow]]
.The message flow for HTLC commitment between channel partners
image::images/HTLC_commitment_message_flow_1.png["The message flow for HTLC commitment between channel partners"]

We've already seen the +commitment_signed+ and +revoke_and_ack+ in <<payment_channels>>. Now we will see how HTLCs fit into the commitment scheme. The two new messages are +update_add_htlc+ which Alice uses to ask Bob to add an HTLC and +update_fulfill_htlc+ which Bob uses to redeem the HTLC once he has received the payment secret (Dina's secret).

=== Forwarding payments with HTLCs

Alice and Bob start with a payment channel that has 70,000 satoshi balance on each side.

As we saw in <<payment_channels>> this means that Alice and Bob have negotiated and each hold commitment transactions. These commitment transactions are asymmetric, delayed and revocable, and look like the example in <<alice_bob_commitment_txs_1>> below:

[[alice_bob_commitment_txs_1]]
.Alice and Bob's initial commitment transactions
image:images/alice_bob_commitment_txs_1.png["Alice and Bob's initial commitment transactions"]

==== Adding an HTLC

Alice wants Bob to accept an HTLC worth 50,200 satoshis to forward to Dina. To do so, Alice must send the details of this HTLC, including the payment hash and amount to Bob. Bob will also need to know where to forward it, which is something we discuss in detail in <<onion_routing>>.

To add the HTLC, Alice starts the flow we saw in <<HTLC_commitment_message_flow>> by sending the +update_add_htlc+ message to Bob.

[[update_add_htlc]]
==== The +update_add_htlc+ message

Alice sends the `update_add_HTLC` Lightning message to Bob. This message is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#adding-an-htlc-update_add_htlc[BOLT #2 - Peer Protocol - update_add_HTLC], and is shown below:

[[update_add_HTLC_message_fields]]
.The update_add_HTLC message
====
----
[channel_id:channel_id]
[u64:id]
[u64:amount_msat]
[sha256:payment_hash]
[u32:cltv_expiry]
[1366*byte:onion_routing_packet]
----
====

+channel_id+:: This is the channel that Alice has with Bob that she wants to add the HTLC. Remember that Alice and Bob may have multiple channels with each other.

+id+:: This is an HTLC counter and starts at +0+ for the first HTLC offered to Bob by Alice and is incremented for each subsequent offered HTLC

+amount_msat+:: This is the amount (value) of the HTLC in milli-satoshis. In our example this is 50,200,000 milli-satoshis (i.e. 50,200 satoshis).

+payment_hash+:: This is the payment hash calculated from Dina's invoice. It is +H = RIPEMD160(SHA256(R))+, where R is Dina's secret that is known only by Dina and will be revealed if Dina is paid.

+cltv_expiry+:: This is the expiry time for this HTLC which will be encoded as a timelocked refund in case the HTLC fails to reach Dina in this time.

+onion_routing_packet+:: This is an onion-encrypted route that tells Bob where to forward this HTLC next (to Chan). Onion routing is covered in detail in <<onion_routing>>.

[TIP]
====
As a reminder, accounting within the Lightning Network is in units of milli-satoshis (thousandths of a satoshi), whereas Bitcoin accounting is in satoshis. Any amounts in HTLCs are milli-satoshis, which are then rounded to the nearest satoshi in the Bitcoin commitment transactions.
====

==== HTLC in commitment transaction

The received information is enough for Bob to create a new commitment transaction. The new commitment transaction has the same two outputs +to_self+ and +to_remote+ for Alice and Bob's balance, and a *new* output representing the HTLC offered by Alice.

We've already seen the basic structure of an HTLC in <<routing>>. The complete script of an offered HTLC is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#offered-htlc-outputs[BOLT #3 - Transactions - Offered HTLC output] and is shown in <<offered_htlc_output_script>> below:

[source,linenum]
[[offered_htlc_output_script]]
.Offered HTLC output script:
====
----
# Revocation <1>
OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL
OP_IF
    OP_CHECKSIG
OP_ELSE
    <remote_HTLCpubkey> OP_SWAP OP_SIZE 32 OP_EQUAL
    OP_IF
        # Redemption <2>
        OP_HASH160 <RIPEMD160(payment_hash)> OP_EQUALVERIFY
        2 OP_SWAP <local_HTLCpubkey> 2 OP_CHECKMULTISIG
    OP_ELSE
        # Refund <3>
        OP_DROP <cltv_expiry> OP_CHECKLOCKTIMEVERIFY OP_DROP
        OP_CHECKSIG
    OP_ENDIF
OP_ENDIF
----
<1> The first clause of the OP_IF conditional is redeemable by Alice with a revocation key. If this commitment is later revoked, Alice will have a revocation key to claim this output in a penalty transaction, taking the whole channel balance.
<2>  The second clause is redeemable by the pre-image (payment secret or in our example Dina's secret) if it is revealed. This allows Bob to claim this output if he has the secret from Dina, meaning he has successfully delivered the payment to Dina.
<3> The third and final clause is a refund of the HTLC to Alice, if the HTLC expires without reaching Dina. It is timelocked with the expiration +cltv_expiry+. This ensures that Alice's balance is not "stuck" in an HTLC that can't be routed to Dina.
====

There are three ways to claim this output. Try to read the script and see if you can figure it out (remember, it is a stack-based language so things appear "backwards").

==== New commitment with HTLC output

Bob now has the necessary information to add this HTLC script as an additional output and create a new commitment transaction. Bob's new commitment will have 50,200 satoshis in the HTLC output. That amount will come from Alice's channel balance, so Alice's new balance will be 19,800 satoshis (70,000 - 50,200 = 19,800). Bob constructs this commitment as a tentative "Commitment #3", shown in <<add_commitment_3b>>, below:

[[add_commitment_3b]]
.Bob's new commitment with an HTLC output
image::images/add_commitment_3b.png["Bob's new commitment with an HTLC output"]

==== Alice commits

Shortly after sending the +update_add_htlc+ message, she will commit to the new state of the channel, so that the HTLC can be safely added by Bob. Bob has the HTLC information and has constructed a new commitment but does not yet have this new commitment signed by Alice.

Alice sends +commitment_signed+ to Bob, with the signature for the new commitment and for the HTLC within. We saw the +commitment_signed+ message in <<payment_channels>>, but now we can understand the rest of the fields. As a reminder, it is shown in <<commitment_signed_message>> below:

[[commitment_signed_message]]
.The commitment_signed message
====
----
[channel_id:channel_id]
[signature:signature]
[u16:num_htlcs]
[num_htlcs*signature:htlc_signature]
----
====

The fields +num_htlcs+ and +htlc_signature+ now make more sense:

+num_htlcs+:: This is the number of HTLCs that are outstanding in the commitment transaction. In our example, just one HTLC, the one Alice offered.

+htlc_signature+:: This is an array of signatures (+num_htlcs+ in length), containing signatures for the HTLC outputs.

Alice can send these signatures without hesitation: she can always get a refund if the HTLC expires without being routed to Dina.

Now, Bob has a new signed commitment transaction, as shown in <<signed_commitment_3b>> below:

[[signed_commitment_3b]]
.Bob has a new signed commitment
image::images/signed_commitment_3b.png[Bob has a new signed commitment]

==== Bob acknowledges new commitment and revokes old one

Now that Bob has a new signed commitment, he needs to acknowledge it and revoke the old commitment. He does so by sending the +revoke_and_ack+ message, as we saw in <<payment_channels>> previously. As a reminder, that message is shown in <<revoke_and_ack_message_2>>, below:

[[revoke_and_ack_message_2]]
.The +revoke_and_ack+ message
====
----
[channel_id:channel_id]
[32*byte:per_commitment_secret]
[point:next_per_commitment_point]
----
====

Bob sends the +per_commitment_secret+ that allows Alice to construct a revocation key to build a penalty transaction spending Bob's old commitment. Once Bob has sent this, he cannot ever publish "Commitment #2" without risking a penalty transaction and losing all his money. So, the old commitment is effectively revoked.

Bob has effectively moved the channel state forward, as shown in <<revoked_commitment_2b>>:

[[revoked_commitment_2b]]
.Bob has revoked the old commitment
image::images/revoked_commitment_2b.png[Bob has revoked the old commitment]

Despite the fact that Bob has a new (signed) commitment transaction and an HTLC output inside he cannot consider his HTLC as being set up successfully.

He first needs to have Alice revoke her old commitment, because otherwise, Alice can roll back her balance to 70,000 satoshis. Bob needs to make sure that Alice also has a commitment transaction containing the HTLC and has revoked the old commitment.

That is why, if Bob is not the final recipient of the HTLC funds, he should not "forward" the HTLC yet, by offering an HTLC on the next channel with Chan.

Alice has constructed a mirror-image new commitment transaction containing the new HTLC, but it is yet to be signed by Bob. We can see it in <<add_commitment_3a>> below:

[[add_commitment_3a]]
.Alice's new commitment with an HTLC output
image::images/add_commitment_3a.png["Alice's new commitment with an HTLC output"]

As we described in <<payment_channels>>, Alice's commitment is the mirror-image of Bob's, as it contains the asymmetric, delayed, revocable construct for revocation and penalty enforcement of old commitments. Alice's 19,800 satoshi balance (after deducting the HTLC value), is delayed and revocable. Bob's 70,000 satoshi balance is immediately redeemable.

Next, the message flow for +commitment_signed+ and +revoke_and_ack+ is now repeated, but in the opposite direction. Bob sends +commitment_signed+ to sign Alice's  new commitment and Alice responds by revoking her old commitment.

For completeness sake, let's quickly review the commitment transactions as this round of commitment/revocation happens.

==== Bob commits

Bob now sends a +commitment_signed+ back to Alice, with his signatures for Alice's new commitment transaction, including the HTLC output she has added.

Now Alice has the signature for the new commitment transaction. The state of the channel is shown in <<signed_commitment_3a>> below:

[[signed_commitment_3a]]
.Alice has a new *signed* commitment
image::images/signed_commitment_3a.png[Alice has a new *signed* commitment]

Alice can now acknowledge the new commitment by revoking the old one. Alice sends the +revoke_and_ack+ message containing the necessary +per_commitment_point+ that will allow Bob to construct a revocation key and penalty transaction. Thus, Alice revokes her old commitment.

The channel state is shown in <<revoked_commitment_2a>> below:

[[revoked_commitment_2a]]
.Alice has revoked the old commitment
image::images/revoked_commitment_2a.png[Alice has revoked the old commitment]

=== Multiple HTLCs

At any point in time, Alice and Bob may have dozens or even hundreds of HTLCs across a single channel. Each HTLC is offered and added to the commitment transaction as an additional output. A commitment transaction therefore always has 2 outputs for the channel partner balances and any number of HTLC outputs, one per HTLC.

As we saw in the +commitment_signed+ message, there is an array for HTLC signatures so that multiple HTLC commitments can be transmitted at the same time.

The current maximum number of HTLCs allowed on a channel is 483 HTLCs to account for the maximum Bitcoin transaction size and ensure that the commitment transactions continue to be valid Bitcoin transactions.

As we will see in the next section, the maximum is only for _pending_ HTLCs, since once an HTLC is fulfilled (or fails due to timeout/error), it is removed from the commitment transaction.

=== HTLC fulfillment

Now Bob and Alice both have a new commitment transaction with an additional HTLC output and we have achieved a major step towards updating a payment channel.

The new balance of Alice and Bob does not reflect yet that Alice has successfully sent 50,200 satoshis to Bob.

However the HTLCs are now set up in a way that secure settlement in exchange for the proof of payment will be possible.

==== HTLC propagation

Let's assume that Bob continues the chain and sets up an HTLC with Chan for 50,100 satoshis. The process will be exactly the same as we just saw between Alice and Bob. Bob will send +update_add_htlc+ to Chan, then they will exchange +commitment_signed+ and +revoke_and_ack+ messages in two rounds, progressing their channel to the next state.

Next, Chan will do the same with Dina: offer a 50,000 satoshi HTLC, commit and revoke etc. However, Dina is the final recipient of the HTLC. Dina is the only one that knows the payment secret (the pre-image of the payment hash). Therefore, Dina can fulfill the HTLC with Chan immediately!

==== Dina fulfills the HTLC with Chan

Dina can settle the HTLC by sending an +update_fulfill_htlc+ message to Chan. The +update_fulfill_htlc+ message is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#removing-an-htlc-update_fulfill_htlc-update_fail_htlc-and-update_fail_malformed_htlc[BOLT #2 - Peer Protocol - update_fulfill_htlc] and is shown in <<update_fulfill_htlc_message>> below:

[[update_fulfill_htlc_message]]
.The +update_fulfill_htlc+ message
----
[channel_id:channel_id]
[u64:id]
[32*byte:payment_preimage]
----

It's a really simple message:

+channel_id+:: The channel ID on which the HTLC is committed

+id+:: The ID of the HTLC (we started with 0 and incremented for each HTLC on the channel)

+payment_preimage+:: The secret that proves payment was made and redeems the HTLC. This is the +R+ value that was hashed by Dina to produce the payment hash in the invoice to Alice.

When Chan receives this message, he will immediately check if the +payment_preimage+ (let's call it +R+) produces the payment hash (let's call it +H+) in the HTLC that he offered to Dina. He hashes it like this:

----
H = RIPEMD160( SHA256 (R) )
----

If the result +H+ matches the payment hash in the HTLC, Chan can do a little dance of celebration. This long-awaited secret can be used to redeem the HTLC, and will be passed back along the chain of payment channels all the way to Alice, resolving every HTLC that was part of this payment to Dina.

Let's go back to Alice and Bob's channel and watch them unwind the HTLC. To get there, let's assume Dina sent the +update_fulfill_htlc+ to Chan, Chan sent +update_fulfill_htlc+ to Bob and Bob sent +update_fulfill_htlc+ to Alice. The payment pre-image has propagated all the way back to Alice.

==== Bob settles the HTLC with Alice

When Bob sends +update_fulfill_htlc+ to Alice, it will contain the same +payment_preimage+ that Dina selected for her invoice. That +payment_preimage+ has traveled backwards along the payment path. At each step, the +channel_id+ will be different and +id+ (HTLC ID) may be different. But the preimage is the same!

Alice will also validate the +payment_preimage+ received from Bob. She will compare its hash to the HTLC payment hash in the HTLC she offered Bob. She will also find this pre-image matches the hash in Dina's invoice. This is proof that Dina was paid.

The message flow between Alice and Bob is shown in <<htlc_fulfillment_message_flow>> below:

[[htlc_fulfillment_message_flow]]
.The HTLC fulfillment message flow
image::images/htlc_fulfillment_message_flow.png[The HTLC fulfillment message flow]

Both Alice and Bob can now remove the HTLC from the commitment transactions and update their channel balances.

They create new commitments (Commitment #4), as shown in <<htlc_fulfillment_commitments_added>> below

[[htlc_fulfillment_commitments_added]]
.The HTLC is removed and balances updated in new commitments
image::images/htlc_fulfillment_commitments_added.png[The HTLC is removed and balances updated in new commitments]

Next, they complete two rounds of commitment and revocation. First, Alice sends +commitment_signed+ to sign Bob's new commitment transaction. Bob responds with +revoke_and_ack+ to revoke his old commitment. Once Bob has moved the state of the channel forward, the commitments look like we see in <<htlc_fulfillment_commitments_bob_commit>> below:

[[htlc_fulfillment_commitments_bob_commit]]
.Alice signs Bob's new commitment and Bob revoked the old one
image::images/htlc_fulfillment_commitments_bob_commit.png[Alice signs Bob's new commitment and Bob revoked the old one]

Finally, Bob signs Alice's commitment by sending Alice a +commitment_signed+ message. Then Alice acknowledges and revokes her old commitment by sending +revoke_and_ack+ to Bob. The end result is that both Alice and Bob have moved their channel state to Commitment #4, have removed the HTLC and have updated their balances. Their current channel state is represented by the commitment transactions that are shown in <<alice_bob_htlc_fulfilled>> below:

[[alice_bob_htlc_fulfilled]]
.Alice and Bob settle the HTLC and update balances
image::images/alice_bob_htlc_fulfilled.png[Alice and Bob settle the HTLC and update balances]

=== Removing an HTLC due to error or expiry

If an HTLC cannot be fulfilled, it can be removed from the channel commitment using the same process of commitment and revocation.

Instead of +update_fulfill_htlc+, Bob would send a +update_fail_htlc+ or +update_fail_malformed_htlc+. These two messages are defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#removing-an-htlc-update_fulfill_htlc-update_fail_htlc-and-update_fail_malformed_htlc[BOLT #2 - Peer Protocol - Removing an HTLC].

The +update_fail_htlc+ message is shown below:

[[update_fail_htlc_message]]
.The +update_fail_htlc+ message
----
[channel_id:channel_id]
[u64:id]
[u16:len]
[len*byte:reason]
----

It's pretty self-explanatory. The multi-byte +reason+ field is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#failure-messages[BOLT #4 - Onion Routing], which we will describe in more detail in <<onion_routing>>.

If Alice received a +update_fail_htlc+ from Bob, the process would unfold in much the same way: The two channel partners would remove the HTLC, create updated commitment transactions and go through two rounds of commitment/revocation to move the channel state forward to the next commitment. The only difference: the end balances would revert back to what they were without the HTLC, essentially refunding Alice for the HTLC value.

=== Making a local payment

At this point, you will easily understand why HTLCs are used for both "remote" and "local" payments. When Alice pays Bob for a coffee, she doesn't just update the channel balance and commit to a new state. Instead, the payment is made with an HTLC, in the same way Alice paid Dina. The fact that there's only one channel "hop" makes no difference. It would work like this:

* Alice orders a coffee from Bob's shop page.
* Bob's shop sends an invoice with a payment hash.
* Alice constructs an HTLC from that payment hash.
* Alice offers the HTLC to Bob with +update_add_htlc+.
* Alice and Bob exchange commitments and revocations adding the HTLC to their commitment transactions.
* Bob sends +update_fulfill_htlc+ to Alice with the payment pre-image.
* Alice and Bob exchange commitments and revocations removing the HTLC and updating the channel balances.

Whether an HTLC is forwarded across many channels, or just fulfilled in a single channel "hop", the process is exactly the same

=== Conclusion

In this chapter we saw how commitment transactions (from <<payment_channels>>) and HTLCs (from <<routing>>) work together. We saw how an HTLC is added to a commitment transaction, and how it is fulfilled. We saw how the asymmetric, delayed, revocable system for enforcing channel state is extended to HTLCs.

We also saw how a local payment and a multi-hop routed payment are handled identically: using HTLCs.

In the next chapter we will look at the encrypted message routing system called _Onion Routing_