mirror of https://github.com/lnbook/lnbook
Merge branch 'develop'
Andreas M. Antonopoulos
3 years ago
commit
6258790ac7
@ -0,0 +1,212 @@
|
||||
[[channel_operation]]
|
||||
== Channel Operation and Payment Forwarding
|
||||
|
||||
=== Forwarding payments with HTLCs
|
||||
In previous chapters we have seen that payment channels are maintained by two nodes by keeping two disjoint sequences of commitment transactions.
|
||||
The pair of latest commitment transactions in both sequences encodes the current, agreed upon balance in the channel.
|
||||
We have stated that two channel partners negotiate a new commitment transaction in order to change the balance and conduct a payment from one to another.
|
||||
We are finally at the point to explain the communications protocol via Lightning messages and the usage of HTLCs that is executed within a payment channel to change the balance.
|
||||
The same protocol will be executed along a path of channels if the network of channels is being utilized to make a payment between two participants without requiring them to have a dedicated payment channel connecting them directly.
|
||||
|
||||
Let us start with the payment channel with a capacity of 100 mBTC between Alice and Bob.
|
||||
At its current state Alice and Bob have agreed that 20 mBTC belong to Bob and 80 mBTC belong to Alice.
|
||||
As Alice bought a coffee flatrate for the week she has to pay 15 mBTC to Bob and wants to use this channel.
|
||||
Just creating a new pair of commitment transactions and signing them is not so easy as the old ones have to be invalidated by sharing the revocation secret.
|
||||
This process should be executed in a way that it is atomic meaning the nodes will either be able to negotiate a new state without giving the other side the chance to play tricks or it should fail.
|
||||
|
||||
[[routing-setup-htlc-0]]
|
||||
.Let us look at the initial pair of most recent commitment transactions for Alice and Bob:
|
||||
image:images/routing-setup-htlc-0.png[]
|
||||
|
||||
Alice sends the `update_add_HTLC` Lightning message to Bob.
|
||||
The message type is 128 and has the following data fields:
|
||||
|
||||
* [`channel_id`:`channel_id`]
|
||||
* [`u64`:`id`]
|
||||
* [`u64`:`amount_msat`]
|
||||
* [`sha256`:`payment_hash`]
|
||||
* [`u32`:`cltv_expiry`]
|
||||
* [`1366*byte`:`onion_routing_packet`]
|
||||
|
||||
As Bob and Alice might have more than one channel thus the `channel_id` is included to the message.
|
||||
The `id` counter counts starts with 0 for the first HTLC that Alice offers to Bob and is increased by 1 with every subsequent offer.
|
||||
The id of the HTLC is used to compute the derivation path of the bitcoin key that is used for the output of this particular HTLC.
|
||||
In this way addresses change with every payment and cannot be monitored by a third party.
|
||||
Next the amount that Alice wants to send to Bob is entered to the `amount_msat` field.
|
||||
As the name suggests the amount is depicted in millisatoshi even those cannot be enforced within the commitment transaction and within bitcoin.
|
||||
Still Lightning nodes keep track of subsatoshi amounts to avoid rounding issues.
|
||||
As in the offline example Alice includes the `payment_hash` in the next data field.
|
||||
This was told to Alice by Bob in case she wants to just send money to him.
|
||||
If Alice was to send Money to Dina the payment hash would have been given to Alice by Dina.
|
||||
We discussed the potential of time lock or deadline of the contract.
|
||||
This is encoded in the `cltv_expiry`.
|
||||
cltv stands for OP_CHECKTIMELOCKVERIFY and is the OP_CODE that will be used in the HTLC output and serve as the deadline in which the contract is valid.
|
||||
Finally in the last data field there are 1336 Bytes of data included which is an `onion routing packet`.
|
||||
The format of this packet will be discussed in the last section of this chapter.
|
||||
For now it is important to note that it includes encrypted routing hints and information of the payment path that can only be partially decrypted by the recipient of the onion routing packet to extract information to whom to forward the payment or to learn that one as the final recipient.
|
||||
In any case the onion routing packet is always of the same size preventing the possibility to guess the position of an intermediary node within a path.
|
||||
In our particular case Bob will be able to decrypt the first couple bytes of the onion routing packet and learn that the payment is not to be forwarded but intended to be for him.
|
||||
|
||||
The received information is enough for Bob to create a new commitment transaction.
|
||||
This commitment transaction now has not only 2 outputs encoding the balance between Alice and Bob but a third output which encodes the hashed time locked contract.
|
||||
|
||||
[[routing-setup-htlc-1]]
|
||||
.Lets look at the newly created commitment transaction for Bob:
|
||||
image:images/routing-setup-htlc-1.png[]
|
||||
|
||||
We can see that Bob Assumes that Alice will agree to lock 15 mBTC of her previous balance and assign it to the HTLC output.
|
||||
Creating this HTLC output can be compared to giving Alice golden coins to the escrow service.
|
||||
In our situation the bitcoin network can enforce the HTLC as Bob and Alice have agreed upon.
|
||||
Bob's Balance has not changed yet.
|
||||
In Bitcoin outputs are mainly described by scripts.
|
||||
The received HTLC in Bob's commitment transaction will use the following bitcoin script to define the output:
|
||||
|
||||
|
||||
# To remote node with revocation key
|
||||
OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL
|
||||
OP_IF
|
||||
OP_CHECKSIG
|
||||
OP_ELSE
|
||||
<remote_HTLCpubkey> OP_SWAP OP_SIZE 32 OP_EQUAL
|
||||
OP_IF
|
||||
# To local node via HTLC-success transaction.
|
||||
OP_HASH160 <RIPEMD160(payment_hash)> OP_EQUALVERIFY
|
||||
2 OP_SWAP <local_HTLCpubkey> 2 OP_CHECKMULTISIG
|
||||
OP_ELSE
|
||||
# To remote node after timeout.
|
||||
OP_DROP <cltv_expiry> OP_CHECKLOCKTIMEVERIFY OP_DROP
|
||||
OP_CHECKSIG
|
||||
OP_ENDIF
|
||||
OP_ENDIF
|
||||
|
||||
We can see that there are basically three conditions to claim the output.
|
||||
|
||||
1. Directly if a revocation key is known. This would happen if at a later state Bob fraudulently publishes this particular commitment transaction. As a newer state could only be agreed upon if Alice has learnt Bob's half of the revocation secret she could directly claim the funds and keep them even if Bob was later able to provide a proof of payment. This is mainly described in this line `OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL` and can be down by using `<revocation_sig> <revocationpubkey> as a witness script.
|
||||
2. If Bob has successfully delivered the payment and learnt the preimage he can spend the HTLC output with the help of the preimage and his `local_HTLC_secret`. This is to make sure that only Bob can spend this output if the commitment transaction hits the chain and not any other third party who might know the preimage because they had been included in the routing process. Claiming this output requires an HTLC-success transaction which we describe later.
|
||||
3. Finally Alice can use her `remote_HTLC_secret` to spend the HTLC output after the timeout of `cltv_expiry` was passed by using the following witness script `<remoteHTLCsig> 0`
|
||||
|
||||
As the commitment transaction spends the 2 out of 2 multisig funding transaction Bob needs two signatures after he constructed this commitment transaction.
|
||||
He can obviously compute his own signature but he needs also the signature from Alice.
|
||||
As Alice initiated the payment and wanted the HTLC to be set up she will be reluctant to provide a signature.
|
||||
|
||||
|
||||
[[routing-setup-htlc-2]]
|
||||
.Alice sends the `commitment_signed` Lightning Message to Bob:
|
||||
image:images/routing-setup-htlc-2.png[]
|
||||
|
||||
We can see in the diagram that Bob now has two valid commitment transactions.
|
||||
Let us have a quick look at the `commitment_signed` Lightning message which has the type 132.
|
||||
It has 4 data fields:
|
||||
|
||||
* [`channel_id`:`channel_id`]
|
||||
* [`signature`:`signature`]
|
||||
* [`u16`:`num_HTLCs`]
|
||||
* [`num_HTLCs*signature`:`HTLC_signature`]
|
||||
|
||||
First it again states which for which of the channels between Alice and Bob this message is intended.
|
||||
Then it has included a signature for the entire commitment transaction.
|
||||
As commitment transactions can have several HTLCs and HTLC success transactions need signatures which might not be provided at the time when they are needed those signatures are all already sent over to Bob.
|
||||
If all signatures are valid Bob has a new commitment transaction.
|
||||
At this time he would be able to publish either the old one or the new one without getting a penality as the old one is not yet revoked and invalidated.
|
||||
However this is safe for Alice as Bob has less money in this old state and is economically not incentivized to publish the old commitment transaction.
|
||||
Alice on the other side has no problem if Bob publishes the new commitment transaction as she wanted to send him money.
|
||||
If Bob can provide the preimage he is by their agreement and expectation entitled to claim the HTLC output.
|
||||
Should Bob decide to sabotage to future steps of the protocol Alice can publish her commitment transaction without Bob being able to punish her.
|
||||
He will just not have received the funds from Alice.
|
||||
This is important!
|
||||
Despite the fact that Bob has a new commitment transaction with two valid signatures and an HTLC output inside he cannot consider his HTLC as being set up successfully.
|
||||
He first needs to have Alice invalidate her old state.
|
||||
That is why - in the case that he is not the final recipient of the funds - he should not forward the HTLC yet by setting up a new HTLC on the next channel with Chan.
|
||||
Alice will not invalidate her commitment transaction yet as she has to first get her new commitment transaction and she wants Bob to invalidate his old commitment transaction which he can safely do at this time.
|
||||
|
||||
[[routing-setup-htlc-3]]
|
||||
.Bob sends a `revoke_and_ack` Lighting message to Alice:
|
||||
image:images/routing-setup-htlc-3.png[]
|
||||
|
||||
The `revoke_and_ack` Lightning message contains three data fields.
|
||||
* [`channel_id`:`channel_id`]
|
||||
* [`32*byte`:`per_commitment_secret`]
|
||||
* [`point`:`next_per_commitment_point`]
|
||||
|
||||
While it is really simple and straight forward it is very crucial.
|
||||
Bob shares the the `per_commitment_secret` of the old commitment transaction which serves as the revocation key and would allow Alice in future to penalize Bob if he publishes the old commitment transaction without the HTLC output.
|
||||
As in a future Alice and Bob might want to negotiate additional commitment transactions he already shares back the `next_per_commitment_point` that he will use in his next commitment transaction.
|
||||
|
||||
Alice checks that the `per_commitment_secret` produces the last `per_commitment_point` and constructs her new commitment transaction with the HTLC output.
|
||||
Alice's version of the HTLC output is slightly different to the one that Bob had.
|
||||
The reason is the asymmetries of the penalty based payment channel construction protocol.
|
||||
Alice is offering in her commitment transaction an HTLC to the `remote` partner of the channel while Bob as accepting and offered HTLC to himself the `local` partner of the channel.
|
||||
Thus the Bitcoin script is adapted slightly.
|
||||
It is a very good exercise to go through both scripts and see where they differ.
|
||||
You could also try to use Bob's HTLC output script to come up with Alice's and vice versa and check your result with the following script.
|
||||
|
||||
# To remote node with revocation key
|
||||
OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL
|
||||
OP_IF
|
||||
OP_CHECKSIG
|
||||
OP_ELSE
|
||||
<remote_HTLCpubkey> OP_SWAP OP_SIZE 32 OP_EQUAL
|
||||
OP_NOTIF
|
||||
# To local node via HTLC-timeout transaction (timelocked).
|
||||
OP_DROP 2 OP_SWAP <local_HTLCpubkey> 2 OP_CHECKMULTISIG
|
||||
OP_ELSE
|
||||
# To remote node with preimage.
|
||||
OP_HASH160 <RIPEMD160(payment_hash)> OP_EQUALVERIFY
|
||||
OP_CHECKSIG
|
||||
OP_ENDIF
|
||||
OP_ENDIF
|
||||
|
||||
Bob can redeem the HTLC with `<remoteHTLCsig> <payment_preimage>` as the witness script and in case the commitment transaction is revoked but published by Alice, Bob can trigger the penality by spending this output immediately with the following witness script `<revocation_sig> <revocationpubkey>`.
|
||||
|
||||
[[routing-setup-htlc-4]]
|
||||
.Bob knows how Alice's commitment transaction will look like and sends over the necessary signatures.
|
||||
image:images/routing-setup-htlc-4.png[]
|
||||
|
||||
This process is completely symmetrical to the one where Alice sent her signatures for Bob's new commitment transaction.
|
||||
Now Alice is the one having two valid commitment transactions.
|
||||
Technically she can still abort the payment by publishing her old commitment transaction to the bitcoin network.
|
||||
No one would lose anything as Bob knows that the contract is still being set up and not fully set up yet.
|
||||
This is a little bit different than how the situation would look like in a real world scenario.
|
||||
Recall Alice and Bob both have set up a new commitment transaction and have exchanged signatures.
|
||||
In the real world one would argue that this contract is now valid.
|
||||
|
||||
[[routing-setup-htlc-5]]
|
||||
.However Bob knows that Alice has to invalidate her previous commitment transaction which she does
|
||||
image:images/routing-setup-htlc-5.png[]
|
||||
|
||||
Now Bob and Alice both have a new commitment transaction with and additional HTLC output and we have achieved a major step towards updating a payment channel.
|
||||
The new Balance of Alice and Bob does not reflect yet that Alice has successfully send 15 mBTC to Bob.
|
||||
However the hashed time locked contracts are now set up in a way that secure settlement in exchange for the proof of payment will be possible.
|
||||
This yields another round of communication with Lightning messages and setting up additional commitment transactions which in case of good cooperation remove the outstanding HTLCs.
|
||||
Interestingly enough the `commitment_signed` and `revoke_and_ack` mechanism that we described to add an HTLC can be reused to update the commitment transaction.
|
||||
|
||||
If Bob was the recipient of the 15 mBTC and knows the preimage to the payment hash Bob can settle the HTLCs by sending and `update_fulfill_htlc` Lightning message to Alice.
|
||||
This message has the type 130 and only 3 data fields:
|
||||
|
||||
* [`channel_id`:`channel_id`]
|
||||
* [`u64`:`id`]
|
||||
* [`32*byte`:`payment_preimage`]
|
||||
|
||||
As other messages Bob uses the `channel_id` field to indicates for which channel he returns the preimage.
|
||||
The htlc that is being removed is identified by the same `id` that was used to set up the HTLC in the commitment transaction initially.
|
||||
You might argue that Alice would not need to know the id of the HTLC for which Bob releases the preimage as the preimage and payment hash could be unique.
|
||||
However with this design the protocol supports that a payment channel has several htlcs with the same preimage but only settles one.
|
||||
One could argue that this does not make too much sense and it is good to be critical but this is how the protocol is designed and what it supports.
|
||||
Finally in the last field Bob provides the `payment_preimage` which Alice can check hashes to the payment hash.
|
||||
|
||||
[WARNING]
|
||||
====
|
||||
When designing, implementing or studying a protocol one should ask: Is it safe to this or that in this moment of the protocol and can this be abused. We discussed for example the messages that where necessary for an HTLC to become valid. We pointed out that Bob should not see the received HTLC as valid even though he already has a new commitment transaction with signatures and invalidated his old commitment transaction before Alice also revoked her old commitment transaction. We also saw that no one is able to mess with the protocol of setting up a commitment transaction as in the worst case the protocol could be aborted and any dispute could be resolved by the Bitcoin Network. In the same way we should ask ourselves is it safe for Bob to just send out and release the preimage even though neither he nor Alice have created the new pair of commitment transactions in which the HTLCs are removed. It is important to take a short break and ask yourself if Bob will in any case be able to claim the funds from the HTLC if the preimage is correct?
|
||||
====
|
||||
|
||||
It is safe for Bob to tell Alice the preimage.
|
||||
Imagine Alice decides that she would not want to pay Bob anymore and does not respond anymore to create a new pair of commitment transactions with the removed HTLC and the Balance on Bob's end.
|
||||
In that case Bob could just force close the channel and publish his latest version of the commitment transaction.
|
||||
As the time lock of the HTLC is not over yet with an onchain success transaction Bob would be able to claim and settle his 15 mBTC as he is the only person who is able to spend the HTLC output in the commitment transaction.
|
||||
The other way around meaning Bob and Alice would negotiate a new commitment transaction with the removed HTLC would never be save for Alice.
|
||||
If the signatures for the new commitment transaction are exchanged Bob has received the money and could decide not to release the preimage.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Isn't it remarkable that even though the process of exchanging funds for an preimage seems to be happening concurrently at the same moment in time in reality it is actually happening one step after another but in the right order.
|
||||
====
|
||||
@ -0,0 +1,8 @@
|
||||
#!/bin/bash
|
||||
|
||||
# a small script to help sanity check the versions of the different node implementations
|
||||
dockerfiles=$(find . -name 'Dockerfile')
|
||||
# print location of dockerfiles
|
||||
echo $dockerfiles
|
||||
# print variables
|
||||
awk '/ENV/ && /VER|COMMIT/' $dockerfiles
|
||||
@ -1,53 +1,37 @@
|
||||
#!/bin/bash
|
||||
|
||||
echo Getting node IDs
|
||||
alice_address=$(docker-compose exec -T Alice bash -c "lncli -n regtest getinfo | jq .identity_pubkey")
|
||||
bob_address=$(docker-compose exec -T Bob bash -c "lightning-cli getinfo | jq .id")
|
||||
wei_address=$(docker-compose exec -T Wei bash -c "eclair-cli -s -j -p eclair getinfo| jq .nodeId")
|
||||
gloria_address=$(docker-compose exec -T Gloria bash -c "lncli -n regtest getinfo | jq .identity_pubkey")
|
||||
|
||||
# The jq command returns JSON strings enclosed in double-quote characters
|
||||
# These will confuse the shell later, because double-quotes have special
|
||||
# meaning in a bash script.
|
||||
|
||||
# We remove the double-quote character by using the shell string manipulation
|
||||
# expression: // removes the " character. Even here, we have to escape the "
|
||||
# character with a backslash because otherwise bash will interpret it as a
|
||||
# string closure.
|
||||
# A bit messy, but it works!
|
||||
alice_address=${alice_address//\"}
|
||||
bob_address=${bob_address//\"}
|
||||
wei_address=${wei_address//\"}
|
||||
gloria_address=${gloria_address//\"}
|
||||
alice_address=$(docker-compose exec -T Alice bash -c "lncli -n regtest getinfo | jq -r .identity_pubkey")
|
||||
bob_address=$(docker-compose exec -T Bob bash -c "lightning-cli getinfo | jq -r .id")
|
||||
chan_address=$(docker-compose exec -T Chan bash -c "eclair-cli -s -j -p eclair getinfo| jq -r .nodeId")
|
||||
dina_address=$(docker-compose exec -T Dina bash -c "lncli -n regtest getinfo | jq -r .identity_pubkey")
|
||||
|
||||
# Let's tell everyone what we found!
|
||||
echo Alice: ${alice_address}
|
||||
echo Bob: ${bob_address}
|
||||
echo Wei: ${wei_address}
|
||||
echo Gloria: ${gloria_address}
|
||||
echo Alice: ${alice_address}
|
||||
echo Bob: ${bob_address}
|
||||
echo Chan: ${chan_address}
|
||||
echo Dina: ${dina_address}
|
||||
|
||||
echo Setting up channels...
|
||||
echo Alice to Bob
|
||||
docker-compose exec -T Alice lncli -n regtest connect ${bob_address}@Bob
|
||||
docker-compose exec -T Alice lncli -n regtest openchannel ${bob_address} 1000000
|
||||
|
||||
echo Bob to Wei
|
||||
docker-compose exec -T Bob lightning-cli connect ${wei_address}@Wei
|
||||
docker-compose exec -T Bob lightning-cli fundchannel ${wei_address} 1000000
|
||||
echo Bob to Chan
|
||||
docker-compose exec -T Bob lightning-cli connect ${chan_address}@Chan
|
||||
docker-compose exec -T Bob lightning-cli fundchannel ${chan_address} 1000000
|
||||
|
||||
echo Wei to Gloria
|
||||
docker-compose exec -T Wei eclair-cli -p eclair connect --uri=${gloria_address}@Gloria
|
||||
docker-compose exec -T Wei eclair-cli -p eclair open --nodeId=${gloria_address} --fundingSatoshis=1000000
|
||||
echo Chan to Dina
|
||||
docker-compose exec -T Chan eclair-cli -p eclair connect --uri=${dina_address}@Dina
|
||||
docker-compose exec -T Chan eclair-cli -p eclair open --nodeId=${dina_address} --fundingSatoshis=1000000
|
||||
|
||||
echo Get 10k sats invoice from Gloria
|
||||
gloria_invoice=$(docker-compose exec -T Gloria bash -c "lncli -n regtest addinvoice 10000 | jq .payment_request")
|
||||
echo Get 10k sats invoice from Dina
|
||||
dina_invoice=$(docker-compose exec -T Dina bash -c "lncli -n regtest addinvoice 10000 | jq -r .payment_request")
|
||||
|
||||
# Remove quotes
|
||||
gloria_invoice=${gloria_invoice//\"}
|
||||
echo Gloria invoice ${gloria_invoice}
|
||||
echo Dina invoice ${dina_invoice}
|
||||
|
||||
echo Wait for channel establishment - 60 seconds for 6 blocks
|
||||
sleep 60
|
||||
|
||||
echo Alice pays Gloria 10k sats, routed around the network
|
||||
docker-compose exec -T Alice lncli -n regtest payinvoice --json --inflight_updates -f ${gloria_invoice}
|
||||
echo Alice pays Dina 10k sats, routed around the network
|
||||
docker-compose exec -T Alice lncli -n regtest payinvoice --json --inflight_updates -f ${dina_invoice}
|
||||
|
||||
@ -0,0 +1,74 @@
|
||||
=== Trust, Fairness and Enforcement
|
||||
|
||||
When people have competing interests, how can they establish enough trust to engage in some cooperative or transactional behavior? The answer to this question lies at the core of several scientific and humanistic disciplines, such as economics, sociology, behavioral psychology, and mathematics. Some of those disciplines give us "soft" answers, that depend on concepts such as reputation, fairness, morality, even religion. Other disciplines give us concrete answers that depend only on the assumption that the participants in these interactions will act rationally.
|
||||
|
||||
In broad terms there are a handful of ways to ensure fair outcomes in interactions between individuals who may have competing interests:
|
||||
|
||||
* Require trust - you only interact with people who you already trust, due to prior interactions, reputation, or familial relationships. This works well enough at small scale, especially within families and small groups, that it is the most common basis for cooperative behavior. Unfortunately, it doesn't scale and it suffers from tribalist (in-group) bias.
|
||||
|
||||
* Rule of law - establish rules for interactions that are enforced by an institution. This scales better, but it cannot scale globally due to differences in customs and traditions, as well as the inability to scale the institutions of enforcement. Nasty side-effect: the institutions become more and more powerful as they get bigger and that leads to corruption.
|
||||
|
||||
* Trusted third parties - put an intermediary in every interaction to enforce fairness. Combined with the "rule of law" to provide oversight of intermediaries, this scales better, but suffers from the same imbalance of power: the intermediaries get very powerful and attract corruption. Concentration of power leads to systemic risk and systemic failure ("Too big to fail").
|
||||
|
||||
* Game theoretical fairness protocols - this last category emerges from the combination of the internet and cryptography and is the subject of this section. Let's see how it works and what its advantages and disadvantages are.
|
||||
|
||||
==== Trusted protocols without intermediaries
|
||||
|
||||
Cryptographic systems like Bitcoin and the Lightning Network are systems that allow you to transact with people (and computers) that you don't trust. This is often referred to as "trustless" operation, even though it is not actually trustless. You have to trust in the software that you run and you have to trust that the protocol implemented by that software will result in fair outcomes.
|
||||
|
||||
The big distinction between a cryptographic system like this and a traditional financial system, is that in traditional finance you have a _trusted third party_, for example a bank, to ensure that outcomes are fair. A significant problem with such systems is that they give too much power to the third party and they are also vulnerable to a _single point of failure_. If the trusted third party itself violates trust or attempts to cheat, the basis of trust breaks.
|
||||
|
||||
As you study cryptographic systems, you will notice a certain pattern: instead of relying on a trusted third party, these systems attempt to prevent unfair outcomes by using a system of incentives and disincentives. In cryptographic systems you place trust in the _protocol_, which is effectively a system with a set of rules that, if properly designed, will correctly apply the desired incentives and disincentives. The advantage of this approach is two fold. Not only do you avoid trusting a third party, you also reduce the need to enforce fair outcomes. So long as the participants follow the agreed protocol and stay within the system, the incentive mechanism in that protocol achieves fair outcomes without enforcement.
|
||||
|
||||
The use of incentives and disincentives to achieve fair outcomes is one aspect of a branch of mathematics called _game theory_, which studies "models of strategic interaction among rational decision makers" footnote:[Wikipedia "Game Theory": https://en.wikipedia.org/wiki/Game_theory]. Cryptographic systems that control financial interactions between participants, such as Bitcoin and the Lightning Network rely heavily on game theory to prevent participants from cheating and allow participants who don't trust each other to achieve fair outcomes.
|
||||
|
||||
While game theory and its use in cryptographic systems may appear confounding and unfamiliar at first, chances are you're already familiar with these systems in your everyday life, you just don't recognize them yet. Below we'll use a simple example from childhood to help us identify the basic pattern. Once you understand the basic pattern you will see it everywhere in the blockchain space and you will come to recognize it quickly and intuitively.
|
||||
|
||||
In this book, we call this pattern a _Fairness Protocol_ defined as a process that uses a system of incentives and/or disincentives to ensure fair outcomes for participants who don't trust each other. Enforcement of a fairness protocol is only necessary to ensure that the participants can't escape the incentives or disincentives.
|
||||
|
||||
==== A fairness protocol in action
|
||||
|
||||
Let's look at an example of a fairness protocol, which may be familiar to any reader, perhaps as a memory from their childhood.
|
||||
|
||||
Imagine a family lunch, with a parent and two children. The parent has prepared a bowl of fried potatoes ("french fries" or "chips" depending on which English dialect you use). The two siblings must share the plate of chips. The parent must ensure a fair distribution of chips to each child, otherwise the parent will have to hear constant complaining (maybe all day) and there's always a possibility of the unfair situation escalating to violence. What is a parent to do?
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Any similarity between the scenario above and Andreas' childhood experiences with his two cousins is entirely coincidental and should not be mentioned again. The battles of the french fries created enough drama and should be left in the past.
|
||||
====
|
||||
|
||||
There are a few different ways that fairness can be achieved in this strategic interaction between two siblings that do not trust each other and have competing interests. The naive but commonly used method is for the parent to use their authority as a trusted third party: they split the bowl of chips into two servings. This is similar to traditional finance, where a bank, accountant or lawyer acts as a trusted third party to prevent any cheating between two parties who want to transact.
|
||||
|
||||
The problem with this scenario is that this puts a lot of power in the hands of the trusted third party. The parent is accused of playing favorites and not sharing the chips equally. The siblings may fight over the chips, dragging the parent into their fight. Eventually the parent threatens to never again cook french fries if it always results in fights. It is an empty threat, and so the cycle repeats daily.
|
||||
|
||||
But a much better solution exists: the siblings are taught to play a game called "split and choose". At each lunch one sibling splits the bowl of chips into two servings and the *other* sibling gets to choose which serving they want. Almost immediately, the siblings figure out the dynamic of this game. If the one splitting makes a mistake or tries to cheat, the other sibling can "punish" them by choosing the bigger bowl. It is in the best interest of both siblings, but especially the one splitting the bowl, to play fair. Only the cheater loses in this scenario. The parent doesn't even have to use their authority or enforce fairness. All the parent has to do is _enforce the protocol_; as long as the siblings cannot escape their assigned roles of "splitter" and "chooser", the protocol itself ensures a fair outcome without the need for any intervention. The parent can't play favorites or distort the outcome.
|
||||
|
||||
==== Security primitives as building blocks
|
||||
|
||||
In order for a fairness protocol like this to work, there need to be certain guarantees, or _security primitives_ that can be combined to ensure enforcement. The first security primitive is _strict time ordering/sequencing_: the "splitting" action must happen before the "choosing" action. It's not immediately obvious, but unless you can guarantee that action A happens before action B, then the protocol falls apart. The second security primitive is _commitment with non-repudiation_. Each sibling must commit to their choice of role: either splitter or chooser. Also, once the splitting has been completed, the splitter is committed to the split they created - they cannot repudiate that choice and go try again.
|
||||
|
||||
Cryptographic systems offer a number of security primitives that can be combined in different ways to construct a fairness protocol. In addition to sequencing and commitment, we can also use many other tools:
|
||||
|
||||
- Hash functions to fingerprint data, as a form of commitment, or as the basis for a digital signature.
|
||||
- Digital signatures for authentication, non-repudiation, and proof of ownership of a secret.
|
||||
- Encryption/decryption to restrict access to information to authorized participants only.
|
||||
|
||||
This is only a small list of a whole "menagerie" of security and cryptographic primitives that are in use. More basic primitives and combinations are invented all the time.
|
||||
|
||||
In our real-life example, we saw one form of fairness protocol called "split and choose". This is just one of a myriad different fairness protocols that can be built by combining the building blocks of security primitives in different ways. But the basic pattern is always the same: two or more participants interact without trusting each other, by engaging in a series of steps that are part of an agreed protocol. The protocol's steps arrange incentives and disincentives to ensure that if the participants are rational, cheating is counter-productive and fairness is the automatic outcome. Enforcement is not necessary to get fair outcomes - it is only necessary to keep the participants from breaking out of the agreed protocol.
|
||||
|
||||
Now that you understand this basic pattern, you will start seeing it everywhere in Bitcoin, the Lightning Network and many other systems. Let's look at some specific examples, next.
|
||||
|
||||
==== Example of the fairness protocol
|
||||
|
||||
The most prominent example of a "fairness protocol", is Bitcoin's consensus algorithm _Proof of Work_ (PoW). In Bitcoin, miners compete to verify transactions and aggregate them in blocks. To ensure that the miners do not cheat, without entrusting them with authority, Bitcoin uses a system of incentives and disincentives. Miners have to use a lot of electricity doing "work", that is embedded as a "proof" inside every block. This is achieved because of a property of hash functions where the output value is randomly distributed across the entire range of possible outputs. If miners succeed in producing a valid block fast enough, they are rewarded by earning the block reward for that block. Forcing miners to use a lot of electricity before the network considers their blocks means that they have an incentive to correctly validate the transactions in the block. If they cheat or make any kind of mistake, their block is rejected and the electricity they used to "prove" it is wasted. No one needs to force miners to produce valid blocks, the reward and punishment incentivize them to do so. All the protocol needs to do is ensure that only valid blocks with proof of work are accepted.
|
||||
|
||||
The "fairness protocol" pattern can also be found in many different aspects of the Lightning Network:
|
||||
|
||||
* Those who fund channels make sure that they have a refund transaction signed before they publish the funding transaction.
|
||||
|
||||
* Whenever a channel is moved to a new state, the old state is "revoked" by ensuring that if anyone tries to broadcast it, they lose the entire balance and get punished.
|
||||
|
||||
* Those who forward payments know that if they commit funds forward, they can either get a refund or they get paid by the node preceding them.
|
||||
|
||||
Again and again, we see this pattern. Fair outcomes are not enforced by any authority. They emerge as the natural consequence of a protocol that rewards fairness and punishes cheating. A fairness protocol that harnesses self-interest by directing it towards fair outcomes.
|
||||
@ -0,0 +1,711 @@
|
||||
# Channel Graph Discovery, Authentication & Maintenance
|
||||
|
||||
## Intro
|
||||
|
||||
As we have seen the Lightning Network uses a sourced base Onion Routing Protocol to deliver a payment from a sender to the recipient.
|
||||
For this the sending node has to be able to construct a path of payment channels that connects it with the recipient.
|
||||
Thus the sender has to be aware of the Channel Graph.
|
||||
The channel graph is the interconnected set of publicly advertised channels (more on that later), and the nodes that these channels interlink.
|
||||
As channels are backed by a funding transaction that is happening on chain one might falsely believe that Lightning Nodes could just extract the existing channels from the Bitcoin Blockchain.
|
||||
However this only possible to a certain extend.
|
||||
The Funding transactions are Pay to Whitness Script addresses and the nature of the script will only be revealed once the funding transaction output is spent.
|
||||
Even if the nature of the script was known we emphasize that not all 2 - of - 2 multisignature scripts correspond to payment channels.
|
||||
Since not even every Pay to Whitness Script address that we see in the Bitcoin Blockchain corresponds to a payment channel this approach is not helpful.
|
||||
There are actually more reasons why looking at the Bitcoin Blockchain might not be helpful.
|
||||
For example on the Lightnign Network the Bitcoin keys that are used for signing are rotated by the nodes for every channel and update.
|
||||
Thus even if we could reliably detect funding transactions on the Bitcoin Blockchain we would not know which two nodes on the Lightning Network own that particular channel.
|
||||
Thus we could node create the Channel Graph that would be used to create candidate paths during the payment delivery via onion routing.
|
||||
|
||||
The Lightning Network solves the afore mentioned problem by implementing a gossip protocol.
|
||||
Gossip protocols are typical for peer 2 peer networks and are being used so that nodes can share information with other nodes without talking to those other nodes directly.
|
||||
For that Lightning Nodes open encrypted peer 2 peer connections to each other and share novel information that they have received from other peers.
|
||||
As soon as a node wants to share some information - for example about a newly created channel - it sends a message to all its peers.
|
||||
Uppon receiving a message a node decides if the received message was novel and if so it will forward the information to its peers.
|
||||
In this way if the peer 2 peer network is connected all new information that is necessary for the operation of the network will eventually be propagated to all other peers.
|
||||
|
||||
Obviously if a new peer initially wants to join the network it needs to know some other peers on the Network initially.
|
||||
Only then the new peer could connect to them in order to learn about the existance of other peers.
|
||||
In this chapter, we'll explore exactly _how_ Lightning nodes discover each
|
||||
other, and also the channel graph itself.
|
||||
When most refer to the _network_ part
|
||||
of the Lightning Network, they're referring to the Channel Graph which itself
|
||||
is a unique authenticated data structure _anchored_ in the base Bitcoin
|
||||
blockchain.
|
||||
However the Lightning network is also a peer to peer network of nodes that are have just opened an encrypted peer connection.
|
||||
Usually for 2 peers to maintain a payment channel they need to talk to each other directly which means that there will be a peer connection between them.
|
||||
This suggests that the Channel Graph is a subnetwork of the peer to peer network.
|
||||
However this is not true.
|
||||
As payment channels do not need to be closed if one or both peers are going offline they can continue to exist even though the peer connection is terminated.
|
||||
It might be usefull to get familiar with the different terminology that we have used througout the book for similar concepts / actions depending on weather they happen on the Channel Graph or on the peer 2 peer network.
|
||||
|
||||
[[network-terminology]]
|
||||
.Terminology of the different Networks
|
||||
[options="header"]
|
||||
|===
|
||||
| | Channel Graph |peer to peer network
|
||||
| Name | channel | connection
|
||||
| initiate | open | (re)connect
|
||||
| finalize | close| terminate
|
||||
| technology | Bitcoin Blockchain | encrypted TCP/IP connection
|
||||
| lifetime | until funding spent | while peers are online
|
||||
|===
|
||||
|
||||
As the Lightning Network is a peer-to-peer network, some initial bootstrapping
|
||||
is required in order for peers to discover each other. Within this chapter
|
||||
we'll follow the story of a new peer connecting to the network for the first
|
||||
time, and examine each step in the bootstrapping process from initial peer
|
||||
discovery, to channel graph syncing and validation.
|
||||
|
||||
As an initial step, our new node needs to somehow _discover_ at least _one_
|
||||
peer that is already connected to the network and has a full channel graph (as
|
||||
we'll see later, there's no canonical version as the update dissemination
|
||||
system is _eventually consistent_). Using one of many initial bootstrapping
|
||||
protocols to find that first peer, after a connection is established, our new
|
||||
peer now needs to _download_ and _validate_ the channel graph. Once the channel
|
||||
graph has been fully validated, our new peer is ready to start opening channels
|
||||
and sending payments on the network.
|
||||
|
||||
After initial bootstrap, a node on the network needs to continue to maintain
|
||||
its view of the channel graph by: processing new channel routing policy
|
||||
updates, discovering and validating new channels, removing channels that have
|
||||
been closed on chain, and finally pruning channels that fail to send out a
|
||||
proper "heartbeat" every 2 weeks or so.
|
||||
|
||||
Upon completion of this chapter, the reader will understand a key component of
|
||||
the p2p Lightning Network: namely how peers discover each other and maintain a
|
||||
personal view of the channel graph. We'll being our story with exploring the
|
||||
story of a new node that has just booted up, and needs to find other peers to
|
||||
connect to on the network.
|
||||
|
||||
## Peer Discovery
|
||||
|
||||
In this section, we'll begin to follow the story of Norman, a new Lightning
|
||||
node that wishes to join the network as he sets out on his journey to which consists of 3 steps:
|
||||
|
||||
. discovery a set of bootstrap peers.
|
||||
. download and validate the channel graph.
|
||||
. begin the process of ongoing maintainance of the channel graph itself.
|
||||
|
||||
|
||||
### P2P Boostrapping
|
||||
|
||||
Before doing any thing else, Norman first needs to discover a set of peers whom
|
||||
are already part of the network. We call this process initial peer
|
||||
bootstrapping, and it's something hat every peer to peer network needs to
|
||||
implement properly in order to ensure a robust, healthy network. Bootstrapping
|
||||
new peers to existing peer to peer networks is a very well studied problem with
|
||||
several known solutions, each with their own distinct trade offs. The simplest
|
||||
solution to this problem is simply to package a set of _hard coded_ bootstrap
|
||||
peers into the packaged p2p node software. This is simple in that each new node
|
||||
can find the bootstrap peer with nothing else but the software they're running,
|
||||
but rather fragile given that if the set of bootstrap peers goes offline, then
|
||||
no new nodes will be able to join the network. Due to this fragility, this
|
||||
option is usually used as a fallback in case none of the other p2p bootstrapping
|
||||
mechanisms work properly.
|
||||
|
||||
Rather than hard coding the set of bootstrap peers within the software/binary
|
||||
itself, we can instead opt to devise a method that allows peers to dynamically
|
||||
obtain a fresh/new set of bootstrap peers they can use to join the network.
|
||||
We'll call this process initial peer discovery. Typically we'll leverage
|
||||
existing Internet protocols to maintain and distribute a set of bootstrapping
|
||||
peers. A non-exhaustive list of protocols that have been used in the past to
|
||||
accomplish initial peer discovery include:
|
||||
|
||||
* DNS
|
||||
* IRC
|
||||
* HTTP
|
||||
|
||||
Similar to the Bitcoin protocol, the primary initial peer discovery mechanism
|
||||
used in the Lightning Network happens via DNS. As initial peer discovery is a critical and
|
||||
universal task for the network, the process has been _standardized_ in a
|
||||
document that is a part of the Basis of Lightning Technology (BOLT)
|
||||
specification. This document is [BOLT
|
||||
10](https://github.com/lightningnetwork/lightning-rfc/blob/master/10-dns-bootstrap.md).
|
||||
|
||||
### DNS Bootstrapping via BOLT 10
|
||||
|
||||
The BOLT 10 document describes a standardized way of implementing peer
|
||||
discovery using the Domain Name System (DNS). Lightning's flavor of DNS based
|
||||
bootstrapping uses up to 3 distinct record types:
|
||||
|
||||
* `SRV` records for discovering a set of _node public keys_.
|
||||
* `A` records for mapping a node's public key to its current `IPv4` address.
|
||||
* `AAA` records for mapping a node's public key to its current `IPv6` address
|
||||
(if one exists).
|
||||
|
||||
Those somewhat familiar with the DNS protocol may already be familiar with the
|
||||
`A` and `AAA` record types, but not the `SRV` type. The `SRV` record type is
|
||||
used less commonly by protocols built on DNS, that's used to determine the
|
||||
_location_ for a specified service. In our context, the service in question is
|
||||
a given Lightning node, and the location its IP address. We need to use this
|
||||
additional record type as unlike nodes within the Bitcoin protocol, we need
|
||||
both a public key _and_ an IP address in order to connect to a node. As we saw
|
||||
in chapter XX on the transport encryption protocol used in LN, by requiring
|
||||
knowledge of the public key of a node before one can connect to it, we're able
|
||||
to implement a form of _identity_ hiding for nodes in the network.
|
||||
|
||||
// TODO(roasbeef): move paragraph below above?
|
||||
|
||||
#### A New Peer's Bootstrapping Workflow
|
||||
|
||||
Before diving into the specifics of BOLT 10, we'll first outline the high level
|
||||
flow of a new node that wishes to use BOLT 10 to join the network.
|
||||
|
||||
First, a node needs to identify a single, or set of DNS servers that understand
|
||||
BOLT 10 so they can be used for p2p bootstrapping.
|
||||
While BOLT 10 uses lseed.bitcoinstats.com as the seed server as the example there exists no "official"
|
||||
set of DNS seeds for this purpose, but each of the major implementations
|
||||
maintain their own DNS seed, and cross query each other's seeds for redundancy
|
||||
purposes.
|
||||
|
||||
[[dns seeds]]
|
||||
.Table of known lightning dns seed servers
|
||||
[options="header"]
|
||||
|===
|
||||
| dns server | Maintainer
|
||||
| lseed.bitcoinstats.com | Christian Decker
|
||||
| nodes.lightning.directory | lightning labs (Olaoluwa Osuntokun)
|
||||
| soa.nodes.lightning.directory | lightning labs (Olaoluwa Osuntokun)
|
||||
| lseed.darosior.ninja | Antoine Poinsot
|
||||
|===
|
||||
|
||||
|
||||
DNS seeds exist for both Bitcoin's mainnet and testnet. For the sake
|
||||
of our example, we'll assume the existence of a valid BOLT 10 DNS seed at
|
||||
`nodes.lightning.directory`.
|
||||
|
||||
Next, we'll now issue an `SRV` query to obtain a set of _candidate bootstrap
|
||||
peers_. The response to our query will be a series of _bech32_ encoded public
|
||||
keys. As DNS is a text based protocol, we can't send raw bytes, so an encoding
|
||||
scheme is required. For this scheme BOLT 10 has selected _bech32_ due to its
|
||||
existing propagation in the wider Bitcoin ecosystem. The number of encoded
|
||||
public keys returned depends on the server returning the query, as well as all
|
||||
the resolver that stand between the client and the authoritative server. Many
|
||||
resolvers may filter out SRV records all together, or attempt to truncate the
|
||||
response size itself.
|
||||
|
||||
Using the widely available `dig` command-line tool, we can query the _testnet_
|
||||
version of the DNS seed mentioned above with the following command:
|
||||
```
|
||||
$ dig @8.8.8.8 test.nodes.lightning.directory SRV
|
||||
```
|
||||
|
||||
We use the `@` argument to force resolution via Google's nameserver as they
|
||||
permit our larger SRV query responses. At the end, we specify that we only want
|
||||
`SRV` records to be returned. A sample response looks something like:
|
||||
```
|
||||
$ dig @8.8.8.8 test.nodes.lightning.directory SRV
|
||||
|
||||
; <<>> DiG 9.10.6 <<>> @8.8.8.8 test.nodes.lightning.directory SRV
|
||||
; (1 server found)
|
||||
;; global options: +cmd
|
||||
;; Got answer:
|
||||
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43610
|
||||
;; flags: qr rd ra; QUERY: 1, ANSWER: 25, AUTHORITY: 0, ADDITIONAL: 1
|
||||
|
||||
;; OPT PSEUDOSECTION:
|
||||
; EDNS: version: 0, flags:; udp: 512
|
||||
;; QUESTION SECTION:
|
||||
;test.nodes.lightning.directory. IN SRV
|
||||
|
||||
;; ANSWER SECTION:
|
||||
test.nodes.lightning.directory. 59 IN SRV 10 10 9735 ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7.test.nodes.lightning.directory.
|
||||
test.nodes.lightning.directory. 59 IN SRV 10 10 15735 ln1qtgsl3efj8verd4z27k44xu0a59kncvsarxatahm334exgnuvwhnz8dkhx8.test.nodes.lightning.directory.
|
||||
|
||||
<SNIP>
|
||||
|
||||
;; Query time: 89 msec
|
||||
;; SERVER: 8.8.8.8#53(8.8.8.8)
|
||||
;; WHEN: Thu Dec 31 16:41:07 PST 2020
|
||||
```
|
||||
|
||||
We've truncated the response for brevity. In our truncated responses, we can
|
||||
see two responses. Starting from the right-most column, we have a candidate
|
||||
"virtual" domain name for a target node, then to the left we have the _port_
|
||||
that this node can be reached using. The first response uses the standard port
|
||||
for LN: `9735`. The second response uses a custom port which is permitted by
|
||||
the protocol.
|
||||
|
||||
Next, we'll attempt to obtain the other piece of information we need to connect
|
||||
to a node: it's IP address. Before we can query for this however, we'll fist
|
||||
_decode_ the returned sub-domain for this virtual host name returned by the
|
||||
server. To do that, we'll first encoded public key:
|
||||
```
|
||||
ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7
|
||||
```
|
||||
|
||||
Using `bech32`, we can decode this public key to obtain the following valid
|
||||
`secp256k1` public key:
|
||||
```
|
||||
026c64f5a7f24c6f7f0e1d6ec877f23b2f672fb48967c2545f227d70636395eaf3
|
||||
```
|
||||
|
||||
Now that we have the raw public key, we'll now ask the DNS server to _resolve_
|
||||
the virtual host given so we can obtain the IP information for the node:
|
||||
```
|
||||
$ dig ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7.test.nodes.lightning.directory A
|
||||
|
||||
; <<>> DiG 9.10.6 <<>> ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7.test.nodes.lightning.directory A
|
||||
;; global options: +cmd
|
||||
;; Got answer:
|
||||
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41934
|
||||
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
|
||||
|
||||
;; OPT PSEUDOSECTION:
|
||||
; EDNS: version: 0, flags:; udp: 4096
|
||||
;; QUESTION SECTION:
|
||||
;ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7.test.nodes.lightning.directory. IN A
|
||||
|
||||
;; ANSWER SECTION:
|
||||
ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7.test.nodes.lightning.directory. 60 IN A X.X.X.X
|
||||
|
||||
;; Query time: 83 msec
|
||||
;; SERVER: 2600:1700:6971:6dd0::1#53(2600:1700:6971:6dd0::1)
|
||||
;; WHEN: Thu Dec 31 16:59:22 PST 2020
|
||||
;; MSG SIZE rcvd: 138
|
||||
```
|
||||
|
||||
In the above command, we've queried the server so we can obtain an `IPv4`
|
||||
address for our target node. Now that we have both the raw public key _and_ IP
|
||||
address, we can connect to the node using the `brontide` transport protocol at:
|
||||
`026c64f5a7f24c6f7f0e1d6ec877f23b2f672fb48967c2545f227d70636395eaf3@X.X.X.X`.
|
||||
Querying for the curent `A` record for a given node can also be used to look up
|
||||
the _latest_ set of addresses for a given node. Such queries can be used to
|
||||
more quickly (compared to waiting on gossip as we'll cover later) sync the
|
||||
latest addressing information for a node.
|
||||
|
||||
At this point in our journey, Norman our new Lightning Node has found its first
|
||||
peer and established its first connect! Now we can being the second phase of
|
||||
new peer bootstrapping: channel graph synchronization and validation, but
|
||||
first, we'll explore more of the intricacies of BOLT 10 itself to take a deeper
|
||||
look into how things work under the hood.
|
||||
|
||||
### A Deep Dive Into BOLT 10
|
||||
|
||||
As we learned earlier in the chapter, BOLT 10 describes the standardized
|
||||
protocol for boostrapping new peer suing the DNS protocol. In this section,
|
||||
we'll dive into the details of BOLT 10 itself in order to explore exactly how
|
||||
bootstrapping queries are made, and also the additional set of options
|
||||
available for querying.
|
||||
|
||||
#### SRV Query Options
|
||||
|
||||
The BOLT 10 standard is highly extensible due to its usage of nested
|
||||
sub-domains as a communication layer for additional query options. The
|
||||
bootstrapping protocol allows clients to further specify the _type_ of nodes
|
||||
they're attempting to query for vs the default of receiving a random subset of
|
||||
nodes in the query responses.
|
||||
|
||||
The query option sub-domain scheme uses a series of key-value pairs where the
|
||||
key itself is a _single letter_ and the remaining set of text is the value
|
||||
itself. The following query types exist in the current version of the BOLT 10
|
||||
standards document:
|
||||
|
||||
* `r`: The "realm" byte which is used to determine which chain or realm
|
||||
queries should be returned for. As is, the only value for this key is `0`
|
||||
which denotes Bitcoin itself.
|
||||
|
||||
* `a`: Allows clients to filter out returned nodes based on the _types_ of
|
||||
addresses they advertise. As an example, this can be used to only obtain
|
||||
nodes that advertise a valid IPv6 address.
|
||||
* The value that follows this type is based on a bitfled that _indexes_
|
||||
into the set of specified address _type_ which are defined in BOLT 7.
|
||||
We'll cover this material shortly later in this chapter once we examine
|
||||
the structure of the channel graph itself.
|
||||
* The default value for this field is `6`, which is `2 || 4`, which denotes
|
||||
bit 1 and 2, which are IPv4 and IPv6.
|
||||
|
||||
* `l`: A valid node public key serialized in compressed format. This allows a
|
||||
client to query for a specified node rather than receiving a set of random
|
||||
nodes.
|
||||
|
||||
* `n`: The number of records to return. The default value for this field is
|
||||
`25`.
|
||||
|
||||
An example query with additional query options looks something like the following:
|
||||
```
|
||||
r0.a2.n10.nodes.lightning.directory
|
||||
```
|
||||
|
||||
Breaking down the query one key-value pair at a time we gain the following
|
||||
insights:
|
||||
|
||||
* `r0`: The query targets the Bitcoin realm.
|
||||
* `a2`: The query only wants IPv4 addresses to be returned.
|
||||
* `n10`: The query requests
|
||||
|
||||
## Channel Graph: Structure and Attributes
|
||||
|
||||
Now that Norman is able to use the DNS boostrapping protocol to connect to his
|
||||
very first peer, we can now start to sync the channel graph! However, before we
|
||||
sync the channel graph, we'll need to learn exactly _what_ we mean by the
|
||||
channel graph. In this section we'll explore the precise _structure_ of the
|
||||
channel graph and examine the unique aspects of the channel graph compared to
|
||||
the typical abstract "graph" data structure which is well known/used in the
|
||||
field of Computer Science.
|
||||
|
||||
### The Channel Graph as a Directed Overlay Data Structure
|
||||
|
||||
A graph in computer science is a special data structure composed of vertices
|
||||
(typically referred to as nodes) and edges (also known as links). Two nodes may
|
||||
be connected by one or more edges. The channel graph is also _directed_ given
|
||||
that a payment is able to flow in either direction over a given edge (a
|
||||
channel). As we're concerned with _routing payments_, in our model a node with
|
||||
no edges isn't considered to be a part of the graph as it isn't "productive".
|
||||
In the context of the Lightning Network, our vertices are the Lightning nodes
|
||||
themselves, with our edges being the channels that _connect_ these nodes. As
|
||||
channels are themselves a special type of multi-sig UTXO anchored in the base
|
||||
Bitcoin blockchain, possible to scan the chain (with the assistance of special
|
||||
meta data proofs) and re-derive the channel graph first-hand (though we'd be
|
||||
missing some information as we see below).
|
||||
|
||||
As channels themselves are UTXOs, we can view the channel graph as a special
|
||||
sub-set of the UTXO set, on top of which we can add some additional information
|
||||
(the nodes, etc) to arrive at the final overlay structure which is the channel
|
||||
graph. This anchoring of fundamental components of the cahnnel graph in the
|
||||
base Bitcoin blockchain means that it's impossible to _fake_ a valid channel
|
||||
graph, which has useful properties when it comes to spam prevention as we'll
|
||||
see later. The channel graph in the Lightning Network is composed of 3
|
||||
individual components which are described in BOLT 7:
|
||||
|
||||
* `node_announcement`: The vertex in our graph which communicates the public
|
||||
key of a node, as well as how to reach the node over the internet and some
|
||||
additional metadata describing the set of _features_ the node supports.
|
||||
|
||||
* `channel_announcement`: A blockchain anchored proof of the existence of a
|
||||
channel between two individual nodes. Any 3rd party can verify this proof in
|
||||
order to ensure that a _real_ channel is actually being advertised. Similar
|
||||
to the `node_announcement` this message also contains information describing
|
||||
the _capabilities_ of the channel which is useful when attempting to route a
|
||||
payment.
|
||||
|
||||
* `channel_update`: A _pair_ of structures that describes the set of _routing
|
||||
policies_ for a given channel. `channel_update`s come in a _pair_ as a
|
||||
channel is a directed edge, so both sides of the channel are able to specify
|
||||
their own custom routing policy. An example of a policy communicated in a
|
||||
|
||||
It's important to note that each of components of the channel graph are
|
||||
themselves _authenticated_ allowing a 3rd party to ensure that the owner of a
|
||||
channel/update/node is actually the one sending out an update. This effectively
|
||||
makes the Channel Graph a unique type of _authenticated data structure_ that
|
||||
cannot be counterfeited. For authentication, we use an `secp256k1` ECDSA
|
||||
digital signature (or a series of them) over the serialized digest of the
|
||||
message itself. We won't get into the specific of the messaging
|
||||
framing/serialization used in the LN in this chapter, as we'll cover that
|
||||
information in Chapter XX on the wire protocol used in in the protocol.
|
||||
|
||||
With the high level structure of the channel graph laid out, we'll now dive
|
||||
down into the precise structure of each of the 3 components of the channel
|
||||
graph. We'll also explain how one can also _verify_ each component of the
|
||||
channel graph as well.
|
||||
|
||||
#### Node Announcement: Structure & Validation
|
||||
|
||||
First, we have the `node_announcement` which plays the role of the vertex in
|
||||
the channel graph. A node's announcement in the network serves to primary
|
||||
purposes:
|
||||
|
||||
1. To advertise connection information so other nodes can connect to it,
|
||||
either to bootstrap to the network, or to attempt to establish a set of new
|
||||
channels.
|
||||
|
||||
2. To communicate the set of features protocol level features a node
|
||||
understands. This communication is critical to the decentralized
|
||||
de-synchronized update nature of the Lightning Network itself.
|
||||
|
||||
Unlike channel announcements, node announcements aren't actually anchored in
|
||||
the base blockchain. As a result, advertising a node announcement in isolation
|
||||
bares no up front cost. As a result, we require that all node announcements are
|
||||
only considered "valid" if it has propagated with a corresponding channel
|
||||
announcement as well. In other words, we always reject unconnected nodes in
|
||||
order to ensure a rogue peer can't fill up our disk with bogus nodes that may
|
||||
not actually be part of the network.
|
||||
|
||||
##### Structure
|
||||
|
||||
The node announcement is a simple data structure that needs to exist for each
|
||||
node that's a part of the channel graph. The node announcement is comprised of
|
||||
the following fields, which are encoded using the wire protocol structure
|
||||
described in BOLT 1:
|
||||
|
||||
* `signature`: A valid ECDSA signature that covers the serialized digest of
|
||||
all fields listed below. This signature MUST be venerated by the private
|
||||
key that backs the public key of the advertised node.
|
||||
|
||||
* `features`: A bit vector that describes the set of protocol features that
|
||||
this node understands. We'll cover this field in more detail in Chapter XX
|
||||
on the extensibility of the Lightning Protocol. At a high level, this field
|
||||
carries a set of bits (assigned in pairs) that describes which new features
|
||||
a node understands. As an example, a node may signal that it understands
|
||||
the latest and greatest channel type, which allows peers that which
|
||||
bootstrap to the network to filter out the set of nodes they want to connect
|
||||
to.
|
||||
|
||||
* `timestamp`: A timestamp which should be interpreted as a unix epoch
|
||||
formatted timestamp. This allows clients to enforce a partial ordering over
|
||||
the updates to a node's announcement.
|
||||
|
||||
* `node_id`: The `secp256k1` public key that this node announcement belongs
|
||||
to. There can only be a single `node_announcement` for a given node in the
|
||||
channel graph at any given time. As a result, a `node_announcement` can
|
||||
superseded a prior `node_announcement` for the same node if it carries a
|
||||
higher time stamp.
|
||||
|
||||
* `rgb_color`: A mostly unused field that allows a node to specify an RGB
|
||||
"color" to be associated with it.
|
||||
|
||||
* `alias`: A UTF-8 string to serve as the nickname for a given node. Note
|
||||
that these aliases aren't required to be globally unique, nor are they
|
||||
verified in any shape or form. As a result, they are always to be
|
||||
interpreted as being "unofficial".
|
||||
|
||||
* `addresses`: A set of public internet reachable addresses that are to be
|
||||
associated with a given node. In the current version of the protocol 4
|
||||
address types are supported: IPv4 (1), IPv6 (2), Tor V2 (3), Tor V3 (4). On
|
||||
the wire, each of these address types are denoted by an integer type which
|
||||
is included in parenthesis after the address type.
|
||||
|
||||
##### Validation
|
||||
|
||||
Validating an incoming `node_announcement` is straight forward, the following
|
||||
assertions should be upheld when examining a node announcement:
|
||||
|
||||
* If an existing `node_announcement` for that node is already known, then the
|
||||
`timestamp` field of a new incoming `node_announcement` MUST be greater
|
||||
than the prior one.
|
||||
|
||||
* With this constraint, we enforce a forced level of "freshness".
|
||||
|
||||
* If no `node_announcement` exist for the given node, then an existing
|
||||
`channel_announcement` that refernces the given node (more on that later)
|
||||
MUST already exist in one's local channel graph.
|
||||
|
||||
* The included `signature` MUST be a valid ECDSA signature verified using the
|
||||
included `node_id` public key and the double-sha256 digest of the raw
|
||||
message encoding (mins the signature and frame header!) as the message.
|
||||
|
||||
* All included `addresses` MUST be sorted in ascending order based on their
|
||||
address identifier.
|
||||
|
||||
* The included `alias` bytes MUST be a valid UTF-8 string.
|
||||
|
||||
#### Channel Announcement: Structure & Validation
|
||||
|
||||
Next, we have the `channel_announcement`. This message is used to _announce_ a
|
||||
new _public_ channel to the wider network. Note that announcing a channel is
|
||||
_optional_. A channel only needs to be announced if its intended to be used for
|
||||
routing by the public network. Active routing nodes may wish to announce all
|
||||
their channels. However, certain nodes like mobile nodes likely don't have the
|
||||
uptime or desire required to be an active routing node. As a result, these
|
||||
mobile nodes (which typically use light clients to connect to the Bitcoin p2p
|
||||
network), instead may have purely _unadvertised_ channels.
|
||||
|
||||
##### Unadvertised Channels & The "True" Channel Graph
|
||||
|
||||
An unadvertised channel isn't part of the known public channel graph, but can
|
||||
still be used to send/receive payments. An astute reader may now be wondering
|
||||
how a channel which isn't part of the public channel graph is able to receive
|
||||
payments. The solution to this problem is a set of "path finding helpers" that
|
||||
we call "routing hints. As we'll see in Chapter XX on the presentation/payment
|
||||
layer, invoices created by nodes with unadvertised channels will include
|
||||
auxiliary information to help the sender route to them assuming the no has at
|
||||
least a single channel with an existing public routing node.
|
||||
|
||||
Due to the existence of unadvertised channels, the _true_ size of the channel
|
||||
graph (both the public and private components) is unknown. In addition, any
|
||||
snapshot of the channel graph that doesn't come directly from one's own node
|
||||
(via a Block Explorer or the like) is to be considered non-canonical as
|
||||
updates to the graph are communicated using a system that only is able to
|
||||
achieve an eventually consistent view of the channel graph.
|
||||
|
||||
##### Locating Channel In the Blockchain via Short Channel IDs
|
||||
|
||||
As mentioned earlier, the channel graph is authenticated due to its usage of
|
||||
public key cryptography, as well as the Bitcoin blockchain as a spam prevention
|
||||
system. In order to have a node accept a new `channel_announcement`, the
|
||||
advertise must _prove_ that the channel actually exists in the Bitcoin
|
||||
blockchain. This proof system adds an upfront cost to adding a new entry to the
|
||||
channel graph (the on-chain fees on must pay to create the UTXO of the
|
||||
channel). As a result, we mitigate spam and ensure that another node on the
|
||||
network can't costless fill up the disk of an honest node with bogus channels.
|
||||
|
||||
Given that we need to construct a proof of the existence of a channel, a
|
||||
natural question that arises is: how to we "point to" or reference a given
|
||||
channel for the verifier? Given that a channel MUST be a UTXO, an initial
|
||||
thought might be to first attempt to just advertise the full outpoint
|
||||
(`txid:index`) of the channel. Given the outpoint of a UTXO is globally unique
|
||||
one confirmed in the chain, this sounds like a good idea, however it has one
|
||||
fatal flow: the verifier must maintain a full copy of the UTXO set in order to
|
||||
verify channels. This works fine for full-node, but light clients which rely on
|
||||
primarily PoW verification don't typically maintain a full UTXO set. As we want
|
||||
to ensure we can support mobile nodes in the Lightning Network, we're forced to
|
||||
find another solution.
|
||||
|
||||
What if rather than referencing a channel by its UTXO, we reference it based on
|
||||
its "location" in the chain? In order to do this, we'll need a scheme that
|
||||
allows us to "index" into a given block, then a transaction within that block,
|
||||
and finally a specific output created by that transaction. Such an identifier
|
||||
is described in BOLT 7 and is referred to as a: short channel ID, or `scid`.
|
||||
The `scid` is used both in `channel_announcement` (and `channel_update`) as
|
||||
well as within the onion encrypted routing packet included within HTLCs as we
|
||||
learned in Chapter XX.
|
||||
|
||||
###### The Short Channel ID Identifier
|
||||
|
||||
Based on the information above, we have 3 piece of information we need to
|
||||
encode in order to uniquely reference a given channel. As we want to very
|
||||
compact representation, we'll attempt to encode the information into a _single_
|
||||
integer using existing known bit packing techniques. Our integer format of
|
||||
choice is an unsigned 64-bit integer, which is comprised of 8 logical bytes.
|
||||
|
||||
First, the block height. Using 3 bytes (24-bits) we can encode 16777216 blocks,
|
||||
which is more than double the number of blocks that are attached to the current
|
||||
mainnet Bitcoin blockchain. That leaves 5 bytes left for us to encode the
|
||||
transaction index and the output index respectively. We'll then use the next 3
|
||||
bytes to encode the transaction index _within_ a block. This is more than
|
||||
enough given that it's only possible to fix tens of thousands of transactions
|
||||
in a block at current block sizes. This leaves 2 bytes left for us to encode
|
||||
the output index of the channel within the transaction.
|
||||
|
||||
Our final `scid` format resembles:
|
||||
```
|
||||
block_height (3 bytes) || transaction_index (3 bytes) || output_index (2 byes)
|
||||
```
|
||||
|
||||
Using bit packing techniques, we first encode the most significant 3 bytes as
|
||||
the block height, the next 3 bytes as the transaction index, and the least
|
||||
significant 2 bytes as the output index of that creates the channel UTXO.
|
||||
|
||||
A short channel ID can be represented as a single integer
|
||||
(`695313561322258433`) or as a more human friendly string: `632384x1568x1`.
|
||||
Here we see the channel was mined in block `632384`, was the `1568` th
|
||||
transaction in the block, with the channel output being found as the second
|
||||
(UTXOs are zero-indexed) output produced by the transaction.
|
||||
|
||||
Now that we're able to succinctly defence a given channel in the chain, we can
|
||||
now examine the full structure of the `channel_announcement` message, as well
|
||||
as how to verify the proof-of-existence included within the message.
|
||||
|
||||
##### Channel Announcement Structure
|
||||
|
||||
A channel announcement primarily communicates two aspects:
|
||||
|
||||
1. A proof that a channel exists between Node A and Node B with both nodes
|
||||
controlling the mulit-sig keys in the refracted channel output
|
||||
|
||||
2. The set of capabilities of the channel (what types of HTLCs can it route,
|
||||
etc)
|
||||
|
||||
When describing the proof, we'll typically refer to node `1` and node `2`. Out
|
||||
of the two nodes that a channel connects, the "first" node is the node that has
|
||||
a "lower" public key encoding when we compare the public key of the two nodes
|
||||
in compressed format hex-encoded in lexicographical order. Correspondingly, in
|
||||
addition to a node public key on the network, each node should also control a
|
||||
public key within the Bitcoin blockchain.
|
||||
|
||||
Similar to the `node_announcement` message, all included signatures of the
|
||||
`channel_announcement` message should be signed/verified against the raw
|
||||
encoding of the message (minus the header) that follows _after_ the final
|
||||
signature (as it isn't possible for a signature to sign itself..)
|
||||
|
||||
With that said, a `channel_announcement` message (the edge descriptor in the
|
||||
channel graph) has the following attributes:
|
||||
|
||||
* `node_signature_1`: The signature of the first node over the message digest.
|
||||
|
||||
* `node_signature_2`: The signature of the second node over the message
|
||||
digest.
|
||||
|
||||
* `bitcoin_signature_1`: The signature of the multi-sig key (in the funding
|
||||
output) of the first node over the message digest.
|
||||
|
||||
* `bitcoin_signature_2`: The signature of the multi-sig key (in the funding
|
||||
output) of the second node over the message digest.
|
||||
|
||||
* `features`: A feature bitvector that describes the set of protocol level
|
||||
features supported by this channel.
|
||||
|
||||
* `chain_hash`: A 32 byte hash which is typically the genesis block hash of
|
||||
the chain the channel was opened within.
|
||||
|
||||
* `short_channel_id`: The `scid` that uniquely locates the given channel
|
||||
within the blockchain.
|
||||
|
||||
* `node_id_1`: The public key of the first node in the network.
|
||||
|
||||
* `node_id_2`: The public key of the second node in the network.
|
||||
|
||||
* `bitcoin_key_1`: The raw multi-sig key for the channel funding output for
|
||||
the first node in the network.
|
||||
|
||||
* `bitcoin_key_2`: The raw multi-sig key for the channel funding output for
|
||||
the second node in the network.
|
||||
|
||||
##### Channel Announcement Validation
|
||||
|
||||
Now that we know what a `channel_announcement` contains. We can now move onto
|
||||
to exactly _how_ to verify such an announcement.
|
||||
|
||||
|
||||
#### Channel Update: Structure & Validation
|
||||
|
||||
|
||||
// TODO(roasbeef): rename to "the structure of the channel graph"?
|
||||
|
||||
## Syncing the Channel Graph
|
||||
|
||||
|
||||
|
||||
* introduce the NodeAnnouncement (purpose structure validation)
|
||||
* go thru fields, ref ability to use Tor, etc
|
||||
* ref feature bits at high level, say will be covered in later chapter
|
||||
* node announcement validation
|
||||
* acceptance critera
|
||||
|
||||
|
||||
### Channel Announcement
|
||||
|
||||
## Ongoing Channel Graph Maintenance
|
||||
|
||||
### Gossip Protocols in the Abstract
|
||||
|
||||
* what is a gossip protocol?
|
||||
* why are they used?
|
||||
* what other famous uses of gossip protocols are out there?
|
||||
* when does it make sense to use a gossip protocol?
|
||||
* what are some use a gossip protocol?
|
||||
* why does LN uise
|
||||
* questions to ask for gossip rptocol
|
||||
* what is being gossiped
|
||||
* what is the expected delay bound?
|
||||
* how is DoS prevented
|
||||
|
||||
## Gossip in LN
|
||||
|
||||
### Channel Announcements
|
||||
|
||||
### Purpose
|
||||
### Structure
|
||||
### Validation
|
||||
|
||||
### Channel Updates
|
||||
|
||||
### Purpose
|
||||
### Structure
|
||||
### Validation
|
||||
|
||||
### Node Announcements
|
||||
|
||||
### Purpose
|
||||
### Structure
|
||||
### Validation
|
||||
|
||||
* anser the three quesitons above
|
||||
|
||||
* what: node info, chan info, channel updates
|
||||
|
||||
* delay: 2 week liveness assumption, otherwise pruned, keep alive updates
|
||||
|
||||
* DoS: real channel, proper validation of sigs, etc
|
||||
|
||||
# Conlusion
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 45 KiB After Width: | Height: | Size: 118 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 40 KiB After Width: | Height: | Size: 155 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 43 KiB After Width: | Height: | Size: 166 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 45 KiB After Width: | Height: | Size: 175 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 46 KiB After Width: | Height: | Size: 178 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 46 KiB After Width: | Height: | Size: 173 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 50 KiB After Width: | Height: | Size: 189 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 46 KiB After Width: | Height: | Size: 178 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 138 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 56 KiB After Width: | Height: | Size: 92 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 88 KiB After Width: | Height: | Size: 148 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 110 KiB After Width: | Height: | Size: 182 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 97 KiB After Width: | Height: | Size: 157 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 130 KiB After Width: | Height: | Size: 217 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 150 KiB After Width: | Height: | Size: 245 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 182 KiB |
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue