Archives/lnbook
2
0
Fork 0
mirror of https://github.com/lnbook/lnbook synced 2024-11-01 03:20:53 +00:00
Code Issues Projects Releases Wiki Activity

ch 5 ce edits partial

Browse Source
This commit is contained in:
Gregory Hyman 2021-10-15 13:49:39 -04:00
parent 561e1bc233
commit 89bfafaeeb
1 changed files with 119 additions and 127 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

246
05_node_operations.asciidoc
View File

@ -6,15 +6,15 @@ After having read this far, you have probably set up a Lightning wallet. In this
There are many reasons why you might want to set up your own Lightning node. They include:
* To be a full, active participant in the Lightning Network, not just an end-user.
* To run an e-commerce store or receive income via Lightning payments.
* To earn income from Lightning routing fees or by renting channel liquidity.
* To develop new services, applications, or plugins for the Lightning Network.
* To increase your financial privacy while using Lightning.
* To use some apps built on top of Lightning, like Lightning-powered instant messaging apps.
* For financial freedom, independence, and sovereignty.
* To be a full, active participant in the Lightning Network, not just an end user
* To run an ecommerce store or receive income via Lightning payments
* To earn income from Lightning routing fees or by renting channel liquidity
* To develop new services, applications, or plug-ins for the Lightning Network
* To increase your financial privacy while using Lightning
* To use some apps built on top of Lightning, like Lightning-powered instant messaging apps
* For financial freedom, independence, and sovereignty
There are costs associated with running a Lightning Network node. You need a computer, a permanent Internet connection, lots of disk space, and lots of time!
There are costs associated with running an LN node. You need a computer, a permanent internet connection, lots of disk space, and lots of time!
Operational costs will include electricity expenses.
But the skills you will learn from this experience are valuable and can be applied to a variety of other tasks too.
@ -26,7 +26,7 @@ Let's get started!
It is important that you set your own expectations correctly on accurate facts.
If you plan to operate a Lightning node _solely_ to gain income by earning routing fees,
please do your homework diligently first. Running a profitable business by operating a Lightning node is
definitely _not_ easy. Calculate all your initial and ongoing costs in a spreadsheet. Study Lightning Network statistics carefully.
definitely _not_ easy. Calculate all your initial and ongoing costs in a spreadsheet. Study LN statistics carefully.
What is the current payment volume? What is the volume per node? What are the current average routing fees? Consult forums and ask
for advice or feedback from other community members who have already gained real-world experience. Form your own educated opinion only
_after_ you have done this due diligence exercise. Most people will find their motivation for running a node not in financial gain,
@ -35,20 +35,20 @@ but somewhere else.
=== Choosing Your Platform
There are many ways you can run a Lightning node ranging from a small mini-PC hosted in your home or a dedicated server to a hosted server in the cloud. The method you choose will depend on the resources you have and how much money you want to spend.
There are many ways you can run a Lightning node ranging from a small mini PC hosted in your home or a dedicated server to a hosted server in the cloud. The method you choose will depend on the resources you have and how much money you want to spend.
[[continuous_operation]]
==== Why Is Reliability Important for Running a Lightning Node?
In Bitcoin, hardware is not particularly important unless one is specifically running a mining node.
The Bitcoin Core node software can be run on any machine that meets its minimum requirements and does not need to be online to receive payments; only to send them.
If a Bitcoin node goes down for an extended period of time, the user can simply reboot the node and once it connects to the rest of the network, it will re-sync the blockchain.
If a Bitcoin node goes down for an extended period of time, the user can simply reboot the node, and once it connects to the rest of the network, it will resync the blockchain.
In Lightning, however, the user needs to be online both to send _and_ to receive payments. If the Lightning node is offline, it cannot receive any payments from anyone and thus its open invoices cannot be fulfilled.
Furthermore, the open channels of an offline node cannot be used to route payments. Your channel partners will notice that you are offline and cannot contact you to route a payment. If you are offline too often, they may consider the bitcoin locked up in their channels with you to be underutilized capacity, and may close those channels. We already discussed the case of a protocol attack where your channel partner tries to cheat you by submitting an earlier commitment transaction. If you are offline and your channels aren't being monitored, then the attempted theft could succeed and you will have no recourse once the timelock expires.
In Lightning, however, the user needs to be online both to send _and_ to receive payments. If the Lightning node is offline, it cannot receive any payments from anyone, and thus its open invoices cannot be fulfilled.
Furthermore, the open channels of an offline node cannot be used to route payments. Your channel partners will notice that you are offline and cannot contact you to route a payment. If you are offline too often, they may consider the bitcoin locked up in their channels with you to be underutilized capacity, and may close those channels. We already discussed the case of a protocol attack in which your channel partner tries to cheat you by submitting an earlier commitment transaction. If you are offline and your channels aren't being monitored, then the attempted theft could succeed, and you will have no recourse once the timelock expires.
Hence, node reliability is extremely important for a Lightning node.
There are also the issues of hardware failure and loss of data. In Bitcoin, a hardware failure can be a trivial problem if the user has a backup of their mnemonic phrase or private keys. The Bitcoin wallet and the bitcoin inside the wallet can be easily restored from the private keys on a new computer. Most information can be re-downloaded from the blockchain.
There are also the issues of hardware failure and loss of data. In Bitcoin, a hardware failure can be a trivial problem if the user has a backup of their mnemonic phrase or private keys. The Bitcoin wallet and the bitcoin inside the wallet can be easily restored from the private keys on a new computer. Most information can be redownloaded from the blockchain.
In contrast, in Lightning the information about the user's channels, including the commitment transactions and revocation secrets, are not publicly known and are only stored on the individual user's hardware.
Thus, software and hardware failures in the Lightning Network can easily result in loss of funds.
@ -57,31 +57,31 @@ Thus, software and hardware failures in the Lightning Network can easily result
There are three main types of hardware Lightning nodes:
* **General-purpose computers**: A Lightning Network node can be run on a home computer or laptop running Windows, Mac OS, or Linux. Typically this is run alongside a Bitcoin node.
* **Dedicated hardware**: A Lightning node can also be run on dedicated hardware like a Raspberry Pi, Rock64, or mini PC. This setup would usually run a software stack including a Bitcoin node and other applications. This setup is popular as the hardware is dedicated to running and maintaining the Lightning node only and is usually set up with an installation "helper".
* **Pre-configured hardware**: A Lightning Network node can also be run on purpose-built hardware specifically selected and configured for it. This would include "out-of-the-box" Lightning node solutions that can be purchased as a kit or a turn-key system.
* **General-purpose computers**: An LN node can be run on a home computer or laptop running Windows, macOS, or Linux. Typically this is run alongside a Bitcoin node.
* **Dedicated hardware**: A Lightning node can also be run on dedicated hardware like a Raspberry Pi, Rock64, or mini PC. This setup would usually run a software stack including a Bitcoin node and other applications. This setup is popular because the hardware is dedicated to running and maintaining the Lightning node only and is usually set up with an installation "helper."
* **Preconfigured hardware**: An LN node can also be run on purpose-built hardware specifically selected and configured for it. This would include "out-of-the-box" Lightning node solutions that can be purchased as a kit or a turnkey system.
==== Running in the "Cloud"
_Virtual Private Server_ (VPS) and "cloud computing" services such as Microsoft Azure, Google Cloud, Amazon Web Services (AWS), or DigitalOcean are quite affordable and can be set up very quickly. A Lightning node can be hosted for between $20 and $40 per month on such a service.
_Virtual private server_ (VPS) and cloud computing services such as Microsoft Azure, Google Cloud, Amazon Web Services (AWS), or DigitalOcean are quite affordable and can be set up very quickly. A Lightning node can be hosted for between $20 and $40 per month on such a service.
However, as the saying goes, "'Cloud' is just other people's computers". Using these services means running your node on other people's computers. This brings along the corresponding advantages and disadvantages. The key advantages are convenience, efficiency, uptime, and possibly even cost. The cloud operator manages and runs the node to a high degree automatically providing you with convenience and efficiency. They provide excellent uptime and availability, often much better than what an individual can achieve at home. If you consider that just the electricity cost of running a server in many western countries is around $10 per month, then add to that the cost of network bandwidth and the hardware itself, the VPS offering becomes financially competitive. Lastly, with a VPS you need no space for a PC at home, and don't have any issues with PC noise or heat.
On the other hand there are several notable disadvantages. A Lightning node running in the "cloud" will always be less secure and less private than one running on your own computer. Additionally, these cloud computing services are very centralized. The vast majority of Bitcoin and Lightning nodes running on such services are located in a handful of data centers in Virginia, Sunnyvale, Seattle, London, and Frankfurt. When the networks or data centers of these providers have service problems, it affects thousands of nodes on so-called "decentralized" networks.
However, as the saying goes, "‘Cloud’ is just other people's computers." Using these services means running your node on other people's computers. This brings along the corresponding advantages and disadvantages. The key advantages are convenience, efficiency, uptime, and possibly even cost. The cloud operator manages and runs the node to a high degree automatically providing you with convenience and efficiency. They provide excellent uptime and availability, often much better than what an individual can achieve at home. If you consider that just the electricity cost of running a server in many Western countries is around $10 per month, then add to that the cost of network bandwidth and the hardware itself, the VPS offering becomes financially competitive. Lastly, with a VPS you need no space for a PC at home and don't have any issues with PC noise or heat.
On the other hand, there are several notable disadvantages. A Lightning node running in the "cloud" will always be less secure and less private than one running on your own computer. Additionally, these cloud computing services are very centralized. The vast majority of Bitcoin and Lightning nodes running on such services are located in a handful of data centers in Virginia, Sunnyvale, Seattle, London, and Frankfurt. When the networks or data centers of these providers have service problems, it affects thousands of nodes on so-called "decentralized" networks.
If you have the possibility and capacity of running a node on your own computer at home or in your office, then this might be preferable to running it
in the cloud. Nonetheless, if running your own server is not an option, by all means consider running one on a VPS.
==== Running a Node At Home
==== Running a Node at Home
If you have a reasonable capacity Internet connection at home or in your office, you can certainly run a Lightning node there. Any "broadband" connection is sufficient for the purpose of running a lightweight node, and a fast connection will allow you to run a Bitcoin full node too.
If you have a reasonable-capacity internet connection at home or in your office, you can certainly run a Lightning node there. Any "broadband" connection is sufficient for the purpose of running a lightweight node, and a fast connection will allow you to run a Bitcoin full node too.
While you can run a Lightning node (and even a Bitcoin node) on your laptop, it will become annoying quite fast. These programs consume your computer's resources and need to run 24/7. Your user applications like your browser or your spreadsheet will find themselves competing against the Lightning background services for your computer's resources. In other words, your browser and other desktop workloads will be slowed down.
And when your word-processing app freezes up your laptop, your Lightning node will go down as well leaving you unable to receive transactions and potentially vulnerable to attacks. Furthermore, you should never turn off your laptop.
All this combined together results in a set-up that is not ideal. The same will apply to your daily-use personal desktop PC.
And when your word-processing app freezes up your laptop, your Lightning node will go down as well, leaving you unable to receive transactions and potentially vulnerable to attacks. Furthermore, you should never turn off your laptop.
All this combined together results in a setup that is not ideal. The same will apply to your daily-use personal desktop PC.
Instead, most users will choose to run a node on a dedicated computer.
Fortunately, you don't need a "server" class computer to do this.
You can run a Lightning node on a single-board computer, such as a Raspberry Pi or on a Mini PC (usually marketed as home theater PCs).
You can run a Lightning node on a single-board computer, such as a Raspberry Pi or on a mini PC (usually marketed as home theater PCs).
These are simple computers which are commonly used as a home automation hub or a media server.
They are relatively inexpensive, when compared to a PC or a laptop.
The advantage of a dedicated device as a platform for Lightning and Bitcoin nodes is that it can run continuously, silently, and unobtrusively on your home network, tucked behind your router or TV.
@ -89,106 +89,100 @@ No one will even know that this little box is actually part of a global banking
[WARNING]
====
Operating a node on a 32-bit operating system and/or 32-bit CPU is not recommended, as the node software may run into resource issues, causing a crash and possibly a loss of funds.
Operating a node on a 32-bit operating system and/or 32-bit CPU is not recommended, because the node software may run into resource issues, causing a crash and possibly a loss of funds.
====
==== What Hardware Is Required to Run a Lightning Node?
At a minimum, the following will be required to run a Lightning node:
* **CPU**: Sufficient processing power will be required to run a Bitcoin node, which will continuously download and validate new blocks. The user also needs to consider the Initial Block Download (IBD) when setting up a new Bitcoin node, which can take anywhere from several hours to several days. A 2-core or 4-core CPU is recommended.
* **CPU**: Sufficient processing power will be required to run a Bitcoin node, which will continuously download and validate new blocks. The user also needs to consider the initial block download (IBD) when setting up a new Bitcoin node, which can take anywhere from several hours to several days. A 2-core or 4-core CPU is recommended.
* **RAM**: A system with 2GB of RAM will _barely_ run both Bitcoin and Lightning nodes. It will perform much better with at least 4GB of RAM. The Initial Block Download will be especially challenging with less than 4GB of RAM. More than 8GB of RAM is unnecessary, because the CPU is the greater bottleneck for these types of services, due to cryptographic operations such as signature validation.
* **RAM**: A system with 2 GB of RAM will _barely_ run both Bitcoin and Lightning nodes. It will perform much better with at least 4 GB of RAM. The IBD will be especially challenging with less than 4 GB of RAM. More than 8 GB of RAM is unnecessary because the CPU is the greater bottleneck for these types of services, due to cryptographic operations such as signature validation.
* **Storage drive**: This can be a Hard Disk Drive (HDD) or a Solid State Drive (SSD).
* **Storage drive**: This can be a hard disk drive (HDD) or a solid state drive (SSD).
An SSD will be significantly quicker (but more expensive) for running a node.
Most of the storage is used for the Bitcoin blockchain, which is hundreds of gigabytes in size.
A fair tradeoff (cost for complexity) is to buy a small SSD to boot the OS from and a larger HDD to store large data objects (mostly databases).
A fair trade-off (cost for complexity) is to buy a small SSD to boot the OS from and a larger HDD to store large data objects (mostly databases).
[NOTE]
====
Raspberry Pis are a common choice for running node software, due to the cost and parts availability.
The OS that runs on the device usually boots from an SD card.
For most use-cases, this is a non-issue, but Bitcoin Core is notorious for being I/O heavy.
The OS that runs on the device usually boots from a secure digital (SD) card.
For most use cases, this is a nonissue, but Bitcoin Core is notorious for being I/O heavy.
You should make sure to place the Bitcoin blockchain and Lightning data directory on a different drive because long-term intensive I/O can cause an SD card to fail.
====
* **Internet connection**: A reliable Internet connection will be required to download new Bitcoin blocks, as well as to communicate with other Lightning peers. During operation the estimated data use ranges from 10GB to 100GB per month, depending on configuration. At startup, a Bitcoin full node downloads the full blockchain.
* **Internet connection**: A reliable internet connection will be required to download new Bitcoin blocks, as well as to communicate with other Lightning peers. During operation the estimated data use ranges from 10 to 100 GB per month, depending on configuration. At startup, a Bitcoin full node downloads the full blockchain.
* **Power supply**: A reliable power supply is required as Lightning nodes need to be online at all times. A power failure will cause in-progress payments to fail. For heavy duty routing nodes, a backup or uninterruptible power supply (UPS) is useful in the event of power outages.
Ideally, you should connect your Internet router to this UPS as well.
* **Power supply**: A reliable power supply is required because Lightning nodes need to be online at all times. A power failure will cause in-progress payments to fail. For heavy duty routing nodes, a backup or uninterruptible power supply (UPS) is useful in the event of power outages.
Ideally, you should connect your internet router to this UPS as well.
* **Backup**: Backup is crucial as a failure can result in loss of data and hence in loss of funds.
* **Backup**: Backup is crucial because a failure can result in loss of data and hence in loss of funds.
The user will want to consider some kind of data backup solution. This could be a cloud-based automated backup to a server or web service the user controls. Alternatively, it could be an automated local hardware backup, such as a second hard drive. For the best results, both local and remote backup can be combined.
==== Switching Server Configuration in the Cloud
When renting a cloud server, it is often cost effective to change the configuration between two phases of operation. A faster CPU and faster storage will be needed during the Initial Block Download (e.g. the first day). After the blockchain has synced, the CPU and storage speed requirements are much less, so the performance can be downgraded to a more cost-effective level.
When renting a cloud server, it is often cost effective to change the configuration between two phases of operation. A faster CPU and faster storage will be needed during the IBD (e.g., the first day). After the blockchain has synced, the CPU and storage speed requirements are much less, so the performance can be downgraded to a more cost-effective level.
For example, on Amazon's cloud, we would use a 8-16GB RAM, 8-core CPU (e.g. t3-large or m3.large) and faster 400GB SSD (1000+ provisioned IOPS) for the Initial Block Download, reducing its time to just 6-8 hours. Once that is complete, we would switch the server instance to a 2GB RAM, 2-core CPU (e.g. t3.small) and storage to a general purpose 1TB HDD. This will cost about the same as if you ran it on the slower server the entire time, but it will get you up and running in less than a day instead of having to wait almost a week for the IBD.
For example, on Amazon's cloud, we would use an 8–16 GB RAM, 8-core CPU (e.g. t3-large or m3.large) and faster 400 GB SSD (1000+ provisioned input/output operations per second [IOPS]) for the IBD, reducing its time to just 6-8 hours. Once that is complete, we would switch the server instance to a 2 GB RAM, 2-core CPU (e.g., t3.small) and storage to a general purpose 1 TB HDD. This will cost about the same as if you ran it on the slower server the entire time, but it will get you up and running in less than a day instead of having to wait almost a week for the IBD.
===== Permanent data storage (drive)
If you use a mini PC or rent a server, the storage can be the most expensive part, costing as much as the computer and connectivity (data) added together.
Let's have a look at the different options available. First there are two main types of drives, Hard Disk Drives (HDDs) and Solid State Drives (SSDs). HDDs are cheaper and SSDs are faster, but both do the job.
Let's have a look at the different options available. First there are two main types of drives, HDDs and SSDs. HDDs are cheaper and SSDs are faster, but both do the job.
The fastest SSDs available today use the NVMe interface. The NVMe SSDs are faster in high-end machines, but also more costly.
The fastest SSDs available today use the Non-Volatile Memory Express (NVMe) interface. The NVMe SSDs are faster in high-end machines, but also more costly.
Traditional SATA-based SSDs are cheaper, but not as fast. SATA SSDs perform sufficiently well for your node setup.
Smaller computers might not be able to take advantage of NVMe SSDs.
For example, the Raspberry Pi 4 cannot benefit from them because of the limited bandwidth of its USB port.
To choose the size, let's look at the Bitcoin blockchain. As of August 2021, its size is 360GB including the transaction index and grows by roughly 60GB/year. If you want to have some margin available for future growth or to install other data on your node, purchase at least a 512GB drive or better yet a 1TB drive.
To choose the size, let's look at the Bitcoin blockchain. As of August 2021, its size is 360 GB including the transaction index and grows by roughly 60 GB per year. If you want to have some margin available for future growth or to install other data on your node, purchase at least a 512 GB drive or better yet a 1 TB drive.
[[helpers]]
=== Using an Installer or Helper
Installing a Lightning node or a Bitcoin node may be daunting if you are not familiar with a command line environment. Luckily, there are a number of projects that make "helpers", i.e. software that installs and configures the various components for you. You will still need to learn some command line incantations to interact with your node, but most of the initial work is done for you.
Installing a Lightning node or a Bitcoin node may be daunting if you are not familiar with a command-line environment. Luckily, there are a number of projects that make "helpers," i.e., software that installs and configures the various components for you. You will still need to learn some command-line incantations to interact with your node, but most of the initial work is done for you.
==== Raspiblitz
==== RaspiBlitz
One of the most popular and complete "helpers" is _RaspiBlitz_ (<<RaspiBlitz>>), a project built by Christian Rotzoll. It is intended to be installed on a Raspberry Pi 4. RaspiBlitz comes with a recommended hardware "kit" that you can build in a matter of hours or at most a weekend. If you attend a Lightning "hackathon" in your city, you are likely to see many people working on their RaspiBlitz setup, swapping tips, and helping each other. You can find the RaspiBlitz project here:
https://github.com/rootzoll/raspiblitz
One of the most popular and complete "helpers" is _RaspiBlitz_ (<<RaspiBlitz>>), a project built by Christian Rotzoll. It is intended to be installed on a Raspberry Pi 4. RaspiBlitz comes with a recommended hardware kit that you can build in a matter of hours or at most a weekend. If you attend a Lightning "hackathon" in your city, you are likely to see many people working on their RaspiBlitz setup, swapping tips, and helping each other. You can find the RaspiBlitz project on https://github.com/rootzoll/raspiblitz[GitHub].
[[RaspiBlitz]]
.caption to be written
.A RaspiBlitz node
image::images/mtln_0501.png[]
In addition to a Bitcoin and Lightning node, RaspiBlitz can install a number of additional services, such as:
* Tor (run as hidden service)
* ElectRS (Electrum server in Rust)
* BTCPayServer (cryptocurrency payment processor)
* BTCPay Server (cryptocurrency payment processor)
* BTC RPC Explorer (Bitcoin blockchain explorer)
* Ride The Lightning (Lightning node management GUI)
* LNbits (Lightning wallet/accounts system)
* Specter Desktop (multisig Trezor, Ledger, Coldcard wallet & Specter-DIY)
* LNDmanage (command line interface for advanced channel management)
* LNDmanage (command-line interface for advanced channel management)
* Loop (submarine swaps service)
* JoinMarket (CoinJoin service)
==== Mynode
_myNode_ is another popular open source "helper" project including a lot of Bitcoin related software. It is easy to install: you "flash" the installer onto an SD card and boot your mini-PC from the SD card. You do not need any monitor to use myNode as the administrative tools are accessible remotely from a browser. If your mini-PC has no monitor, mouse, or keyboard; you can manage it from another computer or even from your smartphone. Once installed, go to http://mynode.local/ and create a Lightning wallet and node in two clicks.
You can find the myNode project here:
https://mynodebtc.com/
https://mynodebtc.com/[_myNode_] is another popular open source "helper" project including a lot of Bitcoin related software. It is easy to install: you "flash" the installer onto an SD card and boot your mini PC from the SD card. You do not need any monitor to use myNode because the administrative tools are accessible remotely from a browser. If your mini PC has no monitor, mouse, or keyboard, you can manage it from another computer or even from your smartphone. Once installed, go to http://mynode.local/ and create a Lightning wallet and node in two clicks.
In addition to a Bitcoin and Lightning node, myNode can optionally install a variety of additional services, such as:
* Ride The Lightning (Lightning node management GUI)
* OpenVPN (VPN Support for remote management or wallet)
* LNDmanage (command line interface for advanced channel management)
* BTC RPC Explorer (A Bitcoin blockchain explorer)
* OpenVPN (virtual private network [VPN] support for remote management or wallet)
* LNDmanage (command-line interface for advanced channel management)
* BTC RPC Explorer (a Bitcoin blockchain explorer)
==== Umbrel
Famous for their UX/UI (shown in <<umbrel>>), _Umbrel_ provides a very easy and accessible way to get your Bitcoin and Lightning node up and running in no time, especially for beginners. A very distinctive feature is that _Umbrel_ utilizes Neutrino/SPV during the Initial Blockchain Download (IBD) so you can instantly start using your node. Once Bitcoin Core is fully synced in the background it automatically switches over and disables SPV mode. Umbrel OS supports the Raspberry Pi 4 and can also be installed on any Linux-based OS or on a virtual machine on macOS or Windows. You can also connect any wallet that supports Bitcoin Core P2P, Bitcoin Core RPC, the Electrum protocol or lndconnect.
Famous for their UX/UI (shown in <<umbrel>>), _Umbrel_ provides a very easy and accessible way to get your Bitcoin and Lightning node up and running in no time, especially for beginners. A very distinctive feature is that Umbrel utilizes Neutrino/SPV during the Initial Blockchain Download (IBD) so you can instantly start using your node. Once Bitcoin Core is fully synced in the background it automatically switches over and disables SPV mode. Umbrel OS supports the Raspberry Pi 4 and can also be installed on any Linux-based OS or on a virtual machine on macOS or Windows. You can also connect any wallet that supports Bitcoin Core P2P, Bitcoin Core RPC, the Electrum protocol, or lndconnect.
There's no need to wait for a rainy day - you can find the project here: https://getumbrel.com
There's no need to wait for a rainy day&mdash;you can go right to https://getumbrel.com[Umbrel] to learn more.
[[umbrel]]
.The Umbrel web interface
@ -196,10 +190,10 @@ image::images/mtln_0502.png["The Umbrel web interface"]
In addition to a Bitcoin and Lightning node, Umbrel introduced the Umbrel App Store, where you can easily install additional services, such as:
* Lightning Terminal (interface for managing channel liquidity, loop-in & loop-out)
* Lightning Terminal (interface for managing channel liquidity, loop-in, and loop-out)
* Ride The Lightning (Lightning node management GUI)
* Specter Desktop (watch-only coordinator for multi-signature and single-key Bitcoin wallets)
* BTCPayServer (cryptocurrency payment processor)
* Specter Desktop (watch-only coordinator for multisignature and single-key Bitcoin wallets)
* BTCPay Server (cryptocurrency payment processor)
* BTC RPC Explorer (Bitcoin blockchain explorer)
* ThunderHub (monitor and manage your node)
* Sphinx Relay (handling connectivity and storage for Sphinx chat)
@ -210,14 +204,12 @@ Umbrel is currently still in beta and not considered secure.
==== BTCPay Server
While not initially designed as an installation "helper", the e-commerce and payment platform _BTCPay Server_ has an incredibly easy installation system that uses Docker containers and +docker-compose+ to install a Bitcoin node, Lightning node, and payment gateway, among many other services. It can be installed on a variety of hardware platforms, from a simple Raspberry Pi 4 (4GB recommended) to a mini PC, old laptop, desktop or server.
While not initially designed as an installation "helper," the ecommerce and payment platform _BTCPay Server_ has an incredibly easy installation system that uses Docker containers and +docker-compose+ to install a Bitcoin node, Lightning node, and payment gateway, among many other services. It can be installed on a variety of hardware platforms, from a simple Raspberry Pi 4 (4 GB recommended) to a mini PC, old laptop, desktop, or server.
BTCPay Server is a fully-featured self-hosted self-custody e-commerce platform that can be integrated with many e-commerce platforms such as Wordpress Woocommerce and others. The installation of the full node is only a step of the e-commerce platform installation.
While originally developed as a feature-for-feature replacement of the _Bitpay_ commercial payment service and API, it has evolved past that to be a complete platform for BTC and Lighting services related to e-commerce. For many sellers or shops it is a one-shop turn-key solution to e-commerce.
BTCPay Server is a fully featured self-hosted, self-custody ecommerce platform that can be integrated with many ecommerce platforms such as WordPress WooCommerce and others. The installation of the full node is only a step of the ecommerce platform installation.
While originally developed as a feature-for-feature replacement of the _Bitpay_ commercial payment service and API, it has evolved past that to be a complete platform for BTC and Lightning services related to ecommerce. For many sellers or shops it is a one-shop turnkey solution to ecommerce.
More information can be found at:
https://btcpayserver.org/
More information can be obtained from https://btcpayserver.org/[BTCPay].
In addition to a Bitcoin and Lightning node, BTCPay Server can also install a variety of services, including:
@ -225,30 +217,30 @@ In addition to a Bitcoin and Lightning node, BTCPay Server can also install a va
* Litecoin support
* Monero support
* Spark server (c-lightning web wallet)
* Charge server (c-lightning e-commerce API)
* Charge server (c-lightning ecommerce API)
* Ride The Lightning (Lightning node management web GUI)
* Many BTC forks
* BTCTransmuter (event-action automation service supporting currency exchange)
The number of additional services and features is growing rapidly, so the list above is only a small subset of what is available on the BTCPay Server platform.
The number of additional services and features is growing rapidly, so the preceding list is only a small subset of what is available on the BTCPay Server platform.
==== Bitcoin Node or Lightweight Lightning
One critical choice for your setup will be the choice of the Bitcoin node and its configuration. _Bitcoin Core_, the reference implementation, is the most common choice but not the only choice available. One alternative choice is _btcd_, which is a Go-language implementation of a Bitcoin node. Btcd supports some features that are useful for running an LND Lightning node and are not available in Bitcoin Core.
A second consideration is whether you will run an _archival_ Bitcoin node with a full copy of the blockchain (some 350GB in mid-2021) or a _pruned_ blockchain that only keeps the most recent blocks. A pruned blockchain can save you some disk space, but you will still need to download the full blockchain at least once (during the Initial Block Download). Hence it won't save you any network traffic. Using a pruned node to run a Lightning node is still an experimental capability and might not support all the functionality. However, many people are running a node like that successfully.
A second consideration is whether you will run an _archival_ Bitcoin node with a full copy of the blockchain (some 350 GB in mid-2021) or a _pruned_ blockchain that only keeps the most recent blocks. A pruned blockchain can save you some disk space, but you will still need to download the full blockchain at least once (during the IBD). Hence it won't save you any network traffic. Using a pruned node to run a Lightning node is still an experimental capability and might not support all the functionality. However, many people are running a node like that successfully.
Finally, you also have the option of not running a Bitcoin node at all. Instead you can operate the LND Lightning node in "lightweight" mode, using the _neutrino_ protocol to retrieve blockchain information from public Bitcoin nodes operated by others. Running like this means that you are taking resources from the Bitcoin network without offering any in return. Instead, you are offering your resources and contributing to the Lightning Network community. For smaller Lightning nodes this will generally reduce network traffic in comparison to running a full Bitcoin node.
Finally, you also have the option of not running a Bitcoin node at all. Instead you can operate the LND Lightning node in "lightweight" mode, using the Neutrino Protocol to retrieve blockchain information from public Bitcoin nodes operated by others. Running like this means that you are taking resources from the Bitcoin network without offering any in return. Instead, you are offering your resources and contributing to the LN community. For smaller Lightning nodes this will generally reduce network traffic in comparison to running a full Bitcoin node.
Keep in mind that operating a Bitcoin node allows you to support other services, besides and on top of a Lightning node. These other services may require an archival (not pruned) Bitcoin node and often can't run without a Bitcoin node. Consider upfront what other services you may want to run now or in the future to make an informed decision on the type of Bitcoin node you select.
The bottom line for this decision is: If you can afford a disk larger than 500GB, run a full archival Bitcoin node. You will be contributing resources to the Bitcoin system and helping others who cannot afford to do so. If you can't afford such a big disk, run a pruned node. If you can't afford the disk or the bandwidth for even a pruned node, run a lightweight LND node over neutrino.
The bottom line for this decision is: If you can afford a disk larger than 500 GB, run a full archival Bitcoin node. You will be contributing resources to the Bitcoin system and helping others who cannot afford to do so. If you can't afford such a big disk, run a pruned node. If you can't afford the disk or the bandwidth for even a pruned node, run a lightweight LND node over Neutrino.
==== Operating System Choice
The next step is to select an operating system for your node. The vast majority of Internet servers run on some variant of Linux. Linux is the platform of choice for the Internet because it is a powerful, open-source operating system. Linux, however, has a steep learning curve and requires familiarity with a command line environment. It is often intimidating for new users.
The next step is to select an operating system for your node. The vast majority of internet servers run on some variant of Linux. Linux is the platform of choice for the internet because it is a powerful open source operating system. Linux, however, has a steep learning curve and requires familiarity with a command-line environment. It is often intimidating for new users.
Ultimately, most of the services can be run on any modern POSIX operating system, which includes Mac OS, Windows, and of course Linux. Your choice should be driven more by your familiarity and comfort with an operating system and your learning objectives. If you want to expand your knowledge and learn how to operate a Linux system, this is a great opportunity to do so with a specific project and a clear goal. If you just want to get a node up and running, go with what you know.
Ultimately, most of the services can be run on any modern POSIX operating system, which includes macOS, Windows, and of course Linux. Your choice should be driven more by your familiarity and comfort with an operating system and your learning objectives. If you want to expand your knowledge and learn how to operate a Linux system, this is a great opportunity to do so with a specific project and a clear goal. If you just want to get a node up and running, go with what you know.
Nowadays, many services are also delivered in the form of containers, usually based on the _Docker_ system. These containers can be deployed on a variety of operating systems, abstracting the underlying OS. You may need to learn some Linux CLI commands nonetheless, as most of the containers run some variant of Linux inside.
@ -262,35 +254,35 @@ Familiarity with the programming language and build system, on the other hand, i
* go utilities for LND
* Java/Maven for Eclair
The programming language doesn't just influence the choice of build system, but also many other aspects of the program. Each programming language comes with a whole design philosophy and affects many other aspects, such as:
The programming language influences not only the choice of build system but also many other aspects of the program. Each programming language comes with a whole design philosophy and affects many other aspects, such as:
* format and syntax of configuration files
* file locations (in the filesystem)
* command line arguments and their syntax
* error message formatting
* prerequisite libraries
* Remote Procedure Call interfaces
* Format and syntax of configuration files
* File locations (in the filesystem)
* Command-line arguments and their syntax
* Error message formatting
* Prerequisite libraries
* Remote procedure call interfaces
When you choose your Lightning node, you are also choosing all of the above characteristics. So your familiarity with these tools and design philosophies will make it easier to run a node. Or harder, if you land in an unfamiliar domain.
When you choose your Lightning node, you are also choosing all the aforementioned characteristics. So your familiarity with these tools and design philosophies will make it easier to run a node. Or harder, if you land in an unfamiliar domain.
On the other hand, if this is your first foray into the command line and server/service environment, you will find yourself unfamiliar with any implementation and have the opportunity to learn something completely new. In that case you might want to decide based on a number of other factors, such as:
On the other hand, if this is your first foray into the command-line and server/service environment, you will find yourself unfamiliar with any implementation and have the opportunity to learn something completely new. In that case you might want to decide based on a number of other factors, such as:
* quality of support forums and chat rooms
* quality of documentation
* degree of integration with other tools you want to run
* Quality of support forums and chat rooms
* Quality of documentation
* Degree of integration with other tools you want to run
As a final consideration, you may want to examine the performance and reliability of different node implementations. This is especially important if you will be using this node in a production environment and expect heavy traffic and high reliability requirements. This might be the case if you plan to run the payment system of a shop on it.
=== Installing a Bitcoin or Lightning Node
You decided not to use an installation "helper" and instead to dive into the command line of a Linux operating system? That is a brave decision and we'll try to help you make it work. If you'd rather not try to do this manually, consider using an application that helps you install the node software or a container-based solution, as described in <<helpers>>.
You decided not to use an installation "helper" and instead to dive into the command line of a Linux operating system? That is a brave decision, and we'll try to help you make it work. If you'd rather not try to do this manually, consider using an application that helps you install the node software or a container-based solution, as described in <<helpers>>.
[WARNING]
====
This section will delve into the advanced topic of system administration from the command line. Linux administration is its own skill set that is outside the scope of this book. It is a complicated topic and there are many pitfalls. Proceed with caution!
====
In the next few sections we will briefly describe how to install and configure a Bitcoin and Lightning node on a Linux operating system. You will need to review the installation instructions for the specific Bitcoin and Lightning node applications you decided to use. You can usually find these in a file called +INSTALL+ or in the +docs+ sub-directory of each project. We will only describe some of the common steps that apply to all such services, and the instructions we offer will be necessarily incomplete.
In the next few sections we will briefly describe how to install and configure a Bitcoin and Lightning node on a Linux operating system. You will need to review the installation instructions for the specific Bitcoin and Lightning node applications you decided to use. You can usually find these in a file called _INSTALL_ or in the +docs+ subdirectory of each project. We will only describe some of the common steps that apply to all such services, and the instructions we offer will be necessarily incomplete.
==== Background Services
@ -300,9 +292,9 @@ This can create some confusion for users who are not used to running background
==== Process Isolation
Background services usually run under a specific user account in order to isolate them from the operating system and each other. For example, Bitcoin Core is configured to run as user +bitcoin+. You will need to use the command line to create a user for each of the services you run.
Background services usually run under a specific user account to isolate them from the operating system and each other. For example, Bitcoin Core is configured to run as user +bitcoin+. You will need to use the command line to create a user for each of the services you run.
In addition, if you have connected an external drive, you will need to tell the operating system to relocate the user's home directory to that drive. That's because a service like Bitcoin Core will create files under the user's home directory. If you are setting it up to download the full Bitcoin blockchain, these files will take up several hundred gigabytes. Here, we assume you have connected the external drive and it is located on the +/external_drive/+ path of the operating system.
In addition, if you have connected an external drive, you will need to tell the operating system to relocate the user's home directory to that drive. That's because a service like Bitcoin Core will create files under the user's home directory. If you are setting it up to download the full Bitcoin blockchain, these files will take up several hundred gigabytes. Here, we assume you have connected the external drive and it is located on the _/external_drive/_ path of the operating system.
On most Linux systems you can create a new user with the +useradd+ command, like this:
@ -310,11 +302,11 @@ On most Linux systems you can create a new user with the +useradd+ command, like
$ sudo useradd -m -d /external_drive/bitcoin -s /dev/null bitcoin
----
The +m+ and +d+ flags create the user's home directory as specified by /external_drive/bitcoin in this case. The +s+ flag assigns the user's interactive shell. In this case, we set it to +/dev/null+ to disable interactive shell use. The last argument is the new user's username +bitcoin+.
The +m+ and +d+ flags create the user's home directory as specified by _/external_drive/bitcoin_ in this case. The +s+ flag assigns the user's interactive shell. In this case, we set it to _/dev/null_ to disable interactive shell use. The last argument is the new user's username +bitcoin+.
==== Node Startup
For both Bitcoin and Lightning node services, "installation" also involves creating a so called _startup script_ to make sure that the node starts when the computer boots. Startup and shutdown of background services is handled by an operating system process, which in Linux is called _init_ or _systemd_. You can usually find a system startup script in the +contrib+ sub-directory of each project. For example, if you are on a modern Linux OS that uses +systemd+, you would find a script called +bitcoind.service+, that can start and stop the Bitcoin Core node service.
For both Bitcoin and Lightning node services, "installation" also involves creating a so-called _startup script_ to make sure that the node starts when the computer boots. Startup and shutdown of background services is handled by an operating system process, which in Linux is called +init+ or +systemd+. You can usually find a system startup script in the +contrib+ subdirectory of each project. For example, if you are on a modern Linux OS that uses +systemd+, you would find a script called _bitcoind.service_ that can start and stop the Bitcoin Core node service.
Here's an example of what a Bitcoin node's startup script looks like, taken from the Bitcoin Core code repository:
@ -430,11 +422,11 @@ In general, it is a good idea to minimize the amount of customization of these s
Network configuration is normally not an issue when configuring a new application. However, peer-to-peer networks like Bitcoin and the Lightning network present some unique challenges for network configuration.
In a centralized service, your computer connects to the "big servers" of some corporation, and not vice-versa. Your home Internet connection is actually configured on the assumption that you are simply a consumer of services provided by others. But in a peer-to-peer system, every peer both consumes from and provides services to other nodes. If you're running a Bitcoin or Lightning node at your home, you're providing a service to other computers on the Internet. Your Internet service by default is not configured to allow you to run servers and may need some additional configuration to enable others to reach your node.
In a centralized service, your computer connects to the "big servers" of some corporation, and not vice-versa. Your home internet connection is actually configured on the assumption that you are simply a consumer of services provided by others. But in a peer-to-peer system, every peer both consumes from and provides services to other nodes. If you're running a Bitcoin or Lightning node at your home, you're providing a service to other computers on the internet. Your internet service by default is not configured to allow you to run servers and may need some additional configuration to enable others to reach your node.
If you want to run a Bitcoin or Lightning node, you need to make it possible for other nodes on the Internet to connect to you. That means enabling incoming TCP connections to the Bitcoin port (port 8333 by default) or Lightning port (port 9735 by default). While you can run a Bitcoin node without incoming connectivity, you can't do that with a Lightning node. A Lightning node must be accessible to others from outside your network.
If you want to run a Bitcoin or Lightning node, you need to make it possible for other nodes on the internet to connect to you. That means enabling incoming TCP connections to the Bitcoin port (port 8333 by default) or Lightning port (port 9735 by default). While you can run a Bitcoin node without incoming connectivity, you can't do that with a Lightning node. A Lightning node must be accessible to others from outside your network.
By default, your home Internet router does not expect incoming connections from the outside, and in fact incoming connections are blocked. Your Internet router IP address is the only externally accessible IP address, and all the computers you run inside your home network share that single IP address. This is achieved by a mechanism called _Network Address Translation (NAT)_ which allows your Internet router to act as an intermediary for all outbound connections. If you want to allow an inbound connection you have to set up _Port Forwarding_, which tells your Internet router that incoming connections on specific ports should be forwarded to specific computers inside the network. You can do this manually by changing your Internet router configuration or, if your router supports it, through an automatic port forwarding mechanism called _Universal Plug and Play (UPNP)_.
By default, your home internet router does not expect incoming connections from the outside, and in fact incoming connections are blocked. Your internet router IP address is the only externally accessible IP address, and all the computers you run inside your home network share that single IP address. This is achieved by a mechanism called _Network Address Translation (NAT)_ which allows your internet router to act as an intermediary for all outbound connections. If you want to allow an inbound connection you have to set up _Port Forwarding_, which tells your internet router that incoming connections on specific ports should be forwarded to specific computers inside the network. You can do this manually by changing your internet router configuration or, if your router supports it, through an automatic port forwarding mechanism called _Universal Plug and Play (UPNP)_.
An alternative mechanism to port forwarding is to enable The Onion Router (Tor), which provides a kind of virtual private network overlay that allows incoming connections to an _onion address_. If you run Tor, you don't need to do port forwarding nor enable incoming connections to Bitcoin or Lightning ports. If you run your nodes using Tor, all traffic goes through Tor and no other ports are used.
@ -442,7 +434,7 @@ Let's look at different ways you can make it possible for others to connect to y
===== It just works!
There is a possibility that your Internet service provider or router is configured to support UPNP by default and everything just works automatically. Let's try this approach first, just in case we are lucky.
There is a possibility that your internet service provider or router is configured to support UPNP by default and everything just works automatically. Let's try this approach first, just in case we are lucky.
Assuming you already have a Bitcoin or Lightning node running, we will try and see if they are accessible from the outside.
@ -467,19 +459,19 @@ In <<ln_port_check>> you can see the result of checking port 9735 on a server ru
===== Automatic port forwarding using upnp
Sometimes, even if your Internet router supports UPNP, it may be turned off by default. In that case you need to change your Internet router configuration from its web administration interface:
Sometimes, even if your internet router supports UPNP, it may be turned off by default. In that case you need to change your internet router configuration from its web administration interface:
. Connect to your Internet router's configuration website. Usually this can be done by connecting to the _gateway address_ of your home network using a web browser. You can find the gateway address by looking at the IP configuration of any computer on your home network. It is often the first address in one of the non-routable networks, like 192.168.0.1 or 10.0.0.1. Check all stickers on your router as well for the _gateway address_. Once found, open a browser and enter the IP address into the browser URL/Search box, e.g. "192.168.0.1" or "http://192.168.0.1".
. Connect to your internet router's configuration website. Usually this can be done by connecting to the _gateway address_ of your home network using a web browser. You can find the gateway address by looking at the IP configuration of any computer on your home network. It is often the first address in one of the non-routable networks, like 192.168.0.1 or 10.0.0.1. Check all stickers on your router as well for the _gateway address_. Once found, open a browser and enter the IP address into the browser URL/Search box, e.g. "192.168.0.1" or "http://192.168.0.1".
. Find the administrator username and password for the web configuration panel of the router. This is often written on a sticker on the router itself and may be as simple as "admin" and "password". A quick web search for your ISP and router model can also help you find this information.
. Find a setting for UPNP and turn it on.
Restart your Bitcoin and/or Lighting node and repeat the open port test with one of the websites we used in the previous section.
Restart your Bitcoin and/or Lightning node and repeat the open port test with one of the websites we used in the previous section.
===== Using tor for incoming connections
_The Onion Router (Tor)_ is a virtual private network with the special property that it encrypts communications between hops, such that any intermediary node cannot determine the origin or destination of a packet. Both Bitcoin and Lightning nodes support operation over Tor, which enables you to operate a node without revealing your IP address or location. Hence, it provides a high level of privacy to your network traffic. An added benefit of running Tor is that because it operates as a VPN, it resolves the problem of port forwarding from your Internet router. Incoming connections are received over the Tor tunnel, and your node can be found through an ad-hoc generated _onion address_ instead of an IP address.
_The Onion Router (Tor)_ is a virtual private network with the special property that it encrypts communications between hops, such that any intermediary node cannot determine the origin or destination of a packet. Both Bitcoin and Lightning nodes support operation over Tor, which enables you to operate a node without revealing your IP address or location. Hence, it provides a high level of privacy to your network traffic. An added benefit of running Tor is that because it operates as a VPN, it resolves the problem of port forwarding from your internet router. Incoming connections are received over the Tor tunnel, and your node can be found through an ad-hoc generated _onion address_ instead of an IP address.
Enabling Tor requires two steps: First you must install the Tor router and proxy on your computer. Second, you must enable the use of the Tor proxy in your Bitcoin or Lightning configuration.
@ -509,24 +501,24 @@ curl --socks5 localhost:9050 --socks5-hostname localhost:9050 -s https://check.t
If everything is working properly, the response of this command should be +"Congratulations. This browser is configured to use Tor."+.
Due to the nature of Tor, you can't easily use an external service to check if your node is reachable via an onion address. Nonetheless, you should see your Tor onion address in the logs of your Lightning node. It is a long string of letters and numbers followed by the suffix +.onion+. Your node should now be reachable from the Internet, with the added bonus of privacy!
Due to the nature of Tor, you can't easily use an external service to check if your node is reachable via an onion address. Nonetheless, you should see your Tor onion address in the logs of your Lightning node. It is a long string of letters and numbers followed by the suffix +.onion+. Your node should now be reachable from the internet, with the added bonus of privacy!
===== Manual port forwarding
This is the most complex process and requires quite a bit of technical skill. The details depend on the type of Internet router you have, your service provider settings and policies, and a lot of other context. Try UPNP or Tor first, before you try this much more difficult mechanism.
This is the most complex process and requires quite a bit of technical skill. The details depend on the type of internet router you have, your service provider settings and policies, and a lot of other context. Try UPNP or Tor first, before you try this much more difficult mechanism.
The basic steps are as follows:
. Find the IP address of the computer your node is on. This is usually dynamically allocated by the Dynamic Host Configuration Protocol (DHCP) and is often somewhere in the 192.168.x.x or 10.x.x.x range.
. Find the Media Access Control (MAC) address of your node's network interface. This can be found in the Internet settings of that computer.
. Find the Media Access Control (MAC) address of your node's network interface. This can be found in the internet settings of that computer.
. Assign a static IP address for your node so that it is always the same one. You can use the IP address it currently has. On your Internet router, look for "Static Leases" under the DHCP configuraiton. Map the MAC address to the IP address you selected. Now your node will always have that IP address allocated to it. Alternatively, you can look at your router's DHCP configuration and find out what its DHCP address range is. Select an unused address _outside_ of the DHCP address range. Then, on the server, configure the network to stop using DHCP and hard-code the selected non-DHCP IP address into the operating system network configuration.
. Assign a static IP address for your node so that it is always the same one. You can use the IP address it currently has. On your internet router, look for "Static Leases" under the DHCP configuration. Map the MAC address to the IP address you selected. Now your node will always have that IP address allocated to it. Alternatively, you can look at your router's DHCP configuration and find out what its DHCP address range is. Select an unused address _outside_ of the DHCP address range. Then, on the server, configure the network to stop using DHCP and hard-code the selected non-DHCP IP address into the operating system network configuration.
. Finally, set up "Port Forwarding" on your Internet router to route incoming traffic on specific ports to the selected IP address of your server.
. Finally, set up "Port Forwarding" on your internet router to route incoming traffic on specific ports to the selected IP address of your server.
Once done re-configuring, repeat the port check using one of the websites from the previous sections.
Once done reconfiguring, repeat the port check using one of the websites from the previous sections.
=== Security of Your Node
@ -552,7 +544,7 @@ To secure your operating system, here are some of the top items to consider:
. Two-factor authentication (2FA): Use two-factor authentication wherever possible, including Universal 2-Factor (U2F) with hardware security keys. This applies to all external services you might be using such as your cloud service provider. You can apply this also to your own set-up such as your own SSH configuration. Use 2FA also for indirect services. For example, say you are using a cloud service. You gave your cloud service provider an email address, so you should also protect your email address with 2FA.
. Backup: Make backups of your system, and make sure you protect the backups with encryption too. Perform these backups periodically. At least once test if you can restore your backup and that your backup is complete and accessible. If possible, keep one copy of your backups on a different disk to avoid that a single hard disk failure destroys _both_ your active node as well as your backup copies.
. Vulnerability and exposure management: Use remote scanning to ensure you have minimized the attack surface of your system. Close any unnecessary services or ports.
- Minimize: Install only software and packages that you really need and use. Uninstall packages that you no longer use. It is recommended that you do _not_ use your node computer for non-node activities that you can perform on another of your computers. Especially, if you can, do _not_ use your node computer for browsing, surfing the Internet, or reading your email.
- Minimize: Install only software and packages that you really need and use. Uninstall packages that you no longer use. It is recommended that you do _not_ use your node computer for non-node activities that you can perform on another of your computers. Especially, if you can, do _not_ use your node computer for browsing, surfing the internet, or reading your email.
This is a list of the most basic security measures. It is by no means exhaustive.
@ -587,7 +579,7 @@ Channel backup mechanisms are still a work-in-progress and a weakness in most Li
==== Static Channel Backups (SCB)
At the time of writing this book, only LND offers a built-in mechanism for static channel backups. Eclair has a similar mechanism deployed for server-side deployments, although Eclair mobile does offer optional backup to a Google Drive. C-lightning recently merged the necessary interfaces for a plugin to implement channel backups. Unfortunately, there is no consistent, agreed upon backup mechanism across different node implementations.
At the time of writing this book, only LND offers a built-in mechanism for static channel backups. Eclair has a similar mechanism deployed for server-side deployments, although Eclair mobile does offer optional backup to a Google Drive. C-lightning recently merged the necessary interfaces for a plug-in to implement channel backups. Unfortunately, there is no consistent, agreed upon backup mechanism across different node implementations.
File-based backups of the Lightning node databases are at best a partial solution because you run the risk of backing up an inconsistent database state. In addition, you may not reliably catch the latest state commitments. It is much better to have a backup mechanism that is triggered every time there is a state change in a channel, thereby ensuring data consistency.
@ -597,7 +589,7 @@ To set up static channel backups in LND, set the +backupfilepath+ parameter eith
As we've discussed previously, the Lightning Network consists of a network of _hot wallets_. The funds you store in a Lightning wallet are *online all the time*. This makes them vulnerable. Hence, you should not store large amounts in a Lightning wallet. Large amounts should be kept in a _cold_ wallet that is _not_ online and which can transact only on-chain.
Even if you start small, as time passes you may still find you have a significant amount of money in a Lightning wallet. This is a typical scenario for store owners. If you use a Lightning node for an e-commerce operation, your wallet will likely receive funds often, but send funds rarely. You will therefore end up having two problems simultaneously: First, your channels will be imbalanced with large local balances outweighing small remote balances. Secondly, you will have too much money in the wallet. Fortunately, you can also solve both of these problems simultaneously.
Even if you start small, as time passes you may still find you have a significant amount of money in a Lightning wallet. This is a typical scenario for store owners. If you use a Lightning node for an ecommerce operation, your wallet will likely receive funds often, but send funds rarely. You will therefore end up having two problems simultaneously: First, your channels will be imbalanced with large local balances outweighing small remote balances. Secondly, you will have too much money in the wallet. Fortunately, you can also solve both of these problems simultaneously.
Let's look at some of the solutions you can use to reduce the funds exposed in a hot wallet.
@ -609,7 +601,7 @@ If your Lightning wallet balance becomes too large for your risk tolerance, you
Sweeping funds on-chain is accomplished by moving the funds from the Lightning wallet to a Bitcoin wallet. You do that by closing channels. When you close a channel, all funds from your local balance are "swept" to a Bitcoin address. The Bitcoin address for on-chain funds is usually generated by your Lightning wallet so it is still a hot-wallet. You may need to do an additional on-chain transaction to move the funds to a more secure address, such as one generated on your hardware wallet.
Closing channels will incur an on-chain fee and will reduce your Lightning node's capacity and connectivity. However, if you run a popular e-commerce node, you will not lack incoming capacity and can strategically close channels with large local balances, essentially "bundling" your funds for movement on-chain. You may need to use some channel re-balancing techniques (see <<channel_rebalancing>>) before you close channels to maximize the benefits of this strategy.
Closing channels will incur an on-chain fee and will reduce your Lightning node's capacity and connectivity. However, if you run a popular ecommerce node, you will not lack incoming capacity and can strategically close channels with large local balances, essentially "bundling" your funds for movement on-chain. You may need to use some channel rebalancing techniques (see <<channel_rebalancing>>) before you close channels to maximize the benefits of this strategy.
===== Off-chain sweep
@ -631,7 +623,7 @@ A node operator can initiate a submarine swap and send all available channel bal
In the future, this could be a paid service offered by nodes on the Lightning Network who advertise exchange rates or charge a flat fee for the conversion.
The advantage of a submarine swap for sweeping funds is that no channel needs to be closed. That means that we preserve our channels, only re-balancing our channels through this operation. As we send a Lightning payment out, we shift some balance from local to remote on one or more of our channels. Not only does that reduce the balance exposed in our node's hot wallet, it also increases the balance available for future incoming payments.
The advantage of a submarine swap for sweeping funds is that no channel needs to be closed. That means that we preserve our channels, only rebalancing our channels through this operation. As we send a Lightning payment out, we shift some balance from local to remote on one or more of our channels. Not only does that reduce the balance exposed in our node's hot wallet, it also increases the balance available for future incoming payments.
You could do this by trusting an intermediary to act as a gateway, but this risks your coins being stolen. But in the case of a submarine swap, the operation does not require trust. Submarine swaps are non-custodial _atomic_ operations. That means that the counterparty in your submarine swap cannot steal your funds because the on-chain payment depends on the completion of the off-chain payment and vice-versa.
@ -686,7 +678,7 @@ If you have the time and skills you should test some basic fault scenarios on th
- Automatic node restart: What happens when your node or one of your nodes crashes? Simulate this fault by killing the corresponding node processes. If a node crashes, it should automatically restart itself. Test it to make sure the node or nodes really restart automatically on failure without human intervention. If this is not the case, most likely your node is not set up correctly as an operating system service.
- Automatic network reconnection: What happens if your network goes down? What happens when your ISP goes temporarily down? What happens when your ISP assigns a new IP address to your router or your computer? When the network comes back, do the nodes you are running automatically reconnect to the network? Simulate this fault by unplugging and later re-plugging the Ethernet cable from the device hosting your nodes. The nodes should automatically reconnect and continue operation without human intervention.
- Automatic network reconnection: What happens if your network goes down? What happens when your ISP goes temporarily down? What happens when your ISP assigns a new IP address to your router or your computer? When the network comes back, do the nodes you are running automatically reconnect to the network? Simulate this fault by unplugging and later replugging the Ethernet cable from the device hosting your nodes. The nodes should automatically reconnect and continue operation without human intervention.
- Configure your log files: All of the above failures should leave textual entries behind in the corresponding log files. Turn up the verbosity of logging if needed. Find these error entries in the log files and use them for monitoring.
@ -751,7 +743,7 @@ OPTIONS:
--help, -h show help
----
C-lightning has the API hooks necessary for a watchtower client plugin, though no such plugin has been implemented yet.
C-lightning has the API hooks necessary for a watchtower client plug-in, though no such plug-in has been implemented yet.
Finally, a popular standalone watchtower server is _The Eye of Satoshi_ (TEOS). It can be found here:
@ -765,7 +757,7 @@ As a Lightning node operator, one of the recurring tasks you will need to perfor
As soon as you get your Lightning node up and running, you can fund its Bitcoin wallet and then start opening channels with those funds.
You must choose channel partners carefully as your node's ability to send payments depends on who your channel partners are and how well connected they are to the rest of the Lightning Network. You also want to have more than one channel to avoid being susceptible to a single point of failure. Since Lightning now supports multi-path payments, you can split your initial funds into several channels and route bigger payments by combining their capacity. At the same time, avoid making your channels too small. Since you need to pay Bitcoin transaction fees to open and close a channel, the channel balance should not be so small that the on-chain fees consume a significant portion. It's all about balance!
You must choose channel partners carefully as your node's ability to send payments depends on who your channel partners are and how well connected they are to the rest of the Lightning Network. You also want to have more than one channel to avoid being susceptible to a single point of failure. Since Lightning now supports multipath payments, you can split your initial funds into several channels and route bigger payments by combining their capacity. At the same time, avoid making your channels too small. Since you need to pay Bitcoin transaction fees to open and close a channel, the channel balance should not be so small that the on-chain fees consume a significant portion. It's all about balance!
To summarize:
@ -786,7 +778,7 @@ Autopilots currently exist in 3 forms.
- +lnd+ incorporates an autopilot that is fully integrated with +lnd+ and runs constantly in the background while turned on.
- +lib_autopilot.py+ can offer autopilot computations for any node implementation based on the gossip and channel data.
- A +c-lighting+ plugin based on +lib_autopilot.py+ exists that provides an easy to use interface for +c-lightning+ users.
- A +c-lightning+ plug-in based on +lib_autopilot.py+ exists that provides an easy to use interface for +c-lightning+ users.
Be aware that the +lnd+ autopilot will start running in the background as soon as it is turned on via the config file. As a result it will start opening channels immediately if you have onchain outputs in your +lnd+ wallet.
If you want to have full control over the bitcoin transactions that you make and the channels that you open, make sure to turn the autopilot off _before_ you load your +lnd+ wallet with bitcoin funds.
@ -820,15 +812,15 @@ As is common, the amounts in the configuration file are enumerated in satoshi.
Currently channels below 1 mBTC are not very useful and we do not recommend you open channels that are too small and below this amount.
With the wider adoption of multipath payments, smaller channels are less of a burden. But for the time being, this is our recommendation.
The +c-lightning+ plugin which was originally written by René Pickhardt works very differently in comparison to the +lnd+ autopilot.
The +c-lightning+ plug-in which was originally written by René Pickhardt works very differently in comparison to the +lnd+ autopilot.
First, it differs in the algorithms used to make the recommendations. We will not cover this here. Secondly, it differs in its user interface.
You will need to download the _autopilot plugin_ from the +c-lightning+ plugin repository at https://github.com/lightningd/plugins/tree/master/autopilot and activate it.
You will need to download the _autopilot plug-in_ from the +c-lightning+ plug-in repository at https://github.com/lightningd/plugins/tree/master/autopilot and activate it.
[NOTE]
====
In order to activate a plugin in +c-lightning+, place it into the `~/.lightning/plugins` directory, ensure that it's executable (e.g. `chmod +x ~/.lightning/plugins/autopilot.py`), then restart +lightningd+.
In order to activate a plug-in in +c-lightning+, place it into the `~/.lightning/plugins` directory, ensure that it's executable (e.g. `chmod +x ~/.lightning/plugins/autopilot.py`), then restart +lightningd+.
Alternatively, if you don't want a plugin to automatically activate when you start +lightningd+ you can place it in a different directory and manually activate it with the +plugin+ argument to +lightningd+:
Alternatively, if you don't want a plug-in to automatically activate when you start +lightningd+ you can place it in a different directory and manually activate it with the +plugin+ argument to +lightningd+:
----
lightningd --plugin=~/lightning-plugins/autopilot.py
@ -861,7 +853,7 @@ These differences reflect personal preferences and could actually be the decidin
Keep in mind that current autopilots will primarily use public information from the gossip protocol about the current topology of the Lightning Network.
It is obvious that your personal requirements for channels can only be reflected to a certain degree.
More advanced autopilots would use historical and usage information that your node has gathered when running in the past, including information about routing successes, who you have paid in the past, and who paid you.
In the future, such improved autopilots might also use this collected data to make recommendations on closing channels and re-allocating funds.
In the future, such improved autopilots might also use this collected data to make recommendations on closing channels and reallocating funds.
Overall, at the time of writing of this book, be cautious not to depend or rely too heavily on autopilots.
@ -909,21 +901,21 @@ Some examples:
* Your channel partner is online and your nodes are negotiating a mutual close, but they become stuck and cannot reach a resolution.
[[channel_rebalancing]]
==== Re-Balancing Channels
==== Rebalancing Channels
In the course of transacting and routing payments on Lightning, the combination of inbound and outbound capacities can become unbalanced.
For example, if one of your channel partners is frequently routing payments through your node, you will exhaust the inbound capacity on that channel, while also exhausting the outbound capacity on the outgoing channels. Once that happens, you can no longer route payments through that route.
There are many ways to re-balance channels, each with different advantages and disadvantages. One way is to use a _submarine swap_ (e.g. +Loop-Out+) as described previously in this chapter. Another way to re-balance is to simply wait for routed payments that flow in the opposite direction. If your node is well connected, when a specific route becomes exhausted in one direction, the same route becomes available in the opposite direction. Other nodes may "discover" that route in the opposite direction and start using it as part of their payment path, thereby re-balancing the funds again.
There are many ways to rebalance channels, each with different advantages and disadvantages. One way is to use a _submarine swap_ (e.g. +Loop-Out+) as described previously in this chapter. Another way to rebalance is to simply wait for routed payments that flow in the opposite direction. If your node is well connected, when a specific route becomes exhausted in one direction, the same route becomes available in the opposite direction. Other nodes may "discover" that route in the opposite direction and start using it as part of their payment path, thereby rebalancing the funds again.
A third way to re-balance channels is to purposefully create a _circular route_ that sends a payment from your node back to your node, via the Lightning Network. By sending a payment out on a channel with large local capacity and arranging the path so that it returns to your node on a channel with large remote capacity, both of those channels will become more balanced. An example of a circular route re-balancing strategy can be seen in <<circular_rebalancing>>.
A third way to rebalance channels is to purposefully create a _circular route_ that sends a payment from your node back to your node, via the Lightning Network. By sending a payment out on a channel with large local capacity and arranging the path so that it returns to your node on a channel with large remote capacity, both of those channels will become more balanced. An example of a circular route rebalancing strategy can be seen in <<circular_rebalancing>>.
[[circular_rebalancing]]
.Circular route re-balancing
.Circular route rebalancing
image::images/mtln_0504.png[]
Circular re-balancing is supported by most Lightning node implementations and can be done on the command line or via one of the web management interfaces such as _Ride the Lightning (RTL)_ (see <<rtl>>).
Circular rebalancing is supported by most Lightning node implementations and can be done on the command line or via one of the web management interfaces such as _Ride the Lightning (RTL)_ (see <<rtl>>).
Channel rebalancing is a complex issue that is the subject of active research and covered in more detail in <<channel_rebalancing>>.
@ -958,7 +950,7 @@ Broadly speaking, you can take one of two approaches to routing fees. You can ro
For most nodes, it is usually best to use the default routing fee values. This way, your node is competing on a mostly level playing field with other nodes who use the default values.
You can also use the routing fee settings to re-balance channels. If most of your channels have the default fees but you want to rebalance a particular channel, just decrease the fees on that specific channel to zero or to very low rates. Then sit back and wait for someone to route a payment over your "cheap" route and re-balance your channels for you as a side-effect.
You can also use the routing fee settings to rebalance channels. If most of your channels have the default fees but you want to rebalance a particular channel, just decrease the fees on that specific channel to zero or to very low rates. Then sit back and wait for someone to route a payment over your "cheap" route and rebalance your channels for you as a side-effect.
=== Node Management