Merge branch 'docker-refactor' into develop

appendix_docker_basics.asciidoc

@@ -0,0 +1,113 @@
+[appendix]
+[[appendix_docker]]
+== Docker Basic Installation and Use
+
+This book contains a number of examples that run inside docker containers, for standardization across different operating systems.
+
+This section will help you install Docker and familiarize yourself with some of the most commonly used Docker commands, so that you can run the book's example containers.
+
+
+=== Installing Docker
+
+Before we begin, you should install the Docker container system on your computer. Docker is an open system that is distributed for free as a _Community Edition_ for many different operating systems including Windows, Mac OS and Linux. The Windows and Mac versions are called _Docker Desktop_ and consist of a GUI desktop application and command-line tools. The Linux version is called _Docker Engine_ and is comprised of a server daemon and command-line tools. We will be using the command-line tools, which are identical across all platforms.
+
+Go ahead and install Docker for your operating system by following the instructions to _"Get Docker"_ from the Docker website found here:
+
+https://docs.docker.com/get-docker/
+
+Select your operating system from the list and follow the installation instructions.
+
+[TIP]
+====
+If you install on Linux, follow the post-installation instructions to ensure you can run Docker as a regular user instead of user _root_. Otherwise, you will need to prefix all +docker+ commands with +sudo+, running them as root like: +sudo docker+.
+====
+
+Once you have Docker installed, you can test your installation by running the demo container +hello-world+ like this:
+
+[docker-hello-world]
+----
+$ docker run hello-world
+
+Hello from Docker!
+This message shows that your installation appears to be working correctly.
+
+[...]
+----
+
+=== Basic Docker commands
+
+In this chapter, we use Docker quite extensively. We will be using the following Docker commands and arguments:
+
+*Building a container*
+
+----
+docker build [-t tag] [directory]
+----
+
+...where +tag+ is how we identify the container we are building, and +directory+ is where the container's "context" (folders and files) and definition file (+Dockerfile+) are found.
+
+*Running a container*
+
+----
+docker run -it [--network netname] [--name cname] tag
+----
+
+...where +netname+ is the name of a Docker network, +cname+ is the name we choose for this container instance and +tag+ is the name tag we gave the container when we built it.
+
+*Executing a command in a container*
+
+----
+docker exec cname command
+----
+
+...where +cname+ is the name we gave the container in the +run+ command, and +command+ is an executable or script that we want to run inside the container.
+
+*Stopping and starting a container*
+
+In most cases, if we are running a container in an _interactive_ as well as _terminal_ mode, i.e. with the +i+ and +t+ flags (combined as +-it+) set, the container can be stopped by simply pressing +CTRL-C+ or by exiting the shell with +exit+ or +CTRL-D+. If a container does not terminate, you can stop it from another terminal like this:
+
+----
+docker stop cname
+----
+
+To resume an already existing container use the `start` command, like so:
+
+----
+docker start cname
+----
+
+*Deleting a container by name*
+
+If you name a container instead of letting Docker name it randomly, you cannot reuse that name until the container is deleted. Docker will return an error like this:
+[source,bash]
+----
+docker: Error response from daemon: Conflict. The container name "/bitcoind" is already in use...
+----
+
+To fix this, delete the existing instance of the container:
+
+----
+docker rm cname
+----
+
+...where +cname+ is the name assigned to the container (+bitcoind+ in the example error message)
+
+*List running containers*
+
+----
+docker ps
+----
+
+...shows the current running containers and their names
+
+*List docker images*
+
+----
+docker image ls
+----
+
+...shows the docker images that have been built or downloaded on your computer
+
+=== Conclusion
+
+These basic Docker commands will be enough to get you started and will allow you to run all the examples in this book.

code/docker/Makefile

@@ -1,16 +1,126 @@
+#!make
+#
+# Makefile to help with building, pulling and pushing containers
+#
+# NOTE: You cannot push to the container registry unless you are authorized
+# in the lnbook organization (i.e. one of the authors or maintainers)
+#
+# Targets:
+#
+# make build # Build all containers
+# make pull # Pull all containers from the registry
+# make build-bitcoind # Build a specific container
+# make clean # remove all images and containers
+# make push # push updated images to Docker Hub (authors/maintainers only)
 
+
+# Latest tested versions of Bitcoin and Lightning clients
+
+# OS base image
+OS=ubuntu
+OS_VER=focal
+
+# bitcoind version
+BITCOIND_VER=0.21.0
+
+# LND version
+GO_VER=1.13
+LND_VER=v0.13.1-beta
+
+# c-lightning version
+CL_VER=0.10.1
+
+# Eclair version
+ECLAIR_VER=0.4.2
+ECLAIR_COMMIT=52444b0
+
+
+
+
+# Docker registry for lnbook
 REGISTRY=docker.com
-NAME=lnbook
+ORG=lnbook
+
+# List of containers
 CONTAINERS=bitcoind lnd eclair c-lightning
 
-all: build-all push-all
+.DEFAULT: pull
 
-build-all:
+
+
+
+
+build-bitcoind:
+	docker build \
+	--build-arg OS=${OS} \
+	--build-arg OS_VER=${OS_VER} \
+	--build-arg BITCOIND_VER=${BITCOIND_VER} \
+	-t ${ORG}/bitcoind:${BITCOIND_VER} \
+	bitcoind -f bitcoind/Dockerfile
+	docker image tag ${ORG}/bitcoind:${BITCOIND_VER} ${ORG}/bitcoind:latest
+
+
+build-cl: build-bitcoind
+	docker build \
+	--build-arg OS=${OS} \
+	--build-arg OS_VER=${OS_VER} \
+	--build-arg CL_VER=${CL_VER} \
+	-t ${ORG}/c-lightning:${CL_VER} \
+	c-lightning -f c-lightning/Dockerfile
+	docker image tag ${ORG}/c-lightning:${CL_VER} ${ORG}/c-lightning:latest
+
+
+build-lnd:
+	docker build \
+	--build-arg OS=${OS} \
+	--build-arg OS_VER=${OS_VER} \
+	--build-arg LND_VER=${LND_VER} \
+	--build-arg GO_VER=${GO_VER} \
+	-t ${ORG}/lnd:${LND_VER}_golang_${GO_VER} \
+	lnd -f lnd/Dockerfile
+	docker image tag ${ORG}/lnd:${LND_VER}_golang_${GO_VER} ${ORG}/lnd:latest
+
+
+build-eclair:
+	docker build \
+	--build-arg OS=${OS} \
+	--build-arg OS_VER=${OS_VER} \
+	--build-arg ECLAIR_VER=${ECLAIR_VER} \
+	--build-arg ECLAIR_COMMIT=${ECLAIR_COMMIT} \
+	-t ${ORG}/eclair:${ECLAIR_VER}-${ECLAIR_COMMIT} \
+	eclair -f eclair/Dockerfile
+	docker image tag ${ORG}/eclair:${ECLAIR_VER}-${ECLAIR_COMMIT} ${ORG}/eclair:latest
+
+
+push-bitcoind: build-bitcoind
+	docker push ${ORG}/bitcoind:${BITCOIND_VER}
+	docker push ${ORG}/bitcoind:latest
+
+push-lnd: build-lnd
+	docker push ${ORG}/lnd:${LND_VER}_golang_${GO_VER}
+	docker push ${ORG}/lnd:latest
+
+push-cl: build-cl
+	docker push ${ORG}/c-lightning:${CL_VER}
+	docker push ${ORG}/c-lightning:latest
+
+push-eclair: build-eclair
+	docker push ${ORG}/eclair:${ECLAIR_VER}-${ECLAIR_COMMIT}
+	docker push ${ORG}/eclair:latest
+
+build: build-bitcoind build-lnd build-cl build-eclair
+
+push: push-bitcoind push-lnd push-cl push-eclair
+
+pull:
 	for container in ${CONTAINERS}; do \
-		docker build -t ${NAME}/$$container $$container -f $$container/Dockerfile; \
+		docker pull ${ORG}/$$container:latest ;\
 	done
 
-push-all:
+clean:
-	for container in ${CONTAINERS}; do \
+	# Try 'make clean-confirm' if you are sure you want to do this.
-		docker push ${NAME}/$$container; \
+	# CAUTION: ALL docker containers and images on your computer will be removed.
-	done
+
+clean-confirm:
+	docker rm -f `docker ps -qa`
+	docker rmi -f `docker image ls -qa`

code/docker/bitcoind/Dockerfile

@@ -1,34 +1,47 @@
-FROM ubuntu:20.04 AS bitcoind-base
+ARG OS=ubuntu
+ARG OS_VER=focal
+FROM ${OS}:${OS_VER} as os-base
 
-RUN apt update && apt install -yqq \
+# Install dependencies
-	curl gosu jq bash-completion
+RUN DEBIAN_FRONTEND=noninteractive \
+    apt-get update -qq && apt-get install -yqq \
+		curl unzip jq bash-completion
 
-ENV BITCOIND_VERSION 0.21.0
+FROM os-base as bitcoind-install
-# Install binaries for Bitcoin Core
-ADD https://bitcoincore.org/bin/bitcoin-core-${BITCOIND_VERSION}/bitcoin-${BITCOIND_VERSION}-x86_64-linux-gnu.tar.gz /usr/local
-RUN cd /usr/local/ \
-	&& tar -zxf bitcoin-${BITCOIND_VERSION}-x86_64-linux-gnu.tar.gz \
-	&& cd bitcoin-${BITCOIND_VERSION} \
-	&& install bin/* /usr/local/bin \
-	&& install include/* /usr/local/include \
-	&& install -v lib/* /usr/local/lib
 
+ARG BITCOIND_VER=0.21.0
+# Install Bitcoin Core binaries and libraries
+RUN cd /tmp && \
+	curl -# -sLO https://bitcoincore.org/bin/bitcoin-core-${BITCOIND_VER}/bitcoin-${BITCOIND_VER}-x86_64-linux-gnu.tar.gz && \
+	tar -zxf bitcoin-${BITCOIND_VER}-x86_64-linux-gnu.tar.gz && \
+	cd bitcoin-${BITCOIND_VER} && \
+	install -vD bin/* /usr/bin && \
+	install -vD lib/* /usr/lib && \
+	cd /tmp && \
+	rm bitcoin-${BITCOIND_VER}-x86_64-linux-gnu.tar.gz && \
+	rm -rf bitcoin-${BITCOIND_VER}
+
+# Install runtime scripts, bash-completion and configuration files
+
+# bash completion for bitcoind and bitcoin-cli
 ENV GH_URL https://raw.githubusercontent.com/bitcoin/bitcoin/master/
 ENV BC /usr/share/bash-completion/completions/
 ADD $GH_URL/contrib/bitcoin-cli.bash-completion $BC/bitcoin-cli
 ADD $GH_URL/contrib/bitcoind.bash-completion $BC/bitcoind
 ADD $GH_URL/contrib/bitcoin-tx.bash-completion $BC/bitcoin-tx
 
-FROM bitcoind-base AS bitcoind
+# Copy bitcoind configuration directory
-
+COPY bitcoind /bitcoind
-ADD bitcoind /bitcoind
 RUN ln -s /bitcoind /root/.
 
-ADD bashrc /root/.bashrc
+# Copy support scripts
-ADD bitcoind-entrypoint.sh /usr/local/bin
+COPY bashrc /root/.bashrc
+COPY bitcoind-entrypoint.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/bitcoind-entrypoint.sh
-ADD mine.sh /usr/local/bin
+COPY mine.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/mine.sh
+COPY cli /usr/local/bin
+RUN chmod +x /usr/local/bin/cli
 
 # bitcoind P2P
 EXPOSE 18444/tcp

code/docker/bitcoind/bitcoind-entrypoint.sh

@@ -1,25 +1,36 @@
 #!/bin/bash
 set -Eeuo pipefail
 
-echo Starting bitcoind...
+
+# Start bitcoind
+echo "Starting bitcoind..."
 bitcoind -datadir=/bitcoind -daemon
+
+# Wait for bitcoind startup
+echo -n "Waiting for bitcoind to start"
 until bitcoin-cli -datadir=/bitcoind -rpcwait getblockchaininfo  > /dev/null 2>&1
 do
+	echo -n "."
 	sleep 1
 done
-echo bitcoind started
+echo
+echo "bitcoind started"
+
+
+# Load private key into wallet
 export address=`cat /bitcoind/keys/demo_address.txt`
 export privkey=`cat /bitcoind/keys/demo_privkey.txt`
+
+# If restarting the wallet already exists, so don't fail if it does,
+# just load the existing wallet:
+bitcoin-cli -datadir=/bitcoind createwallet regtest > /dev/null || bitcoin-cli -datadir=/bitcoind loadwallet regtest > /dev/null
+bitcoin-cli -datadir=/bitcoind importprivkey $privkey > /dev/null || true
+
 echo "================================================"
-echo "Importing demo private key"
+echo "Imported demo private key"
 echo "Bitcoin address: " ${address}
 echo "Private key: " ${privkey}
 echo "================================================"
-# If restarting the wallet already exists, so don't fail if it does,
-# just load the existing wallet:
-bitcoin-cli -datadir=/bitcoind createwallet regtest || bitcoin-cli -datadir=/bitcoind loadwallet regtest
-bitcoin-cli -datadir=/bitcoind importprivkey $privkey || true
 
 # Executing CMD
-echo "$@"
 exec "$@"

code/docker/bitcoind/cli

@@ -0,0 +1,5 @@
+#!/bin/bash
+#
+# Helper script used as an alias for bitcoin-cli with the necessary arguments
+#
+/usr/bin/bitcoin-cli -datadir=/bitcoind -regtest $@

code/docker/bitcoind/mine.sh

@@ -2,18 +2,17 @@
 set -Eeuo pipefail
 
 export address=`cat /bitcoind/keys/demo_address.txt`
-export privkey=`cat /bitcoind/keys/demo_privkey.txt`
 echo "================================================"
-echo "Bitcoin address: " ${address}
-echo "Private key: " ${privkey}
 echo "Balance:" `bitcoin-cli -datadir=/bitcoind getbalance`
 echo "================================================"
 echo "Mining 101 blocks to unlock some bitcoin"
 bitcoin-cli -datadir=/bitcoind generatetoaddress 101 $address
-echo "Mining 1 block every 10 seconds"
+echo "Mining 6 blocks every 10 seconds"
-while sleep 10; do \
+while echo "Balance:" `bitcoin-cli -datadir=/bitcoind getbalance`;
-	bitcoin-cli -datadir=/bitcoind generatetoaddress 1 $address; \
+do
-	echo "Balance:" `bitcoin-cli -datadir=/bitcoind getbalance`; \
+	bitcoin-cli -datadir=/bitcoind generatetoaddress 6 $address; \
+	sleep 10; \
+
 done
 
 # If loop is interrupted, stop bitcoind

code/docker/c-lightning/Dockerfile

@@ -1,14 +1,26 @@
-FROM lnbook/bitcoind AS c-lightning-base
+ARG OS=ubuntu
+ARG OS_VER=focal
+FROM ${OS}:${OS_VER} as os-base
+
+# Install dependencies
+RUN DEBIAN_FRONTEND=noninteractive \
+    apt-get update -qq && apt-get install -yqq \
+		curl unzip jq bash-completion
+
+FROM os-base as cl-install
+COPY --from=lnbook/bitcoind:latest /usr/bin/bitcoin-cli /usr/bin
+
+# Set CL_VER ENV from ARG
+ARG CL_VER=0.10.1
+ENV CL_VER=${CL_VER}
 
-# Install software-properties-common to add apt repositories
 RUN apt-get update -qq && apt-get install -yqq \
-	wget gpg xz-utils libpq5 libsodium23
+		gpg xz-utils libpq5 libsodium23 && \
-
+    apt-get clean && \
-# c-lightning
+    rm -rf /var/lib/apt/lists/*
-ENV C_LIGHTNING_VER 0.10.1
 
 RUN cd /tmp && \
-    wget -q https://github.com/ElementsProject/lightning/releases/download/v${C_LIGHTNING_VER}/clightning-v${C_LIGHTNING_VER}-Ubuntu-20.04.tar.xz
+    curl -# -sLO https://github.com/ElementsProject/lightning/releases/download/v${CL_VER}/clightning-v${CL_VER}-Ubuntu-20.04.tar.xz
 
 # Verify developer signatures. The `gpg --verify` command will print a
 # couple of warnings about the key not being trusted. That's ok. The
@@ -23,25 +35,26 @@ RUN cd /tmp && \
     cat SHA256SUMS && \
     sha256sum --ignore-missing -c SHA256SUMS
 
-RUN tar -xvf /tmp/clightning-v${C_LIGHTNING_VER}-Ubuntu-20.04.tar.xz -C /
+RUN tar -xvf /tmp/clightning-v${CL_VER}-Ubuntu-20.04.tar.xz -C /
 
 ADD https://raw.githubusercontent.com/ElementsProject/lightning/master/contrib/lightning-cli.bash-completion /usr/share/bash-completion/completions/lightning-cli
 
-FROM c-lightning-base AS c-lightning-run
+COPY lightningd /lightningd
-
-ADD lightningd /lightningd
 WORKDIR /lightningd
 RUN ln -s /lightningd /root/.lightning
 
-ADD bashrc /root/.bashrc
+COPY bashrc /root/.bashrc
-ADD c-lightning-entrypoint.sh /usr/local/bin
+COPY c-lightning-entrypoint.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/c-lightning-entrypoint.sh
-ADD fund-c-lightning.sh /usr/local/bin
+COPY fund-c-lightning.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/fund-c-lightning.sh
-ADD logtail.sh /usr/local/bin
+COPY logtail.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/logtail.sh
-ADD wait-for-bitcoind.sh /usr/local/bin
+COPY wait-for-bitcoind.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/wait-for-bitcoind.sh
+COPY cli /usr/local/bin
+RUN chmod +x /usr/local/bin/cli
+
 
 EXPOSE 9735 9835
 ENTRYPOINT ["/usr/local/bin/c-lightning-entrypoint.sh"]

code/docker/c-lightning/c-lightning-entrypoint.sh

@@ -15,5 +15,4 @@ sleep 2
 echo "Funding c-lightning wallet"
 source /usr/local/bin/fund-c-lightning.sh
 
-echo "$@"
 exec "$@"

code/docker/c-lightning/cli

@@ -0,0 +1,5 @@
+#!/bin/bash
+#
+# Helper script used as an alias for lightning-cli with the necessary arguments
+#
+/usr/bin/lightning-cli --lightning-dir=/lightningd $@

code/docker/c-lightning/fund-c-lightning.sh

@@ -5,9 +5,13 @@ set -Eeuo pipefail
 address=$(lightning-cli --lightning-dir=/lightningd --network regtest newaddr | jq '.bech32' -r)
 
 # Ask Bitcoin Core to send 10 BTC to the address, using JSON-RPC call
-bitcoin-cli \
+until bitcoin-cli \
     --rpcuser=regtest \
     --rpcpassword=regtest \
-    --rpcconnect=bitcoind \
+    --rpcconnect=bitcoind:18443 \
     --regtest \
     sendtoaddress ${address} 10 "funding c-lightning"
+do
+	sleep 1;
+	echo Retrying funding...
+done

code/docker/c-lightning/logtail.sh

@@ -2,4 +2,5 @@
 set -Eeuo pipefail
 
 # Show LND log from beginning and follow
+touch /lightningd/lightningd.log
 tail -n +1 -f /lightningd/lightningd.log || true

code/docker/c-lightning/wait-for-bitcoind.sh

@@ -2,14 +2,14 @@
 set -Eeuo pipefail
 
 echo Waiting for bitcoind to start...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getblockchaininfo  > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "cl-node", "method": "getblockchaininfo", "params": []}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result.blocks > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1
 done
 
 echo Waiting for bitcoind to mine blocks...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getbalance  | jq -e ". > 0" > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "cl-node", "method": "getbalance", "params": ["*", 6]}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1

code/docker/check-versions.sh

@@ -1,8 +1,7 @@
 #!/bin/bash
-
 # a small script to help sanity check the versions of the different node implementations
 dockerfiles=$(find . -name 'Dockerfile')
 # print location of dockerfiles
 echo $dockerfiles
 # print variables
-awk '/ENV/ && /VER|COMMIT/' $dockerfiles
+awk '/ARG/ && /VER|COMMIT/' $dockerfiles

code/docker/eclair/Dockerfile

@@ -1,20 +1,29 @@
-FROM ubuntu:20.04 AS eclair-base
+ARG OS=ubuntu
+ARG OS_VER=focal
+FROM ${OS}:${OS_VER} as os-base
 
-RUN apt update && apt install -yqq \
+# Install dependencies
-	curl gosu jq bash-completion
+RUN DEBIAN_FRONTEND=noninteractive \
+    apt-get update -qq && apt-get install -yqq \
+		curl unzip jq bash-completion
 
-RUN apt update && apt install -yqq \
+# Install default Java Runtime Environment
-	openjdk-11-jdk unzip
+RUN DEBIAN_FRONTEND=noninteractive \
-
+    apt-get update -qq && apt-get install -yqq \
-COPY --from=lnbook/bitcoind /usr/local/ /usr/local/
+	  default-jre-headless && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
 
 # Install eclair
-ENV ECLAIR_VER 0.4.2
-ENV ECLAIR_COMMIT 52444b0
 WORKDIR /usr/src
-RUN curl -sLO https://github.com/ACINQ/eclair/releases/download/v${ECLAIR_VER}/eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}-bin.zip \
+
-	&& unzip eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}-bin.zip \
+ARG ECLAIR_VER=0.4.2
-	&& install eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}/bin/eclair-cli /usr/local/bin
+ARG ECLAIR_COMMIT=52444b0
+RUN cd /usr/src && \
+	curl -# -sLO https://github.com/ACINQ/eclair/releases/download/v${ECLAIR_VER}/eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}-bin.zip && \
+	unzip eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}-bin.zip && \
+	install eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}/bin/eclair-cli /usr/local/bin && \
+	rm eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}-bin.zip
 
 ADD https://raw.githubusercontent.com/ACINQ/eclair/master/contrib/eclair-cli.bash-completion /usr/share/bash-completion/completions/eclair-cli
 
@@ -29,6 +38,11 @@ ADD logtail.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/logtail.sh
 ADD wait-for-bitcoind.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/wait-for-bitcoind.sh
+COPY cli /usr/local/bin
+RUN chmod +x /usr/local/bin/cli
+
+ENV ECLAIR_VER=$ECLAIR_VER
+ENV ECLAIR_COMMIT=$ECLAIR_COMMIT
 
 EXPOSE 9735
 ENTRYPOINT ["/usr/local/bin/eclair-entrypoint.sh"]

code/docker/eclair/cli

@@ -0,0 +1,5 @@
+#!/bin/bash
+#
+# Helper script used as an alias for eclair-cli with the necessary arguments
+#
+/usr/local/bin/eclair-cli -s -j -p eclair $@

code/docker/eclair/eclair-entrypoint.sh

@@ -18,5 +18,4 @@ echo Eclair node started
 sleep 2
 
 # Executing CMD
-echo "$@"
 exec "$@"

code/docker/eclair/eclair/eclair.conf

@@ -23,11 +23,18 @@ eclair {
     zmqblock = "tcp://bitcoind:12005"
     zmqtx = "tcp://bitcoind:12006"
   }
-  
+
   on-chain-fees {
     feerate-tolerance {
       ratio-low = 0.000001
       ratio-high = 1000000
     }
   }
+
+  node-alias = "eclair"
+
+  router {
+    channel-exclude-duration = 1 seconds
+    broadcast-interval = 1 seconds
+  }
 }

code/docker/eclair/wait-for-bitcoind.sh

@@ -2,14 +2,14 @@
 set -Eeuo pipefail
 
 echo Waiting for bitcoind to start...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getblockchaininfo  > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "eclair-node", "method": "getblockchaininfo", "params": []}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result.blocks > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1
 done
 
 echo Waiting for bitcoind to mine blocks...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getbalance  | jq -e ". > 0" > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "eclair-node", "method": "getbalance", "params": ["*", 6]}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1

code/docker/lnd/Dockerfile

@@ -1,40 +1,59 @@
-FROM golang:1.13 as lnd-base
+ARG OS=ubuntu
+ARG OS_VER=focal
+ARG GO_VER=1.13
+# Define base images with ARG versions
+FROM ${OS}:${OS_VER} as os
+FROM golang:${GO_VER} as go
 
-ENV GOPATH /go
+# OS image with command-line utilities
-WORKDIR $GOPATH/src
+FROM os AS os-base
 
-# LND
+# Install dependencies
-ENV LND_VER v0.11.1-beta
+RUN DEBIAN_FRONTEND=noninteractive \
-RUN go get -d github.com/lightningnetwork/lnd
+    apt-get update -qq && apt-get install -yqq \
-WORKDIR $GOPATH/src/github.com/lightningnetwork/lnd
+		curl unzip jq bash-completion
-RUN git checkout tags/${LND_VER}
-RUN make && make install
 
-FROM ubuntu:20.04 AS lnd-run
+# Go image for building LND
+FROM go as lnd-build
 
-RUN apt update && apt install -yqq \
+ENV GO_VER=${GO_VER}
-	curl gosu jq bash-completion
+ENV GOPATH=/go
 
-COPY --from=lnd-base /go /go
+# Build LND
-COPY --from=lnbook/bitcoind /usr/local/ /usr/local/
+ARG LND_VER=v0.13.1-beta
+ENV LND_VER=${LND_VER}
+RUN	mkdir -p ${GOPATH}/src && \
+	cd ${GOPATH}/src && \
+	go get -v -d github.com/lightningnetwork/lnd && \
+	cd ${GOPATH}/src/github.com/lightningnetwork/lnd && \
+	git checkout tags/${LND_VER} && \
+	make clean && make && make install
 
-RUN cp /go/src/github.com/lightningnetwork/lnd/contrib/lncli.bash-completion \
+# Runtime image for running LND
+FROM os-base as lnd-run
+
+# Copy only the executables
+COPY --from=lnd-build /go/bin /go/bin
+
+ADD https://raw.githubusercontent.com/lightningnetwork/lnd/master/contrib/lncli.bash-completion \
 	   /usr/share/bash-completion/completions/lncli
 
 ENV GOPATH /go
 ENV PATH $PATH:$GOPATH/bin
 
-ADD lnd /lnd
+COPY lnd /lnd
 RUN ln -s /lnd /root/.lnd
-ADD fund-lnd.sh /usr/local/bin
+COPY fund-lnd.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/fund-lnd.sh
-ADD bashrc /root/.bashrc
+COPY bashrc /root/.bashrc
-ADD lnd-entrypoint.sh /usr/local/bin
+COPY lnd-entrypoint.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/lnd-entrypoint.sh
-ADD logtail.sh /usr/local/bin
+COPY logtail.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/logtail.sh
-ADD wait-for-bitcoind.sh /usr/local/bin
+COPY wait-for-bitcoind.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/wait-for-bitcoind.sh
+COPY cli /usr/local/bin
+RUN chmod +x /usr/local/bin/cli
 
 # LND RPC
 EXPOSE 10009/tcp

code/docker/lnd/cli

@@ -0,0 +1,5 @@
+#!/bin/bash
+#
+# Helper script used as an alias for lncli with the necessary arguments
+#
+/go/bin/lncli --lnddir=/lnd -n regtest $@

code/docker/lnd/lnd-entrypoint.sh

@@ -14,5 +14,4 @@ echo "Startup complete"
 echo "Funding lnd wallet"
 source /usr/local/bin/fund-lnd.sh
 
-echo "$@"
 exec "$@"

code/docker/lnd/wait-for-bitcoind.sh

@@ -2,14 +2,14 @@
 set -Eeuo pipefail
 
 echo Waiting for bitcoind to start...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getblockchaininfo  > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "lnd-node", "method": "getblockchaininfo", "params": []}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result.blocks > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1
 done
 
 echo Waiting for bitcoind to mine blocks...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getbalance  | jq -e ". > 0" > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "lnd-node", "method": "getbalance", "params": ["*", 6]}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1

code/docker/run-payment-demo.sh

@@ -0,0 +1,177 @@
+#!/bin/bash
+
+#
+# Helper functions
+#
+
+
+# run-in-node: Run a command inside a docker container, using the bash shell
+function run-in-node () {
+	docker exec "$1" /bin/bash -c "${@:2}"
+}
+
+# wait-for-cmd: Run a command repeatedly until it completes/exits successfuly
+function wait-for-cmd () {
+		until "${@}" > /dev/null 2>&1
+		do
+			echo -n "."
+			sleep 1
+		done
+		echo
+}
+
+# wait-for-node: Run a command repeatedly until it completes successfully, inside a container
+# Combining wait-for-cmd and run-in-node
+function wait-for-node () {
+	wait-for-cmd run-in-node $1 "${@:2}"
+}
+
+
+# Start the demo
+echo "Starting Payment Demo"
+
+echo "======================================================"
+echo
+echo "Waiting for nodes to startup"
+echo -n "- Waiting for bitcoind startup..."
+wait-for-node bitcoind "cli getblockchaininfo | jq -e \".blocks > 101\""
+echo -n "- Waiting for bitcoind mining..."
+wait-for-node bitcoind "cli getbalance | jq -e \". > 50\""
+echo -n "- Waiting for Alice startup..."
+wait-for-node Alice "cli getinfo"
+echo -n "- Waiting for Bob startup..."
+wait-for-node Bob "cli getinfo"
+echo -n "- Waiting for Chan startup..."
+wait-for-node Chan "cli getinfo"
+echo -n "- Waiting for Dina startup..."
+wait-for-node Dina "cli getinfo"
+echo "All nodes have started"
+
+echo "======================================================"
+echo
+echo "Getting node IDs"
+alice_address=$(run-in-node Alice "cli getinfo | jq -r .identity_pubkey")
+bob_address=$(run-in-node Bob "cli getinfo | jq -r .id")
+chan_address=$(run-in-node Chan "cli getinfo| jq -r .nodeId")
+dina_address=$(run-in-node Dina "cli getinfo | jq -r .identity_pubkey")
+
+# Show node IDs
+echo "- Alice:  ${alice_address}"
+echo "- Bob:    ${bob_address}"
+echo "- Chan:   ${chan_address}"
+echo "- Dina:	${dina_address}"
+
+echo "======================================================"
+echo
+echo "Waiting for Lightning nodes to sync the blockchain"
+echo -n "- Waiting for Alice chain sync..."
+wait-for-node Alice "cli getinfo | jq -e \".synced_to_chain == true\""
+echo -n "- Waiting for Bob chain sync..."
+wait-for-node Bob "cli getinfo | jq -e \".blockheight > 100\""
+echo -n "- Waiting for Chan chain sync..."
+wait-for-node Chan "cli getinfo | jq -e \".blockHeight > 100\""
+echo -n "- Waiting for Dina chain sync..."
+wait-for-node Dina "cli getinfo | jq -e \".synced_to_chain == true\""
+echo "All nodes synched to chain"
+
+echo "======================================================"
+echo
+echo "Setting up connections and channels"
+echo "- Alice to Bob"
+
+# Connect only if not already connected
+run-in-node Alice "cli listpeers | jq -e '.peers[] | select(.pub_key == \"${bob_address}\")' > /dev/null" \
+&& {
+	echo "- Alice already connected to Bob"
+} || {
+	echo "- Open connection from Alice node to Bob's node"
+	wait-for-node Alice "cli connect ${bob_address}@Bob"
+}
+
+# Create channel only if not already created
+run-in-node Alice "cli listchannels | jq -e '.channels[] | select(.remote_pubkey == \"${bob_address}\")' > /dev/null" \
+&& {
+	echo "- Alice->Bob channel already exists"
+} || {
+	echo "- Create payment channel Alice->Bob"
+	wait-for-node Alice "cli openchannel ${bob_address} 1000000"
+}
+echo "Bob to Chan"
+run-in-node Bob "cli listpeers | jq -e '.peers[] | select(.id == \"${chan_address}\")' > /dev/null" \
+&& {
+	echo "- Bob already connected to Chan"
+} || {
+	echo "- Open connection from Bob's node to Chan's node"
+	wait-for-node Bob "cli connect ${chan_address}@Chan"
+}
+run-in-node Bob "cli listchannels | jq -e '.channels[] | select(.destination == \"${chan_address}\")' > /dev/null" \
+&& {
+	echo "- Bob->Chan channel already exists"
+} || {
+	echo "- Create payment channel Bob->Chan"
+	wait-for-node Bob "cli fundchannel ${chan_address} 1000000"
+}
+echo "Chan to Dina"
+run-in-node Chan "cli peers | jq -e '.[] | select(.nodeId == \"${dina_address}\" and .state == \"CONNECTED\")' > /dev/null" \
+&& {
+	echo "- Chan already connected to Dina"
+} || {
+	echo "- Open connection from Chan's node to Dina's node"
+	wait-for-node Chan "cli connect --uri=${dina_address}@Dina"
+}
+run-in-node Chan "cli channels | jq -e '.[] | select(.nodeId == \"${dina_address}\" and .state == \"NORMAL\")' > /dev/null" \
+&& {
+	echo "- Chan->Dina channel already exists"
+} || {
+	echo "- Create payment channel Chan->Dina"
+	wait-for-node Chan "cli open --nodeId=${dina_address} --fundingSatoshis=1000000"
+}
+echo "All channels created"
+echo "======================================================"
+echo
+echo "Waiting for channels to be confirmed on the blockchain"
+echo -n "- Waiting for Alice channel confirmation..."
+wait-for-node Alice "cli listchannels | jq -e '.channels[] | select(.remote_pubkey == \"${bob_address}\" and .active == true)'"
+echo "- Alice->Bob connected"
+echo -n "- Waiting for Bob channel confirmation..."
+wait-for-node Bob "cli listchannels | jq -e '.channels[] | select(.destination == \"${chan_address}\" and .active == true)'"
+echo "- Bob->Chan connected"
+echo -n "- Waiting for Chan channel confirmation..."
+wait-for-node Chan "cli channels | jq -e '.[] | select (.nodeId == \"${dina_address}\" and .state == \"NORMAL\")' > /dev/null"
+echo "- Chan->Dina connected"
+echo "All channels confirmed"
+
+
+echo "======================================================"
+echo -n "Check Alice's route to Dina: "
+run-in-node Alice "cli queryroutes --dest \"${dina_address}\" --amt 10000" > /dev/null 2>&1 \
+&& {
+	echo "Alice has a route to Dina"
+} || {
+	echo "Alice doesn't yet have a route to Dina"
+	echo "Waiting for Alice graph sync. This may take a while..."
+	wait-for-node Alice "cli describegraph | jq -e '.edges | select(length >= 1)'"
+	echo "- Alice knows about 1 channel"
+	wait-for-node Alice "cli describegraph | jq -e '.edges | select(length >= 2)'"
+	echo "- Alice knows about 2 channels"
+	wait-for-node Alice "cli describegraph | jq -e '.edges | select(length == 3)'"
+	echo "- Alice knows about all 3 channels!"
+	echo "Alice knows about all the channels"
+}
+
+echo "======================================================"
+echo
+echo "Get 10k sats invoice from Dina"
+dina_invoice=$(run-in-node Dina "cli addinvoice 10000 | jq -r .payment_request")
+echo "- Dina invoice: "
+echo ${dina_invoice}
+
+echo "======================================================"
+echo
+echo "Attempting payment from Alice to Dina"
+run-in-node Alice "cli payinvoice --json --force ${dina_invoice} | jq -e '.failure_reason == \"FAILURE_REASON_NONE\"'" > /dev/null && {
+	echo "Successful payment!"
+} ||
+{
+	echo "Payment failed"
+}

code/docker/setup-channels.sh

@@ -1,37 +0,0 @@
-#!/bin/bash
-
-echo Getting node IDs
-alice_address=$(docker-compose exec -T Alice bash -c "lncli -n regtest getinfo | jq -r .identity_pubkey")
-bob_address=$(docker-compose exec -T Bob bash -c "lightning-cli getinfo | jq -r .id")
-chan_address=$(docker-compose exec -T Chan bash -c "eclair-cli -s -j -p eclair getinfo| jq -r .nodeId")
-dina_address=$(docker-compose exec -T Dina bash -c "lncli -n regtest getinfo | jq -r .identity_pubkey")
-
-# Let's tell everyone what we found!
-echo Alice:  ${alice_address}
-echo Bob:    ${bob_address}
-echo Chan:    ${chan_address}
-echo Dina: ${dina_address}
-
-echo Setting up channels...
-echo Alice to Bob
-docker-compose exec -T Alice lncli -n regtest connect ${bob_address}@Bob
-docker-compose exec -T Alice lncli -n regtest openchannel ${bob_address} 1000000
-
-echo Bob to Chan
-docker-compose exec -T Bob lightning-cli connect ${chan_address}@Chan
-docker-compose exec -T Bob lightning-cli fundchannel ${chan_address} 1000000
-
-echo Chan to Dina
-docker-compose exec -T Chan eclair-cli -p eclair connect --uri=${dina_address}@Dina
-docker-compose exec -T Chan eclair-cli -p eclair open --nodeId=${dina_address} --fundingSatoshis=1000000
-
-echo Get 10k sats invoice from Dina
-dina_invoice=$(docker-compose exec -T Dina bash -c "lncli -n regtest addinvoice 10000 | jq -r .payment_request")
-
-echo Dina invoice ${dina_invoice}
-
-echo Wait for channel establishment - 60 seconds for 6 blocks
-sleep 60
-
-echo Alice pays Dina 10k sats, routed around the network
-docker-compose exec -T Alice lncli -n regtest payinvoice --json --inflight_updates -f ${dina_invoice}