Merge branch 'standalone_vls' into 'main'

standalone docker image

See merge request lightning-signer/vls-container!13

README.md

@@ -23,64 +23,145 @@ sudo dnf install docker docker-compose containerd runc
 sudo systemctl enable --now docker
 ```
 
-### Docker Compose Compatibility
+### Docker v1/v2 Compatibility
 
-- Currently available `docker-compose` package in different linux distributions (debian, ubuntu, fedora, etc) are not up to date, they are still at version `1` which has been deprecated by `docker` with release of version `2`.
+- Currently available `docker-compose` packages in different linux distributions (debian, ubuntu, fedora, etc) are not up to date, they are still at version `1` which has been deprecated by `docker` with release of version `2`.
 - `docker-compose` version `2` is available through official docker repositories not the distribution ones.
-- Docker Images in this repository work with version 2 and are also __backward compatible__ with version 1.
+- Docker Compose files in this repository work with version 2 and are also __backward compatible__ with version 1.
 - If you are using the distribution installation the `docker compose` command used below has to be changed to `docker-compose` instead.
 
-## Volume Creation
+## VLS standalone Setup
 
+Below are the steps required to run `vlsd2` as a standalone container.
+
+### Docker Image
+
+```bash
+# Pull the latest release image for vlsd
+docker pull registry.gitlab.com/lightning-signer/vls-container/vlsd:latest
+docker tag registry.gitlab.com/lightning-signer/vls-container/vlsd:latest vlsd:latest
+# Build the latest docker image
+cd vlsd
+docker build -t vlsd .
 ```
+
+### Volume Creation
+
+```bash
+docker volume create vls_data
+```
+
+### Environment Variables
+
+VLS container needs the follwing environment variables set:
+- `BITCOIND_RPC_URL`: URL of `bitcoind`'s RPC port.
+- `VLS_NETWORK`: `testnet` or `regtest`.
+
+Frequently used optional environment variables are:
+- `VLS_PERMISSIVE`
+- `RUST_LOG`
+
+### `vlsd2` Command Arguments
+
+Required command arguments:
+- `connect`: URL of `remote_hsmd_socket` running in the lightning node.
+
+For information on all possible arguments to `vlsd` see [documentation](https://gitlab.com/lightning-signer/validating-lightning-signer/-/blob/main/vls-proxy/src/config.rs?ref_type=heads).
+
+### Running container
+
+There is a `docker-compose.yml` in the `vlsd` folder which can be used to run a standalone `vlsd` service with `network_mode` set to host.
+
+```bash
+cd vlsd
+docker compose up
+```
+
+**_Note_**: Make sure to set `BITCOIND_RPC_URL` and `CLN_RMEOTE_HSMD_URL` as either environment variables or in the `docker-compose.yml` file before running the above command.
+
+If you wish to run it as a standalone container without using `docker-compose` you can use the following command:
+
+```bash
+docker run \
+  -d \
+  --rm \
+  --name vlsd \
+  --network host \
+  -e VLS_NETWORK=testnet \
+  -e BITCOIND_RPC_URL=$BITCOIND_RPC_URL \
+  --mount 'type=volume,src=vls_data,dst=/home/vls/.lightning-signer' \
+  vlsd \
+  --connect=$CLN_REMOTE_HSMD_URL
+```
+
+## Single Node Setup
+
+You can run `bitcoind`, `lightningd`, `txood` and `vlsd` on a single node using available docker compose file in the main directory.
+
+**_Note_**: Use this only for experimentation and testing purposes as running `vlsd` on the same machine as `CLN` is not as secure as running it on dedicated hardware.
+
+### Volume Creation
+
+```bash
 docker volume create bitcoin_data
 docker volume create lightning_data
 docker volume create txoo_data
 docker volume create vls_data
 ```
 
-## Docker Compose Run
+### Docker Compose Run
 
-```
-docker compose up --build
+```bash
+docker compose --profile vls up --build
 ```
 
-## Using Bitcoin Chains
+### Selecting Bitcoin Chains
 
-We have three possible overrides over the default `testnet` configuration in `docker-compose.yml`:
+We have two possible overrides over the default `testnet` configuration in `docker-compose.yml`:
 - `docker-compose.testnet.yml`
 - `docker-compose.regtest.yml`
 
 To use override we have to pass it down both the config using `-f` flag:
-```
-export COMPOSE_PROJECT_NAME=<bitcoin_chain>
-docker compose -f docker-compose.yml -f <override_file> up --build
+```bash
+export DOCKER_COMPOSE_OVERRIDE=docker-compose.testnet.yml
+export COMPOSE_PROJECT_NAME=testnet
+docker compose --profile vls -f docker-compose.yml -f $DOCKER_COMPOSE_OVERRIDE up --build
 ```
 
 __Note__: Even while using `testnet` running using the override is recommended as that will expose the `P2P` port for `bitcoind` and `P2P` port for `lightningd` on the host.
 
-## Additional Regtest Commands
+### Single Node without VLS
+
+To run a single node without `vls` service we can use the same `docker-compose.yml` file in the main directory by just removing the `profile` flag `vls` from all commands.
+
+```bash
+docker compose up --build
+```
+
+Above command will run `bitcoind`, `lightningd` and `txood` services on a single node.
+
+### Additional Regtest Commands
 
 We have to run these commands after bitcoind is up and running.
 
 Create Wallet:
-```
+```bash
 docker container exec bitcoind-regtest bitcoin-cli createwallet default
 ```
 
 Generate Address for node:
-```
+```bash
 docker container exec bitcoind-regtest bitcoin-cli getnewaddress
 ```
 
 Generate Blocks
-```
+```bash
 docker container exec bitcoind-regtest bitcoin-cli generatetoaddress 50 $NODE_ADDRESS
 ```
 
-## Testnet CLN + CLBOSS Commands
+### Testnet CLN + CLBOSS Commands
 
-```
+```bash
 # Create a connection to a random node (maybe from 1ML.com)
 docker container exec lightningd-test lightning-cli --testnet connect \
   02ae1e6091d2a9c4db5096558668d2456b1c0e9067cb72273eab1199bcfb208888 67.227.190.47:9735
@@ -98,7 +179,21 @@ docker container exec lightningd-test lightning-cli --testnet clboss-status | le
 docker container exec lightningd-test lightning-cli --testnet summary
 ```
 
-### References
+## Future Work
+
+- [x] standalone dockerfile for vlsd
+- [x] docker compose for vls
+- [x] standalone docker image for `txood`
+- [x] standalone docker image for `lightningd`
+- [x] standalone docker image for `bitcoind`
+- [ ] supporting signet
+- [ ] Profile configuration to run `vls`
+- [ ] Healthcheck for `txoo`
+- [ ] Healthcheck for `vls`
+- [ ] Lightning Storage Server Dockerfile and Compose Service
+- [ ] Postgres Service for Lightning Storage Server
+
+## References
 
 - [bitcoind](https://github.com/ruimarinho/docker-bitcoin-core/blob/master/23/alpine/Dockerfile) by @ruimarinho
 - [lightningd with clboss](https://github.com/tsjk/docker-core-lightning/blob/main/Dockerfile) by @tsjk

bitcoind/Dockerfile

@@ -114,12 +114,12 @@ COPY --from=bitcoin-core ${BITCOIN_PREFIX}/bin/bitcoin-cli /usr/bin/bitcoin-cli
 COPY --from=bitcoin-core ${BITCOIN_PREFIX}/bin/bitcoin-tx /usr/bin/bitcoin-tx
 COPY --from=bitcoin-core ${BITCOIN_PREFIX}/bin/bitcoind /usr/bin/bitcoind
 
-COPY bitcoind/entrypoint.sh /entrypoint.sh
+COPY entrypoint.sh /entrypoint.sh
 RUN chmod +x /entrypoint.sh
 
-COPY assets/bitcoin.conf /bitcoin.conf
+COPY bitcoin.conf /bitcoin.conf
 
-COPY bitcoind/healthcheck.sh /healthcheck.sh
+COPY healthcheck.sh /healthcheck.sh
 RUN chmod +x /healthcheck.sh
 
 RUN mkdir "${BITCOIN_DATA}"

docker-compose.yml

@@ -2,8 +2,7 @@ version: "3.8"
 services:
   bitcoin-core:
     build:
-      dockerfile: ./bitcoind/Dockerfile
-      context: .
+      context: ./bitcoind
     image: bitcoind
     container_name: bitcoind-test
     volumes:
@@ -19,8 +18,7 @@ services:
 
   core-lightning:
     build:
-      dockerfile: ./lightningd/Dockerfile
-      context: .
+      context: ./lightningd
     image: lightningd
     container_name: lightningd-test
     command:
@@ -46,8 +44,7 @@ services:
 
   txoo:
     build:
-      dockerfile: ./txood/Dockerfile
-      context: .
+      context: ./txood
     image: txood
     container_name: txood-test
     restart: unless-stopped
@@ -66,10 +63,11 @@ services:
 
   vls:
     build:
-      dockerfile: ./vlsd/Dockerfile
-      context: .
+      context: ./vlsd
     image: vlsd
     container_name: vlsd-test
+    profiles:
+      - vls
     command:
       - --log-level=info
       - --connect=http://core-lightning:7701

lightningd/Dockerfile

@@ -118,12 +118,12 @@ RUN addgroup -S lightning && adduser -S lightning -G lightning && \
     mkdir -p ${LIGHTNINGD_DATA} && \
     chown -R lightning:lightning ${LIGHTNINGD_DATA}
 
-COPY assets/bitcoin.conf "${BITCOIND_HOME}"/bitcoin.conf
+COPY bitcoin.conf "${BITCOIND_HOME}"/bitcoin.conf
 
-COPY lightningd/entrypoint.sh /entrypoint.sh
+COPY entrypoint.sh /entrypoint.sh
 RUN chmod +x /entrypoint.sh
 
-COPY lightningd/healthcheck.sh /healthcheck.sh
+COPY healthcheck.sh /healthcheck.sh
 RUN chmod +x /healthcheck.sh
 
 VOLUME ["${LIGHTNINGD_DATA}"]
@@ -131,9 +131,9 @@ VOLUME ["${LIGHTNINGD_DATA}"]
 RUN mkdir -p "${BITCOIND_HOME}" && \
     chown -R lightning:lightning "${BITCOIND_HOME}"
 
-COPY assets/bitcoin.conf "${BITCOIND_HOME}"/bitcoin.conf
-COPY assets/testnet-config /testnet-config
-COPY assets/regtest-config /regtest-config
+COPY bitcoin.conf "${BITCOIND_HOME}"/bitcoin.conf
+COPY testnet-config /testnet-config
+COPY regtest-config /regtest-config
 
 USER lightning
 

lightningd/bitcoin.conf

@@ -0,0 +1 @@
+../bitcoind/bitcoin.conf

txood/Dockerfile

@@ -32,7 +32,7 @@ RUN apk update && \
 
 COPY --from=builder /build/txoo/target/release/txood /usr/bin/txood
 
-COPY txood/entrypoint.sh /entrypoint.sh
+COPY entrypoint.sh /entrypoint.sh
 RUN chmod +x /entrypoint.sh
 
 VOLUME ["/root/.txoo/"]

vlsd/Dockerfile

@@ -40,10 +40,10 @@ RUN apk update && \
 
 COPY --from=builder /build/validating-lightning-signer/target/release/vlsd2 /usr/local/bin/vlsd2
 
-COPY vlsd/entrypoint.sh /entrypoint.sh
+COPY entrypoint.sh /entrypoint.sh
 RUN chmod +x /entrypoint.sh
 
-COPY assets/vlsd2.toml /vlsd2.toml
+COPY vlsd2.toml /vlsd2.toml
 
 ENV VLS_DATA=/home/vls/.lightning-signer
 RUN mkdir "${VLS_DATA}"

vlsd/docker-compose.yml

@@ -0,0 +1,21 @@
+version: "3.8"
+services:
+  vls:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    image: vlsd
+    container_name: vlsd-standalone
+    command:
+      - --connect=$CLN_REMOTE_HSMD_URL
+    network_mode: host
+    volumes:
+      - vls_data:/home/vls/.lightning-signer
+    environment:
+      - BITCOIND_RPC_URL=$BITCOIND_RPC_URL
+      - VLS_NETWORK=testnet
+
+volumes:
+  vls_data:
+    name: vls_data
+    external: true