dendi-wang/thingsboard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Use docker volume instead of bind mount

Browse Source
This commit is contained in:
Volodymyr Babak 2023-12-08 13:34:36 +02:00
parent d25b1a4170
commit 88d9128b96
9 changed files with 100 additions and 55 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
application/src/main/data/json/edge/instructions/install/docker/instructions.md
View File

@ -4,22 +4,8 @@ Here is the list of commands, that can be used to quickly install ThingsBoard Ed
Install <a href="https://docs.docker.com/engine/install/" target="_blank"> Docker CE</a> and <a href="https://docs.docker.com/compose/install/" target="_blank"> Docker Compose</a>.
#### Create data and logs folders
Run following commands, before starting docker container(s), to create folders for storing data and logs.
These commands additionally will change owner of newly created folders to docker container user.
To do this (to change user) **chown** command is used, and this command requires *sudo* permissions (command will request password for a *sudo* access):
```bash
mkdir -p ~/.mytb-edge-data && sudo chown -R 799:799 ~/.mytb-edge-data
mkdir -p ~/.mytb-edge-logs && sudo chown -R 799:799 ~/.mytb-edge-logs
{:copy-code}
```
#### Running ThingsBoard Edge as docker service
${LOCALHOST_WARNING}
Create docker compose file for ThingsBoard Edge service:
```bash
@ -30,7 +16,7 @@ nano docker-compose.yml
Add the following lines to the yml file:
```bash
version: '3.0'
version: '3.8'
services:
mytbedge:
restart: always
@ -47,8 +33,9 @@ services:
CLOUD_RPC_PORT: ${CLOUD_RPC_PORT}
CLOUD_RPC_SSL_ENABLED: ${CLOUD_RPC_SSL_ENABLED}
volumes:
- ~/.mytb-edge-data:/data
- ~/.mytb-edge-logs:/var/log/tb-edge
- tb-edge-data:/data
- tb-edge-logs:/var/log/tb-edge
${EXTRA_HOSTS}
postgres:
restart: always
image: "postgres:15"
@ -58,7 +45,15 @@ services:
POSTGRES_DB: tb-edge
POSTGRES_PASSWORD: postgres
volumes:
- ~/.mytb-edge-data/db:/var/lib/postgresql/data
- tb-edge-postgres-data:/var/lib/postgresql/data
volumes:
tb-edge-data:
name: tb-edge-data
tb-edge-logs:
name: tb-edge-logs
tb-edge-postgres-data:
name: tb-edge-postgres-data
{:copy-code}
```

3
application/src/main/data/json/edge/instructions/install/docker/localhost_warning.md
View File

@ -1,3 +0,0 @@
###### WARNING NOTE: 'localhost' can not be used as CLOUD_RPC_HOST
Please note that your ThingsBoard base URL is **'localhost'** at the moment. **'localhost'** cannot be used for docker containers - please update **CLOUD_RPC_HOST** environment variable below to the IP address of your machine (*docker **host** machine*). IP address must be `192.168.1.XX` or similar format. In other case - ThingsBoard Edge service, that is running in docker container, will not be able to connect to the cloud.

4
application/src/main/data/json/edge/instructions/upgrade/docker/start_service.md
View File

@ -5,7 +5,7 @@ nano docker-compose.yml
```
```text
version: '3.0'
version: '3.8'
services:
mytbedge:
restart: always
@ -13,7 +13,7 @@ image: "thingsboard/tb-edge:${TB_EDGE_VERSION}"
...
```
Make sure your image is the set to tb-edge-${TB_EDGE_VERSION}.
Make sure your image is the set to **tb-edge-${TB_EDGE_VERSION}**.
Execute the following commands to up this docker compose directly:
```bash

20
application/src/main/data/json/edge/instructions/upgrade/docker/upgrade_db.md
View File

@ -1,16 +1,14 @@
${CLEAR_DOCKER_UPGRADE}
Create docker compose file for ThingsBoard Edge upgrade process:
```bash
nano docker-compose-upgrade.yml
> docker-compose-upgrade.yml && nano docker-compose-upgrade.yml
{:copy-code}
```
Add the following lines to the yml file:
```bash
version: '3.0'
version: '3.8'
services:
mytbedge:
restart: on-failure
@ -18,8 +16,8 @@ services:
environment:
SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/tb-edge
volumes:
- ~/.mytb-edge-data:/data
- ~/.mytb-edge-logs:/var/log/tb-edge
- tb-edge-data:/data
- tb-edge-logs:/var/log/tb-edge
entrypoint: upgrade-tb-edge.sh
postgres:
restart: always
@ -30,7 +28,15 @@ services:
POSTGRES_DB: tb-edge
POSTGRES_PASSWORD: postgres
volumes:
- ~/.mytb-edge-data/db:/var/lib/postgresql/data
- tb-edge-postgres-data:/var/lib/postgresql/data
volumes:
tb-edge-data:
name: tb-edge-data
tb-edge-logs:
name: tb-edge-logs
tb-edge-postgres-data:
name: tb-edge-postgres-data
{:copy-code}
```

65
application/src/main/data/json/edge/instructions/upgrade/docker/upgrade_preparing.md
View File

@ -15,10 +15,69 @@ docker compose rm mytbedge
docker-compose stop
docker-compose rm mytbedge
```
##### Backup Database
Make a copy of the database folder before upgrading:
##### Migrating Data from Docker Bind Mount Folders to Docker Volumes
Starting with the **3.6.2** release, the ThingsBoard team has transitioned from using Docker bind mount folders to Docker volumes.
This change aims to enhance security and efficiency in storing data for Docker containers and to mitigate permission issues across various environments.
To migrate from Docker bind mounts to Docker volumes, please execute the following commands:
```bash
sudo cp -r ~/.mytb-edge-data/db ~/.mytb-edge-db-BACKUP
docker run --rm -v tb-edge-data:/volume -v ~/.mytb-edge-data:/backup busybox sh -c "cp -a /backup/. /volume"
docker run --rm -v tb-edge-logs:/volume -v ~/.mytb-edge-logs:/backup busybox sh -c "cp -a /backup/. /volume"
docker run --rm -v tb-edge-postgres-data:/volume -v ~/.mytb-edge-data/db:/backup busybox sh -c "cp -a /backup/. /volume"
{:copy-code}
```
After completing the data migration to the newly created Docker volumes, you'll need to update the volume mounts in your Docker Compose configuration.
Modify the `docker-compose.yml` file for ThingsBoard Edge to update the volume settings.
First, please update docker compose file version. Find next snippet:
```text
version: '3.0'
...
```
And replace it with:
```text
version: '3.8'
...
```
Then update volume mounts. Locate the following snippet:
```text
volumes:
- ~/.mytb-edge-data:/data
- ~/.mytb-edge-logs:/var/log/tb-edge
...
```
And replace it with:
```text
volumes:
- tb-edge-data:/data
- tb-edge-logs:/var/log/tb-edge
...
```
Apply a similar update for the PostgreSQL service. Find the section:
```text
volumes:
- ~/.mytb-edge-data/db:/var/lib/postgresql/data
...
```
And replace it with:
```text
volumes:
- tb-edge-postgres-data/:/var/lib/postgresql/data
...
```
##### Backup Database
Make a copy of the database volume before upgrading:
```bash
docker run --rm -v tb-edge-postgres-data:/source -v tb-edge-postgres-data-backup:/backup busybox sh -c "cp -a /source/. /backup"
{:copy-code}
```

5
application/src/main/data/json/edge/instructions/upgrade/docker/upgrade_rm.md
View File

@ -1,5 +0,0 @@
Delete docker compose file, if already exists:
```bash
rm docker-compose-upgrade.yml
{:copy-code}
```

11
application/src/main/java/org/thingsboard/server/service/edge/instructions/DefaultEdgeInstallInstructionsService.java
View File

@ -22,6 +22,7 @@ import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.stereotype.Service;
import org.thingsboard.server.common.data.edge.Edge;
import org.thingsboard.server.common.data.edge.EdgeInstructions;
import org.thingsboard.server.dao.util.DeviceConnectivityUtil;
import org.thingsboard.server.queue.util.TbCoreComponent;
import org.thingsboard.server.service.install.InstallScripts;
@ -75,12 +76,12 @@ public class DefaultEdgeInstallInstructionsService implements EdgeInstallInstruc
private EdgeInstructions getDockerInstallInstructions(Edge edge, HttpServletRequest request) {
String dockerInstallInstructions = readFile(resolveFile("docker", "instructions.md"));
String baseUrl = request.getServerName();
if (baseUrl.contains("localhost") || baseUrl.contains("127.0.0.1")) {
String localhostWarning = readFile(resolveFile("docker", "localhost_warning.md"));
dockerInstallInstructions = dockerInstallInstructions.replace("${LOCALHOST_WARNING}", localhostWarning);
dockerInstallInstructions = dockerInstallInstructions.replace("${BASE_URL}", "!!!REPLACE_ME_TO_HOST_IP_ADDRESS!!!");
if (DeviceConnectivityUtil.isLocalhost(baseUrl)) {
dockerInstallInstructions = dockerInstallInstructions.replace("${EXTRA_HOSTS}", "extra_hosts:\n - \"host.docker.internal:host-gateway\"\n");
dockerInstallInstructions = dockerInstallInstructions.replace("${BASE_URL}", "host.docker.internal");
} else {
dockerInstallInstructions = dockerInstallInstructions.replace("${LOCALHOST_WARNING}", "");
dockerInstallInstructions = dockerInstallInstructions.replace("${EXTRA_HOSTS}", "");
dockerInstallInstructions = dockerInstallInstructions.replace("${BASE_URL}", baseUrl);
}
String edgeVersion = appVersion + "EDGE";

8
application/src/main/java/org/thingsboard/server/service/edge/instructions/DefaultEdgeUpgradeInstructionsService.java
View File

@ -82,7 +82,6 @@ public class DefaultEdgeUpgradeInstructionsService implements EdgeUpgradeInstruc
if (edgeUpgradeInfo == null || edgeUpgradeInfo.getNextEdgeVersion() == null || tbVersion.equals(currentEdgeVersion)) {
return new EdgeInstructions("Edge upgrade instruction for " + currentEdgeVersion + "EDGE is not available.");
}
boolean rmUpgradeCompose = false;
StringBuilder result = new StringBuilder(readFile(resolveFile("docker", "upgrade_preparing.md")));
while (edgeUpgradeInfo.getNextEdgeVersion() != null || !tbVersion.equals(currentEdgeVersion)) {
String edgeVersion = edgeUpgradeInfo.getNextEdgeVersion();
@ -93,13 +92,6 @@ public class DefaultEdgeUpgradeInstructionsService implements EdgeUpgradeInstruc
} else {
ubuntuUpgradeInstructions = ubuntuUpgradeInstructions.replace("${UPGRADE_DB}", "");
}
if (!rmUpgradeCompose) {
rmUpgradeCompose = true;
ubuntuUpgradeInstructions = ubuntuUpgradeInstructions.replace("${CLEAR_DOCKER_UPGRADE}", "");
} else {
String rmUpgrade = readFile(resolveFile("docker", "upgrade_rm.md"));
ubuntuUpgradeInstructions = ubuntuUpgradeInstructions.replace("${CLEAR_DOCKER_UPGRADE}", rmUpgrade);
}
ubuntuUpgradeInstructions = ubuntuUpgradeInstructions.replace("${TB_EDGE_VERSION}", edgeVersion + "EDGE");
ubuntuUpgradeInstructions = ubuntuUpgradeInstructions.replace("${FROM_TB_EDGE_VERSION}", currentEdgeVersion + "EDGE");
currentEdgeVersion = edgeVersion;

2
dao/src/main/java/org/thingsboard/server/dao/util/DeviceConnectivityUtil.java
View File

@ -217,7 +217,7 @@ public class DeviceConnectivityUtil {
return host;
}
private static boolean isLocalhost(String host) {
public static boolean isLocalhost(String host) {
try {
InetAddress inetAddress = InetAddress.getByName(host);
return inetAddress.isLoopbackAddress();