Doing upgrade script. I did add that it's advisable to reinstall over upgrading

Adding backup and restore Extra changes in the install script to use $app instead of 'neutrinet'

Doing upgrade script. I did add that it's advisable to reinstall over upgrading
Adding backup and restore Extra changes in the install script to use $app instead of 'neutrinet'
6ce03a43 · HgO · Ilja · bbeaeb4a · 6ce03a43 · 6ce03a43
Commit 6ce03a43 authored Nov 23, 2019 by HgO Committed by Ilja Nov 23, 2019
11 changed files
--- a/README.md
+++ b/README.md
-# Neutrinet YunoHost App
+# Overview
-This application is for Neutrinet members that have an Internet Cube configured has expected.
+The neutrinet application is for Neutrinet members that have an Internet Cube configured and does 2 things:
+* It renews the vpn-certificates
-For now it does 2 things:
+* Adds a webpage with genral information about Neutrinet
-* fixed the situation where your certificate is outdated
-* install neutrinet app list for YunoHost so you get updates for this app
-* install labrique internet/internet cube app list for YunoHost so you get updates for this app
 # Installation
+## From the webinterface
-Either put `https://github.com/neutrinet/neutrinet_ynh` in the administration interface at the bottom of the installation page or do this command in ssh:
+First make sure you have the neutrinet_app list
+1. Go to the admin interface on your cube
-    yunohost app install https://github.com/Neutrinet/neutrinet_ynh --verbose
+2. Click *Applications* > *Install* > At the bottom click *Manage application lists* > Check in the Application list if you have *neutrinet*
+3. If you don't have it > under Custom applications lists you give *neutrinet* under Name. Under URL you give *https://neutrinet.be/apps.json* > Add
+The we can install the application
+1. Click *Applications* at the top of the page
+2. click *Install* > select *All apps* > search for *neutrinet*> click *Install* > Fill in the form (or just keep the defaults) and press Install just like you would install any app from the webinterface
-# Known and "normal" warning/error messages during installation
+## From the CLI
-If you see those, don't worry, it's not a bug.
+First check if you have a list, probably named *neutrinet*, with *https://neutrinet.be/apps.json* as url.
-After `+ sudo virtualenv ve --system-site-packages`
+`yunohost app listlists`
-    Compiling /tmp/pip-build-0Vk8QK/pexpect/pexpect/async.py ...
+If you don't have the list yet, you can add it using
-      File "/tmp/pip-build-0Vk8QK/pexpect/pexpect/async.py", line 16
-        transport, pw = yield from asyncio.get_event_loop()\
-                                 ^
-    SyntaxError: invalid syntax
-This is due to the fact that we don't use python3 but python2 and that doesn't change anything here.
+`yunohost app fetchlist --name neutrinet -u https://neutrinet.be/apps.json`
-After `Running command 'yunohost app setting vpnclient login_passphrase -v "xxxxxxxxxxxxxxxxxxxxx"' ... done`
+Once you have the list, you can install the app using
-    Failed to open /dev/tty: No such device or address
+`yunohost app install neutrinet --debug`
-    Failed to open /dev/tty: No such device or address
-    Failed to open /dev/tty: No such device or address
-    Failed to open /dev/tty: No such device or address
-    Failed to open /dev/tty: No such device or address
-    Failed to open /dev/tty: No such device or address
-    Failed to open /dev/tty: No such device or address
-It doesn't prevent the script from running.
+# For contributers
+## Publish a new version of the app
-# Publish a new version of the app
+* Edit the [manifest](manifest.json) file to bump the version
+* Edit the [upgrade](scripts/upgrade) script with the needed upgrades for previous installations
+* Test the updated version both for new installs and upgrades and make sure the other scripts ([backup](scripts/backup), [remove](scripts/remove) and [upgrade](scripts/upgrade)) also still work
+* In the [apps.json](https://neutrinet.be/apps.json) file you must update the `revision` with the current `sha` on the `master` branch of the package and update the `lastUpdate` field. If you added things to the manifest file, you should add these changes ass well 
- edit the [upgrade](scripts/upgrade) script to bump the version
- update the `revision` with the current `sha` on the `master` branch, update the `lastUpdate` field in the [apps.json](https://neutrinet.be/apps.json) file
- test the updated version: 
-  - check current version of the app on your cube: `yunohost app setting neutrinet version`
-  - update the app with the latest version from master: `yunohost app upgrade neutrinet -u https://github.com/Neutrinet/neutrinet_ynh`
-  - check the app has been updated: `yunohost app setting neutrinet version`
-  - set the app to a previous version if you want to re-test the update: `yunohost app setting neutrinet version -v "0.2.2"`
--- a/conf/nginx.conf
+++ b/conf/nginx.conf
 location PATHTOCHANGE/ {
    alias ALIASTOCHANGE;
-    if ($scheme = http) {
-        rewrite ^ https://$server_name$request_uri? permanent;
-    }
    index index.html;
 }
--- a/manifest.json
+++ b/manifest.json
@@ -2,20 +2,28 @@
    "name": "Neutrinet",
    "id": "neutrinet",
    "description": {
-        "en": "Specific configs for Neutrinet",
+        "en": "Auto renewall for the Neutrinet vpn-certificates",
-        "fr": "Application gérant les configurations spécifiques à Neutrinet"
+        "fr": "Renouvellement automatique des certificats vpn Neutrinet"
    },
+    "version": "0.3.0~ynh1",
    "license": "GPL-3+",
    "maintainer": {
-        "name": "bram",
+        "name": "ilja",
-        "email": "cortex@worlddomination.be",
+        "email": "neutrinet@spectraltheorem.be",
-        "url": "http://worlddomination.be"
+        "url": "https://gitlab.com/Spctrl"
+    },
+    "requirements": {
+        "yunohost": ">= 3.2.0"
    },
    "multi_instance": "false",
+    "services": [
+        "nginx"
+    ],
    "arguments": {
        "install" : [
            {
                "name": "domain",
+                "type": "domain",
                "ask": {
                    "en": "Choose a domain for Neutrinet application",
                    "fr": "Choisissez un domaine pour l'application Neutrinet"
@@ -23,7 +31,8 @@
                "example": "domain.org"
            },
            {
-                "name": "path",
+                "name": "path_url",
+                "type": "path",
                "ask": {
                    "en": "Choose a path for Neutrinet application",
                    "fr": "Choisissez un chemin pour l'application Neutrinet"

--- a/scripts/_common.sh
+++ b/scripts/_common.sh
+#!/bin/bash
--- a/scripts/backup
+++ b/scripts/backup
+#!/bin/bash
+#=================================================
+# GENERIC START
+#=================================================
+# IMPORT GENERIC HELPERS
+#=================================================
+source /usr/share/yunohost/helpers
+source _common.sh
+#=================================================
+# MANAGE SCRIPT FAILURE
+#=================================================
+ynh_abort_if_errors
+#=================================================
+# LOAD SETTINGS
+#=================================================
+app=$YNH_APP_INSTANCE_NAME
+domain=$(ynh_app_setting_get $app domain)
+path_url=$(ynh_app_setting_get $app path_url)
+app_user=$(ynh_app_setting_get $app app_user)
+www_path=$(ynh_app_setting_get $app www_path)
+opt_path=$(ynh_app_setting_get $app opt_path)
+#=================================================
+# STANDARD BACKUP STEPS
+#=================================================
+# BACKUP THE NGINX CONFIGURATION
+#=================================================
+ynh_print_info "Backing up..."
+ynh_backup "$www_path"
+ynh_backup "/etc/nginx/conf.d/$domain.d/$app.conf"
+ynh_backup "$opt_path"
+ynh_backup "/etc/cron.daily/$app-renew-cert"
--- a/scripts/commons
+++ b/scripts/commons
-RENEW_CERT_PATH=/opt/neutrinet/renew_cert
-get_out_of_testing() {
-    set -e
-    if [ -e /etc/apt/sources.list.d/yunohost.list ]; then
-        sudo sed -ri 's#^(deb http://repo\.yunohost\.org/debian[/]? jessie) (stable )?testing#\1 stable#g' /etc/apt/sources.list.d/yunohost.list
-    fi
-}
--- a/scripts/install
+++ b/scripts/install
-set -e
+#!/bin/bash
-source ./commons
+#=================================================
+# GENERIC START
+#=================================================
+# IMPORT GENERIC HELPERS
+#=================================================
-# Retrieve arguments
+source /usr/share/yunohost/helpers
-domain=$1
+source _common.sh
-path=$2
-path=${path%/}
+#=================================================
+# MANAGE SCRIPT FAILURE
+#=================================================
-# Check domain/path availability
+ynh_abort_if_errors
-sudo yunohost app checkurl $domain$path -a neutrinet
-if [[ ! $? -eq 0 ]]; then
+#=================================================
-    exit 1
+# RETRIEVE ARGUMENTS FROM THE MANIFEST
+#=================================================
+domain=$YNH_APP_ARG_DOMAIN
+path_url=$YNH_APP_ARG_PATH_URL
+app=$YNH_APP_INSTANCE_NAME
+app_user=$app
+#==================================================
+# CHECK IF THE APP CAN BE INSTALLED WITH THESE ARGS
+#==================================================
+www_path=/var/www/$app
+if [[ -e $www_path ]]
+then
+  ynh_die "The path $www_path already contains a folder"
 fi
-install_static_file() {
+opt_path=/opt/$app
-    set -e
+if [[ -e $opt_path ]]
+then
+  ynh_die "The path $opt_path already contains a folder"
+fi
-    final_path=/var/www/neutrinet/
+# Normalize the url path syntax
-    sudo mkdir -p $final_path
+path_url=$(ynh_normalize_url_path $path_url)
-    sudo cp -a ../sources/. $final_path
-    sudo chown -R www-data: $final_path
+# Check web path availability
+if ! ynh_webpath_available $domain $path_url
+then
+  ynh_die "$domain$path_url is not available"
+fi
-    if [[ "$path" == "" ]]; then
+# Register (book) web path
-      sed -i "s@PATHTOCHANGE@/@g" ../conf/nginx.conf
+ynh_webpath_register $app $domain $path_url
-    else
-      sed -i "s@PATHTOCHANGE@$path@g" ../conf/nginx.conf
-    fi
-    sed -i "s@ALIASTOCHANGE@$final_path@g" ../conf/nginx.conf
+#=================================================
+# STORE SETTINGS
+#=================================================
-    sudo cp ../conf/nginx.conf /etc/nginx/conf.d/$domain.d/neutrinet.conf
+ynh_app_setting_set $app domain $domain
+ynh_app_setting_set $app path_url $path_url
+ynh_app_setting_set $app app_user $app_user
+ynh_app_setting_set $app www_path $www_path
+ynh_app_setting_set $app opt_path $opt_path
-    sudo service nginx reload
+#=================================================
-}
+# STANDARD MODIFICATIONS
+#=================================================
+# INSTALL DEPENDENCIES
+#=================================================
-install_renew_cert() {
+ynh_print_info "Installing dependencies…"
-    set -e
-    install_dir=$(pwd)
+ynh_install_app_dependencies git python3-venv libssl-dev libffi-dev python3-dev
-    if [ -e $RENEW_CERT_PATH ]; then
+#=================================================
-        sudo rm -rf $RENEW_CERT_PATH
+# CREATE DEDICATED USER
-    fi
+#=================================================
-    sudo apt-get update
+ynh_print_info "Creating app's user…"
-    sudo apt-get install -y python-virtualenv
-    sudo git clone https://github.com/neutrinet/renew_cert $RENEW_CERT_PATH
+mkdir -p $www_path
+ynh_system_user_create $app_user $www_path
-    cd $RENEW_CERT_PATH
+#=================================================
+# INSTALL STATIC FILE
+#=================================================
-    sudo git checkout ccc5b6b58010bb4166c2ae1a72df340395827f99 || echo ""
+ynh_print_info "Installing static site…"
-    # I need system site packages otherwise moulinette is broken
+cp -a ../sources/. $www_path
-    sudo virtualenv ve --system-site-packages
+chown -R $app_user: $www_path
-    sudo ve/bin/pip install -r requirements.txt
-    cd $install_dir
+sed -i "s@PATHTOCHANGE@$path_url@g" ../conf/nginx.conf
-}
+sed -i "s@ALIASTOCHANGE@$www_path@g" ../conf/nginx.conf
-renew_cert() {
+cp ../conf/nginx.conf /etc/nginx/conf.d/$domain.d/$app.conf
-    set -e
-    install_dir=$(pwd)
+nginx -tq
+service nginx reload
-    cd $RENEW_CERT_PATH
+#=================================================
-    sudo ve/bin/python renew_from_cube.py
+# INSTALL RENEW CERT
+#=================================================
-    cd $install_dir
+renew_cert_repo="https://github.com/neutrinet/renew_cert"
-}
+renew_cert_version=$(jq .version ../manifest.json -r -e | cut -d '~' -f 1)
+renew_cert_path="$opt_path/renew_cert"
+renew_cert_virtualenv="$renew_cert_path/ve"
+renew_cert_python="$renew_cert_virtualenv/bin/python3"
+renew_cert_cron_script="renew_cert_cron.sh"
-install_neutrinet_apps_list() {
+ynh_print_info "Installing automatic VPN certificate renewal…"
-    if ! sudo yunohost app listlists | grep -q "neutrinet:"; then
-      sudo yunohost app fetchlist -n neutrinet -u https://neutrinet.be/apps.json
-    fi
-}
-install_labriqueinternet_apps_list() {
+git clone $renew_cert_repo $renew_cert_path
-    if ! sudo yunohost app listlists | grep -q "labriqueinternet:"; then
+git -C $renew_cert_path checkout $renew_cert_version
-      sudo yunohost app fetchlist -n labriqueinternet -u https://labriqueinter.net/apps/labriqueinternet.json
-    fi
-}
-sudo yunohost app setting neutrinet version -v "0.2"
+# We wrap the python3 script that actually renew the VPN certificate
+# This wrapper will be used as a daily cron task
+cp $renew_cert_cron_script $renew_cert_path/$renew_cert_cron_script
-install_neutrinet_apps_list
+# From now on we work in the renew_cert directory
-install_labriqueinternet_apps_list
+cd $renew_cert_path
-get_out_of_testing
+# We need system site packages otherwise moulinette is broken
+python3 -m venv $renew_cert_virtualenv --system-site-packages
+ve/bin/pip install wheel
+ve/bin/pip install -r requirements.txt
-install_static_file
+ynh_print_info "Setting up permissions"
+chown -R $app_user: $opt_path
-install_renew_cert
+chmod 0755 $renew_cert_cron_script
+chown root: $renew_cert_cron_script
-# vpn is not running, let's assume for now that this mean that the vpn is broken
+#=================================================
-if [ ! "$(grep '^port 1195' /etc/openvpn/client.conf)" ]; then
+# SETTING UP CRONTAB
-    renew_cert
+#=================================================
-fi
+ynh_print_info "Setting up cron job for certificate renewal…"
+cat <<EOF > /etc/cron.daily/$app-renew-cert
+#!/bin/bash
+cd $renew_cert_path
+RENEW_CERT_PYTHON="$renew_cert_python" $renew_cert_path/$renew_cert_cron_script
+EOF
+chown root:root /etc/cron.daily/$app-renew-cert
+chmod 0755 /etc/cron.daily/$app-renew-cert
+#=================================================
+# FINALIZATION
+#=================================================
+ynh_print_info "Checking certificates…"
+/etc/cron.daily/$app-renew-cert
--- a/scripts/remove
+++ b/scripts/remove
-source ./commons
+#!/bin/bash
-domain=$(sudo yunohost app setting neutrinet domain)
+#=================================================
+# GENERIC START
+#=================================================
+# IMPORT GENERIC HELPERS
+#=================================================
-sudo rm -rf $RENEW_CERT_PATH
+source /usr/share/yunohost/helpers
-sudo rm -rf /var/www/neutrinet/
+source _common.sh
+#=================================================
+# LOAD SETTINGS
+#=================================================
+app=$YNH_APP_INSTANCE_NAME
+domain=$(ynh_app_setting_get $app domain)
+path_url=$(ynh_app_setting_get $app path_url)
+app_user=$(ynh_app_setting_get $app app_user)
+www_path=$(ynh_app_setting_get $app www_path)
+opt_path=$(ynh_app_setting_get $app opt_path)
+#=================================================
+# STANDARD REMOVE
+#=================================================
+ynh_print_info "Removing static site..."
+rm -rf $www_path
+rm -f /etc/nginx/conf.d/$domain.d/$app.conf
+nginx -t > /dev/null
+service nginx reload
+ynh_print_info "Removing automatic vpn certificate renewal..."
+rm -rf $opt_path
+rm -rf /etc/cron.daily/$app-renew-cert
+#=================================================
+# REMOVE DEPENDENCIES
+#=================================================
+ynh_print_info "Removing dependencies..."
+ynh_remove_app_dependencies
+#=================================================
+# REMOVE DEDICATED USER
+#=================================================
+ynh_print_info "Removing system user..."
+ynh_system_user_delete $app_user
-sudo rm -f /etc/nginx/conf.d/$domain.d/neutrinet.conf
--- a/scripts/renew_cert_cron.sh
+++ b/scripts/renew_cert_cron.sh
+#!/bin/bash
+set -e
+OPENVPN_CONF_DIR="/etc/openvpn"
+OPENVPN_KEYS_DIR="${OPENVPN_CONF_DIR}/keys"
+OPENVPN_CREDENTIALS_FILE="${OPENVPN_KEYS_DIR}/credentials"
+OPENVPN_AUTH_FILE="${OPENVPN_KEYS_DIR}/auth"
+OPENVPN_USER_CERT="${OPENVPN_KEYS_DIR}/user.crt"
+OPENVPN_USER_KEY="${OPENVPN_KEYS_DIR}/user.key"
+OPENVPN_SERVER_CERT="${OPENVPN_KEYS_DIR}/ca-server.crt"
+OPENVPN_CONF_TEMPLATE="${OPENVPN_CONF_DIR}/client.conf.tpl"
+OPENVPN_CLIENT_LOGS="/var/log/openvpn-client.log"
+NEUTRINET_CONF_TEMPLATE="neutrinet_openvpn_config"
+if [[ -z $RENEW_CERT_PATH ]]
+then
+  RENEW_CERT_PATH=$PWD
+fi
+if [[ -z $RENEW_CERT_PYTHON ]]
+then
+  RENEW_CERT_PYTHON=$(command -v python3)
+fi
+RENEW_CERT_SCRIPT="${RENEW_CERT_PATH}/renew.py"
+if [[ -f $OPENVPN_CREDENTIALS_FILE ]]
+then
+  credentials_file=$OPENVPN_CREDENTIALS_FILE
+elif [[ -f $OPENVPN_AUTH_FILE ]]
+then
+  credentials_file=$OPENVPN_AUTH_FILE
+else
+  >&2 echo "ERROR: Cannot find credentials for Neutrinet VPN since neither ${OPENVPN_CREDENTIALS_FILE} nor ${OPENVPN_AUTH_FILE} exists."
+  exit 1
+fi
+login=$(head -n 1 "$credentials_file")
+password=$(tail -n 1 "$credentials_file")
+run_date=$(date +'%Y-%m-%d_%H:%M:%S')
+renew_dir="certs_$run_date"
+$RENEW_CERT_PYTHON $RENEW_CERT_SCRIPT "$login" -p "$password" -c "$OPENVPN_USER_CERT" -d "$renew_dir" -v
+if [[ ! -d $renew_dir || ! -f $renew_dir/ca.crt || ! -f $renew_dir/client.crt || ! -f $renew_dir/client.key ]]
+then
+  echo "Cleaning $renew_dir directory."
+  rm -rf "$renew_dir"
+  exit 0
+fi
+echo "Saving old OpenVPN config"
+cp -r $OPENVPN_CONF_DIR{,.old_${run_date}}
+echo "Copying new OpenVPN config"
+cp "$NEUTRINET_CONF_TEMPLATE" "$OPENVPN_CONF_TEMPLATE"
+echo "Copying new certificates"
+cp "$renew_dir/ca.crt" "$OPENVPN_SERVER_CERT"
+cp "$renew_dir/client.crt" "$OPENVPN_USER_CERT"
+cp "$renew_dir/client.key" "$OPENVPN_USER_KEY"
+echo "Adding user credentials"
+echo -e "$login\n$password" > "$OPENVPN_CREDENTIALS_FILE"
+echo "Updating VPNClient config"
+yunohost app setting vpnclient server_name -v "vpn.neutrinet.be"
+yunohost app setting vpnclient server_port -v "1195"
+yunohost app setting vpnclient server_proto -v "udp"
+yunohost app setting vpnclient service_enabled -v "1"
+yunohost app setting vpnclient login_user -v "$login"
+yunohost app setting vpnclient login_passphrase -v "$password"
+echo "Critical part 1: reloading VPNClient"
+if ! ynh-vpnclient restart && ynh-vpnclient status
+then
+  >&2 echo "ERROR: Failed to restart VPNClient"
+  tail -n 200 "$OPENVPN_CLIENT_LOGS"
+  exit 1
+fi
+echo "Critical part 2: restarting OpenVPN"
+if ! service openvpn restart
+then
+  >&2 echo "ERROR: Failed to restart OpenVPN"
+  journalctl -u openvpn -n 200 --no-pager
+  exit 1
+fi
+sleep 15
+if ! command -v ynh-hotspot > /dev/null
+then
+  exit 0
+fi
+echo "Few, we're done, let's wait 2min to be sure the VPN is running, then restart hotspot"
+sleep 120
+echo "Restarting hotspot"
+if ! ynh-hotspot restart && ynh-hotspot status
+then
+  >&2 echo "ERROR: Failed to restart hotspot"
+  echo "Since it's not a critical part, let's continue"
+fi
--- a/scripts/restore
+++ b/scripts/restore
+#!/bin/bash
+#=================================================
+# GENERIC START
+#=================================================
+# IMPORT GENERIC HELPERS
+#=================================================
+source /usr/share/yunohost/helpers
+source _common.sh
+#=================================================
+# MANAGE SCRIPT FAILURE
+#=================================================
+ynh_abort_if_errors
+#=================================================
+# LOAD SETTINGS
+#=================================================
+app=$YNH_APP_INSTANCE_NAME
+domain=$(ynh_app_setting_get $app domain)
+path_url=$(ynh_app_setting_get $app path_url)
+app_user=$(ynh_app_setting_get $app app_user)
+www_path=$(ynh_app_setting_get $app www_path)
+opt_path=$(ynh_app_setting_get $app opt_path)
+#=================================================
+# CHECK IF THE APP CAN BE RESTORED
+#=================================================
+ynh_print_info "Checking for conflicts…"
+[[ ! -e $www_path ]] || ynh_die "The path $www_path already contains a folder"
+[[ ! -e $opt_path ]] || ynh_die "The path $opt_path already contains a folder"
+# Check web path availability
+if ! ynh_webpath_available "$domain" "$path_url"
+then
+  ynh_die "$domain$path_url is no longer available"
+fi
+#=================================================
+# STANDARD MODIFICATIONS