Restructure modules (#1572)

* Move portbase into monorepo

* Add new simple module mgr

* [WIP] Switch to new simple module mgr

* Add StateMgr and more worker variants

* [WIP] Switch more modules

* [WIP] Switch more modules

* [WIP] swtich more modules

* [WIP] switch all SPN modules

* [WIP] switch all service modules

* [WIP] Convert all workers to the new module system

* [WIP] add new task system to module manager

* [WIP] Add second take for scheduling workers

* [WIP] Add FIXME for bugs in new scheduler

* [WIP] Add minor improvements to scheduler

* [WIP] Add new worker scheduler

* [WIP] Fix more bug related to new module system

* [WIP] Fix start handing of the new module system

* [WIP] Improve startup process

* [WIP] Fix minor issues

* [WIP] Fix missing subsystem in settings

* [WIP] Initialize managers in constructor

* [WIP] Move module event initialization to constrictors

* [WIP] Fix setting for enabling and disabling the SPN module

* [WIP] Move API registeration into module construction

* [WIP] Update states mgr for all modules

* [WIP] Add CmdLine operation support

* Add state helper methods to module group and instance

* Add notification and module status handling to status package

* Fix starting issues

* Remove pilot widget and update security lock to new status data

* Remove debug logs

* Improve http server shutdown

* Add workaround for cleanly shutting down firewall+netquery

* Improve logging

* Add syncing states with notifications for new module system

* Improve starting, stopping, shutdown; resolve FIXMEs/TODOs

* [WIP] Fix most unit tests

* Review new module system and fix minor issues

* Push shutdown and restart events again via API

* Set sleep mode via interface

* Update example/template module

* [WIP] Fix spn/cabin unit test

* Remove deprecated UI elements

* Make log output more similar for the logging transition phase

* Switch spn hub and observer cmds to new module system

* Fix log sources

* Make worker mgr less error prone

* Fix tests and minor issues

* Fix observation hub

* Improve shutdown and restart handling

* Split up big connection.go source file

* Move varint and dsd packages to structures repo

* Improve expansion test

* Fix linter warnings

* Fix interception module on windows

* Fix linter errors

---------

Co-authored-by: Vladimir Stoilov <vladimir@safing.io>

base/metrics/testdata/.gitignore

@@ -0,0 +1 @@
+data

base/metrics/testdata/README.md

@@ -0,0 +1,4 @@
+# Testing metrics
+
+You can spin up a test setup for pushing and viewing metrics with `docker-compose up`.
+Then use the flag `--push-metrics http://127.0.0.1:8428/api/v1/import/prometheus` to push metrics.

base/metrics/testdata/docker-compose.yml

@@ -0,0 +1,36 @@
+version: '3.8'
+
+networks:
+  pm-metrics-test-net:
+
+services:
+
+  victoriametrics:
+    container_name: pm-metrics-test-victoriametrics
+    image: victoriametrics/victoria-metrics
+    command:
+      - '--storageDataPath=/storage'
+    ports:
+      - 8428:8428
+    volumes:
+      - ./data/victoriametrics:/storage
+    networks:
+      - pm-metrics-test-net
+    restart: always
+
+  grafana:
+    container_name: pm-metrics-test-grafana
+    image: grafana/grafana
+    command:
+      - '--config=/etc/grafana/provisioning/config.ini'
+    depends_on:
+      - "victoriametrics"
+    ports:
+      - 3000:3000
+    volumes:
+      - ./data/grafana:/var/lib/grafana
+      - ./grafana:/etc/grafana/provisioning
+      - ./dashboards:/dashboards
+    networks:
+      - pm-metrics-test-net
+    restart: always

base/metrics/testdata/grafana/config.ini

@@ -0,0 +1,10 @@
+[auth]
+disable_login_form = true
+disable_signout_menu = true
+
+[auth.basic]
+enabled = false
+
+[auth.anonymous]
+enabled = true
+org_role = Admin

base/metrics/testdata/grafana/dashboards/portmaster.yml

@@ -0,0 +1,11 @@
+apiVersion: 1
+
+providers:
+  - name: 'Portmaster'
+    folder: 'Portmaster'
+    disableDeletion: true
+    updateIntervalSeconds: 10
+    allowUiUpdates: true
+    options:
+      path: /dashboards
+      foldersFromFilesStructure: true

base/metrics/testdata/grafana/datasources/datasource.yml

@@ -0,0 +1,8 @@
+apiVersion: 1
+
+datasources:
+    - name: VictoriaMetrics
+      type: prometheus
+      access: proxy
+      url: http://pm-metrics-test-victoriametrics:8428
+      isDefault: true