Restructure modules (#1572)

* Move portbase into monorepo * Add new simple module mgr * [WIP] Switch to new simple module mgr * Add StateMgr and more worker variants * [WIP] Switch more modules * [WIP] Switch more modules * [WIP] swtich more modules * [WIP] switch all SPN modules * [WIP] switch all service modules * [WIP] Convert all workers to the new module system * [WIP] add new task system to module manager * [WIP] Add second take for scheduling workers * [WIP] Add FIXME for bugs in new scheduler * [WIP] Add minor improvements to scheduler * [WIP] Add new worker scheduler * [WIP] Fix more bug related to new module system * [WIP] Fix start handing of the new module system * [WIP] Improve startup process * [WIP] Fix minor issues * [WIP] Fix missing subsystem in settings * [WIP] Initialize managers in constructor * [WIP] Move module event initialization to constrictors * [WIP] Fix setting for enabling and disabling the SPN module * [WIP] Move API registeration into module construction * [WIP] Update states mgr for all modules * [WIP] Add CmdLine operation support * Add state helper methods to module group and instance * Add notification and module status handling to status package * Fix starting issues * Remove pilot widget and update security lock to new status data * Remove debug logs * Improve http server shutdown * Add workaround for cleanly shutting down firewall+netquery * Improve logging * Add syncing states with notifications for new module system * Improve starting, stopping, shutdown; resolve FIXMEs/TODOs * [WIP] Fix most unit tests * Review new module system and fix minor issues * Push shutdown and restart events again via API * Set sleep mode via interface * Update example/template module * [WIP] Fix spn/cabin unit test * Remove deprecated UI elements * Make log output more similar for the logging transition phase * Switch spn hub and observer cmds to new module system * Fix log sources * Make worker mgr less error prone * Fix tests and minor issues * Fix observation hub * Improve shutdown and restart handling * Split up big connection.go source file * Move varint and dsd packages to structures repo * Improve expansion test * Fix linter warnings * Fix interception module on windows * Fix linter errors --------- Co-authored-by: Vladimir Stoilov <vladimir@safing.io>
2024-08-09 17:15:48 +02:00
parent 10a77498f4
commit 80664d1a27
647 changed files with 37690 additions and 3366 deletions
--- a/base/database/query/README.md
+++ b/base/database/query/README.md
@@ -0,0 +1,55 @@
+# Query
+
+## Control Flow
+
+- Grouping with `(` and `)`
+- Chaining with `and` and `or`
+  - _NO_ mixing! Be explicit and use grouping.
+- Negation with `not`
+  - in front of expression for group: `not (...)`
+  - inside expression for clause: `name not matches "^King "`
+
+## Selectors
+
+Supported by all feeders:
+- root level field: `field`
+- sub level field: `field.sub`
+- array/slice/map access: `map.0`
+- array/slice/map length: `map.#`
+
+Please note that some feeders may have other special characters. It is advised to only use alphanumeric characters for keys.
+
+## Operators
+
+| Name                    | Textual            | Req. Type | Internal Type | Compared with             |
+|-------------------------|--------------------|-----------|---------------|---------------------------|
+| Equals                  | `==`               | int       | int64         | `==`                      |
+| GreaterThan             | `>`                | int       | int64         | `>`                       |
+| GreaterThanOrEqual      | `>=`               | int       | int64         | `>=`                      |
+| LessThan                | `<`                | int       | int64         | `<`                       |
+| LessThanOrEqual         | `<=`               | int       | int64         | `<=`                      |
+| FloatEquals             | `f==`              | float     | float64       | `==`                      |
+| FloatGreaterThan        | `f>`               | float     | float64       | `>`                       |
+| FloatGreaterThanOrEqual | `f>=`              | float     | float64       | `>=`                      |
+| FloatLessThan           | `f<`               | float     | float64       | `<`                       |
+| FloatLessThanOrEqual    | `f<=`              | float     | float64       | `<=`                      |
+| SameAs                  | `sameas`, `s==`    | string    | string        | `==`                      |
+| Contains                | `contains`, `co`   | string    | string        | `strings.Contains()`      |
+| StartsWith              | `startswith`, `sw` | string    | string        | `strings.HasPrefix()`     |
+| EndsWith                | `endswith`, `ew`   | string    | string        | `strings.HasSuffix()`     |
+| In                      | `in`               | string    | string        | for loop with `==`        |
+| Matches                 | `matches`, `re`    | string    | string        | `regexp.Regexp.Matches()` |
+| Is                      | `is`               | bool*     | bool          | `==`                      |
+| Exists                  | `exists`, `ex`     | any       | n/a           | n/a                       |
+
+\*accepts strings: 1, t, T, true, True, TRUE, 0, f, F, false, False, FALSE
+
+## Escaping
+
+If you need to use a control character within a value (ie. not for controlling), escape it with `\`.
+It is recommended to wrap a word into parenthesis instead of escaping control characters, when possible.
+
+| Location | Characters to be escaped |
+|---|---|
+| Within parenthesis (`"`) | `"`, `\` |
+| Everywhere else | `(`, `)`, `"`, `\`, `\t`, `\r`, `\n`, ` ` (space) |