Rocky

Rocky is part of the openKAT project, made with Django. To comply to government standards, Manon is used for style and accessibility. Yarn is used as package manager and ParcelJS is used as bundler to compile the frontend (CSS and Javascript). You can find the Manon repository here.

Installation

Containerized

To run rocky from using Docker, run this from the parent directory nl-kat-coordination:

$ make kat

and continue reading this document at “First run”.

Local

For a local set up, you need to start the Django app and compile the frontend.

Django App

This requires a working Python (>3.10) environment. One example of how to create, activate and initialize a development environment is:

$ python3 -m venv $PWD/.venv
$ source .venv/bin/activate
$ python3 -m pip install -r requirements-dev.txt

Copy the .env-dist to a .env and configure the hosts and credentials to PostgreSQL, RabbitMQ and the other services.

$ cp .env-dist .env

For instance, to configure the PostgreSQL database set the following variables:

ROCKY_DB_HOST=
ROCKY_DB_PORT=
ROCKY_DB=
ROCKY_DB_USER=
ROCKY_DB_PASSWORD=
ROCKY_DB_DSN=

Here, ROCKY_DB_DSN is optional (e.g. postgresql://username:password@hostname:port/database_name) and if not set, the other DB variables will be used.

Once your environment variables are set up (see .env-dist, you can initialize Rocky using:

$ make build-rocky-native

To start the Django server, run:

$ make run

Frontend

Yarn is used to bundle CSS and Javascript.

To compile the frontend using yarn locally, run:

$ yarn --ignore-engine
$ yarn build

To compile the frontend using Docker, run:

$ make build-rocky-frontend

The app should be running at localhost:8000.

TL;DR

Given a proper .env file, run:

$ python3 -m venv $PWD/.venv
$ source .venv/bin/activate
$ python3 -m pip install -r requirements-dev.txt
$ make build-rocky-native
$ & make run
$ make build-rocky-frontend

Development

Testing

To run all unit tests, run:

$ make utest

Tip

A local Python environment is useful for unit testing even when using Docker. Follow the first instructions in the local setup to create a Python environment. Then create a rocky/.env from the template rocky/.env-dist and set ROCKY_DB_HOST=localhost. Now for the unit tests you should be able to just run

$ pytest

to run them locally.

You can easily parallelize the tests can be parallelized using pytest-xdist:

$ python -m pip install pytest-xdist
$ time pytest  # 1:08,92 on 13-02-2024
$ time pytest -n 8  # 21,749 on 13-02-2024

Design

Fonts license

All fonts used within Rocky remain under their own license. For example: Fredoka, Open Sans & Tabler icons.

For more information check their respective folders for extra/ more specific license (if available) or visit:

Fredoka

https://fonts.google.com/specimen/Fredoka/about

Open Sans

https://fonts.google.com/specimen/Open+Sans/about

Tabler icons

https://tabler-icons.io/

Technical Design

Running a boefje

The following diagram shows the triggered flows when running a Boefje from Rocky.

sequenceDiagram participant Rocky participant Scheduler participant Boefje participant Bytes participant Normalizer participant Octopoes Rocky->>+Scheduler: Push Boefje Task Boefje->>Scheduler: Pull Boefje Task Scheduler-->>Rocky: boefje_task.status = dispatched Boefje->>Bytes: Save Raw Boefje->>Scheduler: boefje_task.status = completed Scheduler->>-Rocky: boefje_task.status = completed Bytes-->>Scheduler: Raw File Received Scheduler->>+Scheduler: Push Normalizer Task Normalizer->>Scheduler: Pull Normalizer Task Normalizer->>Bytes: Get Raw Scheduler-->>Rocky: normalizer_task.status = dispatched Normalizer->>Octopoes: Add object(s) Normalizer->>Scheduler: normalizer_task.status = completed Scheduler->>-Rocky: normalizer_task.status = completed

Rocky View Structure

Rocky has a hierarchical set of views that are not easy to capture in a single diagram. We therefore made several diagrams to show the most coherent components.

Overview of child Views of the OrganizationViews

classDiagram direction RL class OrganizationView OrganizationView : organization OrganizationView : octopoes_api_connector OrganizationView : organization_member OrganizationView : indemnification_present OrganizationView <|-- View SinglePluginView <|-- OrganizationView BytesRawView <|-- OrganizationView Health <|-- OrganizationView HealthChecks <|-- OrganizationView IndemnificationAddView <|-- OrganizationView OctopoesView <|-- OrganizationView OOIAddTypeSelectView <|-- OrganizationView Report <|-- OrganizationView OrganizationDetailView <|-- OrganizationView OrganizationMemberEditView <|-- OrganizationView DownloadTaskDetail <|-- OrganizationView TaskListView <|-- OrganizationView UploadCSV <|-- OrganizationView UploadRaw <|-- OrganizationView ObjectsBreadcrumbsMixin <|-- OrganizationView OrganizationMemberBreadcrumbsMixin <|-- OrganizationView FindingTypeAddView <|-- OrganizationView

Exhaustive overview of OctopoesViews

classDiagram direction RL class OrganizationView class OctopoesView class SchedulerView class TaskListView OctopoesView <|-- OrganizationView SchedulerView <|-- OctopoesView TaskListView <|-- SchedulerView BoefjeDetailView <|-- TaskListView OOIDetailView <|-- TaskListView OOIDetailView <|-- OOIRelatedObjectAddView OOIDetailView <|-- OOIFindingManager ChangeClearanceLevel <|-- SchedulerView SingleOOIMixin <|-- OctopoesView SingleOOITreeMixin <|-- SingleOOIMixin BaseOOIDetailView <|-- SingleOOITreeMixin BaseOOIDetailView <|-- ConnectorFormMixin OOIDetailView <|-- BaseOOIDetailView OOIFindingListView <|-- OOIFindingManager OOIFindingListView <|-- BaseOOIDetailView MuteFindingView <|-- BaseOOIDetailView BaseReportView <|-- BaseOOIDetailView OOIReportView <|-- BaseOOIDetailView OOITreeView <|-- BaseOOIDetailView OOISummaryView <|-- OOITreeView OOIGraphView <|-- OOITreeView OOIRelatedObjectManager <|-- SingleOOITreeMixin OOIFindingManager <|-- SingleOOITreeMixin OOIRelatedObjectAddView <|-- OOIRelatedObjectManager OOIReportPDFView <|-- SingleOOITreeMixin OnboardingSetupScanOOIDetailView <|-- SingleOOITreeMixin BaseOOIFormView <|-- SingleOOIMixin OOIDeleteView <|-- SingleOOIMixin OnboardingSetupScanOOIAddView <|-- BaseOOIFormView OOIEditView <|-- BaseOOIFormView OOIAddView <|-- BaseOOIFormView FindingAddView <|-- BaseOOIFormView BaseOOIListView <|-- ConnectorFormMixin OOIListView <|-- BaseOOIListView FindingListView <|-- BaseOOIListView OOIListExportView <|-- BaseOOIListView ScanProfileDetailView <|-- OOIDetailView ScanProfileResetView <|-- OOIDetailView

KATalogus Views

This diagram shows the current view structure and what properties are set in each class for the KATalogus.

classDiagram direction RL class FormView class OrganizationView class SinglePluginView class KATalogusView class PluginSettingsAddView class PluginEnableDisableView class SingleSettingView class PluginSettingsListView OrganizationView : organization OrganizationView : octopoes_api_connector OrganizationView : organization_member OrganizationView : indemnification_present SinglePluginView : katalogus_client SinglePluginView : plugin SinglePluginView : plugin_schema SingleSettingView : setting_name class PluginSettingsDeleteView class BoefjeDetailView class TaskListView KATalogusView <|-- OrganizationView KATalogusView <|-- FormView SinglePluginView <|-- OrganizationView SingleSettingView <|-- SinglePluginView BoefjeDetailView <|-- PluginSettingsListView BoefjeDetailView <|-- TaskListView PluginEnableDisableView <|-- SinglePluginView PluginSettingsAddView <|-- FormView PluginSettingsAddView <|-- SinglePluginView PluginSettingsDeleteView <|-- SingleSettingView PluginSettingsListView <|-- SinglePluginView