Merge pull request 'Fixed upload script' (#68) from Qubasa-main into main

Reviewed-on: #68

pkgs/clan-cli/README.md

@@ -13,6 +13,20 @@ clan webui --reload --no-open --log-level debug --populate --emulate
 - The `--emulate` flag will automatically run servers the database with dummy data for the fronted to communicate with (ap, dlg, c1 and c2)
   - To look into the emulated endpoints go to http://localhost:2979/emulate
 
+# Using the Uploaded Docker Image
+
+Pull the image
+
+```bash
+docker pull git.tu-berlin.de:5000/internet-of-services-lab/service-aware-network-front-end:latest
+```
+
+Run the image
+
+```bash
+docker run -p 127.0.0.1:2979:2979 git.tu-berlin.de:5000/internet-of-services-lab/service-aware-network-front-end:latest
+```
+
 # API Documentation
 
 Api documentation can be found in the folder `pkgs/clan-cli/tests/openapi_client/docs/`
@@ -24,12 +38,23 @@ For Entity object go to
 - [tests/openapi_client/docs/ResolutionApi.md](tests/openapi_client/docs/ResolutionApi.md)
 - [tests/openapi_client/docs/RepositoriesApi.md](tests/openapi_client/docs/RepositoriesApi.md)
 
-# Building a Docker Image if the Backend Changed
+# Building a Docker Image if the Frontend Changed
 
-To build a new docker image when the backend code changed be inside the `pkgs/clan-cli` folder and execute:
+To build a new docker image when the frontend code and/or backend code changed you first need
+to get the `GITLAB_TOKEN` go to [repo access tokens](https://git.tu-berlin.de/internet-of-services-lab/service-aware-network-front-end/-/settings/access_tokens) and generate one.
+
+- Make sure the Gitlab token has access to package registry.
+
+Then execute
 
 ```bash
-nix build .#clan-docker
+export GITLAB_TOKEN="<your-access-token>"
+```
+
+Afterwards you can execute:
+
+```bash
+./build_docker.sh
 ```
 
 This will create a symlink directory called `result` to a tar.gz docker file. Import it by executing:
@@ -44,28 +69,60 @@ And then run the docker file by executing:
 docker run -p 127.0.0.1:2979:2979 clan-docker:latest
 ```
 
-- To change parameters in the generated docker image edit the file :
-  [flake-module.nix at line 22](flake-module.nix)
-- Documentation on `dockerTools.buildImage` you can find here: https://nix.dev/tutorials/nixos/building-and-running-docker-images.html
+# Uploading a Docker Image
 
-## Building a Docker Image if the Frontend Changed
-
-To build a new docker image when the frontend code changed you first need
-to get the `GITLAB_TOKEN` go to [repo access tokens](https://git.tu-berlin.de/internet-of-services-lab/service-aware-network-front-end/-/settings/access_tokens) and generate one. Then execute
+You can use the script:
 
 ```bash
-export GITLAB_TOKEN="<your-access-token>"
-```
-
-Afterwards you can execute:
-
-```bash
-./build_docker.sh
+./push_docker.sh
 ```
 
 ### The Script Explained
 
-If changes to the UI have been made, and you want them to propagate to the docker container edit the file: [../ui/nix/ui-assets.nix](../ui/nix/ui-assets.nix).
+Login to the tu docker image server
+
+```bash
+docker login git.tu-berlin.de:5000
+```
+
+Tag the imported image
+
+```bash
+docker image tag clan-docker:latest git.tu-berlin.de:5000/internet-of-services-lab/service-aware-network-front-end:latest
+```
+
+Push the image to the git registry
+
+```bash
+docker image push git.tu-berlin.de:5000/internet-of-services-lab/service-aware-network-front-end:latest
+```
+
+# Upload UI assets as a package
+
+To upload the release build UI assets to gitlab as a package
+first get the `GITLAB_TOKEN`. Go to [repo access tokens](https://git.tu-berlin.de/internet-of-services-lab/service-aware-network-front-end/-/settings/access_tokens) and generate one.
+
+- Make sure the Gitlab token has access to package registry.
+
+To upload the UI assets as a package then execute:
+
+```bash
+./upload_ui_assets.sh
+```
+
+Please commit the changes to ui-assets.nix and push them to the repository.
+If you want clan webui to use the new ui assets.
+
+```bash
+$ git commit -m "Update ui-assets.nix" "$PROJECT_DIR/pkgs/ui/nix/ui-assets.nix"
+$ git push
+```
+
+If you execute `clan webui` the page you will see is a precompiled release version of the UI. This above script will update said precompiled release version. The `./build_docker.sh` script execute this to make sure that the included UI in the docker is up to date.
+
+### The Script Explained
+
+If changes to the UI have been made, and you want them to propagate to the docker container and the `clan webui` command edit the file: [../ui/nix/ui-assets.nix](../ui/nix/ui-assets.nix).
 This is where a release version of the frontend is downloaded and integrated into the cli and the docker build. To do this first execute
 
 ```bash
@@ -75,7 +132,7 @@ nix build .#ui --out-link ui-release
 Make a tarball out of it called `ui-assets.tar.gz`
 
 ```bash
-tar -czvf ui-assets.tar.gz ui-release/lib/node_modules/clan-ui/out/
+tar --transform 's,^\.,assets,' -czvf "ui-assets.tar.gz" -C ui-release/result/lib/node_modules/*/out .
 ```
 
 Upload ui-assets.tar.gz to gitlab.
@@ -112,45 +169,30 @@ And now build the docker image:
 nix build .#clan-docker
 ```
 
-# Uploading a Docker Image
+# Building a Docker Image if only the Backend Changed
 
-You can use the script:
+To build a new docker image only when the backend code changed execute:
 
 ```bash
-./push_docker.sh
+nix build .#clan-docker
 ```
 
-Login to the tu docker image server
+This is much faster then the `./build_docker.sh` script as it needs not to build the frontend and again.
+This will create a symlink directory called `result` to a tar.gz docker file. Import it by executing:
 
 ```bash
-docker login git.tu-berlin.de:5000
+docker load < result
 ```
 
-Tag the imported image
+And then run the docker file by executing:
 
 ```bash
-docker image tag clan-docker:latest git.tu-berlin.de:5000/internet-of-services-lab/service-aware-network-front-end:latest
+docker run -p 127.0.0.1:2979:2979 clan-docker:latest
 ```
 
-Push the image to the git registry
-
-```bash
-docker image push git.tu-berlin.de:5000/internet-of-services-lab/service-aware-network-front-end:latest
-```
-
-# Using the Uploaded Docker Image
-
-Pull the image
-
-```bash
-docker pull git.tu-berlin.de:5000/internet-of-services-lab/service-aware-network-front-end:latest
-```
-
-Run the image
-
-```bash
-docker run -p 127.0.0.1:2979:2979 git.tu-berlin.de:5000/internet-of-services-lab/service-aware-network-front-end:latest
-```
+- To change parameters in the generated docker image edit the file :
+  [flake-module.nix at line 22](flake-module.nix)
+- Documentation on `dockerTools.buildImage` you can find here: https://nix.dev/tutorials/nixos/building-and-running-docker-images.html
 
 # Auto Generating a Python Client
 

pkgs/clan-cli/build_docker.sh

@@ -1,40 +1,14 @@
 #!/usr/bin/env bash
 
-# shellcheck shell=bash
 set -euo pipefail
 
-# GITLAB_TOKEN
-if [[ -z "${GITLAB_TOKEN:-}" ]]; then
-cat <<EOF
-GITLAB_TOKEN environment var is not set. Please generate a new token under
-https://git.tu-berlin.de/internet-of-services-lab/service-aware-network-front-end/-/settings/access_tokens
-EOF
-  exit 1
-fi
-
-# Create a new ui build
-nix build .#ui --out-link ui-release
-tar -czvf ui-assets.tar.gz ui-release/lib/node_modules/clan-ui/out/
-
-# upload ui assets to gitlab
-gitlab_base="https://git.tu-berlin.de/api/v4/projects/internet-of-services-lab%2Fservice-aware-network-front-end"
-curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
-     --upload-file ./ui-assets.tar.gz \
-     "$gitlab_base/packages/generic/ui-assets/1.0.0/ui-assets.tar.gz"
-
-
-# write url and hash to ui-assets.nix
-url="$gitlab_base/packages/generic/ui-assets/1.0.0/ui-assets.tar.gz"
 PROJECT_DIR=$(git rev-parse --show-toplevel)
-cat > "$PROJECT_DIR/pkgs/ui/nix/ui-assets.nix" <<EOF
-{ fetchzip }:
-fetchzip {
-  url = "$url";
-  sha256 = "$(nix-prefetch-url --unpack $url)";
-}
-EOF
+"$PROJECT_DIR"/pkgs/clan-cli/upload_ui_assets.sh
+
+nix build .#clan-docker
 
 cat <<EOF
+==============================
 Please commit the changes to ui-assets.nix and push them to the repository.
 If you want clan webui to use the new ui assets.
 $ git commit -m "Update ui-assets.nix" "$PROJECT_DIR/pkgs/ui/nix/ui-assets.nix"

pkgs/clan-cli/upload_ui_assets.sh

@@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+
+# shellcheck shell=bash
+set -euo pipefail
+
+# GITLAB_TOKEN
+if [[ -z "${GITLAB_TOKEN:-}" ]]; then
+cat <<EOF
+GITLAB_TOKEN environment var is not set. Please generate a new token under
+https://git.tu-berlin.de/internet-of-services-lab/service-aware-network-front-end/-/settings/access_tokens
+EOF
+  exit 1
+fi
+
+tmpdir=$(mktemp -d)
+cleanup() { rm -rf "$tmpdir"; }
+trap cleanup EXIT
+
+# Create a new ui build
+nix build '.#ui' --out-link "$tmpdir/result"
+
+
+tar --transform 's,^\.,assets,' -czvf "$tmpdir/assets.tar.gz" -C "$tmpdir"/result/lib/node_modules/*/out .
+# upload ui assets to gitlab
+gitlab_base="https://git.tu-berlin.de/api/v4/projects/internet-of-services-lab%2Fservice-aware-network-front-end"
+curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
+     --upload-file "$tmpdir/assets.tar.gz" \
+     "$gitlab_base/packages/generic/ui-assets/1.0.0/ui-assets.tar.gz"
+
+
+# write url and hash to ui-assets.nix
+url="$gitlab_base/packages/generic/ui-assets/1.0.0/ui-assets.tar.gz"
+PROJECT_DIR=$(git rev-parse --show-toplevel)
+cat > "$PROJECT_DIR/pkgs/ui/nix/ui-assets.nix" <<EOF
+{ fetchzip }:
+fetchzip {
+  url = "$url";
+  sha256 = "$(nix-prefetch-url --unpack $url)";
+}
+EOF
+
+
+cat <<EOF
+Please commit the changes to ui-assets.nix and push them to the repository.
+If you want clan webui to use the new ui assets.
+$ git commit -m "Update ui-assets.nix" "$PROJECT_DIR/pkgs/ui/nix/ui-assets.nix"
+$ git push
+EOF

pkgs/ui/nix/ui-assets.nix

@@ -1,5 +1,5 @@
 { fetchzip }:
 fetchzip {
-  url = "https://gitea.gchq.icu/api/packages/IoSL/generic/IoSL-service-aware-frontend/0p1dw924f4sdkq26fd3rrb9qmryl84hdn5plz9ds105xv6al4ikf/assets.tar.gz";
-  sha256 = "0p1dw924f4sdkq26fd3rrb9qmryl84hdn5plz9ds105xv6al4ikf";
+  url = "https://git.tu-berlin.de/api/v4/projects/internet-of-services-lab%2Fservice-aware-network-front-end/packages/generic/ui-assets/1.0.0/ui-assets.tar.gz";
+  sha256 = "1npc1f4pa2d0q16bdygbclnx0c2lsi4j1w8r708xc35s1lvn1a4y";
 }