Local Scanner and Hook deployment

If you are integrating a new scanner or hook and want to test from a local build, this document will guide you through it. This guide assumes that you have your Kubernetes cluster already up and running and that we can work in your default namespace. If not, check out the installation for more information. We also assume that you are or have followed the steps in either the Integrating A Scanner or Integrating A Hook guide.

Makefile-based build & deploy (recommended)

To make local testing easier, the secureCodeBox team has provided a Makefile based solution. The specific Make targets and examples to customize them are given in these documents:

This document explains how to use these targets to deploy your scanner locally.

Scanner only: Inspect your scanner's Makefile. If your scanner uses a remote Docker image (such as one hosted on Dockerhub), you can comment out or remove the custom_scanner line. If you have defined your own Dockerfile in the scanner/ directory, you should leave the line as-is.
In the root of the secureCodeBox git repository, under hook-sdk/nodejs/ and parser-sdk/nodejs/, you need to build the Dockerfiles. This ensures that you have the latest version of the SDK available locally. You need this in order to build secureCodeBox parsers and hooks. To build the image:
1. Minikube: run eval $(minikube docker-env) && make docker-build.
2. Kind: run make docker-build.
In your scanner or hook directory, build the Dockerfiles:
1. Minikube: run eval $(minikube docker-env) && make docker-build.
2. Kind: run make docker-build docker-export kind-import.
Run make deploy to install your Helm chart in your active Kubernetes cluster into the integration-tests namespace. The make target ensures that the image name and tag matches that built in the previous step.
Now run an example scan and inspect whether the images are correctly used.

Example shell

securecodebox$ eval $(minikube docker-env)
securecodebox$ cd parser-sdk/nodejs/
securecodebox/parser-sdk/nodejs$ make docker-build
.: ⚙️ Build 'parser-sdk'.
docker build -t securecodebox/parser-sdk-nodejs:"sha-$(git rev-parse --short HEAD)" .
[...]
Successfully built af5faaf0be6e
Successfully tagged securecodebox/parser-sdk-nodejs:sha-a4490167
securecodebox/parser-sdk/nodejs$ cd ../../scanners/nmap/
securecodebox/scanners/nmap$ make docker-build
.: ⚙️ Build 'nmap' parser with BASE_IMG_TAG: 'sha-a4490167'.
[...]
[Warning] One or more build-args [scannerVersion] were not consumed
Successfully built 931ac83a3e42
Successfully tagged securecodebox/parser-nmap:sha-a4490167
.: ⚙️ Build 'nmap' scanner with BASE_IMG_TAG: 'sha-a4490167'.
[...]
[Warning] One or more build-args [baseImageTag namespace] were not consumed
Successfully built 721c154357eb
Successfully tagged securecodebox/scanner-nmap:sha-a4490167
securecodebox/scanners/nmap$ make deploy
.: 💾 Deploying 'nmap' scanner HelmChart with the docker tag 'sha-a4490167' into kind namespace 'integration-tests'.
[...]
Release "nmap" does not exist. Installing it now.
NAME: nmap
LAST DEPLOYED: Thu Nov 18 15:00:14 2021
NAMESPACE: integration-tests
STATUS: deployed
REVISION: 1
TEST SUITE: None
securecodebox/scanners/nmap$ kubectl get scantypes.execution.securecodebox.io -n integration-tests
NAME   IMAGE
nmap   docker.io/securecodebox/scanner-nmap:sha-a4490167
securecodebox/scanners/nmap$ kubectl get parsedefinitions.execution.securecodebox.io -n integration-tests
NAME       IMAGE
nmap-xml   docker.io/securecodebox/parser-nmap:sha-a4490167

note

Notice that the Make target automatically feeds the chart's AppVersion into the scannerVersion build arg.

Manual build & deploy

Minikube only:

$ eval $(minikube docker-env).

Scanner & Parser

Build your scanner image

securecodebox/scanners/your-custom-scanner/scanner$ docker build \
        --build-arg=scannerVersion="7.91-r0" \
        -t your-custom-scanner:local-dev \
        .

Kind only: import your scanner image

securecodebox$ kind load docker-image your-custom-scanner:local-dev

Build Parser SDK

securecodebox/parser-sdk/nodejs$ docker build -t securecodebox/parser-sdk-nodejs:local-dev .

Build your parser image

securecodebox/scanners/your-custom-scanner/parser$ docker build \
        --build-arg=baseImageTag="local-dev" \
        --build-arg=namespace=securecodebox \
        -t your-custom-parser:local-dev \
        .

Kind only: import your parser image

securecodebox$ kind load docker-image your-custom-parser:local-dev

Update deployment image references. Change the fields in values.yaml file like this:

parser:
  image:
    repository: your-custom-parser
    pullPolicy: Never
    tag: local-dev
scanner:
  image:
    repository: your-custom-scanner
    pullPolicy: Never
    tag: local-dev

Hook

Build Hook SDK

securecodebox/hook-sdk/nodejs$ docker build -t securecodebox/hook-sdk-nodejs:local-dev .

Build hook

securecodebox/hooks/your-custom-hook/hook/$ docker build \
        --build-arg=baseImageTag="local-dev" \
        --build-arg=namespace=securecodebox \
        -t your-custom-hook:local-dev \
        .

Kind only: import your hook image

securecodebox$ kind load docker-image your-custom-hook:local-dev

Update deployment image references. Change the fields in values.yaml file like this:

hook:
  image:
    repository: your-custom-hook
    tag: local-dev
    pullPolicy: Never

Deploy the scanner.

securecodebox/hooks/your-custom-hook/$ helm upgrade --install your-custom-hook .

Debugging

`ImagePullBackOff`

Kubernetes is reporting that it isn't able to find the specified image. Check with kubectl describe pod [name of pod] which image your scanner wants to use and whether you have made that image available. Check your Docker build logs to verify that the image has been correctly tagged. You can also check if the image is actually available:

Minikube: minikube ssh docker images
Kind: docker exec kind-control-plane crictl images

Don't forget that all images you want to use in your Minikube Kubernetes cluster must be either remotely available or made available in your Kubernetes cluster.

Minikube: built using eval $(minikube docker-env).
Kind: imported after building
- Using Makefile: make docker-export kind-import.
- Manually: kind load docker-image parser-nmap:[tag].

Namespace

In some cases, you might accidentally install secureCodeBox in an unexpected namespace. Check that you don't have any left-over releases installed.

$ helm list --all-namespaces
NAME                    NAMESPACE               REVISION    UPDATED                                 STATUS      CHART                                   APP VERSION
nmap                    integration-tests       1           2021-11-18 15:00:14.712583292 +0100 CET deployed    nmap-v3.1.0-alpha1                      7.91-r0
securecodebox-operator  securecodebox-system    1           2021-11-18 10:09:24.804277463 +0100 CET deployed    operator-v3.1.0-alpha1
update-category         integration-tests       1           2021-11-18 11:18:45.104860436 +0100 CET deployed    update-field-hook-v3.1.0-alpha1
update-severity         integration-tests       1           2021-11-18 11:18:45.267164462 +0100 CET deployed    update-field-hook-v3.1.0-alpha1

You can install secureCodeBox components in any namespace, however verify that you are starting your scans in the same namespace as where you deployed your scanner or hook.

$ kubectl apply -f ./nmap-scan.yaml -n integration-tests
scan.execution.securecodebox.io/nmap created
$ kubectl get scans -A
NAMESPACE           NAME   TYPE   STATE   FINDINGS
integration-tests   nmap   nmap   Done    1

Makefile-based build & deploy (recommended)​

Example shell​

Manual build & deploy​

Scanner & Parser​

Hook​

Debugging​

ImagePullBackOff​

Namespace​