A RetroSearch Logo

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Search Query:

Showing content from https://github.com/gitpod-io/leeway below:

gitpod-io/leeway: Leeway is a heavily caching build system for Go, Typescript and Docker projects.

Leeway is a heavily caching build system for Go, Yarn and Docker projects. Its features are:

Leeway structures a repository in three levels:

Leeway assumes its running on Linux or macOS. It is very very unlikely that this runs on Windows out-of-the-box. To install, just download and unpack a release.

Place a file named WORKSPACE.yaml in the root of your workspace. For convenience sake you should set the LEEWAY_WORKSPACE_ROOT env var to the path of that workspace. For example:

touch WORKSPACE.yaml
export LEEWAY_WORKSPACE_ROOT=$PWD

The WORKSPACE.yaml may contain some default settings for the workspace:

# defaultTarget is package we build when just running `leeway build`
defaultTarget: some/package:name
#defaultArgs are key=value pairs setting default values for build arguments
defaultArgs:
  key: value

Users can override, and provide additional default arguments using a WORKSPACE.args.yaml file in the workspace root. This is useful for providing local overrides which you might not want to commit to Git. The WORKSPACE.args.yaml takes key value pairs which become available as build arguments. The values herein take precedence over the default arguments in the WORKSPACE.yaml.

Place a BUILD.yaml in a folder somewhere in the workspace to make that folder a component. A BUILD.yaml primarily contains the packages of that components, but can also contain constant values (think of them as metadata). For example:

# const defines component-wide constants which can be used much like build arguments. Only string keys and values are supported.
const:
  internalName: example
  someRandomProperty: value
packages:
- ...
scripts:
- ...

Scripts are a great way to automate tasks during development time (think yarn scripts). Unlike packages they do not run in isolation by default, but have access to the original workspace. What makes scripts special is that they can dependent on packages which become available to a script in the PATH and as environment variables.

Under the scripts key in the component's BUILD.yaml add:

# name is the component-wide unique name of script. Packages and scripts do NOT share a namespace.
# You can have a package called foo and a script called foo within the same component.
name: some-script-name
# description provides a short synopsis of the script. Shown when running `leeway collect scripts`.
description: A sentence describing what the script is good for.
# Deps list dependencies to packages (NOT scripts) which must be built prior to running this script.
# All built dependencies get added to the PATH environment variable. This is handy if your workspace
# contains tools you want to use in a script.
deps:
- some/other:package
# Env sets environment variables which are present during script execution.
env:
- MESSAGE=hello
# Workdir changes the workdir location/layout of working dir of the script. The following choices are available:
# - origin (default): execute the script in the directory of the containing component in the original workspace.
#                     This is the default mode and handy if one wants to automate tasks in the development workspace.
# - packages:         produces a filesystem layout much like during a generic package build where all deps are
#                     found by their name in a temporary directory. This provides some isolation from the original
#                     workspace, while giving full access to the built dependencies.
workdir: origin
# The actual script. For now, only bash scripts are supported. The shebang is added automatically.
script: |
  echo $MESSAGE, this is where the script goes
  if [ "A$(ps -o comm= -p $$)" = "Abash" ]; then
    echo "it's the bash alright"
  fi
  echo "build args work to: ${myBuildArg}"

A package is an entry in a BUILD.yaml in the packages section. All packages share the following fields:

# name is the component-wide unique name of this package
name: must-not-contain-spaces
# Package type must be one of: go, yarn, docker, generic
type: generic
# Sources list all sources of this package. Entries can be double-star globs and are relative to the component root.
# Avoid listing sources outside the component folder.
srcs:
- "**/*.yaml"
- "glob/**/path"
# Deps list dependencies to other packages which must be built prior to building this package. How these dependencies are made
# available during build depends on the package type.
deps:
- some/other:package
# Argdeps makes build arguments version relevant. I.e. if the value of a build arg listed here changes, so does the package version.
argdeps:
- someBuildArg
# Env is a list of key=value pair environment variables available during package build
env:
- CGO_ENABLED=0
# Config configures the package build depending on the package type. See below for details
config:
  ...
config:
  # Packaging method. See https://godoc.org/github.com/gitpod-io/leeway/pkg/leeway#GoPackaging for details. Defaults to library.
  packaging: library
  # If true leeway runs `go generate -v ./...` prior to testing/building. Defaults to false.
  generate: false
  # If true disables `go test -v ./...`
  dontTest: false
  # If true disables the enforcement of `go fmt`. By default, if the code is not gofmt'ed the build fails.
  dontCheckGoFmt: false
  # If true disables the linting stage.
  dontLint: false
  # Overrides the `go build .` command. Supersedes buildFlags.
  buildCommand: []
  # [DEPRECATED: use buildCommand instead] A list of flags passed to `go build`. Useful for passing `ldflags`.
  buildFlags: []
  # Command that's executed to lint the code
  lintCommand: ["golangci-lint", "run"]
  # GoMod can point to a go.mod file outside the component root. Leeway expects a go.sum alongside the go.mod.
  goMod: "../go.mod"
config:
  # yarnlock is the path to the yarn.lock used to build this package. Defaults to `yarn.lock`. Useful when building packages in a Yarn workspace setup.
  # Automatically added to the package sources.
  yarnlock: "yarn.lock"
  # tsconfig is the path to the tsconfig.json used to build this package. Detauls to `tsconfig.json`
  # Automatically added to the package sources.
  tsconfig: "tsconfig.json"
  # packaging method. See https://godoc.org/github.com/gitpod/leeway/pkg/leeway#YarnPackaging for details.
  # Defaults to library
  packaging: library
  # If true disables `yarn test`
  dontTest: false
  # commands overrides the default commands executed during build
  commands:
    install: ["yarn", "install"]
    build: ["yarn", "build"]
    test: ["yarn", "test"]
config:
  # Dockerfile is the name of the Dockerfile to build. Automatically added to the package sources.
  dockerfile: "Dockerfile"
  # Metadata produces a metadata.yaml file in the resulting package tarball.
  metadata:
    foo: bar
  # build args are Docker build arguments. Often we just pass leeway build arguments along here.
  buildArgs:
  - arg=value
  - other=${someBuildArg}
  # image lists the Docker tags leeway will use and push to
  image:
  - gitpod/leeway:latest
  - gitpod/leeway:${__pkg_version}

The first image name of each Docker dependency which pushed an image will result in a build argument. This mechanism enables a package to build the base image for another one, by using the build argument as FROM value. The name of this build argument is the package name of the dependency, transformed as follows:

E.g. component/nested:docker becomes COMPONENT_NESTED__DOCKER.

config:
  # A list of commands to execute. Beware that the commands are not executed in a shell. If you need shell features (e.g. wildcards or pipes),
  # wrap your command in `sh -c`. Generic packages without commands result in an empty tar file.
  commands:
  - ["echo", "hello world"]
  - ["sh", "-c", "ls *"]

Packages can be dynamically produced within a component using a dynamic package script named BUILD.js. This ECMAScript 5.1 file is executed using Goja and produces a packages array which contains the package struct much like they'd exist within the BUILD.yaml. For example:

Leeway interacts with the script using global variables, specifically:

BUILD.js file

let packages = [];

let deps = [];
for(let i = 0; i < 5; i++) {
  const name = "hello-"+i;
  deps.push(name);
  packages.push({
    name: name,
    type: "generic",
    config: {
      commands: [
        ["echo", args.msg + ": hello from "+i]
      ]
    }
  });
}

packages.push({
  name: "all",
  type: "generic",
  deps: deps.map(d => ":" + d),
})

Equivalent BUILD.yaml

pacakages:
- name: all
  type: generic
  deps:
    - hello-1
    - hello-2
    - hello-3
    - hello-4
    - hello-5
- name: hello-1
  type: generic
  config:
    commands:
      - ["echo", "${msg}: hello from 1"]
- name: hello-2
  type: generic
  config:
    commands:
      - ["echo", "${msg}: hello from 2"]
- name: hello-3
  type: generic
  config:
    commands:
      - ["echo", "${msg}: hello from 3"]
...

Note that for a BUILD.js to become effective/be recodnized there needs to a (possibly empty) BUILD.yaml in the same directory.

In a package definition one can use build arguments. Build args have the form of ${argumentName} and are string-replaced when the package is loaded. It's advisable to use build args only within the config section of packages. Constants and built-in build args do not even work outside of the config section.

Leeway supports built-in build arguments:

Leeway supports build-time variance through "package variants". Those variants are defined on the workspace level and can modify the list of sources, environment variables and config of packages. For example consider a WORKSPACE.YAML with this variants section:

variants:
- name: nogo
  srcs:
    exclude:
    - "**/*.go"
  config:
    go:
      buildFlags:
        - tags: foo

This workspace has a (nonsensical) nogo variant that, when enabled, excludes all go source files from all packages. It also changes the config of all Go packages to include the -tags foo flag. You can explore the effects of a variant using collect and describe, e.g. leeway --variant nogo collect files vs leeway collect files. You can list all variants in a workspace using leeway collect variants.

Leeway does not control the environment in which it builds the packages, but assumes that all required tools are available already (e.g. go or yarn). This however can lead to subtle failure modes where a package built in one enviroment ends up being used in another, because no matter of the environment they were built in, they get the same version.

To prevent such issues, leeway computes an environment manifest which contains the versions of the tools used, as well as some platform information. The entries in that manifest depend on the package types used by that workspace, e.g. if only Go packages exist in the workspace, only go version, GOOS and GOARCH will be part of the manifest. You can inspect a workspace's environment manifest using leeway describe environment-manifest.

You can add your own entries to a workspace's environment manifest in the WORKSPACE.yaml like so:

environmentManifest:
  - name: gcc
    command: ["gcc", "--version"]

Using this mechanism you can also overwrite the default manifest entries, e.g. "go" or "yarn".

SBOM and Vulnerability Scanning

Leeway includes built-in support for Software Bill of Materials (SBOM) generation and vulnerability scanning. This feature helps you identify and manage security vulnerabilities in your software supply chain.

SBOM generation is configured in your WORKSPACE.yaml file:

sbom:
  enabled: true                # Enable SBOM generation
  scanVulnerabilities: true    # Enable vulnerability scanning
  failOn: ["critical", "high"] # Fail builds with vulnerabilities of these severities (default: build does not fail)
  ignoreVulnerabilities:       # Workspace-level ignore rules
    - vulnerability: "CVE-2023-1234"
      reason: "Not exploitable in our context"

When enabled, Leeway automatically generates SBOMs for each package during the build process in multiple formats (CycloneDX, SPDX, and Syft JSON) using Syft. These SBOMs are included in the package's build artifacts.

Leeway provides two commands for working with SBOMs:

The sbom export command allows you to export the SBOM of a previously built package:

# Export SBOM in CycloneDX format (default) to stdout
leeway sbom export some/component:package

# Export SBOM in a specific format to a file
leeway sbom export --format spdx --output sbom.spdx.json some/component:package

# Export SBOMs for a package and all its dependencies to a directory
leeway sbom export --with-dependencies --output-dir sboms/ some/component:package

Options:

This command uses existing SBOM files from previously built packages and requires SBOM generation to be enabled in the workspace settings.

The sbom scan command scans a package's SBOM for vulnerabilities and exports the results:

# Scan a package for vulnerabilities
leeway sbom scan --output-dir vuln-reports/ some/component:package

# Scan a package and all its dependencies for vulnerabilities
leeway sbom scan --with-dependencies --output-dir vuln-reports/ some/component:package

Options:

This command uses existing SBOM files from previously built packages and requires SBOM generation to be enabled in the workspace settings (vulnerability scanning does not need to be enabled).

When scanVulnerabilities is enabled, Leeway scans the generated SBOMs for vulnerabilities using Grype. The scan results are written to the build directory in multiple formats:

Configuring Build Failure Thresholds

The failOn setting determines which vulnerability severity levels will cause a build to fail. Omit this configuration to generate only the reports without causing the build to fail. For example:

failOn: ["critical", "high"]  # Fail on critical and high vulnerabilities

Supported severity levels are: critical, high, medium, low, negligible, and unknown.

Leeway provides a flexible system for ignoring specific vulnerabilities. Ignore rules can be defined at both the workspace level (in WORKSPACE.yaml) and the package level (in BUILD.yaml). For detailed documentation on ignore rules, see Grype's documentation on specifying matches to ignore.

Ignore Rule Configuration

Ignore rules use Grype's powerful filtering capabilities:

# In WORKSPACE.yaml (workspace-level rules)
sbom:
  ignoreVulnerabilities:
    # Basic usage - ignore a specific CVE
    - vulnerability: "CVE-2023-1234"
      reason: "Not exploitable in our context"
      
    # Advanced usage - ignore a vulnerability only for a specific package
    - vulnerability: "GHSA-abcd-1234-efgh"
      reason: "Mitigated by our application architecture"
      package:
        name: "vulnerable-pkg"
        version: "1.2.3"
        
    # Using fix state
    - vulnerability: "CVE-2023-5678"
      reason: "Will be fixed in next dependency update"
      fix-state: "fixed"
      
    # Using VEX status
    - vulnerability: "CVE-2023-9012"
      reason: "Not affected as we don't use the vulnerable component"
      vex-status: "not_affected"
      vex-justification: "vulnerable_code_not_in_execute_path"
Package-Level Ignore Rules

You can also specify ignore rules for specific packages in their BUILD.yaml file:

# In package BUILD.yaml
packages:
  - name: my-package
    type: go
    # ... other package configuration ...
    sbom:
      ignoreVulnerabilities:
        - vulnerability: "GHSA-abcd-1234-efgh"
          reason: "Mitigated by our application architecture"

Package-level rules are combined with workspace-level rules during vulnerability scanning.

Available Ignore Rule Fields

Leeway's ignore rules support all of Grype's filtering capabilities:

The package field can contain:

Leeway is configured exclusively through the WORKSPACE.yaml/BUILD.yaml files and environment variables. The following environment variables have an effect on leeway:

Provenance (SLSA) - EXPERIMENTAL

leeway can produce provenance information as part of a build. At the moment only SLSA is supported. This supoprt is experimental.

Provenance generation is enabled in the WORKSPACE.YAML file.

provenance:
  enabled: true
  slsa: true

Once enabled, all packages carry an attestation bundle which is compliant to the SLSA v0.2 spec in their cached archive. The bundle is complete, i.e. not only contains the attestation for the package build, but also those of its dependencies.

Dirty vs clean Git working copy

When building from a clean Git working copy, leeway will use a reference to the Git remote origin as material (part of the SLSA link).

To support SLSA level 2, leeway can sign the attestations it produces. To this end, you can provide the filepath to a key either as part of the WORKSPACE.yaml or through the LEEWAY_PROVENANCE_KEYPATH environment variable.

You can inspect the generated attestation bundle by extracting it from the built and cached archive. For example:

# run a build
leeway build //:app

# export the attestation bundle
leeway provenance export //:app

# export the decoded attestation bundle
leeway provenance export --decode //:app

# verify that all material came from a Git repo
leeway provenance assert --git-only //:app

# verify that all subjects were built using leeway
leeway provenance asert --built-with-leeway //:app

# decode an attestation bundle from a file (also works for assertions)
leeway provenance export --decode file://some-bundle.jsonl

When a build fails, or to get an idea of how leeway assembles dependencies, run your build with leeway build -c local (local cache only) and inspect your $LEEWAY_BUILD_DIR.

How can I build a package in the current component/folder? 
leeway build .:package-name
Is there bash autocompletion?

Yes, run . <(leeway bash-completion) to enable it. If you place this line in .bashrc you'll have autocompletion every time.

How can I find all packages in a workspace? 
# list all packages in the workspace
leeway collect
# list all package names using Go templates
leeway collect -t '{{ range $n := . }}{{ $n.Metadata.FullName }}{{"\n"}}{{end}}'
# list all package names using jq
leeway collect -o json | jq -r '.[].metadata.name'
How can I find out more about a package? 
# print package description on the console
leeway describe some/components:package
# dump package description as json
leeway describe some/components:package -o json
How can I inspect a packages depdencies? 
# print the dependency tree on the console
leeway describe dependencies some/components:package
# print the denendency graph as Graphviz dot
leeway describe dependencies --dot some/components:package
# serve an interactive version of the dependency graph
leeway describe dependencies --serve=:8080 some/components:package
How can I print a component constant? 
# print all constants of the component in the current working directory
leeway describe const .
# print all constants of a component
leeway describe const some/component/name
# print the value of the `someName` constant of `some/component/name`
leeway describe const some/component/name -o json | jq -r '.[] | select(.name=="foo").value'
How can I find all components with a particular constant? 
leeway collect components -l someConstant
How can I export only a workspace the way leeway sees it, i.e. based on the packages? 
LEEWAY_EXPERIMENTAL=true leeway export --strict /some/destination
macOS: leeway fails with "cp --parents" not being a valid command

The way depends on GNU utilities. Install them and make sure they're in your path.

Releases of Leeway are created by the release workflow which uses goreleaser/goreleaser-action.

To create a new release create a new Git tag and push it:

git tag vX.X.X
git push origin vX.X.X

This will trigger the workflow which you can find here. Once it's done a new release will show up in GitHub here.

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4