A RetroSearch Logo

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Search Query:

Showing content from https://github.com/scala/scala-jenkins-infra below:

scala/scala-jenkins-infra: A Chef cookbook that manages Scala's CI infrastructure.

Jenkins-based CI for the Scala project

Used for

Old documentation in doc directory; some pieces are still relevant, some are outdated.

The AWS infra was originally deployed and maintanied using ansible (this repo). We no longer use ansible to update the infra, but manage it manually (AWS console, ssh to the machines, jenkins UI, artifactory UI).

We are gradually moving away from using this infra

But we still need it because

Webhooks, tokens, accounts etc Add the following to your ~/.ssh/config 
Host jenkins-master
  HostName 54.67.111.226
  User admin

Host jenkins-worker-behemoth-1
  HostName 54.153.2.9
  User admin

Host jenkins-worker-behemoth-2
  HostName 54.153.1.99
  User admin

Host jenkins-worker-behemoth-3
  HostName 54.183.156.89
  User admin

# no public ip, jumphost through master
Host influxdb
  HostName 172.31.0.100
  User ubuntu
  ProxyCommand ssh -q -W %h:%p jenkins-master

Nginx for scala-ci.typesafe.com on jenkins-master, /etc/nginx/conf.d/jenkins.conf.

Handles jenkins, /artifactory, /grafana, /benchq, ...

Auth goes via GitHub. In the configuraton there's a field "Admin User Names".

AWS Workers are started / stopped by custom jenkins plugin: https://github.com/lightbend-labs/ec2-start-stop

TODO: how to get logs?

scala-ci.typesafe.com/artifactory/ to log in to the UI.

Repositories:

File locations

Database

Upgrading artifactory

Scabot triggers Jenkins builds and updates their state on github commits / PRs.

Use ncdu -x /path to analyze disk usage.

Jenkins workers disk space

Artifactory disk usage report: https://scala-ci.typesafe.com/ui/admin/monitoring/storage-summary.

Steps to delete old builds from scala-pr-validation-snapshots:

Create a file search.json, adjust the cutoff date on the last line:

items.find({
  "repo": "scala-pr-validation-snapshots",
  "$or": [ { "name": { "$match": "scala-compiler*" } }, {"name": { "$match": "scala-reflect*" } }, { "name": { "$match": "scala-library*" } }, { "name": { "$match": "scala-dist*" } }, { "name": { "$match": "scala-partest*" } }, { "name": { "$match": "scalap*" } } ],
  "created": { "$lt": "2020-01-01" }
})

curl -u 'lukas:SEEEKREET' -X POST "https://scala-ci.typesafe.com/artifactory/api/search/aql" -T search.json > artifacts.json

In an up-to-date Scala 2.13.x checkout, the following tests which of the artifacts correspond to revisions that were actually merged into scala/scala. Builds for those revisions are kept, builds for revisions that never made it are added to to-delete.txt.

n=$(cat artifacts.json | jq -r '.results[] | .path' | uniq | wc -l)
for p in $(cat artifacts.json | jq -r '.results[] | .path' | uniq); do
  n=$((n-1))
  sha=$(echo $p | awk -F'-' '{print $(NF-1)}')
  if git branch --contains $sha | grep 2.13.x > /dev/null; then
    echo "$sha y - $n"
  else
    echo "$sha n - $n"
    echo $p >> to-delete.txt
  fi
done

Delete the artifacts; best run it on ssh jenkins-master for performance.

n=$(cat to-delete.txt | wc -l)
for p in $(cat to-delete.txt); do
  n=$((n-1))
  echo "$p - $n"
  curl -u 'lukas:PASSWORDSEKRET' -X DELETE "https://scala-ci.typesafe.com/artifactory/scala-pr-validation-snapshots/$p"
done

After that

Other measures

Resize drives / file systems

Enlarging drives and their partitions seems to work well, even for the root partition of a running system (Debian).

To recreate a drive / file system (to shrink it, or to move to a different file system), create a new EBS volume, mount it and copy the data over using rsync.

Enabled on master and behemoths

Only a basic jre is installed through apt (eg openjdk-17-jre-headless).

Use sudo su and cd to /usr/lib/jvm, see the README file. Install new JDKs here, we default to adoptium.

On all machines (chronyc tracking to check):

root@ip-172-31-10-237:~# cat /etc/chrony/sources.d/aws.sources
#https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configure-ec2-ntp.html
server 169.254.169.123 prefer iburst minpoll 4 maxpoll 4

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4