This document walks you through the process of setting up a production project using ClusterFuzz.
PrerequisitesMake sure to go through Prerequisites page first.
Create a new Google Cloud projectFollow these instructions to create a new Google Cloud Project.
Verify that your project is successfully created using
gcloud projects describe <your project id>
Export the project id in environment for later use:
export CLOUD_PROJECT_ID=<your project id>
If you’re new to Google Cloud you may be eligible for trial credit.
Enable FirebaseFollow these instructions to add Firebase to the Google Cloud project you just created. This will be used for authentication and should not incur any additional charges.
In the Firebase console, go to the Auth section and enable “Google” as a Sign-in provider.
In the same section, add the domains you plan on using to the “Authorized domains” list. The default domain for App Engine looks like <your project id>.appspot.com
.
To obtain a web API key,
apiKey
value, and export it like so:export FIREBASE_API_KEY=<your api key>
Create OAuth credentials
Follow these instructions to create OAuth credentials for our project setup script. Choose OAuth client ID
credential type. When prompted for an application type, choose Desktop app
. You may also need to fill in the application name on “OAuth consent screen” tab, enter any name of your choice, e.g. MyClusterFuzz
.
Download these credentials as JSON and place it somewhere safe. Export the path for later use:
e.g.
export CLIENT_SECRETS_PATH=/path/to/your/client_secrets.json
Run the project setup script
Now you can run our project setup script to automate the process of setting up a production instance of ClusterFuzz.
This script also creates a config directory for you, which contains some default settings for your deployment and can be later updated.
mkdir /path/to/myconfig # Any EMPTY directory outside the ClusterFuzz source repository.
export CONFIG_DIR=/path/to/myconfig
python butler.py create_config --oauth-client-secrets-path=$CLIENT_SECRETS_PATH \
--firebase-api-key=$FIREBASE_API_KEY --project-id=$CLOUD_PROJECT_ID $CONFIG_DIR
This can take a few minutes to finish, so please be patient. The script also performs a test deployment to verify that the project has been successfully set up.
Check out the configuration yaml files in /path/to/myconfig
directory and change the defaults to suit your use cases. Some common configuration items include:
env.PROJECT_NAME
attribute in project.yaml
.whitelisted_domains
attribute in gae/auth.yaml
.appspot.com
) using domains
attribute in gae/config.yaml
.It’s recommended to check your /path/to/myconfig
directory into your own version control to track your configuration changes and to prevent loss.
To verify that your project is successfully deployed.
Verify that your application is accessible on https://<your project id>.appspot.com
. If you see an error on missing datastore indexes, this may take some time to be generated after the deployment finished. You can check the status here.
Verify that the bots are successfully created using the instructions here. The defaults are 1 regular linux bot and 2 preemptible linux bots on Google Compute Engine.
Now that the initial setup is complete, you may deploy further changes by running:
python butler.py deploy --config-dir=$CONFIG_DIR --prod --force
Configuring number of bots
See this page for instructions to set up the bots.
Once you make changes to the clusters.yaml
file, you must re-deploy by following the previous section. An App Engine cron job will periodically read the contents of this file and create or delete new instances as necessary.
Note that bots do not have to run on Google Compute Engine. It is possible to run your own machines or machines with another cloud provider. To do so, those machines must be running with a service account to access the necessary Google services such as Cloud Datastore and Cloud Storage.
We provide Docker images for running ClusterFuzz bots.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4