This repository contains the FuseML APIs definitions and core service. For the general information about FuseML project, check the main repository page.
If you are using fuseml installer, the core service is installed into your Kubernetes cluster along with other components and the command line client is downloaded to your working directory. It is recommended to copy the client to the location with other executables, e.g. /usr/local/bin/ on the Linux systems.
To download pre-built client and server components, check the releases page.
To build the latest version directly from sources:
Make sure you have go installed, at least version 1.16.
Download protocol buffers. Use the release page for version 3.15.7 and install only the protoc binary.
On Linux, you can proceed this way:
wget https://github.com/protocolbuffers/protobuf/releases/download/v3.15.7/protoc-3.15.7-linux-x86_64.zip unzip protoc-3.15.7-linux-x86_64.zip sudo cp -a bin/protoc /usr/local/bin/
Clone fuseml-core repository
git clone git@github.com:fuseml/fuseml-core.git cd fuseml-core
Install go dependencies
Generate server, client and CLI code and build the binaries.
This will produce server (bin/fuseml_core) and command line client (bin/fuseml) binaries. The client binary is compiled for your current architecture.
Run the server locally
After successful installation using fuseml-installer, fuseml-core server is already running in the Kubernetes cluster. However, for development and testing purposes it is also possible to run it locally by setting the following environment variables:
GITEA_URLGITEA_ADMIN_USERNAMEGITEA_ADMIN_PASSWORDand executing bin/fuseml_core.
The variables contain the values describing connection to the gitea server and credentials for the admin user. If you have used fuseml-installer to set up the environment, there is already default gitea instance installed in your Kubernetes cluster. In such case set the values this way:
export GITEA_URL=http://$(kubectl get VirtualService -n gitea gitea -o jsonpath="{.spec.hosts[0]}")
export GITEA_ADMIN_USERNAME=$(kubectl get Secret -n fuseml-workloads gitea-creds -o jsonpath="{.data.username}" | base64 -d)
export GITEA_ADMIN_PASSWORD=$(kubectl get Secret -n fuseml-workloads gitea-creds -o jsonpath="{.data.password}" | base64 -d)
It is possible to use external gitea server instead. Make sure to provide correct environment variables.
TEKTON_DASHBOARD_URL is the path to the Tekton server. As with other components, Tekton is installed by fuseml-installer into your cluster. To get the right URL, call
export TEKTON_DASHBOARD_URL=http://$(kubectl get VirtualService -n tekton-pipelines tekton -o jsonpath="{.spec.hosts[0]}")
Now it's possible to execute bin/fuseml_core. Use the --help flag to get the command line options that you can supply. By default the server listens on the follwing ports: 8000 (http) and 8080 (grpc)
Run the client
Executing the client with --help option will show the usage instructions
> bin/fuseml --help
FuseML command line client
Usage:
bin/fuseml [command]
Available Commands:
application application management
codeset codeset management
help Help about any command
runnable runnable management
version display version information
workflow Workflow management
Flags:
-h, --help help for bin/fuseml
--timeout int (FUSEML_HTTP_TIMEOUT) maximum number of seconds to wait for response (default 30)
-u, --url string (FUSEML_SERVER_URL) URL where the FuseML service is running
-v, --verbose (FUSEML_VERBOSE) print verbose information, such as HTTP request and response details
Use "bin/fuseml [command] --help" for more information about a command.
Instead of providing --url argument to each command call, you can save the service URL into FUSEML_SERVER_URL environment variable and export it. Use this command to fill the variable with the URL of the fuseml-core server service that is installed in your Kubernetes cluster:
export FUSEML_SERVER_URL=http://$(kubectl get VirtualService -n fuseml-core fuseml-core -o jsonpath="{.spec.hosts[0]}")
The FuseML client allows you to manage the various supported artifacts (application, codeset, runnable and workflow). Use the --help on each available command to get a more detailed description the command and instructions on how to use it.
Codesets contain the code of your ML application, for example MLflow project. They are currently implemented as git repositories.
Create new codeset by calling the bin/fuseml codeset register command.
Example:
bin/fuseml codeset register --name "test" --project "mlflow-project-01" "/tmp/mlflow/mlflow-01"
Last argument points either to the directory on your machine where your ML application code is located or it can actually point to a git repository with the application code.
After registering, use
command to list available codesets and check the value of "URL" in the output of your newly registered codeset. Use this value to git clone the code into another directory. Now you can work in this directory just like with any other project saved in git. Assuming that you assigned a workflow to this codeset (see workflows section), every time you push new code changes a new workflow run will be created and an updated application will be created.
Note: the codeset list command allows filtering the output by project or user defined labels.
Workflows define the full AI/ML workflow. In short, this could be described as a way to process the input (the Codeset) and turn it into the output application (e.g. ML predictor).
To create a new workflow, use the following command:
bin/fuseml workflow create workflow.yaml
After the workflow is created it can be assigned to a codeset. By doing that, the workflow will be automatically executed every time you push a new change to the codeset that the workflow has been assigned to. The first time the workflow is assigned to a codeset a workflow run is also created, which will execute the workflow with its default inputs and the codeset it has been assigned to.
bin/fuseml workflow assign --name "workflow-name" --codeset-name "test" --codeset-project "mlflow-project-01"
To see the progress of running workflow, check the list-runs command:
bin/fuseml workflow list-runs --workflow-name mlflow-sklearn-e2e
To get even more details follow, use the yaml format for the output (--format yaml) which will provide a url to the Tekton Pipeline status on the Tekton Dashboard. Alternatively go to Tekton dashboard in your browser (remember TEKTON_DASHBOARD_URL extracted earlier) and select the correspondent pipeline run under the PipelineRuns menu.
Applications are basically the output services of AI/ML workflow. So if your workflow describes the way from the code, to the trained model, to the serving, the application being served as the last step is considered the FuseML application.
Applications are registered automatically by workflows. Use
bin/fuseml application list
to list existing applications. The output contains the URL where the application can be accessed, e.g. the URL of the prediction service.
Let's look at the example for MLflow model, being trained by MLflow and served with KServe.
Install FuseML and fuseml-core client
Follow the FuseML installation guide to install all necessary services, including fuseml-core.
Install fuseml binary to some place within your PATH so you do not need to execute it with the full path.
Set the value of FUSEML_SERVER_URL, to point to the server URL:
export FUSEML_SERVER_URL=http://$(kubectl get VirtualService -n fuseml-core fuseml-core -o jsonpath="{.spec.hosts[0]}")Get the example code
Check out the examples project:
git clone git@github.com:fuseml/examples.git fuseml-examples
Under codesets/mlflow/sklearn directory there is the example MLflow project. It is a slightly modified example based on upstream MLflow.
Under workflows directory there is an example definition of a FuseML workflow.
Register the codeset
Register the example MLflow project as a codeset:
fuseml codeset register --name sklearn --project mlflow fuseml-examples/codesets/mlflow/sklearn
Create a workflow
fuseml workflow create fuseml-examples/workflows/mlflow-e2e.yaml
Assign the workflow to the codeset
fuseml workflow assign --name mlflow-e2e --codeset-name sklearn --codeset-project mlflow
Now that the Workflow is assigned to the Codeset, a new workflow run was created. To check the workflow run status, execute the following command:
fuseml workflow list-runs --name mlflow-e2e
This command shows you detailed information about workflows runs. You can also access the TEKTON_DASHBOARD_URL to check all available workflow runs.
Once the running workflow reaches the Succeeded state, a new FuseML application has been be created serving the trained model from the Codeset.
Use the prediction service
Check the available applications by running the following command:
This should produce output similar to the following (notice the fake "example.io" domain here):
+---------------------------+-----------+----------------------------------------------+-------------------------------------------------------------------------------------------------------+--------------+ | NAME | TYPE | DESCRIPTION | URL | WORKFLOW | +---------------------------+-----------+------------------------------------------------------+------------------------------------------------------------------------------------------------+------------+ | mlflow-sklearn-mlflow-e2e | predictor | Application generated by mlflow-e2e workflow | http://mlflow-sklearn-mlflow-e2e.fuseml-workloads.example.io/v2/models/mlflow-sklearn-mlflow-e2e/infer | mlflow-e2e | +--------------------------------+-----------+-----------------------------------------+--------------------------------------------------------------------------------------------------------+------------+
Save the application URL in a variable for using in a later step:
PREDICTION_URL=$(fuseml application get -n mlflow-sklearn-mlflow-e2e --format json | jq -r ".url")
Take a look at the example data file from the examples repository, the prediction will be made according to the submitted data:
> cat fuseml-examples/prediction/data-sklearn.json
{
"inputs": [
{
"name": "input-0",
"shape": [1, 11],
"datatype": "FP32",
"data": [
[12.8, 0.029, 0.48, 0.98, 6.2, 29, 7.33, 1.2, 0.39, 90, 0.86]
]
}
]
}
Now perform a request to $PREDICTION_URL with the example data:
curl -d @fuseml-examples/prediction/data-sklearn.json $PREDICTION_URL
The output should look as follows, the prediction result is under outputs/data:
{
"model_name": "mlflow-sklearn-mlflow-e2e",
"model_version": null,
"id": "44d5d037-052b-49b6-aace-1c5346a35004",
"parameters": null,
"outputs": [
{
"name": "predict",
"shape": [1],
"datatype": "FP32",
"parameters": null,
"data": [6.486344809506676]
}
]
}If you find a problem or have a suggestion for an enhancement, use the https://github.com/fuseml/fuseml-core/issues.
design/ - contains specification consumed by Goa out of which REST API server and cli code are generated (HTTP and gRPC)
api.go - defines the http server and a list of services that the server will hostrunnable.go - definition of the runnable serviceopenapi.go - definition fo the openapi service, which exposes a HTTP file server endpoint serving the generated OpenAPI specificationgen/ - contains the boilerplate code generated by Goa (output of $ goa gen github.com/fuseml/fuseml-core/design)
runnable/ - houses the transport-independent runnable service codegrpc/ - contains the protocol buffer descriptions for the runnable gRPC service as well as the server and client code which hooks up the protoc-generated gRPC server and client code along with the logic to encode and decode requests and responses. The cli subdirectory contains the CLI code to build gRPC requests from the command line.http/ - describes the HTTP transport which defines server and client code with the logic to encode and decode requests and responses and the CLI code to build HTTP requests from the command line. It also contains the Open API 2.0/3.0 specification files in both json and yaml formatscmd/ - a basic implementation of the service along with buildable server files that spins up goroutines to start a HTTP and a gRPC server and client files that can make requests to the server (output of $ goa example github.com/fuseml/fuseml-core/design)runnable.go - contains a dummy implementation of the methods described in the design (design/runnable.go) for the runnable service, the actual implementation goes here.goa gen cannot be edited. This directory is re-generated entirely from scratch each time the command is run (e.g. after the design has changed). This is by design to keep the interface between generated and non generated code clean and using standard Go constructs (i.e. function calls). The code generated by goa example however is your code. You should modify it, add tests to it etc. This command generates a starting point for the service to help bootstrap development - in particular it is NOT meant to be re-run when the design changes. Instead simply edit the files accordingly.RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4