This directory contains a sample custom collector to read Trace2 telemetry from Git commands, transform it into OTLP, and export it to various OpenTelemetry data sinks.
You can use this demonstration collector to quickly evaluate the trace2receiver technology and before building your own telemetry solution.
The collector source code (./*.go and ./go.*) distributed in this repository was generated automatically by the OpenTelemetry builder tool (as explained below). Its purpose is to quick start your evaluation.
Also included in this repository is a set of Makefiles and scripts to build distribution/installer packages for the collector on each of the major platforms. The collector is designed to run as a long-running service daemon and these scripts will help you start building your own packages. These scripts cover the basic setup, but omit some details, like package signing.
config.yml (as explained below) to talk to your chosen telemetry data sink or cloud provider.pii.yml, filter.yml, and rulesets so that your distribution packages will be pre-configured when you deploy them to your users.The GOLANG source was generated using the OpenTelemetry builder tool and the builder-config.yml definition. This YML file defines the various receiver, pipeline, and exporter components that will be statically linked into the custom collector executable. If you want to add or remove a component, update the YML file and re-run the builder tool.
You should update the component versions listed in this YML file before running the builder tool, since the builder tool will generate the go.mod file and you don't want it to start with obsolete dependencies.
$ GO111MODULE=on go install go.opentelemetry.io/collector/cmd/builder@latest
$ ~/go/bin/builder --config ./builder-config.yml
$ go build
$ go test
The builder-config.yml file provided here contains a basic set of components to get you started. The trace2receiver component has additional details.
The go commands will automatically pull in the trace2receiver component as a dependent module in the normal GOLANG way. You may want to update the version numbers of any dependent modules at this time.
If you see a warning like the following, update all of the pinned component versions in the builder-config.yml file to match the suggested version (by the newer version of the builder tool) and try again.
% ~/go/bin/builder --config ./builder-config.yml
2023-11-16T12:30:23.932-0500 INFO internal/command.go:123 OpenTelemetry Collector Builder {"version": "dev", "date": "unknown"}
2023-11-16T12:30:23.934-0500 INFO internal/command.go:159 Using config file {"path": "./builder-config.yml"}
2023-11-16T12:30:23.934-0500 INFO builder/config.go:109 Using go {"go-executable": "/usr/local/go/bin/go"}
2023-11-16T12:30:23.934-0500 INFO builder/main.go:67 You're building a distribution with non-aligned version of the builder. Compilation may fail due to API changes. Please upgrade your builder or API {"builder-version": "0.89.0"}
2023-11-16T12:30:23.937-0500 INFO builder/main.go:91 Sources created {"path": "."}
...
In my case, I last used v0.81.0 in my builder-config.yml file and upgraded the tool to v0.89.0, so I had a mismatch.
% git diff -- builder-config.yml
diff --git a/builder-config.yml b/builder-config.yml
index b011468..3b1e23e 100644
--- a/builder-config.yml
+++ b/builder-config.yml
@@ -2,21 +2,21 @@ dist:
module: github.com/git-ecosystem/sample-trace2-otel-collector
name: sample-trace2-otel-collector
description: Custom OTEL Collector to convert and relay Git Trace2 data to OTLP
- otelcol_version: 0.81.0
+ otelcol_version: 0.89.0
output_path: .
version: 0.0.0
exporters:
- - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/azuremonitorexporter v0.81.0
+ - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/azuremonitorexporter v0.89.0
- import: go.opentelemetry.io/collector/exporter/loggingexporter
- gomod: go.opentelemetry.io/collector v0.81.0
+ gomod: go.opentelemetry.io/collector v0.89.0
- import: go.opentelemetry.io/collector/exporter/otlpexporter
- gomod: go.opentelemetry.io/collector v0.81.0
+ gomod: go.opentelemetry.io/collector v0.89.0
receivers:
- gomod: github.com/git-ecosystem/trace2receiver v0.4.3
processors:
- import: go.opentelemetry.io/collector/processor/batchprocessor
- gomod: go.opentelemetry.io/collector v0.81.0
+ gomod: go.opentelemetry.io/collector v0.89.0
Once built, your collector requires another YML file to tell it which (of the statically linked components) to actually instantiate, how to plumb data between them, and set any configuration parameters (such as cloud credentials for your data sink(s) or other pathnames).
The sample-configs/*/config.yml files provided here contains a minimal set of components to log telemetry to the console or system event viewer. They assume that:
/usr/local/sample-trace2-otel-collector/.C:/Program Files/sample-trace2-otel-collector/ and the various YML data files into C:/ProgramData/sample-trace2-otel-collector/.(The named pipe pathname is not in the ProgramData directory, since named pipes must be created on the Named Pipe File System (NPFS) and have the //./pipe/ prefix.)
The trace2receiver component has additional details and several examples.
If you want, you can run the collector in a terminal window to test your configuration. When interactive, the debug log will appear on the console. You can change the two logging verbosity settings in the config.yml file to see the OTLP being emitted.
$ ./sample-trace2-otel-collector --config <pathname-to-config.yml>
In addition to having a running the collector, listening on a Unix Domain Socket (SOCKET) or Windows Named Pipe (PIPE), you'll also need to tell Git to send telemetry to the collector using the trace2.eventtarget Git config variable (using --system or --global scope). The various installer service_start scripts show how to do this.
The provided installer/<platform>/Makefile will build a minimal installer package for your platform.
To build an installer PKG on macOS:
$ go build
$ cd ./installer/<platform>
$ make layout
$ make package
A ".pkg" will be created in ./installer/macos/_out_/_pkg_/. You can use it to install the sample collector in /usr/local/sample-trace2-otel-collector/*. The installer script will copy the files, register it with launchctl(1), and start it.
The scripts in /usr/local/sample-trace2-otel-collector/scripts/* let you stop and restart the service. Use these if you want to try different settings in your installed config.yml or other YML files (such as filter.yml).
The /usr/local/sample-trace2-otel-collector/uninstaller.sh script will stop the service and delete it.
To build an installer DEB on Linux:
$ go build
$ cd ./installer/<platform>
$ make layout
$ make package
A ".deb" will be created in ./installer/linux/_out_/_pkg_/. You can use it to install the sample collector in /usr/local/sample-trace2-otel-collector/*. The installer script will copy the files, register it with systemctl(1), and start it.
The scripts in /usr/local/sample-trace2-otel-collector/scripts/* let you stop and restart the service. Use these if you want to try different settings in your installed config.yml or other YML files (such as filter.yml).
To build a ZIP file for Windows using a Command Prompt and create BAT files. (You can use either a VS Developer Command Prompt or a plain Command Prompt.)
> go build
> cd ./installer/windows_batch_file
> build.bat
A ZIP file will be created in ./installer/windows_batch_file/_out_/ containing the executable, the YML files, and scripts to install and register/unregister the service with Control Panel. You can redistribute the ZIP file and let users (in an elevated Command Prompt) run the install.bat and register.bat scripts. You should then see the collector in the Control Panel Service Manager.
Having three scripts is not as nice as a stand-alone exe installer, but they will let you kick the tires and/or distribute the ZIP file to your users without involving a third-party installer-builder tool.
NOTE: The register.bat script will use git config --global to set some global config values to tell Git to send telemetry data to the collector. These are per-user config values, so telemetry will only be collected from the user who ran the script. If you want to collect telemetry from multiple users on a computer, you should have each user execute those Git commands. This is a limitation of using "global" scope. You might change it to use --system scope, but this causes problems if you have multiple versions of Git installed on the computer, such as the one bundled with Visual Studio vs the one installed in C:\Program Files\Git, since each version has its own notion of where the system configuration is stored. Using --global solves that problem.
If you have the Git SDK installer and are comfortable in an msys2 bash shell, you can create a ZIP file for Windows that uses bash scripts:
$ go build
$ cd ./installer/windows_bash
$ make layout
$ make package
A ZIP file will be created in ./installers/windows_bash/_out_/ containing the executable, the YML files, and scripts to install and register/unregister the service with Control Panel. The ZIP file will contain install.sh, register.sh and unregister.sh. Run these from an elevated bash terminal. You should then see the collector in the Control Panel Service Manager.
I created these scripts during development, but very few people have the Git SDK installed, so I created the above BAT file versions and will retire the bash version eventually.
Once you have the collector running as a service, you can look at its console logs in /usr/local/sample-trace2-otel-collector/logs/* on Linux and macOS. On Windows, these messages are written to the Event Viewer (under "Windows Logs > Application").
There are two settings in config.yml to control the verbosity of the console logs. You can turn them up to get more detailed messages and debug output. (They will generate a lot of data, so don't forget to turn them back down when you're finished.)
In the example config.yml that I included with this sample collector, it only writes to the console log. You'll need to add one or more (real) exporters to send the telemetry somewhere. The documentation in the trace2receiver component has several examples.
The Trace2 feature in Git will send telemetry to a SOCKET or PIPE defined in the trace2.eventtarget system- or global-level config value. The value of this pathname must match the pathname where the collector is listening.
The Git config variable is added to the Git environment using the various service_start or register.sh scripts in the installers.
The collector gets this pathname from the config.yml file.
If you're not seeing any data for your Git commands, verify that these two pathnames match. It may be helpful to use GIT_TRACE2_DST_DEBUG to verify that Git can write to the SOCKET or PIPE:
$ git config --global trace2.eventtarget "af_unix:/foo"
$ GIT_TRACE2_DST_DEBUG=1 git version
warning: trace2: could not connect to socket '/foo' for 'GIT_TRACE2_EVENT' tracing: No such file or directory
git version 2.42.0
If you're sending too much telemetry to your data sink or cloud provider, you can try the filtering built into the trace2receiver component. For example, have it drop data from uninteresting commands or repositories and let you focus on the important commands that are causing your users pain.
The trace2receiver documentation has a whole section on such filtering.
You may also want to consider adding a pipeline component (between the trace2 receiver component and your exporter component) that does some form of sampling. That is outside of the scope of my goals here.
This project is under active development, and loves contributions from the community. Check out the CONTRIBUTING guide for details on getting started.
This project is licensed under the terms of the MIT open source license. Please refer to LICENSE for the full terms.
See CODEOWNERS for a list of current project maintainers.
See SUPPORT for instructions on how to file bugs, make feature requests, or seek help.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4