Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://developer.hashicorp.com/terraform/enterprise/replicated/monitoring/logging below:

Log Forwarding - Infrastructure Administration - Terraform Enterprise | Terraform

Terraform Enterprise supports forwarding its logs to one or more external destinations, a process called log forwarding. Log forwarding provides increased observability, assistance complying with log retention requirements, and information during troubleshooting.

Log forwarding requires:

• Terraform Enterprise running on an instance using systemd-journald. Execute systemctl status systemd-journald to check if the systemd-journald service is started and enabled.
• A version of Docker that supports the journald logging driver. Execute docker info --format '{{.Plugins.Log}}' to check if the journald plugin is listed.
• Network connectivity between Terraform Enterprise and the external destination(s) where logs should be forwarded.

Log forwarding is disabled by default. To enable log forwarding, set the log_forwarding_enabled Terraform Enterprise application setting to the value 1.

tfe-admin app-config -k log_forwarding_enabled -v 1

When log forwarding is enabled, the Terraform Enterprise application settings show the following for log_forwarding_enabled:

    "log_forwarding_enabled": {
        "value": "1"
    },

Note: Automated license utilization reporting (which securely sends HashiCorp the minimum data required to validate license utilization) is on by default.

The log_forwarding_config Terraform Enterprise application setting must contain valid Fluent Bit [OUTPUT] configuration specifying supported external destination(s) where Terraform Enterprise should forward logs. The default configuration does not forward any logs.

Since the Terraform Enterprise application settings are stored as JSON strings, we recommend first creating a fluent-bit.conf file with the valid Fluent Bit [OUTPUT] configuration and then using that file to configure the log_forwarding_config application setting. This method ensures that the configuration is stored in the application settings exactly how it appears in the fluent-bit.conf file.

For a Standalone installation of Terraform Enterprise:

replicatedctl app-config set log_forwarding_config --value "$(cat fluent-bit.conf)"

For an Active/Active installation of Terraform Enterprise:

tfe-admin app-config -k log_forwarding_config -v "$(cat fluent-bit.conf)"

Once configured, the Terraform Enterprise application settings show the log_forwarding_config setting in escaped JSON string format:

    "log_forwarding_config": {
        "value": "# Match all logs and do not forward them anywhere.\n[OUTPUT]\n    Name null\n    Match *\n"
    },

That escaped JSON string renders to the following:

# Match all logs and do not forward them anywhere.
[OUTPUT]
    Name null
    Match *

To forward logs to multiple external destinations, use multiple [OUTPUT] directives.

# Forward all logs to Datadog.
[OUTPUT]
    Name datadog
    Match *
    ...

# Forward all logs to Fluent Bit or Fluentd.
[OUTPUT]
    Name forward
    Match *
    ...

Note: Do not use an [OUTPUT] directive with the stdout Fluent Bit output plugin. Doing this creates a loop that continuously emits logs!

Once log forwarding is enabled and configured, you need to restart Terraform Enterprise for the changes to take effect. Learn how to restart Terraform Enterprise.

You can only forward logs to one of the supported external destinations below. Each supported external destination contains example configuration for convenience.

Sending to Amazon CloudWatch is only supported when Terraform Enterprise is located within AWS due to how Fluent Bit reads AWS credentials.

This example configuration forwards all logs to Amazon CloudWatch. Refer to the cloudwatch_logs Fluent Bit output plugin documentation for more information.

[OUTPUT]
    Name               cloudwatch_logs
    Match              *
    region             us-east-1
    log_group_name     example-log-group
    log_stream_name    example-log-stream
    auto_create_group  On

Note: In Terraform Enterprise installations using AWS external services, Fluent Bit will have access to the same AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables that are used for object storage.

Sending to Amazon S3 is only supported when Terraform Enterprise is located within AWS due to how Fluent Bit reads AWS credentials.

This example configuration forwards all logs to Amazon S3. Refer to the s3 Fluent Bit output plugin documentation for more information.

[OUTPUT]
    Name                          s3
    Match                         *
    bucket                        example-bucket
    region                        us-east-1
    total_file_size               250M
    s3_key_format                 /$TAG/%Y/%m/%d/%H/%M/%S/$UUID.gz
    s3_key_format_tag_delimiters  .-

Note: In Terraform Enterprise installations using AWS external services, Fluent Bit will have access to the same AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables that are used for object storage.

This example configuration forwards all logs to Azure Blob Storage. Refer to the azure_blob Fluent Bit output plugin documentation for more information.

[OUTPUT]
    name                   azure_blob
    match                  *
    account_name           example-account-name
    shared_key             example-access-key
    path                   logs
    container_name         example-container-name
    auto_create_container  on
    tls                    on

This example configuration forwards all logs to Azure Log Analytics. Refer to the azure Fluent Bit output plugin documentation for more information.

[OUTPUT]
    name         azure
    match        *
    Customer_ID  example-log-analytics-workspace-id
    Shared_Key   example-access-key

This example configuration forwards all logs to Datadog. Refer to the datadog Fluent Bit output plugin documentation for more information.

[OUTPUT]
    Name        datadog
    Match       *
    Host        http-intake.logs.datadoghq.com
    TLS         on
    compress    gzip
    apikey      example-api-key
    dd_service  terraform_enterprise
    dd_source   docker
    dd_tags     environment:development,owner:engineering

This example configuration forwards all logs to a listening Fluent Bit or Fluentd instance. Refer to the forward Fluent Bit output plugin documentation for more information.

[OUTPUT]
    Name   forward
    Match  *
    Host   fluent.example.com
    Port   24224

Sending to Google Cloud Platform Cloud Logging is only supported when Terraform Enterprise is located within GCP due to how Fluent Bit reads GCP credentials.

This example configuration forwards all logs to Google Cloud Platform Cloud Logging (formerly known as Stackdriver). Refer to the stackdriver Fluent Bit output plugin documentation for more information.

[OUTPUT]
    Name       stackdriver
    Match      *
    location   us-east1
    namespace  terraform_enterprise
    node_id    example-hostname
    resource   generic_node

Note: In Terraform Enterprise installations using GCP external services, Fluent Bit will have access to the GOOGLE_SERVICE_CREDENTIALS environment variable that points to a file containing the same GCP Service Account JSON credentials that are used for object storage.

This example configuration forwards all logs to Splunk Enterprise via the HTTP Event Collector (HEC) interface. Refer to the splunk Fluent Bit output plugin documentation for more information.

[OUTPUT]
    Name          splunk
    Match         *
    Host          example-splunk-hec-endpoint
    Port          8088
    Splunk_Token  example-splunk-token

This example configuration forwards all logs to a Syslog-compatible endpoint. Refer to the syslog Fluent Bit output plugin documentation for more information.

[OUTPUT]
    Name                 syslog
    Match                *
    host                 example-syslog-host
    port                 514
    mode                 tcp
    syslog_message_key   log
    syslog_severity_key  PRIORITY
    syslog_hostname_key  _HOSTNAME
    syslog_appname_key   SYSLOG_IDENTIFIER
    syslog_procid_key    _PID

Terraform Enterprise emits its audit logs along with its application logs. Currently, log forwarding can forward either all Terraform Enterprise logs or no logs at all. To distinguish audit logs from application logs, audit log entries contain the string [Audit Log].

Here's an example audit log entry formatted for readability:

2021-08-31 04:58:30 [INFO] [7a233ad1-c50c-4737-a925-3be901e55fcb] [Audit Log]
{
  "resource":"run",
  "action":"create",
  "resource_id":"run-nL77p69bsesoF3RK",
  "organization":"example-org",
  "organization_id":"org-pveSPvxocni226Fn",
  "actor":"example-user",
  "timestamp":"2021-08-31T04:58:30Z",
  "actor_ip":"19.115.231.192"
}

If you have a requirement to split audit logs from application logs, we recommend forwarding all Terraform Enterprise logs to a log aggregation system, filtering the audit logs based on the [Audit Log] string, and forwarding just the audit logs to the desired destination.

Log forwarding uses the journald Docker logging driver to send Terraform Enterprise logs to systemd-journald. This can cause increased disk utilization for the /var/log/journal path.

To limit disk utilization, configure the SystemMaxFileSize and SystemMaxFiles settings within /etc/systemd/journald.conf.

The following configuration tells systemd-journald to use up to 7GB of disk space by limiting the size a log file to 1024MB and keeping up to 7 files at any given time:

[Journal]
SystemMaxFileSize=1024M
SystemMaxFiles=7

To apply these changes, restart systemd-journald:

sudo systemctl restart systemd-journald

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4