AWS EC2
This Guide Integrates:
Install OpenTelemetry Collector
Linux
You can follow the following instructions to install the OpenTelemetry collector on your Linux machine:
sudo yum update
sudo yum -y install wget systemctl
wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.81.0/otelcol-contrib_0.81.0_linux_amd64.rpm
sudo rpm -ivh otelcol-contrib_0.81.0_linux_amd64.rpm
sudo yum update
sudo yum -y install wget systemctl
wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.81.0/otelcol-contrib_0.81.0_linux_arm64.rpm
sudo rpm -ivh otelcol-contrib_0.81.0_linux_arm64.rpm
sudo apt-get update
sudo apt-get -y install wget systemctl
wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.81.0/otelcol-contrib_0.81.0_linux_amd64.deb
sudo dpkg -i otelcol-contrib_0.81.0_linux_amd64.deb
sudo apt-get update
sudo apt-get -y install wget systemctl
wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.81.0/otelcol-contrib_0.81.0_linux_arm64.deb
sudo dpkg -i otelcol-contrib_0.81.0_linux_arm64.deb
apk update
apk add wget shadow
wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.81.0/otelcol-contrib_0.81.0_linux_amd64.apk
apk add --allow-untrusted otelcol-contrib_0.81.0_linux_amd64.apk
apk update
apk add wget shadow
wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.81.0/otelcol-contrib_0.81.0_linux_arm64.apk
apk add --allow-untrusted otelcol-contrib_0.81.0_linux_arm64.apk
For example, if the machine is running with Amazon Linux and x86_64 architecture, you can run the following commands to install the collector:
sudoyumupdate
sudoyum-yinstallwgetsystemctl
wgethttps://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.81.0/otelcol-contrib_0.81.0_linux_amd64.rpm
sudorpm-ivhotelcol-contrib_0.81.0_linux_amd64.rpmOther Platforms
For other platforms, you can follow these instructions (opens in a new tab) to install the OpenTelemetry Collector.
Configure OpenTelemetry Collector
Below is an example of a collector configuration that collects logs from a file, as well as host metrics (cpu, memory, etc.) and forwards them to HyperDX.
For setting up additional receivers, you can refer to the available OpenTelemetry receivers (opens in a new tab).
By default, the OpenTelemetry collector reads the config file from
/etc/otelcol-contrib/config.yaml, this can be customized by
following the instructions here (opens in a new tab).
# /etc/otelcol-contrib/config.yaml
receivers:
# Tail plaintext logs from a file
# Assuming log line looks like '20-07-2023 19:50:07.305 [HOST] INFO MESSAGE BODY'
filelog:
include: [/var/log/myapp/*.log]
operators:
- type:regex_parser
regex:
'^(?P<timestamp>\d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}.\d{3})
\[(?P<host>.*)\] (?P<level>[A-Z]*) (?P<message>.*)$'
timestamp:
parse_from:attributes.timestamp
layout:'%d-%m-%Y %H:%M:%S.%f'
severity:# use otel severity parser
parse_from:attributes.level
body:attributes.message# extract the actual body message
# Collect and forward metrics like CPU, memory, etc.
hostmetrics:
collection_interval:30s
scrapers:
cpu:
load:
memory:
disk:
filesystem:
network:
# # If you'd like to tail json logs instead of plaintext logs, you can use the
# # following config instead of the filelog/plaintext receiver.
# filelog/json:
# include: [/var/log/myservice/*.json]
# operators:
# - type: json_parser
# timestamp:
# parse_from: attributes.time
# layout: '%Y-%m-%d %H:%M:%S'
processors:
resourcedetection:# OPTIONAL
detectors:
- env
- system
- docker
- ec2
timeout:5s
override:false
attributes/servicename:# OPTIONAL
actions:
- key:'service.name'
action:upsert
value:<SERVICE_NAME>
batch:
exporters:
logging:
verbosity:detailed
otlphttp/hdx:
endpoint:'https://in-otel.hyperdx.io'
headers:
authorization:<YOUR_API_KEY>
compression:gzip
service:
pipelines:
metrics:
receivers: [hostmetrics]
processors: [resourcedetection,attributes/servicename,batch]
exporters: [logging,otlphttp/hdx]
logs:
receivers: [filelog]
processors: [resourcedetection,attributes/servicename,batch]
exporters: [logging,otlphttp/hdx]Once the config file is saved, run the following command to restart the collector.
sudosystemctlrestartotelcol-contribYou can also run the following command to check for any collector errors.
sudojournalctl-uotelcol-contrib-fAdditional Examples
Postgres Metrics
You can set up the OpenTelemetry collector to collect metrics from your PostgreSQL database. To do so, you can add the following configuration to your collector config file.
receivers:
# Collect postgres metrics
postgresql:
endpoint:localhost:5432
transport:tcp
username:$POSTGRESQL_USERNAME
password:$POSTGRESQL_PASSWORD
databases:
- testDB
collection_interval:10s
tls:
insecure:true
service:
pipelines:
# Make sure to add the postgresql receiver to the metrics pipeline
metrics:
receivers: [postgresql,hostmetrics]
processors: [resourcedetection,batch]
exporters: [logging,otlphttp/hdx]Troubleshooting
Logs not appearing due to insufficient file read permissions
If your log files are located at /home/ec2-user/..., the collector won't have
the necessary permissions to read them. One solution to address this issue is by
updating the systemd configuration file located at
/usr/lib/systemd/system/otelcol-contrib.service:
[Unit]
Description=OpenTelemetryCollectorContrib
After=network.target
[Service]
EnvironmentFile=/etc/otelcol-contrib/otelcol-contrib.conf
ExecStart=/usr/bin/otelcol-contrib $OTELCOL_OPTIONS
KillMode=mixed
Restart=on-failure
Type=simple
User=ec2-user
Group=ec2-user
[Install]
WantedBy=multi-user.target