diff --git a/solutions/observability/get-started/opentelemetry/custom-metrics-quickstart.md b/solutions/observability/get-started/opentelemetry/custom-metrics-quickstart.md new file mode 100644 index 0000000000..ecbe32a1f0 --- /dev/null +++ b/solutions/observability/get-started/opentelemetry/custom-metrics-quickstart.md @@ -0,0 +1,158 @@ +--- +navigation_title: Ingest custom metrics with EDOT +description: Learn how to send custom metrics to Elastic using EDOT and OTLP. This lightweight quickstart covers the minimal setup to ingest and validate metrics in Elastic Observability. +applies_to: + stack: + serverless: + observability: + product: + edot_collector: ga +products: + - id: cloud-serverless + - id: cloud-hosted + - id: observability + - id: edot-collector +--- + +# Quickstart: Ingest custom metrics with EDOT + +Use this quickstart to send custom metrics to Elastic using the Elastic Distribution of the OpenTelemetry Collector (EDOT). + +You’ll install a lightweight EDOT Collector, configure a minimal Open Telemetry Protocol (OTLP) metrics pipeline, and verify the data in {{product.observability}}. + +## Prerequisites + +- An Elastic deployment ({{serverless-short}}, {{ech}}, or self-managed) +- An {{observability}} project {{kib}} instance +- Permissions to create API keys +- A system to run the EDOT Collector (Docker, host, or VM) +- Optional: An application that emits OpenTelemetry metrics + +:::::{stepper} + +::::{step} Create an Elastic API key + +In your {{product.observability}} deployment: + +1. Go to **{{manage-app}}** > **{{stack-manage-app}}** > **API keys**. +2. Create a new API key and copy the value. +3. Note your deployment's OTLP ingest endpoint. +:::: + +::::{step} Run the EDOT Collector with a minimal metrics pipeline + +Update the `collector-config.yaml` file with the following Collector configuration to receive OTLP metrics and export them to Elastic: + +```yaml +receivers: + otlp: + protocols: + http: + grpc: + +processors: + batch: {} + +exporters: + otlphttp: + endpoint: "" + headers: + Authorization: "ApiKey " + +service: + pipelines: + metrics: + receivers: [otlp] + processors: [batch] + exporters: [otlphttp] +``` + +Run the configuration, for example with Docker: + +```bash +docker run --rm \ + -v $(pwd)/collector-config.yaml:/etc/otel/config.yaml \ + -p 4317:4317 -p 4318:4318 \ + docker.elastic.co/observability/otel-collector:latest +``` +:::: + +::::{step} Optional: Port conflict handling + +If you encounter a port conflict error like: + +```bash +bind: address already in use +``` + +Add the following to the `service` section: + +```yaml +service: + telemetry: + metrics: + address: localhost:8889 # Using different port if 8888 is already in use +``` + +You can also verify if the Collector is listening on the correct ports: + +```bash +lsof -i :4318 -i :4317 +``` +:::: + +::::{step} Send a custom metric + +In this Python example, you use an application that emits OTLP metrics. For other languages, refer to the [contrib OpenTelemetry documentation](https://opentelemetry.io/docs/getting-started/dev/). + +```python +from opentelemetry import metrics +from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter +from opentelemetry.sdk.metrics import MeterProvider +from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader + +exporter = OTLPMetricExporter(endpoint="http://localhost:4318/v1/metrics") +reader = PeriodicExportingMetricReader(exporter) +provider = MeterProvider(metric_readers=[reader]) +metrics.set_meter_provider(provider) + +meter = metrics.get_meter("custom-app") + +temperature = meter.create_observable_gauge( + "custom.temperature", + callbacks=[lambda options: [metrics.Observation(26.7)]], +) + +input("Sending metrics periodically... press Enter to stop") +``` +:::: + +::::{step} Verify metrics in {{product.observability}} + +In {{kib}}: + +1. Go to **Infrastructure** > **Metrics Explorer**. +2. Search for `custom.temperature`. +3. Visualize or aggregate the metric data. +:::: + +::::: + +## Explore your metrics + +You've successfully set up a minimal OTLP metrics pipeline with the EDOT Collector. Your custom metrics are flowing into {{product.observability}} and can be visualized in {{kib}}. + +Now you can: + +- Use **Metrics Explorer** to create custom visualizations and dashboards +- Set up alerts based on your custom metrics +- Aggregate and analyze metric trends over time + +## Extend your setup + +You can expand your metrics collection setup in several ways: + +- Add more receivers to collect additional metrics +- Configure the same Collector to send logs and traces alongside metrics + +To learn more, refer to the [Elastic Distribution of the OpenTelemetry Collector documentation](elastic-agent://reference/edot-collector/index.md). \ No newline at end of file diff --git a/solutions/observability/get-started/opentelemetry/quickstart/index.md b/solutions/observability/get-started/opentelemetry/quickstart/index.md index 5706780ffc..e67da58807 100644 --- a/solutions/observability/get-started/opentelemetry/quickstart/index.md +++ b/solutions/observability/get-started/opentelemetry/quickstart/index.md @@ -15,28 +15,29 @@ products: # OpenTelemetry quickstarts -Learn how to set up the Elastic Distributions for OpenTelemetry (EDOT) to monitor Kubernetes, applications, and hosts. +Learn how to set up the Elastic Distributions for OpenTelemetry (EDOT) to monitor {{k8s}}, applications, and hosts. ## Add data from the UI -You can quickly add data from hosts, Kubernetes, applications, and cloud services from the Observability UI. +You can quickly add data from hosts, {{k8s}}, applications, and cloud services from the {{observability}} UI. -1. Open Elastic Observability. +1. Open {{product.observability}}. 2. Go to **Add data**. 3. Select what you want to monitor. 4. Follow the instructions. ## Manual installation guides -The guides cover how to install the EDOT Collector, turn on auto-instrumentation, and configure data collection for metrics, logs, and traces in Elastic Observability. +These guides cover how to install the EDOT Collector, turn on auto-instrumentation, and configure data collection for metrics, logs, and traces in {{product.observability}}. Select a guide based on the environment of your target system and your Elastic deployment model. -| Deployment Model | Kubernetes | Docker | Hosts or VMs | +| Deployment Model | {{k8s}} | Docker | Hosts or VMs | |-------------------------|-----------------------------------------|-----------------------------------------|---------------------------------------| -| Self-managed Elastic Stack | [Kubernetes on self-managed](/solutions/observability/get-started/opentelemetry/quickstart/self-managed/k8s.md) | [Docker on self-managed](/solutions/observability/get-started/opentelemetry/quickstart/self-managed/docker.md) | [Hosts or VMs on self-managed](/solutions/observability/get-started/opentelemetry/quickstart/self-managed/hosts_vms.md) | -| {{serverless-full}} | [Kubernetes on serverless](/solutions/observability/get-started/opentelemetry/quickstart/serverless/k8s.md) | [Docker on serverless](/solutions/observability/get-started/opentelemetry/quickstart/serverless/docker.md) | [Hosts or VMs on serverless](/solutions/observability/get-started/opentelemetry/quickstart/serverless/hosts_vms.md) | -| {{ech}} | [Kubernetes on hosted](/solutions/observability/get-started/opentelemetry/quickstart/ech/k8s.md) | [Docker on hosted](/solutions/observability/get-started/opentelemetry/quickstart/ech/docker.md) | [Hosts or VMs on hosted](/solutions/observability/get-started/opentelemetry/quickstart/ech/hosts_vms.md) | +| {{product.self}} Stack | [{{k8s}} on self-managed](/solutions/observability/get-started/opentelemetry/quickstart/self-managed/k8s.md) | [Docker on self-managed](/solutions/observability/get-started/opentelemetry/quickstart/self-managed/docker.md) | [Hosts or VMs on self-managed](/solutions/observability/get-started/opentelemetry/quickstart/self-managed/hosts_vms.md) | +| {{serverless-full}} | [{{k8s}} on serverless](/solutions/observability/get-started/opentelemetry/quickstart/serverless/k8s.md) | [Docker on serverless](/solutions/observability/get-started/opentelemetry/quickstart/serverless/docker.md) | [Hosts or VMs on serverless](/solutions/observability/get-started/opentelemetry/quickstart/serverless/hosts_vms.md) | +| {{ech}} | [{{k8s}} on hosted](/solutions/observability/get-started/opentelemetry/quickstart/ech/k8s.md) | [Docker on hosted](/solutions/observability/get-started/opentelemetry/quickstart/ech/docker.md) | [Hosts or VMs on hosted](/solutions/observability/get-started/opentelemetry/quickstart/ech/hosts_vms.md) | +| Multiple | [Ingest custom metrics with EDOT](/solutions/observability/get-started/opentelemetry/custom-metrics-quickstart.md) | [Ingest custom metrics with EDOT](/solutions/observability/get-started/opentelemetry/custom-metrics-quickstart.md) | [Ingest custom metrics with EDOT](/solutions/observability/get-started/opentelemetry/custom-metrics-quickstart.md) | ## Troubleshooting diff --git a/solutions/toc.yml b/solutions/toc.yml index 82f9883d84..aea83352fb 100644 --- a/solutions/toc.yml +++ b/solutions/toc.yml @@ -116,6 +116,7 @@ toc: - file: observability/get-started/quickstart-collect-data-with-aws-firehose.md - file: observability/get-started/opentelemetry/quickstart/index.md children: + - file: observability/get-started/opentelemetry/custom-metrics-quickstart.md - file: observability/get-started/opentelemetry/quickstart/self-managed/index.md children: - file: observability/get-started/opentelemetry/quickstart/self-managed/k8s.md