Databricks Apps Telemetry Reaches GA with Native OpenTelemetry Pipeline into Unity Catalog

Databricks has announced the general availability of Databricks Apps telemetry. The capability captures runtime traces, logs, and metrics from custom applications deployed on the platform and persists them directly into Unity Catalog tables using the OpenTelemetry (OTel) protocol. Once enabled, Databricks automatically instruments system-level logs and platform usage events, while allowing developers to emit fine-grained custom spans, metrics, and business attributes using language-native OpenTelemetry SDKs. Historically, data and ML applications hosted on managed platform runtimes lived in an observability blind spot. Engineering teams either had to run out-of-band daemon collectors—often colliding with serverless sandbox constraints—or sacrifice detailed distributed tracing entirely. By adopting OTel as the native ingestion and persistence format for application telemetry, Databricks eliminates proprietary monitoring lock-in while ensuring telemetry data inherits the exact same access control, auditing, and lineage policies established across Unity Catalog. This release aligns with the broader industry transition toward vendor-neutral observability standardization, where major cloud and data platforms integrate OpenTelemetry directly at the infrastructure runtime layer. Rather than treating telemetry as isolated operational exhaust piped exclusively to specialized Application Performance Monitoring (APM) tools, modern architectures increasingly store raw OTel-formatted traces and metrics directly inside analytical lakehouse environments. This allows data practitioners to run SQL-based evaluations, correlate pipeline errors with upstream app requests, and diagnose agentic AI application behaviors using identical data tools. In practice, teams building internal data portals, Streamlit dashboards, and AI agents on Databricks Apps can now configure automated OTel tracing with minimal code changes. Practitioners should adopt standard OpenTelemetry semantic conventions for custom instrumentation—especially when tracing LLM tool calls and database queries—to ensure uniform analysis across Unity Catalog tables. Additionally, platform engineers should define proper data retention and table partitioning policies on the underlying Unity Catalog telemetry tables to manage storage footprint and maintain query performance over time.