diff --git a/observability-tracing-moesif/README.md b/observability-tracing-moesif/README.md new file mode 100644 index 00000000..5afcebe3 --- /dev/null +++ b/observability-tracing-moesif/README.md @@ -0,0 +1,177 @@ +# Observability Tracing Module for Moesif + +This module collects traces using [OpenTelemetry Collector](https://opentelemetry.io) and exports them to [Moesif](https://www.moesif.com). + +## Prerequisites + +- [OpenChoreo](https://github.com/openchoreo/openchoreo) must be installed with the **observability plane** enabled for this module to work. +- A Moesif account and a **Collector Application ID** for each environment from [Moesif](https://www.moesif.com/). + +## Installation + +### Create a Kubernetes Secret + +Create a Kubernetes secret containing your Moesif Collector Application IDs, with one key per environment. + +First, get the environment names and their UIDs: + +```bash +kubectl get environments -o custom-columns="NAME:.metadata.name,UID:.metadata.uid" +``` + +Use the environment **UID** as the secret key and the Moesif **Collector Application ID** as the value. + +For example, if the output is: + +``` +NAME UID +development a1b2c3d4-e5f6-7890-abcd-ef1234567890 +production f9e8d7c6-b5a4-3210-fedc-ba0987654321 +``` + +Create the secret using the UIDs as keys: + +```bash +kubectl create secret generic moesif-tracing-collector-secret \ + --from-literal=a1b2c3d4-e5f6-7890-abcd-ef1234567890="YOUR_DEV_COLLECTOR_APP_ID" \ + --from-literal=f9e8d7c6-b5a4-3210-fedc-ba0987654321="YOUR_PROD_COLLECTOR_APP_ID" \ + --namespace openchoreo-observability-plane +``` + +### (Optional) Create a Search API Secret for Built-in Dashboards + +> **Note:** This step is **optional** and only required if you want to populate the built-in dashboards with trace data from Moesif. +> The Management API key generation is a **paid feature** of Moesif. Configure this only if your Moesif plan supports it. + +To generate an API key in Moesif: + +1. Go to your Moesif dashboard and navigate to the **Management API Keys** section. +2. Create a new API key and select scopes under the **Analytics** section with **read** permission. +3. Create one key per environment, or use a single organization-level key. + +Create the search secret using the environment **UID** as the key and the bearer token as the value: + +```bash +kubectl create secret generic moesif-trace-search-secret \ + --from-literal=a1b2c3d4-e5f6-7890-abcd-ef1234567890="YOUR_DEV_MANAGEMENT_API_BEARER_TOKEN" \ + --from-literal=f9e8d7c6-b5a4-3210-fedc-ba0987654321="YOUR_PROD_MANAGEMENT_API_BEARER_TOKEN" \ + --namespace openchoreo-observability-plane +``` + +### Install the Helm Chart + +```bash +helm upgrade --install observability-tracing-moesif \ + oci://ghcr.io/openchoreo/helm-charts/observability-tracing-moesif \ + --create-namespace \ + --namespace openchoreo-observability-plane \ + --version 0.1.0 +``` + + +### Configuration Options + +For easier configuration management, create a `values.yaml` file: + +```yaml +# values.yaml + +moesif: + # List of environments to collect traces from. + # Get name and id by running: kubectl get environments -o custom-columns="NAME:.metadata.name,UID:.metadata.uid" + environments: + - name: development + id: a1b2c3d4-e5f6-7890-abcd-ef1234567890 + - name: production + id: f9e8d7c6-b5a4-3210-fedc-ba0987654321 + + # Moesif adapter configuration + adapter: + enabled: true + +opentelemetryCollectorCustomizations: + debug: + enabled: false # Enable debug exporter for troubleshooting + + tailSampling: + enabled: true # Enable tail-based sampling + decisionWait: 10s + numTraces: 100 + expectedNewTracesPerSec: 10 + decisionCache: + sampledCacheSize: 10000 + nonSampledCacheSize: 1000 + spansPerSecond: 10 +``` + +Then install with: + +```bash +helm upgrade --install observability-tracing-moesif \ + oci://ghcr.io/openchoreo/helm-charts/observability-tracing-moesif \ + --create-namespace \ + --namespace openchoreo-observability-plane \ + --version 0.1.0 \ + -f moesif-tracing-values.yaml +``` + +#### Configuration Parameters + +| Parameter | Description | Default | +|-----------|-------------|---------| +| `moesif.environments` | List of environments with `name` and `id` (UID) to collect traces from | `[]` | +| `moesif.endpoint` | (Optional) Moesif API endpoint URL | `https://api.moesif.net` | +| `moesif.adapter.enabled` | Enable the Moesif adapter for trace search | `true` | +| `moesif.adapter.searchEndpoint` | Moesif search API endpoint | `https://api.moesif.com` | +| `moesif.adapter.searchSecretName` | Secret name for Moesif search credentials | `moesif-search-credentials` | +| `opentelemetryCollectorCustomizations.debug.enabled` | Enable debug exporter for troubleshooting | `false` | +| `opentelemetryCollectorCustomizations.tailSampling.enabled` | Enable tail-based sampling | `true` | +| `opentelemetryCollectorCustomizations.tailSampling.decisionWait` | Wait time before making a sampling decision | `10s` | +| `opentelemetryCollectorCustomizations.tailSampling.numTraces` | Number of traces kept in memory | `100` | +| `opentelemetryCollectorCustomizations.tailSampling.expectedNewTracesPerSec` | Expected number of new traces per second | `10` | +| `opentelemetryCollectorCustomizations.tailSampling.decisionCache.sampledCacheSize` | Size of sampled decision cache | `10000` | +| `opentelemetryCollectorCustomizations.tailSampling.decisionCache.nonSampledCacheSize` | Size of non-sampled decision cache | `1000` | +| `opentelemetryCollectorCustomizations.tailSampling.spansPerSecond` | Rate limit for spans per second | `10` | + +## How It Works + +This module deploys an **OpenTelemetry Collector** that: + +1. Receives OTLP traces (gRPC on port `4317`, HTTP on port `4318`) from instrumented workloads. +2. Enriches spans with Kubernetes metadata (pod name, deployment, namespace, etc.) using the `k8sattributes` processor. +3. Routes traces to the correct Moesif application based on the environment UID. +4. Exports traces to Moesif using the Moesif Collector Application ID stored in the `moesif-tracing-collector-secret` Kubernetes secret. + +## Troubleshooting + +### Check OpenTelemetry Collector logs + +```bash +kubectl -n openchoreo-observability-plane logs -f deploy/moesif-tracing-collector +``` + +### Verify the secret exists + +```bash +kubectl -n openchoreo-observability-plane get secret moesif-tracing-collector-secret +``` + +### Check pod health + +```bash +kubectl -n openchoreo-observability-plane get pods +``` + +## Uninstalling + +```bash +helm uninstall observability-tracing-moesif \ + --namespace openchoreo-observability-plane +``` + +To also remove the secret: + +```bash +kubectl delete secret moesif-tracing-collector-secret \ + --namespace openchoreo-observability-plane +``` diff --git a/observability-tracing-moesif/VERSION b/observability-tracing-moesif/VERSION new file mode 100644 index 00000000..6e8bf73a --- /dev/null +++ b/observability-tracing-moesif/VERSION @@ -0,0 +1 @@ +0.1.0 diff --git a/observability-tracing-moesif/adaptor-api/.dockerignore b/observability-tracing-moesif/adaptor-api/.dockerignore new file mode 100644 index 00000000..c9665745 --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/.dockerignore @@ -0,0 +1,29 @@ +# Git +.git +.gitignore + +# Documentation +README.md +*.md + +# Kubernetes manifests +k8s/ +helm/ + +# IDE +.vscode +.idea +*.swp +*.swo +*~ + +# Test files +*_test.go + +# Build artifacts +bin/ +*.exe +*.exe~ +*.dll +*.so +*.dylib diff --git a/observability-tracing-moesif/adaptor-api/Dockerfile b/observability-tracing-moesif/adaptor-api/Dockerfile new file mode 100644 index 00000000..8f1bdf00 --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/Dockerfile @@ -0,0 +1,24 @@ +# Copyright 2026 The OpenChoreo Authors +# SPDX-License-Identifier: Apache-2.0 + +FROM golang:1.26-alpine AS builder + +WORKDIR /app +COPY go.mod go.sum* ./ +RUN go mod download +COPY . . +RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main ./cmd + +FROM alpine:latest + +RUN apk --no-cache add ca-certificates && \ + addgroup -g 10500 appuser && \ + adduser -D -u 10500 -G appuser appuser + +WORKDIR /home/appuser +COPY --from=builder --chown=appuser:appuser --chmod=0550 /app/main . + +USER appuser +EXPOSE 9100 + +CMD ["./main"] diff --git a/observability-tracing-moesif/adaptor-api/Makefile b/observability-tracing-moesif/adaptor-api/Makefile new file mode 100644 index 00000000..71d32978 --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/Makefile @@ -0,0 +1,25 @@ +OAPI_CODEGEN_VERSION ?= v2.7.0 +OAS := https://raw.githubusercontent.com/openchoreo/openchoreo/main/openapi/observability-tracing-adapter-api.yaml +MODULE_NAME := $(notdir $(CURDIR)) + +.PHONY: oapi-codegen-install generate build run tidy unit-test + +oapi-codegen-install: + go install github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen@$(OAPI_CODEGEN_VERSION) + +## generate: regenerate server stubs — run after editing the OAS file +generate: oapi-codegen-install + $(shell go env GOPATH)/bin/oapi-codegen -generate types,gin -package gen -o gen/server.gen.go $(OAS) + +build: + go build -o bin/adaptor-api ./cmd + +run: + go run ./cmd + +tidy: + go mod tidy + +unit-test: + go test -coverprofile=coverage.out ./... + mv coverage.out ../$(MODULE_NAME)-coverage.out diff --git a/observability-tracing-moesif/adaptor-api/cmd/main.go b/observability-tracing-moesif/adaptor-api/cmd/main.go new file mode 100644 index 00000000..3c091fe8 --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/cmd/main.go @@ -0,0 +1,36 @@ +// Copyright 2026 The OpenChoreo Authors +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "log/slog" + "os" + + "github.com/gin-gonic/gin" + "github.com/openchoreo/observability-tracing-moesif-cloud/adaptor-api/gen" + "github.com/openchoreo/observability-tracing-moesif-cloud/adaptor-api/internal/config" + "github.com/openchoreo/observability-tracing-moesif-cloud/adaptor-api/internal/handler" + "github.com/openchoreo/observability-tracing-moesif-cloud/adaptor-api/internal/search" +) + +func main() { + logger := slog.New(slog.NewJSONHandler(os.Stdout, nil)) + + cfg, err := config.LoadConfig() + if err != nil { + logger.Error("failed to load configuration", slog.Any("error", err)) + os.Exit(1) + } + + searchClient := search.NewClient(cfg, logger) + + r := gin.Default() + gen.RegisterHandlers(r, handler.New(searchClient)) + + logger.Info("starting server", slog.String("port", cfg.ServerPort)) + if err := r.Run(":" + cfg.ServerPort); err != nil { + logger.Error("server exited", slog.Any("error", err)) + os.Exit(1) + } +} diff --git a/observability-tracing-moesif/adaptor-api/gen/server.gen.go b/observability-tracing-moesif/adaptor-api/gen/server.gen.go new file mode 100644 index 00000000..5aa98fc1 --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/gen/server.gen.go @@ -0,0 +1,393 @@ +// Package gen provides primitives to interact with the openapi HTTP API. +// +// Code generated by github.com/oapi-codegen/oapi-codegen/v2 version v2.7.0 DO NOT EDIT. +package gen + +import ( + "fmt" + "net/http" + "time" + + "github.com/gin-gonic/gin" + "github.com/oapi-codegen/runtime" +) + +// Defines values for ErrorResponseTitle. +const ( + BadRequest ErrorResponseTitle = "badRequest" + Forbidden ErrorResponseTitle = "forbidden" + InternalServerError ErrorResponseTitle = "internalServerError" + Unauthorized ErrorResponseTitle = "unauthorized" +) + +// Valid indicates whether the value is a known member of the ErrorResponseTitle enum. +func (e ErrorResponseTitle) Valid() bool { + switch e { + case BadRequest: + return true + case Forbidden: + return true + case InternalServerError: + return true + case Unauthorized: + return true + default: + return false + } +} + +// Defines values for SpanStatusCode. +const ( + Error SpanStatusCode = "error" + Ok SpanStatusCode = "ok" + Unset SpanStatusCode = "unset" +) + +// Valid indicates whether the value is a known member of the SpanStatusCode enum. +func (e SpanStatusCode) Valid() bool { + switch e { + case Error: + return true + case Ok: + return true + case Unset: + return true + default: + return false + } +} + +// Defines values for TracesQueryRequestSortOrder. +const ( + Asc TracesQueryRequestSortOrder = "asc" + Desc TracesQueryRequestSortOrder = "desc" +) + +// Valid indicates whether the value is a known member of the TracesQueryRequestSortOrder enum. +func (e TracesQueryRequestSortOrder) Valid() bool { + switch e { + case Asc: + return true + case Desc: + return true + default: + return false + } +} + +// ComponentSearchScope defines model for ComponentSearchScope. +type ComponentSearchScope struct { + Component *string `json:"component,omitempty"` + Environment *string `json:"environment,omitempty"` + Namespace string `json:"namespace"` + Project *string `json:"project,omitempty"` +} + +// ErrorResponse defines model for ErrorResponse. +type ErrorResponse struct { + // Detail The error message + Detail *string `json:"detail,omitempty"` + + // ErrorCode The error code from observer service + ErrorCode *string `json:"errorCode,omitempty"` + + // Title The error message + Title *ErrorResponseTitle `json:"title,omitempty"` +} + +// ErrorResponseTitle The error message +type ErrorResponseTitle string + +// SpanStatus Execution status of the span, following the OpenTelemetry span Status model. +type SpanStatus struct { + // Code The status code of the span. One of "ok", "error", or "unset". + Code *SpanStatusCode `json:"code,omitempty"` + + // Message Developer-facing human-readable status description. Typically set only when code is "error". + Message *string `json:"message,omitempty"` +} + +// SpanStatusCode The status code of the span. One of "ok", "error", or "unset". +type SpanStatusCode string + +// TraceSpanDetailsResponse defines model for TraceSpanDetailsResponse. +type TraceSpanDetailsResponse struct { + // Attributes The span attributes as a key/value map + Attributes *map[string]interface{} `json:"attributes,omitempty"` + + // DurationNs The duration of the span in nanoseconds + DurationNs *int64 `json:"durationNs,omitempty"` + + // EndTime The end time of the span + EndTime *time.Time `json:"endTime,omitempty"` + + // ParentSpanId The parent span ID + ParentSpanId *string `json:"parentSpanId,omitempty"` + + // ResourceAttributes The resource attributes as a key/value map + ResourceAttributes *map[string]interface{} `json:"resourceAttributes,omitempty"` + + // SpanId The span ID + SpanId *string `json:"spanId,omitempty"` + + // SpanKind The kind of the span + SpanKind *string `json:"spanKind,omitempty"` + + // SpanName The name of the span + SpanName *string `json:"spanName,omitempty"` + + // StartTime The start time of the span + StartTime *time.Time `json:"startTime,omitempty"` + + // Status Execution status of the span, following the OpenTelemetry span Status model. + Status *SpanStatus `json:"status,omitempty"` +} + +// TraceSpansListResponse defines model for TraceSpansListResponse. +type TraceSpansListResponse struct { + // Spans The list of spans + Spans *[]struct { + // Attributes The span attributes as a key/value map + Attributes *map[string]interface{} `json:"attributes,omitempty"` + + // DurationNs The duration of the span in nanoseconds + DurationNs *int64 `json:"durationNs,omitempty"` + + // EndTime The end time of the span + EndTime *time.Time `json:"endTime,omitempty"` + + // ParentSpanId The parent span ID + ParentSpanId *string `json:"parentSpanId,omitempty"` + + // ResourceAttributes The resource attributes as a key/value map + ResourceAttributes *map[string]interface{} `json:"resourceAttributes,omitempty"` + + // SpanId The span ID + SpanId *string `json:"spanId,omitempty"` + + // SpanKind The kind of the span + SpanKind *string `json:"spanKind,omitempty"` + + // SpanName The name of the span + SpanName *string `json:"spanName,omitempty"` + + // StartTime The start time of the span + StartTime *time.Time `json:"startTime,omitempty"` + + // Status Execution status of the span, following the OpenTelemetry span Status model. + Status *SpanStatus `json:"status,omitempty"` + } `json:"spans,omitempty"` + + // TookMs The time taken to query the spans in milliseconds + TookMs *int `json:"tookMs,omitempty"` + + // Total The total number of matching spans, capped at 1000 + Total *int `json:"total,omitempty"` +} + +// TracesListResponse defines model for TracesListResponse. +type TracesListResponse struct { + // TookMs The time taken to query the traces in milliseconds + TookMs *int `json:"tookMs,omitempty"` + + // Total The total number of matching traces, capped at 1000 + Total *int `json:"total,omitempty"` + + // Traces The list of traces + Traces *[]struct { + // DurationNs The duration of the trace in nanoseconds + DurationNs *int64 `json:"durationNs,omitempty"` + + // EndTime The end time of the trace + EndTime *time.Time `json:"endTime,omitempty"` + + // HasErrors Whether any span in the trace has an error status. + HasErrors *bool `json:"hasErrors,omitempty"` + RootSpanId *string `json:"rootSpanId,omitempty"` + RootSpanKind *string `json:"rootSpanKind,omitempty"` + RootSpanName *string `json:"rootSpanName,omitempty"` + + // SpanCount The number of spans in the trace + SpanCount *int `json:"spanCount,omitempty"` + + // StartTime The start time of the trace + StartTime *time.Time `json:"startTime,omitempty"` + + // TraceId The trace ID + TraceId *string `json:"traceId,omitempty"` + + // TraceName The name of the trace + TraceName *string `json:"traceName,omitempty"` + } `json:"traces,omitempty"` +} + +// TracesQueryRequest defines model for TracesQueryRequest. +type TracesQueryRequest struct { + // EndTime The end time of the query + EndTime time.Time `json:"endTime"` + + // IncludeAttributes Whether to include span attributes in the response. Defaults to false. + IncludeAttributes *bool `json:"includeAttributes,omitempty"` + + // Limit The maximum number of items to return + Limit *int `json:"limit,omitempty"` + SearchScope ComponentSearchScope `json:"searchScope"` + + // SortOrder The sort order of the query + SortOrder *TracesQueryRequestSortOrder `json:"sortOrder,omitempty"` + + // StartTime The start time of the query + StartTime time.Time `json:"startTime"` +} + +// TracesQueryRequestSortOrder The sort order of the query +type TracesQueryRequestSortOrder string + +// QueryTracesJSONRequestBody defines body for QueryTraces for application/json ContentType. +type QueryTracesJSONRequestBody = TracesQueryRequest + +// QuerySpansForTraceJSONRequestBody defines body for QuerySpansForTrace for application/json ContentType. +type QuerySpansForTraceJSONRequestBody = TracesQueryRequest + +// ServerInterface represents all server handlers. +type ServerInterface interface { + // Query traces + // (POST /api/v1alpha1/traces/query) + QueryTraces(c *gin.Context) + // Query spans for a trace + // (POST /api/v1alpha1/traces/{traceId}/spans/query) + QuerySpansForTrace(c *gin.Context, traceId string) + // Get details of a span for a trace + // (GET /api/v1alpha1/traces/{traceId}/spans/{spanId}) + GetSpanDetailsForTrace(c *gin.Context, traceId string, spanId string) + // Health check + // (GET /healthz) + Health(c *gin.Context) +} + +// ServerInterfaceWrapper converts contexts to parameters. +type ServerInterfaceWrapper struct { + Handler ServerInterface + HandlerMiddlewares []MiddlewareFunc + ErrorHandler func(*gin.Context, error, int) +} + +type MiddlewareFunc func(c *gin.Context) + +// QueryTraces operation middleware +func (siw *ServerInterfaceWrapper) QueryTraces(c *gin.Context) { + + for _, middleware := range siw.HandlerMiddlewares { + middleware(c) + if c.IsAborted() { + return + } + } + + siw.Handler.QueryTraces(c) +} + +// QuerySpansForTrace operation middleware +func (siw *ServerInterfaceWrapper) QuerySpansForTrace(c *gin.Context) { + + var err error + _ = err + + // ------------- Path parameter "traceId" ------------- + var traceId string + + err = runtime.BindStyledParameterWithOptions("simple", "traceId", c.Param("traceId"), &traceId, runtime.BindStyledParameterOptions{ParamLocation: runtime.ParamLocationPath, Explode: false, Required: true, Type: "string", Format: ""}) + if err != nil { + siw.ErrorHandler(c, fmt.Errorf("Invalid format for parameter traceId: %w", err), http.StatusBadRequest) + return + } + + for _, middleware := range siw.HandlerMiddlewares { + middleware(c) + if c.IsAborted() { + return + } + } + + siw.Handler.QuerySpansForTrace(c, traceId) +} + +// GetSpanDetailsForTrace operation middleware +func (siw *ServerInterfaceWrapper) GetSpanDetailsForTrace(c *gin.Context) { + + var err error + _ = err + + // ------------- Path parameter "traceId" ------------- + var traceId string + + err = runtime.BindStyledParameterWithOptions("simple", "traceId", c.Param("traceId"), &traceId, runtime.BindStyledParameterOptions{ParamLocation: runtime.ParamLocationPath, Explode: false, Required: true, Type: "string", Format: ""}) + if err != nil { + siw.ErrorHandler(c, fmt.Errorf("Invalid format for parameter traceId: %w", err), http.StatusBadRequest) + return + } + + // ------------- Path parameter "spanId" ------------- + var spanId string + + err = runtime.BindStyledParameterWithOptions("simple", "spanId", c.Param("spanId"), &spanId, runtime.BindStyledParameterOptions{ParamLocation: runtime.ParamLocationPath, Explode: false, Required: true, Type: "string", Format: ""}) + if err != nil { + siw.ErrorHandler(c, fmt.Errorf("Invalid format for parameter spanId: %w", err), http.StatusBadRequest) + return + } + + for _, middleware := range siw.HandlerMiddlewares { + middleware(c) + if c.IsAborted() { + return + } + } + + siw.Handler.GetSpanDetailsForTrace(c, traceId, spanId) +} + +// Health operation middleware +func (siw *ServerInterfaceWrapper) Health(c *gin.Context) { + + for _, middleware := range siw.HandlerMiddlewares { + middleware(c) + if c.IsAborted() { + return + } + } + + siw.Handler.Health(c) +} + +// GinServerOptions provides options for the Gin server. +type GinServerOptions struct { + BaseURL string + Middlewares []MiddlewareFunc + ErrorHandler func(*gin.Context, error, int) +} + +// RegisterHandlers creates http.Handler with routing matching OpenAPI spec. +func RegisterHandlers(router gin.IRouter, si ServerInterface) { + RegisterHandlersWithOptions(router, si, GinServerOptions{}) +} + +// RegisterHandlersWithOptions creates http.Handler with additional options +func RegisterHandlersWithOptions(router gin.IRouter, si ServerInterface, options GinServerOptions) { + errorHandler := options.ErrorHandler + if errorHandler == nil { + errorHandler = func(c *gin.Context, err error, statusCode int) { + c.JSON(statusCode, gin.H{"msg": err.Error()}) + } + } + + wrapper := ServerInterfaceWrapper{ + Handler: si, + HandlerMiddlewares: options.Middlewares, + ErrorHandler: errorHandler, + } + + router.POST(options.BaseURL+"/api/v1alpha1/traces/query", wrapper.QueryTraces) + router.POST(options.BaseURL+"/api/v1alpha1/traces/:traceId/spans/query", wrapper.QuerySpansForTrace) + router.GET(options.BaseURL+"/api/v1alpha1/traces/:traceId/spans/:spanId", wrapper.GetSpanDetailsForTrace) + router.GET(options.BaseURL+"/healthz", wrapper.Health) +} diff --git a/observability-tracing-moesif/adaptor-api/go.mod b/observability-tracing-moesif/adaptor-api/go.mod new file mode 100644 index 00000000..98b791e0 --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/go.mod @@ -0,0 +1,39 @@ +module github.com/openchoreo/observability-tracing-moesif-cloud/adaptor-api + +go 1.26.5 + +require ( + github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect + github.com/bytedance/gopkg v0.1.3 // indirect + github.com/bytedance/sonic v1.15.0 // indirect + github.com/bytedance/sonic/loader v0.5.0 // indirect + github.com/cloudwego/base64x v0.1.6 // indirect + github.com/gabriel-vasile/mimetype v1.4.12 // indirect + github.com/gin-contrib/sse v1.1.0 // indirect + github.com/gin-gonic/gin v1.12.0 // indirect + github.com/go-playground/locales v0.14.1 // indirect + github.com/go-playground/universal-translator v0.18.1 // indirect + github.com/go-playground/validator/v10 v10.30.1 // indirect + github.com/goccy/go-json v0.10.5 // indirect + github.com/goccy/go-yaml v1.19.2 // indirect + github.com/google/uuid v1.6.0 // indirect + github.com/json-iterator/go v1.1.12 // indirect + github.com/klauspost/cpuid/v2 v2.3.0 // indirect + github.com/leodido/go-urn v1.4.0 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect + github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/oapi-codegen/runtime v1.6.0 // indirect + github.com/pelletier/go-toml/v2 v2.2.4 // indirect + github.com/quic-go/qpack v0.6.0 // indirect + github.com/quic-go/quic-go v0.59.0 // indirect + github.com/twitchyliquid64/golang-asm v0.15.1 // indirect + github.com/ugorji/go/codec v1.3.1 // indirect + go.mongodb.org/mongo-driver/v2 v2.5.0 // indirect + golang.org/x/arch v0.22.0 // indirect + golang.org/x/crypto v0.48.0 // indirect + golang.org/x/net v0.51.0 // indirect + golang.org/x/sys v0.41.0 // indirect + golang.org/x/text v0.34.0 // indirect + google.golang.org/protobuf v1.36.10 // indirect +) diff --git a/observability-tracing-moesif/adaptor-api/go.sum b/observability-tracing-moesif/adaptor-api/go.sum new file mode 100644 index 00000000..08d42b04 --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/go.sum @@ -0,0 +1,88 @@ +github.com/RaveNoX/go-jsoncommentstrip v1.0.0/go.mod h1:78ihd09MekBnJnxpICcwzCMzGrKSKYe4AqU6PDYYpjk= +github.com/apapsch/go-jsonmerge/v2 v2.0.0 h1:axGnT1gRIfimI7gJifB699GoE/oq+F2MU7Dml6nw9rQ= +github.com/apapsch/go-jsonmerge/v2 v2.0.0/go.mod h1:lvDnEdqiQrp0O42VQGgmlKpxL1AP2+08jFMw88y4klk= +github.com/bmatcuk/doublestar v1.1.1/go.mod h1:UD6OnuiIn0yFxxA2le/rnRU1G4RaI4UvFv1sNto9p6w= +github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M= +github.com/bytedance/gopkg v0.1.3/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM= +github.com/bytedance/sonic v1.15.0 h1:/PXeWFaR5ElNcVE84U0dOHjiMHQOwNIx3K4ymzh/uSE= +github.com/bytedance/sonic v1.15.0/go.mod h1:tFkWrPz0/CUCLEF4ri4UkHekCIcdnkqXw9VduqpJh0k= +github.com/bytedance/sonic/loader v0.5.0 h1:gXH3KVnatgY7loH5/TkeVyXPfESoqSBSBEiDd5VjlgE= +github.com/bytedance/sonic/loader v0.5.0/go.mod h1:AR4NYCk5DdzZizZ5djGqQ92eEhCCcdf5x77udYiSJRo= +github.com/cloudwego/base64x v0.1.6 h1:t11wG9AECkCDk5fMSoxmufanudBtJ+/HemLstXDLI2M= +github.com/cloudwego/base64x v0.1.6/go.mod h1:OFcloc187FXDaYHvrNIjxSe8ncn0OOM8gEHfghB2IPU= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/gabriel-vasile/mimetype v1.4.12 h1:e9hWvmLYvtp846tLHam2o++qitpguFiYCKbn0w9jyqw= +github.com/gabriel-vasile/mimetype v1.4.12/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s= +github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w= +github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM= +github.com/gin-gonic/gin v1.12.0 h1:b3YAbrZtnf8N//yjKeU2+MQsh2mY5htkZidOM7O0wG8= +github.com/gin-gonic/gin v1.12.0/go.mod h1:VxccKfsSllpKshkBWgVgRniFFAzFb9csfngsqANjnLc= +github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA= +github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY= +github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY= +github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY= +github.com/go-playground/validator/v10 v10.30.1 h1:f3zDSN/zOma+w6+1Wswgd9fLkdwy06ntQJp0BBvFG0w= +github.com/go-playground/validator/v10 v10.30.1/go.mod h1:oSuBIQzuJxL//3MelwSLD5hc2Tu889bF0Idm9Dg26cM= +github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4= +github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M= +github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM= +github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= +github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= +github.com/juju/gnuflag v0.0.0-20171113085948-2ce1bb71843d/go.mod h1:2PavIy+JPciBPrBUjwbNvtwB6RQlve+hkpll6QSNmOE= +github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y= +github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= +github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ= +github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= +github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/oapi-codegen/runtime v1.6.0 h1:7Xx+GlueD6nRuyKoCPzL434Jfi3BetbiJOrzCHp/VPU= +github.com/oapi-codegen/runtime v1.6.0/go.mod h1:GwV7hC2hviaMzj+ITfHVRESK5J2W/GefVwIND/bMGvU= +github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4= +github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8= +github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII= +github.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SAw= +github.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU= +github.com/spkg/bom v0.0.0-20160624110644-59b7046e48ad/go.mod h1:qLr4V1qq6nMqFKkMo8ZTx3f+BZEkzsRUY10Xsm2mwU0= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= +github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= +github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI= +github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08= +github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY= +github.com/ugorji/go/codec v1.3.1/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4= +go.mongodb.org/mongo-driver/v2 v2.5.0 h1:yXUhImUjjAInNcpTcAlPHiT7bIXhshCTL3jVBkF3xaE= +go.mongodb.org/mongo-driver/v2 v2.5.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0= +golang.org/x/arch v0.22.0 h1:c/Zle32i5ttqRXjdLyyHZESLD/bB90DCU1g9l/0YBDI= +golang.org/x/arch v0.22.0/go.mod h1:dNHoOeKiyja7GTvF9NJS1l3Z2yntpQNzgrjh1cU103A= +golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts= +golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos= +golang.org/x/net v0.51.0 h1:94R/GTO7mt3/4wIKpcR5gkGmRLOuE/2hNGeWq/GBIFo= +golang.org/x/net v0.51.0/go.mod h1:aamm+2QF5ogm02fjy5Bb7CQ0WMt1/WVM7FtyaTLlA9Y= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k= +golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk= +golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA= +google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= +google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/observability-tracing-moesif/adaptor-api/internal/config/config.go b/observability-tracing-moesif/adaptor-api/internal/config/config.go new file mode 100644 index 00000000..24003857 --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/internal/config/config.go @@ -0,0 +1,91 @@ +// Copyright 2026 The OpenChoreo Authors +// SPDX-License-Identifier: Apache-2.0 + +package config + +import ( + "fmt" + "net/url" + "os" + "strconv" + "strings" +) + +const ( + AuthModeAPIKey = "api_key" + AuthModeBearer = "bearer" +) + +type Config struct { + ServerPort string + SearchEndpoint string + SearchAuthMode string + TokenDir string + EnvTokens map[string]string // environment name -> bearer token +} + +// AuthMode returns the configured authentication mode. +func (c *Config) AuthMode() string { + return c.SearchAuthMode +} + +// LoadConfig loads and validates configuration from environment variables. +func LoadConfig() (*Config, error) { + serverPort := getEnv("SERVER_PORT", "9100") + searchEndpoint := getEnv("SEARCH_ENDPOINT", "https://api.moesif.com") + searchAuthMode := getEnv("SEARCH_AUTH_MODE", AuthModeBearer) + tokenDir := getEnv("TOKEN_DIR", "/etc/moesif/env") + + if _, err := strconv.Atoi(serverPort); err != nil { + return nil, fmt.Errorf("invalid SERVER_PORT %q: %w", serverPort, err) + } + + if searchEndpoint == "" { + return nil, fmt.Errorf("environment variable SEARCH_ENDPOINT is required") + } + if searchAuthMode != AuthModeAPIKey && searchAuthMode != AuthModeBearer { + return nil, fmt.Errorf("invalid SEARCH_AUTH_MODE %q: must be %q or %q", searchAuthMode, AuthModeAPIKey, AuthModeBearer) + } + parsed, err := url.Parse(searchEndpoint) + if err != nil || parsed.Scheme == "" || parsed.Host == "" { + return nil, fmt.Errorf("SEARCH_ENDPOINT must be a valid URL with scheme and host, got: %q", searchEndpoint) + } + + return &Config{ + ServerPort: serverPort, + SearchEndpoint: searchEndpoint, + SearchAuthMode: searchAuthMode, + TokenDir: tokenDir, + EnvTokens: loadEnvTokens(tokenDir), + }, nil +} + +// loadEnvTokens reads token files from the configured token directory. +// Each file name is the environment name and its content is the token. +// e.g. /etc/moesif/env/development contains "dev.token" → {"development": "dev.token"} +func loadEnvTokens(tokenDir string) map[string]string { + tokens := make(map[string]string) + entries, err := os.ReadDir(tokenDir) + if err != nil { + return tokens + } + for _, entry := range entries { + if entry.IsDir() { + continue + } + name := strings.ToLower(entry.Name()) + data, err := os.ReadFile(tokenDir + "/" + entry.Name()) + if err != nil { + continue + } + tokens[name] = strings.TrimSpace(string(data)) + } + return tokens +} + +func getEnv(key, defaultValue string) string { + if value := os.Getenv(key); value != "" { + return value + } + return defaultValue +} diff --git a/observability-tracing-moesif/adaptor-api/internal/handler/handler.go b/observability-tracing-moesif/adaptor-api/internal/handler/handler.go new file mode 100644 index 00000000..66f7126f --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/internal/handler/handler.go @@ -0,0 +1,450 @@ +// Copyright 2026 The OpenChoreo Authors +// SPDX-License-Identifier: Apache-2.0 + +package handler + +import ( + "net/http" + "strings" + "time" + + "github.com/gin-gonic/gin" + "github.com/openchoreo/observability-tracing-moesif-cloud/adaptor-api/gen" + "github.com/openchoreo/observability-tracing-moesif-cloud/adaptor-api/internal/config" + "github.com/openchoreo/observability-tracing-moesif-cloud/adaptor-api/internal/search" +) + +// defaultSpanDetailsLookback bounds the search window for GetSpanDetailsForTrace, +// whose OpenAPI contract has no startTime/endTime parameters. +const defaultSpanDetailsLookback = 30 * 24 * time.Hour + +// Handler implements gen.ServerInterface. +type Handler struct { + searchClient *search.Client +} + +func New(searchClient *search.Client) *Handler { + return &Handler{searchClient: searchClient} +} + +func (h *Handler) Health(c *gin.Context) { + probe, err := h.searchClient.HealthProbe(c.Request.Context()) + if err != nil { + c.JSON(http.StatusServiceUnavailable, gin.H{"status": "unhealthy", "error": err.Error()}) + return + } + if !probe.Status { + c.JSON(http.StatusServiceUnavailable, gin.H{"status": "unhealthy", "upstream": probe}) + return + } + c.JSON(http.StatusOK, gin.H{"status": "healthy", "upstream": probe}) +} + +func (h *Handler) QueryTraces(c *gin.Context) { + moesifAppID := c.GetHeader("X-Moesif-App-Id") + moesifOrgID := c.GetHeader("X-Moesif-Org-Id") + + if h.searchClient.AuthMode() == config.AuthModeAPIKey { + if moesifAppID == "" || moesifOrgID == "" { + title := gen.ErrorResponseTitle("notFound") + c.JSON(http.StatusNotFound, gen.ErrorResponse{Title: &title, Detail: strPtr("missing required X-Moesif-App-Id or X-Moesif-Org-Id header")}) + return + } + } + + var req gen.QueryTracesJSONRequestBody + if err := c.ShouldBindJSON(&req); err != nil { + title := gen.BadRequest + c.JSON(http.StatusBadRequest, gen.ErrorResponse{Title: &title, Detail: strPtr(err.Error())}) + return + } + + sortOrder := "desc" + if req.SortOrder != nil { + sortOrder = string(*req.SortOrder) + } + + limit := 100 + if req.Limit != nil { + limit = *req.Limit + } + + params := search.TraceSearchParams{ + StartTime: req.StartTime, + EndTime: req.EndTime, + Size: limit, + SortOrder: sortOrder, + MoesifAppID: moesifAppID, + MoesifOrgID: moesifOrgID, + } + applyScope(¶ms, req.SearchScope) + + result, err := h.searchClient.SearchTraceEvents(c.Request.Context(), params) + if err != nil { + title := gen.InternalServerError + c.JSON(http.StatusInternalServerError, gen.ErrorResponse{Title: &title, Detail: strPtr(err.Error())}) + return + } + + traces := mapAggregationBucketsToTraces(result) + + c.JSON(http.StatusOK, gen.TracesListResponse{ + Traces: &traces, + Total: intPtr(len(traces)), + TookMs: intPtr(result.Took), + }) +} + +func (h *Handler) QuerySpansForTrace(c *gin.Context, traceId string) { + moesifAppID := c.GetHeader("X-Moesif-App-Id") + moesifOrgID := c.GetHeader("X-Moesif-Org-Id") + + if h.searchClient.AuthMode() == config.AuthModeAPIKey { + if moesifAppID == "" || moesifOrgID == "" { + title := gen.ErrorResponseTitle("notFound") + c.JSON(http.StatusNotFound, gen.ErrorResponse{Title: &title, Detail: strPtr("missing required X-Moesif-App-Id or X-Moesif-Org-Id header")}) + return + } + } + + var req gen.QuerySpansForTraceJSONRequestBody + if err := c.ShouldBindJSON(&req); err != nil { + title := gen.BadRequest + c.JSON(http.StatusBadRequest, gen.ErrorResponse{Title: &title, Detail: strPtr(err.Error())}) + return + } + + sortOrder := "desc" + if req.SortOrder != nil { + sortOrder = string(*req.SortOrder) + } + size := 100 + if req.Limit != nil { + size = *req.Limit + } + + params := search.TraceSearchParams{ + StartTime: req.StartTime, + EndTime: req.EndTime, + Size: size, + SortOrder: sortOrder, + TraceID: traceId, + MoesifAppID: moesifAppID, + MoesifOrgID: moesifOrgID, + } + applyScope(¶ms, req.SearchScope) + + includeAttributes := req.IncludeAttributes != nil && *req.IncludeAttributes + + result, err := h.searchClient.SearchTraceSpans(c.Request.Context(), params) + if err != nil { + title := gen.InternalServerError + c.JSON(http.StatusInternalServerError, gen.ErrorResponse{Title: &title, Detail: strPtr(err.Error())}) + return + } + + spans := make([]traceSpanItem, 0, len(result.Hits.Hits)) + for _, hit := range result.Hits.Hits { + rec := mapHitToSpanRecord(hit.Source) + spans = append(spans, toSpanItem(rec, includeAttributes)) + } + + c.JSON(http.StatusOK, gen.TraceSpansListResponse{ + Spans: &spans, + Total: intPtr(result.Hits.Total), + TookMs: intPtr(result.Took), + }) +} + +func (h *Handler) GetSpanDetailsForTrace(c *gin.Context, traceId string, spanId string) { + moesifAppID := c.GetHeader("X-Moesif-App-Id") + moesifOrgID := c.GetHeader("X-Moesif-Org-Id") + + if h.searchClient.AuthMode() == config.AuthModeAPIKey { + if moesifAppID == "" || moesifOrgID == "" { + title := gen.ErrorResponseTitle("notFound") + c.JSON(http.StatusNotFound, gen.ErrorResponse{Title: &title, Detail: strPtr("missing required X-Moesif-App-Id or X-Moesif-Org-Id header")}) + return + } + } + + now := time.Now() + params := search.TraceSearchParams{ + StartTime: now.Add(-defaultSpanDetailsLookback), + EndTime: now, + Size: 1, + TraceID: traceId, + SpanID: spanId, + MoesifAppID: moesifAppID, + MoesifOrgID: moesifOrgID, + } + + result, err := h.searchClient.SearchTraceSpanById(c.Request.Context(), params) + if err != nil { + title := gen.InternalServerError + c.JSON(http.StatusInternalServerError, gen.ErrorResponse{Title: &title, Detail: strPtr(err.Error())}) + return + } + + if len(result.Hits.Hits) == 0 { + c.JSON(http.StatusNotFound, gen.ErrorResponse{Detail: strPtr("span not found")}) + return + } + + rec := mapHitToSpanRecord(result.Hits.Hits[0].Source) + item := toSpanItem(rec, true) + + c.JSON(http.StatusOK, gen.TraceSpanDetailsResponse{ + SpanId: item.SpanId, + SpanName: item.SpanName, + SpanKind: item.SpanKind, + StartTime: item.StartTime, + EndTime: item.EndTime, + DurationNs: item.DurationNs, + ParentSpanId: item.ParentSpanId, + Status: item.Status, + Attributes: item.Attributes, + ResourceAttributes: item.ResourceAttributes, + }) +} + +// applyScope copies the ComponentSearchScope fields into the search params. +func applyScope(params *search.TraceSearchParams, scope gen.ComponentSearchScope) { + params.Namespace = scope.Namespace + if scope.Project != nil { + params.Project = *scope.Project + } + if scope.Component != nil { + params.Component = *scope.Component + } + if scope.Environment != nil { + params.Environment = *scope.Environment + } +} + +// traceSpanItem matches the anonymous item type generated for +// TraceSpansListResponse.Spans / TracesListResponse.Traces. +type traceSpanItem = struct { + Attributes *map[string]interface{} `json:"attributes,omitempty"` + DurationNs *int64 `json:"durationNs,omitempty"` + EndTime *time.Time `json:"endTime,omitempty"` + ParentSpanId *string `json:"parentSpanId,omitempty"` + ResourceAttributes *map[string]interface{} `json:"resourceAttributes,omitempty"` + SpanId *string `json:"spanId,omitempty"` + SpanKind *string `json:"spanKind,omitempty"` + SpanName *string `json:"spanName,omitempty"` + StartTime *time.Time `json:"startTime,omitempty"` + Status *gen.SpanStatus `json:"status,omitempty"` +} + +// spanRecord is the parsed representation of a single span extracted from a +// Moesif search hit. Field extraction assumes span data is nested under a +// "span" object and the trace ID under "trace_id" on the event; verify these +// paths against a real Moesif trace event payload. +type spanRecord struct { + TraceID string + SpanID string + SpanName string + SpanKind string + ParentSpanID string + StatusCode string + StatusMessage string + StartTime *time.Time + EndTime *time.Time + DurationNs int64 + Attributes map[string]interface{} + Resource map[string]interface{} +} + +func mapHitToSpanRecord(source map[string]interface{}) spanRecord { + rec := spanRecord{} + + req, _ := source["request"].(map[string]interface{}) + resp, _ := source["response"].(map[string]interface{}) + + if req != nil { + if rt, ok := req["time"].(string); ok { + rec.StartTime = parseTime(rt) + } + } + if resp != nil { + if rt, ok := resp["time"].(string); ok { + rec.EndTime = parseTime(rt) + } + } + + if dm, ok := source["duration_ms"].(float64); ok { + rec.DurationNs = int64(dm * 1e6) + } + if rec.EndTime == nil && rec.StartTime != nil && rec.DurationNs > 0 { + end := rec.StartTime.Add(time.Duration(rec.DurationNs)) + rec.EndTime = &end + } + if rec.DurationNs == 0 && rec.StartTime != nil && rec.EndTime != nil { + rec.DurationNs = rec.EndTime.Sub(*rec.StartTime).Nanoseconds() + } + + rec.TraceID, _ = source["trace_id"].(string) + rec.SpanName, _ = source["action_name"].(string) + + if span, ok := source["span"].(map[string]interface{}); ok { + rec.SpanID, _ = span["id"].(string) + rec.SpanKind, _ = span["kind"].(string) + rec.ParentSpanID, _ = span["parent_id"].(string) + rec.StatusCode, _ = span["status"].(string) + rec.StatusMessage, _ = span["status_message"].(string) + rec.Attributes, _ = span["attributes"].(map[string]interface{}) + } + + rec.Resource, _ = source["resource"].(map[string]interface{}) + + // If resource attributes are stored as flat dot-notation keys (e.g. "resource.openchoreo.dev/namespace") + // at the top level of _source, collect them into the Resource map. + if rec.Resource == nil { + rec.Resource = make(map[string]interface{}) + } + for k, v := range source { + if strings.HasPrefix(k, "resource.") { + rec.Resource[strings.TrimPrefix(k, "resource.")] = v + } + } + if len(rec.Resource) == 0 { + rec.Resource = nil + } + + return rec +} + +func parseTime(s string) *time.Time { + for _, layout := range []string{"2006-01-02T15:04:05.000", time.RFC3339Nano, time.RFC3339} { + if t, err := time.Parse(layout, s); err == nil { + return &t + } + } + return nil +} + +func toSpanItem(rec spanRecord, includeAttributes bool) traceSpanItem { + item := traceSpanItem{ + SpanId: strPtrOrNil(rec.SpanID), + SpanName: strPtrOrNil(rec.SpanName), + SpanKind: strPtrOrNil(rec.SpanKind), + StartTime: rec.StartTime, + EndTime: rec.EndTime, + ParentSpanId: strPtrOrNil(rec.ParentSpanID), + } + if rec.DurationNs != 0 { + item.DurationNs = int64Ptr(rec.DurationNs) + } + if rec.StatusCode != "" { + code := mapSpanStatusCode(rec.StatusCode) + item.Status = &gen.SpanStatus{Code: &code, Message: strPtrOrNil(rec.StatusMessage)} + } + if includeAttributes { + if rec.Attributes != nil { + item.Attributes = &rec.Attributes + } + } + if rec.Resource != nil { + item.ResourceAttributes = &rec.Resource + } + return item +} + +func mapSpanStatusCode(raw string) gen.SpanStatusCode { + switch strings.ToLower(raw) { + case "ok", "status_code_ok": + return gen.Ok + case "error", "status_code_error": + return gen.Error + default: + return gen.Unset + } +} + +// mapAggregationBucketsToTraces converts aggregation buckets from the search +// response into trace summary items. +func mapAggregationBucketsToTraces(result *search.SearchEventsResponse) []struct { + DurationNs *int64 `json:"durationNs,omitempty"` + EndTime *time.Time `json:"endTime,omitempty"` + HasErrors *bool `json:"hasErrors,omitempty"` + RootSpanId *string `json:"rootSpanId,omitempty"` + RootSpanKind *string `json:"rootSpanKind,omitempty"` + RootSpanName *string `json:"rootSpanName,omitempty"` + SpanCount *int `json:"spanCount,omitempty"` + StartTime *time.Time `json:"startTime,omitempty"` + TraceId *string `json:"traceId,omitempty"` + TraceName *string `json:"traceName,omitempty"` +} { + type traceItem = struct { + DurationNs *int64 `json:"durationNs,omitempty"` + EndTime *time.Time `json:"endTime,omitempty"` + HasErrors *bool `json:"hasErrors,omitempty"` + RootSpanId *string `json:"rootSpanId,omitempty"` + RootSpanKind *string `json:"rootSpanKind,omitempty"` + RootSpanName *string `json:"rootSpanName,omitempty"` + SpanCount *int `json:"spanCount,omitempty"` + StartTime *time.Time `json:"startTime,omitempty"` + TraceId *string `json:"traceId,omitempty"` + TraceName *string `json:"traceName,omitempty"` + } + + if result.Aggregations == nil { + return []traceItem{} + } + + buckets := result.Aggregations.Traces.Buckets + items := make([]traceItem, 0, len(buckets)) + + for _, bucket := range buckets { + traceID := bucket.Key + + // Parse start/end times from aggregation values (epoch millis). + startMs := int64(bucket.StartTime.Value) + endMs := int64(bucket.EndTime.Value) + start := time.UnixMilli(startMs).UTC() + end := time.UnixMilli(endMs).UTC() + + durationNs := end.Sub(start).Nanoseconds() + + // Extract root span info from top_hits. + var rootSpanID, rootSpanName, rootSpanKind string + if len(bucket.RootSpan.Hits.Hits) > 0 { + src := bucket.RootSpan.Hits.Hits[0].Source + if span, ok := src["span"].(map[string]interface{}); ok { + rootSpanID, _ = span["id"].(string) + rootSpanKind, _ = span["kind"].(string) + } + if name, ok := src["action_name"].(string); ok { + rootSpanName = name + } + } + + hasErrors := false + item := traceItem{ + TraceId: strPtrOrNil(traceID), + TraceName: strPtrOrNil(rootSpanName), + SpanCount: intPtr(bucket.DocCount), + RootSpanId: strPtrOrNil(rootSpanID), + RootSpanName: strPtrOrNil(rootSpanName), + RootSpanKind: strPtrOrNil(rootSpanKind), + StartTime: &start, + EndTime: &end, + DurationNs: int64Ptr(durationNs), + HasErrors: &hasErrors, + } + items = append(items, item) + } + + return items +} + +func strPtr(s string) *string { return &s } +func strPtrOrNil(s string) *string { + if s == "" { + return nil + } + return &s +} +func intPtr(i int) *int { return &i } +func int64Ptr(i int64) *int64 { return &i } diff --git a/observability-tracing-moesif/adaptor-api/internal/search/client.go b/observability-tracing-moesif/adaptor-api/internal/search/client.go new file mode 100644 index 00000000..33f72028 --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/internal/search/client.go @@ -0,0 +1,510 @@ +// Copyright 2026 The OpenChoreo Authors +// SPDX-License-Identifier: Apache-2.0 + +package search + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "log/slog" + "net/http" + "net/url" + "strings" + "time" + + "github.com/openchoreo/observability-tracing-moesif-cloud/adaptor-api/internal/config" +) + +// Client is an HTTP client for the Moesif search API. +type Client struct { + baseURL string + authMode string + httpClient *http.Client + logger *slog.Logger + envTokens map[string]string // environment name -> bearer token +} + +func NewClient(cfg *config.Config, logger *slog.Logger) *Client { + return &Client{ + baseURL: strings.TrimSuffix(cfg.SearchEndpoint, "/"), + authMode: cfg.AuthMode(), + httpClient: &http.Client{Timeout: 30 * time.Second}, + logger: logger, + envTokens: cfg.EnvTokens, + } +} + +// ResolveEnvToken resolves an environment UID to the corresponding token. +// For AuthModeAPIKey, it looks up "api_key" from the envTokens map. +// For AuthModeBearer, it resolves the environment name and returns the matching token. +// Returns the token and an error if not found. +func (c *Client) ResolveEnvToken(envUID string) (string, error) { + if c.authMode == config.AuthModeAPIKey { + if token, ok := c.envTokens["api_key"]; ok { + return token, nil + } + c.logger.Error("api_key not found in token map") + return "", fmt.Errorf("environment is not supported") + } + + if token, ok := c.envTokens[envUID]; ok { + return token, nil + } + c.logger.Error("no token found for environment", + slog.String("envUID", envUID)) + return "", fmt.Errorf("environment %q is not supported", envUID) +} + +// TraceSearchParams holds parameters for a trace/span search query. +type TraceSearchParams struct { + StartTime time.Time + EndTime time.Time + Size int + SortOrder string // "asc" or "desc" + Namespace string + Project string + Component string + Environment string + TraceID string + SpanID string + MoesifAppID string + MoesifOrgID string +} + +// SearchEventsResponse is the parsed response from the Moesif search events API. +type SearchEventsResponse struct { + Took int `json:"took"` + Hits searchHits `json:"hits"` + Aggregations *SearchAggregations `json:"aggregations,omitempty"` +} + +type searchHits struct { + Total int `json:"total"` + Hits []SearchHit `json:"hits"` +} + +// SearchHit is a single result from the search API. +type SearchHit struct { + Source map[string]interface{} `json:"_source"` +} + +// SearchAggregations holds the aggregation results from the search response. +type SearchAggregations struct { + Traces TracesAggregation `json:"traces"` +} + +type TracesAggregation struct { + Buckets []TraceBucket `json:"buckets"` +} + +type TraceBucket struct { + Key string `json:"key"` + DocCount int `json:"doc_count"` + StartTime AggValue `json:"startTime"` + EndTime AggValue `json:"endTime"` + RootSpan RootSpanAgg `json:"rootSpan"` +} + +type AggValue struct { + Value float64 `json:"value"` + ValueAsString string `json:"value_as_string"` +} + +type RootSpanAgg struct { + Hits RootSpanHits `json:"hits"` +} + +type RootSpanHits struct { + Hits []SearchHit `json:"hits"` +} + +// HealthProbeResponse represents the response from the Moesif search /health/probe endpoint. +type HealthProbeResponse struct { + Name string `json:"name"` + Status bool `json:"status"` + Region string `json:"region"` + Health string `json:"health"` + Build string `json:"build"` +} + +// HealthProbe calls the /health/probe endpoint on the Moesif search API. +// AuthMode returns the configured authentication mode. +func (c *Client) AuthMode() string { + return c.authMode +} + +func (c *Client) HealthProbe(ctx context.Context) (*HealthProbeResponse, error) { + respBody, statusCode, err := c.do(ctx, http.MethodGet, "/health/probe", nil, "") + if err != nil { + return nil, err + } + if statusCode < 200 || statusCode >= 300 { + return nil, fmt.Errorf("health probe returned status %d: %s", statusCode, string(respBody)) + } + + var result HealthProbeResponse + if err := json.Unmarshal(respBody, &result); err != nil { + return nil, fmt.Errorf("failed to unmarshal health probe response: %w", err) + } + return &result, nil +} + +// SearchTraceEvents queries the Moesif search events endpoint for span/trace +// data and returns the parsed response. +func (c *Client) SearchTraceEvents(ctx context.Context, params TraceSearchParams) (*SearchEventsResponse, error) { + body, err := buildSearchRequest(params) + if err != nil { + return nil, fmt.Errorf("failed to build search request: %w", err) + } + + var path string + q := url.Values{} + switch c.authMode { + case config.AuthModeBearer: + path = "/v1/search/~/search/events" + default: + path = fmt.Sprintf("/admin/%s/search/events", params.MoesifOrgID) + q.Set("app_id", params.MoesifAppID) + } + q.Set("from", params.StartTime.UTC().Format(time.RFC3339)) + q.Set("to", params.EndTime.UTC().Format(time.RFC3339)) + q.Set("week_starts_on", "1") + + fullPath := path + "?" + q.Encode() + + envToken, err := c.ResolveEnvToken(params.Environment) + if err != nil { + return nil, fmt.Errorf("internal error: %w", err) + } + respBody, statusCode, err := c.do(ctx, http.MethodPost, fullPath, body, envToken) + + c.logger.Info("search events API response", + slog.Int("statusCode", statusCode), + slog.Int("bytes", len(respBody))) + + if err != nil { + return nil, err + } + if statusCode < 200 || statusCode >= 300 { + c.logger.Error("search events API returned error", + slog.Int("statusCode", statusCode), + slog.String("body", string(respBody))) + return nil, fmt.Errorf("search API returned status %d: %s", statusCode, string(respBody)) + } + + var result SearchEventsResponse + if err := json.Unmarshal(respBody, &result); err != nil { + return nil, fmt.Errorf("failed to unmarshal search response: %w", err) + } + return &result, nil +} + +// SearchTraceSpans queries the Moesif search events endpoint for individual spans +// belonging to a trace and returns the parsed response. +func (c *Client) SearchTraceSpans(ctx context.Context, params TraceSearchParams) (*SearchEventsResponse, error) { + body, err := buildSpansSearchRequest(params) + if err != nil { + return nil, fmt.Errorf("failed to build spans search request: %w", err) + } + + var path string + q := url.Values{} + + switch c.authMode { + case config.AuthModeBearer: + path = "/v1/search/~/search/events" + default: + path = fmt.Sprintf("/admin/%s/search/events", params.MoesifOrgID) + q.Set("app_id", params.MoesifAppID) + } + q.Set("from", params.StartTime.UTC().Format(time.RFC3339)) + q.Set("to", params.EndTime.UTC().Format(time.RFC3339)) + q.Set("week_starts_on", "1") + fullPath := path + "?" + q.Encode() + + envToken, err := c.ResolveEnvToken(params.Environment) + if err != nil { + return nil, fmt.Errorf("internal error: %w", err) + } + respBody, statusCode, err := c.do(ctx, http.MethodPost, fullPath, body, envToken) + + c.logger.Info("search spans API response", + slog.Int("statusCode", statusCode), + slog.String("responseBody", string(respBody))) + + if err != nil { + return nil, err + } + if statusCode < 200 || statusCode >= 300 { + c.logger.Error("search spans API returned error", + slog.Int("statusCode", statusCode), + slog.String("body", string(respBody))) + return nil, fmt.Errorf("search API returned status %d: %s", statusCode, string(respBody)) + } + + var result SearchEventsResponse + if err := json.Unmarshal(respBody, &result); err != nil { + return nil, fmt.Errorf("failed to unmarshal search response: %w", err) + } + return &result, nil +} + +// SearchTraceSpanById queries the Moesif search events endpoint for a single span +// identified by trace ID and span ID. +func (c *Client) SearchTraceSpanById(ctx context.Context, params TraceSearchParams) (*SearchEventsResponse, error) { + body, err := buildSpansSearchRequest(params) + if err != nil { + return nil, fmt.Errorf("failed to build span-by-id search request: %w", err) + } + + var path string + q := url.Values{} + switch c.authMode { + case config.AuthModeBearer: + path = "/v1/search/~/search/events" + default: + path = fmt.Sprintf("/admin/%s/search/events", params.MoesifOrgID) + q.Set("app_id", params.MoesifAppID) + } + q.Set("from", params.StartTime.UTC().Format(time.RFC3339)) + q.Set("to", params.EndTime.UTC().Format(time.RFC3339)) + q.Set("week_starts_on", "1") + fullPath := path + "?" + q.Encode() + + envToken, err := c.ResolveEnvToken(params.Environment) + if err != nil { + return nil, fmt.Errorf("internal error: %w", err) + } + respBody, statusCode, err := c.do(ctx, http.MethodPost, fullPath, body, envToken) + + c.logger.Info("search span by id API response", + slog.Int("statusCode", statusCode), + slog.String("responseBody", string(respBody))) + + if err != nil { + return nil, err + } + if statusCode < 200 || statusCode >= 300 { + c.logger.Error("search span by id API returned error", + slog.Int("statusCode", statusCode), + slog.String("body", string(respBody))) + return nil, fmt.Errorf("search API returned status %d: %s", statusCode, string(respBody)) + } + + var result SearchEventsResponse + if err := json.Unmarshal(respBody, &result); err != nil { + return nil, fmt.Errorf("failed to unmarshal search response: %w", err) + } + return &result, nil +} + +// do executes an authenticated HTTP request and returns the raw response body and status code. +// If tokenOverride is non-empty, it is used as the Bearer token instead of the default. +func (c *Client) do(ctx context.Context, method, path string, body []byte, bearerToken string) ([]byte, int, error) { + u := c.baseURL + path + + var reqBody io.Reader + if len(body) > 0 { + reqBody = bytes.NewBuffer(body) + } + + req, err := http.NewRequestWithContext(ctx, method, u, reqBody) + if err != nil { + return nil, 0, fmt.Errorf("failed to create request: %w", err) + } + + if len(body) > 0 { + req.Header.Set("Content-Type", "application/json") + } + + switch c.authMode { + case config.AuthModeAPIKey: + req.Header.Set("X-Api-Token", bearerToken) + case config.AuthModeBearer: + req.Header.Set("Authorization", "Bearer "+bearerToken) + } + + resp, err := c.httpClient.Do(req) + if err != nil { + c.logger.Error("search API request failed", + slog.String("method", method), + slog.String("path", path), + slog.Any("error", err)) + return nil, 0, fmt.Errorf("failed to execute request: %w", err) + } + defer resp.Body.Close() + + respBody, err := io.ReadAll(resp.Body) + if err != nil { + return nil, resp.StatusCode, fmt.Errorf("failed to read response body: %w", err) + } + + return respBody, resp.StatusCode, nil +} + +func buildSearchRequest(params TraceSearchParams) ([]byte, error) { + size := params.Size + if size <= 0 { + size = 100 + } + + sortOrder := params.SortOrder + if sortOrder == "" { + sortOrder = "desc" + } + + // Always require trace_id.raw to exist. + filter := []interface{}{ + map[string]interface{}{ + "exists": map[string]interface{}{ + "field": "trace_id.raw", + }, + }, + } + + if params.Namespace != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"resource.openchoreo.dev/namespace": params.Namespace}, + }) + } + if params.Project != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"resource.openchoreo.dev/project-uid": params.Project}, + }) + } + if params.Component != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"resource.openchoreo.dev/component-uid": params.Component}, + }) + } + if params.Environment != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"resource.openchoreo.dev/environment-uid": params.Environment}, + }) + } + if params.TraceID != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"trace_id": params.TraceID}, + }) + } + if params.SpanID != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"span.id": params.SpanID}, + }) + } + + req := map[string]interface{}{ + "size": 0, + "query": map[string]interface{}{ + "bool": map[string]interface{}{ + "filter": filter, + }, + }, + "aggs": map[string]interface{}{ + "traces": map[string]interface{}{ + "terms": map[string]interface{}{ + "field": "trace_id.raw", + "size": size, + "order": map[string]interface{}{ + "startTime": sortOrder, + }, + }, + "aggs": map[string]interface{}{ + "startTime": map[string]interface{}{ + "min": map[string]interface{}{ + "field": "request.time", + }, + }, + "endTime": map[string]interface{}{ + "max": map[string]interface{}{ + "field": "request.time", + }, + }, + "rootSpan": map[string]interface{}{ + "top_hits": map[string]interface{}{ + "size": 1, + "sort": []map[string]interface{}{ + { + "request.time": map[string]interface{}{ + "order": "asc", + }, + }, + }, + "_source": []string{ + "span.id", + "action_name", + "span.kind", + }, + }, + }, + }, + }, + }, + } + + return json.Marshal(req) +} + +func buildSpansSearchRequest(params TraceSearchParams) ([]byte, error) { + size := params.Size + if size <= 0 { + size = 10 + } + + var filter []interface{} + + if params.Namespace != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"resource.openchoreo.dev/namespace": params.Namespace}, + }) + } + if params.Project != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"resource.openchoreo.dev/project-uid": params.Project}, + }) + } + if params.Component != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"resource.openchoreo.dev/component-uid": params.Component}, + }) + } + if params.Environment != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"resource.openchoreo.dev/environment-uid": params.Environment}, + }) + } + if params.TraceID != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"trace_id": params.TraceID}, + }) + } + if params.SpanID != "" { + filter = append(filter, map[string]interface{}{ + "match_phrase": map[string]interface{}{"span.id": params.SpanID}, + }) + } + + req := map[string]interface{}{ + "size": size, + } + sortOrder := params.SortOrder + if sortOrder == "" { + sortOrder = "desc" + } + req["sort"] = []map[string]interface{}{ + {"request.time": map[string]interface{}{"order": sortOrder}}, + } + + if len(filter) > 0 { + req["query"] = map[string]interface{}{ + "bool": map[string]interface{}{ + "filter": filter, + }, + } + } + + return json.Marshal(req) +} diff --git a/observability-tracing-moesif/adaptor-api/oapi-codegen.yaml b/observability-tracing-moesif/adaptor-api/oapi-codegen.yaml new file mode 100644 index 00000000..2f2f9815 --- /dev/null +++ b/observability-tracing-moesif/adaptor-api/oapi-codegen.yaml @@ -0,0 +1,14 @@ +# oapi-codegen configuration +# Run `make generate` to regenerate after editing the OAS file. + +package: gen +output-options: + skip-prune: false + +generate: + gin-server: true + models: true + embedded-spec: false + +input-spec: api/observability-tracing-adapter-api.yaml +output: gen/ diff --git a/observability-tracing-moesif/helm/Chart.yaml b/observability-tracing-moesif/helm/Chart.yaml new file mode 100644 index 00000000..55684104 --- /dev/null +++ b/observability-tracing-moesif/helm/Chart.yaml @@ -0,0 +1,21 @@ +# Copyright 2026 The OpenChoreo Authors +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v2 +name: observability-tracing-moesif +description: A Helm chart for OpenChoreo Moesif Tracing module +type: application +version: 0.1.0 +appVersion: "0.1.0" +keywords: + - moesif + - observability + - tracing +home: https://github.com/openchoreo/community-modules +maintainers: + - name: Moesif Team +dependencies: + - name: opentelemetry-collector + repository: https://open-telemetry.github.io/opentelemetry-helm-charts + version: 0.140.0 + condition: opentelemetry-collector.enabled diff --git a/observability-tracing-moesif/helm/templates/adapter/configmap.yaml b/observability-tracing-moesif/helm/templates/adapter/configmap.yaml new file mode 100644 index 00000000..825afb66 --- /dev/null +++ b/observability-tracing-moesif/helm/templates/adapter/configmap.yaml @@ -0,0 +1,16 @@ +# Copyright 2026 The OpenChoreo Authors +# SPDX-License-Identifier: Apache-2.0 + +{{- if .Values.moesif.adapter.enabled }} +apiVersion: v1 +kind: ConfigMap +metadata: + name: moesif-tracing-adapter + namespace: {{ .Release.Namespace }} + labels: + app: moesif-tracing-adapter +data: + SERVER_PORT: "9100" + SEARCH_ENDPOINT: {{ .Values.moesif.adapter.searchEndpoint | quote }} + SEARCH_AUTH_MODE: {{ .Values.moesif.auth_mode | quote }} +{{- end }} diff --git a/observability-tracing-moesif/helm/templates/adapter/deployment.yaml b/observability-tracing-moesif/helm/templates/adapter/deployment.yaml new file mode 100644 index 00000000..4da330a6 --- /dev/null +++ b/observability-tracing-moesif/helm/templates/adapter/deployment.yaml @@ -0,0 +1,52 @@ +# Copyright 2026 The OpenChoreo Authors +# SPDX-License-Identifier: Apache-2.0 + +{{- if .Values.moesif.adapter.enabled }} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: moesif-tracing-adapter + namespace: {{ .Release.Namespace }} + labels: + app: moesif-tracing-adapter +spec: + replicas: 1 + selector: + matchLabels: + app: moesif-tracing-adapter + template: + metadata: + annotations: + checksum/config: {{ include (print $.Template.BasePath "/adapter/configmap.yaml") . | sha256sum }} + labels: + app: moesif-tracing-adapter + spec: + securityContext: + runAsUser: 10500 + runAsGroup: 10500 + runAsNonRoot: true + containers: + - name: moesif-tracing-adapter + image: "{{ .Values.moesif.adapter.image.repository }}:{{ .Values.moesif.adapter.image.tag | default .Chart.AppVersion }}" + imagePullPolicy: {{ .Values.moesif.adapter.image.pullPolicy | default "IfNotPresent" }} + ports: + - containerPort: 9100 + envFrom: + - configMapRef: + name: moesif-tracing-adapter + volumeMounts: + - name: moesif-search-secret + mountPath: /etc/moesif/env + readOnly: true + resources: + limits: + cpu: {{ .Values.moesif.adapter.resources.limits.cpu }} + memory: {{ .Values.moesif.adapter.resources.limits.memory }} + requests: + cpu: {{ .Values.moesif.adapter.resources.requests.cpu }} + memory: {{ .Values.moesif.adapter.resources.requests.memory }} + volumes: + - name: moesif-search-secret + secret: + secretName: {{ .Values.moesif.adapter.searchSecretName }} +{{- end }} diff --git a/observability-tracing-moesif/helm/templates/adapter/service.yaml b/observability-tracing-moesif/helm/templates/adapter/service.yaml new file mode 100644 index 00000000..65344972 --- /dev/null +++ b/observability-tracing-moesif/helm/templates/adapter/service.yaml @@ -0,0 +1,21 @@ +# Copyright 2026 The OpenChoreo Authors +# SPDX-License-Identifier: Apache-2.0 + +{{- if .Values.moesif.adapter.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: tracing-adapter + namespace: {{ .Release.Namespace }} + labels: + app: moesif-tracing-adapter +spec: + type: ClusterIP + ports: + - port: 9100 + targetPort: 9100 + protocol: TCP + name: http + selector: + app: moesif-tracing-adapter +{{- end }} diff --git a/observability-tracing-moesif/helm/templates/opentelemetry-collector/configMap.yaml b/observability-tracing-moesif/helm/templates/opentelemetry-collector/configMap.yaml new file mode 100644 index 00000000..0be3b84b --- /dev/null +++ b/observability-tracing-moesif/helm/templates/opentelemetry-collector/configMap.yaml @@ -0,0 +1,132 @@ +# Copyright 2026 The OpenChoreo Authors +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v1 +kind: ConfigMap +metadata: + name: moesif-tracing-collector-config + namespace: {{ .Release.Namespace }} +data: + relay: | + receivers: + otlp: + protocols: + grpc: + endpoint: 0.0.0.0:4317 + http: + endpoint: 0.0.0.0:4318 + include_metadata: true + + exporters: + {{- if .Values.opentelemetryCollectorCustomizations.debug.enabled }} + debug: + verbosity: detailed + {{- end }} + {{- range .Values.moesif.environments }} + otlphttp/{{ .name }}: + endpoint: {{ $.Values.moesif.endpoint | default "https://api.moesif.net" | quote }} + headers: + {{- if eq $.Values.moesif.auth_mode "api_key" }} + X-Api-Token: ${file:/etc/moesif/env/api_key} + {{- else }} + X-Moesif-Application-Id: ${file:/etc/moesif/env/{{ .id }}} + {{- end }} + auth: + authenticator: headers_setter + {{- end }} + + connectors: + routing: + table: + {{- range .Values.moesif.environments }} + - context: resource + statement: route() where resource.attributes["openchoreo.dev/environment"] == {{ .name | quote }} + pipelines: [ traces/{{ .name }} ] + {{- end }} + + extensions: + health_check: + endpoint: ${env:MY_POD_IP}:13133 + headers_setter: + headers: + - action: upsert + key: X-Moesif-Org-Id + from_context: X-Moesif-Org-Id + - action: upsert + key: X-Moesif-App-Id + from_context: X-Moesif-App-Id + + processors: + resource/cleanup: + attributes: + - key: openchoreo.dev/environment + action: delete + k8sattributes: + auth_type: "serviceAccount" + passthrough: false + + extract: + labels: + - tag_name: $$1 + key_regex: (.*) + from: pod + + metadata: + - k8s.pod.name + - k8s.pod.uid + - k8s.deployment.name + - k8s.namespace.name + - k8s.node.name + {{- if .Values.opentelemetryCollectorCustomizations.tailSampling.enabled }} + tail_sampling: + decision_wait: {{ .Values.opentelemetryCollectorCustomizations.tailSampling.decisionWait }} + num_traces: {{ .Values.opentelemetryCollectorCustomizations.tailSampling.numTraces }} + expected_new_traces_per_sec: {{ .Values.opentelemetryCollectorCustomizations.tailSampling.expectedNewTracesPerSec }} + decision_cache: + sampled_cache_size: {{ .Values.opentelemetryCollectorCustomizations.tailSampling.decisionCache.sampledCacheSize }} + non_sampled_cache_size: {{ .Values.opentelemetryCollectorCustomizations.tailSampling.decisionCache.nonSampledCacheSize }} + policies: [ + { + name: rate_limiting, + type: rate_limiting, + rate_limiting: {spans_per_second: {{ .Values.opentelemetryCollectorCustomizations.tailSampling.spansPerSecond }} } + }, + ] + {{- end }} + batch: + metadata_keys: + - X-Moesif-Org-Id + - X-Moesif-App-Id + metadata_cardinality_limit: 1000 + timeout: 10s + send_batch_size: 1024 + + service: + extensions: [health_check, headers_setter] + pipelines: + traces/in: + receivers: [otlp] + {{- $processors := list }} + {{- $processors = append $processors "resource/cleanup" }} + {{- $processors = append $processors "k8sattributes" }} + {{- if .Values.opentelemetryCollectorCustomizations.tailSampling.enabled }} + {{- $processors = append $processors "tail_sampling" }} + {{- end }} + {{- $processors = append $processors "batch" }} + {{- if $processors }} + processors: [{{ join ", " $processors }}] + {{- end }} + exporters: [ routing ] + {{- range .Values.moesif.environments }} + traces/{{ .name }}: + receivers: [ routing ] + exporters: [ otlphttp/{{ .name }}{{- if $.Values.opentelemetryCollectorCustomizations.debug.enabled }}, debug{{- end }} ] + {{- end }} + telemetry: + metrics: + readers: + - pull: + exporter: + prometheus: + host: '0.0.0.0' + port: 8888 diff --git a/observability-tracing-moesif/helm/values.yaml b/observability-tracing-moesif/helm/values.yaml new file mode 100644 index 00000000..5b7baa44 --- /dev/null +++ b/observability-tracing-moesif/helm/values.yaml @@ -0,0 +1,83 @@ +# Copyright 2026 The OpenChoreo Authors +# SPDX-License-Identifier: Apache-2.0 + +## ----------------------------------------------------------- +## Values for OpenTelemetry Collector configuration +## ----------------------------------------------------------- +opentelemetry-collector: + enabled: true + fullnameOverride: "moesif-tracing-collector" + + clusterRole: + create: true + rules: + - apiGroups: [""] + resources: ["pods"] + verbs: ["list", "watch"] + - apiGroups: ["apps"] + resources: ["replicasets"] + verbs: ["list", "watch"] + + configMap: + create: false + existingName: "moesif-tracing-collector-config" + + image: + repository: otel/opentelemetry-collector-contrib + + mode: deployment + + extraVolumes: + - name: moesif-env-tokens + secret: + secretName: moesif-tracing-collector-secret + + extraVolumeMounts: + - name: moesif-env-tokens + mountPath: /etc/moesif/env + readOnly: true + + resources: + limits: + cpu: 100m + memory: 200Mi + requests: + cpu: 50m + memory: 100Mi + + service: + type: ClusterIP + +opentelemetryCollectorCustomizations: + debug: + enabled: false + + tailSampling: + enabled: true + decisionWait: 10s + numTraces: 100 + expectedNewTracesPerSec: 10 + decisionCache: + sampledCacheSize: 10000 + nonSampledCacheSize: 1000 + spansPerSecond: 10 + +## ----------------------------------------------------------- +## Values for OpenChoreo specific customizations and workloads +## ----------------------------------------------------------- + +moesif: + adapter: + enabled: true + searchEndpoint: "https://api.moesif.com" + searchSecretName: "moesif-trace-search-secret" + image: + repository: "ghcr.io/openchoreo/observability-tracing-moesif-adapter" + tag: "1.0.0" + resources: + limits: + cpu: 200m + memory: 256Mi + requests: + cpu: 50m + memory: 128Mi diff --git a/observability-tracing-moesif/module.yaml b/observability-tracing-moesif/module.yaml new file mode 100644 index 00000000..8f5fdaf3 --- /dev/null +++ b/observability-tracing-moesif/module.yaml @@ -0,0 +1,11 @@ +# Copyright 2026 The OpenChoreo Authors +# SPDX-License-Identifier: Apache-2.0 + +# Module manifest used by the CI workflow to discover Docker images to build. +# Each entry defines the image name, build context, Dockerfile path, and the +# corresponding field in helm/values.yaml to update with the published image URI. + +images: + - name: observability-tracing-moesif-cloud-adapter + context: adaptor-api + dockerfile: adaptor-api/Dockerfile