Overview
The purpose of this page at this time is to capture requirements related to observability of the EMCO services (https://gitlab.com/groups/project-emco/-/epics/7).
...
Also, keep in mind this cautionary note from the Prometheus project
CAUTION: Remember that every unique combination of key-value label pairs represents a new time series, which can dramatically increase the amount of data stored. Do not use labels to store dimensions with high cardinality (many different label values), such as user IDs, email addresses, or other unbounded sets of values.
Unbounded sets of values in the EMCO APIs would include values such as project names, intent names, etc.
...
The resources of a service can be identified from the HTTP resources. The initial labels can be the URL parameters.
| Service | Resource | Labels |
|---|---|---|
| orchestrator | controller | name |
| project | name | |
| compositeApp | version, name, project | |
| app | name, composite_app_version, composite_app, project | |
| dependency | name, app, composite_app_version, composite_app, project | |
| compositeProfile | name, composite_app_version, composite_app, project | |
| appProfile | name, composite_profile, composite_app_version, composite_app, project | |
| deploymentIntentGroup | name, composite_app_version, composite_app, project | |
| genericPlacementIntent | name, deployment_intent_group, composite_app_version, composite_app, project | |
| genericAppPlacementIntent | name, generic_placement_intent, deployment_intent_group, composite_app_version, composite_app, project | |
| groupIntent | name, deployment_intent_group, composite_app_version, composite_app_name, project |
The metrics for these resources should capture the state of the resource, i.e. metrics for creation, deletion, etc. (emco_controller_creation_timestamp, emco_controller_deletion_timestamp, etc.) as described in the guidelines. This approach is suggested as it is unclear how to apply metrics capturing resource utilization to these resources.
The status of a deployment intent group deserves special consideration. The initial idea would be to add metrics describing the contents of the status. This would enable alerting on failed resources for example.
| Metric | Labels |
|---|---|
| deployment_intent_group_resource | name, cluster, cluster_provider, app, deployment_intent_group, composite_profile, composite_app_version, composite_app, project |
It's not clear to me yet whether the rsyncStatus value should be part of the metric name (deployment_intent_group_resource_applied) or a label. Following the kube-state-metrics model would make it part of the metric name. Further complicating the question is the readyStatus field of the cluster.
...