Grafana Agent Observability Plugin Released for Hermes Agent

Plugin Functionality

The new Grafana Agent Observability plugin for Hermes Agent is designed to capture data from Large Language Model (LLM) calls and tool executions. This data is recorded as 'generations', providing a structured way to track the agent's interactions and outputs. The plugin also emits OpenTelemetry (OTel) traces and metrics, which are standard formats for collecting telemetry data.

Integration and Configuration

To integrate the plugin, users need to install it via pip using `pip install grafana-agento11y-hermes` within the same Python environment as Hermes. Configuration involves enabling the `agento11y` plugin in the `~/.hermes/config.yaml` file. The plugin provides a setup page within Grafana for generating necessary environment variables, including endpoint and authentication details, which must be set in Hermes's environment.

Upgrade Path from Previous Version

Users upgrading from the `hermes-plugin-sigil` package must uninstall the old package before installing `grafana-agento11y-hermes`. Additionally, the configuration key in `~/.hermes/config.yaml` needs to be changed from `sigil` to `agento11y`, and environment variables prefixed with `SIGIL_` must be renamed to `AGENTO11Y_`.

Why It Matters

This plugin offers developers a way to gain visibility into the operations of their AI agents built with Hermes. By providing detailed traces and metrics, it allows for better debugging, performance monitoring, and understanding of how LLMs and tools are being utilized within the agent's workflow. This can be crucial for optimizing agent behavior and ensuring reliable operation in production environments.