Google Cloud's Knowledge Catalog Integrates Open Knowledge Format (OKF) for Agent Context

Open Knowledge Format Evolution

The Open Knowledge Format (OKF) is an open specification designed to formalize the LLM-wiki pattern into a portable and interoperable format for AI agents. OKF v0.1 established a portable format using markdown files with YAML frontmatter, while OKF v0.2 added trust signals such as provenance, verification, freshness, and attestation to enhance reliability for machine-authored bundles.

Challenge of Organizational Sharing

While OKF provides a portable format, a key challenge has been sharing and governing access to OKF bundles across an organization. Managing bundles in separate Git repositories does not offer searchability alongside data, lacks integrated security and compliance, and is disconnected from existing technical metadata like schemas and lineage. This approach does not scale beyond a small number of bundles, as each agent needs to know the specific location of every bundle.

Knowledge Catalog Integration

To address organizational scalability, Google Cloud's Knowledge Catalog now integrates with OKF. Knowledge Catalog functions as a context engine for agents, providing a governed index over data from sources like BigQuery, Cloud Storage, and operational databases. By mapping OKF bundles onto Knowledge Catalog's existing types, every concept within a bundle becomes discoverable, governed, and accessible to any agent already utilizing the catalog.

Enhanced Discoverability and Governance

The integration means that OKF bundles benefit from Knowledge Catalog's features, including schema, lineage, ownership, and tagging. The catalog also offers search and cross-project lookup capabilities, retrieving optimized context for agentic queries. Access to this context is secured and governed by IAM controls, ensuring agents only access entries they are authorized to see based on their IAM identity.

Simplified Publishing Process

Publishing an OKF bundle into Knowledge Catalog requires a one-time setup and a single push. This process utilizes OKF sample code found in the Knowledge Catalog repository. Wrappers within this code call gcloud dataplex for the initial setup and delegate the push operation to kcmd, the Metadata-as-Code CLI also located in the same repository. The setup registers specific Knowledge Catalog resources, including an EntryGroup to contain the bundle.