Tools directory

Open model registry, experiment tracking, and deployment packaging.

Open feature store for training/serving parity and point-in-time joins.

Experiment tracking, model registry, and eval visualization for ML teams.

Package and serve ML/LLM models as production APIs with adapters and runners.

Git-backed data and model versioning with reproducible pipelines.

Python-native workflow orchestration for training, eval, and data pipelines.

Data quality expectations as code for training and feature pipelines.

Unified LLM proxy with routing, caching, budgets, and provider failover.

Managed LLM gateway with observability, guardrails, and routing.

LLM observability proxy with cost, latency, and cache analytics.

CLI and CI evals for prompts, RAG, and red-team scenarios.

Rust vector database with filtered search, hybrid retrieval, and self-host or cloud options.

Postgres extension for embeddings - lowest ops cost when you already run Postgres.

RAG-specific eval metrics: faithfulness, context precision/recall, answer relevancy.

Pytest-style LLM evaluation framework with scorers for correctness, toxicity, and RAG.

Graph-based agent orchestration with durable state, cycles, and human-in-the-loop nodes.

Document parsing and chunking for PDFs, tables, and messy enterprise corpora.

Multi-provider LLM API marketplace with unified routing across model vendors.

Managed vector database with serverless and dedicated options for RAG at scale.

LLM traces, prompt versioning, eval datasets, and production analytics.

Data drift, model quality, and LLM eval reports as code.

Vendor-neutral traces, metrics, and logs - including GenAI semantic conventions.

Eval-driven development - datasets, scorers, and regression tracking for LLM apps.

Open LLM observability and eval tracing - spans, embeddings viz, and dataset workflows.

PII detection and anonymization for text before LLM calls.

Programmable conversational guardrails (Colang) for input/output rails and dialog flows.

Output validation and structured correction for LLM responses (schema, toxicity, PII).

Managed prompt-injection and content-risk detection for LLM and agent traffic.

GitOps continuous delivery for Kubernetes and ML manifest promotion.

Kubernetes-native model serving with canary and scale-to-zero.

Durable workflow engine for long-running agent jobs with retries and approvals.

Internal developer portal for golden paths, templates, and service catalog.

High-throughput OpenAI-compatible inference server with PagedAttention for GPU efficiency.

Declarative IaC for cloud landing zones, networking, identity, and AI service baselines.

CNCF GitOps toolkit for continuous reconciliation of Kubernetes manifests.

Kubernetes cost allocation by namespace, label, and workload.

IaC cost estimates in PRs for Terraform and OpenTofu.

In-memory store used for semantic LLM caches, session memory, and low-latency vector indexes.