Knowledge Base

Production-grade Knowledge Graph system with Qdrant vector DB + Neo4j graph database.

Features

• Semantic Search: Vector similarity search via Qdrant HNSW index
• Event Tracking: Task completions, code changes, discussions, decisions
• TLS Security: Encrypted connections for Qdrant and Neo4j
• Rate Limiting: SlowAPI-based per-endpoint limits
• Audit Logging: Append-only JSON security audit trail
• Prometheus Metrics: Full observability with custom KG metrics
• Backup/Restore: Automated scripts with 30-day retention

Architecture

                        API Service
  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐
  │ /kg/health  │  │ /kg/search  │  │/kg/knowledge│
  └─────────────┘  └─────────────┘  └─────────────┘
         │                │                │
  ┌──────┴────────────────┴────────────────┴──────┐
  │              KG Service Layer                  │
  │  - Embedding generation (all-MiniLM-L6-v2)    │
  │  - Event deduplication                         │
  │  - Rate limiting and audit logging             │
  └────────────────────────────────────────────────┘
          │                              │
    ┌─────┴─────┐                 ┌─────┴─────┐
    │  Qdrant   │                 │   Neo4j   │
    │  (Vectors)│                 │  (Graph)  │
    └───────────┘                 └───────────┘

Quick Start

# Start services
docker-compose up -d

# Check health
curl http://localhost:8000/kg/health

# Add knowledge
curl -X POST http://localhost:8000/kg/knowledge \
  -H "Content-Type: application/json" \
  -d '{"event_type": "task_completion", "description": "Implemented feature X"}'

# Search
curl -X POST http://localhost:8000/kg/search \
  -H "Content-Type: application/json" \
  -d '{"query": "feature implementation", "top_k": 10}'

API Endpoints

Endpoint	Method	Description	Rate Limit
/kg/health	GET	Health check for all components	Unlimited
/kg/search	POST	Semantic search	50/min
/kg/knowledge	POST	Store new knowledge	20/min
/metrics	GET	Prometheus metrics	Unlimited

Configuration

Environment variables:

Variable	Description	Required
QDRANT_URL	Qdrant server URL	Yes
NEO4J_URI	Neo4j bolt URI	Yes
NEO4J_USER	Neo4j username	Yes
NEO4J_PASSWORD	Neo4j password	Yes
SECRET_KEY	API secret key	Yes
QDRANT_TLS	Enable TLS for Qdrant	No (default: false)
NEO4J_TLS	Enable TLS for Neo4j	No (default: false)
RATE_LIMIT_ENABLED	Enable rate limiting	No (default: true)
DEDUP_WINDOW_HOURS	Deduplication window	No (default: 1)

Scripts

Script	Purpose
scripts/kg-update.sh	CLI for adding events to knowledge graph
scripts/kg-update-light.sh	Lightweight version for quick updates
deployment/backup-kg.sh	Automated backup (cron-compatible)
deployment/restore-kg.sh	Restore from backup
deployment/consistency-check.py	Verify Qdrant/Neo4j consistency

Monitoring

• Grafana Dashboard: grafana/dashboards/kg.json
• Prometheus Alerts: prometheus/alerts-kg.yml
  • KGSearchLatencyHigh: p99 > 500ms for 5min
  • KGComponentUnhealthy: Any component down for 1min
  • KGErrorRateHigh: Error rate > 5% for 5min

Security

• TLS encryption for Qdrant/Neo4j (configurable)
• Docker network isolation (kg-internal network)
• Audit logging with tamper-evident append-only files
• Dependency vulnerability scanning via GitHub Actions
• Per-endpoint rate limiting

Directory Structure

knowledge-base/
├── services/api/          # FastAPI application
│   ├── kg_routes.py       # KG API endpoints
│   ├── metrics.py         # Prometheus metrics
│   ├── audit_logger.py    # Security audit logging
│   ├── rate_limiter.py    # SlowAPI rate limiting
│   └── ...
├── scripts/               # CLI tools
│   ├── kg-update.sh       # Add events
│   └── kg-api.py          # API client
├── deployment/            # Operations
│   ├── backup-kg.sh       # Backup script
│   ├── restore-kg.sh      # Restore script
│   └── consistency-check.py
├── schemas/               # JSON schemas
├── grafana/dashboards/    # Grafana dashboards
├── prometheus/            # Alerting rules
├── certs/                 # TLS certificates
└── docker-compose.yml     # Service definitions

License

MIT