Mattermost Feedback Bot Deployment Summary

Implementation Complete

Successfully implemented issue #87: Deploy Mattermost Feedback Bot

What Was Built

Feedback Bot Service (`services/feedback-bot/`)

FastAPI application with natural language processing
VADER sentiment analysis - analyzes positive/negative/neutral sentiment
Auto-categorization - 10 categories (UI, Performance, Documentation, CI/CD, Security, API, Feature Request, Bug, Observability, Developer Experience)
Smart rating extraction - extracts or infers ratings from natural language (1-5 stars)
Mattermost integration - /feedback slash command
Prometheus metrics - tracks submissions by sentiment and category
Feedback API integration - submits to centralized feedback service

Key Features

Natural Language Interface
Users type feedback naturally: "The new UI is amazing!"
Bot analyzes and categorizes automatically
No forms or structured input required
Sentiment Analysis
Positive: 😊 (compound score ≥ 0.05)
Neutral: 😐 (compound score -0.05 to 0.05)
Negative: 😞 (compound score ≤ -0.05)
Auto-Categorization
Keyword-based matching for 10 categories
Falls back to "General" if no match
Handles multiple categories intelligently
Smart Rating
Extracts explicit ratings: "5 stars", "4/5", "rate it 3"
Infers from sentiment if no explicit rating
Always assigns 1-5 star rating

Deployment

Kubernetes Manifests (`platform/apps/feedback-bot/`)

deployment.yaml - 2 replicas with health checks
service.yaml - ClusterIP service on port 8000
secret.yaml - Optional credentials for API and bot token
serviceaccount.yaml - Service account for pod
servicemonitor.yaml - Prometheus metrics scraping
kustomization.yaml - Kustomize configuration

Deploy Commands

# Apply all manifests
kubectl apply -k platform/apps/feedback-bot/

# Verify deployment
kubectl get pods -n fawkes -l app=feedback-bot
kubectl logs -n fawkes -l app=feedback-bot

# Test health endpoint
kubectl port-forward -n fawkes svc/feedback-bot 8000:8000
curl http://localhost:8000/health

Mattermost Setup

Create Slash Command

Go to Main Menu → Integrations → Slash Commands
Click "Add Slash Command"
Configure:
Command: feedback
Request URL: http://feedback-bot.fawkes.svc.cluster.local:8000/mattermost/slash/feedback
Request Method: POST
Response Username: @fawkes
Autocomplete: Enable
Autocomplete Hint: <your feedback here>

Usage Examples

/feedback The new UI is amazing! Love the dark mode.
→ ⭐⭐⭐⭐⭐ (5/5), Positive 😊, Category: UI

/feedback Builds are too slow, taking 20+ minutes
→ ⭐⭐ (2/5), Negative 😞, Category: CI/CD

/feedback Rate it 5 stars! Documentation is excellent.
→ ⭐⭐⭐⭐⭐ (5/5), Positive 😊, Category: Documentation

Configuration

Environment variables in deployment.yaml:

FEEDBACK_API_URL - URL of feedback service API
FEEDBACK_API_TOKEN - Optional API authentication token
BOT_TOKEN - Optional Mattermost slash command token for validation
MATTERMOST_URL - Mattermost server URL
EMAIL_DOMAIN - Domain for user emails (default: fawkes.local)

Testing

Unit Tests

cd services/feedback-bot
pip install -r requirements.txt -r requirements-dev.txt
pytest tests/test_main.py -v

Result: ✅ 7/7 tests passing

BDD Tests

Located in tests/bdd/features/feedback-bot.feature

15 scenarios covering all features
Tests for NLP, sentiment analysis, auto-categorization
Integration and error handling tests

Prometheus Metrics

Metrics exposed on /metrics:

feedback_bot_logs_total{platform,status,sentiment,category} - Total submissions
feedback_bot_slash_commands_total{command,platform} - Total slash commands
feedback_bot_request_duration_seconds{endpoint} - Request duration

Security

✅ Non-root container (UID 10001)
✅ Read-only root filesystem
✅ All capabilities dropped
✅ No security vulnerabilities (CodeQL verified)
✅ Secrets in Kubernetes secrets
✅ Optional token validation

Architecture

┌─────────────────────────────────────────┐
│           Mattermost                     │
│  User: /feedback The UI is great!       │
└────────────────┬────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────┐
│       Feedback Bot Service              │
│  ┌──────────┐  ┌──────────┐            │
│  │   NLP    │  │Sentiment │            │
│  │  Parser  │  │ Analyzer │            │
│  └──────────┘  └──────────┘            │
│  ┌──────────┐  ┌──────────┐            │
│  │Auto-Cat  │  │  Rating  │            │
│  │          │  │Extractor │            │
│  └──────────┘  └──────────┘            │
└────────────────┬────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────┐
│        Feedback Service API             │
│         PostgreSQL Database             │
└─────────────────────────────────────────┘

Acceptance Criteria Met

✅ Bot deployed - Kubernetes manifests created and validated
✅ Natural language processing - Smart parsing with multiple patterns
✅ Sentiment analysis - VADER integration with positive/neutral/negative
✅ Auto-categorization - 10 categories with keyword matching
✅ Integration with feedback system - API integration with error handling

Next Steps

Deploy to cluster:

kubectl apply -k platform/apps/feedback-bot/

Configure Mattermost slash command as documented above
Test bot:

/feedback Test message

Monitor metrics:
Check Prometheus for feedback_bot_* metrics
View logs: kubectl logs -n fawkes -l app=feedback-bot
View feedback in Backstage dashboard
Navigate to feedback analytics dashboard
View sentiment analysis and categorization

Files Changed

services/feedback-bot/ - New service implementation
platform/apps/feedback-bot/ - New Kubernetes manifests
tests/bdd/features/feedback-bot.feature - New BDD tests

Documentation

services/feedback-bot/README.md - Comprehensive setup and usage guide
Inline code documentation in app/main.py
BDD feature file with 15 test scenarios

Support

For issues or questions:

View logs: kubectl logs -n fawkes -l app=feedback-bot -f
Check health: curl http://feedback-bot:8000/health
View metrics: curl http://feedback-bot:8000/metrics