Track any topic across Hacker News, Reddit, and Wikipedia — get live sentiment analysis, trend detection, and audience insights in one dashboard.
You want to know what the internet thinks about bitcoin, OpenAI, or Tesla — right now, not yesterday. SocialPulse continuously ingests public data from three major sources, runs NLP sentiment analysis on every message, and renders 10+ live-updating visualizations so you can spot momentum shifts, polarization spikes, and emerging narratives as they happen.
No API keys required to start — Hacker News and Wikipedia work out of the box. Add Reddit credentials for even richer coverage.
| Widget | What it shows |
|---|---|
| Sentiment Line Chart | Rolling average sentiment over time with positive/negative/neutral breakdown |
| Hype Score Gauge | Composite 0–100 score combining message volume, sentiment intensity, and Wikipedia pageviews |
| Polarization Meter | How divided the conversation is — high when opinions cluster at extremes |
| Source Volume | Bar chart comparing message counts across Hacker News, Reddit, and Wikipedia |
| Word Cloud | Most frequent terms in the conversation (stopwords and topic terms filtered out) |
| Wikipedia Pageviews | 30-day sparkline of article traffic — a proxy for mainstream interest |
| Timeline with Spike Detection | Hourly activity chart that flags statistically unusual bursts (>2 sigma) |
| Activity Heatmap | 7x24 grid showing when the conversation is hottest by day-of-week and hour |
| Message Feed | Live scrolling feed with sentiment color-coding and source badges |
| Sentiment Pie Chart | Aggregate distribution of positive, negative, and neutral messages |
- Dual sentiment engines — switch between VADER (fast, rule-based) and a DistilBERT transformer (ML-based) at runtime
- Real-time via WebSocket — messages, stats, and analytics stream through GraphQL subscriptions
- Graceful degradation — runs fine without Reddit credentials; every widget has skeleton loaders and error states
- Polling + dedup — fetchers poll at tuned intervals (HN 30s, Reddit 60s, Wikipedia 5min) and deduplicate by source ID
Hacker News API ──┐
│ ┌───────────────┐ GraphQL/WS ┌───────────────┐
Reddit OAuth API ──┼────────▶│ Analyzer │◀─────────────────────▶│ Gateway │
│ │ FastAPI │ REST │ Apollo/Bun │
Wikipedia API ─────┘ │ :8000 │ │ :4000 │
└──┬─────────┬──┘ └───────┬───────┘
│ │ │
┌──────▼──┐ ┌───▼──────────┐ ┌───────▼───────┐
│ Redis │ │ PostgreSQL │ │ Frontend │
│ :6379 │ │ :5432 │ │ Next.js │
└─────────┘ └──────────────┘ │ :3000 │
└───────────────┘
Data flow: Fetchers poll external APIs → Analyzer runs sentiment analysis → Messages stored in PostgreSQL → Updates published via Redis pub/sub → Gateway translates to GraphQL subscriptions → Frontend renders in real time.
| Layer | Technologies |
|---|---|
| Frontend | Next.js 16, React 19, TypeScript, Tailwind CSS 4, Shadcn/UI, Recharts, Framer Motion, Apollo Client |
| Gateway | Bun, Apollo Server 4, graphql-ws, ioredis |
| Analyzer | Python 3.12, FastAPI, NLTK/VADER, HuggingFace Transformers, SQLAlchemy 2, httpx, asyncpg |
| Database | PostgreSQL 16 |
| Pub/Sub | Redis 7 |
| Infra | Docker Compose, uv (Python), pnpm workspaces |
docker compose up -dThis starts PostgreSQL (5432) and Redis (6379).
cd services/analyzer
uv sync
uv run uvicorn app.main:app --reload --port 8000cd apps/gateway
bun install
bun run devGraphQL playground available at http://localhost:4000/graphql.
cd apps/web
pnpm install
pnpm devOpen http://localhost:3000, type a topic, and watch it come alive.
docker compose --profile full up -dThis also starts the analyzer inside Docker (port 8000). You still need to run the gateway and frontend locally.
- Open
http://localhost:3000and type a topic (e.g. bitcoin) - Click Monitor — the analyzer starts polling Hacker News, Wikipedia, and Reddit (if configured)
- Watch the dashboard populate with real-time charts, stats, and messages
- Toggle between VADER and Transformer sentiment modes in the header
- Click Stop when done — fetchers shut down and the data stays in PostgreSQL
SocialPulse works without Reddit. Adding credentials unlocks Reddit as a third data source.
- Go to reddit.com/prefs/apps
- Click Create App → choose script type
- Name it
socialpulse-local, set redirect URI tohttp://localhost:8080 - Copy the client ID (under app name) and secret
Set the environment variables before starting the analyzer:
# Linux / macOS
export SP_REDDIT_CLIENT_ID="your_client_id"
export SP_REDDIT_CLIENT_SECRET="your_secret"
export SP_REDDIT_USER_AGENT="SocialPulse/1.0 (by u/your_username)"# Windows PowerShell
$env:SP_REDDIT_CLIENT_ID="your_client_id"
$env:SP_REDDIT_CLIENT_SECRET="your_secret"
$env:SP_REDDIT_USER_AGENT="SocialPulse/1.0 (by u/your_username)"All settings use the SP_ prefix and can be set via environment variables:
| Variable | Default | Description |
|---|---|---|
SP_DATABASE_URL |
postgresql+asyncpg://...localhost/socialpulse |
PostgreSQL connection string |
SP_REDIS_URL |
redis://localhost:6379 |
Redis connection string |
SP_SENTIMENT_MODE |
vader |
Sentiment engine: vader or transformer |
SP_HN_POLL_INTERVAL |
30 |
Hacker News polling interval (seconds) |
SP_WIKIPEDIA_POLL_INTERVAL |
300 |
Wikipedia polling interval (seconds) |
SP_REDDIT_POLL_INTERVAL |
60 |
Reddit polling interval (seconds) |
SP_CORS_ORIGINS |
* |
Allowed CORS origins (comma-separated) |
SocialPulse/
├── apps/
│ ├── web/ # Next.js frontend
│ │ ├── src/app/ # Pages (landing + dashboard)
│ │ ├── src/components/ # 13 visualization components
│ │ └── src/lib/graphql/ # Queries, mutations, subscriptions
│ └── gateway/ # GraphQL gateway
│ ├── src/schema/ # Type definitions + resolvers
│ └── src/services/ # Analyzer REST client, Redis pub/sub
├── services/
│ └── analyzer/ # Python sentiment analysis service
│ ├── app/api/ # REST endpoints
│ ├── app/core/ # Config, database, logging
│ ├── app/models/ # Message + Pageview ORM models
│ ├── app/services/ # Sentiment, analytics, fetchers, publisher
│ └── Dockerfile
├── docker-compose.yml # PostgreSQL + Redis (+ optional analyzer)
└── package.json # pnpm workspace scripts
MIT