Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
name: Docker Build and Publish

on:
push:
branches: [ main ]
tags: [ 'v*' ]
pull_request:
branches: [ main ]

env:
REGISTRY: docker.io
IMAGE_NAME_SERVER: observo/server
IMAGE_NAME_DASHBOARD: observo/dashboard

jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_SERVER }}
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DASHBOARD }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha

- name: Build and push Server image
uses: docker/build-push-action@v5
with:
context: ./server
file: ./server/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max

- name: Build and push Dashboard image
uses: docker/build-push-action@v5
with:
context: ./client-dashboard
file: ./client-dashboard/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
196 changes: 196 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
# Observo - Self-Hosted Log Monitoring System

🚀 **Observo** is a complete self-hosted log monitoring and analytics platform that provides real-time log aggregation, storage, and visualization.

## 🚀 Quick Start (No Repository Clone Required!)

### Option 1: One-Command Installation
```bash
curl -sSL https://raw.githubusercontent.com/daddycoder007/Observo/main/get-observo.sh | bash
cd observo
./quick-start.sh
```

### Option 2: Manual Setup
```bash
# Create directory and download files
mkdir observo && cd observo
curl -o docker-compose.yml https://raw.githubusercontent.com/daddycoder007/Observo/main/docker-compose.yml
curl -o quick-start.sh https://raw.githubusercontent.com/daddycoder007/Observo/main/quick-start.sh
chmod +x quick-start.sh

# Start Observo
./quick-start.sh
```

### Option 3: Direct Docker Compose
```bash
# Download and run
curl -o docker-compose.yml https://raw.githubusercontent.com/daddycoder007/Observo/main/docker-compose.yml
docker-compose up -d
```

## 🏗️ Architecture

```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Log Files │───▶│ Log Agent │───▶│ Kafka │───▶│ Server │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Dashboard │◀───│ Web UI │◀───│ REST API │◀───│ MongoDB │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
```

## 📦 Components

### 1. Log Agent (`@observo/log-agent`)
Install the log agent to start monitoring your log files:

```bash
npm install -g @observo/log-agent
```

Create a configuration file:
```yaml
kafka:
brokers: localhost:9092
clientId: my-log-agent

files:
- path: /var/log/myapp.log
topic: myapp-logs
tag: output
- path: /var/log/error.log
topic: myapp-logs
tag: error
```

Start monitoring:
```bash
observo-agent config.yaml
```

### 2. Observo Server (Published Docker Image)
- **Image**: `observo/server:latest`
- **Port**: 3000
- **Features**: Kafka consumer, MongoDB storage, REST API, WebSocket

### 3. Observo Dashboard (Published Docker Image)
- **Image**: `observo/dashboard:latest`
- **Port**: 80
- **Features**: React-based web interface, real-time logs, analytics

## ⚙️ Configuration

### Environment Variables (.env file)
```env
# MongoDB Configuration
MONGO_ROOT_USERNAME=admin
MONGO_ROOT_PASSWORD=password123
MONGO_DATABASE=observo
MONGO_PORT=27017

# Kafka Configuration
KAFKA_TOPIC_PREFIX=observo-logs
KAFKA_PORT=9092

# Server Configuration
SERVER_PORT=3000
DASHBOARD_PORT=80

# Kafka Topics
KAFKA_TOPICS=observo-logs.output,observo-logs.error,observo-logs.access
```

## 🐳 Docker Images

All images are automatically published to Docker Hub:

- **observo/server**: `docker pull observo/server:latest`
- **observo/dashboard**: `docker pull observo/dashboard:latest`

### Available Tags
- `latest` - Latest stable version
- `v1.0.0` - Specific version
- `main` - Latest from main branch

## 📊 Access Points

After starting Observo:

- **Dashboard**: http://localhost
- **API**: http://localhost:3000/api
- **MongoDB**: localhost:27017
- **Kafka**: localhost:9092

## 🛠️ Management Commands

```bash
# Start Observo
./quick-start.sh

# Stop Observo
./stop.sh

# View logs
./logs.sh

# Update to latest version
docker-compose pull
docker-compose up -d
```

## 📊 API Documentation

### Base URL: `http://localhost:3000/api`

#### Get Logs
```http
GET /api/logs?page=1&limit=50&level=error
```

#### Get Statistics
```http
GET /api/logs/stats?startDate=2024-01-01&endDate=2024-01-31
```

#### Get Analytics
```http
GET /api/analytics?startDate=2024-01-01&endDate=2024-01-31
```

## 🔒 Security

### Production Deployment
1. Change default passwords in `.env`
2. Use environment variables for sensitive data
3. Configure firewall rules
4. Enable HTTPS (use reverse proxy)
5. Use Kafka authentication if needed

### Example Production .env
```env
MONGO_ROOT_USERNAME=observo_admin
MONGO_ROOT_PASSWORD=your_secure_password_here
KAFKA_TOPIC_PREFIX=your-company-logs
```

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

## 📄 License

MIT License - see [LICENSE](LICENSE) file for details.

## 🆘 Support

- 📖 [Documentation](https://github.com/daddycoder007/Observo/wiki)
- 🐛 [Issues](https://github.com/daddycoder007/Observo/issues)
- 💬 [Discussions](https://github.com/daddycoder007/Observo/discussions)
29 changes: 29 additions & 0 deletions client-dashboard/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Build stage
FROM node:18-alpine AS builder

WORKDIR /app

# Copy package files
COPY package*.json ./

# Install dependencies
RUN npm ci

# Copy source code
COPY . .

# Build the app
RUN npm run build

# Production stage
FROM nginx:alpine

# Copy built app to nginx
COPY --from=builder /app/build /usr/share/nginx/html

# Copy nginx configuration
COPY nginx.conf /etc/nginx/conf.d/default.conf

EXPOSE 80

CMD ["nginx", "-g", "daemon off;"]
33 changes: 33 additions & 0 deletions client-dashboard/nginx.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
index index.html;

# Handle React Router
location / {
try_files $uri $uri/ /index.html;
}

# API proxy to backend
location /api/ {
proxy_pass http://host.docker.internal:3000/api/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_cache_bypass $http_upgrade;
}

# WebSocket proxy
location /ws {
proxy_pass http://host.docker.internal:3000/ws;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
}
4 changes: 4 additions & 0 deletions client-dashboard/src/components/Sidebar.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
import {
Dashboard as DashboardIcon,
ReceiptLong as LogsIcon,
Settings as SettingsIcon,
ChevronLeft,
ChevronRight,
QueryStats,
Expand Down Expand Up @@ -72,6 +73,9 @@ const Drawer = styled(MuiDrawer, { shouldForwardProp: (prop) => prop !== 'open'
const menuItems = [
{ text: 'Dashboard', icon: <DashboardIcon />, path: '/' },
{ text: 'Logs', icon: <LogsIcon />, path: '/logs' },
{ text: 'Analytics', icon: <QueryStats />, path: '/analytics' },
{ text: 'Settings', icon: <SettingsIcon />, path: '/settings' },

];

interface SidebarProps {
Expand Down
2 changes: 1 addition & 1 deletion client-dashboard/src/services/api.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import axios from 'axios';

const API_BASE_URL = 'http://localhost:8000/api';
const API_BASE_URL = 'http://localhost:3000/api';

// Create axios instance with base configuration
const apiClient = axios.create({
Expand Down
Loading
Loading