SNMP API

A small SNMP v2-based data collection and API service for RWIS NTCIP 1204 weather stations. It periodically polls configured stations, stores normalized observations in a local SQLite database, and exposes a simple HTTP endpoint to retrieve the latest data.

Supported station types: LX, Lufft, and RWS200 (others may work if they comply with NTCIP 1204 and provide compatible OIDs).

How it works

• Scheduler (start.py) triggers fetch jobs every 5 minutes (at :05, :10, :15, ... :55). Each run:
  • Reads stations from config.json
  • Performs SNMP getBulk queries (fetch_bulk_v2.py)
  • Decodes/normalizes values (data_processor.py)
  • Stores flattened records in SQLite (database.py, fetcher.db)
  • Cleans up rows older than 5 minutes (db_cleanup.py)
• A Flask API (API.py) is served by waitress on port 9000 and returns observations from the last ~6 minutes.

Quick start

1. Prerequisites
   • Python 3.12 (recommended)
   • Windows (tested), should work on other OSes with minor changes
2. Create and activate a virtual environment
   • PowerShell:
     • python -m venv .venv
     • ..venv\Scripts\Activate.ps1
3. Install dependencies
   • pip install -r requirements.txt
4. Initialize the database
   • python setup.py
5. Configure stations
   • Edit config.json and add your stations (see Configuration below).
6. Start the service (scheduler + API)
   • python start.py
   • API will listen on http://0.0.0.0:9000

Configuration

• config.json
  • stations: Array of station objects with the following fields:
    • IPaddress: string (IP or hostname)
    • snmp_port: number (default 161)
    • SysTyp: "LX" | "Lufft" | "RWS200" (determines the starting OID range)
    • Site Name: string (human-readable)
    • Local Description: string
    • latitude: number
    • longitude: number
    • Station_ID: string or GUID (used as uuid)
• push.json (optional)
  • Reserved for future push/integration workflows.

Example config.json stations entry:

{
  "stations": [
    {
      "IPaddress": "192.0.2.10",
      "snmp_port": 161,
      "SysTyp": "Lufft",
      "Site Name": "HWY 1 MM 23",
      "Local Description": "Bridge deck sensor",
      "latitude": 35.1234,
      "longitude": -97.5678,
      "Station_ID": "station-0001"
    }
  ]
}

Running

• Start everything:
  • python start.py
• Just perform a one-off fetch (without scheduler/API):
  • From Python code, call fetcher.fetch_data()
• Note: The scheduler runs jobs every 5 minutes (excluding :00). The API and scheduler run in separate threads.

API

• Base URL: http://localhost:9000
• GET /fetcher
  • Returns a JSON array, one object per station with latest fields for the last ~6 minutes. If no data, returns {"status": "FAILED"}.

Example response: [ { "uuid": "station-0001", "Date_Time": "2025-09-06 19:05:00", "Air_Temperature": "12.3", "Wind_Speed": "5.6", "...": "..." } ]

Data model

• SQLite DB file: fetcher.db
• Table: weather_data (see setup.py)
  • ID INTEGER PRIMARY KEY AUTOINCREMENT
  • uuid TEXT
  • Date_Time TEXT (UTC, YYYY-MM-DD HH:MM:SS)
  • Data_Name TEXT (normalized key)
  • Data TEXT (value)

Rows older than 5 minutes are periodically deleted by db_cleanup.py.

Troubleshooting

• No data returned / status FAILED
  • Ensure setup.py was run to create the table.
  • Verify stations in config.json are reachable and community is 'public'.
  • Check Windows Firewall and SNMP ACLs for UDP/161.
  • Ensure the scheduler has run at least once after startup (wait up to 5 minutes).
• ImportError or dependency issues
  • Reinstall using: pip install --upgrade -r requirements.txt
• Port already in use
  • Another service may be listening on 9000. Edit start.py start_server() to change port.

Development

• Code style: standard Python 3.12
• Entry points:
  • fetcher.fetch_data(): perform one fetch cycle
  • start.py: scheduler + waitress API server
  • API.py: Flask app with /fetcher
• Large decoding logic lives in data_processor.py; SNMP bulk logic in fetch_bulk_v2.py.

Security notes

• Uses SNMP v2c with community 'public' (see fetch_bulk_v2.py). Consider restricting network access or upgrading to SNMPv3 for production.
• API is unauthenticated and bound to 0.0.0.0 by default; place behind a firewall or reverse proxy if exposed.

License

MIT

Acknowledgements

• pysnmp, Flask, pandas, waitress, schedule