NatureMapr → iNaturalist uploader

A Python tool to import observations from NatureMapr CSV exports into iNaturalist.

Project Structure

The codebase is organized into modular components:

• main.py - Main entry point and CLI interface
• config.py - Configuration, environment variables, and constants
• observation_builder.py - Builds observation parameters from CSV rows
• file_downloader.py - Downloads images and audio files with retry logic
• inat_api.py - iNaturalist API interactions (create, delete, retries)
• utils.py - Utility functions (file cleanup, etc.)

Setup

1. Create a virtual environment:

   python3 -m venv venv

2. Activate it:

   source venv/bin/activate  # macOS/Linux
   # or
   venv\Scripts\activate     # Windows

3. Install dependencies:

   pip install -r requirements.txt

Configuration

API Token Setup

Get your iNaturalist API token from: https://www.inaturalist.org/users/api_token

You can set the token using either method:

Option 1: Using a .env file (Recommended)

1. Create a .env file in the project root directory
2. Add your token:

   INAT_TOKEN=your_token_here
   

3. The .env file is automatically loaded by the script (via python-dotenv)
4. Important: Make sure .env is in your .gitignore to keep your token secure

Option 2: Setting environment variable directly

• macOS/Linux (bash/zsh):

  export INAT_TOKEN=your_token_here

• Windows (PowerShell):

  $Env:INAT_TOKEN = "your_token_here"

Other Configuration

• Downloads of images/audio are saved to tmp_attachments/ and cleaned up after each upload
• Default timezone is Australia/Sydney (configurable in config.py)
• Default tags: NatureMapr, claire_import (configurable in config.py)

Usage

Basic Import

Import observations from a CSV file:

python main.py --csv path/to/NatureMapr-obs.csv

Command-Line Options

• --csv (required) - Path to the NatureMapr CSV export file
• -n, --dry-run - Test mode: don't upload anything, just print what would happen
• --delete - Delete observations by ID (comma-separated list)
• --limit N - Only process the first N rows
• --resume N - Start processing at row N (0-based index)

Examples

# Dry run to test
python main.py --csv data.csv --dry-run

# Import first 10 rows
python main.py --csv data.csv --limit 10

# Resume from row 50
python main.py --csv data.csv --resume 50

# Delete specific observations
python main.py --delete "12345,67890,11111"

CSV Format

The script expects a CSV file with these columns (minimum required):

Required Columns

• Scientific Name - Scientific name of the species
• Recorded Date Utc - Date in format like 4/28/20
• Recorded Time Utc - Time in 12-hour (1:51 AM) or 24-hour (14:00) format
• Place - Location name
• Lat - Latitude (decimal degrees)
• Long - Longitude (decimal degrees)

Optional Columns

• Image1 through Image5 - URLs to image files (missing values are skipped)
• Audio - URL to audio file
• Description Public - Public description text
• Abundance - Abundance value (observation field 647)
• Animal health - Animal health status (observation field 443)
• Plant health - Plant health status (observation field 443)
• Animal size - Animal size (observation field 779)
• Circumference of trunk - Trunk circumference (observation field 779)
• Plant height - Plant height (observation field 779)
• Flower size - Flower size (observation field 779)
• Gender - Gender information (observation field 41)

Notes

• Timezone is automatically converted to Australia/Sydney before sending to iNaturalist
• Image and audio URLs are automatically downloaded to temporary files before upload
• All temporary files are cleaned up after each observation is processed

Output

The script provides:

• Progress bar showing import status
• Per-row status messages
• Summary statistics at the end:
  • Total rows considered
  • Successfully created
  • Dry-run skipped
  • Failed

Example output:

Dry run: False
Rows to process: 100
Starting at row: 0
Importing observations: 100%|████████████| 100/100 [02:30<00:00]
Created iNat observation 123456 (CSV row 0)
Created iNat observation 123457 (CSV row 1)
...

=== IMPORT SUMMARY ===
Total rows considered: 100
Successfully created:  98
Dry-run skipped:       0
Failed:                2
======================

Troubleshooting

Timeouts

• The script automatically retries with exponential backoff
• To increase timeout, modify timeout parameter in file_downloader.py and inat_api.py

Download Failures

• Bad or inaccessible URLs will show warnings and be skipped
• Check that image/audio URLs are publicly accessible

Date Parsing Errors

• Ensure dates are in format: M/D/YY (e.g., 4/28/20)
• Times can be 12-hour (1:51 AM) or 24-hour (14:00) format

Missing Token

• Ensure INAT_TOKEN is set in .env file or as environment variable
• Check that .env file is in the project root directory

Development

Project Structure

• config.py - All configuration constants and environment setup
• observation_builder.py - Core logic for building observation parameters
  • parse_datetime() - Date/time parsing
  • download_media() - Media file handling
  • build_observation_fields() - Observation field mapping
  • make_observation_params() - Main builder function
• file_downloader.py - File downloading with retry logic
• inat_api.py - API interaction functions
  • create_observation_with_retry() - Create with retry logic
  • delete_observations() - Batch delete function
• utils.py - Utility functions
  • cleanup_temp_files() - Cleanup helper

Dependencies

• Dependencies are pinned in requirements.txt
• To refresh from your environment: pip freeze > requirements.txt

References

• How to upload many observations to iNaturalist
• pyinaturalist GitHub
• pyinaturalist Documentation