Automating Nightly GeoJSON Rebuilds with GitHub Actions

Automating nightly GeoJSON rebuilds with GitHub Actions eliminates manual data exports by running a cron-triggered pipeline that fetches raw spatial data, transforms it into a standardized FeatureCollection, validates it against the RFC 7946 GeoJSON specification, and commits the result directly to your repository. This serverless approach removes infrastructure overhead while guaranteeing reproducible, version-controlled datasets for web maps and dashboards. When paired with cache-aware frontend routing, this pattern becomes a foundational element of Scheduled Map Rebuild Workflows and scales seamlessly into broader enterprise architectures.

Core Workflow Configuration

Place the following YAML in .github/workflows/nightly-geojson.yml. It triggers daily at 02:00 UTC, installs dependencies, runs the transformation, validates the output, and commits changes only when the file actually differs from the previous state.

name: Nightly GeoJSON Rebuild
on:
  schedule:
    - cron: '0 2 * * *' # Runs daily at 02:00 UTC
  workflow_dispatch:    # Manual trigger for debugging
permissions:
  contents: write
jobs:
  rebuild:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          persist-credentials: true
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - name: Install dependencies
        run: npm ci
      - name: Fetch & Transform Data
        env:
          SOURCE_API_URL: ${{ secrets.SOURCE_API_URL }}
          API_TOKEN: ${{ secrets.API_TOKEN }}
        run: node scripts/rebuild-geojson.js
      - name: Validate GeoJSON
        run: npx @mapbox/geojsonhint data/output.geojson
      - name: Commit & Push
        run: |
          git config user.name "github-actions[bot]"
          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
          git add data/output.geojson
          git diff --staged --quiet || git commit -m "chore: nightly GeoJSON rebuild [skip ci]"
          git push

Key configuration notes:

  • persist-credentials: true ensures the default GITHUB_TOKEN retains write access for the final push step.
  • The git diff --staged --quiet guard prevents empty commits, preserving repository history and conserving runner minutes.
  • [skip ci] in the commit message stops GitHub Actions from recursively triggering dependent workflows on the same push.
  • Secrets are injected via env to keep credentials out of logs and source control.

Production-Ready Transformation Script

The script below handles data fetching, coordinate normalization, and file output. It uses ES modules, reduces floating-point precision to shrink payload size, and exits with a non-zero status on failure so the workflow halts before committing invalid data.

// scripts/rebuild-geojson.js
import fs from 'fs/promises';
import * as turf from '@turf/turf';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';

const __dirname = dirname(fileURLToPath(import.meta.url));
const OUT_PATH = join(__dirname, '../data/output.geojson');

async function rebuild() {
  try {
    // 1. Fetch source data with authentication
    const res = await fetch(process.env.SOURCE_API_URL, {
      headers: { 'Authorization': `Bearer ${process.env.API_TOKEN}` }
    });
    if (!res.ok) throw new Error(`Source API returned ${res.status}`);
    const raw = await res.json();

    // 2. Map to GeoJSON FeatureCollection
    const features = raw.items.map(item => ({
      type: 'Feature',
      properties: {
        id: item.id,
        category: item.category,
        updated_at: new Date().toISOString()
      },
      geometry: {
        type: 'Point', // Switch to LineString/Polygon as your schema requires
        coordinates: [item.lng, item.lat]
      }
    }));

    const collection = { type: 'FeatureCollection', features };

    // 3. Spatial optimization: truncate coordinates to 5 decimal places (~1m precision)
    const optimized = turf.truncate(collection, { precision: 5 });

    // 4. Write to disk
    await fs.writeFile(OUT_PATH, JSON.stringify(optimized, null, 2), 'utf8');
    console.log(`✅ Successfully wrote ${features.length} features to ${OUT_PATH}`);
  } catch (error) {
    console.error('❌ GeoJSON rebuild failed:', error.message);
    process.exit(1);
  }
}

rebuild();

Why truncate coordinates? Raw GPS coordinates often contain 12+ decimal places. Reducing precision to 5 decimals drops file size by 30–50% with zero perceptible impact on web map rendering, directly improving dashboard load times and CDN cache efficiency.

Validation, Idempotency & Execution Guarantees

Validation is your safety net. Running @mapbox/geojsonhint catches structural violations like missing type keys, malformed coordinate arrays, or invalid CRS declarations before they reach production. For stricter schema enforcement, pair it with ajv and a custom JSON Schema that enforces your business rules (e.g., required property fields, numeric bounds).

GitHub’s scheduler does not guarantee exact-minute execution. During high-traffic periods, cron jobs may be delayed by up to 15 minutes. Review the official GitHub Actions schedule event documentation to understand execution windows and implement retry logic or webhook-driven fallbacks if your dashboard requires strict SLAs.

To test locally before committing:

  1. Run npm ci && node scripts/rebuild-geojson.js in your terminal.
  2. Validate output with npx @mapbox/geojsonhint data/output.geojson.
  3. Use git diff to verify only expected changes are staged.

Scaling to Enterprise Dashboards

Once the GeoJSON is committed, modern frontend frameworks can consume it via static asset paths or CDN-backed URLs. Implement Cache-Control: public, max-age=86400, stale-while-revalidate=3600 on your hosting layer so browsers and edge networks cache aggressively while still picking up the nightly rebuild. Pair this with ETag validation to avoid redundant network requests when the dataset hasn’t changed.

This architecture naturally extends into broader Data Refresh & Automation Pipelines when you add downstream steps like database seeding, Slack alerts on validation failures, or automated PR creation for staging environments. For teams managing multiple map layers, convert the workflow to a matrix strategy to rebuild several datasets in parallel without duplicating YAML. Combine this with deterministic hashing (e.g., sha256sum data/output.geojson) to trigger frontend cache busting only when spatial content actually changes, ensuring dashboard hydration remains fast and predictable.