Fomo NYC

fomo.nyc is a free, community-built interactive map of upcoming events in the NYC area.

About

It works by visiting the websites of parks, museums, music venues, etc., identifying any upcoming events, and displaying them on a map. My hope is that this becomes a useful resource for people to find events they are interested in and to engage with their local communities.

Project Structure

Main Directories

/src/ Website source files

• src/js/ JavaScript modules (IIFE pattern, loaded in order)
• src/css/ CSS modules (imported via index.css)
• src/data/ Event and location JSON data (generated by pipeline)
• src/api/ PHP API endpoints
• src/admin/ PHP admin dashboard
• src/images/ SVG assets
• src/fonts/ Self-hosted MapLibre glyph fonts

/dist/ Build output (not committed)

• Generated by npm run build (prod) or npm run dev (dev)
• Contains everything needed for the server's public_html/ directory
• Uploaded to production via python scripts/upload_public_html.py

/pipeline/ Python scripts for data processing pipeline

• main.py Main entry point - orchestrates the complete workflow
• crawler.py Crawls websites using Crawl4AI, stores content in database
• extractor.py Uses Gemini AI to extract structured event data
  • Requires GEMINI_API_KEY environment variable (set in .env file)
• processor.py Parses extracted data, enriches with location coordinates
• merger.py Deduplicates events into final events table
• exporter.py Generates JSON files for the website (src/data/)
• uploader.py Uploads data files to FTP server
• db.py Database operations (CRUD for crawl runs, results, events)

/database/ Database schema and setup scripts

• schema.sql Complete database schema
• setup.py Creates empty database tables
• migrate_schema.py Applies schema changes to existing database
• /database/backups/ Database backup files

Build System

The frontend is built with esbuild:

npm run dev     # Dev build + watch (unminified, symlinks, stable filenames)
npm run build   # Prod build (minified, content-hashed, full asset copies)

• JS modules are concatenated in load order and minified (not bundled — they use the IIFE pattern with globals)
• CSS @import chain is resolved and minified
• Flatpickr is installed via npm and bundled into the output
• dist/ contains the complete deployable output: HTML, JS/CSS bundles, plus copies of data/, images/, fonts/, api/, admin/

Data Pipeline Flow

All data flows through the database (crawl_runs → crawl_results → crawl_events → events):

1. Crawl → Query websites table for due sites, crawl and store in crawl_results.crawled_content
2. Extract → Use Gemini AI to extract structured tables, store in crawl_results.extracted_content
3. Process → Parse tables, enrich with location data from locations, store in crawl_events
4. Merge → Deduplicate crawl_events into final events table
5. Export → Generate src/data/*.json from events table
6. Upload → Push JSON files to FTP server

Deployment

npm run build                          # Build to dist/
python scripts/upload_public_html.py   # Upload dist/ to server

How You Can Help

• 📢 Share with your friends

• 📍 Add events and places you know

• 🫱🏾‍🫲🏼 Stay in touch

  • This website is in active development, so keep visiting for regular updates!
  • You can reach out by email or join the Discord server

Database

The project uses a MariaDB/MySQL database to store locations, websites, and crawl data.

Initial Setup

New developers should restore from a database backup rather than starting with an empty database:

# 1. Create the database
mysql -u root -e "CREATE DATABASE fomo CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"

# 2. Restore from backup
mysql -u root fomo < database/backups/fomo_backup_YYYYMMDD.sql

Creating Backups

# Windows (XAMPP)
"C:/xampp/mysql/bin/mysqldump.exe" -u root fomo > database/backups/fomo_backup_YYYYMMDD.sql

# Linux/Mac
mysqldump -u root fomo > database/backups/fomo_backup_YYYYMMDD.sql

Schema Updates

If the schema has changed since your backup, run migrations:

python database/migrate_schema.py

Acknowledgements

• 🧭 Map library: MapLibre GL JS
• 🗺️ Map tiles: © Protomaps, © OpenStreetMap contributors
• 📅 Date picker: Flatpickr
• 🔠 Fonts: Inter, Noto Color Emoji
• 🚀 This project uses Crawl4AI for web data extraction
• 🤖 Gemini and Claude for data processing and vibe coding
• 💖 All the amazing, creative, hard-working people who make the city shine with their light! 🗽