Google Photos Takeout Tool 📸

A fast, lightweight Node.js utility designed to scan, match, and map Google Photos Takeout media archives with their corresponding JSON metadata sidecar files.

---

🌟 The Problem

When exporting your library with Google Takeout, Google Photos provides media alongside accompanying .json metadata sidecars. However, Takeout notoriously mangles filenames:

• 51-Character Truncation: Google Takeout cuts filenames and extensions off after 51 characters (e.g. long_filename_from_camera.jp.json).
• Suffix & Index Chaos: Duplicate names receive arbitrary suffix indexing like photo(1).jpg, photo.jpg(1).json, or photo(1).json.
• Special Characters: Reserved characters (/, \, :, *, ?, ", <, >, |, %, &) are sanitized or replaced with underscores (_).
• Missing Extensions: Metadata titles sometimes omit file extensions entirely.
• Motion Photos, Bursts & Edits: Auxiliary files such as motion video tracks (MVIMG_*.mp4, .MP, MP~1), edited copies (-edited.jpg), and burst shots (00100dPORTRAIT_*_BURST*) are exported without their own dedicated JSON files.

This tool solves these quirks by analyzing directory structures, resolving edge cases, grouping auxiliary files, and exporting clean CSV mapping reports.

---

🚀 Features

• ⚡ High Throughput Scanning: Efficiently handles massive libraries containing hundreds of thousands of photos and videos.
• 🎯 Heuristic Matching Engine:
  • Handles 51-character length truncation.
  • Recovers missing extensions (.jpg, .png, .gif, .jpeg).
  • Normalizes illegal characters.
  • Resolves (n) suffix collisions.
• 🎞️ Auxiliary & Motion Media Pairing: Links Live/Motion photos (MVIMG, .MP), burst sequences, and edited versions back to their primary JSON metadata.
• 📊 CSV Export Reports:
  • match-json.csv: Direct JSON ➔ Media file mapping.
  • match-files.csv: Media file ➔ JSON mapping.
  • match-json-extra.csv: JSON ➔ Primary + auxiliary files (motion videos, bursts, edits).
• 🧪 Skeleton Mock Generator: Create a lightweight 0-byte dummy replica of your entire Takeout archive (preserving full folder hierarchy and JSON content) for safe, rapid testing without terabytes of real media files.
• 📊 Real-time Terminal Progress: Visual status bar and progress indicator powered by log-with-statusbar.
• ⚙️ Configurable Overrides: Map difficult custom filenames and extra files via simple lookup tables.

---

📁 Output Reports

After processing your Takeout folder, CSV reports are written to the output/ directory:

File	Description
output/match-json.csv	Maps each metadata JSON sidecar path to its matching primary media file path.
output/match-files.csv	Reverse lookup mapping each media file path to its source JSON metadata file.
output/match-json-extra.csv	Maps a JSON metadata file to all associated media files, including motion clips, bursts, and edited versions.

---

🛠️ Project Structure

takeout-tool/
├── config/
│   ├── general.js          # Input paths and CSV export target paths
│   └── photos.js           # Custom edge-case mapping tables
├── output/                 # Generated CSV reports
├── scripts/
│   └── create-skeleton.js  # Utility to create 0-byte mock archive for testing
├── src/
│   ├── index.js            # CLI entry point and command router
│   ├── photos.js           # Main Google Photos matching and scanning engine
│   └── progress.js          # Terminal progress bar and logging manager
├── mise.toml               # Tooling & Node version configuration
├── package.json
└── README.md

---

📦 Installation & Requirements

• Node.js: v18.0.0 or higher (uses native ES Modules and node:util parseArgs).

1. Clone the repository:

   git clone https://github.com/nicufarmache/takeout-tool.git
   cd takeout-tool

2. Install dependencies:

   npm install

---

🚦 Usage

1. Configure Input Paths

Open config/general.js to configure where your Takeout directory or skeleton is located:

export const startPath = './takeout-skeleton'; // Path containing 'Takeout/Google Photos'
export const csvPathJson = './output/match-json.csv';
export const csvPathFiles = './output/match-files.csv';
export const csvPathExtra = './output/match-json-extra.csv';

2. Run the Matching Tool

Run the matcher against your configured directory:

npm run photos

or

node src/index.js photos

You'll see real-time console progress as it scans directories, resolves JSON pairs, checks for unmatched/auxiliary files, and writes the CSVs.

---

🧪 Testing with Mock Skeletons

Processing real multi-gigabyte/terabyte Google Takeout dumps can be slow and memory-intensive during testing. You can generate a takeout skeleton that replicates the exact folder structure and JSON sidecars while replacing heavy media files with 0-byte dummy placeholders:

npm run skeleton -- /path/to/real/Takeout [output-skeleton.zip]

Example:

node scripts/create-skeleton.js "/Volumes/Backup/Google Takeout" takeout-skeleton.zip

Unzip takeout-skeleton.zip into your project directory and set startPath in config/general.js to ./takeout-skeleton for instant testing.

---

⚙️ Custom Overrides

If you encounter unique filenames that cannot be resolved automatically, define manual mappings in config/photos.js:

• CustomJsonToFile: Map specific JSON basenames to their actual media filenames.
• CustomExtraFilesToJson: Map extra/unmatched media files to their parent media files.

export const CustomJsonToFile = new Map([
  ['2014-02-28(1)', '2014-02-28.jpg'],
  ['2014-02-28', '2014-02-28.png'],
]);

export const CustomExtraFilesToJson = new Map([
  ['00100dPORTRAIT_00100_BURST20180203004137338_COV(2).jpg', '00100dPORTRAIT_00100_BURST20180203004137338_COV(1).jpg'],
]);

---

📄 License

This project is licensed under the ISC License.