A researcher scraped 4.5 billion TikTok videos through the app's private API and put them on Hugging Face

A pseudonymous developer using the handle kuben-developer has published a 4.5-billion-record dataset of TikTok video metadata -- 289GB across 27 compressed files -- to Hugging Face, the same open-model hub NVIDIA agreed this week to buy for $12.93 billion. The data was collected over about three weeks through TikTok's private Android-app API rather than its public web endpoints, using anonymous device registrations and cryptographically signed requests that mimic the app itself -- no logins, accounts or session cookies involved, according to the dataset's own documentation.

Each of the dataset's 4,501,811,789 rows carries 14 fields: a caption, tagged account IDs, view/like/comment/share/save counts, duration, sound metadata, inferred country and language, and whether the post was an ad. No video or image content is included -- at roughly 60 bytes per row, this is metadata only. The dataset's own card is unusually direct about what that metadata means: "Captions are written by real people and this dataset is personal data under GDPR ... Do not use this to identify, profile, target or contact individuals." It also states plainly that "collection was contrary to TikTok's terms of service," and that the uploader has "no affiliation with, endorsement by, or connection to TikTok or ByteDance."

Reverse-engineering ByteDance's app-signing scheme -- internally named X-Argus, X-Gorgon and X-Ladon -- to spoof legitimate device requests is itself a known, difficult technique. A discussion of the release on Hacker News questioned the commercial value of scraping code being sold separately, but didn't dispute that the underlying dataset is what it claims to be. As of publication, ByteDance had not issued any public response, and the dataset remained live on Hugging Face under a research-and-education-only license.