Web and Blog datasets
Memetracker data
MemeTracker is an approach for extracting short textual phrases from web documents (news articles and blog posts) and then tracking how such prases spread over the Web and how they change and evolve as they spread.
MemeTracker data contains two datasets:
- Phrase cluster data: The data contains all variants of different phrases and corresponding URLs wiht time stamps where these phrases appeared.
- Raw phrases data: The data contains timestamped phrase and link information for news media articles and blog posts from 1.5 million different blogs and news websites. The data spans 10 monts from August 2008 till May 2009, with several hundred million documents.
ICWSM 2009 Spinn3r data
A collection of raw blog posts and news media articles collected by Spinn3r and released as a part of International Conference on Weblogs and Social Media 2009.
- ICWSM online news document data: includes mainstream news media articles and blogs.
Stanford WebBase web crawls
A collection of web crawls from Stanford InfoLab. The web crawls go almost 10 years back.
- Stanford web crawls: several big web crawls.