The Common Crawl Foundation participated at the 30th International Conference on Theory and Practice of Digital Libraries (TPDL 2026) in Faro, Portugal, from 22nd to 25th of September.
The conference was hosted by the University of Algarve at its Penha Campus, on the edge of the Ria Formosa Natural Park. The programme opened with a day of workshops and a doctoral consortium, followed by three main conference days of keynotes, eight paper sessions, and a dedicated demo session.
On the 23rd of September, Michael Paris presented the "Measuring What the Crawler Sees: Discovery Curves, Core Persistence, and Shell Dynamics in Longitudinal Web Crawls", co-authored with Hande Çelikkanat and Luca Foppiano, in the session on “Studying the Research Landscape: Scientometrics and Meta-Research”. The preprint is available on arXiv. The data used for this work is published in the Common Crawl Statistics.
On the 24th of September, Luca Foppiano presented a collaborative work "The INRIA DataLake: A Generic and Scalable Ecosystem of Pipelines for HAL Applied to Software Mentions Tracking" in the demo session, submitted with Vipul Gupta (Hereon), Samuel Scalbert (Inria), Kumar Guha (Inria), Yannick Barborini (CCSD), Alain Monteil (Inria), and Laurent Romary (Inria). The preprint is available on arXiv.
TPDL is the sort of conference that gets librarians, archivists, data engineers, and NLP researchers talking to one another, often leading to collaborations across disciplines.
In conversation with the Portuguese Web-Archive Arquivo.pt we discussed Common Crawl’s operations and our initiatives around languages, data quality and the importance of web preservation. Arquivo.pt has preserved Portuguese web content since 1996 and was formally established at FCCN in 2007.
The Common Crawl Foundation welcomes and encourages donations of seeds as URLs in any language with particular attention to low resource languages. Contributors may use our web-languages repository to suggest new seeds for the crawl.
We had further discussions with representatives from OpenAIRE and the ORKG on the importance of using graphs to create equitable recognition in academic publications.
We plan to release a follow up blog post that includes an interactive deep dive on the presented estimator of the policy induced size of the crawl sample.
Stay tuned.
.svg)
