Decentralized Search over Personal Online Datastores: Architecture and Performance Evaluation

Data privacy and sovereignty are open challenges in today’s Web, which the Solid ecosystem aims to meet by providing personal online datastores (pods) where individuals can control access to their data. Solid allows developers to deploy applications with access to data stored in pods, subject to users’ permission. For the decentralised Web to succeed, the problem of search over pods with varying access permissions must be solved. The ESPRESSO framework takes the first step in exploring such a search architecture, enabling large-scale keyword search across Solid pods with varying access rights. This paper provides a comprehensive experimental evaluation of the performance and scalability of decentralised keyword search across pods on the current ESPRESSO prototype. The experiments specifically investigate how controllable experimental parameters influence search performance across a range of decentralised settings. This includes examining the impact of different text dataset sizes (0.5MB to 50MB per pod, divided into 1 to 10,000 files), different access control levels (10%, 25%, 50%, or 100% file access), and a range of configurations for Solid servers and pods (from 1 to 100 pods across 1 to 50 servers). The experimental results confirm the feasibility of deploying a decentralised search system to conduct keyword search at scale in a decentralised environment.

@inproceedings{ragab_espresso_icwe_2024,
  author = {Ragab, Mohamed and Savateev, Yury and Oliver, Helen and Tiropanis, Thanassis and Poulovassilis, Alex and Chapman, Adriane and Taelman, Ruben and Roussos, George},
  title = {Decentralized Search over Personal Online Datastores: Architecture and Performance Evaluation},
  booktitle = {Proceedings of the 24th International Conference on Web Engineering},
  year = {2024},
  month = jun
}