Efficient Continuous Big Data Integrity Checking for Decentralized Storage

Decentralized storage powered by blockchain is becoming a new trend that allows data owners to outsource their data to remote storage resources offered by various storage providers. Unfortunately, unqualified storage providers easily encounter unpredictable downtime due to security threats, such as malicious attacks or system failures, which is unacceptable in many real-time or data-driven applications. As a result, continuous data integrity should be guaranteed in decentralized storage, which ensures that data is intact and available for the entire storage period. However, this requires frequent checking for long time periods and incurs heavy burdens of both communication and computation, especially in a big data scenario. In this paper, we propose an efficient continuous big data integrity checking approach for decentralized storage. We design a data-time sampling strategy that randomly checks the integrity of multiple files at each time slot with high checking probability. Furthermore, to tackle the fairness problem derived from the sampling strategy, we propose a fair approach by designing an arbitration algorithm with the verifiable random function. Security analysis shows the security of our approach under the random oracle model. Evaluation and experiments demonstrate that our approach is more efficient in the big data scenario compared with the state-of-the-arts.