TY - GEN
T1 - Noise adaptive tensor train decomposition for low-rank embedding of noisy data
AU - Li, Xinsheng
AU - Candan, K. Selçuk
AU - Sapino, Maria Luisa
N1 - Funding Information:
This work has been supported by NSF grants #1633381, #1909555, #1629888, #2026860, #1827757, a DOE CYDRES grant, and a European Commission grant #690817. Experiments for the paper were conducted using NSF testbed: “Chameleon: A Large-Scale Re-configurable Experimental Environment for Cloud Research”.
Publisher Copyright:
© Springer Nature Switzerland AG 2020.
PY - 2020
Y1 - 2020
N2 - Tensor decomposition is a multi-modal dimensionality reduction technique to support similarity search and retrieval. Yet, the decomposition process itself is expensive and subject to dimensionality curse. Tensor train decomposition is designed to avoid the explosion of intermediary data, which plagues other tensor decomposition techniques. However, many tensor decomposition schemes, including tensor train decomposition is sensitive to noise in the input data streams. While recent research has shown that it is possible to improve the resilience of the tensor decomposition process to noise and other forms of imperfections in the data by relying on probabilistic techniques, these techniques have a major deficiency: they treat the entire tensor uniformly, ignoring potential non-uniformities in the noise distribution. In this paper, we note that noise is rarely uniformly distributed in the data and propose a Noise-Profile Adaptive Tensor Train Decompositionmethod, which aims to tackle this challenge. $$\mathtt{NTTD}$$ leverages a model-based noise adaptive tensor train decomposition strategy: any rough priori knowledge about the noise profiles of the tensor enable us to develop a sample assignment strategy that best suits the noise distribution of the given tensor.
AB - Tensor decomposition is a multi-modal dimensionality reduction technique to support similarity search and retrieval. Yet, the decomposition process itself is expensive and subject to dimensionality curse. Tensor train decomposition is designed to avoid the explosion of intermediary data, which plagues other tensor decomposition techniques. However, many tensor decomposition schemes, including tensor train decomposition is sensitive to noise in the input data streams. While recent research has shown that it is possible to improve the resilience of the tensor decomposition process to noise and other forms of imperfections in the data by relying on probabilistic techniques, these techniques have a major deficiency: they treat the entire tensor uniformly, ignoring potential non-uniformities in the noise distribution. In this paper, we note that noise is rarely uniformly distributed in the data and propose a Noise-Profile Adaptive Tensor Train Decompositionmethod, which aims to tackle this challenge. $$\mathtt{NTTD}$$ leverages a model-based noise adaptive tensor train decomposition strategy: any rough priori knowledge about the noise profiles of the tensor enable us to develop a sample assignment strategy that best suits the noise distribution of the given tensor.
UR - http://www.scopus.com/inward/record.url?scp=85093827007&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85093827007&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-60936-8_16
DO - 10.1007/978-3-030-60936-8_16
M3 - Conference contribution
AN - SCOPUS:85093827007
SN - 9783030609351
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 203
EP - 217
BT - Similarity Search and Applications - 13th International Conference, SISAP 2020, Proceedings
A2 - Satoh, Shin’ichi
A2 - Vadicamo, Lucia
A2 - Carrara, Fabio
A2 - Zimek, Arthur
A2 - Bartolini, Ilaria
A2 - Aumüller, Martin
A2 - Jonsson, Bjorn Por
A2 - Pagh, Rasmus
PB - Springer Science and Business Media Deutschland GmbH
T2 - 13th International Conference on Similarity Search and Applications, SISAP 2020
Y2 - 30 September 2020 through 2 October 2020
ER -