Tensor decompositions: short guides

These guides give short, precise answers to common questions about tensors and tensor decompositions: what a tensor is in machine learning terms, how t-SVD and the t-product work, what tc-SVD changes, and how CP, Tucker and t-SVD compare. They are based on Mahdi Molavi's M.Sc. thesis, Big Data Learning with Tensor Representation, written at Tarbiat Modares University.

The guides

• What is a tensor? Order and modes, fibers and slices, unfolding, the mode-n product, and why flattening data into vectors throws information away.
• What is t-SVD? The t-product of Kilmer and Martin (2011), the factorization A = U ∗ S ∗ V^T, why the FFT makes it cheap, and what truncation guarantees.
• What is tc-SVD? The variant I developed in my thesis: a mirror boundary instead of a periodic one, the DCT instead of the FFT, real arithmetic throughout, and the results it gave.
• CP vs Tucker vs t-SVD What each decomposition gives you, what it costs, and when to pick which, with a side-by-side table.

Where they come from

My M.Sc. in computer science at Tarbiat Modares University (2016–2019), supervised by Dr. Mansoor Rezghi, started from one question: how do you learn from large data sets without flattening them first? These pages condense the answers into definitions you can check quickly.

For the longer version, with figures, experiments and code, read the Tensor Notes series, starting with part 1, Stop Flattening Your Data. The thesis summary lists everything the work covers on one page.

Code

A MATLAB implementation of tc-SVD, the FFT-based t-SVD, and the classification, clustering and incremental-update methods from the thesis is open source at github.com/mahdipc/tcSVD under the MIT License. A short, tested NumPy version of the t-product and t-SVD is in the t-SVD guide.

Reading order

If tensors are new to you, read the guides in the order listed. If you already know the matrix SVD well, go straight to the t-SVD guide: after a transform along the third mode, every operation there turns into ordinary matrix work on the frontal slices.