Mahdi MolaviCP writes a tensor as a sum of rank-one tensors; Tucker writes it as a small core tensor multiplied by one factor matrix per mode, usually computed with HOSVD; t-SVD factors a third-order tensor through the t-product into two orthogonal tensors and an f-diagonal one. CP is the most compact and interpretable of the three but is hard to fit, Tucker gives orthogonal factors but a dense core, and t-SVD keeps the optimal truncation of the matrix SVD at the price of treating the third mode specially.
All three try to do for tensors what the SVD does for matrices, and none keeps every property. Below, X is an I×J×K tensor. If modes, unfolding and the mode-n product are new to you, start with What is a tensor?
CP: a sum of rank-one tensors
The CP decomposition (also known as CANDECOMP or PARAFAC) approximates the tensor by R rank-one terms, each the outer product of three vectors:
X ≈ ∑r=1..R ar ∘ br ∘ cr
CP decomposition: a sum of R rank-one tensors.
What you get is three factor matrices with R columns each, R(I + J + K) numbers in total, the most compact of the three models. Under mild conditions the decomposition is unique up to scaling and the order of the terms, so the components can often be read as meaningful parts of the data. The costs: the factors are not orthogonal; there is no closed-form algorithm like the SVD, so CP is fitted iteratively, commonly with alternating least squares (ALS); choosing R is a hard problem in itself; and a best rank-R fit does not always exist, which can make fitting numerically delicate.
Tucker and HOSVD: a core and a basis per mode
The Tucker decomposition uses a small core tensor G of size R1×R2×R3 and one factor matrix per mode, combined through mode-n products:
X ≈ G ×1 A ×2 B ×3 C
HOSVD, the higher-order SVD, is the standard way to compute it. A holds the leading R1 left singular vectors of the mode-1 unfolding, B and C come from the other two unfoldings in the same way, and the core is G = X ×1 AT ×2 BT ×3 CT.
Tucker (HOSVD): orthogonal factors, but a dense core (drawn here as S).
You get an orthogonal basis for every mode and a separate compression rank per mode; the method works for tensors of any order and needs nothing beyond matrix SVDs. The costs: the core is dense, not diagonal, so every component interacts with every other; the core grows as R1R2R3; and truncated HOSVD is in general not the best approximation of its multilinear rank, only a good one that iterative methods such as HOOI can refine.
t-SVD: the matrix SVD in another algebra
t-SVD (Kilmer and Martin, 2011) changes the multiplication instead of the shape of the factors. Under the t-product it writes X = U ∗ S ∗ VT, with orthogonal U and V and f-diagonal S.
Full t-SVD of an n1×n2×n3 tensor: U is n1×n1×n3, V is n2×n2×n3, and S is f-diagonal.
You get orthogonality, a diagonal-like middle factor and an optimal truncation: keeping k singular tubes gives the best Frobenius-norm approximation of tubal rank at most k. It is cheap: after an FFT along the third mode it becomes K independent matrix SVDs that run in parallel, and real data needs only about half of them. The costs: it is defined for third-order tensors; the third mode is treated differently from the other two, so orientation and slice order matter; and the FFT brings a periodic assumption along that mode. tc-SVD keeps the algebra but swaps the FFT for the DCT and the wrap-around for a mirror.
Side by side
| CP | Tucker (HOSVD) | t-SVD | |
|---|---|---|---|
| Model | sum of R rank-one tensors | core tensor times one factor matrix per mode | U ∗ S ∗ VT under the t-product |
| Orthogonal factors | no | yes | yes, in the t-product sense |
| Middle factor | none (optional weights) | dense core | f-diagonal S |
| Rank notion | CP rank R | multilinear rank (R1, R2, R3) | tubal rank k |
| Computed by | iterative fitting, e.g. ALS | SVD of each unfolding | FFT, one SVD per frontal slice, inverse FFT |
| Best truncation | may not exist | good, not optimal in general | optimal for tubal rank |
| Storage for I×J×K | R(I + J + K) | R1R2R3 + IR1 + JR2 + KR3 | k(I + J + 1)K |
| Tensor order | any | any | three |
| Modes | treated alike | treated alike | third mode is special |
When to use which
- CP when you want a few interpretable components and the most compact model, and you can afford iterative fitting and a search over R.
- Tucker / HOSVD when different modes deserve different amounts of compression, when you want an orthogonal basis per mode, or when the tensor has more than three modes.
- t-SVD when the data is naturally a stack of matrices along one direction, such as images stored as lateral slices or frames over time, and you want SVD-like features with a guaranteed best truncation and simple, parallel code.
- tc-SVD in the same situations as t-SVD when wrapping that direction around makes no sense, for example in signals whose start and end are unrelated.
What my experiments showed
There was no single winner. In the clustering experiments of my thesis (best NMI, averaged over ten runs), two-sided tc-SVD came first on ORL and Extended Yale B, two-sided t-SVD on PIE-pose, and HOSVD on COIL-100 with 0.749, ahead of NMF and both tensor SVDs. HOSVD also ran faster than t-SVD and tc-SVD on Yale B and COIL-100. In classification, t-SVD and tc-SVD reached nearly the same MNIST accuracy, around 97%. Pick by the structure of your data and what you need from the factors, then check on your own data. Part 1 of Tensor Notes covers the building blocks in more depth.
Frequently asked questions
Is HOSVD the same as the Tucker decomposition?
Tucker is the model: a core tensor multiplied by a factor matrix along each mode. HOSVD is the standard way to compute it, taking each factor from the SVD of the matching unfolding.
Which tensor decomposition is the real generalization of the SVD?
None of them keeps every property of the matrix SVD. HOSVD keeps orthogonal factors but loses the diagonal core, CP keeps the sum of rank-one terms but loses orthogonality, and t-SVD keeps orthogonality and optimal truncation by changing the multiplication.
What is the difference between CP rank, multilinear rank and tubal rank?
CP rank is the smallest number of rank-one terms that sum to the tensor. Multilinear rank is the tuple of ranks of the unfoldings, which Tucker truncates. Tubal rank is the number of non-zero singular tubes in the t-SVD.
Which decomposition should I use for a stack of images?
If you want fast, SVD-like features with a guaranteed best truncation, t-SVD or tc-SVD with images as slices is a good start; if you want separate compression per mode, use Tucker. In my clustering experiments the best method changed from one data set to another, so test on your own data.