Hierarchical Singular Value Decomposition of Tensors

We define the hierarchical singular value decomposition (SVD) for tensors of order $dgeq2$. This hierarchical SVD has properties like the matrix SVD (and collapses to the SVD in $d=2$), and we prove these. In particular, one can find low rank (almost) best approximations in a hierarchical format ($mathcal{H}$-Tucker) which requires only $mathcal{O}((d-1)k^3+dnk)$ parameters, where d is the order of the tensor, n the size of the modes, and k the (hierarchical) rank. The $mathcal{H}$-Tucker format is a specialization of the Tucker format and it contains as a special case all (canonical) rank k tensors. Based on this new concept of a hierarchical SVD we present algorithms for hierarchical tensor calculations allowing for a rigorous error analysis. The complexity of the truncation (finding lower rank approximations to hierarchical rank k tensors) is in $mathcal{O}((d-1)k^4+dnk^2)$ and the attainable accuracy is just 2–3 digits less than machine precision.