{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Other Tensor Formats\n",
"\n",
"Besides the [natively supported formats](main_formats.ipynb), you can use *tntorch* to emulate other structured tensor decompositions (or at least, some of their functionality). \n",
"\n",
"Reference: all the following models are surveyed in [*\"Tensor Decompositions and Applications\"*, by Kolda and Bader (2009)](https://epubs.siam.org/doi/pdf/10.1137/07070111X)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## INDSCAL\n",
"\n",
"*Individual differences in scaling* (INDSCAL) is just a 3D CP decomposition with two shared factors, say the first two."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"3D CP tensor:\n",
"\n",
" 10 10 64\n",
" | | |\n",
" <0> <1> <2>\n",
" / \\ / \\ / \\\n",
"8 8 8 8\n",
"\n",
"tensor(0.0559, grad_fn=<DivBackward1>)\n"
]
}
],
"source": [
"import tntorch as tn\n",
"import torch\n",
"torch.set_default_dtype(torch.float64)\n",
"\n",
"\n",
"def INDSCAL(shape, rank):\n",
" \n",
" assert len(shape) == 3\n",
" assert shape[0] == shape[1]\n",
" \n",
" A = torch.randn(shape[0], rank, requires_grad=True)\n",
" B = A # The first two cores share the same memory\n",
" C = torch.randn(shape[2], rank, requires_grad=True)\n",
"\n",
" return tn.Tensor([A, B, C])\n",
" \n",
"t = INDSCAL([10, 10, 64], 8)\n",
"print(t)\n",
"print(tn.mean(t))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This tensor's two first factors are the same PyTorch tensor in memory. So if we optimize (fit) the tensor they will stay the same, as is desirable."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## CANDELINC\n",
"\n",
"CANDELINC (*canonical decomposition with linear constraints*) is a CP decomposition such that each factor is compressed along its columns by an additional given matrix (the *linear constraints*). In other words, it is a CP-Tucker format with fixed Tucker factors."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3D CP-Tucker tensor:\n",
"\n",
" 10 11 12\n",
" | | |\n",
" 5 6 7\n",
" <0> <1> <2>\n",
" / \\ / \\ / \\\n",
"3 3 3 3"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def CANDELINC(rank, constraints): # `constraints` are N In x Sn matrices encoding the linear constraints for the N CP factors\n",
" cores = [torch.randn(c.shape[1], rank, requires_grad=True) for c in constraints]\n",
" return tn.Tensor(cores, constraints)\n",
"\n",
"N = 3\n",
"rank = 3\n",
"constraints = [torch.randn(10, 5), torch.randn(11, 6), torch.randn(12, 7)]\n",
"CANDELINC(rank, constraints)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## DEDICOM\n",
"\n",
"In three-way *decomposition into directional components* (DEDICOM), 5 factors interact to encode a 3D tensor (2 of those factors are repeated). All factors use the same rank."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"5D TT-CP tensor:\n",
"\n",
" 10 64 1 64 10\n",
" | | | | |\n",
" <0> <1> (2) <3> <4>\n",
" / \\ / \\ / \\ / \\ / \\\n",
"8 8 8 8 8 8"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def DEDICOM(shape, rank):\n",
" \n",
" assert len(shape) == 3\n",
" assert shape[0] == shape[2]\n",
" \n",
" A = torch.randn(shape[0], rank, requires_grad=True)\n",
" D = torch.randn(shape[1], rank, requires_grad=True)\n",
" R = torch.randn(rank, 1, rank, requires_grad=True)\n",
"\n",
" return tn.Tensor([A, D, R, D, A])\n",
" \n",
"DEDICOM([10, 64, 10], 8)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that this tensor is to be accessed via a special pattern (`t[i, j, k]` should be written as `t[i, j, 0, j, k]`). Some routines (e.g. `numel()`, `torch()`, `norm()`, etc.) that are unaware of this special structure will not work properly."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## PARATUCK2\n",
"\n",
"PARATUCK2 is a variant of DEDICOM in which no factors are repeated, and two different ranks intervene."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"5D TT-CP tensor:\n",
"\n",
" 10 64 1 64 10\n",
" | | | | |\n",
" <0> <1> (2) <3> <4>\n",
" / \\ / \\ / \\ / \\ / \\\n",
"7 7 7 8 8 8"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def PARATUCK2(shape, ranks):\n",
" \n",
" assert len(shape) == 3\n",
" assert shape[0] == shape[2]\n",
" assert len(ranks) == 2\n",
" \n",
" A = torch.randn(shape[0], ranks[0], requires_grad=True)\n",
" DA = torch.randn(shape[1], ranks[0], requires_grad=True)\n",
" R = torch.randn(ranks[0], 1, ranks[1], requires_grad=True)\n",
" DB = torch.randn(shape[1], ranks[1], requires_grad=True)\n",
" B = torch.randn(shape[2], ranks[1], requires_grad=True)\n",
"\n",
" return tn.Tensor([A, DA, R, DB, B])\n",
" \n",
"PARATUCK2([10, 64, 10], [7, 8])"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
