LTN broadcasting

LTN predicate case

In LTNtorch, when a predicate (ltn.core.Predicate), function (ltn.core.Function), or connective (ltn.core.Connective) is called, the framework automatically performs the broadcasting of the inputs.

To make a simple example, assume that we have two variables, \(x\) and \(y\), with the following groundings:

\(\mathcal{G}(x)=[[1.6, 1.8, 2.3], [9.3, 4.5, 3.4]] \in \mathbb{R}^{2 \times 3}\);
\(\mathcal{G}(y)=[[1.2, 1.3, 2.7, 10.4], [4.3, 5.6, 9.5, 1.3], [5.4, 1.5, 9.5, 8.4]] \in \mathbb{R}^{3 \times 4}\).

Variable \(x\) has two individuals with three features each, while variable \(y\) has three individuals with four features each.

Now, let us assume that we have a binary predicate \(P(a, b)\), grounded as \(\mathcal{G}P(a, b | \theta) = a, b \mapsto \sigma(MLP_{\theta}(a, b))\). \(P(a, b)\) is a learnable predicate which maps from \(\mathbb{R}^7\) to \([0., 1.]\). In the notation, \(MLP_{\theta}\) is a neural network, parametrized by \(\theta\), with 7 input neurons, some hidden layers, and 1 output neuron. At the last layer has been applied a logistic function to assure the output to be in the range \([0., 1.]\). By doing so, the output of \(P(a, b)\) can be interpreted as fuzzy truth value.

Now, suppose that we want to compute \(P(x, y)\). LTNtorch automatically broadcasts the two variables before computing the predicate. After the broadcasting, we will have the following inputs for our predicate:

\(\begin{bmatrix} 1.6 & 1.8 & 2.3\\ 1.6 & 1.8 & 2.3\\ 1.6 & 1.8 & 2.3\\ 9.3 & 4.5 & 3.4\\ 9.3 & 4.5 & 3.4\\ 9.3 & 4.5 & 3.4 \end{bmatrix} \in \mathbb{R}^{6 \times 3}\) for \(x\), and \(\begin{bmatrix} 1.2 & 1.3 & 2.7 & 10.4\\ 4.3 & 5.6 & 9.5 & 1.3\\ 5.4 & 1.5 & 9.5 & 8.4\\ 1.2 & 1.3 & 2.7 & 10.4\\ 4.3 & 5.6 & 9.5 & 1.3\\ 5.4 & 1.5 & 9.5 & 8.4 \end{bmatrix} \in \mathbb{R}^{6 \times 4}\) for \(y\).

Now, it is possible to observe that if we concatenate these two tensors on the first dimension (torch.cat([x, y], dim=1)), we obtain the following input for our predicate:

\(\begin{bmatrix} 1.6 & 1.8 & 2.3 & 1.2 & 1.3 & 2.7 & 10.4\\ 1.6 & 1.8 & 2.3 & 4.3 & 5.6 & 9.5 & 1.3\\ 1.6 & 1.8 & 2.3 & 5.4 & 1.5 & 9.5 & 8.4\\ 9.3 & 4.5 & 3.4 & 1.2 & 1.3 & 2.7 & 10.4\\ 9.3 & 4.5 & 3.4 & 4.3 & 5.6 & 9.5 & 1.3\\ 9.3 & 4.5 & 3.4 & 5.4 & 1.5 & 9.5 & 8.4 \end{bmatrix} \in \mathbb{R}^{6 \times 7}\).

This tensor contains all the possible combinations of the individuals of the two variables, that are 6. After the computation of the predicate, LTNtorch organizes the output in a tensor \(\mathbf{out} \in [0., 1.]^{2 \times 3}\), where the first dimension is related with variable \(x\), while the second dimension with variable \(y\). In \(\mathbf{out}[0, 0]\) there will be the result of the evaluation of \(P(x, y)\) on the first individual of \(x\), namely \([1.6, 1.8, 2.3]\), and first individual of \(y\), namely \([1.2, 1.3, 2.7, 10.4]\), in \(\mathbf{out}[0, 1]\) there will be the result of the evaluation of \(P(x, y)\) on the first individual of \(x\), namely \([1.6, 1.8, 2.3]\), and second individual of \(y\), namely \([4.3, 5.6, 9.5, 1.3]\), and so forth.

To conclude this note, in LTNtorch, the output of predicates, functions, connectives, and quantifiers are LTNObject instances. In the case of our example, the output of predicate \(P\) is an LTNObject with the following attributes:

value \(\in [0., 1.]^{2 \times 3}\);
free_vars = [‘x’, ‘y’].

Note that we have analyzed just an atomic formula (predicate) in this scenario. Since the variables appearing in the formula are not quantified, the free variables in the output are both \(x\) and \(y\). If instead of \(P(x, y)\) we had to compute \(\forall x P(x, y)\), the free_vars attribute would have been equal to [‘y’]. Finally, if we had to compute \(\forall x \forall y P(x, y)\), the free_vars attribute would have been an empty list.

LTN function case

The same scenario explained above can be applied to an LTN function (ltn.core.Function) instead of an LTN predicate (ltn.core.Predicate). Suppose we have the same variables, \(x\) and \(y\), with the same groundings, \(\mathcal{G}(x)\) and \(\mathcal{G}(y)\). Then, suppose we have a 2-ary (2 inputs) logical function \(f\), grounded as \(\mathcal{G}f(a, b | \theta) = a, b \mapsto MLP_{\theta}(a, b)\).

In this case, \(MLP_{\theta}\) is a neural network, parametrized by \(\theta\), with 7 input neurons, some hidden layers, and five output neurons. In other words, \(f\) is a learnable function which maps from individuals in \(\mathbb{R}^7\) to individuals in \(\mathbb{R}^5\). Note that, in this case, we do not have applied a logistic function to the output. In fact, logical functions do not have the constraint of having outputs in the range \([0., 1.]\).

LTNtorch applies the same broadcasting that we have seen above to the inputs of function \(f\). The only difference is on how the output is organized. In the case of an LTN function, the output is organized in a tensor where the first \(k\) dimensions are related with the variables given in input, while the remaining dimensions are related with the features of the individuals in output.

In our scenario, the output of \(f(x, y)\) is a tensor \(\mathbf{out} \in \mathbb{R}^{2 \times 3 \times 5}\). The first dimension is related with variable \(x\), the second dimension with variable \(y\), while the third dimension with the features of the individuals in output. In \(\mathbf{out}[0, 0]\) there will be the result of the evaluation of \(f(x, y)\) on the first individual of \(x\), namely \([1.6, 1.8, 2.3]\), and first individual of \(y\), namely \([1.2, 1.3, 2.7, 10.4]\), in \(\mathbf{out}[0, 1]\) there will be the result of the evaluation of \(f(x, y)\) on the first individual of \(x\), namely \([1.6, 1.8, 2.3]\), and second individual of \(y\), namely \([4.3, 5.6, 9.5, 1.3]\), and so forth.

LTN connective case

LTNtorch applies the LTN broadcasting also before computing a logical connective. To make the concept clear, let us make a simple example.

Suppose that we have variables \(x\), \(y\), \(z\), and \(u\), with groundings:

\(mathcal{G}(x) \in \mathbb{R}^{2 \times 3}\), namely \(x\) contains two individuals in \(\mathbb{R}^3\);
\(mathcal{G}(y) \in \mathbb{R}^{4 \times 3 \times 5}\), namely \(y\) contains four individuals in \(\mathbb{R}^{3 \times 5}\);
\(mathcal{G}(z) \in \mathbb{R}^{3 \times 5}\), namely \(z\) contains three individuals in \(\mathbb{R}^5\);
\(mathcal{G}(u) \in \mathbb{R}^{6 \times 2}\), namely \(u\) contains six individuals in \(\mathbb{R}^2\).

Then, suppose that we have two binary predicates, \(P(a, b)\) and \(Q(a, b)\). \(P\) maps from \(\mathbb{R}^18\) to \([0., 1.]\), while \(Q\) maps from \(\mathbb{R}^7\) to \([0., 1.]\).

Suppose now that we want to compute the formula: \(P(x, y) \land Q(z, u)\). In order to evaluate this formula, LTNtorch follows the following procedure:

it computes the result of the atomic formula \(P(x, y)\), which is a tensor \(\mathbf{out_1} \in [0., 1.]^{2 \times 4}\). Note that before the computation of \(P(x, y)\), LTNtorch performed the LTN broadcasting of variables \(x\) and \(y\);
it computes the result of the atomic formula \(Q(z, u)\), which is a tensor \(\mathbf{out_2} \in [0., 1.]^{3 \times 6}\). Note that before the computation of \(P(z, u)\), LTNtorch performed the LTN broadcasting of variables \(z\) and \(u\);
it performs the LTN broadcasting of \(\mathbf{out_1}\) and \(\mathbf{out_2}\);
it applies the fuzzy conjunction \(\mathbf{out_1} \land_{fuzzy} \mathbf{out_2}\). The result is a tensor \(\mathbf{out} \in [0., 1.]^{2 \times 4 \times 3 \times 6}\).

Notice that the output of a logical connective is always wrapped into an LTN object, like it happens for predicates and functions. In this simple example, the LTNObject produced by the fuzzy conjunction has the following attributes:

value = \(\mathbf{out}\);
free_vars = [‘x’, ‘y’, ‘z’, ‘u’].

Notice that free_vars contains the labels of all the variables appearing in \(P(x, y) \land Q(z, u)\). This is due to the fact that all the variables are free in the formula since are not quantified by any logical quantifier. Notice also that \(\mathbf{out}\) has four dimensions, one for each variable appearing in the formula. These dimensions can be indexed to retrieve the evaluation of \(P(x, y) \land Q(z, u)\) on a specific combination of individuals of \(x,y,z,u\). For example, \(\mathbf{out}[0, 0, 0, 0]\) contains the evaluation of \(P(x, y) \land Q(z, u)\) on the first individuals of all the variables, while \(\mathbf{out}[0, 1, 0, 0]\) contains the evaluation of \(P(x, y) \land Q(z, u)\) on the first individuals of \(x,z,u\) and second individual of \(y\).