One of the coolest ideas I’ve come across recently relates to the way patterns form in matrices. Moreover, the way we can induce a pattern in a matrix. Enter the Kronecker Product, the way pattern-based matrix multiplication occurs.
Definition: Kroncker product uses matrix \(A\) as an \(m \times n\) matrix and matrix \(B\) as a \(p \times q\) matrix so that the Kronecker product \(A \otimes B\) is the \(mp \times nq\) block matrix:
\[{ \bf{ A} } \otimes {\bf{ B } } = \left[ {\begin{array}{ *{20}{c} }{ {a_{11} }\bf{ B } }& \cdots &{ {a_{1n} }\bf{ B } }\\ \vdots & \ddots & \vdots \\{ {a_{m1} }\bf{ B } }& \cdots &{ {a_{mn} }\bf{ B } } \end{array} } \right]\]
Substituting in \(B\)’s values then yields:
\[\left[ {\begin{array}{ *{20}{c} }{ { a_{11} }{ b_{11} } }&{ { a_{11} }{ b_{12} } }& \cdots &{ { a_{11} }{ b_{1q} } }& \cdots & \cdots &{ { a_{1n} }{ b_{11} } }&{ { a_{1n} }{ b_{12} } }& \cdots &{ { a_{1n} }{ b_{1q} } }\\{ { a_{11} }{ b_{21} } }&{ { a_{11} }{ b_{22} } }& \cdots &{ { a_{11} }{ b_{2q} } }& \cdots & \cdots &{ { a_{1n} }{ b_{21} } }&{ { a_{1n} }{ b_{22} } }& \cdots &{ { a_{1n} }{ b_{2q} } }\\ \vdots & \vdots & \ddots & \vdots & & & \vdots & \vdots & \ddots & \vdots \\{ { a_{11} }{ b_{p1} } }&{ { a_{11} }{ b_{p2} } }& \cdots &{ { a_{11} }{ b_{pq} } }& \cdots & \cdots &{ { a_{1n} }{ b_{p1} } }&{ { a_{1n} }{ b_{p2} } }& \cdots &{ { a_{1n} }{ b_{pq} } }\\ \vdots & \vdots & & \vdots & \ddots & & \vdots & \vdots & & \vdots \\ \vdots & \vdots & & \vdots & & \ddots & \vdots & \vdots & & \vdots \\{ { a_{m1} }{ b_{11} } }&{ { a_{m1} }{ b_{12} } }& \cdots &{ { a_{m1} }{ b_{1q} } }& \cdots & \cdots &{ { a_{mn} }{ b_{11} } }&{ { a_{mn} }{ b_{12} } }& \cdots &{ { a_{mn} }{ b_{1q} } }\\{ { a_{m1} }{ b_{21} } }&{ { a_{m1} }{ b_{22} } }& \cdots &{ { a_{m1} }{ b_{2q} } }& \cdots & \cdots &{ { a_{mn} }{ b_{21} } }&{ { a_{mn} }{ b_{22} } }& \cdots &{ { a_{mn} }{ b_{2q} } }\\ \vdots & \vdots & \ddots & \vdots & & & \vdots & \vdots & \ddots & \vdots \\{ { a_{m1} }{ b_{p1} } }&{ { a_{m1} }{ b_{p2} } }& \cdots &{ { a_{m1} }{ b_{pq} } }& \cdots & \cdots &{ { a_{mn} }{ b_{p1} } }&{ { a_{mn} }{ b_{p2} } }& \cdots &{ { a_{mn} }{ b_{pq} } }\end{array} } \right]\]
Within both of these forms, we are able to see a distinct placement of matrices within another matrix.
Theory to More Theory
Prove that \(\mathrm{ tr }\left( { A \otimes B } \right) = \mathrm{ tr }\left( A \right)\mathrm{ tr }\left( B \right)\)
In our case, we are particularly interested in the diagonal of two square matrices joined by the Kronecker product.
Why? Well, the trace of a square $n n $ matrix is defined to be the sum of the elements on the main diagonal, which runs from the upper left corner of the matrix to the lower right corner of the matrix. Mathematically, we have: $(A) = a_{ 11 } + a_{ 22 } + + a_{ nn }={ i=1 }^{ n } { a{ ii } } $
Returning back to our previous line of discussion regarding the Kronecker product, we’ll amend the matrices presented in the definition slightly so that matrix \(A\) now has square dimensions of \(a \times a\) and matrix \(B\) now has square dimensions of \(b \times b\). Thus, we will have a Kronecker product in the following form:
\[{ \bf{ A} } \otimes {\bf{ B } } = \left[ {\begin{array}{ *{20}{c} }{ {a_{11} }\bf{ B } }& \cdots &{ {a_{1a} }\bf{ B } }\\ \vdots & \ddots & \vdots \\{ {a_{a1} }\bf{ B } }& \cdots &{ {a_{aa} }\bf{ B } } \end{array} } \right]\]
The dimensions of this matrix are now \(\left( { ab } \right) \times \left( { ab } \right)\).
Thus, from non-expand Kronecker product form, we can see that sum of the main diagonal will be: ${ i = 1 }^a { { a{ ii } }B } $
So, taking the trace of \(A \otimes B\) gives \[\mathrm{ tr }\left( { A \otimes B } \right) = \sum\limits_{ i = 1 }^a { \left( { { a_{ ii } }\sum\limits_{ j = 1 }^b { { b_{ jj } } } } \right) }\].
Note: The trace is also being applied to \(B\).
Therefore, we get: \[\mathrm{ tr }\left( { A \otimes B } \right) = \sum\limits_{ i = 1 }^a { \left( { { a_{ ii } }\sum\limits_{ j = 1 }^b { { b_{ jj } } } } \right) } = \sum\limits_{ i = 1 }^a { \left( { { a_{ ii } }\mathrm{ tr }\left( B \right) } \right) } = \mathrm{ tr }\left( B \right)\sum\limits_{ i = 1 }^a { \left( { { a_{ ii } } } \right) } = \mathrm{ tr }\left( B \right)\mathrm{ tr }\left( A \right) = \mathrm{ tr }\left( A \right)tr\left( B \right)\]