Given line equation of 2-order:
$$x^{2} - 6 x y + 6 y^{2} - 4 y - 4 = 0$$
This equation looks like:
$$a_{11} x^{2} + 2 a_{12} x y + 2 a_{13} x + a_{22} y^{2} + 2 a_{23} y + a_{33} = 0$$
where
$$a_{11} = 1$$
$$a_{12} = -3$$
$$a_{13} = 0$$
$$a_{22} = 6$$
$$a_{23} = -2$$
$$a_{33} = -4$$
To calculate the determinant
$$\Delta = \left|\begin{matrix}a_{11} & a_{12}\\a_{12} & a_{22}\end{matrix}\right|$$
or, substitute
$$\Delta = \left|\begin{matrix}1 & -3\\-3 & 6\end{matrix}\right|$$
$$\Delta = -3$$
Because
$$\Delta$$
is not equal to 0, then
find the center of the canonical coordinate system. To do it, solve the system of equations
$$a_{11} x_{0} + a_{12} y_{0} + a_{13} = 0$$
$$a_{12} x_{0} + a_{22} y_{0} + a_{23} = 0$$
substitute coefficients
$$x_{0} - 3 y_{0} = 0$$
$$- 3 x_{0} + 6 y_{0} - 2 = 0$$
then
$$x_{0} = -2$$
$$y_{0} = - \frac{2}{3}$$
Thus, we have the equation in the coordinate system O'x'y'
$$a'_{33} + a_{11} x'^{2} + 2 a_{12} x' y' + a_{22} y'^{2} = 0$$
where
$$a'_{33} = a_{13} x_{0} + a_{23} y_{0} + a_{33}$$
or
$$a'_{33} = - 2 y_{0} - 4$$
$$a'_{33} = - \frac{8}{3}$$
then equation turns into
$$x'^{2} - 6 x' y' + 6 y'^{2} - \frac{8}{3} = 0$$
Rotate the resulting coordinate system by an angle φ
$$x' = \tilde x \cos{\left(\phi \right)} - \tilde y \sin{\left(\phi \right)}$$
$$y' = \tilde x \sin{\left(\phi \right)} + \tilde y \cos{\left(\phi \right)}$$
φ - determined from the formula
$$\cot{\left(2 \phi \right)} = \frac{a_{11} - a_{22}}{2 a_{12}}$$
substitute coefficients
$$\cot{\left(2 \phi \right)} = \frac{5}{6}$$
then
$$\phi = \frac{\operatorname{acot}{\left(\frac{5}{6} \right)}}{2}$$
$$\sin{\left(2 \phi \right)} = \frac{6 \sqrt{61}}{61}$$
$$\cos{\left(2 \phi \right)} = \frac{5 \sqrt{61}}{61}$$
$$\cos{\left(\phi \right)} = \sqrt{\frac{\cos{\left(2 \phi \right)}}{2} + \frac{1}{2}}$$
$$\sin{\left(\phi \right)} = \sqrt{1 - \cos^{2}{\left(\phi \right)}}$$
$$\cos{\left(\phi \right)} = \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}}$$
$$\sin{\left(\phi \right)} = \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}}$$
substitute coefficients
$$x' = \tilde x \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}} - \tilde y \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}}$$
$$y' = \tilde x \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}} + \tilde y \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}}$$
then the equation turns from
$$x'^{2} - 6 x' y' + 6 y'^{2} - \frac{8}{3} = 0$$
to
$$6 \left(\tilde x \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}} + \tilde y \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}}\right)^{2} - 6 \left(\tilde x \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}} + \tilde y \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}}\right) \left(\tilde x \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}} - \tilde y \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}}\right) + \left(\tilde x \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}} - \tilde y \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}}\right)^{2} - \frac{8}{3} = 0$$
simplify
$$- 6 \tilde x^{2} \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}} \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}} - \frac{25 \sqrt{61} \tilde x^{2}}{122} + \frac{7 \tilde x^{2}}{2} - \frac{30 \sqrt{61} \tilde x \tilde y}{61} + 10 \tilde x \tilde y \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}} \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}} + \frac{25 \sqrt{61} \tilde y^{2}}{122} + 6 \tilde y^{2} \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}} \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}} + \frac{7 \tilde y^{2}}{2} - \frac{8}{3} = 0$$
$$- \frac{\sqrt{61} \tilde x^{2}}{2} + \frac{7 \tilde x^{2}}{2} + \frac{7 \tilde y^{2}}{2} + \frac{\sqrt{61} \tilde y^{2}}{2} - \frac{8}{3} = 0$$
Given equation is hyperbole
$$\frac{\tilde x^{2}}{\frac{8}{3} \frac{1}{- \frac{7}{2} + \frac{\sqrt{61}}{2}}} - \frac{\tilde y^{2}}{\frac{8}{3} \frac{1}{\frac{7}{2} + \frac{\sqrt{61}}{2}}} = -1$$
- reduced to canonical form
The center of canonical coordinate system at point O
(-2, -2/3)
Basis of the canonical coordinate system
$$\vec e_1 = \left( \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}}, \ \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}}\right)$$
$$\vec e_2 = \left( - \sqrt{\frac{1}{2} - \frac{5 \sqrt{61}}{122}}, \ \sqrt{\frac{5 \sqrt{61}}{122} + \frac{1}{2}}\right)$$