Under the given transformation, the corresponding vertices of R in the (u,v)-plane are
• x = 0, y = 0 ===> u = 0, v = 0
• x = 5, y = 1 ===> u = 1, v = 0
• x = 1, y = 5 ===> u = 0, v = 1
so that R is transformed into an isosceles right triangle given by the set
R' = {(u, v) : 0 ≤ u ≤ 1 and 0 ≤ v ≤ 1 - u}
The Jacobian for this transformation is
[tex]J = \begin{bmatrix}\frac{\partial x}{\partial u} & \frac{\partial x}{\partial v} \\ \frac{\partial y}{\partial u} & \frac{\partial y}{\partial v}\end{bmatrix} = \begin{bmatrix}5&1\\1&5\end{bmatrix}[/tex]
so that det(J) = 24, and the area element is
[tex]dA = dx \, dy = |\det(J)| \, du \, dv = 24 \, du \, dv[/tex]
The integral over R is then
[tex]\displaystyle \iint_R (x-6y) \, dA = 24 \iint_{R'} ((5u+v)-6(u+5v)) \, du \, dv[/tex]
[tex]\displaystyle \iint_R (x-6y) \, dA = -24 \int_0^1 \int_0^{1-u} (u+29v) \, dv \, du[/tex]
[tex]\displaystyle \iint_R (x-6y) \, dA = -24 \int_0^1 \left(u(1-u)+\frac{29}2(1-u)^2\right) \, du[/tex]
[tex]\displaystyle \iint_R (x-6y) \, dA = -12 \int_0^1 \left(29 - 56u + 27 u^2)\right) \, du[/tex]
[tex]\displaystyle \iint_R (x-6y) \, dA = -12 \left(29 - 28 + 9\right) = \boxed{-120}[/tex]