Discuss the maxima and minima of \(x^3y^2(1-x-y)\).

\(f=x^3y^2-x^4y^2-x^3y^3\).

\(f_x=3x^2y^2-4x^3y^2-3x^2y^3=x^2y^2(3-4x-3y)\); \(f_y=2x^3y-2x^4y-3x^3y^2=x^3y(2-2x-3y)\).

Stationary points (x, y ≠ 0): \(4x+3y=3,\ 2x+3y=2\Rightarrow x=\frac12,\ y=\frac13\). (Points with x = 0 or y = 0 give \(rt-s^2=0\) and are not extrema of interest; they need further investigation.)

\(r=f_{xx}=6xy^2-12x^2y^2-6xy^3,\ s=f_{xy}=6x^2y-8x^3y-9x^2y^2,\ t=f_{yy}=2x^3-2x^4-6x^3y\).

At \(\left(\frac12,\frac13\right)\): \(r=\frac13-\frac13-\frac19=-\frac19\); \(s=\frac12-\frac13-\frac14=-\frac1{12}\); \(t=\frac14-\frac18-\frac14=-\frac18\).

\(rt-s^2=\frac1{72}-\frac1{144}=\frac1{144}>0\) and \(r<0\) → maximum.

Maximum at \(\left(\frac12,\frac13\right)\); \(f_{max}=\frac18\cdot\frac19\left(1-\frac12-\frac13\right)=\frac1{432}\).