Let

$\theta$ = ∠HCO = ∠ECO

$\varphi = 180^\circ - 2\theta$

From right triangle BFC

$\tan \varphi = \dfrac{20}{CF}$

$CF = \dfrac{20}{\tan \varphi} = \dfrac{20}{\tan (180^\circ - 2\theta)}$

$CF = -\dfrac{20}{\tan 2\theta}$

From right triangle CEO

$\tan \theta = \dfrac{15}{CE}$

$CE = \dfrac{15}{\tan \theta}$

$BG = CE + CF$

$BG = \dfrac{15}{\tan \theta} - \dfrac{20}{\tan 2\theta}$

Area of trapezoid BCEG

$A_{BCEG} = \frac{1}{2}(CE + BG)(20)$

$A_{BCEG} = 10 \left[ \dfrac{15}{\tan \theta} + \left( \dfrac{15}{\tan \theta} - \dfrac{20}{\tan 2\theta} \right) \right]$

$A_{BCEG} = \dfrac{300}{\tan \theta} - \dfrac{200}{\tan 2\theta}$

Area of trapezoid ABCD

$A = A_{ADEG} - A_{BCEG}$

$A = 25(20) - \left( \dfrac{300}{\tan \theta} - \dfrac{200}{\tan 2\theta} \right)$

$A = 500 - 300 \cot \theta + 200 \cot 2\theta$

For maximum area, dA/dθ = 0:

$\dfrac{dA}{d\theta} = -300(-\csc^2 \theta) + 200 (-2 \csc^2 2\theta) = 0$

$300\csc^2 \theta - 400 \csc^2 2\theta = 0$

$3\csc^2 \theta = 4 \csc^2 2\theta$

$\sqrt{3} \csc \theta = 2 \csc 2\theta$

$\dfrac{\sqrt{3}}{\sin \theta} = \dfrac{2}{\sin 2\theta}$

$\dfrac{\sqrt{3}}{\sin \theta} = \dfrac{2}{2\sin \theta \cos \theta}$

$\cos \theta = \dfrac{1}{\sqrt{3}}$

$\theta = 54.74^\circ$

$\varphi = 180^\circ - 2\theta = 180^\circ - 2(54.74^\circ)$

$\varphi = 70.53^\circ$

From right triangle BFC

$\sin \varphi = \dfrac{20}{BC}$

$BC = \dfrac{20}{\sin \varphi} = \dfrac{20}{\sin 70.53^\circ}$

$BC = 21.21 \, \text{ cm}$ *answer*

A solution to the same problem has been done by Geometry without the use of Calculus. In that solution, the location of the tangent point H has been pointed out by direct analogy. See how it was done by following this link: Solution by Geometry alone