Acoustic black hole effects show promise for passive vibration control through the manipulation of flexural wave propagation in an ABH beam with a power-law thickness profile. Under the same excitation level, the transverse displacement at the tapered tip of a typical ABH beam is much larger than that of a uniform beam, thus leading to amplified geometric nonlinearities. In the meantime, the variation of non-uniform cross section may generate complex dynamics and nonlinear effects. In this paper, a total Lagrangian method combined with Jaumann strain theory is utilized to analyze the dynamic behaviors of an ABH beam based on Euler-Bernoulli beam assumption. The developed model considers the shortening effect of a cantilever ABH beam, which allows representing the longitudinal motion by the transverse displacement in the coupled system equation. The developed formulation is then cast into a reduced Galerkin finite element framework, which allows simple but effective estimation of the nonlinear hardening or softening effects for different structural modes. Both experimental and numerical results show that the lower-order modes of the ABH beam mainly exhibit enhanced hardening effects due to dominant nonlinear stiffness, as compared with its uniform counterpart which shows hardening only for the first mode. The variation of the non-uniform cross section plays a vital role in determining the dominant level of the hardening or softening effects. For the tapered thickness of an ABH beam, the weakened nonlinear inertia is the main reason for the observed hardening effects.