Many rotor dynamics problems are nonlinear in nature and faulty rotor dynamics are mostly nonlinear dynamics problems. It is of great significance to develop nonlinear theory to analyze the special rotor dynamics phenomenon caused by various faults. Part looseness is a common fault in rotor systems, which may cause disasters. Rotor system with part looseness turns to be a complicated nonlinear problem, with gyroscopic and dry friction effects occurring simultaneously. The analysis of nonlinear systems is in general confronted to two key challenges: complicated calculation method and long calculation time. This work has developed a fast and efficient numerical scheme with nonlinear modal analysis method. By employing the proposed nonlinear modal analysis method, the essential gyroscopic effect as well as the dry friction influence caused by part looseness are both taken into account when analyzing the dynamic behavior of rotor systems. First, the theoretical aspects and the analysis approach considering the nonlinear force and the gyroscopic effect are proposed for a general rotor system. Second, by employing the proposed nonlinear modal strategy, simulations are conducted on a general rotor system considering gyroscopic effect and dry friction caused by part looseness. Third, modal parameters such as resonance frequencies and modal damping ratios are carefully investigated as a function of rotation speed and modal amplitude. In the realization of a nonlinear modal analysis of a rotor system, the key point is to establish the relationship between modal parameters with both modal amplitude and rotation velocity. The proposed strategy can help better analyzing and interpreting the dynamic characteristics rotor systems from the modal overview.