In this paper, we show the impact of the in-band group-delay ripple typically existing in analog/RF filter in continuous-phase modulation (CPM) systems and propose a coefficients estimation based compensation method to rival against the group-delay ripple. The method employs an compensation filter to eliminate the in-band group-delay ripple, and the coefficients of the filter are determined by a recursive estimation method based on minimum mean square error (MMSE). The steps of the implementation of this method are also presented. It requires only digital hardware and firmware to accomplish the estimation and compensation process. Computer simulation has also been given out to demonstrate that an accurate filter can be achieved to efficiently compensate for the group-delay ripple, obtaining great performance improvements.