We investigated the coupling of single localized defect excitons in a two-dimensional layer with chiral plasmonic nanocavity. By designing a chiral plasmon mode with high quality factor and coupling it with a WSe2 monolayer, we demonstrated cavity-dependent photon output from single quantum emitters. The dynamics of this coupled system shows that the chiral photon output mainly comes from the spontaneous emission of the quantum emitter which is strongly modified by the chiral plasmonic nanocavity, which proves that the valley-dependent optical selection rules are absent in such quantum emitters . The coupling interactions of plasmonic nanocavity with two-dimensional excitons at the level of single quanta by using the single quantum emitter in monolayer, instead of small number of two-dimensional excitons, will provide platform for future research of quantum plasmonic photonics and integrated photonics.