Coupling efficiencies of TE11 and TM01 modes oscillating with radial injected electron beam in coaxial vircator present nearly the same value for coaxial geometry. 3D PIC simulation result shows that non-poloidal uniform mode dominates the output power. The effect of TE11 mode oscillating with injected electron beam on the performance of coaxial vircator can not be ignored. By introducing an unsymmetrical geometry and a TE11 mode reflector, the coupling efficiency between electron beam and TM01 mode is restrained. Consequently, the modes competition in the beam-EM wave interaction cavity is quenched, which leads to the increase of the energy conversion efficiency. The TE11 mode enhanced coaxial vircator with a non-coaxial geometry gives the power efficiency of 7% and 1.0 GW output pow