Abstract: The high-order mode output of high power millimeter wave gyrotron should be converted due to its disadvantages in space transport, which can be hardly solved by waveguide mode converter. Thus the built-in quasi-optical mode converter is studied. The converter consists of Vlasov launcher, quasi-parabolic mirror and phase correcting mirros. The simulation program is developed according to scalar diffraction theory and KSA algorithm, and phase correcting mirrors are designed. The calculation analysis of field distribution at the reference plane shows that, the converter’s scalar conversion efficiency is larger than 98%, and the vector conversion efficiency is larger than 93%. The results indicate that the method can guide the design of quasi-optical mode converter efficiently.