Based on the coupled wave theory, the helical waveguide(serpentine-type) TEm1-TE11 mode converter was studied. A 94 GHz six-period TE01-TE11 mode converter was designed and numerically analysed and simulated. A 10.24 GHz four-period TE31-TE11 mode converter was analysed and simulated. The ratio of mode constituents in the mode converter was obtained by solving coupled wave equation, which was employed to modify the structure parameters. The convert efficiency of these two devices reached 98% at center frequency and the bandwith of more than 95% exceeded 2 GHz. The simulation results agree well with the theoretical ones.