The analysis code for quasi-optical mode converter is developed based on the theory of vector diffraction and the self-adapted Katsenelenbaum-Semenov algorithm for phase correcting. A 3 mm-band quasi-optical mode converter (TE62 to TEM00) is then optimized, and corresponding parameters of its components, i.e. Vlasov radiator and phase correcting mirror, are obtained. A sample is fabricated accordingly, and tests are conducted. The theoretical results show that the output mode purity of the converter is larger than 94%. The experimental results agree well with the theoretical ones.