Abstract: According to the structural and feeding characteristics of beam waveguide system, the physical model for the excitation, transmission and radiation problems in beam waveguide with arbitrary shape is set up by using the volume-surface integral equation. The matching state at the feeding area is described accurately by the mode matching method. The solutions of electric field distribution and the aperture radiation field of beam waveguide are given by using the multilevel fast multipole algorithm (MLFMA), which is a numerical algorithm with high efficiency. The example demonstrates that the method proposed can provide steady and accurate numerical solutions for the excitation-transmission-radiation problems in beam waveguide with arbitrary shape and electrically large size.