Considering the factors such as the left or right turning directions of wave, the wave being forward or backward, all the possible modes in helical wave guide are given by using the coupling wave equations of helical wave guide. The relationships among all the possible modes are analyzed according to the characteristics of coupling wave equations and Blagg conditions, the mode coupling characteristics of traveling wave tube and backward wave tube with the helical corrugation are given correspondingly. The corresponding coupling relationships and the rules of mode coupling are obtained. Combined with the dispersion equations of helical wave guide, the effects of the helical corrugation period and corrugation fluctuation degree on mode coupling are discussed that with the period increase of