Abstract: In order to improve the performance of lithium niobate acousto-electro-optic(AEO) modulator,the acousto-optic operating modes were systematically studied according to tangent condition after the determination of electro-optic operating mode.Acousto-optic merit for sheared ultrasonic wave in x-O-z plane and y-O-z plane were calculated respectively and the optimum operating mode for lithium niobate AEO modulator was determined.Furthermore,the design parameters for central frequency in range of 50MHz～300MHz were also given.By this optimum operating mode,the AEO device can gain a higher acousto-optic merit and achieve a higher diffractive efficiency.