As the largest freshwater lake in China, the water ecological health status of Lake Poyang has been highly concerned. In recent years, extreme floods and droughts occurred frequently in Lake Poyang. Phytoplankton growth has changed significantly under the influence of extreme floods and droughts. In order to investigate the spatial and temporal dynamics of phytoplankton, and to explore the influence mechanism of environmental factors on phytoplankton and the influence of extreme flood and drought events on phytoplankton in Lake Poyang, structural equation modeling was used to develop the influence path model between phytoplankton and environmental factors, and to quantify the extent of the influence of environmental factors on phytoplankton. The results showed that the phytoplankton in Lake Poyang was dominated by cyanobacteria and green algae with seasonal variation. The most critical factors affecting the abundance of phytoplankton were physical factors (water temperature>pH>secchi depth>dissolved oxygen), followed by nutrients (total nitrogen>nitrate nitrogen>total phosphorus>phosphate). Phytoplankton preferred high temperature, nutrients and pH. In the extreme flood in 2020 and the extreme high temperature and drought in 2022, the density of phytoplankton were mainly affected by physical factors such as water temperature, dissolved oxygen and secchi depth. For nutrients, the main limiting factors were phosphorus and nitrogen. Compared with normal years, in the extreme flood year, Lake Poyang suffered from the increase of incoming water, rainfall and lake water volume. The dilution effect exceeded the promotion of phytoplankton growth by water temperature, dissolved oxygen and secchi depth, resulting in the decrease of phytoplankton density and biomass. Among the nutrients, phosphorus was the main limiting factor. In extreme high temperature and drought years, due to the reduction of incoming water and the rapid evaporation of lake water, the concentration effect exceeded the inhibitory effect of water temperature and secchi depth on phytoplankton growth, resulting in a significant increase in phytoplankton density and biomass. Meanwhile, the impacts of nutrients on phytoplankton were enhanced. This study showed the significant impacts of the extreme flood and drought events on phytoplankton in Lake Poyang. Determining the mechanism of their impacts on phytoplankton can provide scientific support for phytoplankton monitoring and management under extreme meteorological events.