Then you have to learn all those chords up and down the fingerboard so you can whip them out when you need them.

You need to learn basic scale harmony so you know which chords can be built out of which scales and modes.

When you see a chord progression that uses chords built from a scale or mode then that is your best clue as to what scale/mode is going to sound appropriate for that group of chords.

If the chords you’re playing over don’t all fit one scale/mode, then you have to switch while you’re playing or face the possibility of sounding like you don’t know what you’re supposed to be playing.

Some progressions have chords that can be found in more than one scale/mode. In that case you could choose any one that fits or use them all based on what sounds good.

Jazz Fusion guys like to use specific scales/modes over certain chords regardless of how the chord progression is put together. They might choose to play Dorian over any minor chord they come across for example. Generally speaking, you have to know what you’re doing to make this sound good. Most rock stuff sounds best if you stick to the scale that fits all or most of the chords.