In the temple, people were forbidden to use ordinary money as an offering to God. Amy heathen symbol or picture rendered coins unacceptable. Money changers pilfered a king's ransom by shoving righteous money across the table, which kept returning to them through the offering receptacles.
Meanwhile, dirty money they received in exchange for holy money stacked up. It wasn't good enough for God, but made Annas' family rich. The money was rolling, and Jesus couldn't stand it.
He cleansed the temple by Himself. Jesus did not have an army with Him. He did this purging alone, yet the sellers "scurried from Him like vermin from the light" (IDB). Why? One, they knew Jesus was popular. They had seen and heard the crowd as He entered Jerusalem. Two, the tradesmen knew the people didn't like them to start with. Decades later the masses rebelled against this merchandizing. Religious swindlers are still considered among the most despicable of any society.
Three, the dealers probably had guilty consciences. They knew what they were doing to honest, God-fearing, hardworking people. David Thomas well said, "Men will flee as panic-struck cowards before the ghosts of their own crimes". Poe's telltale heart still beats loud and clear in guilty people's consciences.