If Money Is Speech, Then We Should Be Able To Buy People's Votes

It's legal for me to persuade you to vote for me; that is, it's permissible for me to talk you into it.

Now, suppose money is speech. (Or, perhaps less imprecisely: spending money can be an act of political expression). If so, then I should be able to spend money as I would "spend" speech, and persuade you to vote for me. And the most direct way to do this  is by just giving you the money in exchange for your vote.

What am I missing?