seems useless. we'll have 5 people votekick and nothing happens. votekick, how does it work?

I'd say most people need to votekick. Like when you have 20 players on server, at least 10 need to votekick.

But that's my theory. I have no idea how votekicking actually works.

I've never seen a votekick happen. Maybe that's a good thing (people are behaving) but on the other hand, I've heard people complaining about griefers and nothing happened

You have to tell people how to votekick.

I believe it's "/votekick [number to the left of the player name]", but the lack of server messages telling you stuff (e.g. x has logged in, y has initiated votekick against z etc) makes it difficult to confirm.

Well, looking at the server console right now - 8 seems to be sufficient on a 32 player server:

Player 6 placed votekick against player 13
10 Disconnected
New Client: 10
Player 18 placed votekick against player 13
Player 27 placed votekick against player 13
Player 24 placed votekick against player 13
Player 19 placed votekick against player 13
11 Disconnected
New Client: 11
Player 0 placed votekick against player 13
Player 15 placed votekick against player 13
Player 31 placed votekick against player 13
Player 13 kicked
13 Disconnected

so, when someone else initiates the vote, how do you respond? Do you just also type the command?

Yup, have to also /votekick with a player ID.

I believe 25% of everyone in the game has to votekick in order for a player to be kicked. If it is a small amount of people in a game, at least 2 people need to use the votekick command. In ComradeFlapjack's post, this gets confirmed since 8 is 25% of 32.