Open /etc/ssh/sshd_config in your favourite editor (as root). Add the following lines below the authentication section.
AllowUsers user1 user2This will allow only users user1 and user2 access to login via SSH.
Save the file /etc/ssh/sshd_config and restart ssh.
Ony users user1 and user2 will be able to login via SSH.