For a more secure terminal (SSH) environment, you can add a second factor to authenticate a user. I personally run Ubuntu and i implemented the Google Authenticator on my Ubuntu system using the Google Authenticator Library of Google in combination with my iPhone with the Google Authenticator App.

See the following howto:

http://wouter.borremans.nl/computer-related-howto/how-to/howto-two-factor-authentication-ubuntu-12-04-lts-using-google-authenticator/