First I want to make clear that koh's understanding was incorrect; that users do NOT have the rights to do anything an admin can do, just because they have ssh.
Second I want to explain that the big problem is that many files in a linux system must be world-readable, and anyone with shell access can read any of those files. Under certain circumstances these files could include mysql passwords, even login passwords for your credit card merchant account.
How you ask? You may have (because a client asked you to) given chmod 777 to a client's files so his/her improperly designed php and/or cgi file would work. Or you may be running a DA server installed long ago, in which users were created with world-readable directory paths. While this is no longer the case, there may be some accounts on the server that still allow this.
Third, even without a specific exploit, any user with shell access can read your /etc/passwd and /etc/group directories. While modern distributions don't include passwords in these files, the user with shell access can see the complete list of unix users on the server, and perhaps even if you've created any users with root privileges. And knowing usernames makes it a lot easier to begin a brute-force attack.
Fourth, and perhaps most common, many users wan't ssh access so they can set up their own outgoing email server that doesn't write logs, for spamming... or their own chat servers they don't want you to find, or whatever.
There was a time on the 'net when most ISPs offered shell accounts to their users. Then most of us (I was an access ISP in those days) stopped because of what our users were doing with their shell accounts.
If you allow shell access most of your users won't abuse it. But you may just attract some of those malicious (or just annoying users) who want to look around your system to see what they can see and do.
That said, we've had good results giving shell access to users who've expressed a valid need for it.
Jeff