I have a domain.. like domain.com.
Domain.com refferals to the server X, and I want that the sub-domain server2.domain.com will refferal to the server Y.
If I am understanding your question correctly, this is doable. Create an A record in the DNS zone for domain.com called server2.domain.com and give it the IP address of the second server.