VNC'ing is taking FORVER to log in

VNC'ing is taking FORVER to log in

I have my Ubuntu 13.10 server set up for vnc. I have VNC installed that I
think came with stock Ubuntu.
I've tried to VNC into it from both my Macbook and Windows Computer. Same
thing happens on both of the machines. Once I put my log in credentials,
it loads, but loads for infinity. It never stops. It just constantly tries
to connect. It doesn't even time out (I don't know if VNC is suppose to
timeout).
I can ping and ssh into the server, but it just tries to VNC into it. I've
left it go for hours, and it would still be sitting there trying to log
in. I'm not sure what the problem is. On both the Macbook and Windows,
they never give me an error.
What am I missing?
I was going off this source.