Three days after the initial ranking drop, I started to see traffic jump across the server. I regretted the move after I found the new server had hardware problems, now I’m very happy. 
There are two possible reasons, if it’s not because of search engines’ random changes:
- I’m using much more unique IPs, which is favored by Google. 14 IPs cost only $7.5/month, I should have ordered more, it’s even cheaper.
- A slightly faster server, which makes more visitors patient enough to wait for the page load.
Anyway, it doesn’t hurt to give your websites a decent hosting.