When default ubuntu language support for Simplified Chinese is installed, wqy fonts will be set as default, and even your own .fonts.conf will be blinded. To fix this, you need to edit /etc/fonts/conf.d/44-wqy-zenhei.conf and comment down this arrogant configuration part:
<!-- <alias> <family>serif</family> <prefer> <family>Bitstream Vera Serif</family> <family>DejaVu Serif</family> <family>WenQuanYi Zen Hei</family> </prefer> </alias> <alias> <family>sans-serif</family> <prefer> <family>Bitstream Vera Sans</family> <family>DejaVu Sans</family> <family>WenQuanYi Zen Hei</family> </prefer> </alias> <alias> <family>monospace</family> <prefer> <family>Bitstream Vera Sans Mono</family> <family>DejaVu Sans Mono</family> <family>WenQuanYi Zen Hei Mono</family> </prefer> </alias> -->
And then your own ~/.fonts.conf will work again.