vsftpd has a very annoying inplementation and configuration compared to pureftpd and alike.
Part of the annoyance is treating virtual users as guest users and guest users as anonymous users! To fix this you can the anonymous umask to be wide open but this has obvious security concerns if you also have local users as I have.
The fix I found is to set the following to YES in the vsftpd.conf:
If enabled, virtual users will use the same privileges as local users. By default, virtual users will use the same privileges as anonymous users, which tends to be more restrictive (especially in terms of write access).
This way only ftp user can mess with the files and no one has access to the ftp account. I recommend leaving hte local umask as default instead of 022 so others can't read.