The Visual Display of Quantitative Information (Tufte)
Designing Web Usability: The Practice of Simplicity (Nielsen)
Homepage Usability: 50 Websites Deconstructed (Nielsen)
For web hosting I use - and highly recommend - Pair Networks. They're not the cheapest service, but I've found them very reliable over the last eight years, much more so than some of the alternatives. And for domain registration I use pairNIC.
Usable Web has a great collection of resources.
Philip Greenspun has written an excellent book Philip and Alex's Guide to Web Publishing (available online; also see my review of an older edition.)
Webmaster World has information on almost everything to do with web sites.
If you want to see examples of web sites I've designed or played a major role in desiging, check out this site itself (a fairly heterogeneous, small-medium personal site) and my book reviews (a medium single-focus site).
I worked as a volunteer managing the Community Aid Abroad (Oxfam Australia) web site for half a decade, but it has massively changed since then.