Its up to you really. Different things are appropriate for different uses.
You will need some understanding of html, even if you're not planning on doing much design work. Your web apps will typically spit out html at some point. I'd advise learning xhtml 1.0 strict or xhtml 1.1 - it will get you into various good habits. Have a look at the w3c tutorial site or get a good book and work through that.
CSS is necessary if you'll be doing your own design. Good web pages seperate structure (html) and presentation (css).
JSP and Servlets are tools for getting a server dynamically to generate html, usually in conjunction with a database. So when you get to these you'll be doing "programming proper". If you're using a database you may need to learn a little SQL and how to install and admin your database; you might also want to learn how to configure your webserver properly.
Charlie Harvey's website - linux, perl, java, anarchism and punk rock: http://charlieharvey.org.uk