An XSLT stylesheet is an XML document and you will get an XML parser error if you include an unescaped <. You can escape the character either by writing it as < or by putting it in a CDATA section - the two approaches are 100% equivalent.
However, the error "Error: Expected ')'" is clearly from the Javascript parser rather than the XML parser.
Either way, the problem has nothing to do with XSLT.
Michael Kay
http://www.saxonica.com/
Author, XSLT Programmer's Reference and XPath 2.0 Programmer's Reference