The template eachresult is going to generate a sequence something like
<b>Some Title</b>
<br/>
Some description
<br/>
Some author, 2007-02-15
<br/>
So the value of $groupedset will be a document node whose contents are something like:
<b>Some Title</b>
<br/>
Some description
<br/>
Some author, 2007-02-15
<br/>
<b>Some other Title</b>
<br/>
Some other description
<br/>
Some other author, 2007-02-15
<br/>
You're then doing $groupset/*[position() mod $X = 1]. If $X is 3, you're going to select
<b>Some Title</b>
<br/>
<br/>
which doesn't make much sense. You're not selecting the text nodes at all. I think you want eachresult to create a wrapper element around this lot, and then you probably want the copy-of to drop the wrapper element. Say:
<xsl:template name="eachresult">
<result>
<b><xsl:value-of select="TITLE"/></b><br /> <xsl:value-of select="DESCRIPTION"/><br /> <xsl:value-of select="AUTHOR"/>, <xsl:value-of select="PUBLISHDATE"/><br />
</result>
</xsl:template>
then
<xsl:for-each select="exsl:node-set($groupedset)/result[position() mod $recordsPerPage = 1]">
<xsl:copy-of select="(.|following-sibling::*[position() < $recordsPerPage])/node()"/>
</xsl:for-each>
except that this isn't actually splitting it into pages; I would expect to see something like
<xsl:for-each select="exsl:node-set($groupedset)/result[position() mod $recordsPerPage = 1]">
<page>
<xsl:copy-of select="(.|following-sibling::*[position() < $recordsPerPage])/node()"/>
</page>
</xsl:for-each>
Michael Kay
http://www.saxonica.com/
Author, XSLT Programmer's Reference and XPath 2.0 Programmer's Reference