>and I don't understand it
As my maths teacher used to ask, which part of it don't you understand?
<xsl:for-each select="x[position() mod $N = 1]"
Assuming $N is 3, this will select x[1], x[4], x[7] and so on. It will then output a new row in your output table once for each one of these selected elements.
<xsl:for-each select=". | following-sibling::x[position() < $N]">
Assuming $N is 3 and you are processing x[1], this will select x[1], x[2], and x[3] and output a new cell for each of these three elements. Similarly when you are processing x[4], it will output a new cell for each of x[4], x[5], and x[6].
I hope that makes it clearer and will enable you to start debugging your code.
Michael Kay
http://www.saxonica.com/
Author, XSLT 2.0 and XPath 2.0 Programmer's Reference