Checking for unique nodes

madowl · June 27th, 2012, 11:52 AM

Good morning,
I am using xslt 1.0 and need to parse the embedded xml file and only pick up lines where the yearIndicator,originalID, and studentSSN are unique with the greatest transactionNumber.
I was using the following but didnt have much luck:
<code>
<xsl:variable name="studentSSN" select="flat:field[@name = 'studentSSN']"/>
<xsl:variable name="yearIndicator" select="./flat:datum[@name = 'yearIndicator']"/>
<xsl:variable name="originalID" select="./flat:datum[@name = 'originalID']"/>
<xsl:variable name="transactionNumber" select="./flat:datum[@name = 'transactionNumber']"/>
<xsl:if test="not(preceding::flat:datum[text() = $yearIndicator]) and not(preceding::flat:datum[text() = $studentSSN]) and not(preceding::flat:datum[text() = $originalID])" >
<xsl:apply-templates select="node()|@*"/>
</xsl:if>
</code>
A sample xml file is as follows:
<code>
<flat:datum type="record">
<flat:field name="yearIndicator">2</flat:field>
<flat:field name="studentSSN">550951236</flat:field>
<flat:field name="originalID">RA</flat:field>
<flat:field name="transactionNumber">02</flat:field>
</flat:datum>
<flat:datum type="record">
<flat:field name="yearIndicator">2</flat:field>
<flat:field name="studentSSN">550951236</flat:field>
<flat:field name="originalID">RA</flat:field>
<flat:field name="transactionNumber">03</flat:field>
</flat:datum>
</code>
Hence in the above example, I would need to pick up the second record since the transactionNumber is greater and ignore the first.

Martin Honnen · June 27th, 2012, 12:06 PM

I would group with Muenchian grouping and then sort in descending order e.g.

Code:

<xsl:key name="k1" match="flat:datum[@type = 'record']"
  use="concat(flat:field[@name = 'yearIndicator'], '|', flat:field[@name = 'studentSSN'], '|', flat:field[@name = 'originalID'])"/>

<xsl:template match="/">
  <xsl:apply-templates select="//flat:datum[@type = 'record'][generate-id() = generate-id(key('k1', concat(flat:field[@name = 'yearIndicator'], '|', flat:field[@name = 'studentSSN'], '|', flat:field[@name = 'originalID']))[1])"/>
</xsl:template>

<xsl:template match="flat:datum[@type = 'record']">
  <xsl:for-each select="key('k1', concat(flat:field[@name = 'yearIndicator'], '|', flat:field[@name = 'studentSSN'], '|', flat:field[@name = 'originalID']))">
    <xsl:sort select="flat:field[@name = 'transactionNumber']" data-type="number" order="descending"/>
    <xsl:if test="position() = 1">
       <xsl:copy-of select="."/>
    </xsl:if>
  </xsl:for-each>
</xsl:template>

Untested but should give you an idea.

madowl · June 27th, 2012, 12:19 PM

Thank you very much Martin. This seems to work great (was missing a ] at the end)- sorry I wasnt familiar with this method. I appreciate your quick response. However, I am noticing that that there are certain elements before these datum tags that are not being copied over. I guess I would need to copy the whole template over?

Martin Honnen · June 27th, 2012, 12:42 PM

You didn't really say what you want to do with other elements, so the sample I posted simply extracts those unique elements with the highest transactionNumber. You have now posted a very long input sample but you haven't said or described what result you want. I am not sure what you want to do with other elements. Consider to post minimal but representative samples of input and desired output. And to allow us to read code samples please use http://p2p.wrox.com/misc.php?do=bbcode#code

madowl · June 27th, 2012, 12:56 PM

Sorry about that Martin. What you did was exactly what I was looking for with regards to sorting..however, the other elements would just need to be copied over..hence the sample output file - after your processing would be similar to:

Code:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<vocado version="1.25.7-SNAPSHOT_20238-120614.1030" xsi:schemaLocation="http://vocado.vsm-systems.com/xsd/vocado-tx vocado-tx.xsd
http://vocado.vsm-systems.com/xsd/flat-plugin flat-plugin.xsd
http://vocado.vsm-systems.com/xsd/xslt-plugin xslt-plugin.xsd
http://vocado.vsm-systems.com/xsd/fop-plugin fop-plugin.xsd
http://vocado.vsm-systems.com/xsd/csv-plugin csv-plugin.xsd
" xmlns="http://vocado.vsm-systems.com/xsd/vocado-tx" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xslt="http://vocado.vsm-systems.com/xsd/xslt-plugin" xmlns:flat="http://vocado.vsm-systems.com/xsd/flat-plugin" xmlns:csv="http://vocado.vsm-systems.com/xsd/csv-plugin" xmlns:fop="http://vocado.vsm-systems.com/xsd/fop-plugin" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <definitions>
        <entities>
            <entity name="isir">
                <field type="text" label="Original Social Security Number" name="studentSsn" global="true">
                    <key global="true" name="ssn"/>
                </field>
                <field type="number" label="Year indicator" name="yearIndicator" global="true">
                    <key global="true" name="yearIndicator"/>
                </field>
                <field type="text" label="Original Name ID" name="originalNameId" global="true">
                    <key global="true" name="originalNameId"/>
                </field>
                <field type="number" label="The CPS transaction number of this ISIR." name="transactionNumber" global="true">
                    <key global="true" name="transactionNumber"/>
                </field>
            </entity>
        </entities>
        <scopes>
            <scope global="true" datum-type="isir" name="ISIR.scope">
                <field datum-type="isir" datum-field="yearIndicator" xpath="isir/yearIndicator" name="yearIndicator"/>
            </scope>
        </scopes>
        <flat:entities>
            <entity name="record">
                <field type="text" name="yearIndicator"/>
                <field type="text" name="studentSSN"/>
                <field type="text" name="originalID"/>
                <field type="text" name="transactionNumber"/>
</entity>
        </flat:entities>
    </definitions>
<flat:data>
        <flat:datum type="record">
            <flat:field name="yearIndicator">2</flat:field>
            <flat:field name="studentSSN">550951236</flat:field>
            <flat:field name="originalID">RA</flat:field>
            <flat:field name="transactionNumber">03</flat:field>
</flat:datum>
    </flat:data>
</vocado>

Martin Honnen · June 27th, 2012, 01:06 PM

I think you want the identity transformation plus code for the 'flat:data' element containing those record elements to find the unique ones with the greatest transaction number so

Code:

<xsl:key name="k1" match="flat:datum[@type = 'record']"
  use="concat(flat:field[@name = 'yearIndicator'], '|', flat:field[@name = 'studentSSN'], '|', flat:field[@name = 'originalID'])"/>


<xsl:template match="@* | node()">
  <xsl:copy>
     <xsl:apply-templates select="@* | node()"/>
  </xsl:copy>
</xsl:template>

<xsl:template match="flat:data">
  <xsl:copy>
    <xsl:apply-templates select="flat:datum[@type = 'record'][generate-id() = generate-id(key('k1', concat(flat:field[@name = 'yearIndicator'], '|', flat:field[@name = 'studentSSN'], '|', flat:field[@name = 'originalID']))[1])]"/>
  </xsl:copy>
</xsl:template>

<xsl:template match="flat:datum[@type = 'record']">
  <xsl:for-each select="key('k1', concat(flat:field[@name = 'yearIndicator'], '|', flat:field[@name = 'studentSSN'], '|', flat:field[@name = 'originalID']))">
    <xsl:sort select="flat:field[@name = 'transactionNumber']" data-type="number" order="descending"/>
    <xsl:if test="position() = 1">
       <xsl:copy-of select="."/>
    </xsl:if>
  </xsl:for-each>
</xsl:template>

Again untested so might contain syntax errors like missing matching brackets but should give you an idea.

madowl · June 27th, 2012, 01:17 PM

Thank you very very much Martin!!! This seems to work very well. I appreciate the quick response and great guidance.

madowl · June 27th, 2012, 01:22 PM

A quick question Martin, what does the "1" at the end of the generate-id signify?

Code:

<xsl:apply-templates select="flat:datum[@type = 'record'][generate-id() = generate-id(key('isirKey', concat(flat:field[@name = 'yearIndicator'], '|', flat:field[@name = 'studentSSN'], '|', flat:field[@name = 'originalID']))[1])]"/>

Martin Honnen · June 27th, 2012, 01:31 PM

It's a positional predicate applied on the result of the key function (not the generate-id) so with

Code:

key('key-name', keyExpression)[1]

we ensure that the first node in document order found by the key function is used to generate the id from. The generate-id might do that automatically I think but for efficiency depending on the XSLT processor the [1] might help. Read more about Muenchian grouping at http://www.jenitennison.com/xslt/grouping/muenchian.xml

madowl · June 27th, 2012, 01:35 PM

Thanks again! much appreciated