Inserting Special Characters From JavaScript

richard.york · June 8th, 2005, 10:06 AM

I am currently working on a user registration PHP script. On the client-side I've included an AJAX application that dynamically updates the states/provinces/municipalities based on the country selected. Where I'm running into problems is with special characters, particularly with German, Swedish, and Spanish municipalities, which all have at least one or two with special characters.

The AJAX application is called, and the server responds with the following XML.

Code:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<response>
    <lang>de</lang>
    <state id='0'>Please select a state or province</state>
    <state id='79'>Niedersachsen</state>
    <state id='80'>Baden-W?rttemberg</state>
    <state id='81'>Bayern</state>
    <state id='82'>Berlin</state>
    <state id='83'>Brandenburg</state>
    <state id='84'>Bremen</state>
    <state id='85'>Hamburg</state>
    <state id='86'>Hessen</state>
    <state id='87'>Mecklenburg-Vorpommern</state>
    <state id='88'>Nordrhein-Westfalen</state>
    <state id='89'>Rheinland-Pfalz</state>
    <state id='90'>Saarland</state>
    <state id='91'>Sachsen</state>
    <state id='92'>Sachsen-Anhalt</state>
    <state id='93'>Schleswig-Holstein</state>
    <state id='94'>Th?ringen</state>
</response>

I've tried switching out the special characters with entities, such as ü, though this doesn't work. I've also tried setting the lang attribute, but that doesn't work either.

I'm using a <select> box, when the response is received the following function parses through the XML, truncates, and then populates the <select> box.

Code:

function process_states()
{
    debug(http.responseText, 'register-debug');

    if (document.getElementById('register-state'))
    {
        var state = document.getElementById('register-state');
        var _lang = root.getElementsByTagName('lang')[0].firstChild.data;

        state.lang = _lang;

        // truncate the options
        state.length = 0;

        var states = root.getElementsByTagName('state');

        for (var i = 0; i < states.length; i++)
        {
            state.options[i] = new Option(states[i].firstChild.data, states[i].getAttribute('id'));
            state.options[i].lang = _lang;
        }

        //document.getElementById('register-state').innerHTML = http.responseText;
        if (state.length == 1)
        {
            alert('Aye! Cucaracha!');
        }
    }
}

The document that this is going into is XHTML 1.0 strict, servered as text/html with no XML declaration.

I'm using a <meta> tag to specify a UTF-8 character set.

Code:

<meta http-equiv='content-type' content='text/html; charset=utf-8' />

Explorer chokes hard on the special characters, and throws JavaScript errors due to their existence. Firefox just displays question marks.

This seems to be a character set issue. Any ideas on how I can internationalize gracefully?

Regards,
Rich

--
[http://www.smilingsouls.net]
Mail_IMAP: A PHP/C-Client/PEAR solution for webmail
Author: Beginning CSS: Cascading Style Sheets For Web Design

nzakas · June 8th, 2005, 10:56 AM

This may be beside the point, but shouldn't the special characters begin with an ampersand? For example:

Baden-W#65533;rttemberg

Instead of

Baden-W#65533;rttemberg

Nicholas C. Zakas
Author, Professional JavaScript for Web Developers (ISBN 0764579088)
http://www.nczonline.net/

richard.york · June 8th, 2005, 11:12 AM

Well, that happened when I submitted the code that I copied and pasted from Firefox. Those actually show up as question marks, and there are no entites being used, though I have tried entities, like ü already. Right now I'm fooling with the content-type HTTP header, and I just discovered my XML wasn't being sent as application/xml but as text/html. Neither of those changes have made any difference so far. The problem seems to center around the new Option() method, which I am experimenting with now.

Regards,
Rich

--
[http://www.smilingsouls.net]
Mail_IMAP: A PHP/C-Client/PEAR solution for webmail
Author: Beginning CSS: Cascading Style Sheets For Web Design

richard.york · June 9th, 2005, 11:12 AM

For those interested, I've narrowed this down a bit.

Code:

state.options[i] = new Option('&amp;', 0);

Here & is not evaluated into an amphersand. I've also tried inserting raw unicode like \u00F1, but that doesn't work when transmitted via the firstChild.data method, but:

Code:

state.options[i] = new Option('\u00F1', 0);

does work, that evaluates to the n with tilde. Now I'm looking for a method of accessing the data that will evaluate the raw unicode contained therein.

Regards,
Rich

--
[http://www.smilingsouls.net]
Mail_IMAP: A PHP/C-Client/PEAR solution for webmail
Author: Beginning CSS: Cascading Style Sheets For Web Design

richard.york · June 9th, 2005, 11:39 AM

At last, a method that works.

Code:

state.options[i] = new Option(eval('"' + states[i].firstChild.data + '";'), states[i].getAttribute('id'));

I've heard that eval() is pretty slow and inefficient, but it seems to do the job.

Regards,
Rich

--
[http://www.smilingsouls.net]
Mail_IMAP: A PHP/C-Client/PEAR solution for webmail
Author: Beginning CSS: Cascading Style Sheets For Web Design

nzakas · June 9th, 2005, 01:04 PM

I'm wondering if you could use a DOM method and avoid the eval() hack, something like:

Code:

var oOption = document.createElement("option");
oOption.setAttribute("value", "whatever");
oOption.appendChild(document.createTextNode(states[i].firstChild.data));
oSelect.appendChild(oOption);

Nicholas C. Zakas
Author, Professional JavaScript for Web Developers (ISBN 0764579088)
http://www.nczonline.net/