I'm trying to store user-input in an XML document on the client-side (javascript), and transmit that to the server for persistence.

One user, for example, pasted in text that included an STX character (0x2). The XMLSerializer did not escape the STX character, and therefore, did not serialize to well-formed XML. Or perhaps the .attr() call should have escaped the STX character, but in either case, invalid XML was produced.

I'm finding the output of in-browser XMLSerializer() isn't always well-formed, (and doesn't even satisfy the browser's own DOMParser()

This example shows that the STX character is not properly encoded by XMLSerializer():

> doc = $.parseXML('<?xml version="1.0" encoding="utf-8" ?>\n<elem></elem>');
    #document
> $(doc).find("elem").attr("someattr", String.fromCharCode(0x2));
    [ <elem someattr=​"">​</elem>​ ]
> serializedDoc = new XMLSerializer().serializeToString(doc);
    "<?xml version="1.0" encoding="utf-8"?><elem someattr=""/></elem>"
> $.parseXML(serializedDoc);
    Error: Invalid XML: <?xml version="1.0" encoding="utf-8"?><elem someattr=""/></elem>

How should I construct an XML document in-browser (with params determined by arbitrary user-input) such that it will always be well-formed (everything properly escaped)? I don't need to support IE8 or IE7.

(And yes, I do validate the XML on the server side, but if the browser hands the server a document that is not well-formed, the best the server can do is reject it, which isn't that helpful to the poor user)

I'm trying to store user-input in an XML document on the client-side (javascript), and transmit that to the server for persistence.

One user, for example, pasted in text that included an STX character (0x2). The XMLSerializer did not escape the STX character, and therefore, did not serialize to well-formed XML. Or perhaps the .attr() call should have escaped the STX character, but in either case, invalid XML was produced.

I'm finding the output of in-browser XMLSerializer() isn't always well-formed, (and doesn't even satisfy the browser's own DOMParser()

This example shows that the STX character is not properly encoded by XMLSerializer():

> doc = $.parseXML('<?xml version="1.0" encoding="utf-8" ?>\n<elem></elem>');
    #document
> $(doc).find("elem").attr("someattr", String.fromCharCode(0x2));
    [ <elem someattr=​"">​</elem>​ ]
> serializedDoc = new XMLSerializer().serializeToString(doc);
    "<?xml version="1.0" encoding="utf-8"?><elem someattr=""/></elem>"
> $.parseXML(serializedDoc);
    Error: Invalid XML: <?xml version="1.0" encoding="utf-8"?><elem someattr=""/></elem>

How should I construct an XML document in-browser (with params determined by arbitrary user-input) such that it will always be well-formed (everything properly escaped)? I don't need to support IE8 or IE7.

(And yes, I do validate the XML on the server side, but if the browser hands the server a document that is not well-formed, the best the server can do is reject it, which isn't that helpful to the poor user)

• javascript
• xml
• xmlserializer
• well-formed

• I'm not sure there's anything much easier than going through the source string character-by-character, translating to entities as necessary. – Pointy Commented Feb 2, 2013 at 19:01
• I wouldn't trust myself to do this (I don't even know XML well enough to look for OTHER possible issues)... is a mon/standard JS library to do this for me makeSafeForXML(inString) ? – Seth Commented Feb 2, 2013 at 19:54
• Also, wouldn't you end up possibly double-entitizing by accident? For example, if in a future browser XMLSerializer()+attr() end up entitizing, you'll end up double-escaping? – Seth Commented Feb 2, 2013 at 19:55
• well I didn't mean to say it was easy in absolute terms :-) I guess somebody may have done something like that and made it available, but it seems like a sort-of unusual problem; usually in JavaScript in the browser, your code is the recipient of XML, not the origin. – Pointy Commented Feb 2, 2013 at 20:29
• You cannot have character 0x2 in XML. It's illegal in XML version 1.0. The only allowed control characters are \r and \n, so that's an error right there. – Tomalak Commented Feb 2, 2013 at 21:21

2 Answers 2

Here's a function sanitizeStringForXML() which can either be used to cleanse strings before assignment, or a derivative function removeInvalidCharacters(xmlNode) which can be passed a DOM tree and will automatically sanitize attributes and textNodes so they are safe to store.

var stringWithSTX = "Bad" + String.fromCharCode(2) + "News";
var xmlNode = $("<myelem/>").attr("badattr", stringWithSTX);

var serializer = new XMLSerializer();
var invalidXML = serializer.serializeToString(xmlNode);

// Now cleanse it:
removeInvalidCharacters(xmlNode);
var validXML = serializer.serializeToString(xmlNode);

I based this on a list of characters from the non-restricted characters section of this wikipedia article, but the supplementary planes require 5-hex-digit unicode characters, and the Javascript regex does not include a syntax for this, so for now, I'm just stripping them out (you aren't missing too much...):

// WARNING: too painful to include supplementary planes, these characters (0x10000 and higher) 
// will be stripped by this function. See what you are missing (heiroglyphics, emoji, etc) at:
// http://en.wikipedia/wiki/Plane_(Unicode)#Supplementary_Multilingual_Plane
var NOT_SAFE_IN_XML_1_0 = /[^\x09\x0A\x0D\x20-\xFF\x85\xA0-\uD7FF\uE000-\uFDCF\uFDE0-\uFFFD]/gm;
function sanitizeStringForXML(theString) {
    "use strict";
    return theString.replace(NOT_SAFE_IN_XML_1_0, '');
}

function removeInvalidCharacters(node) {
    "use strict";

    if (node.attributes) {
        for (var i = 0; i < node.attributes.length; i++) {
            var attribute = node.attributes[i];
            if (attribute.nodeValue) {
                attribute.nodeValue = sanitizeStringForXML(attribute.nodeValue);
            }
        }
    }
    if (node.childNodes) {
        for (var i = 0; i < node.childNodes.length; i++) {
            var childNode = node.childNodes[i];
            if (childNode.nodeType == 1 /* ELEMENT_NODE */) {
                removeInvalidCharacters(childNode);
            } else if (childNode.nodeType == 3 /* TEXT_NODE */) {
                if (childNode.nodeValue) {
                    childNode.nodeValue = sanitizeStringForXML(childNode.nodeValue);
                }
            }
        }
    }
}

Note that this only removes invalid characters from nodeValues of attributes and textNodes. It does not check tag names or attribute names, ments, etc etc.

Check https://gist.github./john-doherty/b9195065884cdbfd2017a4756e6409cc,

very useful gist, example usage:

const resultXml = removeXMLInvalidChars(INPUT_XML_STRING, true);