[wleeaf]

Since 3.3.1, a processing instruction inside the internal subset raises Malformed DOCTYPE: invalid declaration:

REXML::Document.new(<<~XML)
  <!DOCTYPE r [
  <?target content?>
  <!ELEMENT r EMPTY>
  ]>
  <r/>
XML
# 3.3.0:  parses
# master: REXML::ParseException: Malformed DOCTYPE: invalid declaration

PIs are allowed there: [28b] intSubset ::= (markupdecl | DeclSep)* and [29] markupdecl includes PI. Expat (Python's xml.etree) also accepts this. The :in_doctype branch of BaseParser#pull_event handles <!... declarations, parameter entity references and ]>, but not <?. This change parses PIs there with process_instruction, as outside the DOCTYPE. An XML declaration in the internal subset is still rejected ("XML declaration is not at the start").

I found this by running the W3C XML Conformance Test Suite (xmlts20130923) against REXML: six valid documents that only failed because of this now parse (oasis p29pass1 and IBM P02/P03/P28/P29 valid tests).

Tests: test_instruction_element (DOCTYPE mixed content), test_in_internal_subset and test_xml_declaration_in_internal_subset (processing instructions). All three fail before the change (the third got the generic "invalid declaration" error) and pass after it. bundle exec rake test passes (844 tests), as does bundle exec rake test RUBYOPT="--enable-frozen-string-literal".

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.