Repository navigation
Open
Description
I am trying to package python-xmlsec for Arch, but unfortunately most tests are failing here. Versions:
Python 3.6.6/3.7.0
libxml2 2.9.8
xmlsec 1.2.26
Cflags: -DXMLSEC_CRYPTO_DYNAMIC_LOADING=1 -D__XMLSEC_FUNCTION__=__func__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_DL_LIBLTDL=1 -I/usr/include/xmlsec1 -I/usr/include/libxml2
Example test failure (others are very similar):
============================= test session starts ==============================
platform linux -- Python 3.7.0, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /build/python-xmlsec/src/python-xmlsec-1.3.3, inifile:
collected 64 items
tests/test_ds.py ..FE.FEFEFEFEFE..FEFEFEFEFE [ 25%]
tests/test_enc.py FEFEFEFEFE.. [ 35%]
tests/test_keys.py ......................... [ 75%]
tests/test_templates.py FEFEFEFEFEFEFEFEFEFEFEFE [ 93%]
tests/test_tree.py FEFEFEFE [100%]
==================================== ERRORS ====================================
____________ ERROR at teardown of TestSignContext.test_register_id _____________
self = <tests.test_ds.TestSignContext testMethod=test_register_id>
def test_register_id(self):
ctx = xmlsec.SignatureContext()
> root = self.load_xml("sign_template.xml")
tests/test_ds.py:22:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/base.py:97: in load_xml
root = etree.parse(self.path(name)).getroot()
src/lxml/etree.pyx:3426: in lxml.etree.parse
???
src/lxml/parser.pxi:1840: in lxml.etree._parseDocument
???
src/lxml/parser.pxi:1866: in lxml.etree._parseDocumentFromURL
???
src/lxml/parser.pxi:1770: in lxml.etree._parseDocFromFile
???
src/lxml/parser.pxi:1163: in lxml.etree._BaseParser._parseDocFromFile
???
src/lxml/parser.pxi:601: in lxml.etree._ParserContext._handleParseResultDoc
???
src/lxml/parser.pxi:711: in lxml.etree._handleParseResult
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E File "b'/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'", line 0
E lxml.etree.XMLSyntaxError: <no detail available>
src/lxml/parser.pxi:651: XMLSyntaxError
----------------------------- Captured stderr call -----------------------------
func=xmlSecNoXxeExternalEntityLoader:file=xmlsec.c:line=59:obj=unknown:subj=xmlSecNoXxeExternalEntityLoader:error=5:libxml2 library function failed:illegal external entity='/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'; xml error: 0: NULL
func=xmlSecNoXxeExternalEntityLoader:file=xmlsec.c:line=59:obj=unknown:subj=xmlSecNoXxeExternalEntityLoader:error=5:libxml2 library function failed:illegal external entity='/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'; xml error: 0: NULL
func=xmlSecNoXxeExternalEntityLoader:file=xmlsec.c:line=59:obj=unknown:subj=xmlSecNoXxeExternalEntityLoader:error=5:libxml2 library function failed:illegal external entity='/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'; xml error: 0: NULL
func=xmlSecNoXxeExternalEntityLoader:file=xmlsec.c:line=59:obj=unknown:subj=xmlSecNoXxeExternalEntityLoader:error=5:libxml2 library function failed:illegal external entity='/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'; xml error: 0: NULL
func=xmlSecNoXxeExternalEntityLoader:file=xmlsec.c:line=59:obj=unknown:subj=xmlSecNoXxeExternalEntityLoader:error=5:libxml2 library function failed:illegal external entity='/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'; xml error: 0: NULL
func=xmlSecNoXxeExternalEntityLoader:file=xmlsec.c:line=59:obj=unknown:subj=xmlSecNoXxeExternalEntityLoader:error=5:libxml2 library function failed:illegal external entity='/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'; xml error: 0: NULL
func=xmlSecNoXxeExternalEntityLoader:file=xmlsec.c:line=59:obj=unknown:subj=xmlSecNoXxeExternalEntityLoader:error=5:libxml2 library function failed:illegal external entity='/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'; xml error: 0: NULL
func=xmlSecNoXxeExternalEntityLoader:file=xmlsec.c:line=59:obj=unknown:subj=xmlSecNoXxeExternalEntityLoader:error=5:libxml2 library function failed:illegal external entity='/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'; xml error: 0: NULL
func=xmlSecNoXxeExternalEntityLoader:file=xmlsec.c:line=59:obj=unknown:subj=xmlSecNoXxeExternalEntityLoader:error=5:libxml2 library function failed:illegal external entity='/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'; xml error: 0: NULL
func=xmlSecNoXxeExternalEntityLoader:file=xmlsec.c:line=59:obj=unknown:subj=xmlSecNoXxeExternalEntityLoader:error=5:libxml2 library function failed:illegal external entity='/build/python-xmlsec/src/python-xmlsec-1.3.3/tests/data/sign_template.xml'; xml error: 0: NULL
...
=============== 32 failed, 32 passed, 32 error in 1.73 seconds ================
Metadata
Metadata
Assignees
Labels
No labels
Activity
hoefling commentedon May 21, 2020
@felixonmars if this is still an issue, can you please check against the latest 1.3.8 release?
felixonmars commentedon May 21, 2020
Yes, it's still present in 1.3.7/1.3.8 releases.
Full logs w/versions: https://paste.xinu.at/HRD/
hoefling commentedon May 21, 2020
@felixonmars my first guess would be something is borked with the
python-lxmldependency. If I resort to themanylinuxwheel, e.g.in the
PKGBUILD, the tests pass. I will try looking further into it.felixonmars commentedon May 22, 2020
It seems the lxml here has some issues with runpy, that it fails the second time running the same example:
Using
subprocess.Popenhere instead of runpy makes the tests pass.lilydjwg commentedon May 22, 2020
Here is my findings. Put together a xml file and run this:
It reproduces. The second
parsecall only triggers astatcall but no fileopenandreadcalls.lxml, xmlsec and libxml2 are all coming from my Arch Linux.
It looks like the lxml wheel from PyPI (installed by pip) has its own libxml2, but xmlsec dynamically links to the system one, so xmlsec changes one libxml2 while lxml uses another, avoiding this issue. (But I don't know if it's OK to have two libxml2 libraries like this.)
I don't know how to use the C libraries so no reproduction tries without Python.
hoefling commentedon May 22, 2020
@lilydjwg your comment was of an immense help and it brought me on the right track, thank you!
I found out that this issue is caused by
lxmlreusing the XML parser object: e.g.works, but
raises. The same issue arises when calling
etree.parse('doc.xml')- the default parser stored globally is reused.The workaround to this issue is thus simple - resetting the default parser between the
load_xmlinvocations should be enough to fix the failing tests. However, this doesn't fix the underlying issue #96 yet. I'll poke around some more and open an issue againstlxmllaunchpad if won't be able to resolve it myself.hoefling commentedon May 22, 2020
In the meantime, a simple patch like e.g.
will fix the test run, but I'm sure this is not what you're after in the end.
hoefling commentedon May 22, 2020
For reference: https://bugs.launchpad.net/lxml/+bug/1880251
odidev commentedon Jul 13, 2020
@hoefling, Thanks for the above solution.
All tests are passing after upgrading lxml to 4.5.2 in amd64 platform, but in ARM64/AArch64 platform even after upgrading lxml, I am getting same lxml syntax error and all tests are passing after incorporating above mentioned changes.
It will be helpful if the above mentioned changes are added or suggest if I am missing any installation of any other dependencies.
hoefling commentedon Jul 16, 2020
@odidev if you're getting the same error with
lxml==4.5.2, then the issue is not completely resolved on lxml side (I also think there's more to do as the example I posted in this comment fails with 4.5.2). I'd open a new issue with a reproduction example at lxml's launchpad.hoefling commentedon Jul 16, 2020
Follow-up issue: https://bugs.launchpad.net/lxml/+bug/1887848
Update python-xmlsec to version 1.3.12 / rev 6 via SR 976452
xmlsec workaround for gh#xmlsec#84
xmlsec workaround for gh##84
felixonmars commentedon Aug 31, 2022
There are still three failures here with 1.3.13 and lxml 4.9.1:
Looks like there's no activity on the follow-up lxml issue.