JBoss AS configuration needs some updates. Please, see ticket SUPPORT-5799 and Matteo Pelucco coments in documentation page.

1) BouncyCastle: in magnolia 5.4.5 we are using versions: 1.46. Resteasy override some of this bouncycastle libraries and it generates some errors.

2) JBoss deployment structure XML file needs to be updated to version 1.2: <jboss-deployment-structure xmlns="urn:jboss:deployment-structure:1.1"> should be changed to <jboss-deployment-structure xmlns="urn:jboss:deployment-structure:1.2">, otherwise <exclude-subsystems> node is not accepted.

3) JAVA_OPTS memory settings
You mention to edit file run.conf but it does not exist (anymore?). The right file is standalone.conf.bat
And by default JBoss EAP standalone runs with 1024M (1G) of heap (max / initial), so there is the need to change only MaxPermSize (but on Java8 this is not required anymore). Why there is an option "-server"?

4) Still happening in version 5.4.5: https://documentation.magnolia-cms.com/display/DOCS/JBoss+AS+configuration#JBossASconfiguration-MagnoliaWorkflow(prior5.4)

5) Maybe we can include a reference to this issue: Infinispan transactions is a known issue MGNLUI-2652 that will be fixed in version 5.5 (MGNLUI-351 ).