There's currently no intuitive way to configure access logging in the web subsystem. In the CLI I can get to what seems like the right area:

[standalone@localhost:9999 /] cd /subsystem=web/virtual-server=default-host
[standalone@localhost:9999 virtual-server=default-host] ls
access-log rewrite
sso alias=["localhost","example.com"]
default-web-module=ROOT.war enable-welcome-root=true

and even

[standalone@localhost:9999 virtual-server=default-host] cd access-log
[standalone@localhost:9999 access-log] pwd
/subsystem=web/virtual-server=default-host/access-log

but it's absolutely unclear what to do next.