[imps] local install on Centos 6, BUILD SUCCESSFUL, but not quite ...

[Martin Sharratt]

Hi Andrei

Afraid I was a bit ‘brute force’ with this, I configured the firewall to allow my server to connect to any external host on ports 80 and 443.   I didn’t have enough time to work out exactly which IP’s I needed to be able to talk to and our firewall doesn’t do DNS.  Luckily I didn’t have to ask the network people as I could add the relevant rule myself.  As an added bonus, I didn’t have to configure apt to use a proxy so updates are easy. From memory, after you’ve successfully built the validator it will try to start up and fail because it can’t get to the Internet.  Looking at the last log I’ve got, it seems only to need to get to http://s.validator.nu – see the copy of the log entries below.

nu.validator.servlet.VerifierServletTransaction - Parsing set up. Starting to read schemas.
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/html5/html5full.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/html5/html5full-aria.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/html5-aria-svg-mathml.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/xhtml10/xhtml-strict.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/xhtml10/xhtml.sch
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/xhtml10/xhtml-transitional.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/xhtml10/xhtml-frameset.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/html5/xhtml5full-xhtml.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/xhtml5-aria-rdf-svg-mathml.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/xhtml1-rdf-svg-mathml.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/xhtml1-ruby-rdf-svg-mathml.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/xhtml10/xhtml-basic.rnc
nu.validator.servlet.VerifierServletTransaction - Will load schema: http://s.validator.nu/svg-xhtml5-aria-rdf-mathml.rnc
nu.validator.servlet.VerifierServletTransaction - Schemas read.
nu.validator.servlet.VerifierServletTransaction - Reading spec.
nu.validator.servlet.VerifierServletTransaction - Spec read.
nu.validator.servlet.VerifierServletTransaction - Initialization complete.
2011-12-06 11:56:16.141:INFO::jetty-6.1.26
2011-12-06 11:56:16.355:INFO::Started SocketConnector at 0.0.0.0:8888

Martin S


From: implementors-bounces at lists.whatwg.org [mailto:implementors-bounces at lists.whatwg.org] On Behalf Of Rancid Iodine
Sent: 18 January 2012 12:12
To: implementors at lists.whatwg.org
Subject: Re: [imps] local install on Centos 6, BUILD SUCCESSFUL, but not quite ...


Thank you both Martin and Thomas for your feedback. I think the quickest way around this is
to get our network people to arrange for the firewall exception. I've looked into it from the
java perspective, there's too much trial and error / research involved and there are time


constraints.

BTW doing a export _JAVA_OPTS='-Dhttp.proxyPort=myproxy.co.uk<http://myproxy.co.uk> -Dhttp.proxyPort=80' prior
to running the build.py doesn't seem to help either.

So with these firewall exceptions (for outgoing connections on ports 80 and 443) in mind, my question


is now this: to WHICH target hosts on the internet shall we authorise the exceptions? The
network security people need to know, because naturally (as is their nature :) they refuse
to set up an exception to "just anywhere".


Thomas you mentioned various dependencies are needed from http://www.iana.org<http://www.iana.org/assignments/character-sets,> ... and I could do
a big rgrep and some sedding for http://* in checker/ to see about anything else, but there seems


to be so much that I'm not entirely sure where else is relevant in the massive directory tree under
checker/. I can also see more items being needed (possibly) from http://www.w3.org, and http://xml.org,


mentions of http://www.apps.ietf.org/, http://www.junit.org, and possibly other places from other
directories outside checker/dependencies. Are the problem downloads solely from .java modules? I would


assume so from what you're saying, but maybe not.

Does anyone have an idea of the exact list of targets, or how to narrow these down, or shall I just
throw in anything I can find using rgrep/sed? I simply want to avoid doing a to and fro with our


network people, and make build/py work the first time.

Thanks,

Andrei

On Tue, Dec 13, 2011 at 1:40 PM, Martin Sharratt

<Martin.Sharratt at uwe.ac.uk<http://lists.whatwg.org/listinfo.cgi/implementors-whatwg.org>> wrote:

> Are you behind a firewall, and do you need to set a proxy to get to the

> Web?  My server is and I had to create a firewall exception for the server

> outbound on TCP ports 80 and 443.    Re-running the build.py script then

> completed successfully and I got a working validator.

>

>

>

> Ideally java should honour any system proxy settings but it doesn’t seem

> to.  I’ve done the usual cursory research and have not really found any

> simple way of setting a proxy for java – apparently it’s most often done

> within the code.  As usual this was a rush job so I’ve not had time to

> research more thoroughly once I got a working system.



See http://docs.oracle.com/javase/6/docs/technotes/guides/net/proxies.html



I too believe this is a network issue (see

https://bitbucket.org/validator/syntax/src/f370b59cb78a/relaxng/datatype/java/src/org/whattf/datatype/data/CharsetData.java#cl-54

where the connection is made to

http://www.iana.org/assignments/character-sets, and note that you can

make a local copy and override the URL by setting the

-Dorg.whattf.datatype.charset-registry system property; I don't know

how to set it through build.py though, as I never used it)



--

Thomas Broyer

/tɔ.ma.bʁwa.je/<http://xn--nna.ma.xn--bwa-xxb.je/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.whatwg.org/pipermail/implementors-whatwg.org/attachments/20120119/7bfe3c38/attachment-0003.htm>