Server side browser that can execute JavaScript

Question

Is there any programming libraries available that will parse an HTML document, execute JavaScript and then allow me to navigate the DOM? This needs to be performed server side, not client side. Any language will do, but Java, PHP, or Ruby are preferred.

Answer 1

Have you tried Bringing the Browser to the Server?

• Pepijn: +1 Forgot about that one... On my Mac I'd just use Python's Applescript capability to run the JS straight on Safari though.
• machine elf: Links for updated community version: envjs.com and github.com/thatcher/env-js

Answer 2

in java: http://lobobrowser.org/cobra/java-html-parser.jsp
this is a a Javascript-aware, CSS-aware HTML parser
the most important feature in relation to your question: It is Javascript-aware. DOM modifications that occur during parsing will be reflected in the resulting DOM.

• Florida: Link off, mirror please.

Answer 3

Java has support for javascript with Rhino, also look at this page for server side javascript solutions: http://en.wikipedia.org/wiki/Server-side_JavaScript

Answer 4

For Java, be sure to check out HtmlUnit and HttpUnit.

Answer 5

PhantomJS does this and can be used with any server side language. See some integration modules below for NodeJS and PHP

NodeJS

https://npmjs.org/package/node-phantom

https://github.com/sgentle/phantomjs-node

PHP

https://github.com/diggin/php-PhantomjsRunner

Answer 6

PHP has DOMDocument for navigating the DOM. I haven't heard of anything for executing JavaScript.

Answer 7

Start from this post and follow a links. Or just search for Rhino.

• NilColor: Oh... same link as Luca Matteis gave... Sorry!

Answer 8

There are now several projects that do a really good job of this:

• PhantomJS is a headless version of WebKit, and there are some helpful wrappers such as CasperJS.

• Zombie.js which is a wrapper over jsdom written in Javascript (Node.js).

You need to write JavaScript code to interact with both of these projects. I like Zombie.js better so far, since it is easier to set up, and you can use any Node.js/npm modules in your code.

Answer 9

node.js ?

Node can run any javascript file in its console. I would try node first & see if it can do what you want as it likely has the largest user base & documentation.