Simple GET request

Problem

You want to make a simple GET request to a web site and extract useful information.

Solution

Use the HTTP Request node to make HTTP requests and an HTML node to extract elements from the retrieved html document.

Example

[{"id":"d88dd470.0ac7b8","type":"inject","z":"18c99b30.cf9d35","name":"make request","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"x":130,"y":180,"wires":[["874a3d4e.9b666"]]},{"id":"874a3d4e.9b666","type":"http request","z":"18c99b30.cf9d35","name":"","method":"GET","ret":"txt","url":"https://www.npmjs.com/package/node-red","tls":"","x":294.5,"y":180,"wires":[["90243cc1.87edc"]]},{"id":"7403c68f.21d7c8","type":"debug","z":"18c99b30.cf9d35","name":"","active":true,"console":"false","complete":"false","x":650,"y":180,"wires":[]},{"id":"90243cc1.87edc","type":"html","z":"18c99b30.cf9d35","name":"","property":"","tag":".pretty-number","ret":"text","as":"single","x":471.5,"y":180,"wires":[["7403c68f.21d7c8"]]}]

To find content in a web page, the Chrome browser’s ‘Inspect Element’ can be a useful tool. Using the browser, right click on a page element to see the tags, ids and classes applied to an element as shown.

In this example we extract the download statistics for Node-RED from the npm page at https://www.npmjs.com/package/node-red shown below. Using the inspector we can see the download statistics are conveniently located in a <strong> tag with the class pretty-number. The CSS selector .pretty-number works. This flow displays an array of download statistics in the debug window.