Scrappy and JavaScript Integration

If you cannot find an API like this, you need to execute JavaScript using some kind of headless browser like Selenium or a Puppeteer. You can integrate splash with scrappy really easily. It's literally just like a middleware in scrappy. And then it will execute JavaScript for you.