Today I Learned - Rocky Kev

TIL about the CORS-ANYWHERE project

POSTED ON:

TAGS: webscraping heroku

I was building a web scrapper for a game using Vue, Axios, and Cheerio.

I kept running into the CORS policy error.

... has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Since my tool was a pure Frontend application (Not Node), I had to find a workaround.

The solution is cors-anywhere.

Request Examples:
http://localhost:8080/http://google.com/ - Google.com with CORS headers

It's best to create your own microservice for it.

I did it by:

  1. Signing up for a free Heroku account

  2. Forking the cors-anywhere repo.

  3. Creating a new app in Heroku.

  4. Finally, connecting the forked repo to Heroku.

With that new URL, I can now web scrape away!

Related TILs

Tagged: webscraping

TIL Web scraping with node

Fetch the data, traverse the dom, and return the data.

TIL about the CORS-ANYWHERE project

I was building a web scrapper for a game using Vue, Axios, and Cheerio.

TIL about the CORS-ANYWHERE project

I was building a web scrapper for a game using Vue, Axios, and Cheerio.