Welcome to OGP-bot!

This project was used to help shachomeshi project showing sharing media like youtubes or articles from online news, etc...

Preview Image

Why do we need this project?

OGP or Meta tags could be found in responsed HTMLs of some multimedia websites. So all things we need to do is just send a GET method to that link, then find data in those responsed HTMLs.

But the problem is CORS(Cross-Origin Resource Sharing). Some kind of policy that almost any web use to protect themselves from being attacked. All the thing CORS does is block request from other websites working on browser.

Due to CORS, front-end cannot make request any parse meta data. Further more, Re-implement the logic on each front-end could exhaust resource of time or human.

How this OGP-bot works?

enter image description here

OGP-bot would help you communicate with media resource. Then OGP-bot filter and compress the needed data, then response to you as JSON.

API docs

/ogpbot - POST

Method: POST

Params:

param type Samples Default
url (option) string "http://tinyurl.com/98sw2"
urls (option) array ["http://tinyurl.com/98sw2", "htt...
json boolean 1 0

Response Sample:

{
    "meta": [
        {
            "title": "warm summer afternoons / study lofi playlist - YouTube",
            "image": "https://i.ytimg.com/vi/JNFR_gsFMvw/hqdefault.jpg",
            "host_name": "www.youtube.com",
            "url": "https://www.youtube.com/watch?v=JNFR_gsFMvw&t=1058s"
        },
        {
            "title": "Physics in 6 minutes - YouTube",
            "image": "https://i.ytimg.com/vi/t215Q1oBoXQ/maxresdefault.jpg",
            "host_name": "www.youtube.com",
            "url": "https://www.youtube.com/watch?v=t215Q1oBoXQ"
        },
        {
            "title": "How to use Memoize to cache JavaScript function results and speed up your code",
            "image": "https://cdn-media-1.freecodecamp.org/images/1*UPyVG04cujAGglMNgF1bTA.jpeg",
            "host_name": "freecodecamp.org",
            "url": "https://freecodecamp.org/news/understanding-memoize-in-javascript-51d07d19430e/"
        },
        {
            "error": true,
            "errorMessage": "Fail to call url",
            "url": "a"
        }
    ]
}

*if you set the json param to 1, response will be slightly different. It is a hashtable with key is your url, and value is the metadata we fetched. The response now would look like this

{
    "a": {
        "error": true,
        "errorMessage": "Fail to call url"
    },
    "https://www.youtube.com/watch?v=t215Q1oBoXQ": {
        "title": "Physics in 6 minutes - YouTube",
        "image": "https://i.ytimg.com/vi/t215Q1oBoXQ/maxresdefault.jpg",
        "host_name": "www.youtube.com"
    },
    "https://www.youtube.com/watch?v=JNFR_gsFMvw&t=1058s": {
        "title": "warm summer afternoons / study lofi playlist - YouTube",
        "image": "https://i.ytimg.com/vi/JNFR_gsFMvw/hqdefault.jpg",
        "host_name": "www.youtube.com"
    },
    "https://freecodecamp.org/news/understanding-memoize-in-javascript-51d07d19430e/": {
        "title": "How to use Memoize to cache JavaScript function results and speed up your code",
        "image": "https://cdn-media-1.freecodecamp.org/images/1*UPyVG04cujAGglMNgF1bTA.jpeg",
        "host_name": "freecodecamp.org"
    }
}

/ogpbot - GET

Method: GET QueryString: everthing is similar to POST method, you only need to change post body params into querystring. Result: similar to above.

Deployment

environment variables:

"PORT"                         :                       3000
"OGP_BOT_CACHE_DURATION"     :                       14400000
"OGP_BOT_LOG_INTERVAL"          :                       5000
"OGP_BOT_REDIS_URL"             :                       "redis://************"