What is the Buffer API ?

API Endpoint

https://api.buffer.dk/

Buffer is a private little project that grew large over time. I think i have like 5TB data buffered :) It's not widely used nor known, so I'm not going to advertise to use it. It's simply there, if you have any use for it ... Then enjoy!

It all started back when a web page I accessed a lot to lookup ip range and whois detailed info was taken down. Luckily I found an archived version and crawled it and stored it locally. I think back then I did most of it with httrack on a linux box. Did this for many years when ever i found some data on the internet i was afraid would get lost. So i made a online web site buffer.dk where i could create like a buffered version of the data i found on the internet

Later some of the data would make sense to parse, index and build a small database around it. But then came the problem now I had to code something to access it. So over time I ended up with all these small scripts looking at data and or grabbing data from online sources. Some of it was utterly useless, some you can already do a million other places on the net, some made sense and others is simply unique :)

So as time went on a words like API became a thing, I saw the cool in creating a little website where you could access all these useful and useless scripts in a more unified solution. So this is basically what this is, I'm currently slowly converting these scripts and data into a api data endpoints

Donations

So far I just pay for it out of my own pocket, the ec2 instance is also not very big but that might change over time if we get to much traffic for it to handle. Any donations made to this BTC address will go into this project for future improvements or maintenance cost. Again, I don't expect any donations, but they are however recieved with open arms!

BTC Address - 31jzgaJZuzGhDZQYrDTHQzmRFYnvHhzr57

How to add data or api data endpoints

You can contact me on Github, via telepathy or Show Email :)

Security, Limits, Protection and Server Load

Again, this is a little private project on a small server. I'm not that important if you know what I mean :) If you really wanted to obstruct this service you could, why you wanted to though - I don't know.

I don't impose limits on the data and api calls. I mean the whole point of this site is to make it available. So if you decide to crawl everything, you can do so. I only ask you to be polite when "talking" to this service etc doing loops/crawl. Again, as the server is small it will see high load when there are many requests. I might eventually upgrade it along the way, might even put it behind a load balancer if needed. But I'm not paying for that - lol ! I put it behind CloudFlare just to catch the most obviously filth

There isn't much to this site, I don't have any security in place like oAuth or "secure" web tokens that you need to parse along to view specific data. It's all public.

I do have a IP whitelist that gives access to any API endpoint i have marked as "protected", you wont be able to see these unless you're whitelisted. This is only for internal use

Buffer API Statistics

We all love statistics, atleast i know i do!

I have tried to come up with what i think is fun to look at, i don't really know what to use this for but it's always cool to have :) I like that i calculate average execution time on each API also i like the bytes sent. The last call is perhaps not that important to show but i made it so that i can someday cleanup in unsused api's etc...

OVERALL NUMBERS

Total API Count Total API Requests Total Bytes Sent
9 317 5.30MB

INDIVIDUAL API STATISTICS

API Name Last Called Count GET POST PUT DELETE Sent Avg Time
cve-lookup 2020-08-28 2 0 2 0 0 2.17kB 1ms
exploit-lookup 2020-09-23 44 19 25 0 0 4.42MB 579ms
google-search 2020-09-04 31 0 31 0 0 616.39kB 684ms
myip 2020-09-15 106 105 1 0 0 7.26kB 9ms
phone-lookup-scam 2020-09-23 76 0 76 0 0 245.17kB 338ms
tor-is-exitnode 2020-09-15 9 8 1 0 0 831B 2sec
tor-onion-lookup 2020-08-28 19 0 19 0 0 6.13kB 6.2sec
trustpilot 2020-08-27 16 0 16 0 0 11.47kB 6ms

Buffer API overview


Ever wanted to know how to call us from a cli? Try cURL!

# Basic GET call
curl http://api.buffer.dk/api-name

# Basic POST call urlencoded
curl -X POST \
 -d "param1=value1&param2=value2" \
 http://api.buffer.dk/api-name

# Basic POST call json
curl -X POST \
 -d '{"key1":"value1", "key2":"value2"}' \
 -H "Content-Type: application/json" \
 http://api.buffer.dk/api-name
                

This is a overview list that shows all the API's that are currently active and public on the site. I know it might not be the best overview, but at least it's there. More text for search optimization etc.

INTERNAL Everything is happening internally. (Should always be working)
EXTERNAL Everything is happening externally. (Might break depending on source)
INT-CACHED Like INTERNAL, but we cache the results in an effort to make it faster!
EXT-CACHED Like EXTERNAL, but we cache the results in an effort to make it faster!
HARDWARE Talks to hardware, ie. microcontroller via serial (Might break depending on connections)

BUFFER API LIST

API Name API Type API Description
cve-lookup INTERNAL Lookup Common Vulnerabilities and Exposures (CVE)
exploit-lookup INTERNAL Search Exploit Database
google-search EXTERNAL Search Google for up to 10 results
myip INTERNAL Show your client IP address and reverse DNS
phone-lookup-scam EXTERNAL Get a quick overview if the phone number might be a scammer!
tor-is-exitnode EXTERNAL Lookup IP address to see if it's from Tor network
tor-onion-lookup EXTERNAL Lookup Tor Onion address, get title and http return code
trustpilot INTERNAL Search for Trustpilot (DK) reviews

/ cve-lookup



# Request method - POST
curl -X POST \
 -F 'fieldname=fieldvalue' \
 https://api.buffer.dk/cve-lookup
 

This is a lookup tool for Common Vulnerabilities and Exposures (CVE) it is lookup up the CVE cases in a local database that is updated every week.

API Type INTERNAL
Avg Latency 1ms
Cached No
Accepted methods POST
Endpoint URL https://api.buffer.dk/cve-lookup


Result example :
{
  "id": "148275",
  "cve_name": "CVE-2019-16220",
  "cve_status": "Candidate",
  "cve_description": "In WordPress before 5.2.3, validation and sanitization of a URL in ...",
  "cve_references": [
    "BUGTRAQ:20200108 [SECURITY] [DSA 4599-1] wordpress security update",
    "URL:https://seclists.org/bugtraq/2020/Jan/8",
    "DEBIAN:DSA-4599",
    "URL:https://www.debian.org/security/2020/dsa-4599",
    "DEBIAN:DSA-4677",
    "URL:https://www.debian.org/security/2020/dsa-4677",
    "MISC:https://core.trac.wordpress.org/changeset/45971",
    "MISC:https://github.com/WordPress/WordPress/commit/c86ee39ff4c1a79b93c967...",
    "MISC:https://wordpress.org/news/2019/09/wordpress-5-2-3-security-and-mainten.../",
    "MISC:https://wpvulndb.com/vulnerabilities/9863",
    "MLIST:[debian-lts-announce] 20191017 [SECURITY] [DLA 1960-1] wordpress sec...",
    "URL:https://lists.debian.org/debian-lts-announce/2019/10/msg00023.html"
  ],
  "cve_phase": "Assigned (20190911)",
  "cve_votes": [
    "None (candidate not yet proposed)"
  ],
  "cve_comments": ""
}

POST :: QUERY PARAMETERS

Field Type Required Default Description
cve String Yes CVE case number (Format cve-xxxx-xxxx to cve-xxxx-xxxxxxxxxx)

/ exploit-lookup



# Request method - GET
curl https://api.buffer.dk/exploit-lookup

# Request method - POST
curl -X POST \
 -F 'fieldname=fieldvalue' \
 https://api.buffer.dk/exploit-lookup
 

Search the very infamous exploit database, you can also view latest exploits. If no arguments are givin in GET mode, it will just show todays exploits. The ExploitDB is updateded every 30min 24/7. So it should always contain the latests exploits. This is only ment for research. If you want to download the files directly from us when searching, include the include-script option. Or else you can visit the url link and see the content there.

API Type INTERNAL
Avg Latency 579ms
Cached No
Accepted methods GET, POST
Endpoint URL https://api.buffer.dk/exploit-lookup


Result example :
[
   {
      "script" : false,
      "platform" : "php",
      "author" : "Riemann",
      "url" : "https://www.exploit-db.com/exploits/46982",
      "id" : "46982",
      "type" : "webapps",
      "date" : "2019-06-11",
      "title" : "phpMyAdmin 4.8 - Cross-Site Request Forgery"
   },
   {
      "script" : false,
      "platform" : "php",
      "title" : "phpMyAdmin 4.8.0 < 4.8.0-1 - Cross-Site Request Forgery",
      "url" : "https://www.exploit-db.com/exploits/44496",
      "author" : "revengsh",
      "type" : "webapps",
      "date" : "2018-04-23",
      "id" : "44496"
   },
   ...
]

GET :: QUERY PARAMETERS

Field Type Required Default Description
lastmonth Boolean No false Get all exploits from last month
thismonth Boolean No false Get all exploits from this month
today Boolean No true Get all exploits from today
yesterday Boolean No false Get all exploits from yesterday

POST :: QUERY PARAMETERS

Field Type Required Default Description
search String Yes Search for exploit, accept [a-zA-Z0-9\-\.\s_]
include-script Boolean No false Include exploit script as base64 encoded
strict Boolean No false Do a strict search

/ myip



# Request method - GET
curl https://api.buffer.dk/myip
 

This simple api endpoint will show you your IP address and the reverse DNS of that ip address if it exists. Yes yes i know! So many sites out there can do this, but i like to keep it inhouse, hate asking others to show my ip :)

API Type INTERNAL
Avg Latency 9ms
Cached No
Accepted methods GET
Endpoint URL https://api.buffer.dk/myip


Result example :
{
   "ip":"111.111.111.111",
   "dns":"111-111-111-111-reverse.dns.isp.com"
}

/ phone-lookup-scam



# Request method - POST
curl -X POST \
 -F 'fieldname=fieldvalue' \
 https://api.buffer.dk/phone-lookup-scam
 

This utilized the 118.dk website to lookup a phone number and gather intel. We then show the parsed output in a nice little package. This might break depending on 118.dk if they change their html :)

API Type EXTERNAL
Avg Latency 338ms
Cached No
Accepted methods POST
Endpoint URL https://api.buffer.dk/phone-lookup-scam


Result example :
{
  "number": 20762861,
  "times_searched": "2427",
  "country_code": "DK",
  "country_name": "Denmark",
  "risk_pct": "80%",
  "risk_scale": "5",
  "risk_level": "Very high",
  "last_3months_activity_calls": 1315,
  "last_3months_activity_detailed": {
    "2020": {
      "August": {
        "31": "231",
        ...
        "1": "8"
      },
      "July ": {
        "31": "20",
        ...
        "1": "13"
      },
     ...
    }
  },
  "reported_number": 90,
  "reported_types": [
    {
      "type": "Svindel",
      "reports": "36"
    },
    {
      "type": "Telemarketing",
      "reports": "28"
    },
    ...
  ]
}

POST :: QUERY PARAMETERS

Field Type Required Default Description
number String Yes Phone number (Can include country-code like +XX)

/ tor-is-exitnode



# Request method - GET
curl https://api.buffer.dk/tor-is-exitnode

# Request method - POST
curl -X POST \
 -F 'fieldname=fieldvalue' \
 https://api.buffer.dk/tor-is-exitnode
 

This will lookup a IP address on the torpoject website and see if it belongs to any exit nodes or relays. Please note the the exit node has to be at least 2 days old to show up. Also looks like they only accept IPv4 addresses for now. Use GET for client_ip and POST to specify ip address

API Type EXTERNAL
Avg Latency 2sec
Cached No
Accepted methods GET, POST
Endpoint URL https://api.buffer.dk/tor-is-exitnode


Result example :
{
   "ip":"185.220.100.243",
   "tor":true,
   "exitrelay":true,
   "nickname":"F3Netze",
   "fingerprint":"9D5A7A81BC6194555CE751F75FB196E4A3982CC5"
}

POST :: QUERY PARAMETERS

Field Type Required Default Description
ip String Yes IP address to lookup

/ tor-onion-lookup



# Request method - POST
curl -X POST \
 -F 'fieldname=fieldvalue' \
 https://api.buffer.dk/tor-onion-lookup
 

You can check if a Tor Onion hostname / address is responding on the Tor Network. I will return the HTTP return-code and also the website title if i can find it. Also use follow field set to true to follow redirects. If you follow redirects there is a better chance to grab website title. I show individual status for both http and https. So toggle follow to get best data.

API Type EXTERNAL
Avg Latency 6.2sec
Cached No
Accepted methods POST
Endpoint URL https://api.buffer.dk/tor-onion-lookup


Result example :
{
   "onion_address":"facebookcorewwwi.onion",
   "follow":true,
   "valid":true,
   "url_http":{
      "return-code":200,
      "status":"up",
      "title":"Facebook - Log In or Sign Up",
   },
   "url_https":{
      "return-code":200,
      "status":"up",
      "title":"Facebook - Log In or Sign Up",
   }
}

POST :: QUERY PARAMETERS

Field Type Required Default Description
onionaddr String Yes Tor Onion hostname / address
follow Boolean No false Toggle if we follow redirects

/ trustpilot



# Request method - POST
curl -X POST \
 -F 'fieldname=fieldvalue' \
 https://api.buffer.dk/trustpilot
 

Search for reviews on the danish Trustpilot website. This data is crawled and parsed about every 24 hour unless they change their website it might be older as the parser will fail. You can see the last updated date for your results. NOTE, The data is like 7 years old the scraper have not been running. Encoded in UTF-8 Unicode and ASCII.

API Type INTERNAL
Avg Latency 6ms
Cached No
Accepted methods POST
Endpoint URL https://api.buffer.dk/trustpilot


Result example :
[
   {
      "id":"330",
      "name":"legetojsgiganten.dk",
      "rating_cur":"6.6",
      "rating_min":"0",
      "rating_max":"10",
      "reviews":"36",
      "email":"[email protected]",
      "street":"H\u00f8gevej 8B",
      "zip":"3400",
      "city":"Hiller\u00f8d",
      "country":"Denmark",
      "website":"http://www.trustpilot.dk/review/legetojsgiganten.dk",
      "date_added":"2013-03-13 18:06:01",
      "date_updated":"2013-10-16 15:06:26",
      "category":"kids"
   },
   {
      "id":"487",
      "name":"www.elgiganten.dk",
      "rating_cur":"8.1",
      "rating_min":"0",
      "rating_max":"10",
      "reviews":"27445",
      "email":"[email protected]",
      "street":"Arne Jacobsens All\u00e9 16.2",
      "zip":"2300",
      "city":"K\u00f8benhavn S",
      "country":"Denmark",
      "website":"http://www.trustpilot.dk/review/www.elgiganten.dk",
      "date_added":"2013-03-13 18:08:04",
      "date_updated":"2013-10-16 15:38:46",
      "category":"mobile_internet"
   },
   ...
]

POST :: QUERY PARAMETERS

Field Type Required Default Description
search String Yes Search string to look for in field type
type Enum No name Can be name or email