Read Time:9 Minute, 12 Second

Shodan : “The U1timate Beginner’s Guide”


Shodan is a search engine for Internet-connected devices. Web search engines, such as Google and
Bing, are great for finding websites. But what if you’re interested in finding computers running a
certain piece of software (such as Apache)? Or if you want to know which version of Microsoft IIS
is the most popular? Or you want to see how many anonymous FTP servers there are? Maybe a new
vulnerability came out and you want to see how many hosts it could infect? Traditional web search
engines don’t let you answer those questions.

The main interface for accessing the data gathered by Shodan is via its search engine located at
https://www.shodan.io.

By default, the search query will look at the data collected within the past 30 days. This is a change
from the old website at shodanhq.com, which searched the entire Shodan database by default. This
means that the results you get from the website are recent and provide an accurate view of the
Internet at the moment.
In addition to searching, the website also provides the following functionality:

Download Data

After completing a search there will be a button at the top called Download Data. Clicking on that
button will provide you with the option of downloading the search results in JSON, CSV or XML
formats.

The JSON format generates a file where each line contains the full banner and all accompanying
meta-data that Shodan gathers. This is the preferred format as it saves all available information.
And the format is compatible with the Shodan command-line client, meaning you can download
data from the Shodan website then process it further using the terminal.
The CSV format returns a file containing the IP, port, banner, organization and hostnames for the
banner. It doesn’t contain all the information that Shodan gathers due to limitations in the CSV file
format. Use this if you only care about the basic information of the results and want to quickly load
it into external tools such as Excel.
The XML format is the old, deprecated way of saving search results. It is harder to work with than
JSON and consumes more space, thereby making it suboptimal for most situations.
Downloading data consumes export credits, which are one-time use and purchased on the website.
They aren’t associated in any way with the Shodan API and they don’t automatically renew every
month. 1 export credit can be used to download up to 10,000 results.
Data files generated by the website can be retrieved in the Downloads section of the website, which
you can visit by clicking on the button in the upper right corner.

Generate Report

The website lets you generate a report based off of a search query. The report contains graphs/ charts
providing you a big picture view of how the results are distributed across the Internet. This feature
is free and available to anyone.

When you generate a report you are asking Shodan to take a snapshot of the search results
and provide an aggregate overview. Once the report has been generated, it doesn’t change or
automatically update as new data is being collected by Shodan. This also means that you can
generate a report once a month and keep track of changes over time by comparing it to reports
of previous months. By clicking on the button in the top right corner you can get a listing
of previously generated reports.

Shared Search Queries

Finding specific devices requires knowledge about the software they run and how they respond to
banner grabs over the Internet. Fortunately, it is possible to leverage the shared knowledge of the
community using the search directory on Shodan. People are able to readily describe, tag and share
their search queries for others to use. If you’re interested in getting started with Shodan, the shared
searches should be your first stop.

Shodan Maps

Shodan Maps1 provides a way to explore search results visually instead of the text-based main
website. It displays up to 1,000 results at a time and as you zoom in/ out Maps adjusts the search
query to only show results for the area you’re looking at.
All search filters that work for the main Shodan website also work on Maps.

Shodan Exploits

Shodan Exploits2 collects vulnerabilities and exploits from CVE, Exploit DB and Metasploit to make
it searchable via web interface.

The search filters available for Exploits are different than the rest of Shodan, though an attempt was
made to keep them similar when possible.

Important: By default, Exploits will search the entire content of the available exploit
information including meta-data. This is unlike Shodan, which only searches the banner text if no other filters are specified.

The following search filters are available:

Name Description
author Author of the vulnerability/ exploit
description Description
platform Platform that it targets (ex: php, windows, linux)
type Exploit type (ex: remote, dos)

Shodan Images

For a quick way to browse all the screenshots that Shodan collects check out Shodan Images
.It is a user-friendly interface around the has_screenshot filter.

The search box at the top uses the same syntax as the main Shodan search engine. It is most useful
to use the search box to filter by organization or netblock. However, it can also be used to filter the
types of images that are shown.
Image data is gathered from 5 different sources:
• VNC
• Remote Desktop (RDP)
• RTSP
• Webcams
• X Windows
Each image source comes from a different port/ service and therefor has a different banner. This
means that if you only want to see images from webcams you could search for.


All About the Data




The basic unit of data that Shodan gathers is the banner. The banner is textual information that
describes a service on a device. For web servers this would be the headers that are returned or for
Telnet it would be the login screen.
The content of the banner varies greatly depending on the type of service. For example, here is a
typical HTTP banner:

HTTP/1.1 200 OK
Server: nginx/1.1.19
Date: Sat, 03 Oct 2015 06:09:24 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 6466
Connection: keep-alive

The above banner shows that the device is running the nginx web server software with a version
of 1.1.19. To show how different the banners can look like, here is a banner for the Siemens S7
industrial control system protocol:

Copyright: Original Siemens Equipment
PLC name: S7_Turbine
Module type: CPU 313C
Unknown (129): Boot Loader A
Module: 6ES7 313-5BG04-0AB0 v.0.3
Basic Firmware: v.3.3.8
Module name: CPU 313C
Serial number of module: S Q-D9U083642013
Plant identification:
Basic Hardware: 6ES7 313-5BG04-0AB0 v.0.3

The Siemens S7 protocol returns a completely different banner, this time providing information
about the firmware, its serial number and a lot of detailed data to describe the device.
You have to decide what type of service you’re interested in when searching in Shodan because the
banners vary greatly.

Note: Shodan lets you search for banners – not hosts. This means that if a single IP
exposes many services they would be represented as separate results.


Device Metadata


In addition to the banner, Shodan also grabs meta-data about the device such as its geographic
location, hostname, operating system and more (see Appendix A). Most of the meta-data is
searchable via the main Shodan website, however a few fields are only available to users of the
developer API.


IPv6


As of October 2015, Shodan gathers millions of banners per month for devices accessible on IPv6.
Those numbers still pale in comparison to the hundreds of millions of banners gathered for IPv4 but
it is expected to grow over the coming years.


Data Collection


Frequency


The Shodan crawlers work 24/7 and update the database in real-time. At any moment you query
the Shodan website you’re getting the latest picture of the Internet.


Distributed


Crawlers are present in countries around the world, including:
• USA (East and West Coast)
• China
• Iceland
• France
• Taiwan
• Vietnam
• Romania
• Czech Republic
Data is collected from around the world to prevent geographic bias. For example, many system
administrators in the USA block entire Chinese IP ranges. Distributing Shodan crawlers around the
world ensures that any sort of country-wide blocking won’t affect data gathering.


Randomized 


The basic algorithm for the crawlers is:
1. Generate a random IPv4 address
2. Generate a random port to test from the list of ports that Shodan understands
3. Check the random IPv4 address on the random port and grab a banner
4. Goto 1
This means that the crawlers don’t scan incremental network ranges. The crawling is performed
completely random to ensure a uniform coverage of the Internet and prevent bias in the data at any
given time.


SSL In Depth


SSL is becoming an evermore important aspect of serving and consuming content on the Internet, so
it’s only fit that Shodan extends the information that it gathers for every SSL-capable service. The
banners for SSL services, such as HTTPS, include not just the SSL certificate but also much more.
All the collected SSL information discussed below is stored in the ssl property on the banner (see
Appendix A and Appendix E).


Vulnerability Testing


Heartbleed


If the service is vulnerable to Heartbleed then the banner contains 2 additional properties. opts.heartbleed
contains the raw response from running the Heartbleed test against the service. Note that for the test
the crawlers only grab a small overflow to confirm the service is affected by Heartbleed but it doesn’t
grab enough data to leak private keys. The crawlers also added CVE-2014-0160 to the opts.vulns list
if the device is vulnerable. However, if the device is not vulnerable then it adds “!CVE-2014-0160”.
If an entry in opts.vulns is prefixed with a ! or – then the service is not vulnerable to the given
CVE.

{
"opts": {
"heartbleed": "... 174.142.92.126:8443 - VULNERABLE\n",
"vulns": ["CVE-2014-0160"]
}
} 

Shodan also supports searching by the vulnerability information. For example, to search Shodan for
devices in the USA that are affected by Heartbleed use:

country:US vuln:CVE-2014-0160 

FREAK


If the service supports EXPORT ciphers then the crawlers add the “CVE-2015-0204” item to the
opts.vulns property:

"opts": {
"vulns": ["CVE-2015-0204"]
}

Logjam


The crawlers try to connect to the SSL service using ephemeral Diffie-Hellman ciphers and if the
connection succeeds the following information is stored:

"dhparams": {
"prime": "bbbc2dcad84674907c43fcf580e9...",
"public_key": "49858e1f32aefe4af39b28f51c...",
"bits": 1024,
"generator": 2,
"fingerprint": "nginx/Hardcoded 1024-bit prime"
}

STAY HOME! STAY SAFE! JAI HIND!!


DO FOLLOW US ON SOCIAL MEDIA:

Instagram1 : https://www.instagram.com/indiancybertroops/

Instagram2 : https://www.instagram.com/ict_message/

Facebook : http://faceboo.com/indiancybertroopsnews

Youtube : https://youtube.com/u/indiancybertroops


 

About Post Author

Indian Cyber Troops

Indian Cyber Work For Nation's Wellness And Nation's Security We Share new and unique things with you Jai Hind Jai Shri Ram