When looking for pages with interesting content we have Google or other search engines at our disposal. But what should we use if we are not interested in pages, but in network devices accessible from the Internet, or if we want to check whether we accidentally exposed unsecured remote desktop or database to the Internet? This article presents Shodan search engine, which is sometimes called Google for network devices or search engine for hackers. We will learn how to use it to verify network devices exposed to the Internet and their security level. The tool will be useful for pentesters as well as administrators and security team members.
The Rise of Shodan
We first heard about Shodan over 10 years ago, when a bioinformatics graduate student at the University of California San Diego decided to create a searchable online metadata repository on November 23, 2009. Originally, Shodan was intended as a platform to monitor trends and statistics of software usage (e.g., where Apache or Nginx is most commonly used). However, it was noticed among others by security specialists and quickly became a commonly used platform by them.
Currently Shodan has hundreds of servers storing petabytes of data, over 3 million registered users and over 27 million monitored IP addresses. Shodan's crawlers (indexing programs that collect information about the structure, pages, and content found on sites) crawl and collect information about services exposed to the Internet on a round-the-clock basis. Shodan's scanning is done from machines distributed around the world, avoiding geographically dependent results (such as the effects of some U.S. server administrators blocking entire IP ranges of China).
How does Shodan work?
Technically, Shodan can be classified as a port scanner that searches public IP ranges and indexes data such as banners obtained (e.g., the Server response header received with the response to an HTTP request), SSL certificate information, geographic location, operating system, or potential vulnerabilities. The search of public IP addresses is done randomly. IP addresses and ports to be scanned are randomized to get the most coverage and no predictable scanning order.
The main information you can see in Shodan when displaying details for an IP address of interest is geographic location, technology information, a list of open ports and available services, and banners for those services. Each service exposed to the Internet on a network device has a port assigned to it.
As a reminder,** a banner is the information returned when a connection is established**. For example, to display this article, the browser makes an HTTP GET request and receives an HTML page with the article content and additional information in the HTTP response headers. By default, HTTP header information is not presented to the browser, but is used by the browser, for example, to display a page correctly or to configure security while displaying that page.
To see what information is hidden in http headers, you can use any browser. They allow you to view the full content of the response headers using developer tools. For example, in the Chrome browser, to do this, go to the Chrome menu (the three dots in the right corner) and then select More Tools > Developer Tools.
Below we see the response headers for an example HTTP request to view profile details. You can see the response headers after selecting the request, in the Response Headers tab. The headers often contain interesting information such as the version of the software used (Server header) and in extreme cases even information about passwords or the use of default passwords for a given service. In the screenshot below we see only the standard headers, without headers that are a security risk.
It is from the information in these headers that Shodan determines what service and what software is exposed on a given port.
When displaying information in Shodan about a domain of interest, for example Google, you can see, among other things, the http service exposed to the Internet on port 443/tcp. When you click on port 443 in the details you will see just the HTTP request response headers, which is similar to the ones you can see in the browser's developer tools. To search results for google.co.uk, we can enter an IP address filter in the search box and go into the details of the results. Shodan allows you to search for such results using a number of filters and logical operators. Results can be filtered by ip address (eg ip:18.104.22.168), location (eg country:PL), port (port:443). Searching using filters is available only for logged in users. An account with access to basic functionality can be created for free.
Filters and logical operators allow to prepare interesting queries returning e.g. unsecured remote desktops, databases exposed to the Internet, headers containing passwords or headers informing about the use of vulnerable software.
Shodan is useful for both administrators and pentesters. It can also be used for private purposes, such as checking that the image from our surveillance camera is not available to anyone on the Internet.
Shodan dla pentestera
Shodan is a tool often used during reconnaissance, the first phase of penetration testing. When performing penetration tests, queries can be narrowed down to the IP addresses under test (IP addresses of the organization for which the tests are performed). This provides a passive reconnaissance tool to determine open ports, available services, technologies used, and potential vulnerabilities. All this information can be obtained without direct contact with the tested systems, i.e. without leaving any traces in logs. In the first phase of penetration testing, all information is relevant.
Interesting search results can be found for queries searching for unsecured remote desktops. For example, for the query port:5901 authentication disabled the results will return VNC remote desktops accessible without logging in.
Below we see an example of a control panel made available by an unsecured VNC service.
We can also search for remote desktops of the RDP service. Using the port:3389 filter you can get a list of remote desktops shared with the Internet, which gives an opportunity to e.g. display available logins and makes it easier for a potential attacker to try to guess or bruteforce the password.
Using the mentioned filter and going to the Images tab, you can view screenshots of shared remote desktops.
When performing penetration tests or just looking for interesting findings, it is also worthwhile to search the standard ports of services that are part of the ELK stack (a set of tools that allow collecting, searching, grouping and filtering huge data sets), e.g. by searching for instances of Elasticsearch (a central server for indexing and searching data) or Kibana (a tool for presenting and visualizing data from Elasticsearch).
More than one data leak was caused by exposing the Elasticsearch database to the Internet. An example is the leak of 19 million records from the logs of the AVON web and mobile service Using the port:9200 json filter (Elasticsearch's default port and JSON format) over 35,000 results can be found, and for the port:5601 kibana filter (Kibana's default port) over 13,000 results can be found. It is very likely that these are not issued intentionally in every case.
Of course, there can be many such search filters. Other interesting queries can be found in Shodan's list of most popular queries https://www.shodan.io/explore/popular. Many of them are attempts to search for webcams by standard headings. For example, selecting a category query and going to the Images tab yields images from over 3,000 cameras from stores, parking lots, and even apartments.
Note that you should not enter such unsecured systems without the server owner's permission. You cannot connect through an unsecured remote desktop, or attempt to retrieve data from a database.
Below is a brief summary and steps for the pentester to follow.
- Creating a free account and logging in.
- Provide an IP range (net filter) or individual IP addresses (ip filter) for verification, preferably the full address range of your organization. You can also provide host and domain names (hostname filter).
- If the organization has very many IP addresses and verification of all results would be too time consuming, adding phrases such as "unauthorized", "default password", "camera" or other previously described to the query.
- Analyze the results and gather information about the tested organization, e.g. unsecured remote desktops, outdated and vulnerable service versions.
Shodan for the administrator or security team member
Administrators and security team members can also use Shodan to verify services exposed to the Internet. The steps to be taken will be the same as in the case of pentester. The result of the verification will be a list of services/ports unnecessarily accessible from the Internet, which should be disabled or e.g. blocked on the firewall.
For administrators, Shodan may also be useful as an alternative to regular scanning of resources exposed to the Internet. The cyclic scanning can be replaced by the Shodan Monitor service, setting it to send notifications of detected changes to a Slack channel or email. If there is a new available port or vulnerability for our version of the software a notification is sent. Shodan Monitor service is available when you upgrade your account to Shodan Membership. The fee is a one-time fee of $49, but there is usually a promotion on Black Friday during which the account can be purchased for $5.
Below is a summary of the information in the form of steps for an administrator or security team member to follow.
- Creating an account, purchasing Shodan Membership and logging in.
- Specifying the IP range to monitor, preferably the full IP range of the organization.
- Selecting a communication channel for notifications e.g. emails.
Shodan can also be used for entertainment. You can even turn on music while browsing interesting results at https://2000.shodan.io/. Among the results presented are databases, cameras or unsecured remote desktops.
The original article in Polish can be found here.