Digital asset discovery across your organisation starts with scanning.
The Scan module provides visibility across your total external attack surface, comprising all your organisation’s managed and unmanaged infrastructure and applications. It automatically detects domains, IP addresses, certificates, software, and other key assets.
Our Scan function uses a combination of passive and active crawling techniques to uncover both known and unknown assets (“Things”) that are exposed externally to the Internet.
Based on your priorities or current specific requirement, you can choose what to scan, when, and how.
Note: Although we use multiple techniques to map out your external attack surface, no vulnerabilities are exploited, no forms are filled, and so on. The Discovery scan behaviour is similar to a visitor browsing the application.
Accessing the Scan functionality
From the Dashboard (which is where you’ll land after logging in), click the Scan Icon in the left-hand navigation panel.
Choosing what to scan
With the ThingsRecon Discovery platform you can go as broad and as deep as necessary to discover and assess digital assets or touchpoints. These might range from points of integration and data exchange (in the form of APIs), to the full spectrum of web applications.
The reach of your scan could be right across your entire organisation, or within the parameters of individual locations, business units, or specific digital assets
Adding scan items
Via the Items view (accessed via the left-hand tab at the top), click Add Item in the upper right corner to enter what, specifically, you want to scan.
Choose between:
- Manually adding IPs, IP Ranges, Domains, FQDNs, or URLs using the “Add Item” button (located in the top-right corner)
- Importing from File: Upload a list of scan targets in supported file formats (use the template to import).
To add multiple items at once, use the bulk import function.
Note: Once added, items will appear in the Items Tab with default attributes. (E.g. every Work Queue item is assigned a “trust” level from zero to 100. This refers to the level of confidence that the asset belongs to the organisation. Any item which is manually added to the Work Queue is automatically assigned a 100% trust level.
Choosing the type of scan
Two main types of Discovery scans are available:
Static – scanning only the specific item you entered, staying within the same domain.
- From a predefined list of domains entered by the user, discover all related Fully Qualified Domain Names (FQDNs) URLs, applications, IPs, etc and all relations between them.
Deep Discovery – a more dynamic scan exploring everything related to your company, giving you a broader view of your digital footprint.
- From a domain starting point, discover other domains belonging to the organisation
- From discovered domains, find all other things that are part of your external attack surface
- Deep Discovery can tell you the likelihood of a domain belonging to your organisation and what techniques have been used to find it.
Tip:
|
Discovery scans are typically done first using the default settings in the user interface. However, subsequent Discovery scans can be further configured with a number of parameters. (Setting Scan Parameters). A “reset to default” button can be found at the bottom of the page to undo any previous parameters.
Setting Scan Parameters
Select the Parameters view (top-right tab) for additional scan configuration settings. These parameters allow you to be more granular in your Discovery and adjust the scope of your scans if needed based on initial results.
This view is divided into three tabs (switch between them using the left-hand panel).
1. Deep Discovery Settings
The Deep Discovery settings control how deeply the scan should expand beyond original items and enable passive expansion into related domains and IPs.
These settings comprise:
- Enable Google Analytics
- Enable Redirect To
- Enable Reverse Whois on Company
- Enable Reverse Whois on Domain
- Enable Artificial Intelligence
- Enable IP Range Finding
- Enable Other Countries
- Enable Certificate DB Check.
(See Key to All Scan Settings)
2. Discovery Settings (Static and Deep Discovery)
The Discovery settings define how certificates, WHOIS records, or DNS data are used during asset identification. They also include or exclude external service lookups.
These settings comprise:
- Enable Subdomain Finder
- Whitelisted IP Locations
- HTTP Ports
- HTTPS Ports
- Enable Port Scanning
- Port Scanning Blacklisted IPs
- Enable Responsive IP Search.
(See Key to All Scan Settings)
3. Crawling Settings (Static and Deep Discovery)
The crawling settings determine how web crawling will be handled (depth, timeouts, limits). They are useful for applications and URLs with complex structures.
These settings comprise:
- Max Number of Pages
- Enable Number of Pages Per Application
- Find Application From Certificates
- Enable Sitemap Check
- Enable Fast http Port Checking
- Page Timeout
- Render Delay
- Time To Render
- Max Concurrent Crawling.
(See Key to All Scan Settings)
Key to All Scan Settings:
- Blacklisting Feature: A list of regexes (sequences of characters that defines a search pattern) that will exclude matching findings from results. This feature can also be used in reverse
- Enable Artificial Intelligence: Uses AI techniques such as image recognition to improve trust level accuracy on discovered domains. (Default value: True)
- Enable Certificate DB Check: Checks external certificate data sources to find more domains. (Default value: True)
- Enable Google Analytics: Uses reverse techniques on Google Analytics digital IDs. (Default value: True)
- Enable IP Range Discovery: A technique that checks for IP ranges based on trusted company names. (You will need to have at least one entry in the work queue here.) (Default value: True)
- Enable Other Countries: Allows the discovery of any top-level domain (TLD). If this option is disabled, only domains with the same TLD present in the work queue, entered by the user, will be considered. (Default value: True)
- Enable Redirect To: Identifies web sites/URLs that are redirecting to any discovered domains with a trust level of 100%. (Default value: True)
- Enable Reverse WHOIS on Company name: Finds domains by querying reverse WHOIS data sources with the company name as a criterion. (Default value: True)
- Enable Reverse WHOIS on Domain: Finds domains by querying reverse WHOIS data sources where a domain email address has been used to register the domain
- Enable Port Scanning: Performs port scanning on all IP addresses discovered. (Default value: False)
- Enable Responsive IP Search: Discovers applications responding on HTTP or HTTPS requests based on an IP address rather than an FQDN. (Default value: True)
- Enable Subdomain Enumeration: Finds all subdomains related to the domains discovered
- HTTP Ports: List of HTTP ports used to perform web application discovery. (Default value: [80,8080] ). Note: You can add ports for hidden interfaces
- HTTPS Ports: List of HTTPS ports used to perform web application discovery. (Default value: [443,8443] ). Note: you can add ports for hidden interfaces
- Blacklisting IP(s) for Port Scanning: The Scanner will exclude this list of IPs for port scanning. (Default value: [none] )
- Enable Fast HTTP Port checking: Port scanning to check whether a web application exists. Some firewalls can detect port scanning and blacklist ThingsRecon outbound IP(s). Disabling this option prevents being blocked, but will slow down the Discovery. (Default value: True)
- Max Number of Pages: This refers to the maximum number of pages the Scanner can crawl. This will apply to the whole Project if the “Number of Pages Per Application" option is disabled. (Default value: 60)
- Enable Number of Pages Per Application: Applies the “Max Number of Pages” per Application, instead of for the whole Project. (Default value: True)
- Enable Sitemap Check: The Scanner will try to find sitemap files to crawl more URLs. (Default value: True)
- Find Application from Certificates: Discovers new Applications from certificates found from already-known applications. The Scanner does this by checking the Subject Alternative Name (SAN) attributes. (Default value: True)
- Max Concurrent Crawling: This is the maximum number of web pages that can be crawled concurrently. Decreasing this value will help ensure the Scanner is not blacklisted or blocked. (Default value: 20)
- Page Timeout: Timeout option for HTTP-type connections. (Default value: 10)
- Rendering Delay: The browser may receive a notification indicating that a page is fully loaded, however this information may not be accurate (depending on how the page was constructed). A specific time duration (in seconds) is established for security reasons, to ensure the retrieval of all pertinent information from the page. We recommend extending this duration, especially for Applications with a sluggish response. (Default value: 5)
- Time To Render: This timeout value represents the duration the Scanner waits for a page to fully load in render mode, which involves executing scripts and other processes. (Default value: 60).
Tips:
|
Scan execution: how our scans work
Our Discovery scans use a range of different techniques to map out external attack surface. Most are passive. The only exception is when the scan connects to an application and renders it to build a map (similar to what a visitor sees on browsing your application).
Scan results returned are of a discovery or hygiene nature for applications, certificates and domains.
See also:
- Running a Scan
- Working with Scan Items
Was this article helpful?
That’s Great!
Thank you for your feedback
Sorry! We couldn't be helpful
Thank you for your feedback
Feedback sent
We appreciate your effort and will try to fix the article