Many of my recent talks on automation strategies have referred to a tool called “Marinus.” The tool is designed to help solve the challenges large organizations face in having an accurate view of their external facing infrastructure. “Marinus” can be a useful component of your broader security risk strategy and toolkit by helping you more quickly uncover potential problem areas. Today, Adobe is releasing an open-source version of Marinus enabling those in the security community to leverage it within their organizations.
Shadow IT, legacy systems, acquisitions, and other aspects of running a large infrastructure can make it difficult to keep track of your Internet-facing footprint. However, these are often the very systems that attackers will target as their first step to encroaching on your critical systems. For an organization to have an accurate risk assessment of their exposure, they first need to identify their “unknown unknowns.”
The name “Marinus” refers to Marinus of Tyre who was an early pioneer in geography. Similarly, the goal of the Marinus project is to assist organizations in creating complete maps of their networks. Marinus collects a myriad of data such as DNS records, reverse DNS records, TLS certificates, open ports, HTTP headers, and several other types of information that is publicly viewable from the Internet. Once collected, Marinus data can be used to identify risks such as sub-domain takeovers, internal services that are unintentionally exposed, VirusTotal detections, and much more. It can also provide a visual summary of the network. A very simple example of the domain adobesign.com is shown below:
In the picture above, the “www” and “trust” sub-domains share the same coloring as their parent domain “adobesign.com.” Domains that are unrelated to adobesign.com are given different colors and called out in the legend. In this case, both “www.adobesign.com” and “trust.adobesign.com” are CNAME references to “orgin.acrobat.adobe.com.” The AWS IP addresses used by adobesign.com are grouped together by color, as well. If a traditional data center IP had been identified, then it would have been colored differently and grouped by its Class C. These visual summaries can be a great help when dealing with more complex domains. The legend will show all of the Class C’s and third-party domains that are in use and how they relate to the core domain. The map will show you all the sub-domains that are viewable from the Internet and how they are related to IP addresses and third-party domains.
Marinus is capable of collecting data on any given network from a wide variety of free, commercial, and internal sources. Marinus uses numerous sources in order to create as complete a view of the network as possible. If Marinus were to rely solely on internal services that IT regularly uses, then it would be blind to issues such as shadow IT or forgotten legacy systems. By leveraging third-party data sources, Marinus is able to provide an unbiased view of the network. Marinus can also gain perspective on new acquisitions that haven’t yet been integrated into corporate management solutions.
Projects such as Rapid7’s Open Data project, VirusTotal, Certificate Transparency logs, and Common Crawl can provide free data on the hosts and services that are currently exposed. Marinus provides support for several commercial services, such as PassiveTotal and Censys. In addition, Marinus can collect data from internal DNS tracking services such as InfoBlox, UltraDNS, AWS Route53, and Azure DNS. By mixing internal and external data, Marinus can provide perspective on how much is known vs. unknown within the organization. Internal data can also be used to help identify which teams own the systems that require a follow-on internal investigation. By using data from public sources, Marinus does not need to conduct its own network scans. Although, there is an option for having Marinus collect data using tools from the ZMap project if it will help your organization.
The value of combining multiple data sources is not limited to gaining a complete set of DNS records for the entire organization. These sources often contain addition information such as TLS certificates, HTTP headers, and handshake information from open ports. This allows a Marinus user to quickly search their entire organization for best practice adoption, out of date environments, policy violations, and much more. For a central security team in a large organization, being able to quickly search a database of information that covers the entire company saves a lot of phone calls and emails. In addition, since this data was collected by external third-parties, there is no confusion about whether the data that you are searching would be visible to the outside world.
The Marinus suite requires three components: A Mongo database is required for storing collected data. One or more servers will be needed to run the individual Python scripts that will collect the data from the remote sources and store it in the Mongo database. Finally, Marinus requires a server to run the interactive NodeJS web site that is used for viewing the data. The NodeJS web server also provides REST APIs so that organizations can extract data from Marinus for their own custom reports or internal automation. The GitHub README files contain the details on the specifics of what is required in these environments.
Marinus has been in development for over two years. As with any project, there is always more that can be done. For instance, I am currently investigating possible integrations for Shodan and Google Compute. That said, the goal of Marinus is not meant to be all things to all people. The UI for the Marinus web site is designed to allow quick one-off searches that would otherwise be too onerous to write a script to find. There are also a few sample reports to inspire people on how the data can be turned into actionable information. However, the real power of Marinus is that it is a database of useful information on every host in the organization that is made available via REST APIs. Users can access the data via the REST APIs to power their existing automation, perform company-wide searches, and create their own reporting.
I would like to thank fellow Adobe team members Mayank Goyal and Bhumika Singhal for their contributions to the project during development. The Marinus project is now available on the Adobe GitHub portal.
Peleus Uhley
Principal Scientist