How to Automate OSINT with Spiderfoot on Kasm Workspaces

In this post, I'll walk you through how to install Spiderfoot on Kasm Workspaces and automate your Open Source Intelligence (OSINT) research with it.

SpiderFoot is an open source intelligence (OSINT) automation tool. It integrates with just about every data source available and utilises a range of methods for data analysis, making that data easy to navigate.

Kasm Workspaces is a container streaming platform that lets you create docker containers and stream them directly from your web browser with an enhanced web streaming technology. It is a great way to enable the delivery of containerized workloads to your browser.

The advantage of using Spiderfoot inside Kasm workspaces is that you get to conduct your investigations in a private container which can be streamed directly from your browser. You also do not need to use your browser for any task other than accessing the containerized Spiderfoot session on Kasm. This lets you keep your OSINT investigation inside an isolated sandbox and prevent risks of tracking, exposing self, targeted my malware, etc.

Install Kasm Workspaces

Kasm can be installed on Linux machines with at least 2 cores CPU, 4GB RAM and 50GB SSD storage.

The list of supported Operating Systems is listed here: https://kasmweb.com/docs/latest/install/system_requirements.html

Execute these four simple commands to install Kasm (official installation guide is here)

cd /tmp
curl -O https://kasm-static-content.s3.amazonaws.com/kasm_release_1.14.0.3a7abb.tar.gz
tar -xf kasm_release_1.14.0.3a7abb.tar.gz
sudo bash kasm_release/install.sh

Once Kasm is installed, you will be displayed the randomly generated credentials that you can use to login. Simply go to https://localhost in your browser and use these credentials to login to your Dashboard.

Install Spiderfoot in Kasm

The official image of Spiderfoot is available in Kasm's registry: https://registry.kasmweb.com/1.0/

You can simply install it from the official registry by clicking on the image and select "Install". This will do everything for you and install Spiderfoot on Kasm Workspaces.

Another way of installing Spiderfoot on Kasm is from a custom image. This is completely optional, and the easier way is to install it directly from the registry.

Step 1. Clone the Spiderfoot docker image for Kasm

git clone https://github.com/teja156/spiderfoot-kasm

Step 2. Build the docker image

cd spiderfoot-kasm/
sudo docker build -t spiderfoot -f Dockerfile .

Step 3. Install the custom image on Kasm

• Go to Workspaces -> Add Manually -> Add Workspaces

• Select Workspaces Type as "Container"

• Choose a friendly name, description, and make sure to enable the image.

• For the docker image field, put in your docker image name along with its tag: spiderfoot:latest.

• Set CPU cores as 2, Memory as 2768, GPU count as 0, CPU Allocation Method as Inherit and save the image.

And that's it! Spiderfoot should be available to use instantly in your Workspaces.

Just create a session, and you will be presented with a browser that has the Spiderfoot web interface.

Automating OSINT with Spiderfoot

Spiderfoot supports many type of target types like a website name, IP address, Email address, Bitcoin address, etc.

Let's perform a scan against a website victorlivestockfarm[.]co[.]za which is a scam website. I'll do an "All" scan which executes all the available modules against the target and produces as much data as it can about the target.

AS the scan is running, you can see the data it produced in real-time from executing different modules.

For example, let's have a look at the "SSL Certificate - Issued to" module results

We were able to uncover a new domain abakhisa[.]co[.].za which is potentially hosted by the same scammer (or team of scammers) who is hosting our current target.

We were also able to find multiple subdomains of the website like cpanel[.]victorlivestockfarm[.]co[.]za which hosts the CPanel Login page for the website admins to login.

Similarly, we were able to uncover multiple other subdomains like mail, webdisk, whm, etc. All of these subdomains can be further researched to gather even more information about the target.

The "Email Address - Generic" module scan shows us a list of email addresses found on the target.

The "Co-Hosted Site" module results shows a list of sites that are co-hosted by the same owner/entity that is running the current target. Do note that not all the results are accurate and there can be false positives too.

Similarly, there are many other modules that produce lots of useful, interesting information about the target. The most powerful aspect of Spiderfoot is that gathering these results is all automated which saves you a lot of time. Manual enumeration and reconnaissance is definitely required but Spiderfoot helps you find useful information about the target quickly and helps you map your target so that you can head in the right direction with the manual enumeration.