SpiderFoot -- Most Complete OSINT Reconnaissance Tool

During a penetration testing, the most crucial aspects of engaging a target is information gathering. The more information we collected, the more likely the attack is to succeed. In this tutorial we look at SpiderFoot. SpiderFoot is a reconnaissance tool that automatically queries over 100 public data sources to gather intelligence on IP address, domain names, email address, names and more. We just simply specify the target we want to investigate and pick which modules to enable, then SpiderFoot will collect data to build up an understanding of all the entities and shows the relation between each.

SpiderFoot id a OSINT tool. Open-source intelligence (OSINT) is data that can be gathered from public sources. This isn't just limited to the internet,This also can gather through print media, government records, academic publications and many more.

spiderfoot kali linux tutorial

SpiderFoot is written in Python 2.7 and it has an interactive web-based interface or a powerful command-line interface.
SpiderFoot is highly configurable and supports cross platform. It's available for Windows, Linux and MacOS.

Kali Linux doesn't comes with SpiderFoot installed by default yet, so it comes in custom tools category. First we open our Kali Linux terminal window and type following command to download SpiderFoot from its GitHub repository:

git clone https://github.com/smicallef/spiderfoot
The screenshot is following:

git clone spiderfoot

Then we need to go to the SpiderFoot directory by using cd command :

cd spiderfoot
Now we install all the requirements by using following command:

pip install -r requirements.txt
 The screenshot of the command is following:


requirements for spiderfoot

Then we can run this tool by applying following command:

./sf.py
Then SpiderFoot will start it's server for web-based interface as following screenshot:

spiderfoot web server

Now we can use SpiderFoot by starting our web browser and navigate to 127.0.0.1:5001 . The screenshot is following:

spiderfoot


Here we can see the web-based interface of SpiderFoot. To start a scan we simply click on "New Scan" and the screenshot as following:

scan using spiderfoot


We can enter name of the scan, we can put any name here. Then we can set our target. The target will be a domain name, IP address, host name/sub-domain, sub-net,even we can enter e-mail address or phone number or a name of someone.

Then we need to select the scanning method. We can use  the scan as foot-printing or investigation or passive scan, even we can choose first option to choose all the options.

Here we can choose a scan by our required data from another menu.

new scan


We can choose modules to do a better scan. Some modules are locked, these modules will work if we add the API keys. To do that we need to go to the official website of modules and register to get API key. Then we paste the API key in the modules setting in the settings menu. Adding API keys will make our scan batter. Then we can scan a target. Here is a scan result of an IP address.

Spiderfoot scan



7 comments:

  1. hey, no module named openxmllib,, why ????

    ReplyDelete
    Replies
    1. Thanks for your comment.
      We are happy to help.

      It appears you are missing a module required for SpiderFoot to function. Please refer to the documentation for the list of dependencies and install them.

      Python modules required are:

      M2Crypto
      netaddr
      dns
      cherrypy
      mako
      socks
      whois
      PyPDF2
      openxmllib
      stem
      bs4
      gexf
      phonenumbers
      ipaddr
      ipwhois

      Please note that if you are seeing this after doing a git pull then you just need to do a pip install -r requirements.txt as dependencies previously bundled with SpiderFoot are now unbundled.

      Delete
    2. try following command:
      pip install openxmllib

      Delete
    3. thank you, sir... Thank you for your answer

      Delete
    4. Did it solve the problem? I keep getting the same error and is unable to run sf.py.

      Delete
  2. Hi ! any wat to get spidefoot installed instead of cloning it each time I need to use it ? Many thanks in advance

    ReplyDelete
    Replies
    1. Nope, you don't need to clone it neither need to install dependencies. You just need to go to the directory by using the cd command and run this tool by using ./sf.py command.. That's all. Thanks.

      Delete

Please do not spam here. It is comment box not a spambox. Promotional links are not allowed.

Powered by Blogger.