Haxf4rall
  • Home
  • Become a Hacker
    • Get Started
    • Hacker Mindset
    • Roadmap
    • Simple Setup – Hacker 101
    • Types of Hackers
    • Recommended Courses
  • Boot People Offline
  • Courses
    • All Hacking Courses
    • Cyber Security School
  • CTF
    • Beginners to Advanced Guide
    • Create your own CTF box
    • Field and Resources Guide
    • Platforms & Wargames
    • Tools Used for Solving CTF
    • Writeups
  • Dark Web
    • Beginners Guide
    • Darknet Markets
    • Darkweb 101 (Anonymity Guide)
    • Dark Web OSINT Tools
    • Hacking Forums
    • Latest News
    • Onion Links
  • Hacker Gadgets
  • Hacking Books
  • Tools Directory
  • Home
  • 2022
  • June
  • 18
  • OSIPs – Gathers All Valid IP Addresses From All Text Files From A Directory, And Checks Them Against Whois Database, TOR Relays And Location

OSIPs – Gathers All Valid IP Addresses From All Text Files From A Directory, And Checks Them Against Whois Database, TOR Relays And Location

June 18, 2022 Comments Off on OSIPs – Gathers All Valid IP Addresses From All Text Files From A Directory, And Checks Them Against Whois Database, TOR Relays And Location
OSIPs - Gathers All Valid IP Addresses From All Text Files From A Directory, And Checks Them Against Whois Database, TOR Relays And Location cybersecurity ethical hacking hack android hack app hack wordpress hacker news hacking hacking tools for windows keylogger kit kitploit password brute force penetration testing pentest pentest android pentest linux pentest toolkit pentest tools spy tool kit spyware tools

This script scans every file from a given folder recursively, extracts every IPv4 and IPv6 address, filters out the public IP addresses and then queries these IPs against TOR relays, WhoIs service and Location service. It outputs 4 files: a CSV file and a JSON file with all details of the IP addresses, a CSV file containing an index of every found IP address and a KML file with all the locations gathered.

Features

  • Parses any number of files in a single run
  • Can also be run for a single input file
  • Extracts all unique valid IPv4 and IPv6 addresses (correctly comparing two addresses, even if they are written differently)
  • Collects publicly available Who Is information for all public IP addresses
  • Queries all public IP addresses against TOR relays
  • Collects geographical location information for all public IP addresses
  • Can be run with command line parameters in non-interactive mode for easy integration in other scripts
  • Can be run without parameters and the input values can be provided interactively
  • Allows user to select what steps to perform or exclude
  • Verbosely outputs information to console at every step that it performs
  • Exports all IP addresses details in both a CSV file and a JSON file
  • Exports a KML file that can be easily loaded on top of a map
  • Keeps an index of all found IP addresses in a separate CSV file for easy traceback

Pre-requisites:

Python 3.9.x

Installation

To install dependencies, run:

pip install -r requirements.txt

Compile to executable

pip install pyinstaller

pyinstaller --onefile osips.py

Usage

python osips.py [-h] [-F INPUTFOLDER] [-f INPUTFILE] [-p INPUTFILESPATTERN] [-o OUTPUTFILESNAME] [-t {y,n,Y,N}] [-w {y,n,Y,N}] [-l {y,n,Y,N}] [-locationEndpoint LOCATIONENDPOINT]

Interactively

python osips.py

Non-interactively

python osips.py --checkTor Y --checkWhoIs Y --checkLocations Y --inputFolder testFolder/ --inputFilesPattern * --outputFilesName test

Help

python osips.py -h

Options

  • -h, --help Show the help message and exit
  • -F INPUTFOLDER, --inputFolder INPUTFOLDER A folder containing files with IPs (log files, email files, text files etc). The IPs can be IPv4 and/or IPv6 and can be placed anywhere in the files. The script will parse every file in the folder and will scan for every IP
  • -f INPUTFILE, --inputFile INPUTFILE Use this option to scan only one file instead of a folder with files. The file can be any text file that contains IP addresses (log files, email files, text files etc.)
  • -p INPUTFILESPATTERN, --inputFilesPattern INPUTFILESPATTERN Enter the pattern to use for the files in the input folder. Example: * for all files, *.txt for text files.
  • -o OUTPUTFILESNAME, --outputFilesName OUTPUTFILESNAME The name for the output files, WITHOUT EXTENSION. 4 different files will be created with this name: a CSV file storing the IP details, an index file storing the IP indexes, a JSON file storing the details and a KML file storing the locations.
  • -t {y,n,Y,N}, --checkTor {y,n,Y,N} Specify if you want to check every public IP address against the TOR relays IP addresses
  • -w {y,n,Y,N}, --checkWhoIs {y,n,Y,N} Specify if you want to extract WhoIs information for every public IP address
  • -l {y,n,Y,N}, --checkLocations {y,n,Y,N} Specify if you want to geocode every public IP address and extract location info using https://reallyfreegeoip.org
  • -locationEndpoint LOCATIONENDPOINT Default REST endpoint for location is “https://reallyfreegeoip.org/json/“. You can specify a diffrent end point, includding placeholder. The result must be in JSON and must include the following fields: country_name, region_code, region_name, city, zip_code, time_zone, latitude, longitude, metro_code

Remarks

  • The tool extracts IPv4 addresses only in dot decimal notation (xxx.xxx.xxx.xxx). It doesn’t extract IPv4 addresses in decimal, octal or hexadecimal notation because of the high rate of false positives.
  • Depending on the number of public IP addresses found, querying the online services could take a long time. If you are dealing with a lot of IP addresses, it is recommended to first run the tool without querying any of the services and just to extract the IP addresses. After seeing the extraction result, you can calculate an estimate of the total run time of the tool if the services are queried, and eventually choose to query only one or two of the services.
  • TOR relays are checked at the moment of running the tool. Because these IPs are constantly changing, this information may not be very accurate. If a date can also be identified for an IP, then it should be checked against ExoneraTor at this address: https://metrics.torproject.org/exonerator.html
  • Who Is information as well as the location information is gathered live from publicly available services, so the IP addresses are sent to these services. There are downloadable databases that would allow you to collect this information locally, but these downloads are not free and would need constant updates. This tool is not built to work with local databases.
  • The default service used for location gathering is https://reallyfreegeoip.org. This is a free service with no limitations, but it may not be very accurate. You can change this service with another, by using the -locationEndpoint parameter of the script. You can provide another service that takes an IP address as a parameter and returns a JSON response. Use the as a placemark for the IP address location. Examples: https://ipapi.co//json, https://api.ipstack.com/?access_key=ACCESS_KEY, https://freeapi.robtex.com/ipquery/. Please note that some of these other REST endpoints may not return the required fields and thus will break the script’s logic. In this case, the method named RequestInfoFromRestEndpoint should be modified accordingly.

Acknowledgements

  • IPv6 regex –https://gist.github.com/dfee/6ed3a4b05cfe7a6faf40a2102408d5d8
  • Onion Peeler: Batch Tor Lookup Program – http://az4n6.blogspot.com/2017/02/onion-peeler-batch-tor-lookup-program.html
Download OSIPs

Post navigation

Cocaine Vendor Sentenced to Nine Years in Prison
UK Home Secretary Approves Assange's Extradition

Related Articles

PlutoCrypt Ransomware Decryptor

PlutoCrypt Ransomware Decryptor

- Hack Tools
May 27, 2023
CISA Adds CVE-2023-2868 Vulnerability to KEV Catalog

CISA Adds CVE-2023-2868 Vulnerability to KEV Catalog

- Hack Tools
May 26, 2023
PoC Exploit Released for GitLab CVE-2023-2825 Vulnerability

PoC Exploit Released for GitLab CVE-2023-2825 Vulnerability

- Hack Tools
May 26, 2023
hacker gadgets
hacker phone covers

Recent Posts

PlutoCrypt Ransomware Decryptor

PlutoCrypt Ransomware Decryptor

May 27, 2023
Galaxy Fold 4

Samsung to improve the durability of the waterdrop hinges in the foldable smartphones

May 26, 2023
CISA Adds CVE-2023-2868 Vulnerability to KEV Catalog

CISA Adds CVE-2023-2868 Vulnerability to KEV Catalog

May 26, 2023
Google releases Chrome version 111 to fix 40 security vulnerabilities

Google releases Chrome version 111 to fix 40 security vulnerabilities

May 26, 2023
PoC Exploit Released for GitLab CVE-2023-2825 Vulnerability

PoC Exploit Released for GitLab CVE-2023-2825 Vulnerability

May 26, 2023
CVE View

Mondoo v7.17.1 releases: Cloud-Native Security & Vulnerability Risk Management

February 16, 2023

Social Media Hacking

SocialPath – Track users across Social Media Platforms

SocialPath – Track users across Social Media Platforms

- Social Media Hacking
October 16, 2019October 16, 2019

SocialPath is a django application for gathering social media intelligence on specific username. It checks for Twitter, Instagram, Facebook, Reddit...

SocialScan – Check Email Address and Username Availability on Online Platforms

SocialScan – Check Email Address and Username Availability on Online Platforms

June 17, 2019
Shellphish – Phishing Tool For 18 Social Media Apps

Shellphish – Phishing Tool For 18 Social Media Apps

June 10, 2019July 27, 2019
WhatsApp Hacking using QRLJacking

WhatsApp Hacking using QRLJacking

May 2, 2019May 19, 2019
How to Hack any Facebook Account with Z-Shadow

How to Hack any Facebook Account with Z-Shadow

April 26, 2019June 29, 2020
hacker buffs

About Us

Haxf4rall is a collective, a good starting point and provides a variety of quality material for cyber security professionals.

Active Members

Submit a Tool

Hackers Handbook 2018


Grab your copy here

About Us

Haxf4rall is a collective, a good starting point and provides a variety of quality material for cyber security professionals.

Categories

  • Secure Coding
  • Documentary
  • Courses & Ebooks
  • Hack Tools
  • Hacking Tutorials
  • Mobile Hacking
  • News
  • Operating Systems
  • TOR
  • Tricks & How To’s

Active Members

Useful Links

Contact Us

Disclaimer

Privacy Policy

Submit a Tool

Copyright 2019. All rights reserved | Theme: OMag by LilyTurf Themes