The Book of Secret Knowledge

The Book of Secret Knowledge

Knowledge is powerful, be careful how you use it!

A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools, and more.

What is it?

This repository is a collection of various materials and tools that I use every day in my work. It contains a lot of useful information gathered in one piece. It is an invaluable source of knowledge for me that I often look back on.

restroom  For whom?

For everyone, really. Here everyone can find their favourite tastes. But to be perfectly honest, it is aimed towards System and Network administrators, DevOps, Pentesters, and Security Researchers.

information_source  Contributing

If you find something which doesn’t make sense, or something doesn’t seem right, please make a pull request and please add valid and well-reasoned explanations about your changes or comments.

A few simple rules for this project:

  • inviting and clear
  • not tiring
  • useful

These below rules may be better:

  • easy to contribute to (Markdown + HTML …)
  • easy to find (simple TOC, maybe it’s worth extending them?)

Url marked * is temporary unavailable. Please don’t delete it without confirming that it has permanently expired.

Before adding a pull request, please see the contributing guidelines. You should also remember about this:

+ This repository is not meant to contain everything but only good quality stuff.

All suggestions/PR are welcome!

Code Contributors

This project exists thanks to all the people who contribute.

Financial Contributors

 

Individuals

Become a financial contributor and help us sustain our community » contribute.

Organizations

Support this project with your organization. Your logo will show up here with a link to your website » contribute.

gift_heart  Support

If this project is useful and important for you or if you really like the-book-of-secret-knowledge, you can bring positive energy by giving some good words or supporting this project. Thank you!

newspaper  RSS Feed & Updates

GitHub exposes an RSS/Atom feed of the commits, which may also be useful if you want to be kept informed about all changes.

ballot_box_with_check  ToDo

  •  Add new stuff…
  •  Add useful shell functions
  •  Add one-liners for collection tools (eg. CLI Tools)
  •  Sort order in lists

New items are also added on a regular basis.

anger  Table of Contents

Only main chapters:

trident  The Book of Secret Knowledge (Chapters)

CLI Tools  [TOC]

black_small_square Shells

small_orange_diamond GNU Bash – is an sh-compatible shell that incorporates useful features from the Korn shell and C shell.
small_orange_diamond Zsh – is a shell designed for interactive use, although it is also a powerful scripting language.
small_orange_diamond tclsh – is a very powerful cross-platform shell, suitable for a huge range of uses.
small_orange_diamond bash-it – is a framework for using, developing and maintaining shell scripts and custom commands.
small_orange_diamond Oh My ZSH! – is the best framework for managing your Zsh configuration.
small_orange_diamond Oh My Fish – the Fishshell framework.
small_orange_diamond Starship – the cross-shell prompt written in Rust.
small_orange_diamond powerlevel10k – is a fast reimplementation of Powerlevel9k ZSH theme.

black_small_square Shell plugins

small_orange_diamond z – tracks the folder you use the most and allow you to jump, without having to type the whole path.
small_orange_diamond fzf – is a general-purpose command-line fuzzy finder.
small_orange_diamond zsh-autosuggestions – Fish-like autosuggestions for Zsh.
small_orange_diamond zsh-syntax-highlighting – Fish shell like syntax highlighting for Zsh.

black_small_square Managers

small_orange_diamond Midnight Commander – is a visual file manager, licensed under GNU General Public License.
small_orange_diamond ranger – is a VIM-inspired filemanager for the console.
small_orange_diamond nnn – is a tiny, lightning fast, feature-packed file manager.
small_orange_diamond screen – is a full-screen window manager that multiplexes a physical terminal.
small_orange_diamond tmux – is a terminal multiplexer, lets you switch easily between several programs in one terminal.
small_orange_diamond tmux-cssh – is a tool to set comfortable and easy to use functionality, clustering and synchronizing tmux-sessions.

black_small_square Text editors

small_orange_diamond vi – is one of the most common text editors on Unix.
small_orange_diamond vim – is a highly configurable text editor.
small_orange_diamond emacs – is an extensible, customizable, free/libre text editor – and more.
small_orange_diamond micro – is a modern and intuitive terminal-based text editor.
small_orange_diamond neovim – is a free open source, powerful, extensible and usable code editor.
small_orange_diamond spacemacs – a community-driven Emacs distribution.

black_small_square Files and directories

small_orange_diamond fd – is a simple, fast and user-friendly alternative to find.
small_orange_diamond ncdu – is an easy to use, fast disk usage analyzer.

black_small_square Network

small_orange_diamond PuTTY – is an SSH and telnet client, developed originally by Simon Tatham.
small_orange_diamond nmap – is a free and open source (license) utility for network discovery and security auditing.
small_orange_diamond zmap – is a fast single packet network scanner designed for Internet-wide network surveys.
small_orange_diamond masscan – is the fastest Internet port scanner, spews SYN packets asynchronously.
small_orange_diamond pbscan – is a faster and more efficient stateless SYN scanner and banner grabber.
small_orange_diamond hping – is a command-line oriented TCP/IP packet assembler/analyzer.
small_orange_diamond mtr – is a tool that combines the functionality of the ‘traceroute’ and ‘ping’ programs in a single network diagnostic tool.
small_orange_diamond mylg – is an open source utility which combines the functions of the different network probes in one diagnostic tool.
small_orange_diamond netcat – is a networking utility which reads and writes data across network connections, using the TCP/IP protocol.
small_orange_diamond tcpdump – is a powerful command-line packet analyzer.
small_orange_diamond tshark – is a tool that allows us to dump and analyze network traffic (wireshark cli).
small_orange_diamond Termshark – is a simple terminal user-interface for tshark.
small_orange_diamond ngrep – is like GNU grep applied to the network layer.
small_orange_diamond netsniff-ng – is a Swiss army knife for your daily Linux network plumbing if you will.
small_orange_diamond sockdump – dump unix domain socket traffic.
small_orange_diamond stenographer – is a packet capture solution which aims to quickly spool all packets to disk.
small_orange_diamond tcpterm – visualize packets in TUI.
small_orange_diamond bmon – is a monitoring and debugging tool to capture networking related statistics and prepare them visually.
small_orange_diamond iptraf-ng – is a console-based network monitoring program for Linux that displays information about IP traffic.
small_orange_diamond vnstat – is a network traffic monitor for Linux and BSD.
small_orange_diamond iPerf3 – is a tool for active measurements of the maximum achievable bandwidth on IP networks.
small_orange_diamond ethr – is a Network Performance Measurement Tool for TCP, UDP & HTTP.
small_orange_diamond Etherate – is a Linux CLI based Ethernet and MPLS traffic testing tool.
small_orange_diamond echoip – is a IP address lookup service.
small_orange_diamond Nemesis – packet manipulation CLI tool; craft and inject packets of several protocols.
small_orange_diamond packetfu – a mid-level packet manipulation library for Ruby.
small_orange_diamond Scapy – packet manipulation library; forge, send, decode, capture packets of a wide number of protocols.
small_orange_diamond impacket – is a collection of Python classes for working with network protocols.
small_orange_diamond ssh-audit – is a tool for SSH server auditing.
small_orange_diamond aria2 – is a lightweight multi-protocol & multi-source command-line download utility.
small_orange_diamond iptables-tracer – observe the path of packets through the iptables chains.

black_small_square Network (DNS)

small_orange_diamond dnsdiag – is a DNS diagnostics and performance measurement tools.
small_orange_diamond fierce – is a DNS reconnaissance tool for locating non-contiguous IP space.
small_orange_diamond subfinder – is a subdomain discovery tool that discovers valid subdomains for websites.
small_orange_diamond sublist3r – is a fast subdomains enumeration tool for penetration testers.
small_orange_diamond amass – is tool that obtains subdomain names by scraping data sources, crawling web archives and more.
small_orange_diamond namebench – provides personalized DNS server recommendations based on your browsing history.
small_orange_diamond massdns – is a high-performance DNS stub resolver for bulk lookups and reconnaissance.
small_orange_diamond knock – is a tool to enumerate subdomains on a target domain through a wordlist.
small_orange_diamond dnsperf – DNS performance testing tools.
small_orange_diamond dnscrypt-proxy 2 – a flexible DNS proxy, with support for encrypted DNS protocols.
small_orange_diamond dnsdbq – API client providing access to passive DNS database systems (pDNS at Farsight Security, CIRCL pDNS).
small_orange_diamond grimd – fast dns proxy, built to black-hole internet advertisements and malware servers.

black_small_square Network (HTTP)

small_orange_diamond Curl – is a command line tool and library for transferring data with URLs.
small_orange_diamond kurly – is an alternative to the widely popular curl program, written in Golang.
small_orange_diamond HTTPie – is an user-friendly HTTP client.
small_orange_diamond wuzz – is an interactive cli tool for HTTP inspection.
small_orange_diamond htrace.sh – is a simple Swiss Army knife for http/https troubleshooting and profiling.
small_orange_diamond httpstat – is a tool that visualizes curl statistics in a way of beauty and clarity.
small_orange_diamond httplab – is an interactive web server.
small_orange_diamond Lynx – is a text browser for the World Wide Web.
small_orange_diamond HeadlessBrowsers – a list of (almost) all headless web browsers in existence.
small_orange_diamond ab – is a single-threaded command line tool for measuring the performance of HTTP web servers.
small_orange_diamond siege – is an http load testing and benchmarking utility.
small_orange_diamond wrk – is a modern HTTP benchmarking tool capable of generating significant load.
small_orange_diamond wrk2 – is a constant throughput, correct latency recording variant of wrk.
small_orange_diamond vegeta – is a constant throughput, correct latency recording variant of wrk.
small_orange_diamond bombardier – is a fast cross-platform HTTP benchmarking tool written in Go.
small_orange_diamond gobench – http/https load testing and benchmarking tool.
small_orange_diamond hey – HTTP load generator, ApacheBench (ab) replacement, formerly known as rakyll/boom.
small_orange_diamond boom – is a script you can use to quickly smoke-test your web app deployment.
small_orange_diamond SlowHTTPTest – is a tool that simulates some Application Layer Denial of Service attacks by prolonging HTTP.
small_orange_diamond gobuster – is a free and open source directory/file & DNS busting tool written in Go.
small_orange_diamond ssllabs-scan – command-line reference-implementation client for SSL Labs APIs.
small_orange_diamond http-observatory – Mozilla HTTP Observatory cli version.

black_small_square SSL

small_orange_diamond openssl – is a robust, commercial-grade, and full-featured toolkit for the TLS and SSL protocols.
small_orange_diamond gnutls-cli – client program to set up a TLS connection to some other computer.
small_orange_diamond sslyze – fast and powerful SSL/TLS server scanning library.
small_orange_diamond sslscan – tests SSL/TLS enabled services to discover supported cipher suites.
small_orange_diamond testssl.sh – testing TLS/SSL encryption anywhere on any port.
small_orange_diamond cipherscan – a very simple way to find out which SSL ciphersuites are supported by a target.
small_orange_diamond spiped – is a utility for creating symmetrically encrypted and authenticated pipes between socket addresses.
small_orange_diamond Certbot – is EFF’s tool to obtain certs from Let’s Encrypt and (optionally) auto-enable HTTPS on your server.
small_orange_diamond mkcert – simple zero-config tool to make locally trusted development certificates with any names you’d like.
small_orange_diamond certstrap – tools to bootstrap CAs, certificate requests, and signed certificates.
small_orange_diamond Sublert – is a security and reconnaissance tool to automatically monitor new subdomains.
small_orange_diamond mkchain – open source tool to help you build a valid SSL certificate chain.

black_small_square Security

small_orange_diamond SELinux – provides a flexible Mandatory Access Control (MAC) system built into the Linux kernel.
small_orange_diamond AppArmor – proactively protects the operating system and applications from external or internal threats.
small_orange_diamond grapheneX – Automated System Hardening Framework.
small_orange_diamond DevSec Hardening Framework – Security + DevOps: Automatic Server Hardening.

black_small_square Auditing Tools

small_orange_diamond ossec – actively monitoring all aspects of system activity with file integrity monitoring.
small_orange_diamond auditd – provides a way to track security-relevant information on your system.
small_orange_diamond Tiger – is a security tool that can be use both as a security audit and intrusion detection system.
small_orange_diamond Lynis – battle-tested security tool for systems running Linux, macOS, or Unix-based operating system.
small_orange_diamond LinEnum – scripted Local Linux Enumeration & Privilege Escalation Checks.
small_orange_diamond Rkhunter – scanner tool for Linux systems that scans backdoors, rootkits and local exploits on your systems.
small_orange_diamond PE-sieve – is a light-weight tool that helps to detect malware running on the system.

black_small_square System Diagnostics/Debuggers

small_orange_diamond strace – diagnostic, debugging and instructional userspace utility for Linux.
small_orange_diamond DTrace – is a performance analysis and troubleshooting tool.
small_orange_diamond ltrace – is a library call tracer, used to trace calls made by programs to library functions.
small_orange_diamond ptrace-burrito – is a friendly wrapper around ptrace.
small_orange_diamond perf-tools – performance analysis tools based on Linux perf_events (aka perf) and ftrace.
small_orange_diamond bpftrace – high-level tracing language for Linux eBPF.
small_orange_diamond sysdig – system exploration and troubleshooting tool with first class support for containers.
small_orange_diamond Valgrind – is an instrumentation framework for building dynamic analysis tools.
small_orange_diamond gperftools – high-performance multi-threaded malloc() implementation, plus some performance analysis tools.
small_orange_diamond glances – cross-platform system monitoring tool written in Python.
small_orange_diamond htop – interactive text-mode process viewer for Unix systems. It aims to be a better ‘top’.
small_orange_diamond nmon – a single executable for performance monitoring and data analysis.
small_orange_diamond atop – ASCII performance monitor. Includes statistics for CPU, memory, disk, swap, network, and processes.
small_orange_diamond lsof – displays in its output information about files that are opened by processes.
small_orange_diamond FlameGraph – stack trace visualizer.
small_orange_diamond lsofgraph – small utility to convert Unix lsof output to a graph showing FIFO and UNIX interprocess communication.
small_orange_diamond rr – is a lightweight tool for recording, replaying and debugging execution of applications.
small_orange_diamond Performance Co-Pilot – a system performance analysis toolkit.
small_orange_diamond hexyl – a command-line hex viewer.

black_small_square Log Analyzers

small_orange_diamond angle-grinder – slice and dice log files on the command line.
small_orange_diamond lnav – log file navigator with search and automatic refresh.
small_orange_diamond GoAccess – real-time web log analyzer and interactive viewer that runs in a terminal.
small_orange_diamond ngxtop – real-time metrics for nginx server.

black_small_square Databases

small_orange_diamond usql – universal command-line interface for SQL databases.
small_orange_diamond pgcli – postgres CLI with autocompletion and syntax highlighting.
small_orange_diamond mycli – terminal client for MySQL with autocompletion and syntax highlighting.
small_orange_diamond litecli – SQLite CLI with autocompletion and syntax highlighting.
small_orange_diamond OSQuery – is a SQL powered operating system instrumentation, monitoring, and analytics framework.

black_small_square TOR

small_orange_diamond Nipe – script to make Tor Network your default gateway.
small_orange_diamond multitor – a tool that lets you create multiple TOR instances with a load-balancing.

black_small_square Messengers/IRC Clients

small_orange_diamond Irssi – is a free open source terminal based IRC client.
small_orange_diamond WeeChat – is an extremely extensible and lightweight IRC client.

black_small_square Other

small_orange_diamond sysadmin-util – tools for Linux/Unix sysadmins.
small_orange_diamond incron – is an inode-based filesystem notification technology.
small_orange_diamond lsyncd – synchronizes local directories with remote targets (Live Syncing Daemon).
small_orange_diamond GRV – is a terminal based interface for viewing Git repositories.
small_orange_diamond Tig – text-mode interface for Git.
small_orange_diamond tldr – simplified and community-driven man pages.
small_orange_diamond archiver – easily create and extract .zip, .tar, .tar.gz, .tar.bz2, .tar.xz, .tar.lz4, .tar.sz, and .rar.
small_orange_diamond commander.js – minimal CLI creator in JavaScript.
small_orange_diamond gron – make JSON greppable!
small_orange_diamond bed – binary editor written in Go.

GUI Tools  [TOC]

black_small_square Terminal emulators

small_orange_diamond Guake – is a dropdown terminal made for the GNOME desktop environment.
small_orange_diamond Terminator – is based on GNOME Terminal, useful features for sysadmins and other users.
small_orange_diamond Kitty – is a GPU based terminal emulator that supports smooth scrolling and images.

black_small_square Network

small_orange_diamond Wireshark – is the world’s foremost and widely-used network protocol analyzer.
small_orange_diamond Ettercap – is a comprehensive network monitor tool.
small_orange_diamond EtherApe – is a graphical network monitoring solution.
small_orange_diamond Packet Sender – is a networking utility for packet generation and built-in UDP/TCP/SSL client and servers.
small_orange_diamond Ostinato – is a packet crafter and traffic generator.
small_orange_diamond JMeter™ – open source software to load test functional behavior and measure performance.
small_orange_diamond locust – scalable user load testing tool written in Python.

black_small_square Browsers

small_orange_diamond TOR Browser – protect your privacy and defend yourself against network surveillance and traffic analysis.

black_small_square Password Managers

small_orange_diamond KeePassXC – store your passwords safely and auto-type them into your everyday websites and apps.
small_orange_diamond Enpass – password manager and secure wallet.

black_small_square Messengers/IRC Clients

small_orange_diamond HexChat – is an IRC client based on XChat.
small_orange_diamond Pidgin – is an easy to use and free chat client used by millions.

black_small_square Messengers (end-to-end encryption)

small_orange_diamond Signal – is an encrypted communications app.
small_orange_diamond Wire – secure messaging, file sharing, voice calls and video conferences. All protected with end-to-end encryption.
small_orange_diamond TorChat – decentralized anonymous instant messenger on top of Tor Hidden Services.
small_orange_diamond Matrix – an open network for secure, decentralized, real-time communication.

black_small_square Text editors

small_orange_diamond Sublime Text – is a lightweight, cross-platform code editor known for its speed, ease of use.
small_orange_diamond Visual Studio Code – an open-source and free source code editor developed by Microsoft.
small_orange_diamond Atom – a hackable text editor for the 21st Century.

Web Tools  [TOC]

black_small_square Browsers

small_orange_diamond SSL/TLS Capabilities of Your Browser – test your browser’s SSL implementation.
small_orange_diamond Can I use – provides up-to-date browser support tables for support of front-end web technologies.
small_orange_diamond Panopticlick 3.0 – is your browser safe against tracking?
small_orange_diamond Privacy Analyzer – see what data is exposed from your browser.
small_orange_diamond Web Browser Security – it’s all about Web Browser fingerprinting.
small_orange_diamond How’s My SSL? – help a web server developer learn what real world TLS clients were capable of.
small_orange_diamond sslClientInfo – client test (incl TLSv1.3 information).

black_small_square SSL/Security

small_orange_diamond SSLLabs Server Test – free online service performs a deep analysis of the configuration of any SSL web server.
small_orange_diamond SSLLabs Server Test (DEV) – free online service performs a deep analysis of the configuration of any SSL web server.
small_orange_diamond ImmuniWeb® SSLScan – test SSL/TLS (PCI DSS, HIPAA and NIST).
small_orange_diamond SSL Check – scan your website for non-secure content.
small_orange_diamond CryptCheck – test your TLS server configuration (e.g. ciphers).
small_orange_diamond urlscan.io – service to scan and analyse websites.
small_orange_diamond Report URI – monitoring security policies like CSP and HPKP.
small_orange_diamond CSP Evaluator – allows developers and security experts to check if a Content Security Policy.
small_orange_diamond Useless CSP – public list about CSP in some big players (might make them care a bit more).
small_orange_diamond Why No HTTPS? – list of the world’s top 100 websites by Alexa rank not automatically redirecting insecure requests.
small_orange_diamond TLS Cipher Suite Search
small_orange_diamond cipherli.st – strong ciphers for Apache, Nginx, Lighttpd and more.*
small_orange_diamond dhtool – public Diffie-Hellman parameter service/tool.
small_orange_diamond badssl.com – memorable site for testing clients against bad SSL configs.
small_orange_diamond tlsfun.de – registered for various tests regarding the TLS/SSL protocol.
small_orange_diamond CAA Record Helper – generate a CAA policy.
small_orange_diamond Common CA Database – repository of information about CAs, and their root and intermediate certificates.
small_orange_diamond CERTSTREAM – real-time certificate transparency log update stream.
small_orange_diamond crt.sh – discovers certificates by continually monitoring all of the publicly known CT.
small_orange_diamond Hardenize – deploy the security standards.
small_orange_diamond Cipher suite compatibility – test TLS cipher suite compatibility.
small_orange_diamond urlvoid – this service helps you detect potentially malicious websites.
small_orange_diamond security.txt – a proposed standard (generator) which allows websites to define security policies.
small_orange_diamond ssl-config-generator – help you follow the Mozilla Server Side TLS configuration guidelines.

black_small_square HTTP Headers & Web Linters

small_orange_diamond Security Headers – analyse the HTTP response headers (with rating system to the results).
small_orange_diamond Observatory by Mozilla – set of tools to analyze your website.
small_orange_diamond webhint – is a linting tool that will help you with your site’s accessibility, speed, security and more.

black_small_square DNS

small_orange_diamond ViewDNS – one source for free DNS related tools and information.
small_orange_diamond DNSLookup – is an advanced DNS lookup tool.
small_orange_diamond DNSlytics – online DNS investigation tool.
small_orange_diamond DNS Spy – monitor, validate and verify your DNS configurations.
small_orange_diamond Zonemaster – helps you to control how your DNS works.
small_orange_diamond Leaf DNS – comprehensive DNS tester.
small_orange_diamond Find subdomains online – find subdomains for security assessment penetration test.
small_orange_diamond DNSdumpster – dns recon & research, find & lookup dns records.
small_orange_diamond DNS Table online – search for DNS records by domain, IP, CIDR, ISP.
small_orange_diamond intoDNS – DNS and mail server health checker.
small_orange_diamond DNS Bajaj – check the delegation of your domain.
small_orange_diamond BuddyDNS Delegation LAB – check, trace and visualize delegation of your domain.
small_orange_diamond dnssec-debugger – DS or DNSKEY records validator.
small_orange_diamond PTRarchive.com – this site is responsible for the safekeeping of historical reverse DNS records.
small_orange_diamond xip.io – wildcard DNS for everyone.
small_orange_diamond dnslookup (ceipam) – one of the best DNS propagation checker (and not only).
small_orange_diamond What’s My DNS – DNS propagation checking tool.

black_small_square Mail

small_orange_diamond smtp-tls-checker – check an email domain for SMTP TLS support.
small_orange_diamond MX Toolbox – all of your MX record, DNS, blacklist and SMTP diagnostics in one integrated tool.
small_orange_diamond Secure Email – complete email test tools for email technicians.
small_orange_diamond blacklistalert – checks to see if your domain is on a Real Time Spam Blacklist.
small_orange_diamond MultiRBL – complete IP check for sending Mailservers.
small_orange_diamond DKIM SPF & Spam Assassin Validator – checks mail authentication and scores messages with Spam Assassin.

black_small_square Encoders/Decoders and Regex testing

small_orange_diamond URL Encode/Decode – tool from above to either encode or decode a string of text.
small_orange_diamond Uncoder – the online translator for search queries on log data.
small_orange_diamond Regex101 – online regex tester and debugger: PHP, PCRE, Python, Golang and JavaScript.
small_orange_diamond RegExr – online tool to learn, build, & test Regular Expressions (RegEx / RegExp).
small_orange_diamond RegEx Testing – online regex testing tool.
small_orange_diamond RegEx Pal – online regex testing tool + other tools.
small_orange_diamond The Cyber Swiss Army Knife – a web app for encryption, encoding, compression and data analysis.

black_small_square Net-tools

small_orange_diamond Netcraft – detailed report about the site, helping you to make informed choices about their integrity.*
small_orange_diamond RIPE NCC Atlas – a global, open, distributed Internet measurement platform.
small_orange_diamond Robtex – uses various sources to gather public information about IP numbers, domain names, host names, routes etc.
small_orange_diamond Security Trails – APIs for Security Companies, Researchers and Teams.
small_orange_diamond Online Curl – curl test, analyze HTTP Response Headers.
small_orange_diamond Online Tools for Developers – HTTP API tools, testers, encoders, converters, formatters, and other tools.
small_orange_diamond Ping.eu – online Ping, Traceroute, DNS lookup, WHOIS and others.
small_orange_diamond Network-Tools – network tools for webmasters, IT technicians & geeks.
small_orange_diamond BGPview – search for any ASN, IP, Prefix or Resource name.
small_orange_diamond Riseup – provides online communication tools for people and groups working on liberatory social change.
small_orange_diamond VirusTotal – analyze suspicious files and URLs to detect types of malware.

black_small_square Privacy

small_orange_diamond privacytools.io – provides knowledge and tools to protect your privacy against global mass surveillance.
small_orange_diamond DNS Privacy Test Servers – DNS privacy recursive servers list (with a ‘no logging’ policy).

black_small_square Code parsers/playgrounds

small_orange_diamond ShellCheck – finds bugs in your shell scripts.
small_orange_diamond explainshell – get interactive help texts for shell commands.
small_orange_diamond jsbin – live pastebin for HTML, CSS & JavaScript and more.
small_orange_diamond CodeSandbox – online code editor for web application development. Supports React, Vue, Angular, CxJS, Dojo, etc.
small_orange_diamond PHP Sandbox – test your PHP code with this code tester.
small_orange_diamond Repl.it – an instant IDE to learn, build, collaborate, and host all in one place.

black_small_square Performance

small_orange_diamond GTmetrix – analyze your site’s speed and make it faster.
small_orange_diamond Sucuri loadtimetester – test here the performance of any of your sites from across the globe.
small_orange_diamond Pingdom Tools – analyze your site’s speed around the world.
small_orange_diamond PingMe.io – run website latency tests across multiple geographic regions.
small_orange_diamond PageSpeed Insights – analyze your site’s speed and make it faster.
small_orange_diamond web.dev – helps developers like you learn and apply the web’s modern capabilities to your own sites and apps.
small_orange_diamond Lighthouse – automated auditing, performance metrics, and best practices for the web.

black_small_square Mass scanners (search engines)

small_orange_diamond Censys – platform that helps information security practitioners discover, monitor, and analyze devices.
small_orange_diamond Shodan – the world’s first search engine for Internet-connected devices.
small_orange_diamond Shodan 2000 – do you use Shodan for everyday work? This tool looks for randomly generated data from Shodan.
small_orange_diamond GreyNoise – mass scanner such as Shodan and Censys.
small_orange_diamond ZoomEye – search engine for cyberspace that lets the user find specific network components.
small_orange_diamond netograph – tools to monitor and understand deep structure of the web.
small_orange_diamond FOFA – is a cyberspace search engine.
small_orange_diamond onyphe – is a search engine for open-source and cyber threat intelligence data collected.
small_orange_diamond IntelligenceX – is a search engine and data archive.
small_orange_diamond binaryedge – it scan the entire internet space and create real-time threat intelligence streams and reports.
small_orange_diamond wigle – is a submission-based catalog of wireless networks. All the networks. Found by Everyone.
small_orange_diamond PublicWWW – find any alphanumeric snippet, signature or keyword in the web pages HTML, JS and CSS code.
small_orange_diamond IntelTechniques – this repository contains hundreds of online search utilities.
small_orange_diamond hunter – lets you find email addresses in seconds and connect with the people that matter for your business.
small_orange_diamond GhostProject? – search by full email address or username.
small_orange_diamond databreaches – was my email affected by data breach?
small_orange_diamond We Leak Info – world’s fastest and largest data breach search engine.
small_orange_diamond Pulsedive – scans of malicious URLs, IPs, and domains, including port scans and web requests.
small_orange_diamond scylla – db dumps and more.
small_orange_diamond Buckets by Grayhatwarfar – database with public search for Open Amazon S3 Buckets and their contents.
small_orange_diamond Vigilante.pw – the breached database directory.
small_orange_diamond builtwith – find out what websites are built with.
small_orange_diamond NerdyData – search the web’s source code for technologies, across millions of sites.
small_orange_diamond Mamont’s open FTP Index – if a target has an open FTP site with accessible content it will be listed here.
small_orange_diamond OSINT Framework – focused on gathering information from free tools or resources.
small_orange_diamond maltiverse – is a service oriented to cybersecurity analysts for the advanced analysis of indicators of compromise.
small_orange_diamond Leaked Source – is a collaboration of data found online in the form of a lookup.
small_orange_diamond We Leak Info – to help everyday individuals secure their online life, avoiding getting hacked.
small_orange_diamond pipl – is the place to find the person behind the email address, social username or phone number.
small_orange_diamond abuse.ch – is operated by a random swiss guy fighting malware for non-profit.
small_orange_diamond malc0de – malware search engine.
small_orange_diamond Cybercrime Tracker – monitors and tracks various malware families that are used to perpetrate cyber crimes.
small_orange_diamond shhgit – find GitHub secrets in real time.
small_orange_diamond searchcode – helping you find real world examples of functions, API’s and libraries.
small_orange_diamond Insecam – the world biggest directory of online surveillance security cameras.
small_orange_diamond index-of – contains great stuff like: security, hacking, reverse engineering, cryptography, programming etc.

black_small_square Generators

small_orange_diamond thispersondoesnotexist – generate fake faces in one click – endless possibilities.
small_orange_diamond AI Generated Photos – 100.000 AI generated faces.
small_orange_diamond fakeface – fake faces browser.
small_orange_diamond Intigriti Redirector – open redirect/SSRF payload generator.

black_small_square Passwords

small_orange_diamond have i been pwned? – check if you have an account that has been compromised in a data breach.
small_orange_diamond dehashed – is a hacked database search engine.
small_orange_diamond Leaked Source – is a collaboration of data found online in the form of a lookup.

black_small_square CVE/Exploits databases

small_orange_diamond CVE Mitre – list of publicly known cybersecurity vulnerabilities.
small_orange_diamond CVE Details – CVE security vulnerability advanced database.
small_orange_diamond Exploit DB – CVE compliant archive of public exploits and corresponding vulnerable software.
small_orange_diamond 0day.today – exploits market provides you the possibility to buy zero-day exploits and also to sell 0day exploits.
small_orange_diamond sploitus – the exploit and tools database.
small_orange_diamond cxsecurity – free vulnerability database.
small_orange_diamond Vulncode-DB – is a database for vulnerabilities and their corresponding source code if available.
small_orange_diamond cveapi – free API for CVE data.

black_small_square Mobile apps scanners

small_orange_diamond ImmuniWeb® Mobile App Scanner – test security and privacy of mobile apps (iOS & Android).
small_orange_diamond Quixxi – free Mobile App Vulnerability Scanner for Android & iOS.
small_orange_diamond Ostorlab – analyzes mobile application to identify vulnerabilities and potential weaknesses.

black_small_square Private Search Engines

small_orange_diamond Startpage – the world’s most private search engine.
small_orange_diamond searX – a privacy-respecting, hackable metasearch engine.
small_orange_diamond darksearch – the 1st real Dark Web search engine.
small_orange_diamond Qwant – the search engine that respects your privacy.

black_small_square Secure Webmail Providers

small_orange_diamond CounterMail – is a secure and easy to use online email service, designed to provide maximum security and privacy.
small_orange_diamond Mail2Tor – is a Tor Hidden Service that allows anyone to send and receive emails anonymously.
small_orange_diamond Tutanota – is the world’s most secure email service and amazingly easy to use.
small_orange_diamond Protonmail – is the world’s largest secure email service, developed by CERN and MIT scientists.
small_orange_diamond Startmail – private & encrypted email made easy.

black_small_square Crypto

small_orange_diamond Keybase – it’s open source and powered by public-key cryptography.

black_small_square PGP Keyservers

small_orange_diamond SKS OpenPGP Key server – services for the SKS keyservers used by OpenPGP.

Systems/Services  [TOC]

black_small_square Operating Systems

small_orange_diamond Slackware – the most “Unix-like” Linux distribution.
small_orange_diamond OpenBSD – multi-platform 4.4BSD-based UNIX-like operating system.
small_orange_diamond HardenedBSD – HardenedBSD aims to implement innovative exploit mitigation and security solutions.
small_orange_diamond Kali Linux – Linux distribution used for Penetration Testing, Ethical Hacking and network security assessments.
small_orange_diamond Parrot Security OS – cyber security GNU/Linux environment.
small_orange_diamond Backbox Linux – penetration test and security assessment oriented Ubuntu-based Linux distribution.
small_orange_diamond BlackArch – is an Arch Linux-based penetration testing distribution for penetration testers and security researchers.
small_orange_diamond Pentoo – is a security-focused livecd based on Gentoo.
small_orange_diamond Security Onion – Linux distro for intrusion detection, enterprise security monitoring, and log management.
small_orange_diamond Tails – is a live system that aims to preserve your privacy and anonymity.
small_orange_diamond vedetta – OpenBSD router boilerplate.

black_small_square HTTP(s) Services

small_orange_diamond Varnish Cache – HTTP accelerator designed for content-heavy dynamic web sites.
small_orange_diamond Nginx – open source web and reverse proxy server that is similar to Apache, but very light weight.
small_orange_diamond OpenResty – is a dynamic web platform based on NGINX and LuaJIT.
small_orange_diamond Tengine – a distribution of Nginx with some advanced features.
small_orange_diamond Caddy Server – is an open source, HTTP/2-enabled web server with HTTPS by default.
small_orange_diamond HAProxy – the reliable, high performance TCP/HTTP load balancer.

black_small_square DNS Services

small_orange_diamond Unbound – validating, recursive, and caching DNS resolver (with TLS).
small_orange_diamond Knot Resolver – caching full resolver implementation, including both a resolver library and a daemon.
small_orange_diamond PowerDNS – is an open source authoritative DNS server, written in C++ and licensed under the GPL.

black_small_square Other Services

small_orange_diamond 3proxy – tiny free proxy server.

black_small_square Security/hardening

small_orange_diamond Emerald Onion – is a 501(c)(3) nonprofit organization and transit internet service provider (ISP) based in Seattle.
small_orange_diamond pi-hole – the Pi-hole® is a DNS sinkhole that protects your devices from unwanted content.
small_orange_diamond maltrail – malicious traffic detection system.
small_orange_diamond security_monkey – monitors AWS, GCP, OpenStack, and GitHub orgs for assets and their changes over time.
small_orange_diamond firecracker – secure and fast microVMs for serverless computing.
small_orange_diamond streisand – sets up a new server running your choice of WireGuard, OpenSSH, OpenVPN, Shadowsocks, and more.

Networks  [TOC]

black_small_square Tools

small_orange_diamond CapAnalysis – web visual tool to analyze large amounts of captured network traffic (PCAP analyzer).
small_orange_diamond netbox – IP address management (IPAM) and data center infrastructure management (DCIM) tool.

black_small_square Labs

small_orange_diamond NRE Labs – learn automation by doing it. Right now, right here, in your browser.

black_small_square Other

small_orange_diamond LBNL’s Network Research Group – home page of the Network Research Group (NRG); tools, talks, papers and more.

Containers/Orchestration  [TOC]

black_small_square CLI Tools

small_orange_diamond gvisor – container runtime sandbox.
small_orange_diamond ctop – top-like interface for container metrics.
small_orange_diamond docker-bench-security – is a script that checks for dozens of common best-practices around deploying Docker.

black_small_square Web Tools

small_orange_diamond Moby – a collaborative project for the container ecosystem to assemble container-based system.
small_orange_diamond Traefik – open source reverse proxy/load balancer provides easier integration with Docker and Let’s encrypt.
small_orange_diamond kong – The Cloud-Native API Gateway.
small_orange_diamond rancher – complete container management platform.
small_orange_diamond portainer – making Docker management easy.
small_orange_diamond nginx-proxy – automated nginx proxy for Docker containers using docker-gen.

black_small_square Manuals/Tutorials/Best Practices

small_orange_diamond docker-cheat-sheet – a quick reference cheat sheet on Docker.
small_orange_diamond awesome-docker – a curated list of Docker resources and projects.
small_orange_diamond docker_practice – learn and understand Docker technologies, with real DevOps practice!
small_orange_diamond labs – is a collection of tutorials for learning how to use Docker with various tools.
small_orange_diamond dockerfiles – various Dockerfiles I use on the desktop and on servers.
small_orange_diamond kubernetes-the-hard-way – bootstrap Kubernetes the hard way on Google Cloud Platform. No scripts.
small_orange_diamond kubernetes-the-easy-way – bootstrap Kubernetes the easy way on Google Cloud Platform. No scripts.
small_orange_diamond cheatsheet-kubernetes-A4 – Kubernetes CheatSheets in A4.
small_orange_diamond k8s-security – kubernetes security notes and best practices.
small_orange_diamond kubernetes-production-best-practices – checklists with best-practices for production-ready Kubernetes.
small_orange_diamond kubernetes-production-best-practices – kubernetes security – best practice guide.
small_orange_diamond kubernetes-failure-stories – is a compilation of public failure/horror stories related to Kubernetes.

Manuals/Howtos/Tutorials  [TOC]

black_small_square Shell/Command line

small_orange_diamond pure-bash-bible – is a collection of pure bash alternatives to external processes.
small_orange_diamond pure-sh-bible – is a collection of pure POSIX sh alternatives to external processes.
small_orange_diamond bash-guide – is a guide to learn bash.
small_orange_diamond bash-handbook – for those who wanna learn Bash.
small_orange_diamond The Bash Hackers Wiki – hold documentation of any kind about GNU Bash.
small_orange_diamond Shell & Utilities – describes the commands and utilities offered to application programs by POSIX-conformant systems.
small_orange_diamond the-art-of-command-line – master the command line, in one page.
small_orange_diamond Shell Style Guide – a shell style guide for Google-originated open-source projects.

black_small_square Text Editors

small_orange_diamond Vim Cheat Sheet – great multi language vim guide.

black_small_square Python

small_orange_diamond Awesome Python – a curated list of awesome Python frameworks, libraries, software and resources.
small_orange_diamond python-cheatsheet – comprehensive Python cheatsheet.
small_orange_diamond pythoncheatsheet.org – basic reference for beginner and advanced developers.

black_small_square Sed & Awk & Other

small_orange_diamond F’Awk Yeah! – advanced sed and awk usage (Parsing for Pentesters 3).

black_small_square *nix & Network

small_orange_diamond nixCraft – linux and unix tutorials for new and seasoned sysadmin.
small_orange_diamond TecMint – the ideal Linux blog for Sysadmins & Geeks.
small_orange_diamond Omnisecu – free Networking, System Administration and Security tutorials.
small_orange_diamond linux-cheat – Linux tutorials and cheatsheets. Minimal examples. Mostly user-land CLI utilities.
small_orange_diamond Unix Toolbox – collection of Unix/Linux/BSD commands and tasks which are useful for IT work or for advanced users.
small_orange_diamond htop explained – explanation of everything you can see in htop/top on Linux.
small_orange_diamond Linux Guide and Hints – tutorials on system administration in Fedora and CentOS.
small_orange_diamond strace-little-book – a little book which introduces strace.
small_orange_diamond http2-explained – a detailed document explaining and documenting HTTP/2.
small_orange_diamond http3-explained – a document describing the HTTP/3 and QUIC protocols.
small_orange_diamond HTTP/2 in Action – an excellent introduction to the new HTTP/2 standard.
small_orange_diamond Let’s code a TCP/IP stack – great stuff to learn network and system programming at a deeper level.
small_orange_diamond Nginx Admin’s Handbook – describes how to improve NGINX performance, security and other important things.
small_orange_diamond nginxconfig.io – NGINX config generator on steroids.
small_orange_diamond openssh guideline – is to help operational teams with the configuration of OpenSSH server and client.
small_orange_diamond SSH Handshake Explained – is a relatively brief description of the SSH handshake.
small_orange_diamond PacketLife.net – a place to record notes while studying for Cisco’s CCNP certification.

black_small_square Microsoft

small_orange_diamond AD-Attack-Defense – attack and defend active directory using modern post exploitation adversary tradecraft activity.

black_small_square Large-scale systems

small_orange_diamond The System Design Primer – learn how to design large-scale systems.
small_orange_diamond Awesome Scalability – best practices in building High Scalability, High Availability, High Stability and more.
small_orange_diamond Web Architecture 101 – the basic architecture concepts.

black_small_square System hardening

small_orange_diamond CIS Benchmarks – are secure configuration settings for over 100 technologies, available as a free PDF download.
small_orange_diamond Security Harden CentOS 7 – this walks you through the steps required to security harden CentOS.
small_orange_diamond CentOS 7 Server Hardening Guide – great guide for hardening CentOS; familiar with OpenSCAP.
small_orange_diamond awesome-security-hardening – is a collection of security hardening guides, tools and other resources.
small_orange_diamond The Practical Linux Hardening Guide – provides a high-level overview of hardening GNU/Linux systems.

black_small_square Security & Privacy

small_orange_diamond Hacking Articles – LRaj Chandel’s Security & Hacking Blog.
small_orange_diamond AWS security tools – make your AWS cloud environment more secure.
small_orange_diamond Rawsec’s CyberSecurity Inventory – an inventory of tools and resources about CyberSecurity.
small_orange_diamond The Illustrated TLS Connection – every byte of a TLS connection explained and reproduced.
small_orange_diamond SSL Research – SSL and TLS Deployment Best Practices by SSL Labs.
small_orange_diamond SELinux Game – learn SELinux by doing. Solve Puzzles, show skillz.
small_orange_diamond Certificates and PKI – everything you should know about certificates and PKI but are too afraid to ask.
small_orange_diamond The Art of Subdomain Enumeration – a reference for subdomain enumeration techniques.
small_orange_diamond Quitting Google – the comprehensive guide to quitting Google.

black_small_square Web Apps

small_orange_diamond OWASP – worldwide not-for-profit charitable organization focused on improving the security of software.
small_orange_diamond OWASP ASVS 3.0.1 – OWASP Application Security Verification Standard Project.
small_orange_diamond OWASP ASVS 3.0.1 Web App – simple web app that helps developers understand the ASVS requirements.
small_orange_diamond OWASP ASVS 4.0 – is a list of application security requirements or tests.
small_orange_diamond OWASP Testing Guide v4 – includes a “best practice” penetration testing framework.
small_orange_diamond OWASP Dev Guide – this is the development version of the OWASP Developer Guide.
small_orange_diamond OWASP API Security Project – focuses specifically on the top ten vulnerabilities in API security.
small_orange_diamond Mozilla Web Security – help operational teams with creating secure web applications.
small_orange_diamond security-bulletins – security bulletins that relate to Netflix Open Source.
small_orange_diamond API-Security-Checklist – security countermeasures when designing, testing, and releasing your API.
small_orange_diamond Enable CORS – enable cross-origin resource sharing.
small_orange_diamond Application Security Wiki – is an initiative to provide all application security related resources at one place.
small_orange_diamond Weird Proxies – reverse proxy related attacks; it is a result of analysis of various reverse proxies, cache proxies, etc.
small_orange_diamond Webshells – great series about malicious payloads.
small_orange_diamond Practical Web Cache Poisoning – show you how to compromise websites by using esoteric web features.
small_orange_diamond Hidden directories and files – as a source of sensitive information about web application.
small_orange_diamond Explosive blog – great blog about cybersec and pentests.
small_orange_diamond Security Cookies – this paper will take a close look at cookie security.
small_orange_diamond APISecurityBestPractices – help you keep secrets (API keys, db credentials, certificates) out of source code.

black_small_square All-in-one

small_orange_diamond LZone Cheat Sheets – all cheat sheets.
small_orange_diamond Dan’s Cheat Sheets’s – massive cheat sheets documentation.
small_orange_diamond Rico’s cheatsheets – this is a modest collection of cheatsheets.
small_orange_diamond DevDocs API – combines multiple API documentations in a fast, organized, and searchable interface.

black_small_square Other

small_orange_diamond CTF Series : Vulnerable Machines – the steps below could be followed to find vulnerabilities and exploits.
small_orange_diamond 50M_CTF_Writeup – $50 million CTF from Hackerone – writeup.
small_orange_diamond ctf-tasks – an archive of low-level CTF challenges developed over the years.
small_orange_diamond How to start RE/malware analysis? – collection of some hints and useful links for the beginners.
small_orange_diamond The C10K problem – it’s time for web servers to handle ten thousand clients simultaneously, don’t you think?
small_orange_diamond poor man’s profiler – sampling tools like dtrace’s don’t really provide methods to see what programs are blocking on.
small_orange_diamond HTTPS on Stack Overflow – this is the story of a long journey regarding the implementation of SSL.
small_orange_diamond Julia’s Drawings – some drawings about programming and unix world, zines about systems & debugging tools.
small_orange_diamond Hash collisions – this great repository is focused on hash collisions exploitation.
small_orange_diamond BGP Meets Cat – after 3072 hours of manipulating BGP, Job Snijders has succeeded in drawing a Nyancat.
small_orange_diamond bgp-battleships – playing battleships over BGP.
small_orange_diamond What happens when… – you type google.com into your browser and press enter?
small_orange_diamond how-web-works – based on the ‘What happens when…’ repository.
small_orange_diamond HTTPS in the real world – great tutorial explain how HTTPS works in the real world.
small_orange_diamond Gitlab and NFS bug – how we spent two weeks hunting an NFS bug in the Linux kernel.
small_orange_diamond Gitlab melts down – postmortem on the database outage of January 31 2017 with the lessons we learned.
small_orange_diamond How To Become A Hacker – if you want to be a hacker, keep reading.
small_orange_diamond Operation Costs in CPU – an infographics which should help to estimate costs of certain operations in CPU clocks.
small_orange_diamond Let’s Build a Simple Database – writing a sqlite clone from scratch in C.
small_orange_diamond simple-computer – great resource to understand how computers work under the hood.
small_orange_diamond The story of “Have I been pwned?” – working with 154 million records on Azure Table Storage.
small_orange_diamond TOP500 Supercomputers – shows the 500 most powerful commercially available computer systems known to us.
small_orange_diamond How to build a 8 GPU password cracker – any “black magic” or hours of frustration like desktop components do.
small_orange_diamond CERN Data Centre – 3D visualizations of the CERN computing environments (and more).
small_orange_diamond How fucked is my database – evaluate how fucked your database is with this handy website.
small_orange_diamond Five Whys – you know what the problem is, but you cannot solve it?
small_orange_diamond howhttps.works – how HTTPS works …in a comic!
small_orange_diamond howdns.works – a fun and colorful explanation of how DNS works.
small_orange_diamond POSTGRESQLCO.NF – your postgresql.conf documentation and recommendations.

Inspiring Lists  [TOC]

black_small_square SysOps/DevOps

small_orange_diamond Awesome Sysadmin – amazingly awesome open source sysadmin resources.
small_orange_diamond Awesome Shell – awesome command-line frameworks, toolkits, guides and gizmos.
small_orange_diamond Command-line-text-processing – from finding text to search and replace, from sorting to beautifying text and more.
small_orange_diamond Awesome Pcaptools – collection of tools developed by other researchers to process network traces.
small_orange_diamond awesome-ebpf – a curated list of awesome projects related to eBPF.
small_orange_diamond Linux Network Performance – learn where some of the network sysctl variables fit into the Linux/Kernel network flow.
small_orange_diamond Awesome Postgres – list of awesome PostgreSQL software, libraries, tools and resources.
small_orange_diamond quick-SQL-cheatsheet – a quick reminder of all SQL queries and examples on how to use them.
small_orange_diamond Awesome-Selfhosted – list of Free Software network services and web applications which can be hosted locally.
small_orange_diamond List of applications – huge collection of applications sorted by category, as a reference for those looking for packages.
small_orange_diamond CS-Interview-Knowledge-Map – build the best interview map.
small_orange_diamond DevOps-Guide – DevOps Guide from basic to advanced with Interview Questions and Notes.
small_orange_diamond devops-interview-questions – contains interview questions on various DevOps and SRE related topics.

black_small_square Developers

small_orange_diamond Web Developer Roadmap – roadmaps, articles and resources to help you choose your path, learn and improve.
small_orange_diamond Front-End-Checklist – the perfect Front-End Checklist for modern websites and meticulous developers.
small_orange_diamond Front-End-Performance-Checklist – the only Front-End Performance Checklist that runs faster than the others.
small_orange_diamond Python’s Magic Methods – what are magic methods? They’re everything in object-oriented Python.
small_orange_diamond wtfpython – a collection of surprising Python snippets and lesser-known features.
small_orange_diamond js-dev-reads – a list of books and articles for the discerning web developer to read.
small_orange_diamond Commit messages guide – a guide to understand the importance of commit messages.

black_small_square Security/Pentesting

small_orange_diamond Awesome Web Security – a curated list of Web Security materials and resources.
small_orange_diamond awesome-cyber-skills – a curated list of hacking environments where you can train your cyber skills.
small_orange_diamond awesome-devsecops – an authoritative list of awesome devsecops tools.
small_orange_diamond awesome-osint – is a curated list of amazingly awesome OSINT.
small_orange_diamond awesome-threat-intelligence – a curated list of Awesome Threat Intelligence resources.
small_orange_diamond Red-Teaming-Toolkit – a collection of open source and commercial tools that aid in red team operations.
small_orange_diamond awesome-burp-extensions – a curated list of amazingly awesome Burp Extensions.
small_orange_diamond Free Security eBooks – list of a Free Security and Hacking eBooks.
small_orange_diamond Hacking-Security-Ebooks – top 100 Hacking & Security E-Books.
small_orange_diamond privacy-respecting – curated list of privacy respecting services and software.
small_orange_diamond reverse-engineering – list of awesome reverse engineering resources.
small_orange_diamond linux-re-101 – a collection of resources for linux reverse engineering.
small_orange_diamond reverseengineering-reading-list – a list of Reverse Engineering articles, books, and papers.
small_orange_diamond Awesome-WAF – a curated list of awesome web-app firewall (WAF) stuff.
small_orange_diamond awesome-shodan-queries – interesting, funny, and depressing search queries to plug into shodan.io.
small_orange_diamond RobotsDisallowed – a curated list of the most common and most interesting robots.txt disallowed directories.
small_orange_diamond HackingNeuralNetworks – is a small course on exploiting and defending neural networks.
small_orange_diamond wildcard-certificates – why you probably shouldn’t use a wildcard certificate.
small_orange_diamond Don’t use VPN services – which is what every third-party “VPN provider” does.
small_orange_diamond awesome-yara – a curated list of awesome YARA rules, tools, and people.
small_orange_diamond macOS-Security-and-Privacy-Guide – guide to securing and improving privacy on macOS.
small_orange_diamond awesome-sec-talks – is a collected list of awesome security talks.
small_orange_diamond Movies for Hackers – list of movies every hacker & cyberpunk must watch.

black_small_square Other

small_orange_diamond Cheatography – over 3,000 free cheat sheets, revision aids and quick references.
small_orange_diamond awesome-static-analysis – static analysis tools for all programming languages.
small_orange_diamond computer-science – path to a free self-taught education in Computer Science.
small_orange_diamond post-mortems – is a collection of postmortems (config errors, hardware failures, and more).
small_orange_diamond build-your-own-x – build your own (insert technology here).
small_orange_diamond Project-Based-Tutorials-in-C – is a curated list of project-based tutorials in C.
small_orange_diamond The-Documentation-Compendium – various README templates & tips on writing high-quality documentation.
small_orange_diamond awesome-python-applications – free software that works great, and also happens to be open-source Python.
small_orange_diamond awesome-public-datasets – a topic-centric list of HQ open datasets.

Blogs/Podcasts/Videos  [TOC]

black_small_square SysOps/DevOps

small_orange_diamond Varnish for PHP developers – very interesting presentation of Varnish by Mattias Geniar.
small_orange_diamond A Netflix Guide to Microservices – Josh Evans talks about the chaotic and vibrant world of microservices at Netflix.

black_small_square Developers

small_orange_diamond Comparing C to machine language – compare a simple C program with the compiled machine code of that program.

black_small_square Geeky Persons

small_orange_diamond Brendan Gregg’s Blog – is an industry expert in computing performance and cloud computing.
small_orange_diamond Gynvael “GynDream” Coldwind – is a IT security engineer at Google.
small_orange_diamond Michał “lcamtuf” Zalewski – white hat hacker, computer security expert.
small_orange_diamond Mattias Geniar – developer, sysadmin, blogger, podcaster and public speaker.
small_orange_diamond Nick Craver – software developer and systems administrator for Stack Exchange.
small_orange_diamond Scott Helme – security researcher, international speaker and founder of securityheaders.com and report-uri.com.
small_orange_diamond Brian Krebs – The Washington Post and now an Independent investigative journalist.
small_orange_diamond Bruce Schneier – is an internationally renowned security technologist, called a “security guru”.
small_orange_diamond Chrissy Morgan – advocate of practical learning, Chrissy also takes part in bug bounty programs.
small_orange_diamond Andy Gill – is a hacker at heart who works as a senior penetration tester.
small_orange_diamond Daniel Miessler – cybersecurity expert and writer.
small_orange_diamond Samy Kamkar – is an American privacy and security researcher, computer hacker.
small_orange_diamond Javvad Malik – is a security advocate at AlienVault, a blogger event speaker and industry commentator.
small_orange_diamond Graham Cluley – public speaker and independent computer security analyst.
small_orange_diamond Kacper Szurek – detection engineer at ESET.
small_orange_diamond Troy Hunt – web security expert known for public education and outreach on security topics.
small_orange_diamond raymii.org – sysadmin specializing in building high availability cloud environments.
small_orange_diamond Robert Penz – IT security expert.

black_small_square Geeky Blogs

small_orange_diamond Linux Audit – the Linux security blog about auditing, hardening and compliance by Michael Boelen.
small_orange_diamond Linux Security Expert – trainings, howtos, checklists, security tools and more.
small_orange_diamond The Grymoire – collection of useful incantations for wizards, be you computer wizards, magicians, or whatever.
small_orange_diamond Secjuice – is the only non-profit, independent and volunteer led publication in the information security space.
small_orange_diamond Decipher – security news that informs and inspires.

black_small_square Geeky Vendor Blogs

small_orange_diamond Tenable Podcast – conversations and interviews related to Cyber Exposure, and more.
small_orange_diamond Sophos – threat news room, giving you news, opinion, advice and research on computer security issues.
small_orange_diamond Tripwire State of Security – blog featuring the latest news, trends and insights on current information security issues.
small_orange_diamond Malwarebytes Labs Blog – security blog aims to provide insider news about cybersecurity.
small_orange_diamond TrustedSec – latest news, and trends about cybersecurity.
small_orange_diamond PortSwigger Web Security Blog – about web app security vulns and top tips from our team of web security.
small_orange_diamond AT&T Cybersecurity blog – news on emerging threats and practical advice to simplify threat detection.
small_orange_diamond Thycotic – where CISOs and IT Admins come to learn about industry trends, IT security, data breaches, and more.

black_small_square Geeky Cybersecurity Podcasts

small_orange_diamond Risky Business – is a weekly information security podcast featuring news and in-depth interviews.
small_orange_diamond Cyber, by Motherboard – stories, and focus on the ideas about cybersecurity.
small_orange_diamond Tenable Podcast – conversations and interviews related to Cyber Exposure, and more.
small_orange_diamond Cybercrime Investigations – podcast by Geoff White about cybercrimes.
small_orange_diamond The many hats club – featuring stories from a wide range of Infosec people (Whitehat, Greyhat and Blackhat).
small_orange_diamond Darknet Diaries – true stories from the dark side of the Internet.
small_orange_diamond OSINTCurious Webcasts – is the investigative curiosity that helps people be successful in OSINT.
small_orange_diamond Security Weekly – the latest information security and hacking news.

black_small_square Geeky Cybersecurity Video Blogs

small_orange_diamond rev3rse security – offensive, binary exploitation, web application security, vulnerability, hardening, red team, blue team.
small_orange_diamond LiveOverflow – a lot more advanced topics than what is typically offered in paid online courses – but for free.
small_orange_diamond J4vv4D – the important information regarding our internet security.
small_orange_diamond CyberTalks – talks, interviews, and article about cybersecurity.

black_small_square Best Personal Twitter Accounts

small_orange_diamond @blackroomsec – a white-hat hacker/pentester. Intergalactic Minesweeper Champion 1990.
small_orange_diamond @MarcoCiappelli – Co-Founder @ITSPmagazine, at the intersection of IT security and society.
small_orange_diamond @binitamshah – Linux Evangelist. Malwares. Kernel Dev. Security Enthusiast.
small_orange_diamond @joe_carson – an InfoSec Professional and Tech Geek.
small_orange_diamond @mikko – CRO at F-Secure, Reverse Engineer, TED Speaker, Supervillain.
small_orange_diamond @esrtweet – often referred to as ESR, is an American software developer, and open-source software advocate.
small_orange_diamond @gynvael – security researcher/programmer, @DragonSectorCTF founder/player, technical streamer.
small_orange_diamond @x0rz – Security Researcher & Cyber Observer.
small_orange_diamond @hasherezade – programmer, malware analyst. Author of PEbear, PEsieve, libPeConv.
small_orange_diamond @TinkerSec – tinkerer, cypherpunk, hacker.
small_orange_diamond @alisaesage – independent hacker and researcher.
small_orange_diamond @SwiftOnSecurity – systems security, industrial safety, sysadmin, author of decentsecurity.com.
small_orange_diamond @dakami – chief scientist at White Ops, is one of just seven people with the authority to restore the DNS root keys.
small_orange_diamond @samykamkar – is a famous “grey hat” hacker, security researcher, creator of the MySpace “Samy” worm.
small_orange_diamond @securityweekly – founder & CTO of Security Weekly podcast network.
small_orange_diamond @jack_daniel – @SecurityBSides co-founder.
small_orange_diamond @thegrugq – Security Researcher.
small_orange_diamond @matthew_d_green – a cryptographer and professor at Johns Hopkins University.

black_small_square Best Commercial Twitter Accounts

small_orange_diamond @haveibeenpwned – check if you have an account that has been compromised in a data breach.
small_orange_diamond @bugcrowd – trusted by more of the Fortune 500 than any other crowdsourced security platform.
small_orange_diamond @Malwarebytes – most trusted security company. Unmatched threat visibility.
small_orange_diamond @sansforensics – the world’s leading Digital Forensics and Incident Response provider.
small_orange_diamond @attcyber – AT&T Cybersecurity’s Edge-to-Edge technologies provide threat intelligence, and more.
small_orange_diamond @TheManyHatsClub – an information security focused podcast and group of individuals from all walks of life.
small_orange_diamond @hedgehogsec – Hedgehog Cyber. Gibraltar and Manchester’s top boutique information security firm.
small_orange_diamond @NCSC – the National Cyber Security Centre. Helping to make the UK the safest place to live and work online.
small_orange_diamond @Synacktiv – IT security experts.

black_small_square A piece of history

small_orange_diamond How to Do Things at ARL – how to configure modems, scan images, record CD-ROMs, and other useful techniques.*

black_small_square Other

small_orange_diamond Diffie-Hellman Key Exchange (short version) – how Diffie-Hellman Key Exchange worked.

Hacking/Penetration Testing  [TOC]

black_small_square Pentesters arsenal tools

small_orange_diamond Sandcat Browser – a penetration-oriented browser with plenty of advanced functionality already built in.
small_orange_diamond Metasploit – tool and framework for pentesting system, web and many more, contains a lot a ready to use exploit.
small_orange_diamond Burp Suite – tool for testing web application security, intercepting proxy to replay, inject, scan and fuzz HTTP requests.
small_orange_diamond OWASP Zed Attack Proxy – intercepting proxy to replay, inject, scan and fuzz HTTP requests.
small_orange_diamond w3af – is a Web Application Attack and Audit Framework.
small_orange_diamond mitmproxy – an interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
small_orange_diamond Nikto2 – web server scanner which performs comprehensive tests against web servers for multiple items.
small_orange_diamond sqlmap – tool that automates the process of detecting and exploiting SQL injection flaws.
small_orange_diamond Recon-ng – is a full-featured Web Reconnaissance framework written in Python.
small_orange_diamond AutoRecon – is a network reconnaissance tool which performs automated enumeration of services.
small_orange_diamond Faraday – an Integrated Multiuser Pentest Environment.
small_orange_diamond Photon – incredibly fast crawler designed for OSINT.
small_orange_diamond XSStrike – most advanced XSS detection suite.
small_orange_diamond Sn1per – automated pentest framework for offensive security experts.
small_orange_diamond vuls – is an agent-less vulnerability scanner for Linux, FreeBSD, and other.
small_orange_diamond aquatone – a tool for domain flyovers.
small_orange_diamond BillCipher – information gathering tool for a website or IP address.
small_orange_diamond WhatWaf – detect and bypass web application firewalls and protection systems.
small_orange_diamond Corsy – CORS misconfiguration scanner.
small_orange_diamond Raccoon – is a high performance offensive security tool for reconnaissance and vulnerability scanning.
small_orange_diamond dirhunt – find web directories without bruteforce.
small_orange_diamond John The Ripper – is a fast password cracker, currently available for many flavors of Unix, Windows, and other.
small_orange_diamond hashcat – world’s fastest and most advanced password recovery utility.
small_orange_diamond p0f – is a tool to identify the players behind any incidental TCP/IP communications.
small_orange_diamond ssh_scan – a prototype SSH configuration and policy scanner.
small_orange_diamond LeakLooker – find open databases – powered by Binaryedge.io
small_orange_diamond exploitdb – searchable archive from The Exploit Database.
small_orange_diamond getsploit – is a command line utility for searching and downloading exploits.
small_orange_diamond ctf-tools – some setup scripts for security research tools.
small_orange_diamond pwntools – CTF framework and exploit development library.
small_orange_diamond security-tools – collection of small security tools created mostly in Python. CTFs, pentests and so on.
small_orange_diamond pentestpackage – is a package of Pentest scripts.
small_orange_diamond python-pentest-tools – python tools for penetration testers.
small_orange_diamond fuzzdb – dictionary of attack patterns and primitives for black-box application fault injection and resource discovery.
small_orange_diamond syzkaller – is an unsupervised, coverage-guided kernel fuzzer.
small_orange_diamond pwndbg – exploit development and reverse engineering with GDB made easy.
small_orange_diamond GDB PEDA – Python Exploit Development Assistance for GDB.
small_orange_diamond IDA – multi-processor disassembler and debugger useful for reverse engineering malware.
small_orange_diamond radare2 – framework for reverse-engineering and analyzing binaries.
small_orange_diamond routersploit – exploitation framework for embedded devices.
small_orange_diamond Ghidra – is a software reverse engineering (SRE) framework.
small_orange_diamond Vulnreport – open-source pentesting management and automation platform by Salesforce Product Security.
small_orange_diamond Mentalist – is a graphical tool for custom wordlist generation.
small_orange_diamond archerysec – vulnerability assessment and management helps to perform scans and manage vulnerabilities.
small_orange_diamond Osmedeus – fully automated offensive security tool for reconnaissance and vulnerability scanning.
small_orange_diamond beef – the browser exploitation framework project.
small_orange_diamond AutoSploit – automated mass exploiter.
small_orange_diamond SUDO_KILLER – is a tool to identify and exploit sudo rules’ misconfigurations and vulnerabilities.
small_orange_diamond yara – the pattern matching swiss knife.
small_orange_diamond mimikatz – a little tool to play with Windows security.

black_small_square Pentests bookmarks collection

small_orange_diamond PTES – the penetration testing execution standard.
small_orange_diamond Pentests MindMap – amazing mind map with vulnerable apps and systems.
small_orange_diamond WebApps Security Tests MindMap – incredible mind map for WebApps security tests.
small_orange_diamond Brute XSS – master the art of Cross Site Scripting.
small_orange_diamond XSS cheat sheet – contains many vectors that can help you bypass WAFs and filters.
small_orange_diamond Offensive Security Bookmarks – security bookmarks collection, all things that author need to pass OSCP.
small_orange_diamond Awesome Pentest Cheat Sheets – collection of the cheat sheets useful for pentesting.
small_orange_diamond Awesome Hacking by HackWithGithub – awesome lists for hackers, pentesters and security researchers.
small_orange_diamond Awesome Hacking by carpedm20 – a curated list of awesome hacking tutorials, tools and resources.
small_orange_diamond Awesome Hacking Resources – collection of hacking/penetration testing resources to make you better.
small_orange_diamond Awesome Pentest – collection of awesome penetration testing resources, tools and other shiny things.
small_orange_diamond Awesome-Hacking-Tools – is a curated list of awesome Hacking Tools.
small_orange_diamond Hacking Cheat Sheet – author hacking and pentesting notes.
small_orange_diamond blackhat-arsenal-tools – official Black Hat arsenal security tools repository.
small_orange_diamond Penetration Testing and WebApp Cheat Sheets – the complete list of Infosec related cheat sheets.
small_orange_diamond Cyber Security Resources – includes thousands of cybersecurity-related references and resources.
small_orange_diamond Pentest Bookmarks – there are a LOT of pentesting blogs.
small_orange_diamond Cheatsheet-God – Penetration Testing Reference Bank – OSCP/PTP & PTX Cheatsheet.
small_orange_diamond ThreatHunter-Playbook – to aid the development of techniques and hypothesis for hunting campaigns.
small_orange_diamond Beginner-Network-Pentesting – notes for beginner network pentesting course.
small_orange_diamond OSCPRepo – is a list of resources that author have been gathering in preparation for the OSCP.
small_orange_diamond PayloadsAllTheThings – a list of useful payloads and bypass for Web Application Security and Pentest/CTF.
small_orange_diamond payloads – git all the Payloads! A collection of web attack payloads.
small_orange_diamond command-injection-payload-list – command injection payload list.
small_orange_diamond AwesomeXSS – is a collection of Awesome XSS resources.
small_orange_diamond php-webshells – common php webshells.
small_orange_diamond Pentesting Tools Cheat Sheet – a quick reference high level overview for typical penetration testing engagements.
small_orange_diamond OWASP Cheat Sheet Series – is a collection of high value information on specific application security topics.
small_orange_diamond OWASP dependency-check – is an open source solution the OWASP Top 10 2013 entry.
small_orange_diamond OWASP ProActive Controls – OWASP Top 10 Proactive Controls 2018.
small_orange_diamond PENTESTING-BIBLE – hacking & penetration testing & red team & cyber security & computer science resources.
small_orange_diamond pentest-wiki – is a free online security knowledge library for pentesters/researchers.
small_orange_diamond DEF CON Media Server – great stuff from DEFCON.
small_orange_diamond Awesome Malware Analysis – a curated list of awesome malware analysis tools and resources.
small_orange_diamond SQL Injection Cheat Sheet – detailed technical information about the many different variants of the SQL Injection.
small_orange_diamond Entersoft Knowledge Base – great and detailed reference about vulnerabilities.
small_orange_diamond HTML5 Security Cheatsheet – a collection of HTML5 related XSS attack vectors.
small_orange_diamond XSS String Encoder – for generating XSS code to check your input validation filters against XSS.
small_orange_diamond GTFOBins – list of Unix binaries that can be exploited by an attacker to bypass local security restrictions.
small_orange_diamond Guifre Ruiz Notes – collection of security, system, network and pentest cheatsheets.
small_orange_diamond SSRF Tips – a collection of SSRF Tips.
small_orange_diamond shell-storm repo CTF – great archive of CTFs.
small_orange_diamond ctf – CTF (Capture The Flag) writeups, code snippets, notes, scripts.
small_orange_diamond My-CTF-Web-Challenges – collection of CTF Web challenges.
small_orange_diamond MSTG – The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security testing.
small_orange_diamond Internal-Pentest-Playbook – notes on the most common things for an Internal Network Penetration Test.
small_orange_diamond KeyHacks – shows quick ways in which API keys leaked by a bug bounty program can be checked.
small_orange_diamond securitum/research – various Proof of Concepts of security research performed by Securitum.
small_orange_diamond public-pentesting-reports – is a list of public penetration test reports released by several consulting security groups.
small_orange_diamond hackso.me – a great journey into security.

black_small_square Backdoors/exploits

small_orange_diamond PHP-backdoors – a collection of PHP backdoors. For educational or testing purposes only.

black_small_square Wordlists and Weak passwords

small_orange_diamond Weakpass – for any kind of bruteforce find wordlists or unleash the power of them all at once!
small_orange_diamond Hashes.org – is a free online hash resolving service incorporating many unparalleled techniques.
small_orange_diamond SecLists – collection of multiple types of lists used during security assessments, collected in one place.
small_orange_diamond Probable-Wordlists – sorted by probability originally created for password generation and testing.
small_orange_diamond skullsecurity passwords – password dictionaries and leaked passwords repository.
small_orange_diamond Polish PREMIUM Dictionary – official dictionary created by the team on the forum bezpieka.org.* 1
small_orange_diamond 
statistically-likely-usernames – wordlists for creating statistically likely username lists for use in password attacks.

black_small_square Bounty platforms

small_orange_diamond YesWeHack – bug bounty platform with infosec jobs.
small_orange_diamond Openbugbounty – allows any security researcher reporting a vulnerability on any website.
small_orange_diamond hackerone – global hacker community to surface the most relevant security issues.
small_orange_diamond bugcrowd – crowdsourced cybersecurity for the enterprise.
small_orange_diamond Crowdshield – crowdsourced security & bug bounty management.
small_orange_diamond Synack – crowdsourced security & bug bounty programs, crowd security intelligence platform and more.
small_orange_diamond Hacktrophy – bug bounty platform.

black_small_square Web Training Apps (local installation)

small_orange_diamond OWASP-VWAD – comprehensive and well maintained registry of all known vulnerable web applications.
small_orange_diamond DVWA – PHP/MySQL web application that is damn vulnerable.
small_orange_diamond metasploitable2 – vulnerable web application amongst security researchers.
small_orange_diamond metasploitable3 – is a VM that is built from the ground up with a large amount of security vulnerabilities.
small_orange_diamond DSVW – is a deliberately vulnerable web application written in under 100 lines of code.
small_orange_diamond OWASP Mutillidae II – free, open source, deliberately vulnerable web-application.
small_orange_diamond OWASP Juice Shop Project – the most bug-free vulnerable application in existence.
small_orange_diamond OWASP Node js Goat Project – OWASP Top 10 security risks apply to web applications developed using Node.js.
small_orange_diamond juicy-ctf – run Capture the Flags and Security Trainings with OWASP Juice Shop.
small_orange_diamond SecurityShepherd – web and mobile application security training platform.
small_orange_diamond Security Ninjas – open source application security training program.
small_orange_diamond hackazon – a modern vulnerable web app.
small_orange_diamond dvna – damn vulnerable NodeJS application.
small_orange_diamond django-DefectDojo – is an open-source application vulnerability correlation and security orchestration tool.
small_orange_diamond Google Gruyere – web application exploits and defenses.
small_orange_diamond Bodhi – is a playground focused on learning the exploitation of client-side web vulnerabilities.
small_orange_diamond Websploit – single vm lab with the purpose of combining several vulnerable appliations in one environment.
small_orange_diamond vulhub – pre-built Vulnerable Environments based on docker-compose.
small_orange_diamond CloudGoat 2 – the new & improved “Vulnerable by Design” AWS deployment tool.
small_orange_diamond secDevLabs – is a laboratory for learning secure web development in a practical manner.
small_orange_diamond CORS-vulnerable-Lab – sample vulnerable code and its exploit code.
small_orange_diamond RootTheBox – a Game of Hackers (CTF Scoreboard & Game Manager).

black_small_square Labs (ethical hacking platforms/trainings/CTFs)

small_orange_diamond Offensive Security – true performance-based penetration testing training for over a decade.
small_orange_diamond Hack The Box – online platform allowing you to test your penetration testing skills.
small_orange_diamond Hacking-Lab – online ethical hacking, computer network and security challenge platform.
small_orange_diamond pwnable.kr – non-commercial wargame site which provides various pwn challenges regarding system exploitation.
small_orange_diamond Pwnable.tw – is a wargame site for hackers to test and expand their binary exploiting skills.
small_orange_diamond picoCTF – is a free computer security game targeted at middle and high school students.
small_orange_diamond CTFlearn – is an online platform built to help ethical hackers learn and practice their cybersecurity knowledge and skills.
small_orange_diamond ctftime – CTF archive and a place, where you can get some another CTF-related info.
small_orange_diamond Silesia Security Lab – high quality security testing services.
small_orange_diamond Practical Pentest Labs – pentest lab, take your Hacking skills to the next level.
small_orange_diamond Root Me – the fast, easy, and affordable way to train your hacking skills.
small_orange_diamond rozwal.to – a great platform to train your pentesting skills.
small_orange_diamond TryHackMe – learning Cyber Security made easy.
small_orange_diamond hackxor – is a realistic web application hacking game, designed to help players of all abilities develop their skills.
small_orange_diamond Hack Yourself First – it’s full of nasty app sec holes.
small_orange_diamond OverTheWire – can help you to learn and practice security concepts in the form of fun-filled games.
small_orange_diamond Wizard Labs – is an online Penetration Testing Lab.
small_orange_diamond PentesterLab – provides vulnerable systems that can be used to test and understand vulnerabilities.
small_orange_diamond RingZer0 – tons of challenges designed to test and improve your hacking skills.
small_orange_diamond try2hack – several security-oriented challenges for your entertainment.
small_orange_diamond Ubeeri – preconfigured lab environments.
small_orange_diamond Pentestit – emulate IT infrastructures of real companies for legal pen testing and improving penetration testing skills.
small_orange_diamond Microcorruption – reversal challenges done in the web interface.
small_orange_diamond Crackmes – download crackmes to help improve your reverse engineering skills.
small_orange_diamond DomGoat – DOM XSS security learning and practicing platform.
small_orange_diamond Stereotyped Challenges – upgrade your web hacking techniques today!
small_orange_diamond Vulnhub – allows anyone to gain practical ‘hands-on’ experience in digital security.
small_orange_diamond W3Challs – is a penetration testing training platform, which offers various computer challenges.
small_orange_diamond RingZer0 CTF – offers you tons of challenges designed to test and improve your hacking skills.
small_orange_diamond Hack.me – a platform where you can build, host and share vulnerable web apps for educational and research purposes.
small_orange_diamond HackThis! – discover how hacks, dumps and defacements are performed and secure your website against hackers.
small_orange_diamond Enigma Group WebApp Training – these challenges cover the exploits listed in the OWASP Top 10 Project.
small_orange_diamond Reverse Engineering Challenges – challenges, exercises, problems and tasks – by level, by type, and more.
small_orange_diamond 0x00sec – the home of the Hacker – Malware, Reverse Engineering, and Computer Science.
small_orange_diamond We Chall – there are exist a lots of different challenge types.
small_orange_diamond Hacker Gateway – is the go-to place for hackers who want to test their skills.
small_orange_diamond Hacker101 – is a free class for web security.
small_orange_diamond contained.af – a stupid game for learning about containers, capabilities, and syscalls.
small_orange_diamond flAWS challenge! – a series of levels you’ll learn about common mistakes and gotchas when using AWS.
small_orange_diamond CyberSec WTF – provides web hacking challenges derived from bounty write-ups.
small_orange_diamond CTF Challenge – CTF Web App challenges.
small_orange_diamond gCTF – most of the challenges used in the Google CTF 2017.
small_orange_diamond Hack This Site – is a free, safe and legal training ground for hackers.
small_orange_diamond Attack & Defense – is a browser-based cloud labs.

black_small_square CTF platforms

small_orange_diamond fbctf – platform to host Capture the Flag competitions.
small_orange_diamond ctfscoreboard – scoreboard for Capture The Flag competitions.

black_small_square Other resources

small_orange_diamond Bugcrowd University – open source education content for the researcher community.
small_orange_diamond OSCPRepo – a list of resources and scripts that I have been gathering in preparation for the OSCP.
small_orange_diamond OWASP Top 10: Real-World Examples – test your web apps with real-world examples (two-part series).
small_orange_diamond phrack.org – an awesome collection of articles from several respected hackers and other thinkers.

Your daily knowledge and news  [TOC]

black_small_square RSS Readers

small_orange_diamond Feedly – organize, read and share what matters to you.
small_orange_diamond Inoreader – similar to feedly with a support for filtering what you fetch from rss.

black_small_square IRC Channels

small_orange_diamond #hackerspaces – hackerspace IRC channels.

black_small_square Security

small_orange_diamond The Hacker News – leading news source dedicated to promoting awareness for security experts and hackers.
small_orange_diamond Latest Hacking News – provides the latest hacking news, exploits and vulnerabilities for ethical hackers.
small_orange_diamond Security Newsletter – security news as a weekly digest (email notifications).
small_orange_diamond Google Online Security Blog – the latest news and insights from Google on security and safety on the Internet.
small_orange_diamond Qualys Blog – expert network security guidance and news.
small_orange_diamond DARKReading – connecting the Information Security Community.
small_orange_diamond Darknet – latest hacking tools, hacker news, cybersecurity best practices, ethical hacking & pen-testing.
small_orange_diamond publiclyDisclosed – public disclosure watcher who keeps you up to date about the recently disclosed bugs.
small_orange_diamond Reddit – Hacking – a subreddit dedicated to hacking and hackers.
small_orange_diamond Packet Storm – information security services, news, files, tools, exploits, advisories and whitepapers.
small_orange_diamond Sekurak – about security, penetration tests, vulnerabilities and many others (PL/EN).
small_orange_diamond nf.sec – basic aspects and mechanisms of Linux operating system security (PL).

black_small_square Other/All-in-one

small_orange_diamond Changelog – is a community of hackers; news & podcasts for developers and hackers.

Other Cheat Sheets  [TOC]

Build your own DNS Servers

small_orange_diamond Unbound DNS Tutorial – a validating, recursive, and caching DNS server.
small_orange_diamond Knot Resolver on Fedora – how to get faster and more secure DNS resolution with Knot Resolver on Fedora.
small_orange_diamond DNS-over-HTTPS – tutorial to setup your own DNS-over-HTTPS (DoH) server.
small_orange_diamond dns-over-https – a cartoon intro to DNS over HTTPS.
small_orange_diamond DNS-over-TLS – following to your DoH server, setup your DNS-over-TLS (DoT) server.
small_orange_diamond DNS Servers – how (and why) i run my own DNS Servers.

Build your own Certificate Authority

small_orange_diamond OpenSSL Certificate Authority – build your own certificate authority (CA) using the OpenSSL command-line tools.
small_orange_diamond step-ca Certificate Authority – build your own certificate authority (CA) using open source step-ca.

Build your own System/Virtual Machine

small_orange_diamond os-tutorial – how to create an OS from scratch.
small_orange_diamond Write your Own Virtual Machine – how to write your own virtual machine (VM).
small_orange_diamond x86 Bare Metal Examples – dozens of minimal operating systems to learn x86 system programming.
small_orange_diamond simple-computer – the scott CPU from “But How Do It Know?” by J. Clark Scott.

DNS Servers list (privacy)
IPURL
84.200.69.80dns.watch
94.247.43.254opennic.org
64.6.64.6verisign.com
89.233.43.71censurfridns.dk
1.1.1.1cloudflare.com
94.130.110.185dnsprivacy.at
TOP Browser extensions
Extension nameDescription
IPvFooDisplay the server IP address and HTTPS information across all page elements.
FoxyProxySimplifies configuring browsers to access proxy-servers.
HTTPS EverywhereAutomatically use HTTPS security on many sites.
uMatrixPoint & click to forbid/allow any class of requests made by your browser.
uBlock OriginAn efficient blocker: easy on memory and CPU footprint.
Session BuddyManage browser tabs and bookmarks with ease.
SuperSorterSort bookmarks recursively, delete duplicates, merge folders and more.
Clear CacheClear your cache and browsing data.
d3coderEncoding/Decoding plugin for various types of encoding.
Web DeveloperAdds a toolbar button with various web developer tools.
ThreatPinch LookupAdd threat intelligence hover tool tips.
TOP Burp extensions
Extension nameDescription
AutorizeAutomatically detects authorization enforcement.
ReflectionAn efficient blocker: easy on memory and CPU footprint.
Logger++Logs requests and responses for all Burp tools in a sortable table.
Bypass WAFAdds headers useful for bypassing some WAF devices.
JSON BeautifierBeautifies JSON content in the HTTP message viewer.
JSON Web TokensEnables Burp to decode and manipulate JSON web tokens.
CSP AuditorDisplays CSP headers for responses, and passively reports CSP weaknesses.
CSP-BypassPassively scans for CSP headers that contain known bypasses.
HackvertorConverts data using a tag-based configuration to apply various encoding.
Active Scan++Extends Burp’s active and passive scanning capabilities.
HTML5 AuditorScans for usage of risky HTML5 features.
Software Vulnerability ScannerSoftware vulnerability scanner based on Vulners.com audit API.
Hack Mozilla Firefox addressbar

In Firefox’s addressbar, you can limit results by typing special characters before or after your term:

  • ^ – for matches in your browsing history
  • * – for matches in your bookmarks.
  • % – for matches in your currently open tabs.
  • # – for matches in page titles.
  • @ – for matches in web addresses.
Bypass WAFs by Shortening IP Address (by 0xInfection)

IP addresses can be shortened by dropping the zeroes:

http://1.0.0.1 → http://1.1
http://127.0.0.1 → http://127.1
http://192.168.0.1 → http://192.168.1

http://0xC0A80001 or http://3232235521 → 192.168.0.1
http://192.168.257 → 192.168.1.1
http://192.168.516 → 192.168.2.4

This bypasses WAF filters for SSRF, open-redirect, etc where any IP as input gets blacklisted.

For more information please see How to Obscure Any URL and Magic IP Address Shortcuts.

One-liners  [TOC]

Table of Contents
Tool: terminal
Reload shell without exit
exec $SHELL -l
Close shell keeping all subprocess running
disown -a && exit
Exit without saving shell history
kill -9 $$
unset HISTFILE && exit
Perform a branching conditional
true && echo success
false || echo failed
Pipe stdout and stderr to separate commands
some_command > >(/bin/cmd_for_stdout) 2> >(/bin/cmd_for_stderr)
Redirect stdout and stderr each to separate files and print both to the screen
(some_command 2>&1 1>&3 | tee errorlog ) 3>&1 1>&2 | tee stdoutlog
List of commands you use most often
history | \
awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | \
grep -v "./" | \
column -c3 -s " " -t | \
sort -nr | nl |  head -n 20
Sterilize bash history
function sterile() {

  history | awk '$2 != "history" { $1=""; print $0 }' | egrep -vi "\
curl\b+.*(-E|--cert)\b+.*\b*|\
curl\b+.*--pass\b+.*\b*|\
curl\b+.*(-U|--proxy-user).*:.*\b*|\
curl\b+.*(-u|--user).*:.*\b*
.*(-H|--header).*(token|auth.*)\b+.*|\
wget\b+.*--.*password\b+.*\b*|\
http.?://.+:.+@.*\
" > $HOME/histbuff; history -r $HOME/histbuff;

}

export PROMPT_COMMAND="sterile"

Look also: A naive utility to censor credentials in command history.

Quickly backup a file
cp filename{,.orig}
Empty a file (truncate to 0 size)
>filename
Delete all files in a folder that don’t match a certain file extension
rm !(*.foo|*.bar|*.baz)
Pass multi-line string to a file
# cat  >filename ... - overwrite the file
# cat >>filename ... - append to a file
cat > filename << __EOF__
data data data
__EOF__
Edit a file on a remote host using vim
vim scp://user@host//etc/fstab
Create a directory and change into it at the same time
mkd() { mkdir -p "$@" && cd "$@"; }
Convert uppercase files to lowercase files
rename 'y/A-Z/a-z/' *
Print a row of characters across the terminal
printf "%`tput cols`s" | tr ' ' '#'
Show shell history without line numbers
history | cut -c 8-
fc -l -n 1 | sed 's/^\s*//'
Run command(s) after exit session
cat > /etc/profile << __EOF__
_after_logout() {

  username=$(whoami)

  for _pid in $(ps afx | grep sshd | grep "$username" | awk '{print $1}') ; do

    kill -9 $_pid

  done

}
trap _after_logout EXIT
__EOF__
Generate a sequence of numbers
for ((i=1; i<=10; i+=2)) ; do echo $i ; done
# alternative: seq 1 2 10

for ((i=5; i<=10; ++i)) ; do printf '%02d\n' $i ; done
# alternative: seq -w 5 10

for i in {1..10} ; do echo $i ; done
Simple Bash filewatching
unset MAIL; export MAILCHECK=1; export MAILPATH='$FILE_TO_WATCH?$MESSAGE'

Tool: busybox
Static HTTP web server
busybox httpd -p $PORT -h $HOME [-c httpd.conf]

Tool: mount
Mount a temporary ram partition
mount -t tmpfs tmpfs /mnt -o size=64M
  • -t – filesystem type
  • -o – mount options
Remount a filesystem as read/write
mount -o remount,rw /

Tool: fuser
Show which processes use the files/directories
fuser /var/log/daemon.log
fuser -v /home/supervisor
Kills a process that is locking a file
fuser -ki filename
  • -i – interactive option
Kills a process that is locking a file with specific signal
fuser -k -HUP filename
  • --list-signals – list available signal names
Show what PID is listening on specific port
fuser -v 53/udp
Show all processes using the named filesystems or block device
fuser -mv /var/www

Tool: lsof
Show process that use internet connection at the moment
lsof -P -i -n
Show process that use specific port number
lsof -i tcp:443
Lists all listening ports together with the PID of the associated process
lsof -Pan -i tcp -i udp
List all open ports and their owning executables
lsof -i -P | grep -i "listen"
Show all open ports
lsof -Pnl -i
Show open ports (LISTEN)
lsof -Pni4 | grep LISTEN | column -t
List all files opened by a particular command
lsof -c "process"
View user activity per directory
lsof -u username -a +D /etc
Show 10 largest open files
lsof / | \
awk '{ if($7 > 1048576) print $7/1048576 "MB" " " $9 " " $1 }' | \
sort -n -u | tail | column -t
Show current working directory of a process
lsof -p <PID> | grep cwd

Tool: ps
Show a 4-way scrollable process tree with full details
ps awwfux | less -S
Processes per user counter
ps hax -o user | sort | uniq -c | sort -r
Show all processes by name with main header
ps -lfC nginx

Tool: find
Find files that have been modified on your system in the past 60 minutes
find / -mmin 60 -type f
Find all files larger than 20M
find / -type f -size +20M
Find duplicate files (based on MD5 hash)
find -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 33
Change permission only for files
cd /var/www/site && find . -type f -exec chmod 766 {} \;
cd /var/www/site && find . -type f -exec chmod 664 {} +
Change permission only for directories
cd /var/www/site && find . -type d -exec chmod g+x {} \;
cd /var/www/site && find . -type d -exec chmod g+rwx {} +
Find files and directories for specific user
find . -user <username> -print
Find files and directories for all without specific user
find . \!-user <username> -print
Delete older files than 60 days
find . -type f -mtime +60 -delete
Recursively remove all empty sub-directories from a directory
find . -depth  -type d  -empty -exec rmdir {} \;
How to find all hard links to a file
find </path/to/dir> -xdev -samefile filename
Recursively find the latest modified files
find . -type f -exec stat --format '%Y :%y %n' "{}" \; | sort -nr | cut -d: -f2- | head
Recursively find/replace of a string with sed
find . -not -path '*/\.git*' -type f -print0 | xargs -0 sed -i 's/foo/bar/g'
Recursively find/replace of a string in directories and file names
find . -depth -name '*test*' -execdir bash -c 'mv -v "$1" "${1//foo/bar}"' _ {} \;
Recursively find suid executables
find / \( -perm -4000 -o -perm -2000 \) -type f -exec ls -la {} \;

Tool: top
Use top to monitor only all processes with the specific string
top -p $(pgrep -d , <str>)
  • <str> – process containing string (eg. nginx, worker)

Tool: vmstat
Show current system utilization (fields in kilobytes)
vmstat 2 20 -t -w
  • 2 – number of times with a defined time interval (delay)
  • 20 – each execution of the command (count)
  • -t – show timestamp
  • -w – wide output
  • -S M – output of the fields in megabytes instead of kilobytes
Show current system utilization will get refreshed every 5 seconds
vmstat 5 -w
Display report a summary of disk operations
vmstat -D
Display report of event counters and memory stats
vmstat -s
Display report about kernel objects stored in slab layer cache
vmstat -m
Tool: iostat
Show information about the CPU usage, and I/O statistics about all the partitions
iostat 2 10 -t -m
  • 2 – number of times with a defined time interval (delay)
  • 10 – each execution of the command (count)
  • -t – show timestamp
  • -m – fields in megabytes (-k – in kilobytes, default)
Show information only about the CPU utilization
iostat 2 10 -t -m -c
Show information only about the disk utilization
iostat 2 10 -t -m -d
Show information only about the LVM utilization
iostat -N

Tool: strace
Track with child processes
# 1)
strace -f -p $(pidof glusterfsd)

# 2)
strace -f $(pidof php-fpm | sed 's/\([0-9]*\)/\-p \1/g')
Track process with 30 seconds limit
timeout 30 strace $(< /var/run/zabbix/zabbix_agentd.pid)
Track processes and redirect output to a file
ps auxw | grep '[a]pache' | awk '{print " -p " $2}' | \
xargs strace -o /tmp/strace-apache-proc.out
Track with print time spent in each syscall and limit length of print strings
ps auxw | grep '[i]init_policy' | awk '{print " -p " $2}' | \
xargs strace -f -e trace=network -T -s 10000
Track the open request of a network port
strace -f -e trace=bind nc -l 80
Track the open request of a network port (show TCP/UDP)
strace -f -e trace=network nc -lu 80

Tool: kill
Kill a process running on port
kill -9 $(lsof -i :<port> | awk '{l=$2} END {print l}')

Tool: diff
Compare two directory trees
diff <(cd directory1 && find | sort) <(cd directory2 && find | sort)
Compare output of two commands
diff <(cat /etc/passwd) <(cut -f2 /etc/passwd)

Tool: vimdiff
Highlight the exact differences, based on characters and words
vimdiff file1 file2
Compare two JSON files
vimdiff <(jq -S . A.json) <(jq -S . B.json)
Compare Hex dump
d(){ vimdiff <(f $1) <(f $2);};f(){ hexdump -C $1|cut -d' ' -f3-|tr -s ' ';}; d ~/bin1 ~/bin2
diffchar

Save diffchar @ ~/.vim/plugins

Click F7 to switch between diff modes

Usefull vimdiff commands:

  • qa to exit all windows
  • :vertical resize 70 to resize window
  • set window width Ctrl+W [N columns]+(Shift+)<\>

Tool: tail
Annotate tail -f with timestamps
tail -f file | while read ; do echo "$(date +%T.%N) $REPLY" ; done
Analyse an Apache access log for the most common IP addresses
tail -10000 access_log | awk '{print $1}' | sort | uniq -c | sort -n | tail
Analyse web server log and show only 5xx http codes
tail -n 100 -f /path/to/logfile | grep "HTTP/[1-2].[0-1]\" [5]"

Tool: tar
System backup with exclude specific directories
cd /
tar -czvpf /mnt/system$(date +%d%m%Y%s).tgz --directory=/ \
--exclude=proc/* --exclude=sys/* --exclude=dev/* --exclude=mnt/* .
System backup with exclude specific directories (pigz)
cd /
tar cvpf /backup/snapshot-$(date +%d%m%Y%s).tgz --directory=/ \
--exclude=proc/* --exclude=sys/* --exclude=dev/* \
--exclude=mnt/* --exclude=tmp/* --use-compress-program=pigz .

Tool: dump
System backup to file
dump -y -u -f /backup/system$(date +%d%m%Y%s).lzo /
Restore system from lzo file
cd /
restore -rf /backup/system$(date +%d%m%Y%s).lzo

Tool: cpulimit
Limit the cpu usage of a process
cpulimit -p pid -l 50

Tool: pwdx
Show current working directory of a process
pwdx <pid>

Tool: taskset
Start a command on only one CPU core
taskset -c 0 <command>

Tool: tr
Show directories in the PATH, one per line
tr : '\n' <<<$PATH

Tool: chmod
Remove executable bit from all files in the current directory
chmod -R -x+X *
Restore permission for /bin/chmod
# 1:
cp /bin/ls chmod.01
cp /bin/chmod chmod.01
./chmod.01 700 file

# 2:
/bin/busybox chmod 0700 /bin/chmod

# 3:
setfacl --set u::rwx,g::---,o::--- /bin/chmod

Tool: who
Find last reboot time
who -b
Detect a user sudo-su’d into the current shell
[[ $(who -m | awk '{ print $1 }') == $(whoami) ]] || echo "You are su-ed to $(whoami)"

Tool: last
Was the last reboot a panic?
(last -x -f $(ls -1t /var/log/wtmp* | head -2 | tail -1); last -x -f /var/log/wtmp) | \
grep -A1 reboot | head -2 | grep -q shutdown && echo "Expected reboot" || echo "Panic reboot"

Tool: screen
Start screen in detached mode
screen -d -m <command>
Attach to an existing screen session
screen -r -d <pid>

Tool: script
Record and replay terminal session
### Record session
# 1)
script -t 2>~/session.time -a ~/session.log

# 2)
script --timing=session.time session.log

### Replay session
scriptreplay --timing=session.time session.log

Tool: du
Show 20 biggest directories with ‘K M G’
du | \
sort -r -n | \
awk '{split("K M G",v); s=1; while($1>1024){$1/=1024; s++} print int($1)" "v[s]"\t"$2}' | \
head -n 20

Tool: inotifywait
Init tool everytime a file in a directory is modified
while true ; do inotifywait -r -e MODIFY dir/ && ls dir/ ; done;

Tool: openssl
Testing connection to the remote host
echo | openssl s_client -connect google.com:443 -showcerts
Testing connection to the remote host (debug mode)
echo | openssl s_client -connect google.com:443 -showcerts -tlsextdebug -status
Testing connection to the remote host (with SNI support)
echo | openssl s_client -showcerts -servername google.com -connect google.com:443
Testing connection to the remote host with specific ssl version
openssl s_client -tls1_2 -connect google.com:443
Testing connection to the remote host with specific ssl cipher
openssl s_client -cipher 'AES128-SHA' -connect google.com:443
Verify 0-RTT
_host="example.com"

cat > req.in << __EOF__
HEAD / HTTP/1.1
Host: $_host
Connection: close
__EOF__

openssl s_client -connect ${_host}:443 -tls1_3 -sess_out session.pem -ign_eof < req.in
openssl s_client -connect ${_host}:443 -tls1_3 -sess_in session.pem -early_data req.in
Generate private key without passphrase
# _len: 2048, 4096
( _fd="private.key" ; _len="4096" ; \
openssl genrsa -out ${_fd} ${_len} )
Generate private key with passphrase
# _ciph: des3, aes128, aes256
# _len: 2048, 4096
( _ciph="aes128" ; _fd="private.key" ; _len="4096" ; \
openssl genrsa -${_ciph} -out ${_fd} ${_len} )
Remove passphrase from private key
( _fd="private.key" ; _fd_unp="private_unp.key" ; \
openssl rsa -in ${_fd} -out ${_fd_unp} )
Encrypt existing private key with a passphrase
# _ciph: des3, aes128, aes256
( _ciph="aes128" ; _fd="private.key" ; _fd_pass="private_pass.key" ; \
openssl rsa -${_ciph} -in ${_fd} -out ${_fd_pass}
Check private key
( _fd="private.key" ; \
openssl rsa -check -in ${_fd} )
Get public key from private key
( _fd="private.key" ; _fd_pub="public.key" ; \
openssl rsa -pubout -in ${_fd} -out ${_fd_pub} )
Generate private key and CSR
( _fd="private.key" ; _fd_csr="request.csr" ; _len="4096" ; \
openssl req -out ${_fd_csr} -new -newkey rsa:${_len} -nodes -keyout ${_fd} )
Generate CSR
( _fd="private.key" ; _fd_csr="request.csr" ; \
openssl req -out ${_fd_csr} -new -key ${_fd} )
Generate CSR (metadata from existing certificate)

Where private.key is the existing private key. As you can see you do not generate this CSR from your certificate (public key). Also you do not generate the “same” CSR, just a new one to request a new certificate.

( _fd="private.key" ; _fd_csr="request.csr" ; _fd_crt="cert.crt" ; \
openssl x509 -x509toreq -in ${_fd_crt} -out ${_fd_csr} -signkey ${_fd} )
Generate CSR with -config param
( _fd="private.key" ; _fd_csr="request.csr" ; \
openssl req -new -sha256 -key ${_fd} -out ${_fd_csr} \
-config <(
cat << __EOF__
[req]
default_bits        = 2048
default_md          = sha256
prompt              = no
distinguished_name  = dn
req_extensions      = req_ext

[ dn ]
C   = "<two-letter ISO abbreviation for your country>"
ST  = "<state or province where your organisation is legally located>"
L   = "<city where your organisation is legally located>"
O   = "<legal name of your organisation>"
OU  = "<section of the organisation>"
CN  = "<fully qualified domain name>"

[ req_ext ]
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = <fully qualified domain name>
DNS.2 = <next domain>
DNS.3 = <next domain>
__EOF__
))

Other values in [ dn ]:

Look at this great explanation: How to create multidomain certificates using config files

countryName            = "DE"                     # C=
stateOrProvinceName    = "Hessen"                 # ST=
localityName           = "Keller"                 # L=
postalCode             = "424242"                 # L/postalcode=
streetAddress          = "Crater 1621"            # L/street=
organizationName       = "apfelboymschule"        # O=
organizationalUnitName = "IT Department"          # OU=
commonName             = "example.com"            # CN=
emailAddress           = "webmaster@example.com"  # CN/emailAddress=
List available EC curves
openssl ecparam -list_curves
Generate ECDSA private key
# _curve: prime256v1, secp521r1, secp384r1
( _fd="private.key" ; _curve="prime256v1" ; \
openssl ecparam -out ${_fd} -name ${_curve} -genkey )

# _curve: X25519
( _fd="private.key" ; _curve="x25519" ; \
openssl genpkey -algorithm ${_curve} -out ${_fd} )
Print ECDSA private and public keys
( _fd="private.key" ; \
openssl ec -in ${_fd} -noout -text )

# For x25519 only extracting public key
( _fd="private.key" ; _fd_pub="public.key" ; \
openssl pkey -in ${_fd} -pubout -out ${_fd_pub} )
Generate private key with CSR (ECC)
# _curve: prime256v1, secp521r1, secp384r1
( _fd="domain.com.key" ; _fd_csr="domain.com.csr" ; _curve="prime256v1" ; \
openssl ecparam -out ${_fd} -name ${_curve} -genkey ; \
openssl req -new -key ${_fd} -out ${_fd_csr} -sha256 )
Generate self-signed certificate
# _len: 2048, 4096
( _fd="domain.key" ; _fd_out="domain.crt" ; _len="4096" ; _days="365" ; \
openssl req -newkey rsa:${_len} -nodes \
-keyout ${_fd} -x509 -days ${_days} -out ${_fd_out} )
Generate self-signed certificate from existing private key
# _len: 2048, 4096
( _fd="domain.key" ; _fd_out="domain.crt" ; _days="365" ; \
openssl req -key ${_fd} -nodes \
-x509 -days ${_days} -out ${_fd_out} )
Generate self-signed certificate from existing private key and csr
# _len: 2048, 4096
( _fd="domain.key" ; _fd_csr="domain.csr" ; _fd_out="domain.crt" ; _days="365" ; \
openssl x509 -signkey ${_fd} -nodes \
-in ${_fd_csr} -req -days ${_days} -out ${_fd_out} )
Generate DH public parameters
( _dh_size="2048" ; \
openssl dhparam -out /etc/nginx/ssl/dhparam_${_dh_size}.pem "$_dh_size" )
Display DH public parameters
openssl pkeyparam -in dhparam.pem -text
Extract private key from pfx
( _fd_pfx="cert.pfx" ; _fd_key="key.pem" ; \
openssl pkcs12 -in ${_fd_pfx} -nocerts -nodes -out ${_fd_key} )
Extract private key and certs from pfx
( _fd_pfx="cert.pfx" ; _fd_pem="key_certs.pem" ; \
openssl pkcs12 -in ${_fd_pfx} -nodes -out ${_fd_pem} )
Convert DER to PEM
( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \
openssl x509 -in ${_fd_der} -inform der -outform pem -out ${_fd_pem} )
Convert PEM to DER
( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \
openssl x509 -in ${_fd_pem} -outform der -out ${_fd_der} )
Verification of the private key
( _fd="private.key" ; \
openssl rsa -noout -text -in ${_fd} )
Verification of the public key
# 1)
( _fd="public.key" ; \
openssl pkey -noout -text -pubin -in ${_fd} )

# 2)
( _fd="private.key" ; \
openssl rsa -inform PEM -noout -in ${_fd} &> /dev/null ; \
if [ $? = 0 ] ; then echo -en "OK\n" ; fi )
Verification of the certificate
( _fd="certificate.crt" ; # format: pem, cer, crt \
openssl x509 -noout -text -in ${_fd} )
Verification of the CSR
( _fd_csr="request.csr" ; \
openssl req -text -noout -in ${_fd_csr} )
Check whether the private key and the certificate match
(openssl rsa -noout -modulus -in private.key | openssl md5 ; \
openssl x509 -noout -modulus -in certificate.crt | openssl md5) | uniq
Check whether the private key and the CSR match
(openssl rsa -noout -modulus -in private.key | openssl md5 ; \
openssl req -noout -modulus -in request.csr | openssl md5) | uniq

Tool: secure-delete
Secure delete with shred
shred -vfuz -n 10 file
shred --verbose --random-source=/dev/urandom -n 1 /dev/sda
Secure delete with scrub
scrub -p dod /dev/sda
scrub -p dod -r file
Secure delete with badblocks
badblocks -s -w -t random -v /dev/sda
badblocks -c 10240 -s -w -t random -v /dev/sda
Secure delete with secure-delete
srm -vz /tmp/file
sfill -vz /local
sdmem -v
swapoff /dev/sda5 && sswap -vz /dev/sda5

Tool: dd
Show dd status every so often
dd <dd_params> status=progress
watch --interval 5 killall -USR1 dd
Redirect output to a file with dd
echo "string" | dd of=filename

Tool: gpg
Export public key
gpg --export --armor "<username>" > username.pkey
  • --export – export all keys from all keyrings or specific key
  • -a|--armor – create ASCII armored output
Encrypt file
gpg -e -r "<username>" dump.sql
  • -e|--encrypt – encrypt data
  • -r|--recipient – encrypt for specific
Decrypt file
gpg -o dump.sql -d dump.sql.gpg
  • -o|--output – use as output file
  • -d|--decrypt – decrypt data (default)
Search recipient
gpg --keyserver hkp://keyserver.ubuntu.com --search-keys "<username>"
  • --keyserver – set specific key server
  • --search-keys – search for keys on a key server
List all of the packets in an encrypted file
gpg --batch --list-packets archive.gpg
gpg2 --batch --list-packets archive.gpg

Tool: system-other
Reboot system from init
exec /sbin/init 6
Init system from single user mode
exec /sbin/init
Show current working directory of a process
readlink -f /proc/<PID>/cwd
Show actual pathname of the executed command
readlink -f /proc/<PID>/exe
Tool: curl
curl -Iks https://www.google.com
  • -I – show response headers only
  • -k – insecure connection when using ssl
  • -s – silent mode (not display body)
curl -Iks --location -X GET -A "x-agent" https://www.google.com
  • --location – follow redirects
  • -X – set method
  • -A – set user-agent
curl -Iks --location -X GET -A "x-agent" --proxy http://127.0.0.1:16379 https://www.google.com
  • --proxy [socks5://|http://] – set proxy server
curl -o file.pdf -C - https://example.com/Aiju2goo0Ja2.pdf
  • -o – write output to file
  • -C – resume the transfer
Find your external IP address (external services)
curl ipinfo.io
curl ipinfo.io/ip
curl icanhazip.com
curl ifconfig.me/ip ; echo
Repeat URL request
# URL sequence substitution with a dummy query string:
curl -ks https://example.com/?[1-20]

# With shell 'for' loop:
for i in {1..20} ; do curl -ks https://example.com/ ; done
Check DNS and HTTP trace with headers for specific domains
### Set domains and external dns servers.
_domain_list=(google.com) ; _dns_list=("8.8.8.8" "1.1.1.1")

for _domain in "${_domain_list[@]}" ; do

  printf '=%.0s' {1..48}

  echo

  printf "[\\e[1;32m+\\e[m] resolve: %s\\n" "$_domain"

  for _dns in "${_dns_list[@]}" ; do

    # Resolve domain.
    host "${_domain}" "${_dns}"

    echo

  done

  for _proto in http https ; do

    printf "[\\e[1;32m+\\e[m] trace + headers: %s://%s\\n" "$_proto" "$_domain"

    # Get trace and http headers.
    curl -Iks -A "x-agent" --location "${_proto}://${_domain}"

    echo

  done

done

unset _domain_list _dns_list

Tool: httpie
http -p Hh https://www.google.com
  • -p – print request and response headers
    • H – request headers
    • B – request body
    • h – response headers
    • b – response body
http -p Hh https://www.google.com --follow --verify no
  • -F, --follow – follow redirects
  • --verify no – skip SSL verification
http -p Hh https://www.google.com --follow --verify no \
--proxy http:http://127.0.0.1:16379
  • --proxy [http:] – set proxy server
Tool: ssh
Escape Sequence
# Supported escape sequences:
~.  - terminate connection (and any multiplexed sessions)
~B  - send a BREAK to the remote system
~C  - open a command line
~R  - Request rekey (SSH protocol 2 only)
~^Z - suspend ssh
~#  - list forwarded connections
~&  - background ssh (when waiting for connections to terminate)
~?  - this message
~~  - send the escape character by typing it twice
Compare a remote file with a local file
ssh user@host cat /path/to/remotefile | diff /path/to/localfile -
SSH connection through host in the middle
ssh -t reachable_host ssh unreachable_host
Run command over SSH on remote host
cat > cmd.txt << __EOF__
cat /etc/hosts
__EOF__

ssh host -l user $(<cmd.txt)
Get public key from private key
ssh-keygen -y -f ~/.ssh/id_rsa
Get all fingerprints
ssh-keygen -l -f .ssh/known_hosts
SSH authentication with user password
ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@remote_host
SSH authentication with publickey
ssh -o PreferredAuthentications=publickey -o PubkeyAuthentication=yes -i id_rsa user@remote_host
Simple recording SSH session
function _ssh_sesslog() {

  _sesdir="<path/to/session/logs>"

  mkdir -p "${_sesdir}" && \
  ssh $@ 2>&1 | tee -a "${_sesdir}/$(date +%Y%m%d).log"

}

# Alias:
alias ssh='_ssh_sesslog'
Using Keychain for SSH logins
### Delete all of ssh-agent's keys.
function _scl() {

  /usr/bin/keychain --clear

}

### Add key to keychain.
function _scg() {

  /usr/bin/keychain /path/to/private-key
  source "$HOME/.keychain/$HOSTNAME-sh"

}
SSH login without processing any login scripts
ssh -tt user@host bash
SSH local port forwarding

Example 1:

# Forwarding our local 2250 port to nmap.org:443 from localhost through localhost
host1> ssh -L 2250:nmap.org:443 localhost

# Connect to the service:
host1> curl -Iks --location -X GET https://localhost:2250

Example 2:

# Forwarding our local 9051 port to db.d.x:5432 from localhost through node.d.y
host1> ssh -nNT -L 9051:db.d.x:5432 node.d.y

# Connect to the service:
host1> psql -U db_user -d db_dev -p 9051 -h localhost
  • -n – redirects stdin from /dev/null
  • -N – do not execute a remote command
  • -T – disable pseudo-terminal allocation
SSH remote port forwarding
# Forwarding our local 9051 port to db.d.x:5432 from host2 through node.d.y
host1> ssh -nNT -R 9051:db.d.x:5432 node.d.y

# Connect to the service:
host2> psql -U postgres -d postgres -p 8000 -h localhost

Tool: linux-dev
Testing remote connection to port
timeout 1 bash -c "</dev/<proto>/<host>/<port>" >/dev/null 2>&1 ; echo $?
  • <proto – set protocol (tcp/udp)
  • <host> – set remote host
  • <port> – set destination port
Read and write to TCP or UDP sockets with common bash tools
exec 5<>/dev/tcp/<host>/<port>; cat <&5 & cat >&5; exec 5>&-

Tool: tcpdump
Filter incoming (on interface) traffic (specific ip:port)
tcpdump -ne -i eth0 -Q in host 192.168.252.1 and port 443
  • -n – don’t convert addresses (-nn will not resolve hostnames or ports)
  • -e – print the link-level headers
  • -i [iface|any] – set interface
  • -Q|-D [in|out|inout] – choose send/receive direction (-D – for old tcpdump versions)
  • host [ip|hostname] – set host, also [host not]
  • [and|or] – set logic
  • port [1-65535] – set port number, also [port not]
Filter incoming (on interface) traffic (specific ip:port) and write to a file
tcpdump -ne -i eth0 -Q in host 192.168.252.1 and port 443 -c 5 -w tcpdump.pcap
  • -c [num] – capture only num number of packets
  • -w [filename] – write packets to file, -r [filename] – reading from file
Capture all ICMP packets
tcpdump -nei eth0 icmp
Check protocol used (TCP or UDP) for service
tcpdump -nei eth0 tcp port 22 -vv -X | egrep "TCP|UDP"
Display ASCII text (to parse the output using grep or other)
tcpdump -i eth0 -A -s0 port 443
Grab everything between two keywords
tcpdump -i eth0 port 80 -X | sed -n -e '/username/,/=ldap/ p'
Grab user and pass ever plain http
tcpdump -i eth0  port http -l -A | egrep -i \
'pass=|pwd=|log=|login=|user=|username=|pw=|passw=|passwd=|password=|pass:|user:|username:|password:|login:|pass |user ' \
--color=auto --line-buffered -B20
Extract HTTP User Agent from HTTP request header
tcpdump -ei eth0 -nn -A -s1500 -l | grep "User-Agent:"
Capture only HTTP GET and POST packets
tcpdump -ei eth0 -s 0 -A -vv \
'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420' or 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354'

or simply:

tcpdump -ei eth0 -s 0 -v -n -l | egrep -i "POST /|GET /|Host:"
Rotate capture files
tcpdump -ei eth0 -w /tmp/capture-%H.pcap -G 3600 -C 200
  • -G <num> – pcap will be created every <num> seconds
  • -C <size> – close the current pcap and open a new one if is larger than <size>
Top hosts by packets
tcpdump -ei enp0s25 -nnn -t -c 200 | cut -f 1,2,3,4 -d '.' | sort | uniq -c | sort -nr | head -n 20
Excludes any RFC 1918 private address
tcpdump -nei eth0 'not (src net (10 or 172.16/12 or 192.168/16) and dst net (10 or 172.16/12 or 192.168/16))'

Tool: tcpick
Analyse packets in real-time
while true ; do tcpick -a -C -r dump.pcap ; sleep 2 ; clear ; done

Tool: ngrep
ngrep -d eth0 "www.domain.com" port 443
  • -d [iface|any] – set interface
  • [domain] – set hostname
  • port [1-65535] – set port number
ngrep -d eth0 "www.domain.com" src host 10.240.20.2 and port 443
  • (host [ip|hostname]) – filter by ip or hostname
  • (port [1-65535]) – filter by port number
ngrep -d eth0 -qt -O ngrep.pcap "www.domain.com" port 443
  • -q – quiet mode (only payloads)
  • -t – added timestamps
  • -O [filename] – save output to file, -I [filename] – reading from file
ngrep -d eth0 -qt 'HTTP' 'tcp'
  • HTTP – show http headers
  • tcp|udp – set protocol
  • [src|dst] host [ip|hostname] – set direction for specific node
ngrep -l -q -d eth0 -i "User-Agent: curl*"
  • -l – stdout line buffered
  • -i – case-insensitive search

Tool: hping3
hping3 -V -p 80 -s 5050 <scan_type> www.google.com
  • -V|--verbose – verbose mode
  • -p|--destport – set destination port
  • -s|--baseport – set source port
  • <scan_type> – set scan type
    • -F|--fin – set FIN flag, port open if no reply
    • -S|--syn – set SYN flag
    • -P|--push – set PUSH flag
    • -A|--ack – set ACK flag (use when ping is blocked, RST response back if the port is open)
    • -U|--urg – set URG flag
    • -Y|--ymas – set Y unused flag (0x80 – nullscan), port open if no reply
    • -M 0 -UPF – set TCP sequence number and scan type (URG+PUSH+FIN), port open if no reply
hping3 -V -c 1 -1 -C 8 www.google.com
  • -c [num] – packet count
  • -1 – set ICMP mode
  • -C|--icmptype [icmp-num] – set icmp type (default icmp-echo = 8)
hping3 -V -c 1000000 -d 120 -S -w 64 -p 80 --flood --rand-source <remote_host>
  • --flood – sent packets as fast as possible (don’t show replies)
  • --rand-source – random source address mode
  • -d --data – data size
  • -w|--win – winsize (default 64)

Tool: nmap
Ping scans the network
nmap -sP 192.168.0.0/24
Show only open ports
nmap -F --open 192.168.0.0/24
Full TCP port scan using with service version detection
nmap -p 1-65535 -sV -sS -T4 192.168.0.0/24
Nmap scan and pass output to Nikto
nmap -p80,443 192.168.0.0/24 -oG - | nikto.pl -h -
Recon specific ip:service with Nmap NSE scripts stack
# Set variables:
_hosts="192.168.250.10"
_ports="80,443"

# Set Nmap NSE scripts stack:
_nmap_nse_scripts="+dns-brute,\
                   +http-auth-finder,\
                   +http-chrono,\
                   +http-cookie-flags,\
                   +http-cors,\
                   +http-cross-domain-policy,\
                   +http-csrf,\
                   +http-dombased-xss,\
                   +http-enum,\
                   +http-errors,\
                   +http-git,\
                   +http-grep,\
                   +http-internal-ip-disclosure,\
                   +http-jsonp-detection,\
                   +http-malware-host,\
                   +http-methods,\
                   +http-passwd,\
                   +http-phpself-xss,\
                   +http-php-version,\
                   +http-robots.txt,\
                   +http-sitemap-generator,\
                   +http-shellshock,\
                   +http-stored-xss,\
                   +http-title,\
                   +http-unsafe-output-escaping,\
                   +http-useragent-tester,\
                   +http-vhosts,\
                   +http-waf-detect,\
                   +http-waf-fingerprint,\
                   +http-xssed,\
                   +traceroute-geolocation.nse,\
                   +ssl-enum-ciphers,\
                   +whois-domain,\
                   +whois-ip"

# Set Nmap NSE script params:
_nmap_nse_scripts_args="dns-brute.domain=${_hosts},http-cross-domain-policy.domain-lookup=true,"
_nmap_nse_scripts_args+="http-waf-detect.aggro,http-waf-detect.detectBodyChanges,"
_nmap_nse_scripts_args+="http-waf-fingerprint.intensive=1"

# Perform scan:
nmap --script="$_nmap_nse_scripts" --script-args="$_nmap_nse_scripts_args" -p "$_ports" "$_hosts"

Tool: netcat
nc -kl 5000
  • -l – listen for an incoming connection
  • -k – listening after client has disconnected
  • >filename.out – save receive data to file (optional)
nc 192.168.0.1 5051 < filename.in
  • < filename.in – send data to remote host
nc -vz 10.240.30.3 5000
  • -v – verbose output
  • -z – scan for listening daemons
nc -vzu 10.240.30.3 1-65535
  • -u – scan only udp ports
Transfer data file (archive)
server> nc -l 5000 | tar xzvfp -
client> tar czvfp - /path/to/dir | nc 10.240.30.3 5000
Launch remote shell
# 1)
server> nc -l 5000 -e /bin/bash
client> nc 10.240.30.3 5000

# 2)
server> rm -f /tmp/f; mkfifo /tmp/f
server> cat /tmp/f | /bin/bash -i 2>&1 | nc -l 127.0.0.1 5000 > /tmp/f
client> nc 10.240.30.3 5000
Simple file server
while true ; do nc -l 5000 | tar -xvf - ; done
Simple minimal HTTP Server
while true ; do nc -l -p 1500 -c 'echo -e "HTTP/1.1 200 OK\n\n $(date)"' ; done
Simple HTTP Server

Restarts web server after each request – remove while condition for only single connection.

cat > index.html << __EOF__
<!doctype html>
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
        <title></title>
        <meta name="description" content="">
        <meta name="viewport" content="width=device-width, initial-scale=1">
    </head>
    <body>

    <p>

      Hello! It's a site.

    </p>

    </body>
</html>
__EOF__
server> while : ; do \
(echo -ne "HTTP/1.1 200 OK\r\nContent-Length: $(wc -c <index.html)\r\n\r\n" ; cat index.html;) | \
nc -l -p 5000 \
; done
  • -p – port number
Simple HTTP Proxy (single connection)
#!/usr/bin/env bash

if [[ $# != 2 ]] ; then
  printf "%s\\n" \
         "usage: ./nc-proxy listen-port bk_host:bk_port"
fi

_listen_port="$1"
_bk_host=$(echo "$2" | cut -d ":" -f1)
_bk_port=$(echo "$2" | cut -d ":" -f2)

printf "  lport: %s\\nbk_host: %s\\nbk_port: %s\\n\\n" \
       "$_listen_port" "$_bk_host" "$_bk_port"

_tmp=$(mktemp -d)
_back="$_tmp/pipe.back"
_sent="$_tmp/pipe.sent"
_recv="$_tmp/pipe.recv"

trap 'rm -rf "$_tmp"' EXIT

mkfifo -m 0600 "$_back" "$_sent" "$_recv"

sed "s/^/=> /" <"$_sent" &
sed "s/^/<=  /" <"$_recv" &

nc -l -p "$_listen_port" <"$_back" | \
tee "$_sent" | \
nc "$_bk_host" "$_bk_port" | \
tee "$_recv" >"$_back"
server> chmod +x nc-proxy && ./nc-proxy 8080 192.168.252.10:8000
  lport: 8080
bk_host: 192.168.252.10
bk_port: 8000

client> http -p h 10.240.30.3:8080
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=31536000
Content-Length: 2748
Content-Type: text/html; charset=utf-8
Date: Sun, 01 Jul 2018 20:12:08 GMT
Last-Modified: Sun, 01 Apr 2018 21:53:37 GMT
Create a single-use TCP or UDP proxy
### TCP -> TCP
nc -l -p 2000 -c "nc [ip|hostname] 3000"

### TCP -> UDP
nc -l -p 2000 -c "nc -u [ip|hostname] 3000"

### UDP -> UDP
nc -l -u -p 2000 -c "nc -u [ip|hostname] 3000"

### UDP -> TCP
nc -l -u -p 2000 -c "nc [ip|hostname] 3000"

Tool: gnutls-cli
Testing connection to remote host (with SNI support)
gnutls-cli -p 443 google.com
Testing connection to remote host (without SNI support)
gnutls-cli --disable-sni -p 443 google.com

Tool: socat
Testing remote connection to port
socat - TCP4:10.240.30.3:22
  • - – standard input (STDIO)
  • TCP4:<params> – set tcp4 connection with specific params
    • [hostname|ip] – set hostname/ip
    • [1-65535] – set port number
Redirecting TCP-traffic to a UNIX domain socket under Linux
socat TCP-LISTEN:1234,bind=127.0.0.1,reuseaddr,fork,su=nobody,range=127.0.0.0/8 UNIX-CLIENT:/tmp/foo
  • TCP-LISTEN:<params> – set tcp listen with specific params
    • [1-65535] – set port number
    • bind=[hostname|ip] – set bind hostname/ip
    • reuseaddr – allows other sockets to bind to an address
    • fork – keeps the parent process attempting to produce more connections
    • su=nobody – set user
    • range=[ip-range] – ip range
  • UNIX-CLIENT:<params> – communicates with the specified peer socket
    • filename – define socket

Tool: p0f
Set iface in promiscuous mode and dump traffic to the log file
p0f -i enp0s25 -p -d -o /dump/enp0s25.log
  • -i – listen on the specified interface
  • -p – set interface in promiscuous mode
  • -d – fork into background
  • -o – output file

Tool: netstat
Graph # of connections for each hosts
netstat -an | awk '/ESTABLISHED/ { split($5,ip,":"); if (ip[1] !~ /^$/) print ip[1] }' | \
sort | uniq -c | awk '{ printf("%s\t%s\t",$2,$1) ; for (i = 0; i < $1; i++) {printf("*")}; print "" }'
Monitor open connections for specific port including listen, count and sort it per IP
watch "netstat -plan | grep :443 | awk {'print \$5'} | cut -d: -f 1 | sort | uniq -c | sort -nk 1"
Grab banners from local IPv4 listening ports
netstat -nlt | grep 'tcp ' | grep -Eo "[1-9][0-9]*" | xargs -I {} sh -c "echo "" | nc -v -n -w1 127.0.0.1 {}"

Tool: rsync
Rsync remote data as root using sudo
rsync --rsync-path 'sudo rsync' username@hostname:/path/to/dir/ /local/

Tool: host
Resolves the domain name (using external dns server)
host google.com 9.9.9.9
Checks the domain administrator (SOA record)
host -t soa google.com 9.9.9.9

Tool: dig
Resolves the domain name (short output)
dig google.com +short
Lookup NS record for specific domain
dig @9.9.9.9 google.com NS
Query only answer section
dig google.com +nocomments +noquestion +noauthority +noadditional +nostats
Query ALL DNS Records
dig google.com ANY +noall +answer
DNS Reverse Look-up
dig -x 172.217.16.14 +short

Tool: certbot
Generate multidomain certificate
certbot certonly -d example.com -d www.example.com
Generate wildcard certificate
certbot certonly --manual --preferred-challenges=dns -d example.com -d *.example.com
Generate certificate with 4096 bit private key
certbot certonly -d example.com -d www.example.com --rsa-key-size 4096

Tool: network-other
Get all subnets for specific AS (Autonomous system)
AS="AS32934"
whois -h whois.radb.net -- "-i origin ${AS}" | \
grep "^route:" | \
cut -d ":" -f2 | \
sed -e 's/^[ \t]//' | \
sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 | \
cut -d ":" -f2 | \
sed -e 's/^[ \t]/allow /' | \
sed 's/$/;/' | \
sed 's/allow  */subnet -> /g'
Resolves domain name from dns.google.com with curl and jq
_dname="google.com" ; curl -s "https://dns.google.com/resolve?name=${_dname}&type=A" | jq .
Tool: git
Log alias for a decent view of your repo
# 1)
git log --oneline --decorate --graph --all

# 2)
git log --graph \
--pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' \
--abbrev-commit

Tool: python
Static HTTP web server
# Python 3.x
python3 -m http.server 8000 --bind 127.0.0.1

# Python 2.x
python -m SimpleHTTPServer 8000
Static HTTP web server with SSL support
# Python 3.x
from http.server import HTTPServer, BaseHTTPRequestHandler
import ssl

httpd = HTTPServer(('localhost', 4443), BaseHTTPRequestHandler)

httpd.socket = ssl.wrap_socket (httpd.socket,
        keyfile="path/to/key.pem",
        certfile='path/to/cert.pem', server_side=True)

httpd.serve_forever()

# Python 2.x
import BaseHTTPServer, SimpleHTTPServer
import ssl

httpd = BaseHTTPServer.HTTPServer(('localhost', 4443),
        SimpleHTTPServer.SimpleHTTPRequestHandler)

httpd.socket = ssl.wrap_socket (httpd.socket,
        keyfile="path/tp/key.pem",
        certfile='path/to/cert.pem', server_side=True)

httpd.serve_forever()
Encode base64
python -m base64 -e <<< "sample string"
Decode base64
python -m base64 -d <<< "dGhpcyBpcyBlbmNvZGVkCg=="
Tool: awk
Search for matching lines
# egrep foo
awk '/foo/' filename
Search non matching lines
# egrep -v foo
awk '!/foo/' filename
Print matching lines with numbers
# egrep -n foo
awk '/foo/{print FNR,$0}' filename
Print the last column
awk '{print $NF}' filename
Find all the lines longer than 80 characters
awk 'length($0)>80{print FNR,$0}' filename
Print only lines of less than 80 characters
awk 'length < 80 filename
Print double new lines a file
awk '1; { print "" }' filename
Print line numbers
awk '{ print FNR "\t" $0 }' filename
awk '{ printf("%5d : %s\n", NR, $0) }' filename   # in a fancy manner
Print line numbers for only non-blank lines
awk 'NF { $0=++a " :" $0 }; { print }' filename
Print the line and the next two (i=5) lines after the line matching regexp
awk '/foo/{i=5+1;}{if(i){i--; print;}}' filename
Print the lines starting at the line matching ‘server {‘ until the line matching ‘}’
awk '/server {/,/}/' filename
Print multiple columns with separators
awk -F' ' '{print "ip:\t" $2 "\n port:\t" $3' filename
Remove empty lines
awk 'NF > 0' filename

# alternative:
awk NF filename
Delete trailing white space (spaces, tabs)
awk '{sub(/[ \t]*$/, "");print}' filename
Delete leading white space
awk '{sub(/^[ \t]+/, ""); print}' filename
Remove duplicate consecutive lines
# uniq
awk 'a !~ $0{print}; {a=$0}' filename
Remove duplicate entries in a file without sorting
awk '!x[$0]++' filename
Exclude multiple columns
awk '{$1=$3=""}1' filename
Substitute foo for bar on lines matching regexp
awk '/regexp/{gsub(/foo/, "bar")};{print}' filename
Add some characters at the beginning of matching lines
awk '/regexp/{sub(/^/, "++++"); print;next;}{print}' filename
Get the last hour of Apache logs
awk '/'$(date -d "1 hours ago" "+%d\\/%b\\/%Y:%H:%M")'/,/'$(date "+%d\\/%b\\/%Y:%H:%M")'/ { print $0 }' \
/var/log/httpd/access_log

Tool: sed
Print a specific line from a file
sed -n 10p /path/to/file
Remove a specific line from a file
sed -i 10d /path/to/file
# alternative (BSD): sed -i'' 10d /path/to/file
Remove a range of lines from a file
sed -i <file> -re '<start>,<end>d'
Replace newline(s) with a space
sed ':a;N;$!ba;s/\n/ /g' /path/to/file

# cross-platform compatible syntax:
sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g' /path/to/file
  • :a create a label a
  • N append the next line to the pattern space
  • $! if not the last line, ba branch (go to) label a
  • s substitute, /\n/ regex for new line, / / by a space, /g global match (as many times as it can)

Alternatives:

# perl version (sed-like speed):
perl -p -e 's/\n/ /' /path/to/file

# bash version (slow):
while read line ; do printf "%s" "$line " ; done < file
Delete string +N next lines
sed '/start/,+4d' /path/to/file

Tool: grep
Search for a “pattern” inside all files in the current directory
grep -rn "pattern"
grep -RnisI "pattern" *
fgrep "pattern" * -R
Show only for multiple patterns
grep 'INFO*'\''WARN' filename
grep 'INFO\|WARN' filename
grep -e INFO -e WARN filename
grep -E '(INFO|WARN)' filename
egrep "INFO|WARN" filename
Except multiple patterns
grep -vE '(error|critical|warning)' filename
Show data from file without comments
grep -v ^[[:space:]]*# filename
Show data from file without comments and new lines
egrep -v '#|^$' filename
Show strings with a dash/hyphen
grep -e -- filename
grep -- -- filename
grep "\-\-" filename
Remove blank lines from a file and save output to new file
grep . filename > newfilename
Tool: perl
Search and replace (in place)
perl -i -pe's/SEARCH/REPLACE/' filename
Edit of *.conf files changing all foo to bar (and backup original)
perl -p -i.orig -e 's/\bfoo\b/bar/g' *.conf
Prints the first 20 lines from *.conf files
perl -pe 'exit if $. > 20' *.conf
Search lines 10 to 20
perl -ne 'print if 10 .. 20' filename
Delete first 10 lines (and backup original)
perl -i.orig -ne 'print unless 1 .. 10' filename
Delete all but lines between foo and bar (and backup original)
perl -i.orig -ne 'print unless /^foo$/ .. /^bar$/' filename
Reduce multiple blank lines to a single line
perl -p -i -00pe0 filename
Convert tabs to spaces (1t = 2sp)
perl -p -i -e 's/\t/  /g' filename
Read input from a file and report number of lines and characters
perl -lne '$i++; $in += length($_); END { print "$i lines, $in characters"; }' filename

Shell functions  [TOC]

Table of Contents
Domain resolve
# Dependencies:
#   - curl
#   - jq

function DomainResolve() {

  local _host="$1"

  local _curl_base="curl --request GET"
  local _timeout="15"

  _host_ip=$($_curl_base -ks -m "$_timeout" "https://dns.google.com/resolve?name=${_host}&type=A" | \
  jq '.Answer[0].data' | tr -d "\"" 2>/dev/null)

  if [[ -z "$_host_ip" ]] || [[ "$_host_ip" == "null" ]] ; then

    echo -en "Unsuccessful domain name resolution.\\n"

  else

    echo -en "$_host > $_host_ip\\n"

  fi

}

Example:

shell> DomainResolve nmap.org
nmap.org > 45.33.49.119

shell> DomainResolve nmap.org
Unsuccessful domain name resolution.
Get ASN
# Dependencies:
#   - curl
#   - python

function GetASN() {

  local _ip="$1"

  local _curl_base="curl --request GET"
  local _timeout="15"

  _asn=$($_curl_base -ks -m "$_timeout" "http://ip-api.com/json/${_ip}" | \
  python -c 'import sys, json; print json.load(sys.stdin)["as"]' 2>/dev/null)

  _state=$(echo $?)

  if [[ -z "$_ip" ]] || [[ "$_ip" == "null" ]] || [[ "$_state" -ne 0 ]]; then

    echo -en "Unsuccessful ASN gathering.\\n"

  else

    echo -en "$_ip > $_asn\\n"

  fi

}

Example:

shell> GetASN 1.1.1.1
1.1.1.1 > AS13335 Cloudflare, Inc.

shell> GetASN 0.0.0.0
Unsuccessful ASN gathering.

Source: Github