You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

README.md 35 KiB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269
  1. <p align="center">
  2. <img src="https://github.com/trimstray/awesome-ninja-admins/blob/master/doc/img/awesome_ninja_admins.png"
  3. alt="Master">
  4. </p>
  5. <br>
  6. <h4 align="center">A collection of awesome lists, manuals, blogs, hacks, one-liners and tools for <b>Awesome Ninja Admins</b>.</h4>
  7. <br>
  8. <p align="center">
  9. <a href="https://github.com/trimstray/awesome-ninja-admins/tree/master">
  10. <img src="https://img.shields.io/badge/Branch-master-green.svg?longCache=true"
  11. alt="Branch">
  12. </a>
  13. <a href="https://awesome.re">
  14. <img src="https://awesome.re/badge.svg"
  15. alt="Awesome">
  16. <a href="https://github.com/trimstray/awesome-ninja-admins">
  17. <img src="https://img.shields.io/badge/Status-09072018-red.svg?longCache=true"
  18. alt="Status">
  19. </a>
  20. <a href="http://www.gnu.org/licenses/">
  21. <img src="https://img.shields.io/badge/License-GNU-blue.svg?longCache=true"
  22. alt="License">
  23. </a>
  24. </p>
  25. <div align="center">
  26. <sub>Created by
  27. <a href="https://twitter.com/trimstray">trimstray</a> and
  28. <a href="https://github.com/trimstray/awesome-ninja-admins/graphs/contributors">
  29. contributors
  30. </a>
  31. </div>
  32. <br>
  33. ***
  34. ## Who is Ninja Admins?
  35. - race of pure evil who rule the network through a monarchistic feudelic system
  36. - they never opened the door for strangers (or anyone at all)
  37. - they know very nasty piece of code like a **[fork bombs](https://en.wikipedia.org/wiki/Fork_bomb)**
  38. - they can make dd is not a **[destroyer of disks](http://www.noah.org/wiki/Dd_-_Destroyer_of_Disks)**
  39. - they know that `#!/usr/bin/env bash` superior to `#!/bin/bash`
  40. - they know that `su -` logs in completely as root
  41. - they miss and cry for **[Slackware](http://www.slackware.com/)** on production
  42. - they love the old admin nix-world
  43. ## :ballot_box_with_check: Todo
  44. - [ ] Add useful shell functions
  45. - [ ] Add one-liners for collection tools (eg. CLI Tools)
  46. - [ ] Add Ninja Admins T-Shirt stickers
  47. - [ ] Generate Awesome Ninja Admins book (eg. pdf format)
  48. ## Ninja Admins Collection
  49. #### CLI Tools
  50. ##### :black_small_square: Shells
  51. <p>
  52. &nbsp;&nbsp;:small_orange_diamond: <a href="https://ohmyz.sh/"><b>Oh My ZSH!</b></a> - the best framework for managing your Zsh configuration.<br>
  53. &nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/Bash-it/bash-it"><b>bash-it</b></a> - a community Bash framework.<br>
  54. </p>
  55. ##### :black_small_square: Managers
  56. <p>
  57. &nbsp;&nbsp;:small_orange_diamond: <a href="https://midnight-commander.org/"><b>Midnight Commander</b></a> - visual file manager, licensed under GNU General Public License.<br>
  58. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.gnu.org/software/screen/"><b>screen</b></a> - full-screen window manager that multiplexes a physical terminal.<br>
  59. &nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/tmux/tmux/wiki"><b>tmux</b></a> - terminal multiplexer, lets you switch easily between several programs in one terminal.<br>
  60. </p>
  61. ##### :black_small_square: Network
  62. <p>
  63. &nbsp;&nbsp;:small_orange_diamond: <a href="https://curl.haxx.se/"><b>Curl</b></a> - command line tool and library
  64. for transferring data with URLs.<br>
  65. &nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/jkbrzt/httpie"><b>HTTPie</b></a> - a user-friendly HTTP client.<br>
  66. &nbsp;&nbsp;:small_orange_diamond: <a href="https://gnutls.org/manual/html_node/gnutls_002dcli-Invocation.html"><b>gnutls-cli</b></a> - client program to set up a TLS connection to some other computer.<br>
  67. &nbsp;&nbsp;:small_orange_diamond: <a href="http://netcat.sourceforge.net/"><b>netcat</b></a> - networking utility which reads and writes data across network connections, using the TCP/IP protocol.<br>
  68. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.tcpdump.org/"><b>tcpdump</b></a> - powerful command-line packet analyzer.<br>
  69. </p>
  70. ##### :black_small_square: Databases
  71. <p>
  72. &nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/dbcli/pgcli"><b>pgcli</b></a> - postgres CLI with autocompletion and syntax highlighting.<br>
  73. &nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/dbcli/mycli"><b>mycli</b></a> - terminal client for MySQL with autocompletion and syntax highlighting.<br>
  74. </p>
  75. #### Web Tools
  76. ##### :black_small_square: SSL
  77. <p>
  78. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.ssllabs.com/ssltest/"><b>SSL Server Test</b></a> - free online service performs a deep analysis of the configuration of any SSL web server.<br>
  79. &nbsp;&nbsp;:small_orange_diamond: <a href="https://dev.ssllabs.com/ssltest/"><b>SSL Server Test (DEV)</b></a> - free online service performs a deep analysis of the configuration of any SSL web server.<br>
  80. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.htbridge.com/ssl/"><b>ImmuniWeb® SSLScan</b></a> - test SSL/TLS (PCI DSS, HIPAA and NIST).<br>
  81. &nbsp;&nbsp;:small_orange_diamond: <a href="https://report-uri.com/home/tools"><b>Report URI</b></a> - monitoring security policies like CSP and HPKP.<br>
  82. &nbsp;&nbsp;:small_orange_diamond: <a href="https://csp-evaluator.withgoogle.com/"><b>CSP Evaluator</b></a> - allows developers and security experts to check if a Content Security Policy.<br>
  83. &nbsp;&nbsp;:small_orange_diamond: <a href="https://ccadb.org/resources"><b>Common CA Database</b></a> - repository of information about CAs, and their root and intermediate certificates.<br>
  84. &nbsp;&nbsp;:small_orange_diamond: <a href="https://certstream.calidog.io/"><b>CERTSTREAM</b></a> - real-time certificate transparency log update stream.<br>
  85. </p>
  86. ##### :black_small_square: HTTP Headers
  87. <p>
  88. &nbsp;&nbsp;:small_orange_diamond: <a href="https://securityheaders.com/"><b>Security Headers</b></a> - analyse the HTTP response headers (with rating system to the results).<br>
  89. &nbsp;&nbsp;:small_orange_diamond: <a href="https://observatory.mozilla.org/"><b>Observatory by Mozilla</b></a> - set of tools to analyze your website.<br>
  90. </p>
  91. ##### :black_small_square: DNS
  92. <p>
  93. &nbsp;&nbsp;:small_orange_diamond: <a href="http://viewdns.info/"><b>ViewDNS</b></a> - one source for free DNS related tools and information.<br>
  94. &nbsp;&nbsp;:small_orange_diamond: <a href="https://dnsspy.io/"><b>DNS Spy</b></a> - monitor, validate and verify your DNS configurations.<br>
  95. &nbsp;&nbsp;:small_orange_diamond: <a href="https://dnslytics.com/"><b>DNSlytics</b></a> - online investigation tool.<br>
  96. </p>
  97. ##### :black_small_square: Mail
  98. <p>
  99. &nbsp;&nbsp;:small_orange_diamond: <a href="https://mxtoolbox.com/SuperTool.aspx"><b>MX Toolbox</b></a> - all of your MX record, DNS, blacklist and SMTP diagnostics in one integrated tool.<br>
  100. </p>
  101. ##### :black_small_square: Mass scanners (search engines)
  102. <p>
  103. &nbsp;&nbsp;:small_orange_diamond: <a href="https://censys.io/"><b>Censys</b></a> - platform that helps information security practitioners discover, monitor, and analyze devices.<br>
  104. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.shodan.io/"><b>Shodan</b></a> - the world's first search engine for Internet-connected devices.<br>
  105. &nbsp;&nbsp;:small_orange_diamond: <a href="https://viz.greynoise.io/table"><b>GreyNoise</b></a> - mass scanner (such as Shodan and Censys).<br>
  106. </p>
  107. ##### :black_small_square: Net-tools
  108. <p>
  109. &nbsp;&nbsp;:small_orange_diamond: <a href="https://toolbar.netcraft.com/site_report"><b>Netcraft</b></a> - detailed report about the site, helping you to make informed choices about their integrity.<br>
  110. &nbsp;&nbsp;:small_orange_diamond: <a href="https://securitytrails.com/"><b>Security Trails</b></a> - APIs for Security Companies, Researchers and Teams.<br>
  111. &nbsp;&nbsp;:small_orange_diamond: <a href="https://tools.keycdn.com/curl"><b>Online Curl</b></a> - curl test, analyze HTTP Response Headers.<br>
  112. &nbsp;&nbsp;:small_orange_diamond: <a href="http://ping.eu/"><b>Ping.eu</b></a> - online Ping, Traceroute, DNS lookup, WHOIS and others.<br>
  113. &nbsp;&nbsp;:small_orange_diamond: <a href="https://network-tools.com/"><b>Network-Tools</b></a> - network tools for webmasters, IT technicians & geeks.<br>
  114. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.url-encode-decode.com/"><b>URL Encode/Decode</b></a> - tool from above to either encode or decode a string of text.<br>
  115. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.hardenize.com/"><b>Hardenize</b></a> - deploy the security standards.<br>
  116. </p>
  117. ##### :black_small_square: Performance
  118. <p>
  119. &nbsp;&nbsp;:small_orange_diamond: <a href="https://gtmetrix.com/"><b>GTmetrix</b></a> - analyze your site’s speed and make it faster.<br>
  120. &nbsp;&nbsp;:small_orange_diamond: <a href="https://performance.sucuri.net/"><b>Sucuri loadtimetester</b></a> - test here the
  121. performance of any of your sites from across the globe.<br>
  122. </p>
  123. ##### :black_small_square: Passwords
  124. <p>
  125. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.random.org/passwords/"><b>Random.org</b></a> - generate random passwords.<br>
  126. &nbsp;&nbsp;:small_orange_diamond: <a href="https://gotcha.pw/"><b>Gotcha?</b></a> - list of 1.4 billion accounts circulates around the Internet.<br>
  127. &nbsp;&nbsp;:small_orange_diamond: <a href="https://haveibeenpwned.com/"><b>have i been pwned?</b></a> - check if you have an account that has been compromised in a data breach.<br>
  128. </p>
  129. #### Manuals/Howtos/Tutorials
  130. ##### :black_small_square: Bash
  131. <p>
  132. &nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/dylanaraps/pure-bash-bible"><b>pure-bash-bible</b></a> - a collection of pure bash alternatives to external processes.<br>
  133. &nbsp;&nbsp;:small_orange_diamond: <a href="http://wiki.bash-hackers.org/start"><b>The Bash Hackers Wiki</b></a> - hold documentation of any kind about GNU Bash.<br>
  134. </p>
  135. ##### :black_small_square: Unix tutorials
  136. <p>
  137. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.cyberciti.biz/"><b>nixCraft</b></a> - linux and unix tutorials for new and seasoned sysadmin.<br>
  138. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.tecmint.com/"><b>TecMint</b></a> - the ideal Linux blog for Sysadmins & Geeks.<br>
  139. </p>
  140. ##### :black_small_square: Hacking
  141. <p>
  142. &nbsp;&nbsp;:small_orange_diamond: <a href="http://www.hackingarticles.in/"><b>Hacking Articles</b></a> - LRaj Chandel's Security & Hacking Blog.<br>
  143. </p>
  144. #### Blogs
  145. <p>
  146. &nbsp;&nbsp;:small_orange_diamond: <a href="http://www.brendangregg.com/"><b>Brendan Gregg's Blog</b></a> - Brendan Gregg is an industry expert in computing performance and cloud computing.<br>
  147. &nbsp;&nbsp;:small_orange_diamond: <a href="http://gynvael.coldwind.pl/"><b>Gynvael "GynDream" Coldwind</b></a> - Gynvael is a IT security engineer at Google.<br>
  148. &nbsp;&nbsp;:small_orange_diamond: <a href="http://lcamtuf.coredump.cx/"><b>Michał "lcamtuf" Zalewski</b></a> - "white hat" hacker, computer security expert.<br>
  149. &nbsp;&nbsp;:small_orange_diamond: <a href="https://ma.ttias.be/"><b>Mattias Geniar</b></a> - developer, Sysadmin, Blogger, Podcaster and Public Speaker.<br>
  150. &nbsp;&nbsp;:small_orange_diamond: <a href="https://nickcraver.com/"><b>Nick Craver</b></a> - Software Developer and Systems Administrator for Stack Exchange.<br>
  151. &nbsp;&nbsp;:small_orange_diamond: <a href="https://robert.penz.name/"><b>Robert Penz</b></a> - IT security Expert.<br>
  152. &nbsp;&nbsp;:small_orange_diamond: <a href="https://scotthelme.co.uk/"><b>Scott Helme</b></a> - Security Researcher, international speaker and founder of securityheaders.com and report-uri.com.<br>
  153. &nbsp;&nbsp;:small_orange_diamond: <a href="https://security.szurek.pl/"><b>Kacper Szurek</b></a> - Detection Engineer at ESET.<br>
  154. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.troyhunt.com/"><b>Troy Hunt</b></a> - Microsoft Regional Director and Microsoft Most Valuable Professional for Developer Security.<br>
  155. &nbsp;&nbsp;:small_orange_diamond: <a href="https://linux-audit.com/"><b>Linux Audit</b></a> - the Linux security blog about Auditing, Hardening, and Compliance by Michael Boelen.<br>
  156. &nbsp;&nbsp;:small_orange_diamond: <a href="http://www.grymoire.com/"><b>The Grymoire</b></a> - collection of useful incantations for wizards, be you computer wizards, magicians, or whatever.<br>
  157. </p>
  158. #### Systems/Services
  159. ##### :black_small_square: Systems
  160. <p>
  161. &nbsp;&nbsp;:small_orange_diamond: <a href="http://www.slackware.com/"><b>Slackware</b></a> - the most "Unix-like" Linux distribution.<br>
  162. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.openbsd.org/"><b>OpenBSD</b></a> - multi-platform 4.4BSD-based UNIX-like operating system.<br>
  163. &nbsp;&nbsp;:small_orange_diamond: <a href="https://hardenedbsd.org/"><b>HardenedBSD</b></a> - HardenedBSD aims to implement innovative exploit mitigation and security solutions.<br>
  164. </p>
  165. ##### :black_small_square: HTTP(s) Services
  166. <p>
  167. &nbsp;&nbsp;:small_orange_diamond: <a href="https://varnish-cache.org/"><b>Varnish HTTP Cache</b></a> - HTTP accelerator designed for content-heavy dynamic web sites.<br>
  168. </p>
  169. ##### :black_small_square: Security/hardening
  170. <p>
  171. &nbsp;&nbsp;:small_orange_diamond: <a href="https://emeraldonion.org/"><b>Emerald Onion</b></a> - Seattle-based encrypted-transit internet service provider.<br>
  172. </p>
  173. #### Lists
  174. <p>
  175. &nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/kahun/awesome-sysadmin"><b>Awesome Sysadmin</b></a> - amazingly awesome open source sysadmin resources.<br>
  176. &nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/alebcay/awesome-shell"><b>Awesome Shell</b></a> - awesome command-line frameworks, toolkits, guides and gizmos.<br>
  177. &nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/Hack-with-Github/Awesome-Hacking"><b>Awesome-Hacking</b></a> - awesome lists for hackers, pentesters and security researchers.<br>
  178. </p>
  179. #### Hacking/Penetration testing
  180. ##### :black_small_square: Bounty programs
  181. <p>
  182. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.openbugbounty.org/"><b>Openbugbounty</b></a> - allows any security researcher reporting a vulnerability on any website.<br>
  183. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.hackerone.com/"><b>hackerone</b></a> - global hacker community to surface the most relevant security issues.<br>
  184. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.bugcrowd.com/"><b>bugcrowd</b></a> - crowdsourced cybersecurity for the enterprise.<br>
  185. &nbsp;&nbsp;:small_orange_diamond: <a href="https://crowdshield.com/"><b>Crowdshield</b></a> - crowdsourced Security & Bug Bounty Management.<br>
  186. </p>
  187. ##### :black_small_square: Web Training Apps
  188. <p>
  189. &nbsp;&nbsp;:small_orange_diamond: <a href="http://www.dvwa.co.uk/"><b>DVWA</b></a> - PHP/MySQL web application that is damn vulnerable.<br>
  190. &nbsp;&nbsp;:small_orange_diamond: <a href="https://sourceforge.net/projects/mutillidae/"><b>OWASP Mutillidae II</b></a> - free, open source, deliberately vulnerable web-application.<br>
  191. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.owasp.org/index.php/OWASP_Juice_Shop_Project"><b>OWASP Juice Shop Project</b></a> - the most bug-free vulnerable application in existence.<br>
  192. &nbsp;&nbsp;:small_orange_diamond: <a href="https://www.owasp.org/index.php/Category:OWASP_WebGoat_Project"><b>OWASP WebGoat Project</b></a> - insecure web application maintained by OWASP designed to teach web app security.<br>
  193. &nbsp;&nbsp;:small_orange_diamond: <a href="https://github.com/opendns/Security_Ninjas_AppSec_Training"><b>Security Ninjas</b></a> - open source application security training program.<br>
  194. </p>
  195. #### One-liners
  196. ##### Table of Contents
  197. - **[System](#system)**
  198. * [terminal](#tool-terminal)
  199. * [mount](#tool-mount)
  200. * [fuser](#tool-fuser)
  201. * [ps](#tool-ps)
  202. * [top](#tool-top)
  203. * [find](#tool-find)
  204. * [diff](#tool-diff)
  205. * [tail](#tool-tail)
  206. * [cpulimit](#tool-cpulimit)
  207. * [pwdx](#tool-pwdx)
  208. * [tr](#tool-tr)
  209. * [chmod](#tool-chmod)
  210. * [who](#tool-who)
  211. * [screen](#tool-screen)
  212. * [du](#tool-du)
  213. * [inotifywait](#tool-inotifywait)
  214. * [openssl](#tool-openssl)
  215. - **[HTTP/HTTPS](#http-https)**
  216. * [curl](#tool-curl)
  217. * [httpie](#tool-httpie)
  218. - **[Network](#network)**
  219. * [ssh](#tool-ssh)
  220. * [linux-dev](#tool-linux-dev)
  221. * [tcpdump](#tool-tcpdump)
  222. * [ngrep](#tool-ngrep)
  223. * [hping3](#tool-hping3)
  224. * [netcat](#tool-netcat)
  225. * [socat](#tool-socat)
  226. * [lsof](#tool-lsof)
  227. * [netstat](#tool-netstat)
  228. * [rsync](#tool-rsync)
  229. - **[Programming](#programming)**
  230. * [awk](#tool-awk)
  231. * [sed](#tool-sed)
  232. * [grep](#tool-grep)
  233. <a name="system"><b>System</b></a>
  234. ##### Tool: [terminal](https://en.wikipedia.org/wiki/Linux_console)
  235. ###### Close shell keeping all subprocess running
  236. ```bash
  237. disown -a && exit
  238. ```
  239. ###### Exit without saving shell history
  240. ```bash
  241. kill -9 $$
  242. unset HISTFILE && exit
  243. ```
  244. ###### Perform a branching conditional
  245. ```bash
  246. true && { echo success;} || { echo failed; }
  247. ```
  248. ###### Pipe stdout and stderr to separate commands
  249. ```bash
  250. some_command > >(/bin/cmd_for_stdout) 2> >(/bin/cmd_for_stderr)
  251. ```
  252. ###### Pipe stdout and stderr to separate commands
  253. ```bash
  254. (some_command 2>&1 1>&3 | tee errorlog ) 3>&1 1>&2 | tee stdoutlog
  255. ```
  256. ###### List of commands you use most often
  257. ```bash
  258. history | awk '{ a[$2]++ } END { for(i in a) { print a[i] " " i } }' | sort -rn | head
  259. ```
  260. ###### Quickly backup a file
  261. ```bash
  262. cp filename{,.orig}
  263. ```
  264. ###### Delete all files in a folder that don't match a certain file extension
  265. ```bash
  266. rm !(*.foo|*.bar|*.baz)
  267. ```
  268. ###### Edit a file on a remote host using vim
  269. ```bash
  270. vim scp://user@host//etc/fstab
  271. ```
  272. ###### Create a directory and change into it at the same time
  273. ```bash
  274. mkd () { mkdir -p "$@" && cd "$@"; }
  275. ```
  276. ###### Convert uppercase files to lowercase files
  277. ```bash
  278. rename 'y/A-Z/a-z/' *
  279. ```
  280. ###### Print a row of characters across the terminal
  281. ```bash
  282. printf "%`tput cols`s" | tr ' ' '#'
  283. ```
  284. ###### Show shell history without line numbers
  285. ```bash
  286. history | cut -c 8-
  287. fc -l -n 1 | sed 's/^\s*//'
  288. ```
  289. ___
  290. ##### Tool: [mount](https://en.wikipedia.org/wiki/Mount_(Unix))
  291. ###### Mount a temporary ram partition
  292. ```bash
  293. mount -t tmpfs tmpfs /mnt -o size=64M
  294. ```
  295. * `-t` - filesystem type
  296. * `-o` - mount options
  297. ___
  298. ##### Tool: [fuser](https://en.wikipedia.org/wiki/Fuser_(Unix))
  299. ###### Kills a process that is locking a file
  300. ```bash
  301. fuser -k filename
  302. ```
  303. ###### Show what PID is listening on specific port
  304. ```bash
  305. fuser -v 53/udp
  306. ```
  307. ___
  308. ##### Tool: [ps](https://en.wikipedia.org/wiki/Ps_(Unix))
  309. ###### Show a 4-way scrollable process tree with full details
  310. ```bash
  311. ps awwfux | less -S
  312. ```
  313. ###### Processes per user counter
  314. ```bash
  315. ps hax -o user | sort | uniq -c | sort -r
  316. ```
  317. ___
  318. ##### Tool: [find](https://en.wikipedia.org/wiki/Find_(Unix))
  319. ###### Find files that have been modified on your system in the past 60 minutes
  320. ```bash
  321. find / -mmin 60 -type f
  322. ```
  323. ###### Find all files larger than 20M
  324. ```bash
  325. find / -type f -size +20M
  326. ```
  327. ###### Find duplicate files (based on MD5 hash)
  328. ```bash
  329. find -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 33
  330. ```
  331. ###### Change permission only for files
  332. ```bash
  333. cd /var/www/site && find . -type f -exec chmod 766 {} \;
  334. cd /var/www/site && find . -type f -exec chmod 664 {} +
  335. ```
  336. ###### Change permission only for directories
  337. ```bash
  338. cd /var/www/site && find . -type d -exec chmod g+x {} \;
  339. cd /var/www/site && find . -type d -exec chmod g+rwx {} +
  340. ```
  341. ###### Find files and directories for specific user
  342. ```bash
  343. find . -user <username> -print
  344. ```
  345. ###### Find files and directories for all without specific user
  346. ```bash
  347. find . \!-user <username> -print
  348. ```
  349. ###### Delete older files than 60 days
  350. ```bash
  351. find . -type f -mtime +60 -delete
  352. ```
  353. ___
  354. ##### Tool: [top](https://en.wikipedia.org/wiki/Top_(software))
  355. ###### Use top to monitor only all processes with the specific string
  356. ```bash
  357. top -p $(pgrep -d , <str>)
  358. ```
  359. * `<str>` - process containing str (eg. nginx, worker)
  360. ___
  361. ##### Tool: [diff](https://en.wikipedia.org/wiki/Diff)
  362. ###### Compare two directory trees
  363. ```bash
  364. diff <(cd directory1 && find | sort) <(cd directory2 && find | sort)
  365. ```
  366. ___
  367. ##### Tool: [tail](https://en.wikipedia.org/wiki/Tail_(Unix))
  368. ###### Annotate tail -f with timestamps
  369. ```bash
  370. tail -f file | while read; do echo "$(date +%T.%N) $REPLY"; done
  371. ```
  372. ###### Analyse an Apache access log for the most common IP addresses
  373. ```bash
  374. tail -10000 access_log | awk '{print $1}' | sort | uniq -c | sort -n | tail
  375. ```
  376. ___
  377. ##### Tool: [cpulimit](http://cpulimit.sourceforge.net/)
  378. ###### Limit the cpu usage of a process
  379. ```bash
  380. cpulimit -p pid -l 50
  381. ```
  382. ##### Tool: [pwdx](https://www.cyberciti.biz/faq/unix-linux-pwdx-command-examples-usage-syntax/)
  383. ###### Show current working directory of a process
  384. ```bash
  385. pwdx <pid>
  386. ```
  387. ___
  388. ##### Tool: [taskset](https://www.cyberciti.biz/faq/taskset-cpu-affinity-command/)
  389. ###### Start a command on only one CPU core
  390. ```bash
  391. taskset -c 0 <command>
  392. ```
  393. ___
  394. ##### Tool: [tr](https://en.wikipedia.org/wiki/Tr_(Unix))
  395. ###### Show directories in the PATH, one per line
  396. ```bash
  397. tr : '\n' <<<$PATH
  398. ```
  399. ##### Tool: [chmod](https://en.wikipedia.org/wiki/Chmod)
  400. ###### Remove executable bit from all files in the current directory
  401. ```bash
  402. chmod -R -x+X *
  403. ```
  404. ___
  405. ##### Tool: [who](https://en.wikipedia.org/wiki/Who_(Unix))
  406. ###### Find last reboot time
  407. ```bash
  408. who -b
  409. ```
  410. ___
  411. ##### Tool: [screen](https://en.wikipedia.org/wiki/GNU_Screen)
  412. ###### Start screen in detached mode
  413. ```bash
  414. screen -d -m [<command>]
  415. ```
  416. ___
  417. ##### Tool: [du](https://en.wikipedia.org/wiki/GNU_Screen)
  418. ###### Show 20 biggest directories with 'K M G'
  419. ```bash
  420. du | sort -r -n | awk '{split("K M G",v); s=1; while($1>1024){$1/=1024; s++} print int($1)" "v[s]"\t"$2}' | head -n 20
  421. ```
  422. ##### Tool: [inotifywait](https://en.wikipedia.org/wiki/GNU_Screen)
  423. ###### Init tool everytime a file in a directory is modified
  424. ```bash
  425. while true ; do inotifywait -r -e MODIFY dir/ && ls dir/ ; done;
  426. ```
  427. ##### Tool: [openssl](https://www.openssl.org/)
  428. ###### Testing connection to remote host
  429. ```bash
  430. echo | openssl s_client -connect google.com:443 -showcerts
  431. ```
  432. ###### Testing connection to remote host with specific ssl version
  433. ```bash
  434. openssl s_client -tls1_2 -connect google.com:443
  435. ```
  436. ###### Testing connection to remote host with specific ssl cipher
  437. ```bash
  438. openssl s_client -cipher 'AES128-SHA' -connect google.com:443
  439. ```
  440. ###### Generate private key
  441. ```bash
  442. # _ciph: des3, aes
  443. ( _ciph="des3" ; _fd="private.key" ; _len="2048" ; \
  444. openssl genrsa -${_ciph} -out ${_fd} ${_len} )
  445. ```
  446. ###### Remove password from private key
  447. ```bash
  448. ( _fd="private.key" ; _fd_unp="private_unp.key" ; \
  449. openssl rsa -in ${_fd} -out ${_fd_unp} )
  450. ```
  451. ###### Get public key from private key
  452. ```bash
  453. ( _fd="private.key" ; _fd_pub="public.key" ; \
  454. openssl rsa -pubout -in ${_fd} -out ${_fd_pub} )
  455. ```
  456. ###### Generate private key + csr
  457. ```bash
  458. ( _fd="private.key" ; _fd_csr="request.csr" ; _len="2048" ; \
  459. openssl req -out ${_fd_csr} -new -newkey rsa:${_len} -nodes -keyout ${_fd} )
  460. ```
  461. ###### Generate csr
  462. ```bash
  463. ( _fd="private.key" ; _fd_csr="request.csr" ; \
  464. openssl req -out ${_fd_csr} -new -key ${_fd} )
  465. ```
  466. ###### Generate csr (metadata from exist certificate)
  467. ```bash
  468. ( _fd="private.key" ; _fd_csr="request.csr" ; _fd_crt="cert.crt" ; \
  469. openssl x509 -x509toreq -in ${_fd_crt} -out ${_fd_csr} -signkey ${_fd} )
  470. ```
  471. ###### Generate csr with -config param
  472. ```bash
  473. ( _fd="private.key" ; _fd_csr="request.csr" ; \
  474. openssl req -new -sha256 -key ${_fd} -out ${_fd_csr} \
  475. -config <(
  476. cat <<-EOF
  477. [req]
  478. default_bits = 2048
  479. prompt = no
  480. default_md = sha256
  481. req_extensions = req_ext
  482. distinguished_name = dn
  483. [ dn ]
  484. C=<two-letter ISO abbreviation for your country>
  485. ST=<state or province where your organization is legally located>
  486. L=<city where your organization is legally located>
  487. O=<legal name of your organization>
  488. OU=<section of the organization>
  489. CN=<fully qualified domain name>
  490. [ req_ext ]
  491. subjectAltName = @alt_names
  492. [ alt_names ]
  493. DNS.1 = <fully qualified domain name>
  494. DNS.2 = <next domain>
  495. DNS.3 = <next domain>
  496. EOF
  497. ))
  498. ```
  499. ###### Convert DER to PEM
  500. ```bash
  501. ( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \
  502. openssl x509 -in ${_fd_der} -inform der -outform pem -out ${_fd_pem} )
  503. ```
  504. ###### Convert PEM to DER
  505. ```bash
  506. ( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \
  507. openssl x509 -in ${_fd_pem} -outform der -out ${_fd_der} )
  508. ```
  509. ###### Checking whether the private key and the certificate match
  510. ```bash
  511. (openssl rsa -noout -modulus -in private.key | openssl md5 ; openssl x509 -noout -modulus -in certificate.crt | openssl md5) | uniq
  512. ```
  513. ___
  514. ##### Tool: [gnutls-cli](https://gnutls.org/manual/html_node/gnutls_002dcli-Invocation.html)
  515. ###### Testing connection to remote host (with sni)
  516. ```bash
  517. gnutls-cli -p 443 google.com
  518. ```
  519. ###### Testing connection to remote host (without sni)
  520. ```bash
  521. gnutls-cli --disable-sni -p 443 google.com
  522. ```
  523. <a name="http-https"><b>HTTP/HTTPS</b></a>
  524. ##### Tool: [curl](https://curl.haxx.se)
  525. ```bash
  526. curl -Iks https://www.google.com
  527. ```
  528. * `-I` - show response headers only
  529. * `-k` - insecure connection when using ssl
  530. * `-s` - silent mode (not display body)
  531. ```bash
  532. curl -Iks --location -X GET -A "x-agent" https://www.google.com
  533. ```
  534. * `--location` - follow redirects
  535. * `-X` - set method
  536. * `-A` - set user-agent
  537. ```bash
  538. curl -Iks --location -X GET -A "x-agent" --proxy http://127.0.0.1:16379 https://www.google.com
  539. ```
  540. * `--proxy [socks5://|http://]` - set proxy server
  541. ___
  542. ##### Tool: [httpie](https://httpie.org/)
  543. ```bash
  544. http -p Hh https://www.google.com
  545. ```
  546. * `-p` - print request and response headers
  547. * `H` - request headers
  548. * `B` - request body
  549. * `h` - response headers
  550. * `b` - response body
  551. ```bash
  552. http -p Hh --follow --max-redirects 5 --verify no https://www.google.com
  553. ```
  554. * `-F, --follow` - follow redirects
  555. * `--max-redirects N` - maximum for `--follow`
  556. * `--verify no` - skip SSL verification
  557. ```bash
  558. http -p Hh --follow --max-redirects 5 --verify no --proxy http:http://127.0.0.1:16379 https://www.google.com
  559. ```
  560. * `--proxy [http:]` - set proxy server
  561. <a name="network"><b>Network</b></a>
  562. ##### Tool: [ssh](https://www.openssh.com/)
  563. ###### Compare a remote file with a local file
  564. ```bash
  565. ssh user@host cat /path/to/remotefile | diff /path/to/localfile -
  566. ```
  567. ###### SSH connection through host in the middle
  568. ```bash
  569. ssh -t reachable_host ssh unreachable_host
  570. ```
  571. ###### Run command over ssh on remote host
  572. ```bash
  573. cat > cmd.txt << __EOF__
  574. cat /etc/hosts
  575. __EOF__
  576. ssh host -l user $(<cmd.txt)
  577. ```
  578. ###### Get public key from private key
  579. ```bash
  580. ssh-keygen -y -f ~/.ssh/id_rsa
  581. ```
  582. ___
  583. ##### Tool: [linux-dev](https://www.tldp.org/LDP/abs/html/devref1.html)
  584. ###### Testing remote connection to port
  585. ```bash
  586. timeout 1 bash -c "</dev/<proto>/<host>/<port>" >/dev/null 2>&1 ; echo $?
  587. ```
  588. * `<proto` - set protocol (tcp/udp)
  589. * `<host>` - set remote host
  590. * `<port>` - set destination port
  591. ###### Read and write to TCP or UDP sockets with common bash tools
  592. ```bash
  593. exec 5<>/dev/tcp/<host>/<port>; cat <&5 & cat >&5; exec 5>&-
  594. ```
  595. ___
  596. ##### Tool: [tcpdump](http://www.tcpdump.org/)
  597. ```bash
  598. tcpdump -ne -i eth0 -Q in host 192.168.252.1 and port 443
  599. ```
  600. * `-n` - don't convert addresses
  601. * `-e` - print the link-level headers
  602. * `-i [iface]` - set interface
  603. * `-Q|-D [in|out|inout]` - choose send/receive direction (`-D` - for old tcpdump versions)
  604. * `host [ip|hostname]` - set host, also `[host not]`
  605. * `[and|or]` - set logic
  606. * `port [1-65535]` - set port number, also `[port not]`
  607. ```bash
  608. tcpdump -ne -i eth0 -Q in host 192.168.252.1 and port 443 -c 5 -w tcpdump.pcap
  609. ```
  610. * `-c [num]` - capture only num number of packets
  611. * `-w [filename]` - write packets to file, `-r [filename]` - reading from file
  612. ___
  613. ##### Tool: [ngrep](http://ngrep.sourceforge.net/usage.html)
  614. ```bash
  615. ngrep -d eth0 "www.google.com" port 443
  616. ```
  617. * `-d [iface|any]` - set interface
  618. * `[domain]` - set hostname
  619. * `port [1-65535]` - set port number
  620. ```bash
  621. ngrep -d eth0 "www.google.com" (host 10.240.20.2) and (port 443)
  622. ```
  623. * `(host [ip|hostname])` - filter by ip or hostname
  624. * `(port [1-65535])` - filter by port number
  625. ```bash
  626. ngrep -d eth0 -qt -O ngrep.pcap "www.google.com" port 443
  627. ```
  628. * `-q` - quiet mode (only payloads)
  629. * `-t` - added timestamps
  630. * `-O [filename]` - save output to file, `-I [filename]` - reading from file
  631. ```bash
  632. ngrep -d eth0 -qt 'HTTP' 'tcp'
  633. ```
  634. * `HTTP` - show http headers
  635. * `tcp|udp` - set protocol
  636. * `[src|dst] host [ip|hostname]` - set direction for specific node
  637. ```bash
  638. ngrep -l -q -d eth0 -i "User-Agent: curl*"
  639. ```
  640. * `-l` - stdout line buffered
  641. * `-i` - case-insensitive search
  642. ___
  643. ##### Tool: [hping3](http://www.hping.org/)
  644. ```bash
  645. hping3 -V -p 80 -s 5050 <scan_type> www.google.com
  646. ```
  647. * `-V|--verbose` - verbose mode
  648. * `-p|--destport` - set destination port
  649. * `-s|--baseport` - set source port
  650. * `<scan_type>` - set scan type
  651. * `-F|--fin` - set FIN flag, port open if no reply
  652. * `-S|--syn` - set SYN flag
  653. * `-P|--push` - set PUSH flag
  654. * `-A|--ack` - set ACK flag (use when ping is blocked, RST response back if the port is open)
  655. * `-U|--urg` - set URG flag
  656. * `-Y|--ymas` - set Y unused flag (0x80 - nullscan), port open if no reply
  657. * `-M 0 -UPF` - set TCP sequence number and scan type (URG+PUSH+FIN), port open if no reply
  658. ```bash
  659. hping3 -V -c 1 -1 -C 8 www.google.com
  660. ```
  661. * `-c [num]` - packet count
  662. * `-1` - set ICMP mode
  663. * `-C|--icmptype [icmp-num]` - set icmp type (default icmp-echo = 8)
  664. ```bash
  665. hping3 -V -c 1000000 -d 120 -S -w 64 -p 80 --flood --rand-source <remote_host>
  666. ```
  667. * `--flood` - sent packets as fast as possible (don't show replies)
  668. * `--rand-source` - random source address mode
  669. * `-d --data` - data size
  670. * `-w|--win` - winsize (default 64)
  671. ___
  672. ##### Tool: [netcat](http://netcat.sourceforge.net/)
  673. ```bash
  674. nc -kl 5000
  675. ```
  676. * `-l` - listen for an incoming connection
  677. * `-k` - listening after client has disconnected
  678. * `>filename.out` - save receive data to file (optional)
  679. ```bash
  680. nc 192.168.0.1 5051 < filename.in
  681. ```
  682. * `< filename.in` - send data to remote host
  683. ```bash
  684. nc -vz 10.240.30.3 5000
  685. ```
  686. * `-v` - verbose output
  687. * `-z` - scan for listening daemons
  688. ```bash
  689. nc -vzu 10.240.30.3 1-65535
  690. ```
  691. * `-u` - scan only udp ports
  692. ###### Transfer data file (archive)
  693. ```bash
  694. server> nc -l 5000 | tar xzvfp -
  695. client> tar czvfp - /path/to/dir | nc 10.240.30.3 5000
  696. ```
  697. ###### Launch remote shell
  698. ```bash
  699. server> nc -l 5000 -e /bin/bash
  700. client> nc 10.240.30.3 5000
  701. ```
  702. ###### Simple file server
  703. ```bash
  704. while true ; do nc -l 5000 | tar -xvf - ; done
  705. ```
  706. ###### Simple HTTP Server
  707. > Restarts web server after each request - remove `while` condition for only single connection.
  708. ```bash
  709. cat > index.html << __EOF__
  710. <!doctype html>
  711. <head>
  712. <meta charset="utf-8">
  713. <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
  714. <title></title>
  715. <meta name="description" content="">
  716. <meta name="viewport" content="width=device-width, initial-scale=1">
  717. </head>
  718. <body>
  719. <p>
  720. Hello! It's a site.
  721. </p>
  722. </body>
  723. </html>
  724. __EOF__
  725. ```
  726. ```bash
  727. server> while : ; do \
  728. (echo -ne "HTTP/1.1 200 OK\r\nContent-Length: $(wc -c <index.html)\r\n\r\n" ; cat index.html;) \
  729. | nc -l -p 5000 \
  730. ; done
  731. ```
  732. * `-p` - port number
  733. ###### Simple HTTP Proxy (single connection)
  734. ```bash
  735. #!/usr/bin/env bash
  736. if [[ $# != 2 ]] ; then
  737. printf "%s\\n" \
  738. "usage: ./nc-proxy listen-port bk_host:bk_port"
  739. fi
  740. _listen_port="$1"
  741. _bk_host=$(echo "$2" | cut -d ":" -f1)
  742. _bk_port=$(echo "$2" | cut -d ":" -f2)
  743. printf " lport: %s\\nbk_host: %s\\nbk_port: %s\\n\\n" \
  744. "$_listen_port" "$_bk_host" "$_bk_port"
  745. _tmp=$(mktemp -d)
  746. _back="$_tmp/pipe.back"
  747. _sent="$_tmp/pipe.sent"
  748. _recv="$_tmp/pipe.recv"
  749. trap 'rm -rf "$_tmp"' EXIT
  750. mkfifo -m 0600 "$_back" "$_sent" "$_recv"
  751. sed "s/^/=> /" <"$_sent" &
  752. sed "s/^/<= /" <"$_recv" &
  753. nc -l -p "$_listen_port" <"$_back" \
  754. | tee "$_sent" \
  755. | nc "$_bk_host" "$_bk_port" \
  756. | tee "$_recv" >"$_back"
  757. ```
  758. ```bash
  759. server> chmod +x nc-proxy && ./nc-proxy 8080 192.168.252.10:8000
  760. lport: 8080
  761. bk_host: 192.168.252.10
  762. bk_port: 8000
  763. client> http -p h 10.240.30.3:8080
  764. HTTP/1.1 200 OK
  765. Accept-Ranges: bytes
  766. Cache-Control: max-age=31536000
  767. Content-Length: 2748
  768. Content-Type: text/html; charset=utf-8
  769. Date: Sun, 01 Jul 2018 20:12:08 GMT
  770. Last-Modified: Sun, 01 Apr 2018 21:53:37 GMT
  771. ```
  772. ###### Create a single-use TCP or UDP proxy
  773. ```bash
  774. ### TCP -> TCP
  775. nc -l -p 2000 -c "nc [ip|hostname] 3000"
  776. ### TCP -> UDP
  777. nc -l -p 2000 -c "nc -u [ip|hostname] 3000"
  778. ### UDP -> UDP
  779. nc -l -u -p 2000 -c "nc -u [ip|hostname] 3000"
  780. ### UDP -> TCP
  781. nc -l -u -p 2000 -c "nc [ip|hostname] 3000"
  782. ```
  783. ___
  784. ##### Tool: [socat](http://www.dest-unreach.org/socat/doc/socat.html/)
  785. ###### Testing remote connection to port
  786. ```bash
  787. socat - TCP4:10.240.30.3:22
  788. ```
  789. * `-` - standard input (STDIO)
  790. * `TCP4:<params>` - set tcp4 connection with specific params
  791. * `[hostname|ip]` - set hostname/ip
  792. * `[1-65535]` - set port number
  793. ###### Redirecting TCP-traffic to a UNIX domain socket under Linux
  794. ```bash
  795. socat TCP-LISTEN:1234,bind=127.0.0.1,reuseaddr,fork,su=nobody,range=127.0.0.0/8 UNIX-CLIENT:/tmp/foo
  796. ```
  797. * `TCP-LISTEN:<params>` - set tcp listen with specific params
  798. * `[1-65535]` - set port number
  799. * `bind=[hostname|ip]` - set bind hostname/ip
  800. * `reuseaddr` - allows other sockets to bind to an address
  801. * `fork` - keeps the parent process attempting to produce more connections
  802. * `su=nobody` - set user
  803. * `range=[ip-range]` - ip range
  804. * `UNIX-CLIENT:<params>` - communicates with the specified peer socket
  805. * `filename` - define socket
  806. ___
  807. ##### Tool: [lsof](https://en.wikipedia.org/wiki/Lsof)
  808. ###### Show process that use internet connection at the moment
  809. ```bash
  810. lsof -P -i -n
  811. ```
  812. ###### Show process that use specific port number
  813. ```bash
  814. lsof -i tcp:443
  815. ```
  816. ###### Lists all listening ports together with the PID of the associated process
  817. ```bash
  818. lsof -Pan -i tcp -i udp
  819. ```
  820. ###### List all open ports and their owning executables
  821. ```bash
  822. lsof -i -P | grep -i "listen"
  823. ```
  824. ###### Show open ports
  825. ```bash
  826. lsof -Pni4 | grep LISTEN | column -t
  827. ```
  828. ###### List all files opened by a particular command
  829. ```bash
  830. lsof -c "process"
  831. ```
  832. ###### View user activity per directory
  833. ```bash
  834. lsof -u username -a +D /etc
  835. ```
  836. ___
  837. **Tool: [netstat](https://en.wikipedia.org/wiki/Netstat)**
  838. ###### Graph # of connections for each hosts
  839. ```bash
  840. netstat -an | grep ESTABLISHED | awk '{print $5}' | awk -F: '{print $1}' | grep -v -e '^[[:space:]]*$' | sort | uniq -c | awk '{ printf("%s\t%s\t",$2,$1) ; for (i = 0; i < $1; i++) {printf("*")}; print "" }'
  841. ```
  842. ###### Monitor open connections for specific port including listen, count and sort it per IP
  843. ```bash
  844. watch "netstat -plan | grep :443 | awk {'print \$5'} | cut -d: -f 1 | sort | uniq -c | sort -nk 1"
  845. ```
  846. ___
  847. **Tool: [rsync](https://en.wikipedia.org/wiki/Rsync)**
  848. ###### Rsync remote data as root using sudo
  849. ```bash
  850. rsync --rsync-path 'sudo rsync' username@hostname:/path/to/dir/ /local/
  851. ```
  852. <a name="programming"><b>Programming</b></a>
  853. ##### Tool: [awk](http://www.grymoire.com/Unix/Awk.html)
  854. ###### Remove duplicate entries in a file without sorting
  855. ```bash
  856. awk '!x[$0]++' filename
  857. ```
  858. ###### Exclude multiple columns using AWK
  859. ```bash
  860. awk '{$1=$3=""}1' filename
  861. ```
  862. ___
  863. ##### Tool: [sed](http://www.grymoire.com/Unix/Sed.html)
  864. ###### To print a specific line from a file
  865. ```bash
  866. sed -n 10p /path/to/file
  867. ```
  868. ###### Remove a specific line from a file
  869. ```bash
  870. sed -i 10d /path/to/file
  871. ```
  872. ###### Remove a range of lines from a file
  873. ```bash
  874. sed -i <file> -re '<start>,<end>d'
  875. ```
  876. ___
  877. ##### Tool: [grep](http://www.grymoire.com/Unix/Grep.html)
  878. ###### Search for a "pattern" inside all files in the current directory
  879. ```bash
  880. grep -RnisI "pattern" *
  881. fgrep "pattern" * -R
  882. ```
  883. ###### Remove blank lines from a file and save output to new file
  884. ```bash
  885. grep . filename > newfilename
  886. ```
  887. ###### Except multiple patterns
  888. ```bash
  889. grep -vE '(error|critical|warning)' filename
  890. ```
  891. ###### Show data from file without comments
  892. ```bash
  893. grep -v ^[[:space:]]*# filename
  894. ```
  895. ###### Show data from file without comments and new lines
  896. ```bash
  897. egrep -v '#|^$' filename
  898. ```