A next-generation crawling and spidering framework.
-ps, -passive enable passive sources to discover target endpoints
-pss, -passive-source string[] passive source to use for url discovery (waybackarchive,commoncrawl,alienvault)
katana -u tesla.com -passive -f qurl
...
https://static-assets-pay.tesla.com/api/payment-schema/creditcard?countrycode=dk
https://static-assets-pay.tesla.com/api/paymenttypes/?clientlibrary=payment-website
https://tradepartnertickets.tesla.com/dist/main.css?029b26e9be3aef4fc82c
https://tradepartnertickets.tesla.com/dist/vendors.vendors.css?029b26e9be3aef4fc82c
https://workforce.tesla.com/auth/callback?code=na_921f11c72db1d416c2fb624ea94ab5e1bad5f803
...
...
...
[INF] Found 208114 endpoints for https://tesla.com in 2m11.65937825s (commoncrawl: 128, alienvault: 1615, waybackarchive: 206371)
-e, -exclude string[] exclude host matching specified filter ('cdn', 'private-ips', cidr, ip, regex)
-kf option to skip if the maximum depth is exceeded by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/799
Full Changelog: https://github.com/projectdiscovery/katana/compare/v1.0.5...v1.1.0
-disable-redirects by @ErikOwen in https://github.com/projectdiscovery/katana/pull/630
-dr, -disable-redirects disable following redirects (default false)
-aff config option by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/676
-kf option by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/655
Full Changelog: https://github.com/projectdiscovery/katana/compare/v1.0.4...v1.0.5
-fs option by @c3l3si4n in https://github.com/projectdiscovery/katana/pull/571
katana -fs '(company-staging.io|company.com)' -u company.com
-dr, -disable-redirects disable following redirects (default false)
-resume crawl option by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/538
-resume string resume crawl using resume.cfg
-match-regex option by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/565
ExtractRelativeEndpoints() in scriptJSFileRegexParser() by @0x123456789 in https://github.com/projectdiscovery/katana/pull/587
go-tree-sitter -> tree-sitter by @dogancanbakir in https://github.com/projectdiscovery/katana/pull/547
Full Changelog: https://github.com/projectdiscovery/katana/compare/v1.0.3...v1.0.4
-mdc, -match-condition string match response with dsl based condition
-fdc, -filter-condition string filter response with dsl based condition
-tlsi, -tls-impersonate enable experimental client hello (ja3) tls randomization
-jsl, -jsluice enable jsluice parsing in javascript file (memory intensive)
-fx, -form-extraction extract form, input, textarea & select elements in jsonl output
-xhr, -xhr-extraction extract xhr request url,method in jsonl output
-j used with -sr option by @ErikOwen in https://github.com/projectdiscovery/katana/pull/532
-ct option by @RamanaReddy0M in https://github.com/projectdiscovery/katana/pull/500
-ct, -crawl-duration value maximum duration to crawl the target for (s, m, h, d) (default s)
-cwu, -chrome-ws-url string use chrome browser instance launched elsewhere with the debugger listening at this URL
Full Changelog: https://github.com/projectdiscovery/katana/compare/v1.0.2...v1.0.3
katana -H cookie.txt -u https://mail.google.com -headless
-or, -omit-raw omit raw requests/responses from jsonl output
-ob, -omit-body omit response body from jsonl output
Issues closed in this release - https://github.com/projectdiscovery/katana/milestone/6?closed=1
Full Changelog: https://github.com/projectdiscovery/katana/compare/v1.0.1...v1.0.2
-iqp, -ignore-query-params ignore crawling same path with different query-param values
katana -u hackerone.com
net/url with urlutil by @iamargus95 in https://github.com/projectdiscovery/katana/pull/399
-do option by @Mzack9999 in https://github.com/projectdiscovery/katana/pull/387
Issues closed in this release - https://github.com/projectdiscovery/katana/milestone/5?closed=1
Full Changelog: https://github.com/projectdiscovery/katana/compare/v1.0.0...v1.0.1
Warning: breaking changes in json output structure.
{
"timestamp": "2022-11-05T22:33:27.745815+05:30",
"endpoint": "https://www.iana.org/domains/example",
"source": "https://example.com",
"tag": "a",
"attribute": "href"
}
{
"timestamp": "2023-03-20T16:23:58.027559+05:30",
"request": {
"method": "GET",
"endpoint": "https://example.com",
"tag": "a",
"attribute": "href",
"source": "http://www.iana.org/domains/reserved",
"raw": "GET / HTTP/1.1\r\nHost: example.com\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36\r\nAccept-Encoding: gzip\r\n\r\n"
},
"response": {
"status_code": 200,
"headers": {
"accept_ranges": "bytes",
"expires": "Mon, 27 Mar 2023 10:53:58 GMT",
"last_modified": "Thu, 17 Oct 2019 07:18:26 GMT",
"content_type": "text/html; charset=UTF-8",
"server": "ECS (dcb/7EA3)",
"vary": "Accept-Encoding",
"etag": "\"3147526947\"",
"cache_control": "max-age=604800",
"x_cache": "HIT",
"date": "Mon, 20 Mar 2023 10:53:58 GMT",
"age": "331239"
},
"body": "<!doctype html>\n<html>\n<head>\n <title>Example Domain</title>\n\n <meta charset=\"utf-8\" />\n <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n <style type=\"text/css\">\n body {\n background-color: #f0f0f2;\n margin: 0;\n padding: 0;\n font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n \n }\n div {\n width: 600px;\n margin: 5em auto;\n padding: 2em;\n background-color: #fdfdff;\n border-radius: 0.5em;\n box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n }\n a:link, a:visited {\n color: #38488f;\n text-decoration: none;\n }\n @media (max-width: 700px) {\n div {\n margin: 0 auto;\n width: auto;\n }\n }\n </style> \n</head>\n\n<body>\n<div>\n <h1>Example Domain</h1>\n <p>This domain is for use in illustrative examples in documents. You may use this\n domain in literature without prior coordination or asking for permission.</p>\n <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n",
"technologies": [
"Azure",
"Amazon ECS",
"Amazon Web Services",
"Docker",
"Azure CDN"
],
"raw": "HTTP/1.1 200 OK\r\nContent-Length: 1256\r\nAccept-Ranges: bytes\r\nAge: 331239\r\nCache-Control: max-age=604800\r\nContent-Type: text/html; charset=UTF-8\r\nDate: Mon, 20 Mar 2023 10:53:58 GMT\r\nEtag: \"3147526947\"\r\nExpires: Mon, 27 Mar 2023 10:53:58 GMT\r\nLast-Modified: Thu, 17 Oct 2019 07:18:26 GMT\r\nServer: ECS (dcb/7EA3)\r\nVary: Accept-Encoding\r\nX-Cache: HIT\r\n\r\n<!doctype html>\n<html>\n<head>\n <title>Example Domain</title>\n\n <meta charset=\"utf-8\" />\n <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n <style type=\"text/css\">\n body {\n background-color: #f0f0f2;\n margin: 0;\n padding: 0;\n font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n \n }\n div {\n width: 600px;\n margin: 5em auto;\n padding: 2em;\n background-color: #fdfdff;\n border-radius: 0.5em;\n box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n }\n a:link, a:visited {\n color: #38488f;\n text-decoration: none;\n }\n @media (max-width: 700px) {\n div {\n margin: 0 auto;\n width: auto;\n }\n }\n </style> \n</head>\n\n<body>\n<div>\n <h1>Example Domain</h1>\n <p>This domain is for use in illustrative examples in documents. You may use this\n domain in literature without prior coordination or asking for permission.</p>\n <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n"
}
}
...
"technologies": [
"Azure",
"Amazon ECS",
"Amazon Web Services",
"Docker",
"Azure CDN"
],
...
-mr, -match-regex string[] regex or list of regex to match on output url (cli, file)
-fr, -filter-regex string[] regex or list of regex to filter on output url (cli, file)
-r, -resolvers string[] list of custom resolver (file or comma separated)
-up, -update update katana to latest version
ufile field based filter by @Mzack9999 in https://github.com/projectdiscovery/katana/pull/357
katana -f ufile -u https://www.tesla.com
-chrome-data-dir / -cdd flag to implement authenticated crawling #230 by @parthmalhotra in https://github.com/projectdiscovery/katana/pull/231
rdn, fqdn, etc. scope filter in readme by @olearycrew in https://github.com/projectdiscovery/katana/pull/288
Dockerfile to avoid version mismatch by @olearycrew in https://github.com/projectdiscovery/katana/pull/279
Issues closed in release - https://github.com/projectdiscovery/katana/milestone/4
Full Changelog: https://github.com/projectdiscovery/katana/compare/v0.0.3...v1.0.0
-scp) by @parthmalhotra in https://github.com/projectdiscovery/katana/pull/202
-elog) option by @Mzack9999 in https://github.com/projectdiscovery/katana/pull/267
-hc) option by @Mzack9999 in https://github.com/projectdiscovery/katana/pull/260
Issues closed in release - https://github.com/projectdiscovery/katana/milestone/3?closed=1
Full Changelog: https://github.com/projectdiscovery/katana/compare/v0.0.2...v0.0.3
-no-sandbox option to run headless mode as root user by @edoardottt in https://github.com/projectdiscovery/katana/pull/132
-headless-options option to include additional chrome options by @edoardottt in https://github.com/projectdiscovery/katana/pull/132
-version option by @mschader in https://github.com/projectdiscovery/katana/pull/140
Full Changelog: https://github.com/projectdiscovery/katana/compare/v0.0.1...v0.0.2