Pown Recon 一个由图论提供支持的强大目标侦察框架

Pown Recon 一个由图论提供支持的强大目标侦察框架

Pown Recon

Pown Recon is a target reconnaissance framework powered by graph theory. The benefit of using graph theory instead of flat table representation is that it is easier to find the relationships between different types of information which comes quite handy in many situations. Graph theory algorithms also help with diffing, searching, like finding the shortest path, and many other helpful tasks to aid information discovery and intelligence gathering.

Credits

This tool is part of secapps.com open-source initiative.

___ ___ ___ _ ___ ___ ___ / __| __/ __| /_\ | _ \ _ \/ __| \__ \ _| (__ / _ \| _/ _/\__ \ |___/___\___/_/ \_\_| |_| |___/ https://secapps.com

NB: Pown Recon is the result of an almost direct copy of SecApps' excellent Recon tool.

Quickstart

This tool is meant to be used as part of Pown.js but it can be invoked separately as an independent tool.

Install Pown first as usual:

$ npm install -g pown@latest

Invoke directly from Pown:

$ pown recon

Otherwise, install this module locally from the root of your project:

$ npm install @pown/recon --save

Once done, invoke pown cli:

$ POWN_ROOT=. ./node_modules/.bin/pown-cli recon

You can also use the global pown to invoke the tool locally:

$ POWN_ROOT=. pown recon

Usage

WARNING: This pown command is currently under development and as a result will be subject to breaking changes.

pown recon Target reconCommands: pown recon transform Perform inline transformation [aliases: t] pown recon select Select nodes [aliases: s] pown recon traverse Traverse nodes [aliases: v] pown recon options Manage options pown recon add Add nodes [aliases: a] pown recon remove Remove nodes [aliases: r] pown recon merge Perform a merge between at least two recon files [aliases: m] pown recon diff Perform a diff between two recon files [aliases: d] pown recon group Group nodes [aliases: g] pown recon ungroup Ungroup nodes [aliases: u] pown recon load Load a file [aliases: l] pown recon save Save to file [aliases: o] pown recon import Import file [aliases: i] pown recon export Export to file [aliases: e] pown recon remote Remote managment [aliases: f] pown recon layout Layout the graph [aliases: r] pown recon summary Create a summary [aliases: y]Options: --version Show version number [boolean] --help Show help [boolean]

pown recon transform

pown recon transform Perform inline transformationCommands: pown recon transform archiveindex [options] Obtain archive.org index for specific URL. [aliases: archive_index, arci] pown recon transform bitbucketlistrepos [options] List Bitbucket repositories. [aliases: bitbucket_list_repos, bblr] pown recon transform bitbucketlistsnippets [options] List Bitbucket snippets. [aliases: bitbucket_list_snippets, bbls] pown recon transform bitbucketlistteamrepos [options] List Bitbucket team repos. [aliases: bitbucket_list_team_repos, bbltr] pown recon transform bitbucketlistteammembers [options] List Bitbucket team members. [aliases: bitbucket_list_team_members, bbltm] pown recon transform bufferoverrunsubdomainsearch [options] Obtain a list of subdomains using bufferover.run DNS service [aliases: bufferoverrun_subdomain_search, brss] pown recon transform builtwithscraperelationships [options] Performs scrape of builtwith.com relationships. [aliases: builtwith_scrape_relationships, bwsr] pown recon transform certspotterissuances [options] Obtain issuances from Certspotter. [aliases: certspotter_issuances, csi] pown recon transform cloudflarednsquery [options] Query CloudFlare DNS API. [aliases: cloudflare_dns_query, cfdq] pown recon transform commoncrawlindex [options] Obtain a CommonCraw index for specific URL. [aliases: commoncrawl_index, cci] pown recon transform crtshcndomainreport [options] Obtain crt.sh domain report which helps enumerating potential target subdomains. [aliases: crtsh_cn_domain_report, crtshcdr] pown recon transform crtshsandomainreport [options] Obtain crt.sh domain report which helps enumerating potential target subdomains. [aliases: crtsh_san_domain_report, crtshsdr] pown recon transform bakeimages [options] Convert external image into data URIs for self-embedding purposes. [aliases: bake_images, be] pown recon transform dnsresolve [options] Does not do anything. [aliases: dr, dns] pown recon transform dockerhublistrepos [options] List DockerHub repositories for a given member or org. [aliases: dockerhub_list_repos, dhlr] pown recon transform githubuser [options] Map GitHub user for a given term. [aliases: github_user, ghu] pown recon transform githuborg [options] Map GitHub org for a given term. [aliases: github_org, gho] pown recon transform githubsearchusers [options] Search GitHub users for a given term. [aliases: github_search_users, ghsu] pown recon transform githublistorgs [options] List GitHub orgs for a given member. [aliases: github_list_orgs, ghlo] pown recon transform githublistmembers [options] List GitHub members for a given org. [aliases: github_list_members, ghlm] pown recon transform githublistrepos [options] List GitHub repos for a given user/org. [aliases: github_list_repos, ghlr] pown recon transform githublistgists [options] List GitHub gists for a given user/org. [aliases: github_list_gists, ghlg] pown recon transform gravatar [options] Get gravatar. pown recon transform hackertargetreverseiplookup [options] Obtain reverse IP information from hackertarget.com. [aliases: hackertarget_reverse_ip_lookup, htril] pown recon transform hackertargetonlineportscan [options] Obtain port information from hackertarget.com. [aliases: hackertarget_online_port_scan, htps] pown recon transform hibpreport [options] Obtain haveibeenpwned.com breach report. [aliases: hibp_report, hibpr] pown recon transform httpfingerprint [options] Performs a fingerprint on the HTTP server and application. [aliases: http_fingerprint, hf] pown recon transform pkslookupkeys [options] Look the the PKS database at pool.sks-keyservers- which pgp.mit.edu is part of. [aliases: pks_lookup_keys, pkslk] pown recon transform pwndbsearch [options] Searching the PownDB database [aliases: pwndb_search, pds] pown recon transform riddleripsearch [options] Searches for IP references using F-Secure riddler.io. [aliases: riddler_ip_search, rdis] pown recon transform riddlerdomainsearch [options] Searches for Domain references using F-Secure riddler.io. [aliases: riddler_domain_search, rdds] pown recon transform securitytrailssuggestions [options] Get a list of domain suggestions from securitytrails.com. [aliases: securitytrails_domain_suggestions, stds] pown recon transform securitytrailsdomainreport [options] Get a domain report from securitytrails.com. [aliases: securitytrails_domain_report, stdr] pown recon transform shodanorgsearch [options] Performs search using ORG filter. [aliases: shodan_org_search, sos] pown recon transform shodansslsearch [options] Performs search using SSL filter. [aliases: shodan_ssl_search, sss] pown recon transform tcpportscan [options] Simple, full-handshake TCP port scanner (very slow and perhaps inaccurate) [aliases: tcp_port_scan, tps] pown recon transform threatcrowddomainreport [options] Obtain threatcrowd domain report which helps enumerating potential target subdomains and email addresses. [aliases: threatcrowd_domain_report, tcdr] pown recon transform threatcrowdipreport [options] Obtain threatcrowd ip report which helps enumerating virtual hosts. [aliases: threatcrowd_ip_report, tcir] pown recon transform urlscanliveshot [options] Generates a liveshot of any public site via urlscan. [aliases: urlscan_liveshot, usls] pown recon transform urlscanliveshotsearchengines [options] Generates a liveshot of a number of search engines. [aliases: urlscan_liveshot_search_engines, uslsse] pown recon transform noop [options] Does not do anything. [aliases: nop] pown recon transform sleep [options] Sleeps for predefined time. [aliases: sleep] pown recon transform duplicate [options] Duplicate input. [aliases: dup] pown recon transform extract [options] Extract property. [aliases: ext] pown recon transform prefix [options] Adds a prefix. [aliases: prepand] pown recon transform suffix [options] Adds a suffix. [aliases: append] pown recon transform augment [options] Augment with prefix or suffix. pown recon transform splitemail [options] Split email. [aliases: split_email, se] pown recon transform buildemail [options] Build email. [aliases: build_email, be] pown recon transform splitdomain [options] Split domain. [aliases: split_domain, ss] pown recon transform builddomain [options] Build domain. [aliases: build_domain, bd] pown recon transform splituri [options] Split URI. [aliases: split_uri, su] pown recon transform builduri [options] Build URI. [aliases: build_uri, bu] pown recon transform virustotalsubdomains [options] Obtain subdomains from Virustotal. [aliases: virustotal_subdomains, vtsd] pown recon transform wappalyzerprofile [options] Enumerate technologies with api.wappalyzer.com. [aliases: wappalyzer_profile, wzp] pown recon transform whoaretheyreport [options] Find social accounts with the help of whoarethey database. [aliases: whoarethey_report, whoarethey, wmnr, wmn] pown recon transform auto [options] Select the most appropriate methods of transformationOptions: --version Show version number [boolean] --help Show help [boolean] --read, -r Read file [string] --write, -w Write file [string]

pown recon select

pown recon select Select nodesOptions: --version Show version number [boolean] --help Show help [boolean] --read, -r Read file [string] --write, -w Write file [string] --output-format, -o Output format [string] [choices: "table", "grid", "csv", "json"] [default: "table"] --output-fields Output fields [string] [default: ""] --output-ids Output ids [boolean] [default: false] --output-labels Output labels [boolean] [default: false] --output-images Output images [boolean] [default: false] --output-parents Output parents [boolean] [default: false] --output-tags Output tags [boolean] [default: false]

pown recon traverse

pown recon traverse Traverse nodesOptions: --version Show version number [boolean] --help Show help [boolean] --read, -r Read file [string] --write, -w Write file [string] --output-format, -o Output format [string] [choices: "table", "grid", "csv", "json"] [default: "table"] --output-fields Output fields [string] [default: ""] --output-ids Output ids [boolean] [default: false] --output-labels Output labels [boolean] [default: false] --output-images Output images [boolean] [default: false] --output-parents Output parents [boolean] [default: false] --output-tags Output tags [boolean] [default: false]

pown recon options

pown recon options Manage optionsCommands: pown recon options list List option [aliases: l] pown recon options set Set option [aliases: s] pown recon options get Get option [aliases: g] pown recon options delete Delete option [aliases: d] pown recon options clear Clear options [aliases: c]Options: --version Show version number [boolean] --help Show help [boolean]

pown recon add

pown recon add Add nodesOptions: --version Show version number [boolean] --help Show help [boolean] --group, -g Group nodes [string] [default: ""] --node-type The type for new nodes [string] [default: "string"] --node-props The props for new nodes [string] [default: ""] --node-props-file A file for the props for new nodes [string] [default: ""] --select, -s Select graph. Nodes will be added and linked only if graph contains at least one node. [string] [default: ""] --traverse, -v Traverse graph. Nodes will be added and linked only if graph contains at least one node. [string] [default: false] --read, -r Read file [string] --write, -w Write file [string] --output-format, -o Output format [string] [choices: "table", "grid", "csv", "json"] [default: "table"] --output-fields Output fields [string] [default: ""] --output-ids Output ids [boolean] [default: false] --output-labels Output labels [boolean] [default: false] --output-images Output images [boolean] [default: false] --output-parents Output parents [boolean] [default: false] --output-tags Output tags [boolean] [default: false]

pown recon remove

pown recon remove Remove nodesOptions: --version Show version number [boolean] --help Show help [boolean] --read, -r Read file [string] --write, -w Write file [string] --output-format, -o Output format [string] [choices: "table", "grid", "csv", "json"] [default: "table"] --output-fields Output fields [string] [default: ""] --output-ids Output ids [boolean] [default: false] --output-labels Output labels [boolean] [default: false] --output-images Output images [boolean] [default: false] --output-parents Output parents [boolean] [default: false] --output-tags Output tags [boolean] [default: false] --traverse, -v Traverse graph [boolean] [default: false]

pown recon merge

pown recon merge Perform a merge between at least two recon filesOptions: --version Show version number [boolean] --help Show help [boolean] --write, -w Write file [string]

pown recon diff

pown recon diff Perform a diff between two recon filesOptions: --version Show version number [boolean] --help Show help [boolean] --subset, -s The subset to select [choices: "left", "right", "both"] [default: "left"] --write, -w Write file [string] --output-format, -o Output format [string] [choices: "table", "grid", "csv", "json"] [default: "table"] --output-fields Output fields [string] [default: ""] --output-ids Output ids [boolean] [default: false] --output-labels Output labels [boolean] [default: false] --output-images Output images [boolean] [default: false] --output-parents Output parents [boolean] [default: false] --output-tags Output tags [boolean] [default: false]

pown recon group

pown recon group Group nodesOptions: --version Show version number [boolean] --help Show help [boolean] --read, -r Read file [string] --write, -w Write file [string] --output-format, -o Output format [string] [choices: "table", "grid", "csv", "json"] [default: "table"] --output-fields Output fields [string] [default: ""] --output-ids Output ids [boolean] [default: false] --output-labels Output labels [boolean] [default: false] --output-images Output images [boolean] [default: false] --output-parents Output parents [boolean] [default: false] --output-tags Output tags [boolean] [default: false] --traverse, -v Traverse graph [boolean] [default: false]

pown recon ungroup

pown recon ungroup Ungroup nodesOptions: --version Show version number [boolean] --help Show help [boolean] --read, -r Read file [string] --write, -w Write file [string] --output-format, -o Output format [string] [choices: "table", "grid", "csv", "json"] [default: "table"] --output-fields Output fields [string] [default: ""] --output-ids Output ids [boolean] [default: false] --output-labels Output labels [boolean] [default: false] --output-images Output images [boolean] [default: false] --output-parents Output parents [boolean] [default: false] --output-tags Output tags [boolean] [default: false] --traverse, -v Traverse graph [boolean] [default: false]

pown recon load

pown recon load Load a fileOptions: --version Show version number [boolean] --help Show help [boolean] --write, -w Write file [string]

pown recon save

pown recon save Save to fileOptions: --version Show version number [boolean] --help Show help [boolean] --read, -r Read file [string]

pown recon import

pown recon import Import fileOptions: --version Show version number [boolean] --help Show help [boolean] --file-type Import file type [choices: "json"] --node-type Imported nodes types [default: "string"] --id-field, -i The id field --type-field, -i The type field --label-field, -l The label field --image-field, -m The image field --group, -g Group nodes [string] [default: ""] --output-format, -o Output format [string] [choices: "table", "grid", "csv", "json"] [default: "table"] --output-fields Output fields [string] [default: ""] --output-ids Output ids [boolean] [default: false] --output-labels Output labels [boolean] [default: false] --output-images Output images [boolean] [default: false] --output-parents Output parents [boolean] [default: false] --output-tags Output tags [boolean] [default: false]

pown recon export

pown recon export Export to fileOptions: --version Show version number [boolean] --help Show help [boolean]

pown recon remote

pown recon remote Remote managmentCommands: pown recon remote list List remotes [aliases: l] pown recon remote add Add remote [aliases: a] pown recon remote remove Remove remote [aliases: a]Options: --version Show version number [boolean] --help Show help [boolean]

pown recon summary

pown recon summaryCreate a summaryOptions: --version Show version number [boolean] --help Show help [boolean] --read, -r Read file [string] --kind, -k Summary kind [string] [default: "type"] --summary-file-name, --summary-file Write summary to file [string] --summary-file-type, --summary-type Write summary to file with type [string] [default: "text"]

Preview

Generated graphs can be previewed in SecApps Recon for convenience, which this tool is based on. You can access SecApps Recon from your browser but you can also invoke it from the command line.

First you need @pown/apps installed:

$ pown modules install @pown/apps

This will install the optional apps command package.

Generate your graph using the write options:

$ pown recon transform auto -w path/to/file-work --node-type brand target

Once the recon is complete, open the graph for preview in SecApps Recon:

$ pown apps recon < path/to/file-work

You will see a screen similar to the following:

Scripting

Pown recon is designed to be scripted either via your favorite shell environment or Pown Script. If you use Pown Script you will benefit from preserved context between each command execution. This means that you can build a graph without then need to save and restore into intermediate files.

Using your favourite editor create a file called example.pown with the following contents:

echo This is scriptrecon add --node-type brand targetrecon t auto

Execute the script from pown:

$ pown script path/to/example.pown

For more information, see the examples.

Selectors

Some commands expect graph selectors. The rest of the documentation is copy of cytoscape.js selectors manual with some minor differences.

A selector functions similar to a CSS selector on DOM elements, but selectors in Recon instead work on collections of graph elements. This mechanism is provided by the powerful cytoscape.js.

The selectors can be combined together to make powerful queries, for example:

pown select 'node[weight >= 50][height < 180]'

Selectors can be joined together (effectively creating a logical OR) with commas:

pown select 'node#j, edge[source = "j"]'

It is important to note that strings need to be enclosed by quotation marks:

pown select 'node[type = "domain"]'

Note that metacharacters ( ^ $ \ / ( ) | ? + * [ ] { } , . ) need to be escaped:

pown select '#some\$funky\@id'

Group, class, & ID

node, edge, or * (group selector) Matches elements based on group (node for nodes, edge for edges, * for all)..className Matches elements that have the specified class (e.g. use .foo for a class named "foo").The #id Matches element with the matching ID (e.g. #foo is the same as [id = 'foo'])

Data

[name] Matches elements if they have the specified data attribute defined, i.e. not undefined (e.g. [foo] for an attribute named “foo”). Here, null is considered a defined value.[^name] Matches elements if the specified data attribute is not defined, i.e. undefined (e.g [^foo]). Here, null is considered a defined value.[?name] Matches elements if the specified data attribute is a truthy value (e.g. [?foo]).[!name] Matches elements if the specified data attribute is a falsey value (e.g. [!foo]).[name = value] Matches elements if their data attribute matches a specified value (e.g. [foo = 'bar'] or [num = 2]).[name != value] Matches elements if their data attribute doesn’t match a specified value (e.g. [foo != 'bar'] or [num != 2]).[name > value] Matches elements if their data attribute is greater than a specified value (e.g. [foo > 'bar'] or [num > 2]).[name >= value] Matches elements if their data attribute is greater than or equal to a specified value (e.g. [foo >= 'bar'] or [num >= 2]).[name < value] Matches elements if their data attribute is less than a specified value (e.g. [foo < 'bar'] or [num < 2]).[name <= value] Matches elements if their data attribute is less than or equal to a specified value (e.g. [foo <= 'bar'] or [num <= 2]).[name *= value] Matches elements if their data attribute contains the specified value as a substring (e.g. [foo *= 'bar']).[name ^= value] Matches elements if their data attribute starts with the specified value (e.g. [foo ^= 'bar']).[name $= value] Matches elements if their data attribute ends with the specified value (e.g. [foo $= 'bar']).@ (data attribute operator modifier) Prepended to an operator so that is case insensitive (e.g. [foo @$= 'ar'], [foo @>= 'a'], [foo @= 'bar'])! (data attribute operator modifier) Prepended to an operator so that it is negated (e.g. [foo !$= 'ar'], [foo !>= 'a'])[[]] (metadata brackets) Use double square brackets in place of square ones to match against metadata instead of data (e.g. [[degree > 2]] matches elements of degree greater than 2). The properties that are supported include degree, indegree, and outdegree.

Compound nodes

> (child selector) Matches direct children of the parent node (e.g. node > node). (descendant selector) Matches descendants of the parent node (e.g. node node).$ (subject selector) Sets the subject of the selector (e.g. $node > node to select the parent nodes instead of the children).

Transforms

GitHub Search of Repos, Gists and MembersBitbucket Search of Repos, Snippets and MembersCloudFlare 1.1.1.1 DNS APICRTSH (CN & SAN)DockerHub Repo SearchGravatar URLsHacker Target Reverse IP LookupHave I Been Pwned LookupPKS LookupBufferover.runUrlscan Live ShotThreatcrowd LookupWappalyzerAWS IAM PagesBuiltwithRiddlerCommonCrawArchive.orgShodanWhatsMyNameCertspotterVirustotalUtility TransformsSecurity TrailsAuto Recon

Tutorial

To demonstrate the power of Pown Recon and graph-based OSINT (Open Source Intelligence), let's have a look at the following trivial example.

Let's start by querying everyone who is a member of Google's engineering team and contributes to their GitHub account.

pown recon t -w google-work ghlm google

This command will generate a table similar to this:

github:member┌─────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────┐│ uri │ login │ avatar │├─────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────┤│ https://github.com/3rf │ 3rf │ https://avatars1.githubusercontent.com/u/1242478?v=4 │├─────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────┤│ https://github.com/aaroey │ aaroey │ https://avatars0.githubusercontent.com/u/31743510?v=4 │├─────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────┤│ https://github.com/aarongable │ aarongable │ https://avatars3.githubusercontent.com/u/2474926?v=4 │.........│ https://github.com/alexpennace │ alexpennace │ https://avatars1.githubusercontent.com/u/2506548?v=4 │├─────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────┤│ https://github.com/alexv │ alexv │ https://avatars0.githubusercontent.com/u/30807372?v=4 │├─────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────┤│ https://github.com/alexwhouse │ alexwhouse │ https://avatars3.githubusercontent.com/u/1448490?v=4 │└─────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────┘

You just created your first network!

The representation is tabular for convenience but underneath we've got a model which consists of nodes connected by edges.

If you are wondering what that looks like you can use SecApps Recon. The command line does not have the necessary level of interactivity to present the complexity of graphs.

The -w google-work command line option exported the network to a file. You can load the file directly into SecApps Recon with the file open feature. The result will look like this:

Now imagine that we want to query what repositories these Google engineers are working on. This is easy. First, we need to select the nodes in the graph and then transform them with the "GitHub List Repositories" transformation. This is how we do it from the command line:

pown recon t ghlr -r google-work -w google2.nework -s 'node[type="github:member"]'

If you don't hit GitHub API rate limits, you will be presented with this:

github:repo┌──────────────────────────────────────────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────────┐│ uri │ fullName │├──────────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤│ https://github.com/3rf/2015-talks │ 3rf/2015-talks │├──────────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤│ https://github.com/3rf/codecoroner │ 3rf/codecoroner │├──────────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤│ https://github.com/3rf/DefinitelyTyped │ 3rf/DefinitelyTyped │.........│ https://github.com/agau4779/ultimate-tic-tac-toe │ agau4779/ultimate-tic-tac-toe │├──────────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤│ https://github.com/agau4779/worm_scraper │ agau4779/worm_scraper │├──────────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤│ https://github.com/agau4779/zsearch │ agau4779/zsearch │└──────────────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────────┘

Since now we have two files google-work and google2-work you might be wondering what is the difference between them. Well, we have a tool for doing just that.

pown recon diff google-work google2-work

Now we know! This feature is quite useful if you are building large recon maps and you would like to know what are the key differences. Imagine your cron job performs the same recon every day and you would like to know if something new just appeared which might be worth exploring further. Hello, bug bounty hunters!

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。