Principles
IVRE is a network cartography (or network recon) framework.
Purposes
IVRE has five purposes (we use this word to refer to the different types of data IVRE handles), which can be stored by one or more backend databases:
data: associates IP ranges to Autonomous Systems (AS numbers and names), and geographical information (country, region, city), based on data from Maxmind GeoIP. It can be queried using:Python API: the
db.dataobject from theivre.dbmodule.Command line: the
ivre ipdatatool.Web (JSON) API: the
/cgi/ipdata/<address>URL.
nmap(sometimes also referred to asscans): contains Nmap, Masscan, Dismap, Zgrab2, ZDNS, Nuclei, httpx, tlsx and dnsx scan results, as well asivre auditdomresults. Each record represents one host seen during one network scan. It can be queried using:Python API: the
db.nmapobject from theivre.dbmodule.Command line: the
ivre scanclitool.Web (JSON) API: the
/cgi/scansand/cgi/scans/*URLs.
passive: contains host intelligence captured from the network using a Zeek dedicated module calledpassiverecon, p0f and airodump-ng logs. Each record represents one piece of information (e.g., the HTTPServer:header valueApachehas been seen 10 times on port 80 of host 1.2.3.4). It can be queried using:Python API: the
db.passiveobject from theivre.dbmodule.Command line: the
ivre ipinfoandivre iphosttools. The latter is dedicated to passive DNS queries.Web (JSON) APIs: the
/cgi/passiveand/cgi/passivednsURLs. The latter is dedicated to passive DNS and is compatible with the Common Output Format implemented for example in CIRCL’s PyPDNS.
view: contains a consolidated view of hosts based on data fromnmapandpassive. The structure of the records is similar tonmap, but each record represents a host, seen during one or more network scans and/or seen from network captures. It can be queried using:Python API: the
db.viewobject from theivre.dbmodule.Command line: the
ivre viewtool.Web (JSON) API: the
/cgi/viewand/cgi/view/*URLs.Web UI: the
/or/index.htmlWeb page.
flow: contains aggregated network flows, as seen by Zeek, Argus or Netflows (using Nfdump). It can be queried using:Python API: the
db.flowobject from theivre.dbmodule.Command line: the
ivre flowclitool.Web (JSON) API: the
/flowsURL.Web UI: the
/flow.htmlWeb page.
The following (non-exhaustive) figure shows how the data gets from your favorite open-source tools to IVRE’s databases.
Storing data
![digraph {
graph [rankdir=LR];
"maxmind.com";
"Nmap";
"Masscan";
"ivre auditdom";
"Zgrab2";
"Zdns";
"Nuclei";
"httpx";
"tlsx";
"dnsx";
"Dismap";
"airodump-ng";
"p0f";
"Zeek";
"Zeek";
"Argus";
"Nfdump";
XML [label="XML scan result"];
JSON [label="JSON scan result"];
CSV_LOG [label="airodump .csv files"];
P0F_LOG [label="p0f output files"];
PASS_LOG [label="passive_recon.log"];
FLOW_LOG [label=".log files"];
FLOWS [label="flow files"];
db_data [label="db.data" shape="box" style="filled"];
db_nmap [label="db.nmap" shape="box" style="filled"];
db_passive [label="db.passive" shape="box" style="filled"];
db_flow [label="db.flow" shape="box" style="filled"];
db_view [label="db.view" shape="box" style="filled"];
"maxmind.com" -> db_data [label="ivre\nipdata"];
"Nmap" -> XML [label="-oX"];
"Masscan" -> XML [label="-oX"];
"ivre auditdom" -> XML;
"ivre auditdom" -> JSON [label="--json"];
"Zgrab2" -> JSON [label="-o"];
"Zdns" -> JSON [label="-o"];
"Nuclei" -> JSON [label="-json -o"];
"httpx" -> JSON [label="-json -o"];
"tlsx" -> JSON [label="-json -o"];
"dnsx" -> JSON [label="-json -o"];
"Dismap" -> JSON [label="-j"];
"airodump-ng" -> CSV_LOG [label="-w"];
"p0f" -> P0F_LOG [label="-o"];
"Zeek" -> PASS_LOG [label="passiverecon"];
"Zeek" -> FLOW_LOG;
"Argus" -> FLOWS;
"Nfdump" -> FLOWS;
XML -> db_nmap [label="ivre\nscan2db"];
JSON -> db_nmap [label="ivre\nscan2db"];
CSV_LOG -> db_passive [label="ivre\nairodump2db"];
P0F_LOG -> db_passive [label="ivre\np0f2db"];
PASS_LOG -> db_passive [label="ivre\npassiverecon2db"];
FLOW_LOG -> db_flow [label="ivre\nzeek2db"];
FLOWS -> db_flow [label="ivre\nflow2db"];
db_passive -> db_view [label="ivre\ndb2view"];
db_nmap -> db_view [label="ivre\ndb2view"];
{
rank = same;
edge[style=invis];
"maxmind.com" -> "Nmap" -> "Masscan" -> "ivre auditdom" -> "Zgrab2" -> "Zdns" -> "Nuclei" -> "httpx" -> "tlsx" -> "dnsx" -> "Dismap" -> "airodump-ng" -> "p0f" -> "Zeek" -> "Zeek" -> "Argus" -> "Nfdump";
rankdir = UD;
}
}](../_images/graphviz-2f6369a26823883990bcb83b9ac3555fffef6cc9.png)
Accessing data
The following (also non-exhaustive) figures show how the data gets from IVRE’s databases back into your hands.
![digraph {
db_data [label="db.data" shape="box" style="filled"];
db_flow [label="db.flow" shape="box" style="filled"];
db_nmap [label="db.nmap" shape="box" style="filled"];
web_api_data [label="Web API\n/ipdata"];
web_api_flows [label="Web API\n/flows"];
web_api_scans [label="Web API\n/scans"];
web_ui_flow [label="Web UI\n/flow.html"];
cli_ipdata [label="CLI\nipdata"];
cli_flow [label="CLI\nflowcli"];
cli_scancli [label="CLI\nscancli"];
db_data -> web_api_data;
db_flow -> web_api_flows;
db_flow -> cli_flow;
db_nmap -> web_api_scans;
web_api_flows -> web_ui_flow;
db_data -> cli_ipdata;
db_nmap -> cli_scancli;
}](../_images/graphviz-f3a9d1551609259fb7dbeca90e2770cd757af72a.png)
![digraph {
db_passive [label="db.passive" shape="box" style="filled"];
db_view [label="db.view" shape="box" style="filled"];
web_api_passive [label="Web API\n/passive"];
web_api_passivedns [label="Web API\n/passivedns"];
web_api_view [label="Web API\n/view"];
web_ui_view [label="Web UI /"];
cli_ipinfo [label="CLI\nipinfo"];
cli_iphost [label="CLI\niphost"];
cli_view [label="CLI\nview"];
db_view -> web_api_view;
web_api_view -> web_ui_view;
db_view -> cli_view;
db_passive -> web_api_passive;
db_passive -> web_api_passivedns;
db_passive -> cli_ipinfo;
db_passive -> cli_iphost;
}](../_images/graphviz-895f16ae20541f9b2ee5370c2ec2d9ff25b1a4d9.png)