The SiLK Analysis Suite is a collection of command-line tools for
querying packed NetFlow data. The most important tool is rwfilter,
an application for querying the central NetFlow data repository for
NetFlow records that satisfy a set of filtering options.
rwaddrcount
|
rwaddrcount {--print-recs|--print-stat|--print-ips} [SWITCHES] [FILES]
Summarize SiLK Flow records by source or destination IP; with
the --print-recs option will produce textual ouput with counts of
bytes, packets, and flow records for each IP, and the time range
when the IP was active. When no files are given on command line,
flows are read from STDIN.
|
rwbag
|
rwbag [SWITCHES] [FILES]
Read SiLK Flow records and builds binary Bag(s) containing
key-count pairs. Key is protocol or source or destination
address or port. Counter is sum of flows, packets, or bytes.
Reads SiLK Flows from named files or from the standard input.
|
rwbagbuild
|
rwbagbuild {--set-input=FILE | --bag-input=FILE} [SWITCHES]
Create a binary Bag file from either a binary IPset file or from
a textual input file. Use "stdin" as the file name to read
from the standard input. The Bag is written to the standard
output or the location specified with the --output switch.
|
rwbagcat
|
rwbagcat [SWITCHES] [BAG_FILES]
Print binary Bag files as text.
|
rwbagtool
|
rwbagtool [SWITCHES] BAG_FILE [BAG_FILES...]
Perform operations on bag files. Requires at least one bag
file read from stdin or given on the command line.
|
rwcount
|
rwcount [SWITCHES] [FILES]
Summarize SiLK Flow records across time, producing textual output
with counts of bytes, packets, and flow records for each time bin.
When no files given on command line, flows are read from STDIN.
|
rwcut
|
rwcut [SWITCHES] [FILES]
Print SiLK Flow records in a |-delimited, columnar, human-readable
format. Use --fields to select columns to print. When no files are
given on the command line, flows are read from the standard input.
|
rwfilter
|
rwfilter <app-opts> <partition-opts> {<selection-opts> | <inputFiles>}
Partitions SiLK Flow records into one or more 'pass' and/or
'fail' output streams. The source of the SiLK records can
be stdin, a named pipe, files listed on the command line, or
files selected from the data-store via the selection switches.
There is no default input or output; these must be specified.
|
rwgroup
|
rwgroup [SWITCHES]
Groups flows together by specified id-fields and delta-field;
marks the group ID in next hop IP; requires pre-sorting.
|
rwmatch
|
rwmatch --relate=FIELD_PAIR QUERY_FILE RESPONSE_FILE MATCHED_PATH
Groups records as queries and responses
|
rwnetmask
|
rwnetmask [--source-prefix=N] [--destination-prefix=N] [--next-hop-prefix=N]
Read SiLK Flow records from STDIN, mask-off the lower (32-N) bits
of the specified IP address(es), and write the resulting records to
stdout. Does not process files.
|
rwpcut
|
Outputs a tcpdump file as ASCII, in a form similar to rwcut.
|
rwpdedupe
|
rwpdedupe <SWITCHES>
Detects and eliminates duplicate records. Duplicate
records are defined as having the same 5-tuple and payload,
and whose timestamps are within a user-configurable amount
of time of each other.
|
rwpmapbuild
|
rwpmapbuild --input=<INPUT_FILE> --input=<INPUT_FILE>
Reads textual input and creates a binary prefixmap file
for use with the Address Type (addrtype) and Prefix Map
(pmapfilter) Plug-Ins.
|
rwpmatch
|
rwpmatch <SWITCHES>
Filter a tcpdump file by outputting only packets whose 5-tuple
and timestamp match corresponding flows in a rw-file. Outputs
the filtered tcpdump file to stdout.
|
rwptoflow
|
rwptoflow [SWITCHES] TCPDUMP_FILE
Read TCPDUMP_FILE and generate a SiLK Flow record for every
packet. Write the SiLK Flows to stdout, which must not be
connected to a terminal.
|
rwset
|
rwset [SWITCHES] [FILES]
Read SiLK Flow records and generate binary IPset file(s). When no
files are given on command line, flows are read from STDIN.
|
rwsetbuild
|
rwsetbuild <INPUT_FILE> <OUTPUT_FILE>
Reads IP addresses in dotted-quad or CIDR notation from input-file
and writes a binary IPset file to output-file. Use "stdin" as
the input-file to read the IPs from the standard input, and use
"stdout" as the output-file to write the IPset to the standard
output when the standard output is not a terminal.
|
rwsetcat
|
rwsetcat [SWITCHES] [IPSET_FILES]
By default, prints the IPs in the specified IPSET_FILES. Use
switches to control format of the outout and to optionally or
additionally print the number of IPs in the file, the network
structure, or other statistics. If no IPSET_FILEs are given on
the command line, the IPset will be read from the standard input.
|
rwsetintersect
|
rwsetintersect --add=<file> [options] {--print-ips | --set-file=<file>}
Generates a new IP-set by intersecting all --add-set binary
IP-set files, then removing all --remove-set binary IP-set
files from the new IP-set. Prints the resulting IP-set to
stdout and/or writes it to the specified file.
Supports 1-4 add sets and 0-4 remove sets.
|
rwsetunion
|
rwsetunion <OUTPUT_IPSET> <INPUT_IPSET1> [<INPUT_IPSET2> ...]
Merge the input binary IPSet files into the output IPSet;
an IP in any input file will be in the output file.
|
rwsort
|
rwsort --fields=<FIELDS> [SWITCHES] [FILES]
Read SiLK Flow records, sort them by the specified FIELD(S), and
write the records to the named output path or to the standard
output. When no FILES are given on command line, flows are read
from the standard input.
|
rwstats
|
rwstats <SWITCHES> [FILES]
Summarize SiLK Flow records by one of a limited number of
key/value pairs and display the results as a Top-N or Bottom-N
list. The N can be a fixed value, a certain percentage of the
innput, or a threshold value.
Alternatively, provide statistics for each of bytes, packets, and
bytes-per-packet giving minima, maxima, quartile, and interval
flow-counts across all flows or across user-specified protocols.
When no files are given on command line, flows are read from STDIN.
|
rwtotal
|
rwtotal <KEY> [SWITCHES] [FILES]
Summarize SiLK Flow records by a specified key and print the byte,
packet, and flow counts for flows matching the key. When no files
are given on the command line, flows are read from STDIN.
|
rwuniq
|
rwuniq --fields=N [SWITCHES] [FILES]
Summarize SiLK Flow records into user-defined keyed bins specified
with the --fields switch. For each keyed bin, print byte, packet,
and/or flow counts and/or the time window when key was active.
When no files are given on command line, flows are read from STDIN.
|
Several analysis tools load shared libraries to enhance their
functionality. These shared libraries are:
The SiLK Packing System is a server application that receives
NetFlow V5 PDU's and converts them into a more space efficient
format, recording the packed records into service-specific binary
flat files. Files are organized in a time-based directory heirarchy
with files cover an hour at the leaves.
mapsid
|
mapsid [SENSORS]
Maps between sensor names and sensor IDs. Prints a list of
all sensors when no command line arguments are given.
|
num2dot
|
num2dot [SWITCHES]
Read pipe (|) delimited text from the standard input, convert
integer values in the specified column(s) (default first column)
to dotted-decimal IP addresss, and print result to standard output.
|
rwappend
|
rwappend [SWITCHES] TARGET-FILE SOURCE-FILE1 [SOURCE-FILE2...]
Append the SiLK Flow records contained in the second through
final filename arguments to the records contained in the
first filename argument. All files must be SiLK flow files;
the TARGET-FILE must not be compressed.
|
rwcat
|
rwcat [SWITCHES] [FILES]
Reads SiLK Flow records from the FILES named on the comamnd
line, or from the standard input when no FILES are provided,
and writes the SiLK records to the specified output file or
to the standard output if it is not connected to a terminal.
|
rwfglob
|
rwfglob [SWITCHES]
A utility to simply print to stdout the list of files
that rwfilter would normally process for a given set of
file selection switches.
|
rwfileinfo
|
rwfileinfo [SWITCHES] <FILES>
Print information (type, version, etc.) about a SiLK Flow,
IPset, or Bag file. Use the fields switch to control what
information is printed.
|
rwip2cc
|
rwip2cc [SWITCHES]
Maps from IP address to country code using the specified MAP file
or the default map. Either one or more addresses must be specified.
|
rwrandomizeip
|
rwrandomizeip [SWITCHES] <INPUT_FILE> <OUTPUT_FILE>
Substitute a non-routable IP address for the source and
destination IP addresses of <input-file> and write the result
to <output-file>. You may use "stdin" for <input-file> and
"stdout" for <output-file>. Gzipped files are o.k.
|
rwswapbytes
|
rwswapbytes [SWITCHES] <ENDIAN_SWITCH> <INPUT_FILE> <OUTPUT_FILE>
Change the byte-order of <INPUT_FILE> as specified
by <ENDIAN_SWITCH> and write result to <OUTPUT_FILE>.
You may use "stdin" for <INPUT_FILE> and "stdout"
for <OUTPUT_FILE>. Gzipped files are o.k.
|
rwtuc
|
rwtuc [SWITCHES] [FILES]
Generate SiLK flow records from textual input; the input should be
in a form similar to what rwcut generates.
|