Argus
From NSMWiki
Contents
|
Introduction
Argus is an open source IP Audit tool written by Carter Bullard and has been under development for over 10 years. This wiki is an attempt to fill out the documentation for Argus. Argus distribution contains man pages which describe the various programs and their commandline switches, what is missing is tutorial type material describing how one might use Argus day to day. The material here should be used in conjunction with the man pages and information on the Argus Home.
Argus consists of two parts, a server which records network traffic visible from one or more of the NICs on the machine. Argus assembles these information on the traffic into network flows. Data about flows is either written to disk or to a network socket if a client has connected. The second part of Argus is a collection of clients which read flow data either from Argus log files for direct from an Argus server via a network socket.
Who uses Argus
Many universities, corporations, and government entities use Argus to record both internal traffic flows and flows entering and leaving their network(s). These records are used in both immediate network utilization analysis, and historical analysis or trending. With a sensor network using Argus, organizations may validate the connectivity of end-hosts through multiple routers. If routers A, B, and C are passing traffic for hosts Y and Z, Argus may be used to determine latency and other problems between routers B and C (which may not be apparent in packet captures).
Historical netflow data can be used in forensic investigations several months, or years, after an incident has taken place. Argus' netflow records offer up to a 10,000:1 ratio from the packet size to the record written to disk, which allows installations to save records for much longer than full packet captures. When network security is very important, non-repudiation becomes a very important requirement that must be provided throughout the network. Argus provides the basic data needed to establish a centralized network activity audit system. If done properly, this system can account for all network activity in and out of an enclave, which can provide the basic system needed to assure that someone can't deny having done something in the network.
Network research labs have used Argus to provide network performance measurements of unique protocols, such as Infiniband over IPv6. Argus can be quickly adapted to new protocols, and in some cases, provides the basic metrics without extension. Individuals use Argus in their home networks to give them a heads up on DSL and Cable Modem based networks. Argus provides a higher order view into packet data, that allows a network user the ability to see problems quickly.
Argus server
man page
Clients
What sorts of things can argus do
Argus is primarily a network activity monitoring system. Historically, Argus has been used to support network security management and network forensics. through its ability to establish an audit trail of network activity.
Stéphane Peters's Cheat sheet
List originally contributed by Stéphane Peters (v3).
Examples
ragrep example: Finding Palevo / Sality virus activity
As of V3.0.2 ragrep() is obsolete. You should use the newer argus-clients-3.0.2 programs, all of which allow you to grep, using the "-e" option. bash code :
#!/bin/bash # File : ragrep-sality.sh s="solfire.aljosaborkovic.com" s="$s|kukutrustnet777.info" s="$s|www.kjwre.*fqwieluoi.info" s="$s|l33t.brand-clothes.net" s="$s|pica.banjalucke-ljepotice.ru" s="$s|maellisromance.com" s="$s|217.32.75.74" s="$s|pingaksh.com" s="$s|radio.irib.ir" s="$s|regal-mont.pondi.hr" s="$s|sandra.prichaonica.com" s="$s|sasgrowth.com" s="$s|snowboard619.w.interia.pl" s="$s|spargeunid.go.ro" s="$s|stakrix.st.funpic.de" s="$s|us516757.bizhostnet.com" s="$s|www.abassiehmunicipality.com" s="$s|www.polaris.ge" s="$s|www.railwayservices.be" s="$s|www.senaauto.ge" s="$s|ziyagokalpilkogretim72.meb.k12.tr" ra -s "+suser:50 -bytes" -e "$s" $* - udp port 53
It is really a one-liner like this, split on several lines for editing.
ra -s "+suser:50 -bytes" -e "solfire.aljosaborkovic.com|kukutrustnet777.info|www.kjwre.*fqwieluoi.info" -nr $file - udp port 53
You need to use "ragrep" in previous versions of argus-clients(3.0.0 for example).
Knowing that Palevo and Sality viruses try to connect to one of these sites, this script permits to identify the computers that have done such DNS requests, and that are infected (with a high degree of probability).
The resulting RE is an ORing of several strings and another RE (www.kjwre.*fqwieluoi.info) to cach a probably random number. The script is launched like this:
ragrep-sality.sh -nr $file ragrep-sality.sh -nr $file -w /tmp/sality-traces.ra
Here is an output:
StartTime Flgs Proto SrcAddr Sport Dir DstAddr Dport SrcPkts DstPkts State srcUdata 01/03 08:21 e udp 1.0.4.1.44177 <-> 100.0.1.1.53 1 1 CON s[40]=.............sandra.prichaonica.com..... 01/03 08:21 e udp 1.0.4.1.40419 <-> 100.0.1.1.53 1 1 CON s[44]=.............solfire.aljosaborkovic.com..... 01/03 08:21 e udp 1.0.5.1.32200 <-> 100.0.1.1.53 1 1 CON s[40]=.Y...........sandra.prichaonica.com..... 01/03 08:22 e udp 1.0.5.1.29661 <-> 100.0.1.1.53 1 1 CON s[44]=.............solfire.aljosaborkovic.com..... 01/03 08:29 e udp 1.0.5.1.32554 <-> 100.0.1.1.53 1 1 CON s[40]=.............sandra.prichaonica.com..... 01/03 08:30 e udp 1.0.5.1.44465 <-> 100.0.1.1.53 1 1 CON s[44]=.............solfire.aljosaborkovic.com..... 01/03 08:30 e udp 1.0.4.1.29810 <-> 100.0.1.1.53 1 1 CON s[40]=b............sandra.prichaonica.com..... 01/03 08:31 e udp 1.0.4.1.41186 <-> 100.0.1.1.53 1 1 CON s[44]=yc...........solfire.aljosaborkovic.com..... ... 01/03 10:27 * udp 1.0.9.2.42875 <-> 100.0.1.1.53 1 1 CON s[44]=e............solfire.aljosaborkovic.com..... 01/03 10:42 e udp 1.0.15.1.46746 -> 197.0.7.1.53 2 0 INT s[50]= O................V...........sandra.prichaonica.c 01/03 10:42 e udp 1.0.12.1.45079 <-> 100.0.1.1.53 1 1 CON s[40]=.............sandra.prichaonica.com..... 01/03 10:42 * udp 1.0.9.3.31681 <-> 100.0.1.1.53 1 1 CON s[40]=.............sandra.prichaonica.com..... 01/03 10:42 e udp 1.0.15.1.46746 -> 197.0.2.1.53 3 0 INT s[50]= O................V...........sandra.prichaonica.c 01/03 10:42 e udp 1.0.15.1.46746 -> 197.0.3.1.53 3 0 INT s[50]= O................V...........sandra.prichaonica.c 01/03 10:42 e udp 1.0.15.1.46746 -> 197.0.4.1.53 3 0 INT s[50]= O................V...........sandra.prichaonica.c
other
Flow filtering on certain port range :
ra -r $file - dst port gt 1024 and dst port lt 2048
Use racluster() to generate the counts you are looking for:
racluster -m proto -r $file -s proto spkts dpkts sbytes dbytes
Proto SrcPkts DstPkts SrcBytes DstBytes
udp 15567 12390 2912004 3240927
tcp 900187 866302 410506598 722771403
icmp 645 522 123240 61250
Packet Loss (with IP address):
ragraph loss saddr daddr -M 10s -r $file -title 'Packet Loss / IPs' -w ploss.png
Packet Loss (number of packets)
ragraph loss spkts dpkts -M 10s -r $file -title 'Packet Loss / Packets' -w ploss2.png
Jitter (number of packets)
ragraph jitter saddr daddr -M 10s -r $file -title 'Jitter' -w jitter.png
Concurrent transactions:
ragraph trans -M 10s -r $file -title 'Concurrent Transactions' -w transac.png2
Note (2010-0617): It does look, from the code, that it is trans/sec. We have explicit
code for controlling that, and it looks like "Trans" doesn't correct for
the the GAUGE/AVERAGE artifacts rrd and rrd_graph generates.
If you make this change to ragraph():
thoth:~ carter$ diff `which ragraph` /tmp/ragraph
1093c1093
< /Trans/ and do {$power[$x] = 1.0 ; };
---
> /Trans/ and do {$power[$x] = $STEP ; };
It will graph the actual 'trans' value in each time bin.
Top talkers & Listeners
racluster -m matrix -r $file -w - | rasort -m bytes | less
Note: piping through 'ra -n' again was redundant and a waste of CPU cycles (FYI: the -s switch is also available for rasort when one requires a different output)
Rastrip always removes argus management transactions, thus having the same effect as a
’not man’
filter expression.
To remove the tcp network DSR (data structure record?):
rastrip "-m -net"
(or something like it)
To see if you get something useful:
rastrip "-M time flow metric"
Yes, you can pipe rastrip(). Try something like this:
rastrip -S $server -w - | rasplit [options] -r -
racluster -r $file -M net 192.168.0.0/16 -m daddr/16 - "host 192.168.0.10 or host 192.168.0.11"
% ra -nr $file -s saddr sport daddr dport SrcAddr Sport DstAddr Dport 1.2.3.58.1140 1.2.4.5.41460 1.2.3.55.4100 1.2.4.5.41460 1.2.3.3.3336 1.2.5.6.135
Split records into 5 minute files
rasplit -M time 5m -S argus-north... -w /var/log/argus/\$srcid/%Y/%m/%d/file.%Y.%m%d.%H.%M.%S
one for every day
rasplit -S radium -M 1d -w /path/argus-\$srcid.%Y.%m.%d.log
It is possible to execute some command after each file, ie compress it or insert data in a database;
rastream -S argus -B 15s -w /archive/\$srcid/%Y/%m/%d/ntam.%Y.%m.%d.%H.%M.%S \
-f /usr/local/bin/rastreamshell
There is an example file in the distribution, SRC/support/Config/rastream.sh :
#!/bin/sh
#
# Argus Client Software. Tools to read, analyze and manage Argus data.
# Copyright (C) 2000-2011 QoSient, LLC.
# All Rights Reserved
#
# Script called by rastream, to process files.
#
# Since this is being called from rastream(), it will have only a single
# parameter, filename,
#
# Carter Bullard <carter@qosient.com>
#
PATH="/usr/local/bin:$PATH"; export PATH
package="argus-clients"
version="3.0.2"
OPTIONS="$*"
FILES=
while test $# != 0
do
case "$1" in
-r) shift; FILES="$1"; break;;
esac
shift
done
racluster -M replace -r $FILES
gzip $FILES
exit 0
Comma separated value
%cat ra3.conf.t RA_PRINT_LABELS=0 RA_FIELD_DELIMITER=',' RA_PRINT_NAMES=proto RA_TIME_FORMAT="%y-%m-%d %T" RA_PRINT_DURATION=no RA_PRINT_LASTIME=yes
%ra3 -F ra3.conf.t -r icmp3.argus | more StartTime,Flgs,Proto,SrcAddr,Sport,Dir,DstAddr,Dport,SrcPkts,DstPkts,SrcBytes,DstBytes,State 06-06-27 11:20:28.911941, v ,icmp,142.58.201.99,,->,142.58.201.254,,1,0,102,0,ECO 06-06-27 11:20:28.911946, v ,icmp,142.58.201.99,,->,142.58.201.254,,1,0,102,0,ECO 06-06-27 11:20:28.911951, v ,icmp,142.58.201.99,,->,142.58.201.254,,1,0,102,0,ECO
racluster -m saddr/23 daddr proto dport -w -r $file - dst net 10.1.2.0/23 \
| rasort -m proto daddr dport dbytes - \
-s ltime saddr sport daddr dport spkts dpkts sbytes dbytes \
|less
To do a top talkers for say IP addresses
(racluster can do it for any object in the record, top mac addrs, top
tos bytes, top mpls label, top vlan, top port, top ttl, etc....):
racluster -M rmon -m saddr -r $file - ip
A list with 2 columns, IP-address and bytes used:
racluster -M rmon -m saddr -r $file -w - - ip \ | rasort -m bytes -s saddr bytes |head -20
... not to be confused with :
racluster -M rmon -m saddr -r $file -w - - ip \ | rasort -N 20 -m bytes -s saddr bytes
... equivalent to :
racluster -M rmon -m saddr -r $file -w - - ip \ | ra -N 20 | rasort -m bytes -s saddr bytes
A list with 2 columns, IP-address and bytes used (carter version):
racluster -M rmon -m proto sport -r $file -w - - ip | \ rasort -m bytes proto sport -s stime dur proto sport spkts dpkts sbytes dbytes
802.1q packets monitoring already there. If you have vlan input traffic adding
-s +svlan +dvlan
to your ra command will display the VLAN tag values in hex form and you can filter ra (or other clients) traffic on vlan tags.
To see the VLAN in decimal form, use these options:
-s +svid +dvid
Top src address based on src bytes in a collection of records
racluster -m saddr -w - -R 2006/09/28 - ip | rasort -m sbytes
Top address, regardless of direction (The "-M rmon" folds the src and dst addresses together, putting the values into the saddr field.):
racluster -M rmon -m saddr -w - -R 2006/09/28 - ip | rasort -m sbytes
2007-0305 (Argus-info Digest, Vol 19, Issue 5) What is the current best way to get a report like :
ramon -nn -L0 -M svc -r $file - | head -25
racluster -M rmon -m proto sport -r $file -w - - tcp or udp | \
ra -N 25 -s proto sport spkts dpkts sbytes dbytes
2007-0321 (Argus-info Digest, Vol 19, Issue 30) Looking for functionality like: ramon -M TopN or -M Matrix try this:
racluster -r $file -M rmon -m saddr - ip ( this generates stats based on IP address)
racluster -r $file -m matrix - ip (based on IP matrix)
to do whatever TopN you want, pipe the output to rasort(). So to get the Top10 in packets received and transmitted:
racluster -r $file -M rmon -m saddr -w - | rasort -m pkts -w - | ra -N 10
To get the Top5 in bytes per second transmitted:
racluster -r $file -M rmon -m saddr -w - | rasort -m srate -w - | ra -N 5 -s +srate
2007-1102 (Argus-info Digest, Vol 27, Issue 2) I(Terry) run the following collectors:
/opt/argus/sbin/argus -X -d -A -i eth2 -P 561 /opt/argus/sbin/radium -X -d -C -S 1006 -P 564 /opt/argus/sbin/radium -X -d -C -S 1007 -P 565
I(Terry) have another process that aggregates these:
/opt/argus/sbin/radium -X -d -S localhost:561 -S localhost:564 -S \ localhost:565 -P 569
2008-0215 Some examples of ragraph: ( search.gmane.org/?query=ragraph&group=gmane.network.argus )
ragraph bytes proto -M 60s -r strange-broadcast-10000.argus -fill -stack \
-w ./strange-broadcast-10000.png
ragraph -r inputfiles* -t 12-13
ragraph spkts dport -M 1h -n -n -r argus.dat.04 - src net X/20
ragraph pkts dport -M 10s -T 60 -S 192.168.1.101 -p0
ragraph bytes saddr -M 1m -m saddr/24
rabins -M soft zero -p6 -GL0 -s ltime bytes -nn -M 1m \
-r $files - srcid eligate1 and icmp | head
ragraph sbytes dbytes -M rmon time 1m -m smac -t 2007/10/04 \
-r $file -w ragraph.png -- ether host 00:15:F2:64:92:13
ragraph pkts proto -M 1m -title 'eligate2: protocol distribution' \
-height 200 -t 2007/10/04 -r /var/log/argus/argus.log \
-w /var/www/argus/eligate2/proto/current.png - srcid eligate2
rahisto -r datafile -H drate 140:100-170K
bash> for i in 1s 2s 5s 10s 15s 20s 30s 45s 1m 2m 5m 10m 15m 20m 30m 1h 2h; do echo $i ;\
ragraph rate dport -M $i -r output.file -t 18-20 -m proto dport -upper 5000 -lower 7000 \
-title "Aggregation Metric Distribution Analysis - Resolution $i" ;\
mv ragraph.png aggregation.$i.png; done
rasort -R ${stats_dir}/.../day -m bytes smac saddr -w - \
| ra -N 20 -w top20.talkers.list
; ra -s addr -r top20.talkers.list > addrs.list
; rafilteraddr -f addrs.list -R ${stats_dir}/..../daily > /tmp/data
; ragraph spkts dpkts saddr -M 1m -w /tmp/ragraph.png
2008-0228 (Argus-info Digest, Vol 30, Issue 41)
to insert data every 5 minutes, it can be as easy as:
rastream -S live.argus.stream -f yourMysqlImport.sh -M time 5m -B 15s \
-w /opt/ARGUS/OUTBOUND/%Y/%m/%d/argus.%Y.%m.%d.%H.%M.%S
This would generate an argus archive broken out by year/month/day containing files every 5 minutes, and 15 seconds after then end of each 5 minute clock boundary, your script would be run against the file, indexing the data and then compressing the file. It could remove the file if you're not interested in keeping the archive etc......
2008-0305 (Argus-info Digest, Vol 31, Issue 6)
When the records are not well formed, you need the "-M rmon" option
to make the records direction-less. Because of the direction-less nature
you can use "dport" or "sport" as the merge key, but you have to be consistent,
as you will need to pipe the output to ra() to select the ports you're interested in:
racluster -M rmon -r $file -m proto dport -w - | \
ra -L 0 -s stime dur proto dport spkts dpkts sbytes dbytes - dst port 80 or 443
equivalent to (in argus clients v2.0.6)
ramon -M Svc -nn -r argus-$DATE.arg - port 80 or 443".
Bandwidth usage flow by flow on 26th Feb from 19h to 20h,
unnecessary columns have been cut to keep every record on a single line
( from : www.vorant.com/nsmwiki/Argus#How_do_I_do_IP_accounting_by_IP :-)
cd /archive/2008/02/26 racluster -w - -M rmon -m saddr daddr -r argus.19.00.00.gz -w - - ip and dur gt 1 \ | rasort -m sload -w - \ | ra -N 15 -p 0 -s "-flgs -proto -dir -state +avgdur +sload +dload +trans"
List all possible state fields of a file
% ra -r $file -nn | awk '{print $NF}' | sort | uniq -c | sort -nr
91104 CON
77066 FIN
65763 TIM
55618 ECO
41232 INT
28724 RST
798 ECR
467 URP
2 CLO
1 STA
2008-0312 (Argus-info Digest, Vol 31, Issue 15) Print headers in ra* version 3.*
"-L 0" will print the headers once, "-L 40" will print the headers every 40 lines, etc ...
2008-0312 (Argus 3: Statistics for Major Protocols) (C.S. Lee) Here you go, you can cluster or merge the records based on the flow key and it is suitable for data mining, data management and report generation, let's generate the statistical report using protocol as flow key. Notice I specify -m proto in command line below and using -s to print the field I want
racluster -L0 -m proto -r $file -s proto trans pkts bytes appbytes -\ tcp or udp or icmp
2008-0317 When (on which date) did start this long-running argus file (by default, ra* clients use the "%T" format ie HH:MM:SS) ?
cat /tmp/rarc
RA_TIME_FORMAT="%D %T"'
ra -s "stime" -F /tmp/rarc.$$ -N 1 -L 0 -nr $file
StartTime
02/29/08 18:42:55
2008-02-28 simple gnuplot plot file to generate a graph of "Total Bytes By Protocol" using argus data; assuming gnuplot is installed in /opt/local/bin/gnuplot (Carter Bullard).
% chmod 755 barchart.bytesxproto.plt
% racluster -m proto -r argus.out -s proto spkts dpkts sbytes dbytes > racluster.dat
% ./barchart.bytesxproto.plt
------ begin barchart.bytesxproto.plt ------
#!/opt/local/bin/gnuplot -persist
#
# G N U P L O T
# Version 4.2 patchlevel 2
# last modified 31 Aug 2007
# System: Darwin 9.2.0
#
# Copyright (C) 1986 - 1993, 1998, 2004, 2007
# Thomas Williams, Colin Kelley and many others
#
# Type `help` to access the on-line reference manual.
# The gnuplot FAQ is available from www.gnuplot.info/faq/
#
# Send bug reports and suggestions to <sourceforge.net/projects/gnuplot>
#
#
reset
#
# Create simple barchart of Total Bytes by Protocol
# The racluster.dat file was generated using:
#
# racluster -m proto -r argus.out -s proto spkts dpkts sbytes dbytes
#
# And is of the format:
#
# Proto SrcPkts DstPkts SrcBytes DstBytes
# pim 53267 18086 48793554 1085160
# ospf 1764 0 213220 0
# [more]
#
set termoption font "Verdana, 12"
set size square 0.90,0.90
set bmargin 4
set title "Total Bytes By Protocol" font "Verdana,22"
set style data histogram
set style histogram cluster gap 1
set style fill solid border -1
set tics font "Verdana,14"
set boxwidth 0.80
set grid
set ylabel "Log Total Bytes" font "Verdana,18"
set logscale y 10
set auto y
set label 1 "Generated by Argus using Gnuplot"
set label 1 at graph 1.02, 0.62 rotate by 90 font "Verdana,9"
#
set key autotitle columnhead
plot 'racluster.dat' using 4:xticlabels(1) ti col, \
using 5 ti col
#
------ end barchart.bytesxproto.plt ------
2008-0326 Count flows by groups of 10 minutes : show only the flow start times, cut after the 10ths of minutes, strip first line (headers), add a trailing zero and delete heading spaces to show a nice HH:MM line, count them, invert columns, insert a delimitor. Ready to be feed in your favorite spreadsheet.
echo 'RA_TIME_FORMAT="%H:%M"' > raTime.conf
ra -F raTime.conf -s stime -nr $file | \
cut -c -4 | \
uniq -c | \
sed -e '1d' \
-e 's/$/0/' \
-e 's/^ *//' \
-e 's/\(.*\) *\(.*\)/\2,\1/' > flowcounts.csv
2008-0409 Carter's version, thanks to Nick Diel - This example assumes you have already merged status flow records, so records = flows, if not add another pipe of racluster. If you have multiple collectors, you can have rabins merge on something else
such as proto if you are filtering on tcp.
echo 'RA_TIME_FORMAT="%H:%M"' > raTime.conf # (you could also add this to your rarc file)
rastrip -r $file -M -agr -w - | \
rabins -M nomodify time 10m -m srcid -s stime trans -c , -F raTime.conf > flowcounts.csv
2008-0409 Carter's note : When you only want a single flow counted once, in the time bin
when it started. To do this you don't want to modify/split the flow records, so use this option:
rabins -M nomodify
2008-0409 Stéphane Peters : Small all-purpose script to count and totalize all columns : /bin/tot
#!/bin/awk -f
BEGIN{max=0}
{if ( NF > max ) max = NF;
for ( i=1 ; i <= NF ; i++ ) {
tot[i]+=$(i);
}
}
END { for ( i=1 ; i <= max ; i++ ) {
if ( tot[i] > 1000000 )
printf "%sm\t", tot[i]/1000000;
else if ( tot[i] > 1000 )
printf "%sk\t", tot[i]/1000;
else if ( tot[i] == 0 )
printf "-\t";
else
printf "%s\t",tot[i];
}
printf "\n";
}
2007-10-04 Wolfgang Barth : "I'm using the following code for graphing interface load:" (thread.gmane.org/gmane.network.argus/5338/focus=5348)
/usr/local/bin/rabins -M rmon 1m -m smac -t 2007/10/04 \ -r /var/log/argus/argus.log -w - - srcid eligate2 | \ /usr/local/bin/ragraph sbytes dbytes -M 1m -title 'eligate2: Load' \ -height 200 -upper 1000000 -rigid -lower 1000000 -rigid -t 2007/10/04 \ -w /var/www/argus/eligate2/load/current.png -r - - ether dst 00:15:F2:64:92:13
2008-06-25 From Peter Van Epp: How to put commas in large numbers (article.gmane.org/gmane.network.argus/6062)
The following perl fragment will add commas if you run the ra output through an appropriate perl script:
sub commas {
local($_) = @_;
1 while s/(.*\d)(\d\d\d)/$1,$2/;
$_;
}
and called like this:
$pcount = &commas($count);
2008-12-29 (Argus-info Digest, Vol 40, Issue 5) ragraph with large files
Carter : When you are graphing objects like ports, you can use the aggregation features of ragraph() to minimize the memory use. For example, you can use "-m proto dport" in :
ragraph dbytes sbytes dport -M 5m -t $time -fill -stack -invert -title \"$title\" \ $log -w $filename $filter
That should constrain your graph so that it doesn't use much memory at all (max should be, what, 64K ports for udp and tcp in memory for each 5m period). Thing to note : the destination port field doesn't decode without the protocol field having a valid value.
2009-02-13 (Argus-info Digest, Vol 42, Issue 15) Radium repository example
rasplit -M time 5m -S radium -w experiment/\$srcid/%Y/%m/%d/argus.%Y.%m.%d.%H.%M.%S ra -S remoteRadium/path/to/specific/argus/file/argus.2009.02.13.15.20.00.gz
2009-04-24 (Argus-info Digest, Vol 44, Issue 35) argus reads tcpdump files
First, creation of the tcpdumpfile (CAP / PCAP format), followed by the conversion with argus
tcpdump -i eth0 -n -w testdump ; argus -mAJZR -r testdump -w testdump.arg3
ralabel example
(Argus-info Digest, Vol 59, Issue 33)
First, create a "ralabel.conf" file:
RALABEL_ARGUS_FLOW=yes RALABEL_ARGUS_FLOW_FILE="argus-flow-file"
Second, create an "argus-flow-file" :
# Argus-flow-file # # Our application filter="host 10.1.2.3 and port 80" label="Appserver - web traffic" filter="host 10.1.2.3" label="Appserver - other traffic" # Proxy filter="host 10.1.2.4 and port 8080" label="Proxy server - normal traffic" filter="host 10.1.2.4 and port 80" label="Proxy server - web traffic" filter="host 10.1.2.4" label="Proxy server - other traffic" filter="udp and port 53" label="DNS traffic"
Use it (some fields have been removed to fit the wiki page) :
ralabel -f ralabel.conf -nr $f -s "-status -sbytes -dbytes +label:40"
Result:
26/07 11:59 tcp 1.0.2.2.9405 -> 10.2.3.4.80 503846 RST flow=Proxy server - web traffic 26/07 11:59 tcp 10.2.3.4.8080 <?> 1.0.3.1.8248 163 CON flow=Proxy server - normal traffic 26/07 11:59 tcp 1.0.4.1.8820 -> 10.1.2.3.80 9895 FIN flow=Appserver - web traffic 26/07 11:59 icmp 1.0.5.1.8 <-> 10.2.3.4.11736 204 ECO flow=Proxy server - other traffic 26/07 11:59 tcp 1.0.6.1.9286 -> 10.2.3.4.8080 5381 FIN flow=Proxy server - normal traffic 26/07 11:59 tcp 1.0.4.1.8821 -> 10.1.2.3.80 1475 FIN flow=Appserver - web traffic 26/07 11:59 icmp 1.0.5.1.8 <-> 10.2.3.4.11736 204 ECO flow=Proxy server - other traffic 26/07 11:59 icmp 1.0.5.1.8 <-> 10.2.3.4.11736 204 ECO flow=Proxy server - other traffic 26/07 11:59 tcp 1.0.7.1.57268 -> 10.2.3.4.8080 1208 CON flow=Proxy server - normal traffic 26/07 11:59 tcp 1.0.8.1.9265 <?> 10.2.3.4.8080 242 CON flow=Proxy server - normal traffic 26/07 11:59 tcp 1.0.9.1.22513 -> 10.2.3.4.8080 9252 FIN flow=Proxy server - normal traffic 26/07 11:59 tcp 1.0.9.1.22516 -> 10.2.3.4.8080 9200 FIN flow=Proxy server - normal traffic 26/07 11:59 tcp 1.0.9.1.22518 -> 10.2.3.4.8080 155672 FIN flow=Proxy server - normal traffic
rasplit example, working with pipes
(Argus-info Digest, Vol 76, Issue 21, Jesse Bowling)
To have argus generate both a flow file as well as a pcap file of the data as it's captured... a hackish way to go about it...YMMV:
mkfifo tcpdump.fifo mkfifo argus.fifo tcpdump -r tcpdump.fifo -w /pcaps/%Y_%m_%d_%H%M_test.pcap -G 300 & argus -r argus.fifo -w - | rasplit -r - -M time 5m -w /argus/%Y_%m_%d_%H%M_test.argus & tcpdump -i eth0 -s 2048 -w - | tee argus.fifo > tcpdump.fifo &
ratop filters
(2007-02-14 04:46:06, Carter Bullard)
There are three types of filters in ratop(), the first is a remote filter, which will be transmitted to a remote argus source, thus limiting the amount of traffic on the wire. The second is a local input filter. You would use this type of filter if the remote does not support the type of filter you want to use. This is a compatibility feature. The third is a display filter, which will control what records are displayed, without affecting the internal buffers of ratop().
You differentiate the filter types using the keywords "remote", "local" and "display".
Without a keyword, you get "remote", and the remote filter is sent, if there is an argus server to send it to, and it is used as an input filter for ratop().
So ... try this:
ratop -r file
This causes ratop() to process the file without any type of input filtering. Once the data is done, then in ratop(), call up the "Specify filter: " prompt by typing:
f
and then at the prompt type:
display tcp and dst port 80
ratop various commands
(2012-01-23, Carter Bullard)
Command '/','?', 'n', 'N' - search
ratop.1 is like 'vi', in that you will have at the bottom of the screen a status line. If you type '/', you go into forward search mode, and you can type any string, then carriage return, and like 'vi', the cursor will bounce to that string in the developing flow cache display that ratop.1 is printing. That is a regex, so you can put really bizarre things in there. It will search across multiple pages, and then 'N', and 'n' allow you to go to the next or previous. These search on the actual strings on the screen, so you have to have fields displayed in order to search on them. If you type '?' you go into backwards search mode.
Control-r: Reverse flow direction
One that is important is the control-r command, as that reverses the direction of a specific flow record on the screen. Get the cursor to a line you want to reverse, then hit 'control-r'.
Command ':' - options
if you type ':' you will be in command mode and you can type options and commands. Command 'h', will print out the help screen. Using the ':' command method, you can for example, change the sorting algorithm on the fly (command 's'), you can change the fields (command 'F').
Command ':H' - Human bytes
At anytime type command 'H' while ratop is reading data, and most of the numeric metrics, such as bytes, appbytes, packet counts, rates, loads, etc…. will be converted to the appropriate abbreviations. 'H' is a toggle, so you can hit it as many time as you like to flip the abbreviations, and when you're done, carriage return will put ratop.1 back into navigation mode.
Command ':s' - Save cache
Because you can do a command 's' at anytime to save the cache that ratop.1 is working with, you can use ratop.1 to do corrections on flow records.
Command ':a' - add
One that is not mentioned in the help screen is the 'A' option, to add new lines to the display. The only one supported right now is 'totals', which puts the aggregate of the entire cache that ratop.1 is working with as the first line on the display. Remove it with "-totals", so type:
:atotals
gives you an "Add: " prompt, then type "totals", then carriage return.
As in 'vi', the 'h', 'j', 'k', 'l' navigation works, so you can move the cursor around if your arrow keys don't work.
various parameters
"-M hex" : ra hex dump
(2012-01-22 20:28:50, Carter Bullard, "Re: argus client obfuscation")
ra* programs currently support the "-M ascii", "-M hex", "-M encode32", and "-M encode64" command-line options, which are undocumented. I will change this support to " -M printer=ascii", "-M printer=hex" ...
Useful Links
Argus - Downloads (official download page, actual v3.0.0 stable since Apr 2008 !)
www.qosient.com/argus/downloads.shtml
Argus - Home
www.qosient.com/argus/index.shtml
Argus - FAQ
www.qosient.com/argus/faq.shtml
Argus - Development File Listing dev (download page for the next release in development)
qosient.com/argus/dev/
Argus - Previous versions
qosient.com/argus/src/
News Argus sur Gmane
news.gmane.org/gmane.network.argus (thread look) blog.gmane.org/gmane.network.argus (blog look)
Argus - NSMWiki (but ... it's here !)
nsmwiki.org/Argus
Argus - WTFWiki (another one, updated 2012-07)
wtf.hijacked.us/wiki/index.php/Argus
Argus - Documentation / How To File
www.qosient.com/argus/howto.shtml web.archive.org/web/20080119143705/qosient.com/argus/how-to.htm (last copy of old version)
[ARGUS] rahisto dialog
blog.gmane.org/gmane.network.argus/month=20061121
Argus Tips and Tricks: more than 17 extensive posts from C.S. Lee about Argus (When {Puffy} Meets ^RedDevil^: of C.S. Lee)
A good starting point to understand how the argus records work
geek00l.blogspot.com/2007/12/network-flow-demystified.html
Packets -> Flows -> CSV -> Graph
geek00l.blogspot.com/2007/11/packet-flow-csv-graph.html
Argus 3: Statistics for Major Protocols
geek00l.blogspot.com/2008/01/argus-3-statistics-for-major-protocols.html
Argus 3: German Article
geek00l.blogspot.com/2008/01/argus-3-german-article.html
... for the remaining ones, look at C.S. Lee's blog posts talking about Argus3 :