Interface - total bandwidth

[centerstage]

Hello all,

I am trying to find a simple solution to a seemingly common requirement. We need to keep track of total bandwidth by month on our external interface as our
new ISP charges $15/gb over our monthly cap

We also don't need fancy graphs just text is fine.

If anyone can recommend something it would be greatly appreciated.

[s0xxx]

Run into it a while ago, see if it helps you:

https://calomel.org/bandwidth_usage.html

[mish]

Have a look at the following and see if any one suits your needs:

/usr/ports/net/bwm-ng <= probably your best bet
/usr/ports/net/ifstat
/usr/ports/net/iftop

There's also ipband (Google it), but it's not in OpenBSD ports. See if you can compile it anyway.

Good luck.

[centerstage]

Thanks for the suggestions. I have set them all up and will see what i get after they have a chance to run for a bit. The camomel script seems to have the best output. I put it in the daily cron so I am emailed the stats every day. The only issue i can see is that the statistics get reset on reboot.

I am going to keep searching as ideally i would like the output to look like:

Today's Bandwidth: x/gb
Yesterday's Bandwidth: x/gb
Current Month: x/gb
Previous Month: x/gb

Don't know if i am expecting too much......

Anyways I will give an update tomorrow

[ephemera]

Originally Posted by centerstage

I am going to keep searching as ideally i would like the output to look like:

Today's Bandwidth: x/gb
Yesterday's Bandwidth: x/gb
Current Month: x/gb
Previous Month: x/gb

Don't know if i am expecting too much......

Anyways I will give an update tomorrow

#!/usr/bin/perl
# Copyright: ephemera @ daemonforums.org

use POSIX("mktime");
die "Usage: $0 interface\n" if $#ARGV;
$logfile="/var/log/bwmond.$ARGV[0]";
open LOG,"< $logfile" or die "$logfile: $!\n"; 
@abbr = qw( Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec);
($min,$date,$mon,$year)=(localtime)[1,3..5]; 
($ydate,$ymon,$yyear)=(localtime mktime 0, 0, 0, $date-1, $mon, $year)[3..5];
($ldate,$lmon,$lyear)=(localtime mktime 0, 0, 0, $date, $mon-1, $year)[3..5]; 
$year+=1900; $yyear+=1900; $lyear+=1900;
%what=(d=>'Today',yd=>'Yesterday',m=>'This Month',lm=>'Last Month');
for (reverse <LOG>) {
	$end=1;
	if (!defined $lent) {
		/(.*):.*/;
		$lent=$1;
	}	
	tr/:/ /;
	@F = split;
	if ($F[5] == $year && $F[0] eq $abbr[$mon]) { 
		push @m, [@F];
		if ($F[1] == $date) {
			push @d, [@F];
		}
		$end=0;
	} 
	if ($F[5] == $yyear && $F[0] eq $abbr[$ymon] && $F[1]==$ydate) {
		push @yd,[@F]; $end=0;
	} 
	if ($F[5] == $lyear && $F[0] eq $abbr[$lmon]) {
		push @lm,[@F]; $end=0;
	} 
	last if $end;
}
printf "\n%23s%10s\n", '| Dnload(MB)', '| Upload(MB)|';
printf "%s\n", '-----------+-----------+-----------+';
for (('d','yd','m','lm')) {
 if (defined @{"$_"}) {
  $up = ${"$_"}[0]->[12] - ${"$_"}[-1]->[12] + ${"$_"}[-1]->[6]; 
  $down = ${"$_"}[0]->[14] - ${"$_"}[-1]->[14] + ${"$_"}[-1]->[9];
  printf "%-10s |  %8.1f |  %8.1f |\n",$what{"$_"},$down/1024,$up/1024;
 } else {
  printf("%-10s |         - |         - |\n", $what{"$_"});
 }
}
print "\nLast updated on $lent\n";

[BSDfan666]

This is a raw extraction of the byte-counting code from the netstat command.. this may be useful to someone who's attempting to write their own application.

#include <sys/param.h>
#include <sys/socket.h>
#include <sys/sysctl.h>

#include <net/if.h>
#include <net/if_dl.h>
#include <net/route.h>

int
main(void)
{
	struct if_msghdr ifm;
	int mib[6] = { CTL_NET, AF_ROUTE, 0, 0, NET_RT_IFLIST, 0 };
	struct rt_msghdr *rtm;
	struct if_data *ifd;
	struct sockaddr *sa, *rti_info[RTAX_MAX];
	struct sockaddr_dl *sdl;
	char *buf, *next, *lim;
	char name[IFNAMSIZ];
	size_t len;
	int loop_index;

	if (sysctl(mib, 6, NULL, &len, NULL, 0) == -1)
		err(1, "sysctl");

	if ((buf = (char *)malloc(len)) == NULL)
		err(1, NULL);

	if (sysctl(mib, 6, buf, &len, NULL, 0) == -1)
		err(1, "sysctl");

	lim = buf + len;
	for (next = buf; next < lim; next += rtm->rtm_msglen) {
		rtm = (struct rt_msghdr *)next;
		if (rtm->rtm_version != RTM_VERSION)
			continue;

		if (rtm->rtm_type == RTM_IFINFO) {
			bcopy(next, &ifm, sizeof(ifm));
			ifd = &ifm.ifm_data;

			sa = (struct sockaddr *)(next + rtm->rtm_hdrlen);
			for (loop_index = 0; loop_index < RTAX_MAX; loop_index++) {
				if (ifm.ifm_addrs & (1 << loop_index)) {
					rti_info[loop_index] = sa;
					sa = (struct sockaddr *)((char *)(sa) + 
						roundup(sa->sa_len, sizeof(long)));
				} else {
					rti_info[loop_index] = NULL;
				}
			}

			sdl = (struct sockaddr_dl *)rti_info[RTAX_IFP];
			if (sdl == NULL || sdl->sdl_family != AF_LINK)
				continue;

			bzero(name, sizeof(name));
			if (sdl->sdl_nlen >= IFNAMSIZ) {
				memcpy(name, sdl->sdl_data, IFNAMSIZ - 1);
			} else if (sdl->sdl_nlen > 0) {
				memcpy(name, sdl->sdl_data, sdl->sdl_nlen);
			}

			if(!strcmp(name, "rl0")) {
				printf("%10llu %10llu\n", ifd->ifi_ibytes, ifd->ifi_obytes);
			}
		}
	}
}

The second strcmp argument must be altered to match your Ethernet interface, any LAN traffic over the link is also recorded.

The equivalent command is:
$ netstat -ib
Take care.

[ephemera]

Originally Posted by centerstage

Hello all,

I am trying to find a simple solution to a seemingly common requirement. We need to keep track of total bandwidth by month on our external interface as our
new ISP charges $15/gb over our monthly cap

We also don't need fancy graphs just text is fine.

I have written a daemon for my own amusement. You can try it out.

It can track the b/w usage correctly across reboots or even an OS crash.

the log file /var/log/bwmon.[ifn] can be used to find the b/w usage across any time range.

startup and error messages are logged to /var/log/daemon.

Source code:

/*
 *      Persistant net bandwidth usage monitor daemon 
 *      
 *      Copyright: ephemera @ daemonforums.org
 */
  
#include <sys/types.h>
#include <sys/mman.h>
#include <sys/stat.h>
#ifdef BSD
# include <sys/socket.h>
# include <ifaddrs.h>
#endif /* BSD */
#ifdef SOLARIS
# include <kstat.h>
#endif /* SOLARIS */

#include <net/if.h>

#include <errno.h>
#include <fcntl.h>
#include <limits.h>
#ifdef LINUX
# include <regex.h>
#endif /* LINUX */
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <syslog.h>
#include <time.h>
#include <unistd.h>

#define ARGV0   "bwmond"

typedef unsigned long long      Bwbyte_t;

static char     LOGFILE[PATH_MAX] = "/var/log/"ARGV0".";
static char     RECFILE[PATH_MAX] = "/var/spool/"ARGV0".";
static char     interface[IFNAMSIZ];

static void     usage(void);
static int      update(Bwbyte_t *, Bwbyte_t *);
static void     logpr(int, char *, ...);
static void     err_sys(int, char *, char *);
static void     sig_term(int);

int
main(int argc, char *argv[])
{
        int             i, recfd, logfd;
        time_t          t, t2;
        Bwbyte_t        u, d, last_up, last_down, hr_up, hr_down;
        Bwbyte_t        *record;
        enum            {UP, DOWN};
        int    		record_len = 2 * sizeof(*record);
        const int       interval = 60;     
        int		log_interval = 60 * 5; 
        pid_t           pid;
        sigset_t        sigset;
        struct sigaction       sigact;

        switch (--argc) {
        case 2:
		log_interval = 60 * strtoul(*++argv, (char **)NULL, 10);
                if (log_interval <= 0) 
                        usage();
                /* FALLTHROUGH */

        case 1:
		strncpy(interface, *++argv, sizeof(interface));
                interface[sizeof(interface) - 1] = '\0';
                strncat(RECFILE, interface, sizeof(interface) - 1);
                strncat(LOGFILE, interface, sizeof(interface) - 1);
                break;

        default:
		usage();
        }

        if ( (pid = fork()) < 0) 
                err_sys(1, "fork", "");

        if (pid > 0)
                exit(0);

        setsid();

        chdir("/");

        umask(0);

        for (i = getdtablesize(); i >= 0; --i) 
                close(i); 

        i = open("/dev/null", O_RDWR);
        dup2(i, STDIN_FILENO);
        dup2(i, STDOUT_FILENO);
        dup2(i, STDERR_FILENO);

        sigfillset(&sigset);    
        sigdelset(&sigset, SIGTERM);   
        sigprocmask(SIG_BLOCK, &sigset, NULL);
        sigact.sa_handler = sig_term;
        sigemptyset(&sigact.sa_mask);
        sigact.sa_flags = 0;
        sigaction(SIGTERM, &sigact, NULL);

        openlog(ARGV0, LOG_PID, LOG_DAEMON);
        syslog(LOG_INFO, "Started on %s.", interface);

        if (access(LOGFILE, F_OK) != 0) 
                syslog(LOG_NOTICE, "Creating Logfile: %s. "
			       "If this is not the first time that "ARGV0" is being run then this indicates a Problem."
			       , LOGFILE);

        logfd = open(LOGFILE, O_WRONLY | O_APPEND | O_CREAT, S_IRUSR | S_IWUSR);
        if (logfd < 0) 
                err_sys(1, "open", LOGFILE);

        if (access(RECFILE, F_OK) != 0) {
                recfd = open(RECFILE
                             ,O_RDWR | O_CREAT | O_SYNC
                             ,S_IRUSR | S_IWUSR);

                if (recfd >= 0) {
                        syslog(LOG_NOTICE, "Creating Recordfile: %s. "
			       "If this is not the first time that "ARGV0" is being run then this indicates a Problem."
			       , RECFILE);
                        if (lseek(recfd, record_len, SEEK_SET) < 0)
                                err_sys(1, "lseek", RECFILE);

                        if (write(recfd, "", 1) < 1)
                                err_sys(1, "write", RECFILE);
                }
        } else {
                recfd = open(RECFILE, O_RDWR | O_SYNC);
        }
        if (recfd < 0)
                err_sys(1, "open", RECFILE);

        record = (Bwbyte_t *) mmap(NULL
                                   ,record_len
                                   ,PROT_READ | PROT_WRITE
                                   ,MAP_SHARED
                                   ,recfd
                                   ,0);

        if ( (caddr_t)record == (caddr_t)-1)
                err_sys(1, "mmap", RECFILE);

        close(recfd);

        u = d = last_up = last_down = hr_up = hr_down = 0;
	t = time(NULL);

        while (!update(&hr_up, &hr_down) || !update(&last_up, &last_down))
		sleep(interval);

        for ( ; ; ) {
                if (update(&u, &d)) {
                         if (u < last_up || d < last_down) {
                                last_up = last_down = hr_up = hr_down = 0;
                                continue;
                         }
                         record[UP] += u - last_up;
                         record[DOWN] += d - last_down;
                         last_up = u;
                         last_down = d;
			 t2 = time(NULL);
			 if (t2 - t >= log_interval) {
                                 logpr(logfd
                                       ,"%6llu kb Up  %6llu kb Dn  %10llu / %llu\n"
                                       ,(last_up - hr_up) / 1024 
                                       ,(last_down - hr_down) / 1024
                                       ,record[UP] / 1024
                                       ,record[DOWN] / 1024);
			 	 t = t2;
                                 hr_up = last_up;
                                 hr_down = last_down;
                         }
                         msync((void *)record, record_len, MS_SYNC);
                }
                sleep(interval);
        }
        /* NOTREACHED */
}

static void
err_sys(int quit, char *fn, char *arg)
{

        if (!*arg)
                syslog(LOG_ERR, "Error: %s: %m\n", fn); 
        else
                syslog(LOG_ERR, "Error: %s: %s: %m\n", fn, arg); 

        if (quit) {
                syslog(LOG_ERR, "Terminated due to error on %s.", interface);
                closelog();
                exit(quit);
        }
}

static int
update(Bwbyte_t *up, Bwbyte_t *down)
{
        int    updated = 0;

        errno = 0;

#ifdef BSD
        struct ifaddrs *ifa, *f;

        if (getifaddrs(&ifa) == 0) {
                for (f = ifa; f; f = f->ifa_next) {
                        if (strcmp(interface, f->ifa_name) == 0 && 
                            f->ifa_addr->sa_family == AF_LINK) {
                                *down = ((struct if_data*)f->ifa_data)->ifi_ibytes;
                                *up =((struct if_data*)f->ifa_data)->ifi_obytes;
                                updated = 1;
                                break;
                        }
                }
                freeifaddrs(ifa);
        }
#endif  /* BSD */
#ifdef LINUX
        char            line[2048];
	static char	repatrn[128] = "";
        regmatch_t      rematch[3];
        static regex_t  regex;
        FILE            *pfd;

	if (!*repatrn) {
	        snprintf(repatrn, sizeof(repatrn)
       	        	 ,"^ *%s: *([0-9]+) +[0-9]+ +[0-9]+ +[0-9]+ +[0-9]+" 
       	        	  " +[0-9]+ +[0-9]+ +[0-9]+ +([0-9]+) "
       	        	 ,interface);

                if (regcomp(&regex, repatrn, REG_EXTENDED) != 0)
			repatrn[0] = 0;
	}
        if (*repatrn && (pfd = fopen("/proc/net/dev", "r")) != NULL) {
                while(fgets(line, sizeof(line), pfd) != NULL) {
                        if (regexec(&regex, line, 3, rematch, 0) == 0) {
                                *down = atoll(line + rematch[1].rm_so);
                                *up = atoll(line + rematch[2].rm_so);
                                updated = 1;
                                break;
                        }
                }
        	fclose(pfd);
        }
#endif  /* LINUX */
#ifdef SOLARIS
        kstat_ctl_t     *kc;
        kstat_t         *ksp;
        kstat_named_t   *knp;

        if ( (kc = kstat_open()) != NULL) {
                /* workaround? kstat_open sets errno even on success */
                errno = 0;
                for (ksp = kc->kc_chain; ksp != NULL; ksp = ksp->ks_next) {
                        if (strcmp(ksp->ks_class, "net") == 0 &&
                            strcmp(ksp->ks_name, interface) == 0 &&
                            kstat_read(kc, ksp, NULL) != -1) {
                                if ((knp=kstat_data_lookup(ksp,"rbytes64")) != NULL)
                                        *down = knp->value.ui64;
                                if ((knp=kstat_data_lookup(ksp,"obytes64")) != NULL) {
                                        *up = knp->value.ui64;
                                        updated = 1;
                                        break;
                                }
                        } 
                }
                kstat_close(kc);
        }
#endif  /* SOLARIS */

        if (errno)
                err_sys(0, "update", "");

        return updated;
}

static void
logpr(int logfd, char *msg, ...)
{
#define LOG_LINE_MAX	256
        char    logent[LOG_LINE_MAX];        
	size_t	r;
        time_t  clk;
        va_list ap;

        va_start(ap, msg);
        clk = time(NULL);
        r = strftime(logent, sizeof(logent)
		     ,"%b %e %H:%M:%S %Y: "
		     ,localtime(&clk));
        vsnprintf(logent + r, sizeof(logent) - r, msg, ap);
	if (write(logfd, logent, strlen(logent)) < 0)
		err_sys(0, "logpr", "");

        va_end(ap);
}

static void
usage(void)
{

        fprintf(stderr, "Usage: %s [log interval] interface\n",
                ARGV0);
        exit(1);
}

static void
sig_term(int signo)
{
        
        syslog(LOG_INFO, "Stopped on %s.", interface);   
        closelog();
        exit(0);
}

save the above to a file bwmon.c.

then compile: cc -O2 -DBSD -o bwmond bwmon.c && strip bwmond

you might want to run it from rc.local at startup. Example:

/path/to/bwmond em0 && echo -n ' bwmon'

(note: the interface argument is mandatory.)

[BSDfan666]

Whoo, ephemera used my alterations to the netstat code.

EDIT: While you're not likely to run into this yet, consider changing the size of the interface variable to IFNAMSIZ.
EDIT2: You're using OpenBSD, also consider using strlcpy/strlcat instead of strncpy/strncat..
EDIT3: I'm nagging now, ignore me.. -pedantic, anyway, consider making this daemon privsep, there shouldn't be a need to run this as root.

[ephemera]

Originally Posted by BSDfan666

Whoo, ephemera used my alterations to the netstat code.

I ripped your code in true OSS spirit.
but didn't you say earlier that it was a 'raw' copy?

[BSDfan666]

Originally Posted by ephemera

I ripped your code in true OSS spirit.
but didn't you say earlier that it was a 'raw' copy?

Well, raw-ish, changes some things to make it pretty, and drop a local function. (get_rtaddrs).

It should be noted, Copyright (c) 1983, 1988, 1993 - The Regents of the University of California. All rights reserved.

[J65nko]

Originally Posted by centerstage

We need to keep track of total bandwidth by month on our external interface as our new ISP charges $15/gb over our monthly cap

BTW running a caching nameserver and/or a proxy like squid are great for reducing your bandwidth and thus your costs.

[centerstage]

Wow I am truly blown away by the responses. Thanks a bunch!

Ephemera: "I have written a daemon for my own amusement." that's amazing I will be trying this out asap.

J65nko: All of our "regular" traffic goes out an unlimited connection so there is no worry there. We are already running a proxy to filer out ad's and other various crap.

If anyone was wondering, I ran the script s0xxx suggested for a few days and something is messed up with it. Here is the output:

External interface bandwidth usage:
uptime 10 days
ExtIf in total -1 GBytes
ExtIf out total 1 GBytes
ExtIf in/day 0 MBytes/day
ExtIf out/day 0 MBytes/day
ExtIf in/30day 0 GBytes/month
ExtIf out/30day 0 GBytes/month