* Running the programs Should've been started by init: - postgres Should've been started by init. If not, do sudo /etc/init.d/postgresql-8.3 start - Filter port 50001, where BGPMon runs sudo ~linus/fw.sh - BGPMon -- runs in foreground cd ~linus/bgpmon && sudo ~frank/bgpmon_v6/bgpmon - bgpstore -- runs in foreground sudo linus ~linus/src/bgp-logger/src/start-bgpstore.sh victoria 50001 bgpstore bgpstore bgpstore localhost - psql -- for peeking in the database (pw = bgpstore) psql -h localhost bgpstore bgpstore * BGPMon host: victoria.tug.nordu.net telnet localhost 50000 # CLI for the logger software, pw=nordunet nc localhost 50001 # Stream of log data. The product is called [[http://bgpmon.netsec.colostate.edu/][BGPmon]]. There's XFB, an [[http://tools.ietf.org/html/draft-cheng-grow-bgp-xml-00][XML format]] for BGP messages and control information, a.k.a. XFB. There are two scripts in tumleren.pilsnet.sunet.se:~frank called client_dom.pl and client_sax.pl. They've been modified by Frank. There's a version 6.1 of the BGPmon sw announced on their website but one needs userid & pw to get at it. I've contacted He Yan ([[gnus:nnimap%2Bimap.nordu.net:INBOX.sent-mail.2009-06#52][Email from Linus Nordberg: BGPmon version]]) about that. <2009-06-22 Mon 22:23> Turns out that what they mailed Frank was actually 6.1, which is what we're running. The program (bpgmon) must be run as root since it uses source port 179 for the BGP session with se-tug. Also, I had to fix a syntax error in the config file found in Franks home directory which looked like it should've read 1 so now it does just that. * Running postgresql To start the server sudo sudo -u postgres /opt/local/lib/postgresql83/bin/postgres -D /opt/local/var/db/postgresql83/defaultdb or sudo -i -u postgres /opt/local/lib/postgresql83/bin/pg_ctl -D /opt/local/var/db/postgresql83/defaultdb -l logfile start * db layout <2009-06-24 Wed> Discussions with Fredrik. - We should store one entry per prefix that has changed. This means that one UPDATE message can result in more than one entry. - TIMESTAMP isn't enough, there's often more than one message per second. We'll have to store PRECISION_TIME too. - Because of TIMESTAMP (above), we need some other primary key, like an id with AUTO_INCREMENT. (Can we make postmodern add that?) Update: It's called [[http://www.postgresql.org/docs/8.3/interactive/datatype-numeric.html#DATATYPE-SERIAL][SERIAL]] in PostgreSQL. Use 'bigserial' if we anticipate more than 2^31 entries. ':col-type serial' should probably do it. - We've identified, from the perl program output, the following fields to be of interest: - TIMESTAMP (int32) - PRECISION_TIME (int) - WITHDRAWN (list of prefixes), possibly including label - NLRI (list of prefixes), possibly including label - AS_PATH (list of integers) - NEXT_HOP (int32) - I think we care only about UPDATE messages. - We also should store OCTETS in OCTET_MSG when TYPE is UPDATE (2). Just in case we find out that we missed something. This is the complete BGP message. * TODO - [X] store timestamp and precision-time - [X] store as-path - [X] move to victoria - [X] store nexthop - [ ] store octets as binary, in separate table - [ ] limit access to bgpmon stream (acl's) - [ ] get an idea of disk footprint - [ ] get an idea of RAM footprint - [ ] get an idea of CPU load, sbcl and postgres - [ ] auto-start on boot - [ ] reconnect * bgpstore Disk used (whole system) early morning 2009-06-26: 1781468 Kb. | date | ~ 10^6 rows | disk used (Kb) | RAM used (Kb) | sbcl VSZ/RSS | |------------------------+-------------+----------------+---------------+--------------| | <2009-06-26 Fri 17:12> | | 1822492 | 499964 | | | <2009-06-27 Sat 12:30> | | 1866388 | 496028 | 536224 36420 | | <2009-06-27 Sat 22:42> | | 1885236 | 496924 | 536224 43300 | | <2009-06-28 Sun 07:43> | | 1899988 | 500164 | 536224 42852 | | <2009-06-29 Mon 19:57> | | 1983632 | 499472 | 536224 41612 | | <2009-07-01 Wed 12:36> | | 2111188 | 485120 | 536224 42840 | | <2009-07-03 Fri 16:28> | | 2242152 | 500028 | 536224 44992 | | <2009-07-06 Mon 00:01> | | 2392316 | 499656 | 536224 45388 | | <2009-07-12 Sun 12:54> | 2.9 | 2831324 | 499128 | 536224 42864 | | <2009-07-16 Thu 22:04> | 3.86 | 3330048 | 499752 | 536224 45452 | | | | | | |