* Running the programs
Should've been started by init:
- postgres
  Should've been started by init.  If not, do
  sudo /etc/init.d/postgresql-8.3 start

- Filter port 50001, where BGPMon runs
  sudo ~linus/fw.sh

- BGPMon -- runs in foreground
  cd ~linus/bgpmon && sudo ~frank/bgpmon_v6/bgpmon

- bgpstore -- runs in foreground
  sudo linus ~linus/src/bgp-logger/src/start-bgpstore.sh victoria 50001 bgpstore bgpstore bgpstore localhost

- psql -- for peeking in the database (pw = bgpstore)
  psql -h localhost bgpstore bgpstore

* BGPMon
host: victoria.tug.nordu.net

telnet localhost 50000  # CLI for the logger software, pw=nordunet
nc localhost 50001	# Stream of log data.

The product is called [[http://bgpmon.netsec.colostate.edu/][BGPmon]].  There's XFB, an [[http://tools.ietf.org/html/draft-cheng-grow-bgp-xml-00][XML format]] for BGP
messages and control information, a.k.a. XFB.

There are two scripts in tumleren.pilsnet.sunet.se:~frank called
client_dom.pl and client_sax.pl.  They've been modified by Frank.

There's a version 6.1 of the BGPmon sw announced on their website but
one needs userid & pw to get at it.  I've contacted He Yan ([[gnus:nnimap%2Bimap.nordu.net:INBOX.sent-mail.2009-06#52][Email from
Linus Nordberg: BGPmon version]]) about that.  <2009-06-22 Mon 22:23>
Turns out that what they mailed Frank was actually 6.1, which is what
we're running.

The program (bpgmon) must be run as root since it uses source port 179
for the BGP session with se-tug.  Also, I had to fix a syntax error in
the config file found in Franks home directory which looked like it
should've read

                <LABEL_ACTION>1</LABEL_ACTION>

so now it does just that.
* Running postgresql
To start the server

    sudo sudo -u postgres /opt/local/lib/postgresql83/bin/postgres -D /opt/local/var/db/postgresql83/defaultdb
or
    sudo -i -u postgres
    /opt/local/lib/postgresql83/bin/pg_ctl -D /opt/local/var/db/postgresql83/defaultdb -l logfile start
* db layout
<2009-06-24 Wed> Discussions with Fredrik.  

- We should store one entry per prefix that has changed.  This means
that one UPDATE message can result in more than one entry.

- TIMESTAMP isn't enough, there's often more than one message per
  second.  We'll have to store PRECISION_TIME too.

- Because of TIMESTAMP (above), we need some other primary key, like
  an id with AUTO_INCREMENT.  (Can we make postmodern add that?)
  Update: It's called [[http://www.postgresql.org/docs/8.3/interactive/datatype-numeric.html#DATATYPE-SERIAL][SERIAL]] in PostgreSQL.  Use 'bigserial' if we
  anticipate more than 2^31 entries.  ':col-type serial' should
  probably do it.

- We've identified, from the perl program output, the following fields
  to be of interest:
  - TIMESTAMP (int32)
  - PRECISION_TIME (int)
  - WITHDRAWN (list of prefixes), possibly including label
  - NLRI (list of prefixes), possibly including label
  - AS_PATH (list of integers)
  - NEXT_HOP (int32)

- I think we care only about UPDATE messages.

- We also should store OCTETS in OCTET_MSG when TYPE is UPDATE (2).
  Just in case we find out that we missed something.  This is the
  complete BGP message.
* TODO
- [X] store timestamp and precision-time
- [X] store as-path
- [X] move to victoria
- [X] store nexthop
- [ ] store octets as binary, in separate table
- [ ] limit access to bgpmon stream (acl's)
- [ ] get an idea of disk footprint
- [ ] get an idea of RAM footprint
- [ ] get an idea of CPU load, sbcl and postgres
- [ ] auto-start on boot
- [ ] reconnect
* bgpstore
Disk used (whole system) early morning 2009-06-26: 1781468 Kb.
| date                   | ~ 10^6 rows | disk used (Kb) | RAM used (Kb) | sbcl VSZ/RSS |
|------------------------+-------------+----------------+---------------+--------------|
| <2009-06-26 Fri 17:12> |             |        1822492 |        499964 |              |
| <2009-06-27 Sat 12:30> |             |        1866388 |        496028 | 536224 36420 |
| <2009-06-27 Sat 22:42> |             |        1885236 |        496924 | 536224 43300 |
| <2009-06-28 Sun 07:43> |             |        1899988 |        500164 | 536224 42852 |
| <2009-06-29 Mon 19:57> |             |        1983632 |        499472 | 536224 41612 |
| <2009-07-01 Wed 12:36> |             |        2111188 |        485120 | 536224 42840 |
| <2009-07-03 Fri 16:28> |             |        2242152 |        500028 | 536224 44992 |
| <2009-07-06 Mon 00:01> |             |        2392316 |        499656 | 536224 45388 |
| <2009-07-12 Sun 12:54> |         2.9 |        2831324 |        499128 | 536224 42864 |
| <2009-07-16 Thu 22:04> |        3.86 |        3330048 |        499752 | 536224 45452 |
|                        |             |                |               |              |