In Progress

Before I started today I wanted to give a quick description of what I did yesterday. Jumped right into refactoring collectd finally! Not that I have some DAOs written and some ideas for how the others should be written I just took my machete out and started hacking away. Since collectd has very little in the way of tests I have been trying hard to just move code around and not change any real function. But hey! It already looks much better and I’ve been working on only two files! Collectd.java and SnmpCollector.java. Today I’m going to try and tackle CollectableService.java and then get back to SnmpCollector.java. I just can’t wait! When I’m done Collectd will hopefully be ‘DAO’ed’ and easy to understand! After that I think I’ll have to take on graphing in the web app. Oh Baby!!!

Matt

Well the interest in OpenNMS has been growing a lot lately and as I result it seems that I have more work than I could possibly do. I’m very tempted to be stressed out and it seems that a lot of us at The OpenNMS Group are, but… this is exactly what we’ve been wanting. To see OpenNMS grow into the project it should always have been — A true enterprise grade network management system able to manage tens of thousands of nodes, millions of data points and many users. Fortunately there are a number of companies and individualts that would like to see us reach that same goal. They’ve either sponsored a number of development projects that will move OpenNMS forward or worked hard to contrib effort to the project. And some have done both! What an exciting opportunity!

The unfortunately reality however is that there is a TON of work to do and what seems to be only a very short time to do it. There are a number of initiatives that we’ve started to add more features as well as to make OpenNMS easier to develop. But will all the work that we’ve been doing none of these initiatiatives have made it all the way to completion. Sooooo….

For the last two days Dave and I have worked to develop a comprehensive development plan that covers all of the sponsored development work as well as all the ‘initiatives’ that we need to get some of them done. Early next week we hope to put the beginnings of a detailed plan up on the wiki that we will be driving toward for the next few months.

The objective of the plan is to divide the project into manageable steps that some one that wants to contribute will be able to just grab a hold of one small piece and make it happen. And to give some clarity to everyone who watches the OpenNMS project to know where we are and where we are going. Lastly, it will help us to provide some organization for our time so we can make progress.

So… Stay Tuned for some good work that will give you a good idea of where we are going and how you can help us get there.

See you next time!
Matt

Well today started a little slow. Spent the morning at home perusing some technology tutorials for some things I’d like to learn more about. Finally headed into the office for ‘Cue Friday with TOG and was just about to get started on some more of the ‘build’ work I was doing (see yesterday’s entry) when Dave leans over and says, “You got a minute to look at the RTC code with me?”
I figured that I may as well help out now since I hadn’t really started anything. Dave goes on to tell me that a customer of TOG has a VERY large network they are trying to monitor (They have over 40,000 nodes and even more ipInterfaces in their database and are hoping to grow to over 80,000.) They have been doing some performance tuning by adjusting polling and collection intervals and such and is having a difficult time because RTCd is talking 3.5 hours to finish its ‘init’ method. This makes experimental changes very painful not to mention how it impacts the users so we decided to have a look and figure out what’s taking so long.

Well, it turns out that I have seldom seen such poorly thought out DB code. I turns out the the number of queries that this thing did start was something of the order O(numCategories*numIfsPerCat*numSvcPerIf) This customer had 30 categories and about 3-4 services per interface. So that meant 30 * 3-4 * numIfsPerCategory = 100 * numIfsPerCat. So if they average 1000 interfaces per category, they get 100,000 DB calls to make! This is a huge! With that many queries in 3.5 hours the queries are only taking about 120ms each which seems pretty fast.

Anyway, the code was pretty convoluted so it wasn’t any wonder its never been ‘improved.’ Some pretty major refactoring and getting rid of some really stupid code we were able to know the number of queries down to about numCategories + 1 which is quite a bit better. Dave dumped the database they were using and loaded it onto his 1.25GHz Powerbook and the new code was able to start in only about 16 minutes.

I haven’t heard yet how much faster it is on the customer system but I expect it will be very fast. I’ll let you know when I hear. So if you’ve been having trouble with a lot of categories on your system then you might think about experimenting with the code in HEAD.

Anyway, off to help with a scollable cursor problem I just heard about.

Have a nice weekend!

Matt