Archive for April, 2006

Taming Monsters – Refactoring with Method Objects

Saturday, April 22nd, 2006

I’ve started refactoring the actual SnmpCollector code in an attempt to simplify a few things as a prepare to reorganize. I figured I’d start at the beginning. WHOA! Have you seen the initialize method in there. Check it out in 1.2! This one methods was hundreds of lines longs generated and/or manipulated at least 3 Maps, piles of Lists, and on top of that dove into the database for a number of different things. In one case it even made database calls inside a loop processing a result set! Needless to say when I look at it I was completely overwhelmed!

Fortunately for me I had just been discussing refactoring with Dave and Mike Huot. I had been going through the examples from the refactoring book and had been showing Dave some of the work I had done on Collectd. I had noticed a comment that said [paraphrased], “If you run across a very complex method with large number of local variables, turn it into a method object and then refactor.” I had been a little skeptical with respect to how this would really help but when I came across this method I decided I would give it a try.

I created an object called Initializer. The purpose of this object was just to have one public method called execute that did all the same things as the ‘initialize method did. I created an instance of the object and moved the method into the Initializer class using Eclipse. Next I went through the object and, again using Eclipse, I made all the locals into fields. This at first doesn’t really do anything to help the complexity of the code but it makes the next steps possible which make things MUCH easier.

Once all of the locals are fields, Then it is a simple matter to go through the code and extract methods that perform various useful functions. By simple doing this I was able to turn the initialize into the following:


	initialize();
	/*
	 * All database calls wrapped in try/finally block so we make certain
	 * that the connection will be closed when we are finished.
	 */
	allocateDBConn();
	try {
		getPrimarySnmpInfo();

		getSysObjId();

		createNodeInfo();

		determineSnmpVersion();

		getSnmpInfoForInterfaces();

		verifyCollectionIsNecessary();

	} finally {
		closeDBConn();
	}

	logCompletion();

The code in these methods is still quite ugly but now I have a number of methods with a much more easily defined purpose. I am also able to do the same thing to those method as I need to. The ultimate goal of course is to turn most of this function into operations on the appropriate objects as this makes things MUCH simpler. I’m not finished with this object yet but I suspect by the time I finish this object will ‘appear’ to do nothing more than validate the parameters to the initialize method.

This refactoring thing just keeps getting better and better. But knowing refactoring and actually doing refactoring sure are very different and real mastery comes with practice. I sure wish I had this rectoring skilll when I worked on the poller!

Well, happy refactoring!
Matt

DAOs. They’re Not Just for Breakfast Anymore!

Wednesday, April 19th, 2006

Well, The last day or two has been quite a challenge. I’ve made some significant progress on the refactoring of collectd, but got held up for two key reasons. One is that I had gotten past the completely mechanical refactorings to the more sophisticated and I realized that the tests I had (or didn’t have) were not going to do it. Soooo… I had to start figuring out how to write some collectd tests. The second hindrance to progress was the fact that I had been ‘pussy footing’ around the ConfigFactories. They all but stopped my efforts to write tests and were making the refactoring difficult because some of the functions in the factory need to be on other objects.

So all this to say that today, I started refactoring the CollectionConfigFactory to simplify it. This effort has been very educational and has helped to solidify in my mind exactly what a ‘configuration DAO’ needs to look like and how it should be implemented. Hopefully I will be able to complete this soon as this is a prerequisite to abstracting the RRD code out.

Anyway, stay tuned as there is much more to come.

Matt

Are we there yet?

Tuesday, April 18th, 2006

Well I didn’t make it on Thursday and I had Friday off for the holiday weekend. So it’s now Tuesday and I’m finally getting back to things. I spent some more time Monday working on Collectd and feel like things have gotten quite a bit simpler in there. I’ve been working primarily with Collectd.java and CollectableService.java trying to ‘find’ a good set of objects that describes what is going on in this set of code. I’ve introduced a few objects that make the code seem quite a bit simpler. Still need to work on the event handling though as well as the scheduling. THEN there’s SNMPCollector. That will be where the fun really begins.

Oh yeah, I also wanted to mention that I found a few places in the main collector that assume we are only collectiong SNMP data so I’ll have to figure out what to do about that.

Anyway, more to come,

Matt

DataSources Galore!

Tuesday, April 11th, 2006

Well, mhuot and i had a great time pairing this morning. It was one of the more product pairing sessions we’ve had in a while. A while back Dave and I implemented the EventTrasnlator service that can create ‘standard’ kinds (or any kind actually) of events from other events such as snmp traps. This is very handy for setting up notifications about unusual things or things that are difficult to monitor directly with OpenNMS. One of the features we added was the ability to put SQL queries in. Mike wanted to be able to define databases other than the OpenNMS database and use queries against them to aid in the translation of events. This is a great idea! In order to help do this Mike and I added pretty simple multi-database support to the DatabaseConnectionFactory. This will make it simple for a user to configure a new database into opennms and for the EventTranslator to use it to make queries.

After that I merged a few fixes into a branch a for Chuck Thier at Rackspace and then up to HEAD from 1.2. These were JDBCMonitor and JDBCPlugin stuff from Tobey Pasheilich along with a Pllugin/Monitor pair for calling a stored procedure in the database. Pretty cool actually.

Heading off to the North Carolina Zoo tomorrow.. So see y’all on Thursday!

Good Job today Mike!
Matt

Bonzai!!!!

Tuesday, April 11th, 2006

Before I started today I wanted to give a quick description of what I did yesterday. Jumped right into refactoring collectd finally! Not that I have some DAOs written and some ideas for how the others should be written I just took my machete out and started hacking away. Since collectd has very little in the way of tests I have been trying hard to just move code around and not change any real function. But hey! It already looks much better and I’ve been working on only two files! Collectd.java and SnmpCollector.java. Today I’m going to try and tackle CollectableService.java and then get back to SnmpCollector.java. I just can’t wait! When I’m done Collectd will hopefully be ‘DAO’ed’ and easy to understand! After that I think I’ll have to take on graphing in the web app. Oh Baby!!!

Matt

If you fail to plan, you plan to fail

Saturday, April 8th, 2006

Well the interest in OpenNMS has been growing a lot lately and as I result it seems that I have more work than I could possibly do. I’m very tempted to be stressed out and it seems that a lot of us at The OpenNMS Group are, but… this is exactly what we’ve been wanting. To see OpenNMS grow into the project it should always have been — A true enterprise grade network management system able to manage tens of thousands of nodes, millions of data points and many users. Fortunately there are a number of companies and individualts that would like to see us reach that same goal. They’ve either sponsored a number of development projects that will move OpenNMS forward or worked hard to contrib effort to the project. And some have done both! What an exciting opportunity!

The unfortunately reality however is that there is a TON of work to do and what seems to be only a very short time to do it. There are a number of initiatives that we’ve started to add more features as well as to make OpenNMS easier to develop. But will all the work that we’ve been doing none of these initiatiatives have made it all the way to completion. Sooooo….

For the last two days Dave and I have worked to develop a comprehensive development plan that covers all of the sponsored development work as well as all the ‘initiatives’ that we need to get some of them done. Early next week we hope to put the beginnings of a detailed plan up on the wiki that we will be driving toward for the next few months.

The objective of the plan is to divide the project into manageable steps that some one that wants to contribute will be able to just grab a hold of one small piece and make it happen. And to give some clarity to everyone who watches the OpenNMS project to know where we are and where we are going. Lastly, it will help us to provide some organization for our time so we can make progress.

So… Stay Tuned for some good work that will give you a good idea of where we are going and how you can help us get there.

See you next time!
Matt