WebSpider Improvement

Thanks to the two days extension, now I am finally able to finish this assignment. I improved my program so that now it can perform all the tasks, although I am not sure is it performing it correctly since there is no official correct answer. My source code can be found in here.

My external resources were mostly from my classmates’ codes and the discussion from google group, especially the ones from Jared Sunouchi and Ka Yee Leung. Also, the test website created by Randy Cox was very helpful. I could have a general idea of whether I am doing the correct things or not.

During the time I tried to accomplish task 2, I found out that finding the most popular page and finding the total number links share a lot of same code. I tried to merge these two methods together, but these methods return two different types of values, which is troublesome to fix. Therefore, I kept them separated. For the task 3, the java logging package turned out to be easier to use than I expected. It is like a more sophisticated System.out.println. I think I will start using it more for the later programs in this class.

The QA tools’ testing was ok. However, findbugs and PMD sometimes will report error that is counterpart for each other, which took me a while to figure out a solution. For the Emma, I only got the 100% coverage for classes and methods level, but I think that should be sufficient since the JUnit tests were mostly written to satisfy the coverage in Emma, they did not guarantee high quality codes anyways.

Overall, this assignment is very challenging. I feel lucky that I could have the extension to finish it, also thanks to my smart classmates. Their clever ideas have helped me a lot in this assignment.