java - Keeping track of utility classes -

i've been more , more frustrated problem see emerging in projects code-base.

i'm working on large scale java project has >1m lines of code. interfaces , class structure designed , engineers writing code proficient. problem in attempt make code cleaner people write utility classes whenever need reuse functionality, result on time , project grows more , more utility methods crop up. however, when next engineer comes across need same functionality has no way of knowing had implemented utility class (or method) somewhere in code , implements copy of functionality in different class. result lot of code duplication , many utility classes overlapping functionality.

are there tools or design principles team can implement in order prevent duplication , low visibility of utility classes?

example: engineer has 3 places needs transform xml string writes utility class called xmlutil , places static tostring(document) method in it. engineer b has several places serializes documents various formats including string, writes utility class called serializationutil , has static method called serialize(document) returns string.

note more code-duplication quite possible 2 implementations of above example different (say 1 uses transformer api , other uses xerces2-j) can seen "best-practices" problem well...

update: guess better describe current environment develop in. use hudson ci, clover code coverage , checkstyle static code analysis. use agile development including daily talks , (perhaps insufficient) code reviews. define our utility classes in .util due it's size has 13 sub-packages , 60 classes under root (.util) class. use 3rd party libraries such of apache commons jars , of jars make guava.

i'm positive can reduce amount of utilities half if put on task of refactoring entire package, wondering if there tools can make operation less costly, , if there methodologies can delay as possible problem recurring.

your problem common one. , real problem too, because there no solution.

we in same situation here, i'd worse, 13 millions line of code, turnover , more 800 developers working on code. discuss same problem describe.

the first idea - developers have used - refactor common code in utility classes. our problem solution, pair programming, mentoring , discussion, many effective. in fact grow in subteams, people sharing knowledge in subteam, knowledge doesn't transit between subteams. maybe wrong think pair programming , talks can't in case.

we have architecture team. team responsible deal design , architecture concerns , make common utilities might need. team in fact produces call corporate framework. yes, framework, , works well. team responsible push best practices , raise awareness of should done or not, available or not.

good core java api design 1 of reason java success. third party open sources libraries count lot too. small crafted api allows offer useful abstraction , can reduce code size lot. know, making framework , public api not same thing @ coding utility class in 2 hours. has high cost. utility class costs 2 hours initial coding, maybe 2 days debugging , unit tests. when start sharing common code on big projects/teams, make api. must ensure perfect documentation then, readable , maintainable code. when release new version of code, must stay backward compatible. have promote company wide (or @ least team wide). 2 days small utility class grow 10 days, 20 days or 50 days full-fledged api.

and api design may not great. well, not engineers not bright - indeed are. willing let them work 50 days on small utility class parsing number in consistent way ui? willing let them redesign whole thing when start using mobile ui totally different needs? have noticed how brightest engineers in word make apis never popular or fade slowly? see, first web project made used internal frameworks or no framework @ all. added php/jsp/asp. in java added struts. jsf standard. , thinking using spring web flow, vaadin or lift...

all want there no solution, overhead grows exponentially code size , team size. sharing big codebase restricts agility , responsiveness. change must done carefully, must think of potential integration problems , must trained of new specificities , features.

but main productivity point in software company not gain 10 or 50 lines of code when parsing xml. generic code grow thousand lines of code anyway , recreates complex api layered utility classes. when guy make utility class parsing xml, abstraction. give name 1 dozen or 1 hundred lines of specialized code. code useful because specialized. common api allows work on streams, url, strings, whatever. has factory can choose parser implementation. utility class because work parser , strings. , because need 1 line of code call it. of course, utility code of limited use. works mobile application, or loading xml configuration. , that's why developer added utility class in first place.

in conclusion, consider instead of trying consolidate code whole codebase split code responsibility teams grow:

• transform big team work on 1 big project small teams work on several subprojects;
• ensure interfacing minimize integration problems, let team have own code;
• inside theses teams , corresponding codebases, ensure have best practices. no duplicate code, abstractions. use existing proven apis community. use pair programming, strong api documentation, wikis... should let different teams make choices, build own code, if means duplicate code across teams or different design decisions. know, if design decisions different may because needs different.

what managing complexity. in end if make 1 monolithic codebase, generic , advanced one, increase time newcomers ramp up, increase risk developers not use common code @ all, , slow down because change has far greater chances break existing functionality.