[evla-sw-discuss] Repository/Code Organization

[Joe Chavez]

There are really several levels of build containment that need to be 
considered here.

Based on what I've been able to glean from th e-mail traffic I think the 
content here is pertinent. Advanced apologies as I am not directly 
familiar with the current practices and processes at the NRAO.

The basic idea is that each project is stored in a unique repository 
using this structure:

subsytem_1/
  project_1
    tags
    branches
    trunk
      bin
      config
      db
      dist
      docs
      component_1
        bin
        build
        config
        dist
        docs
        module_1
          bin
          build
          config
          docs
          src
          test
        module_2
     OR
        src
        test
    
Under this model a project consists of a set of components which in turn 
are composed of a set of modules. If the module construct is overkill 
particular component it may be omitted.
At each level (project, component, module) there is a set of artifacts 
that represent the script, configuration, distribution, documentation, 
source/test code for that level.
Each level also contains a build file for that is responsible for 
building the source code into distributable components and the packaging 
the related documentation, configuration files, etc. If a module or 
component is intending to export some feature it would put those 
compiled and packaged artifacts in the dist directory. For example a 
typical Java application build would produce compiled Java classes (in 
the build directory), distributable JAR files (in the dist directory) 
and Java docs (in docs/javadocs). Compiling an XML schema into Java 
beans would produce source code, Java classes, distributable JAR files 
and Java docs.
This will encourage binary level sharing between projects so that the 
pedigree of the set of components imported into another project is known 
at all times. This is similar to the concept of using binaries from an 
open source project, sure you can get the source code and build it 
yourself but the stable binary is much more useful. The latter should 
also be placed in the "lib" or "ext" 

 
  evla/
   project_1/
     tags/
     branches/
     trunk/
       pom.xml AND/OR build.xml AND/OR Makefile
       config
       build
       db
       dist/
         jars
         ears
         lib
         wars
       docs
         javadocs  
       lib/
         (imported libraries from internal and external sources)
       modules
         web1
           pom.xml AND/OR build.xml AND/OR Makefile
           pages
           src
           resources
         component1
           pom.xml AND/OR build.xml AND/OR Makefile
           config
           dist
           src
           test
       src
         java
           src/edu/nrao/common/util/
             pom.xml AND/OR build.xml AND/OR Makefile
         python
       test
         java
           src/edu/nrao/common/util/
             pom.xml AND/OR build.xml AND/OR Makefile
         python

I would also recommend placing the top level (below nrao) subsystems 
(evla, sss and vlba) in separate repositories.  Any sharing should be 
done at the binary level (artifacts in the "dist" sub-directories). 
There are several advantages to this approach:

1. A build can be done at any level (component, module, project). A 
subsystem build (builds multiple projects) can be handled by creating a 
project with the same name as the subsystem (at the project level).
2. Each project is versioned independently of the other and avoids 
"revision creep" between unrelated projects.
3. If for some reason a repository becomes corrupt (Subversion IS open 
source) then recovery is limited to the victimized project.
4. In addition to source code otherr artifacts are also versioned such 
as binary distributables, documentation, configuration files, etc.
5. System integration is improved though the use of binary level sharing 
with a known pedigree (svn version number). For example, project_2 
requires components from project_1. project_2 will checkout 
corresponding tag or branch and perform an integration build. If at a 
later date project_2 requires an updated tag or branch then it can use 
"svn switch" to change the URL to the working copy.

That's my 2 cents...

-Joe