Ruby on Rails
TheBigPicture

The following is the partial online version of a thread started on 25-Nov-2004 regarding conceptual approaches to structuring developer’s material for re-use and efficiency of development in rails@lists.rubyonrails.org. Carl Youngblood? suggested I post it here shortly after Nick Van Weerdenburg and Marcel Molina Jr.? responded to my question. Many thanks go to them for those great answers, they helped me a lot. Enjoy…

The Big Picture

Hi,

Tonight I sat down to play arround with Rails like I have done for the last few days and I realized, being new at all of this framework and OO stuff, that it’d be great to be able to have a personal repository of classes that I could share between different projects. This may sound obvious for most of you but OO is an approach that I’m getting familiar with through Ruby. I know it is possible if I make the effort of creating well designed classes maintaining dependencies to a minimum, etc… but what about the big picture as far as how I use all of this from a practical point of view?

How do experienced developpers do it? Do you usually keep a library of classes that you know well available for every project and then mix and match? Is there a way of maybe having a server “serve” these functionality so it can be used by my clients (web services?)? If so, have you seen a setup like this one?

I’ve learnt most of what I know about programming because I enjoy it but my background is in design so I don’t know how a “freelance” developper would go about organizing his material. I’d like think I can focus on an approach that will save me from repeating myself and feeling like I have done everything 100 times.

For example, as far as Rails goes, say I’ve been using it for some time and have a collection of nice controllers scattered arround in various applications. How can I share these between apps while keeping the code in one place or at least a “master” in one place? I know Needle or Needle-like functionality is being integrated but don’t know if this helps.

Imagine having a Rails application with a stucture like this one (this is only for illustration purposses):

/appsrv/
/appsrv/cart/create
/appsrv/cart/list
/appsrv/cart/add_item
/appsrv/cart/mod_item
/appsrv/cart/…

/appsrv/inventory/create
/appsrv/inventory/list
/appsrv/inventory/locations
/appsrv/inventory/add_locations
/appsrv/inventory/add_item
/appsrv/inventory/mod_item
/appsrv/inventory/…

/appsrv/catalog/create
/appsrv/catalog/add_item
/appsrv/catalog/…

Then being able to create an application (maybe even on a different server – a client’s server) and “calling” these different controllers (mixing and matching). Maybe a particular client only needs to have inventory functionality so you only use that controller. I realize this has many performance and security implications (and probably more) but I’m dicussing conceptual approaches to structuring a developpers material for re-use and efficiency of development not necessarily propossing this as the right approach.

Maybe Railing all of it is not the right approach and having organized folders with classes that I can share petween Rail apps is better. But then how do you consume these services or controllers from other servers? I don’t know….

Do you mind sharing your opinions or recommendations on how one should or could approach development?

Thank you,

Luis G. Gómez

Re: The Big Picture

Hi Luis,

Great question. It is a great idea to get an infrastructure in place- that will encourage reuse and continued development of code you often use. Without, you end up repeating yourself a lot or getting lost among a raft of half finished code on your system. The following is my take, which, I think, is pretty generic nowadays. The Pragmatic Starter Kit , is good introduction to a number of the topics.

First off, even as a solo developer, a lot of it depends on the tools you use- version control, test/admin automation, and ide/project tools. Unit testing is also a good idea- it provides documentation and security when you revisit something years later. It also encourages continual development and refactoring, since you aren’t so worried about breaking things.

I’d recommend looking at things from a source perspective, not a distributed application perspective. Distributed apps should be avoided unless it absolutely makes sense due to complexity and performance reasons.

Core considerations are:

1. version control and classification of projects/modules
2. code structure (design, refactoring) and documentation
3. code inclusion techniques to project
4. automation (rake is a great tool for Ruby)
5. evolve your practices to fit your working style- everyone is different

Version Control and Classification

First off, I’d recommend source control – cvs or subversion.
Pragmatic Version Control (one of three books in The Pragmatic Starter Kit)

Then, you need to find a place for code. Not too many, or too few projects.
Maybe something like,

• rails-utils
  All your simple rails utilities. subfolders are fine, but don’t over do it. you generally checkin and checkout the entire module.

• ruby-utils
  Similar to rails_utilities, but for ruby as a whole

• ruby-play
  Hack-space/sandbox, to not make a mess of your nicely resuable stuff. When something becomes more interesting, move to one of the others. I put example code from mailing lists, etc. here for future reference as well as my learning experiments,

• projectX
  Significant work gets it’s own project space. This would akin to the various things on rubyforge packaged as gems. In cvs, you could do this as a subdirectory in a module, or a module by itself.

Code Structure and Documentation

Documentation is critical for resuse, even with yourself. You’ll forget the purpose and nuances of the code, as well as it’s state, pretty quickly if you write lots of code. Learn RDoc , and start using in your codes comments right away.

More importantly, code needs to be written to be reusable, and reusability can be hard – and even not wanted in certain cases.

From a file structure perspective, consider Rails unique needs. Rails has certain requirements due to introspection and directory expectation that need to conformed too (more on code inclusion techniques below). This makes putting a controller and model in the ruby library path unlikely to be successful – they need to know where they are in the context of the rails directory structure.

I think this is fine, because view and controller logic are unlikely to be very resuable anyhow, and there are many, many cool ruby techniques for including reusable code into a framework. Generally, I don’t think I’d want to (or need to) share controller across apps so much as factor out the code, and share that. Controllers needs to uniquely represent the application control flow and logic, and it would strange to find reuse at a controller-level without creating difficult to maintain code. Helpers are probably closer to resusable since they don’t do control/flow logic so much.

What then, when resuability starts to make it’s needs known- you have code in a controller that’s screaming “resuse me! I’m da bomb!”. The code is talking to you, and you should listen.

Thankfully, Ruby is great for managing code structure (and resulting file structures). Refactor, and put the code into a helper, or library that’s included into the view or controller. If the application is growing, and more resuse is needed, create another module or layer for your application – have the controller talke to a service that bundles up the functionality you want to resuse. Rails introduced an actionmailer module recently. It wasn’t appropriate to put the logic in a controller, but rather to reference it. That type of design practice is how you’ll best achieve reuse in rails – low coupling (minimal dependencies between modules), high-cohesion (like functionality grouped together, which in turn helps achieve low coupling and reuse), and DRY (don’t repeat yourself).

In the case that you have something great that is very controler specific, you may want to place your shareable controller code into a module, and include that into a skeleton controller. Or create a singleton (class << self). Or extend the base rails classes via your included ruby file. But in most cases, you can aggregate the functionality without needing to mixin a module, or extend the class (see actionmailer again).

Needle helps with code reuse by enforcing good practice – high modularity with good interfaces – and reducing/controlling coupling between modules. However, good design practices can achieve most of the same thing, and Needle adds a certain complexity. On the other hand, having the environment explicitly define modularity and interfaces is great – it helps push you to continually make the right choices. Test-first development helps similarly, by continually forcing you to use the code and refine the interfaces and modularity. There are a few great artciles on DI with Needle and rails. Dave Thomas made an interesting point on achieving DI without losing clarity of code ( a problem with DI where the code now longer tells the whole story). It sounds like Rails+Needle will be going in this direction. I’m pretty keen on DI after working on a legacy Java codebase with 7 core “singletons” creating the most obscure and horrid spagetti-flow I’ve ever seen.

Botton line- resuability is a design requirement that when present, influences how you design and code. Mix-modules, aggregation, etc. Different types of code have different successful design patterns. MVC, as rails implements, is a great one. Helper classes, also great. Aggregation/Composition is generally better for reuse then inheritence (mix-ins are composition with automatic delegation more then inheritence, so they don’t suffer from the same issues).

Code Inclusion

Now, you need to include your code in your projects. with the goal of not “forking” by cutting and pasting. As mentioned above, certain application frameworks may complicate this, in which case you need to look at code structure. But you still need to get the code into the project you are working on! Start simple (e.g. symlink your work directory into the ruby library parth), and evolve. Here are some options:

1. use the ruby lib path to include your work directories. This allows “require ….” into any script on your system. The downside to this is visibility into your project space if you are continuing to enhance the code while working on the project. Note that you can do this via symlinks, environment variables, or in scripts themselves ($: represents the load path. Add directories to it and you can require away – though it’s probably not an ideal practice if avoidable).

   e.g. $:.push File.dirname(__FILE__) puts the directory of the current file into the loadpath. I often use it at the top of the main ruby file for a package I have in my classpath.

   You could either modify the lib path, or deploy your code into it. While this is great for external code that your don’t modify (e.g. all ruby libraries you download and install), it may not be so good for your own work in progress where you are modifying your library code as much as your app code. Then modifying the ruby lib path to include your work is probably better. Currently I symlink my work directories to /usr/local/lib/ruby/site_ruby/1.8/.

2. use symbolic links to include directories from your main work directory (as checked out of cvs) into your project. Note: you may want deployment directories for your utility code, so that work-in-progress doesn’t break existing apps.

   /work/ruby_utilities
   where you do the development
   /my_libs/ruby/ruby_utilities
   where you depoy various “builds” to (not used by your work-in-progess, which is pointing to the work directory)

   The benefit of this approach is that you can modify the files within the project your are working on, and you modify the source files directly. This way, even if switching about 3 rails projects, the “rails_utilities.rb” file in the project tree is the same file. This can also be done virtually by editors and ideas with project management tools and browsers – you can include multiply projects into the same workspace, hiding the fact they are in different directories.

   Windows has symbolic links for directories- there are just no tools included with windows to create them (the are called junctions). http://www.sysinternals.com/ntw2k/utilities.shtml has a tool called junction that I use a lot.

3. copy the actual files into the project…I’d only recommend this if it’s automated nicely from a the actual work or deployment directory and you don’t modify the copied code (maybe make it read-only). This is an ugly option, and probably a bad idea without really good automation for deloying and refreshing code. It also makes it harder to work on the code at the same time your working on your project in a convenient manner. However, some frameworks may not give you much of a choice due to their structure or nature. This seems less likely in Ruby then in Java. Also, option 2, symbolic links, may be a better option that makes the framework happy. I’d worry about symlinks at the file level though- that could be messy to maintain. I’d be more comfortable at the directory level.

4. what might be cool is to develop a level of automation to package your own code as gems, and then run your own local gems server, for you own use. That would be pretty cool, enabling versions and providing a higher-level deployment architecture (as opposed to the my_libs/ruby directory mentioned above).

You probably want to start simply- maybe just deploy to the ruby environment lib directory (/usr/local/lib/ruby/site_ruby/1.8/ in Unix) on successful tests/builds (using rake). Or symlink your working directories there (my current strategy) . The you can do “require "rails_utilities/myutil” from anywhere.

Later, when more is going on, refine to something that fits you and your code better (e.g. seperation of development and deployment directories).

Automation

Builds, cvs actions, deployments, rdoc generation, unit testing…makes all the best and useful practices automatic

Evolve Your Practices

The above suits my tastes- everyone is different. Also, the volume and nature of the code you write is going to effect practices a lot. However, I think the core principles above are good ones (Pragmattic Programmer Starter Kit is a great overview of most of the above), and while it’s a bit of effort to get it in place initial, there is really low overhead once that’s done. In fact, it easily pays for itself in time-savings. For instance, my commiting to cvs, I don’t have to pollute my directory with mytest.rb.version1 etc. to save history in case I break something. And once you have basic automation in place, all actions are quicker then the manual alternatives, and the collection of automation scripts provide both a source for the next script, as well as great documentation and repeatability of your processes.

Oops. My response is longer then expected. Hope it helps. Helped me – I just cleaned up my code while writing it. I should also note that I work on 4 different machines in a given week, so cvs is essential.

Regards,
Nick

Re: The Big Picture

Great of you to take the time to write this all up and share
Nicholas. Just /w/r/t the two suggestions above, I’d like to point out
that if one is using subversion, externals definitions (the svn:externals
prop) can come in very handy for tasks similar to what you outline above.

More information at:
http://svnbook.red-bean.com/en/1.1/ch07s03.html

marcel

Re: The Big Picture

Thanks. Good points, and a good link. I plan to switch to subversion in the near future, once I’m a bit more comfortable with cvs.

In a similar vein, I realized after sending I forgot to mention using CVS to synch code by checking out into multiple directories/projects and then doing updates/commits to keep the multiple copies synched (one of my original reasons for mentioning cvs). You can do this manually, by checking out at various levels in your project structure. But better yet, you can do a similar thing as mentioned by that subversion link by defining modules that include other modules (called ampersand modules)- so one checkout grabs everything.

e.g. CVSROOT/modules
entirerailsproject1 &railsproject1 &rails-utils &ruby-utils
entirerailsproject2 &railsproject2 &rails-utils &ruby-utils

cvs co entirerailsproject1
cvs co entirerailsproject1
gets the rails project plus the my personal libraries. The libraries are now duplicated, but they are relatively easy to synch via regular update/commits such that it shouldn’t be an issue.

Modules can create nice aliases to project subdirectories, e.g.
railsprojectdocs -a railsproject1/docs

I’ll probably switch to this kind of management structure once I have a bit more ruby code. This is particulary good if working on multiple machines, since it doesn’t require you to update various symbolic links. And if working on a team, you don’t subject your team members to your symbolic link hacks.

Nick