• Call: +1 (858) 429-9131

Archive for May, 2012

Managing Multiple Branches in Subversion

Subversion (SVN) is a centralized storage for sharing information. The Subversion repository is like a time machine. It keeps all the records of every changes ever committed and allows you to explore this history by examining previous versions of files and directories as well as the metadata that accompanies them. With a single Subversion command, you can check out the repository exactly as it was at any date or revision number in the past.

At the core of the Subversion is a repository, which is the central store of that system’s data. Any number of Users connect to the repository, and then read or write to these files according to their permissions. The first Subversion tool we will use is svnadmin. This tool is for administration tasks, like creating repositories, making backup dumps, and the like.

Creating the Repository

To create a repository, open the command line, change the current directory to where you want to create it, and run svnadmin:

[shell]$ cd /usr/local
$ svnadmin create repo[/shell]

Where,
repo = repository name.You can call it whatever you like.

Now the SVN URL is like “svn://localhost:/usr/local/repo“. Subversion uses this repository to store information about your projects, like file revisions.

Importing Projects

The svn import command is a quick way to copy an unversioned tree of files into a repository, creating intermediate directories as necessary. svn import doesn’t require a working copy, and your files are immediately committed to the repository. You typically use this when you have an existing tree of files that you want to begin tracking in your Subversion repository. Each project have a recognizable project root in the repository, a directory under which all of the versioned information for that project—and only that project—lives.To create a project under our repository use the following command:

[shell]$ svn mkdir myproject1 svn://localhost:/usr/local/repo[/shell]

Most projects have a recognizable “main line”, or trunk, of development; some branches, which are divergent copies of development lines; and some tags, which are named, stable snapshots of a particular line of development.

Creating trunk,branches and tags

we suggest that each project root contain a trunk subdirectory for the main development line, a branches subdirectory in which specific branches (or collections of branches) will be created, and a tags subdirectory in which specific tags (or collections of tags) will be created.

[shell]$ svn mkdir svn://localhost:/usr/local/repo/myproject1/trunk -m “trunk created”
$ svn mkdir svn://localhost:/usr/local/repo/myproject1/branches -m “branches created”
$ svn mkdir svn://localhost:/usr/local/repo/myproject1/tags -m “tags created”[/shell]

Here “-m” option is using to add the label. Commonly the label is related to the changes or bug fixes which we made in that commit. It is very important to recognize that particular change which we made in each commit. In the above commands we are creating directories, so we gave the label as “trunk created”.

Then we can list the myproject as showing below:

[shell]$ svn list svn://localhost:/usr/local/repo/myproject1[/shell]

Output:

[shell]trunk/
branches/
tags/[/shell]

We can import our project code to the repository like:

[shell]$ svn import mycode svn://localhost:/usr/local/repo/myproject1/trunk -m “Initial Import”[/shell]

Output:

[shell]Adding mycode/file1
Adding mycode/file2
Adding mycode/file3
Adding mycode/file4

Committed revision 1.[/shell]

Checking out the code

Most of the time, you will start using a Subversion repository by performing a checkout of your project. Checking out a directory from a repository creates a working copy of that directory on your local machine. We can check out the repository by  using the following command:

[shell]$ svn checkout svn://localhost:/usr/local/repo/myproject/trunk[/shell]

Output:

[shell]A trunk/file1
A trunk/file2
A trunk/file3
A trunk/file4

Checked out revision 1.[/shell]

Subversion chose to create a working copy in a directory named for the final component of the checkout URL. This occurs only as a convenience to the user when the checkout URL is the only bit of information provided to the svn checkout command. Subversion’s command-line client gives you additional flexibility, though, allowing you to optionally specify the local directory name that Subversion should use for the working copy it creates.

[shell]$ svn checkout svn://localhost:/usr/local/repo/myproject/trunk code[/shell]

Output:

[shell]A code/file1
A code/file2
A code/file3
A code/file4

Checked out revision 1.[/shell]

If there are more developers working in our project, it is better to use the branches. This allows you to save your not-yet-completed work frequently without interfering with others’ changes and while still selectively sharing information with your collaborators.

To Create branches

Creating a branch is very simple. You make a copy of your project tree in the repository using the svn copy command. Since your project’s source code is rooted in the /repo/myproject/trunk directory, it’s that directory that you’ll copy. You can give the branch name as you wish. Here we are using names of  the users.

[shell]$ svn copy svn://localhost:/usr/local/repo/myproject/trunk svn://localhost:/usr/local/repo/myproject/branches/sam -m “branch 1.0.0″[/shell]

Output:

[shell] Committed revision 2[/shell]

This command causes a near-instantaneous commit in the repository, creating a new directory in revision 2. The new directory is a copy of /repo/myproject/trunk. So each user can copy the /repo/myproject/trunk as their own branch and checkout it like

[shell]$ svn checkout svn://localhost:/usr/local/repo/myproject/branches/sam mycode-sam[/shell]

Output:

[shell]A    mycode-sam/file1
A    mycode-sam/file2
A    mycode-sam/file3
A    mycode-sam/file4

Checked out revision 2.[/shell]

There’s nothing special about this working copy; it simply mirrors a different directory in the repository. When you commit changes, however, the other users won’t see them when they updates, because they are using their own working copy. Whenever someone needs to make a long-running change that is likely to disrupt the trunk, a standard procedure is to create a private branch and commit changes there until all the work is complete. Subversion gives you the ability to selectively “copy” changes between branches. And when you’re completely finished with your branch, your entire set of branch changes can be copied to the staging branch and checkout this to the staging server for testing. If the code is ready to move to the production server then merge into the production branch and then to production server. If we are doing this first time there is no staging and production branches, so we need to copy the trunk as staging and production branches. After that we have to move the changes in the user branches.

Merging

In Subversion terminology, the general act of replicating changes from one branch to another is called merging, and it is performed using the command svn merge.

cd "directory you merge into"

[shell]$ svn switch svn://localhost:/usr/local/repo/myproject/staging
$ svn up
$ svn merge -rAA:BB svn://localhost:/usr/local/repo/myproject/branches/sam
$ svn ci -m ‘Merged branch sam into staging'[/shell]

Where,

AA = the revision number of the staging branch

BB = the revision number of the user branch (sam)

Now the staging branch is updated with the user branch, and we can test this code by checking out to the staging server. For this we need to login to the staging server and checkout the code in the correct location.

[shell]$ svn co svn://localhost:/usr/local/repo/myproject/staging /path/to/code[/shell]

Now the code in the staging branch is checked out to the staging server. We have to test this and if all are fine then merge the staging branch to the production branch and then checkout it to the production server as we done with the staging. Also update the trunk with the latest code in the production by merging it with the production branch, because we need to create branches from trunk for the next release. once we have merged the trunk we can retire from the user branch.

Taging

Now the trunk is same as the branch “production” repository. Then we have to tag this for the future reference. The tag is used to save the releases which are going to the production. So we can easily track the revisions which we updated to the production.

[shell]$ svn copy svn://localhost:/usr/local/repo/myproject/trunk svn://localhost:/usr/local/repo/myproject/tags/first-release[/shell]

You can give the tag name as you wish. Here we gave the tag name as first-release. This is the flow of branching, you have to repeat these process on the future releases too. Because of using the tag we can easily track the releases. So if any issue occurs it is easy to revert the production.

Some useful commands in svn are given below

svn add – To add files and directories to the repository

svn mkdir – Create a directory in the repository

svn copy – Copy from one location to other

svn move – To rename

svn delete – Delete from the repository

svn ci – Commit the changes in the working copy

svn diff – Shows line-level details of a particular change

svn log – Shows you broad information: log messages with date and author information attached to revisions and which paths changed in each revision

svn cat – Retrieves a file as it existed in a particular revision number and displays it on your screen

svn list – Displays the files in a directory for any given revision

svn log – Examine merge-sensitive history

Subversion has commands to help you maintain parallel branches of your files and directories. It allows you to create branches by copying your data, and remembers that the copies are related to one another. The main advantage of branching is that the users can work independently with their own repository and so it will not affect the other users and also easy to maintain.

From CAP, Puppet Now Chef, Evolution of Configuration Management Tools

CHEF, PUPPET & CAPISTRANO are used basically for two purposes  :

Application Deployment is all of the activities that make a software system available for use.

Configuration Management is software configuration management is the task of tracking and controlling changes in the software. Configuration management practices include revision control and the establishment of baselines.

Let me enlighten on how we evolved from the beginning when we were using tools like ssh, scp to the point where we began to abstract and began to equip our-self with these sophisticated yet simple to use tools. Earlier the following tools like

  • ssh which is used as a configuration management solution for admins.
  • scp act as a secure channel for application deployment.

The need for any other tools was out of question until things got complicated!!!

HISTORY

Earlier an Application Deployment  was just a few steps away such as

  1. scp app to production box
  2. restart server (optional)
  3. profit

And these software refreshing/updates were done

  1. Manual (ssh)
  2. with shell scripts living on the servers
  3. or not done at all

CAPISTRANO
(Introduced by Jamis Buck, written in Ruby, initially for Rails project)

Capistrano is a developer tool for deploying web applications. It is typically installed on a workstation, and used to deploy code from your source code management (SCM) to one, or more servers.In its sim­plest form, Capis­trano al­lows you to copy code from your source con­trol repos­i­tory (SVN or Git) to your server via SSH, and per­form pre & post-de­ploy func­tions like restart­ing a web­server, bust­ing cache, re­nam­ing files, run­ning data­base mi­gra­tions and so on.

Nice things cap introduced :

  1. Automate deploys with one set of files
  2. The files don’t have to live on the production server
  3. The language (Ruby) allows some abstraction

Now application deployment step can be coded and tested like rest of the project. It has also become the de facto way to deploy the Ruby on Rails applications. It has also had tools like webistrano build on top of it to provide a graphical interface to the command line tool.

Drawback : The tool seems to be widely used but not well supported.

PUPPET

(Written in Ruby and evolved from cfengine)

Luke Kanies came up with the idea for Puppet in 2003 after getting fed up with existing server-management software in his career as a systems administrator. In 2005 he quit his job at BladeLogic, a maker of data-center management software, and spent the next 10 months writing code to automate the dozens of steps required to set up a server with the right software, storage space, and network configurations. The result: scores of templates for different kinds of servers, which let systems administrators become, in Kanies’s metaphor, puppet masters, pulling on strings to give computers particular personalities and behaviors. He formed Puppet Labs to begin consulting for some of the thousands of companies using the software—the list includes Google, Zynga, and Twitter etc

Puppet is typically used in a client server formation, with all your clients talking to one or more servers. Each client contacts the servers periodically (every half an hour by default), downloads the latest configuration and makes sure it is sync with that configuration.

The Server in Puppet is called Puppet Master.
Puppet Manifests contains all the configuration details which are declarative as opposed to imperative.

The DSL is not Ruby as you are not writing scripts you are writing definitions, Install order is determined through dependencies.
The Puppet Master is idempotent which will make sure the client machines match the definitions.This is good as you can implement changes across machines automatically just by updating the manifest in the Puppet Master.

CHEF
(written in ruby evolved from puppet)

CHEF is an open source configuration management tool using pure-Ruby, the chef domain specific language for writing system configuration related stuff (recipes and cookbook)
CHEF brings a new feel with its interesting naming conventions relating to cookery like Cookbooks (they contain codes for a software package installation and configuration in the form of Recipes), Knife (API tool), Databags (act like global variables) etc

Chef Server – deployment scripts called Cookbooks and Recipes, configuration instructions called Nodes, security details etc. The clients in the chef infrastructure are called Nodes. Chef recipes are imperative as opposed to declarative. The DSL is extended Ruby so you can write scripts as well as definitions. Install order is script order NO dependency checking.

CHEF & PUPPET

Chef and Puppet automatically set up and tweak the operating systems and programs that run in massive data centers and the new-age “cloud” services, designed to replace massive data centers.

Chef Recipes is more programmer friendly as it is easily understood by a developer unlike a Puppet Manifest.

And when it comes to features in comparison to puppet, chef is rather more intriguing .
For example “Chef’s ability to search an environment and use that information at run time is very appealing.

Knife is Chef’s powerful command line interface. Knife allows you to interact with your entire infrastructure and Chef code base. Use knife to bootstrap a server, build the scaffolding for a new cookbook, or apply a role to a set of nodes in your environment. You can use knife ssh to execute commands on any number of nodes in your environment. knife ssh + search is a very powerful combination.

The part of defining dependencies in Puppet was overly verbose and cumbersome. With Chef, order matters and dependencies would be met if we specified them in the proper order.

We can deploy additional software applications on virtual machine instances without dealing with the overhead of doing everything manually,” Stowe explains. “We can do it with code — recipes that define how various applications and libraries are deployed and configured.” According to Stowe, creating and deploying a new software image now takes minutes or hours rather than hours or weeks. They call this technique DevOps because it applies traditional programming techniques to system administration tasks. “It’s just treating IT operations as a software development problem, – Stowe, CEO of Cycle Computing, a Greenwich, Connecticut-based start-up that uses Chef to manage the software underpinning the online “supercomputing” service it offers to big businesses and academic outfits. “Before this, there were ways of configuring servers and managing them, but DevOps has gotten it right.”

Lets CATEGORIZE

Let me help you to know onto which buckets does the above tools fell into and other similar tools…

App Deploy Capistrano, ControlTier, Fabric, Fun, mCollective
SysConfig Chef, Puppet, cfengine, Smart Frog, Bcfg2
Cloud/VM Xen, Ixc, openVZ, Eucalyptus, KVM
OS Install Kickstart, Jumpstart, Cobbler, OpenQRM, xCAT