Using Bamboo for Deployments

As part of a project involving the complete redevelopment of a large UK based loyalty programme website, it was also necessary to assess the development processes that were being used and identify areas of improvement.

The status quo of the software lifecycle was that changes that were required on the server side were developed offshore and then the skeleton JSP files were provided to the onshore web team who would apply the styling (CSS) and front end validation (Javascript). These JSP files were managed and edited within the CMS which resulted in content editors having to know technologies such as HTML, CSS and JSP. Consequently, the web team were burdened with managing the content for the live website, usually at very short notice. Furthermore, the packaging of the server side, CMS and additional applications were bundled as one monolithic Java Enterprise Archive (EAR) distribution and deployed manually on application servers. Testing was also completely manual and although there were a number of test environments, each was configured slightly differently and not representative of the production environment. The entire deployment process was very labour intensive and consequently release into production (performed by a separate production support team) was a very exhausting and painful experience for those involved. Releases had to be scheduled every two weeks and performed in the early hours of the morning with a number of people on-call to provide support.

Instead of reusing any of the existing codebase, it was decided to start from the ground-up using the a new CMS as the platform to develop the new website that would serve multiple channels and also facilitate the marketing team to manage its content.

The business was also undergoing a transition from a traditional 'waterfall' method of development to using 'agile' processes. Behaviour Driven Development (BDD) was employed to capture and refine requirements.

There were also a number of technologies and tools adopted to deliver enhance productivity and deliver value. We decided to abandon the heavyweight enterprise application server as there was no need for the additional features that it provided, and use the more lightweight Tomcat servlet container. Jira [1] was used to capture requirements as stories, to raise issues and defects, and generally plan and track the project. Confluence [2] was adopted for documentation purposes and as a collaboration tool to discuss designs. Bamboo [3] was used as the continuous integration and build tool and all new source code was required to have unit tests. Functional and acceptance tests were developed using Cucumber [4] and Ruby [5] to create an automated regression testing suite.

Bamboo was configured to monitor every time code was committed to version control and trigger a build and test. With third-party plugins such as Sonar, the team was provided with visibility of test coverage and code quality. Using Bamboo to perform deployments into all environments also provided confidence in the deployment process so that there were no surprises during release into production. As the acceptance tests were also built and executed by Bamboo, it was possible to trigger them from Bamboo after a deployment using Bamboo's REST interface.

Bamboo deployment plans allow software artefacts created during the Bamboo build plans to be deployed into a given environment.

Bamboo Deployment Plans

The above screenshot shows how Bamboo can be used to deploy into different environments. Each environment needs to be configured with tasks that determine how the artefacts will be deployed.

Bamboo Deployment Tasks

The screenshot above shows the tasks used to deploy the website into the performance testing environment. The artefact is downloaded on to the Bamboo server and then a script is executed to perform variable substitution of environment specific configuration. The artefacts are then copied to the remote server using the SCP Task and finally Tomcat is stopped, the artefacts deployed and Tomcat restarted by executing a script on the remote server (SSH Task). These tasks are then repeated for the second server in the clustered environment. The tasks are replicated for each environment to ensure consistency in the deployment process but each environment also has some specific configuration such as the address of the database server and external web services. These can be specified as environment variables and then referenced from the deployment tasks such as $bamboo_catalina_base.

Bamboo Deployment Plan Variables

Using Bamboo for deployment made life so much easier for developers, testers and production support. Nevertheless, managing the Bamboo configuration was becoming cumbersome as with every change to the deployment process, the numerous deployment tasks and variables had to be updated for every environment. To streamline the deployment pipeline further, it was necessary to move the environment specific variables and tasks into property files and scripts that could be executed remotely. The deployment scripts were maintained in version control just like the website source code and packaged along with the application as a separate artefact by Bamboo. This had the benefit of decoupling the deployment process from Bamboo and reducing the number of tasks required to be configured in Bamboo for additional environments. So although Bamboo is still being used for deployments and executing the acceptance tests, it is much easier to manage and test changes to the deployment process.

Deployment Script

Environment Properties

The above screenshots show the deployment script that is now executed by Bamboo along with a sample properties file. The properties file captures the environment specific variables that were previously managed within Bamboo. Line 33 of the script shows how the properties file is read into the script. Lines 62 to 67 shows how an array is declared and the app_cluster from the properties file is read into the array. The array is then iterated over to configure the node that matches the ip address of the server. This allows additional nodes to be added to the cluster very easily by simply updating the properties file. Lines 86-100 perform the variable substitution on a number of configuration files by using the variable values specified within the properties file. Lines 107-118 then copy these configuration files to the relevant directories and restart Tomcat.

Bamboo now simply needs to invoke this script as shown below.

Invoke Deployment Script

The post-deployment smoke tests are invoked from Bamboo through its REST API.

curl --user $bamboo_smoke_test_user:$bamboo_smoke_test_password -X POST http://galbamboo:8085/rest/api/latest/queue/CMS-SMK?bamboo.variable.CMS_ENV=qa

References:

[1] https://www.atlassian.com/software/jira/
[2] https://www.atlassian.com/software/confluence/
[3] https://www.atlassian.com/software/bamboo/
[4] https://cucumber.io/
[5] https://www.ruby-lang.org/en/

Migrating from SVN to Git

Our project team had been using SVN for the last 8 months as its source control management tool. All development was done on trunk and releases were created from a release branch. After 8 months, when the project (new responsive website on new multi-channel CMS platform) went live, we continued developing on trunk and any hot-fixes were made on the release branch and then merged into trunk. Merges were intentionally kept small and frequent and we avoided feature branches altogether.

Strategically the business wanted to adopt a number of new tools including Slack (team messaging, collaboration), Asana (Project and Task management) and Git, so I was tasked with migrating our project from SVN to Git. The repository would be hosted on github and thereby minimise maintenance as there was no server to manage to ensure there was sufficient disk space, or upgrading the scm tool to a later version.

This post describes the process.

The atlassian tutorial [1] provides 5 simple steps to help migrate users from SVN to Git.

1. Preparing the environment
2. Converting the SVN repository to a local Git repository
3. Synchronizing the local Git repository when the SVN repository changes
4. Sharing the Git repository with developers.
5. Migrating development effort from SVN to Git.

1. Preparing the Environment

After downloading and executing the migration script, it was found that the versions of Git and SVN that were available with the Linux distribution [2] were not of the required versions. It was therefore necessary to install the latest version of Git (2.5.2) from source [3].

Once again, after downloading the Git source, it was found that the necessary dependencies such as the gcc compiler and make tool were missing. These were installed using the yum Development Tools package.

The atlassian migration-script was still complaining about the SVN version so the latest version (1.8.14) was also installed by configuring an additional yum repository [4].

The final piece of the jigsaw was to install the correct subversion perl bindings for the updated SVN by executing yum install subversion-perl

Finally, the svn-migration-scripts verify command was able to successfully execute.

[root@devapp1 git-migration]# java -jar svn-migration-scripts.jar verify
svn-migration-scripts: using version 0.1.56bbc7f
Git: using version 2.5.2
Subversion: using version 1.8.14
git-svn: using version 2.5.2
[root@devapp1 git-migration]#

2. Converting the SVN repository to a local Git repository

The tutorial mentions running git svn clone --stdlayout --authors-file=authors.txt

 <svn-repo>/<project> <git-repo-name>

This fails with the following error if the SVN repository requires authentication.

Password for 'myuser': Can't locate Term/ReadKey.pm in @INC (@INC contains: /usr/lib/perl/site_perl/5.14
/usr/lib/perl5/site_perl/5.14/x86_64-cygwin-threads /usr/lib/perl5/vendor_perl
/5.14/x86_64-cygwin-threads /usr/lib/perl5/vendor_perl/5.14 /usr/lib/perl5/5.14/x86_64-cygwin-
threads /usr/lib/perl5/5.14 .) at /usr/lib/perl5/vendor_perl/5.14/Git.pm line 565.

Therefore it is necessary to first run the svn checkout command so that svn caches the credentials and then also install the Perl Term ReadKey package:

perl -MCPAN -e shell

After the shell initiation, install the ReadKey package:

install Term::ReadKey

It should now be possible to create a local Git repository from the SVN repository.

The cloning of the SVN repository creates the SVN branches and tags as remote branches. The clean-git script included in the svn-migration-scripts.jar kept on deleting the branches and tags that were being created so git branch only showed the master branch.

Executing "java -Dfile.encoding=utf-8 -jar ../svn-migration-scripts.jar clean-git --force --no-delete" allowed us to create the local git branches including all the obsolete and deleted ones.

Executing git branch now showed the local git branches, but git tag was still not returning any tags.

The following script was used to create the git tags changes the path to the tags.[5]

Create Git Tags

3. Synchronizing the local Git repository when SVN changes

The synchronizing of the local Git repository with SVN changes works as described in the documentation by simply fetching the latest changes, rebasing the Git repository and then doing a clean up.

git svn fetch

java -Dfile.encoding=utf-8 -jar ~/svn-migration-scripts.jar sync-rebase

java -Dfile.encoding=utf-8 -jar ~/svn-migration-scripts.jar clean-git --force

4. Sharing the Git repository

This step involves pushing the local Git repository to a public Git repository server and then allowing developers to pull and push commits to and from the public repository.

To facilitate pushing changes to a remote repository, Git uses a 'remote' reference as an alias for the remote repository URL. The remote reference can be anything one chooses but if the remote repository is going to serve as the official codebase for the project, it is conventionally referred to as 'origin',

git remote add origin https://\@bitbucket.org/\/\.git

Once the remote has been set up, the changes can be pushed to the remote repository.

git push -u origin --all

git push --tags

Instead of bitbucket, our project team used github to host our remote repository. The uploading of the git repository failed as there were some files with a history of being larger than 100MB in size which github doesn't allow. The file contained the binary data of images that were pruned once development matured (as the images were moved to be managed in the production environment and not within the codebase). However, to allow the uploading of the Git repository, it would be necessary to remove the history of the offending file. This is where the BFG utility came to the rescue [6].

To Be Continued....

References:

[1] https://www.atlassian.com/git/tutorials/migrating-overview
[2] Red Hat Enterprise Linux Server release 6.6 (Santiago)
[3] http://johnathanmarksmith.com/linux/git/programming/2013/05/15/how-to-install-git-182-on-fedora-centos-red-hat-and-scientific-linux/
[4] http://tecadmin.net/install-subversion-1-8-on-centos-rhel/
[5] http://blogs.atlassian.com/2012/01/moving-confluence-from-subversion-to-git/
[6] https://rtyley.github.io/bfg-repo-cleaner/

Hippo CMS with Spring Security

Hippo CMS already uses Spring for dependency injection as part of the HST framework used to develop websites. By default, it uses JAAS as its authentication mechanism and the CMS repository (via the HippoAuthenticationProvider) for authenticating login credentials and providing security roles for the authenticated user.

The existing website application already had in excess of 10 million users and there was no way that these users would be moved into the CMS repository. Fortunately Hippo can be configured with a custom AuthenticationProvider to use a database or LDAP service rather than the CMS repository for authenticating login credentials.

As part of the redevelopment of the website, the login UI was also redesigned. The default form based authentication uses the Hippo UI templates to render the login form. Again this was quite easy to replace with a custom login form template by following the online documentation [1].

At this point the development was looking good, the show and tell at the end of the first sprint demonstrated the new login story and the home page with integration to internal services for authentication and customer transactional data.

The next sprint required some changes to the login functionality based on insights gathered from user testing and UI/UX designers. Due to the business requirements, the login functionality was required to display a number of different messages depending on the cause of authentication failure. This may involve a simple error message, or asking the user that they need to activate their account or to contact the call centre if their account had been locked.

Using the default login mechanism proved to be difficult in propagating different error messages from the custom AuthenticationProvider back through the Hippo JAAS LoginModule and to the Hippo LoginServlet to render the login form. This was due to the Hippo LoginModule catching any exceptions from the AuthenticationProvider and throwing a new JAAS LoginException with a default error message thereby losing the error code and message from the original exception. There were also other issues with CSS when the login form was redisplayed after an authentication failure.

Furthermore, there was also a lurking requirement that the new mobile apps would be required to integrate with Hippo CMS for authentication and content. This would require developing a RESTful API to expose the content for the mobile apps to use. The mobile apps would not use http form based authentication for a RESTful API. Consequently the new website would need to provide a different authentication method depending on the access channel (web browser or mobile app).

The above issues and the complexity involved in customising the JAAS implementation were the main deciding factor to adopt Spring Security. Fortunately, we were not the first team that needed to do this and the Hippo community provides a project that details the integration of Spring Security with HST based website applications.[2]

The configuration is pretty simple and requires configuring the Spring Security filter within the web.xml file.

Spring Security Configuration in web.xml

This will now ensure all requests are passed through the Spring Security filter chain. The context files can be used to specify any security related Spring beans such as a custom authentication manager, authentication filters and success and failure handlers.

Thereafter, to integrate Spring Security within the HST based web application, it is simply a matter of adding the SpringSecurityValve into the HST request processing pipeline. As the HST pipeline employs the 'chain of responsibilty' pattern [3], by adding the SpringSecurityValve into the pipeline, HST will ensure the SpringSecurityValve has the opportunity to handle the request and establish a javax.security.auth.Subject if it finds a spring security Authentication instance.

The Hippo SpringSecurityValve allows securing the HST based website by configuring 'hst:sitemapitem' or 'hst:mount' nodes with security settings.

References
[1] http://www.onehippo.org/library/concepts/security/hst-2-authentication-and-authorization-support.html
[2] http://hst-springsec.forge.onehippo.org/
[3] http://en.wikipedia.org/wiki/Chain-of-responsibility_pattern

Hippo CMS Forms

As part of migrating a website application that has been continuously growing for the last 12 years over to a new CMS platform, it was necessary to understand how forms could be implemented using Hippo CMS.

After downloading the community edition of Hippo and working through the developer trail and reference documentation, there wasn't any clear example of how a form could be implemented. This post provides details of the proof-of-concept work carried out in developing a website form using registration as an example.

The existing website application provides a registration process that allows users to register with the site and also captures some additional information through a survey. The registration process is therefore spread over multiple pages.

Hippo CMS Forms

The above diagram shows how  the CMS repository is configured for the registration process. When a request for registration comes in, it will be matched to the register.html sitemap item. This is configured with the component hst:pages/register. This component reuses the hst:pages/standard component for the general page layout and provides the specific contents through its child body component. The body component shows how by specifying the SpringBridgeHstComponent as the componentclassname and the spring bean id (enrolmentComponent) as the parameter value for the spring-delegated-bean parameter name, we can define the component controller classes using Spring configuration and leverage its dependency injection functionality. The register/body node is also specifies a redirect-page parameter with 'survey' as its value. This parameter will be made available to the Spring configured enrolmentComponent to indicate which page the user should be redirected to after the form has been submitted.

Form Processing EnrolmentComponent

The above code extract shows the doAction method for the EnrolmentComponent. It shows how a list of the fieldnames is constructed and passed to the constructor of the Hippo FormMap object. At this point, the form would be used to construct an object that would be passed to the enrolment service via the Spring injected ServiceFacade. However, for the sake of simplicity and to serve as a PoC a previously generated identifier is used to retrieve existing enrolment details. The submitted form is then temporarily persisted in the repository for retrieval by the SurveyComponent and the configured redirect-page parameter is retrieved and used to redirect the response. By passing a StoreFormResult instance when persisting the form, we are able to retrieve the UUID to send back in the response. The UUID identifies the node where the form is stored in the repository.

The code below shows how the SurveyComponent then retrieves the form from the repository ready to be used as required.

SurveyComponent

As can be seen the UUID is extracted from the request and then used to populate a FormMap instance using the form data stored in the repository.

The above demonstrates how a website form can be developed using Hippo. It doesn't demonstrate error handling but its obvious how easily that can be implemented.

The above solution is very specific and doesn't allow for a generic form processing component that could be reused in different contexts. With this in mind the Hippo developer community provides the 'easyforms' plugin that makes it very easy to create forms using drag and drop widgets. This will be investigated in an upcoming post.

Hippo CMS

On a current project, my team are involved in the replacement of an old content management system (CMS) with one that can serve multiple channels and allow business users to update the website without intervention from developers. After reviewing a number of CMS products, we decided to settle on Hippo. It's open source, JCR-283 compliant, well designed with an intuitive UI and was better aligned with the technology stack in use.

To build a website with Hippo it is important to understand the underlying architecture and model. The Hippo developer trails and documentation provide a good reference and is used to present the summary below.

A Hippo CMS implementation is deployed as two Java web-applications. The Hippo CMS web application (cms.war) is the authoring tool used to create and edit content. The Hippo delivery tier-based (HST) site web application (site.war) is the end-user website.

Developing a website with the delivery tier (HST) is based on the Hierarchical MVC architectural pattern that allows a hierarchical structure of content and for content to be reused across multiple pages. The content stored in the repository and accessed using HST Content Beans represents the model. Java classes providing HST components encapsulate the controller and JSP or Freemarker templates are responsible for the view.

The configuration that connects the model, controllers and views is also stored in the repository under the /hst:hst node. The /hst:hst node stores all the configuration required for the website and processing of requests such as hosts, sites, channels and URLs.

Once a request has been matched to a host, site and channel it is further matched against the sitemap that contains a number of hst:sitemapitem nodes. We can think of the sitemapitem as representing the URL to a webpage. The sitemapitem references a HMVC root hst:component node typically under /hst:pages. This hst:component node can contain child hst:component nodes to provide the composite structure of a page. The hst:component node references a JSP or Freemark template (through its renderpath property) to provide the view and optionally references a  Java HST component class (through its componentclassname property) that implements the controller. If a component class is not provided, the component will use the default component class.

The diagram below will help understand how a page is typically configured in the Hippo repository.

Hippo CMS Page Composition

When a request that matches the sitemap item about.html it will be rendered as a textpage due to the hst:componentconfigurationid configured to reference hst:pages/textpage. The content that will be displayed in the rendered page is retrieved through the hst:relativecontentpath property. The hst:pages/textpage nodes models the structure of the webpage. The diagram above shows that the page reuses the hst:pages/standard page structure through the hst:referencecomponent property and references hst:component/content for its content child node.

The standard page is composed of an header and main child components and a template for rendering the view. The main component is further composed of a leftmenu and right child components and a template for rendering the view.

The child components that require some business logic such as header and leftmenu, again reference components under hst:components that specify the componentclassname that implements the logic and a template for the view. Similarly the content component also specifies its componentclassname that is responsible for retrieving its document from the repository and a template responsible for the view.

The configuration of templates is found under hst:templates where each template has a renderpath property that specifies the location of the JSP or Freemarker file.

This covers how the Hippo repository is typically configured to deliver modular web pages..

Groovy Multimethods

One of the features of the Groovy language is multi-methods and how it differs from Java in the way it invokes methods. Java selects which methods to invoke based on the compile-time declared types of arguments. Groovy selects which methods to invoke based on the runtime types of the objects and arguments.

As an example, consider the following listing:

The x and y arguments are both of static (compile-time) type Object but x is of dynamic (runtime) type Integer and y is of dynamic type String. As a consequence, in Java both would dispatch to the same oracle(Object) method. However, in Groovy due to method dispatch being dynamic, the oracle(String) method is used when argument y is passed.

This has the benefit of avoiding duplicate code by being able to override more selective behaviour. Consider the following equals implementation that overrides Object's default equals method only for the argument type Point.

When an object of type Point is passed to the equals method, the specialized implementation is chosen. When an arbitrary object is passed, the default implementation of its superclass Object.equals is called. This gives the impression that equals(Point) is overriding equals(Object), which is impossible in Java. In Java, it would be necessary to check for null, check that argument is of the required Point type, cast the object to Point, and then perform the specific equals logic. This forces the developer to duplicate the logic of overriding equals for every custom type.

Apache Web Server, Tomcat AJP: ajp_read_header: ajp_ilink_receive failed

Problem:

Apache Web Server configured to proxy requests to web application running on Tomcat (7.0.39) over AJP. The applications were installed on virtual machines in a cloud environment. On the completion of load tests (more than 24 hours), the system became unresponsive. Whenever a request was made to Apache an HTTP 503 status code was returned. Looking at the Tomcat logs showed no errors and requests could still be sent over the HTTP channel directly to Tomcat. CPU and memory consumption was also very low. Looking at the Apache log files showed errors of the following nature:

[error] ajp_read_header: ajp_ilink_receive failed
[error] (70007)The timeout specified has expired: proxy: read response failed from 10.1.3.3:8009 (10.1.3.3)

Tomcat could no longer handle any more requests from Apache over AJP and required a restart.

Analysis:

Doing a 'netstat' for port 8009 on the app server VM showed that there were 200 connections still in an ESTABLISHED state, with 100 connections in a CLOSE_WAIT state.
 From this initial analysis, a number of questions arose:

• How did the number of AJP connections grow so large?
• Why did the number of connections not close after a period of inactivity?
• What was causing Tomcat from accepting any more requests?

Reading the AJP documentation confirmed that by default the AJP connection pool is configured with a size of 200 and an accept count (request queue when all connections are busy) of 100. To confirm the findings, Tomcat was configured with a smaller AJP connection pool (20) and as expected the errors in Apache occurred sooner and more frequently. 

To address the issue, Apache (MaxClients) and Tomcat (maxConnections) were both configured to support 25 concurrent requests. This worked perfectly (Apache no longer returned 503 responses and the log files no longer showed the ajp_link errors). The test was then repeated after increasing the connection pool to 50. Running a load test for an hour showed the servers working well, response times improved and no 503 responses. However, after the test completed, the Tomcat VM still showed 50 connections in an ESTABLISHED state. A further read of the Tomcat AJP documentation revealed that the connections remain open indefinitely until the client closes them. The next thing to try was the 'keepAliveTimeout' on the AJP connection pool. This had the effect of closing the connections after a period of inactivity and therefore seems to have resolved the issue. Ideally, the AJP connections should grow as load increases and then reduce back to an optimal number when load decreases. The 'keepAliveTimeout' has the effect of closing all connections that are inactive.

Solution:

Configure Apache 'MaxClients' to be equal to the Tomcat AJP 'maxConnections' configuration.

Configure Tomcat AJP 'keepAliveTimeout' to close connections after a period of inactivity.

References:
Tomcat AJP: http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html
Apache MPM Worker: http://httpd.apache.org/docs/2.2/mod/worker.html

Runtime Configuration of log4j properties

log4j is used widely to provide logging in applications. There are numerous articles about how to configure log4j.properties or log4j.xml to enable/disable logging at various levels and categories. Often it is necessary to enable the debug log level for a short duration to identify the cause of a production issue. Without runtime configuration, this would require stopping the application, changing the log configuration (after extracting the properties file from the war) and then redeploying.

This post describes how log4j can be configured outside of the application and also changed at runtime.

File Watchdog

Log4j provides runtime configuration through the DOMConfigurator.configureAndWatch for XML files, or the PropertyConfigurator.configureAndWatch for properties file. Both these methods take the absolute path for the configuration file and a refresh interval. This allows the configuration file to be located outside of a web application war file and allow administrators to change the logging levels at runtime.

The configureAndWatch API can be invoked from a custom servlet listener. For users of the Spring framework, there already exists the Log4jConfigListener that is a wrapper to the Log4j configureAndWatch API. This listener is configured in the web applications web.xml.

Spring Log4jConfigListener

By specifying the log4jConfigLocation to a file outside the web application, it allows different environments (dev, test, prod) to have different levels of logging enabled. The log4jRefreshInterval specifies how often the log4j.properties file should be checked for changes and reloaded. This allows runtime changes to the log4j configuration to be performed.

JMX

Log4j also has built-in support for JMX and two classes org.apache.log4j.jmx.LoggerDynamicMBean and org.apache.log4j.HierarchyDynamicMBean are used to expose and register loggers as MBeans. However, it still requires wrappers to work to add loggers that are not defined at start-up.

Tomcat JMX Connectivity through firewall

JMX is great for monitoring the JVM to identify potential problems with memory and concurrency. Usually this is quite simple as a local running JVM can be connected to using tools such as JConsole. When connecting remotely however there is some additional configuration required.

During a recent project, the team faced an issue where a web application was deployed using Tomcat 7 on a cloud environment.

To enable JMX monitoriing for Tomcat 7, the following system properties were configured via setenv.sh:

Tomcat JMX configuration

The above configuration properties specify that JMX remote access should be enabled on port 7099 and authenticated using the credentials found in jmxremote.access and jmxremote.password.

For more information about configuring JMX see the Oracle documentation in the references section.

The firewall on the VPN between the cloud environment and local network was configured to allow connections on port 7099. However this still didn't prove successful.

After reading numerous online resources, the problem was identified. Basically, when the JMX server starts up, it opens two ports, one for the JMX registry and another dynamically generated port for the RMI server. The above configuration only specifies the JMX registry port and there isn't a system property to configure the RMI server port. Both these ports need to be opened in the firewall, but as the RMI server port is dynamically chosen, how can the firewall be configured?

One solution is to develop a custom JMX agent and configure the java runtime to use it. This is outlined in the references below.

For Tomcat there is a more elegant solution for out-of-the-box JMX monitoring.
Tomcat provides a JMXRemoteLifecycleListener that allows specifying both the JMX registry port and the RMI server port.

Tomcat JMXRemoteLifecycleListener

The rmiRegistryPortPlatform replaces the use of the com.sun.management.jmxremote.port system property.

The JMXRemoteLifecycleListener requires the deployment of the catalina-jmx-remote.jar in the ${CATALINA_HOME}/lib directory.

Tomcat can now be remotely monitored and managed using the following JMX service URL:
service:jmx:rmi://<host>:<rmiServerPort>/jndi/rmi://<host>:<rmiRegistryPort>/jmxrmi

References:

http://docs.oracle.com/javase/7/docs/technotes/guides/management/agent.html

http://olegz.wordpress.com/2009/03/23/jmx-connectivity-through-the-firewall/

http://beautifulbytes.wordpress.com/2012/02/16/using-a-java-agent-to-monitor-remote-applications-via-jmx-through-a-firewall/

http://tomcat.apache.org/tomcat-7.0-doc/config/listeners.html#JMX_Remote_Lifecycle_Listener_-_org.apache.catalina.mbeans.JmxRemoteLifecycleListener

Environment Specific Configuration of Web Applications

We've all faced this situation. A web application needs to be deployed to a number of different environments (dev, test, prod) and each of these requires slightly different configuration such as which database to use etc.

Assuming a typical Maven-based Spring application, these configuration parameters are usually held within a properties file in the src/main/resources folder.

sample properties file

And then used to substitute place-holders within Spring application context files.

sample application context file

Now how do we provide environment specific configuration for every deployment?

• Manually unpack the artefact and modify the properties file and then repackage it. This is clearly unacceptable from a maintenance and quality control perspective.
• Use Maven profiles to include an environment specific properties file at build time.
• Use Maven filtering in combination with profiles to produce an environment specific build.
• Use Maven assembly plug-in to generate multiple environment specific artefacts in a single build.
• Use Spring profiles for runtime configuration

Our initial approach was to use Maven filtering with profiles and further augment the artefact by using the profile id as a classifier.

Maven filters

There exists a properties file for every required environment (dev, test, prod), but instead of placing these files within the src/main/resources directory, they are placed within the src/main/filters directory. The property files follow a naming convention such as config-{profileid}.

Maven pom filtering configuration

The above Maven pom extract specifies that when the resources are copied from the source to target directory, place-holder substitution or filtering will take place using the environment specific filter. The artefact that is built with the maven-war-plugin will also be tagged with the environment variable (env).

The env variable is a Maven property that is set by the active profile.

Maven profiles

The above Maven pom extract specifies a number of profiles that simply set the env property, that effectively points to a configuration file that contains all the properties. This is a cleaner approach than to list all the environment specific properties within the Maven profile.

The above approach works perfectly well but there are still some deficiencies such as:

• Requires running the build multiple times for environment specific builds. This is not recommended as the artefact should be built only once and deployed anywhere.
• Doesn't work well with the Maven release plug-in and automated build tools.
• Every new environment requires a new profile, new configuration filter and a new artefact.

Taking a step back, we first need to identify and understand the problem we are trying to solve. 

Ideally the configuration should not be tightly coupled to the build but external and applied at runtime. By storing the configuration property file outside of the application in a specified location on the deployment environment, the application can be configured with environment specific properties at runtime.

External Runtime Configuration

This is achieved simply by configuring the Spring application context property-placeholder to load properties from a file location that is outside the application. Note, that there is also a properties file in the application (src/main/resources), that is used to apply default properties. The environment specific property files can then choose to override the default properties if required. This approach is much simpler (no profiles, no filtering, single artefact) than the initial approach and easier to maintain. For additional deployment environments, there is no need to modify the build process, all that is required is a customized properties file in the required location.

Spring MVC, Jackson and Filtering JSON Serialization

The default Spring JSON configuration will show all the fields of an object during serialization. Often we would like to return a subset of an entity’s properties and hide some which are only used internally such as the id and version.  Or perhaps return different properties depending on who/what is requesting the data.

As Spring uses Jackson for JSON serialization it supports the Jackson Json annotations. Jackson provides a number of options for “narrowing” or “filtering” the properties of an entity that gets serialized.

Jackson JsonViews (requires adding @JsonView to entity classes which pollutes them)

Jackson mix-in (requires extending the entity to specify @JsonIgnoreProperties)

Jackson Filtering options

http://www.cowtowncoder.com/blog/archives/2011/02/entry_443.html

Example of using Jackson views and mixin for filtering:

http://vard-lokkur.blogspot.co.uk/2010/10/json-jackson-serialization-narrowed.html

This one creates a new annotation to use with JsonViews but is a little too involved for my liking as it requires changing some Spring classes.

http://martypitt.wordpress.com/2012/11/05/custom-json-views-with-spring-mvc-and-jackson/

This link provides a good demo of Jackson Views and Mixins

http://www.jroller.com/RickHigh/entry/working_with_jackson_json_views

Within my SearchController I was returning a SearchSummary object that referenced a Collector object. However, I didn’t want to return all the Collector details within the SearchSummary as its not relevant. Shown below is all I required:

@RequestMapping("/rest/loyalty")

@Controller

public class SearchController {

    @JsonIgnoreProperties({"loyaltyId","earnings","id","version"})

    private static class CollectorIdOnlyView extends Collector {

        // Empty by design ...

    } 

      

    private final ObjectMapper objectMapper = new ObjectMapper();

    private final MappingJacksonJsonView view = new MappingJacksonJsonView();      

    

    public SearchController() {         

        objectMapper.getSerializationConfig().addMixInAnnotations(Collector.class, CollectorIdOnlyView.class);

        view.setObjectMapper(objectMapper);

          

    }

    @Autowired

    private AccountService accountService;

      

    @Autowired

    private SearchService searchService;

      

    @RequestMapping(value = "/search/{id}", method = RequestMethod.GET)

    public View getSummary(@PathVariable("id") String id, Model uiModel) {

        Collector collector = accountService.findCollectorById(id);

        if(collector == null) {

              throw new RuntimeException("Collector Not Found");

        } else {

              SearchSummary searchSummary = searchService.getSearchSummary(collector);

              uiModel.addAttribute("summary",searchSummary);

              return view;

        }

       

    }

}

For the filtering I had to create a subclass of my entity to define the properties I wanted to ignore. (CustomerIdOnlyView).

This is then used to configure the ObjectMapper and View.

The change I had to make to the getSummary method was to change the return type from SearchSummary to View, add the SearchSummary to the Model and remove the @ResponseBody annotation.