How Alfresco is powering the ECM industry transformation and move to a cloud powered digital Enterprise – Introducing the Alfresco SPK

http://www.amsimaging.com/blog/bid/146509/3-Reasons-Why-You-Should-Move-Your-Documents-to-the-Cloud-With-ECM
ECM in the Cloud

The move to cloud is happening, and to be fair ECM has been one of the most lagging businesses to finally take the leap and seriously look into the new delivery models (XaaS) the cloud opens up.

Metrics are off the roof, all the indicators part of the global intel we gather from customers, partners and prospects point in one direction: the future of ECM, like any other enterprise information system, is rapidly moving to cloud, and AWS is driving the space.

SaaS, PaaS, IaaS and company are growingly looked at as major IT optimization factors and concerns over data security are being fast overcome. And this is happening much more rapidly that you’d think and that was predicted just 12/24 months ago. Even in large organization, typically in “closed” vertical, often lagging from technology advancement standpoint.

At Alfresco, we are heavily investing in providing a seamless unified content platform which will ubiquitously enable the deployment of your ECM solutions on premise or on cloud (private, public, hybrid), leveraging natively cloud scale and availability features to power mission critical massive content repositories.

But what’s driving this transformation? In other words, from a customer standpoint, what the key drivers that are increasingly driving customers to choose cloud as a primary ECM deployment vehicle?

Scale (and elasticity) is definitely a key driver. You might have recently read about our joint effort with Amazon that ingested, processed and served 1B documents on an Alfresco instance running on AWS (EC2 + Aurora). As you can appreciate in the technical details of this benchmark, Aurora as a super-scalable database and the flexibility of growing up to 20 Solr shards + 10 Alfresco instances were the key winning factors, validating the choice of basing this benchmark on a flexible and scalable infrastructure like AWS and be able to leverage native features like availability and auto-scaling.

But the most compelling reason remains optimization of the IT processes and the mirage of handing off  maintenance of thicker and thicker layers of the stack to 3rd party providers. While this is not exactly big news (see this post from 2 years ago), especially with the flourishing plethora of DevOps tools to automate provisioning (Chef, Puppet, Ansible, Salt, etc.), orchestration (Kubernetes, Terraform, etc) and more in general the whole Dev -> Ops workflow with immutable containers (Docker, Unikernels, etc), organizations are quickly realizing the end to end benefits of a cloud strategy that starts from the bottom layers of the stacks to rapidly become pervasive and drive to the outsourcing of shared services, application layer, even application development itself and some of the back office processes.

At Alfresco, we are very aware of this movement and real life customer requirements springing off the DevOps movement, and so we have focused our efforts to provide an answer to our customers, partners and community members trying to deploying even growing and arbitrarily complex architectures seamlessly in Cloud and On premise, with native support for virtualized environments and containers.

As an initial deliverable and project to follow, less than 2 weeks ago at the Alfresco Day Roma, we have launched a community preview of the Alfresco SPK (Software Provisioning Kit), a toolbelt for the Alfresco admins and DevOps (much like the alfresco-sdk for Developers), which constitutes a common layer to easily provision Alfresco instances and stacks, either from scratch or starting from pre-existing images, based on pre-existing and commonly used DevOps technology like chef, Vagrant, Packer and virtual image formats (AMIs and Docker to start with). I want to give kudos to our DevOps department and especially Maurizio to have patiently worked with me to convey our internal DevOps experience into a customer facing tool. The SPK is hosted on Github and it’s a 100% community open source effort, so we welcome your feedback!

The SPK constitutes the first key building block to enable a set of very real customer use cases:

  1. First and foremost, provide a modern way of consuming Alfresco software, in the form of immutable pre-baked instance templates (see OOTB) which can be configured by the customer and composed in arbitrarily complex stacks (OOTB or customer define) that can then be ran in your favorite cloud and orchestration system (currently AWS and Cloud formation)
  2. Allow more advanced customer DevOps departments to consume Alfresco software and build images from scratch, to produce stacks that can be then packaged in the virtual format of choice (AMI to start with, Docker later)
  3. Enable the immutable container driven (e.g. via Docker) extended Developer workflow, to allow building of reliable and repeatable stacks locally and remotely, and enable Alfresco to take part to modern processes of continuous delivery and leverage cloud scalability / failover capabilities natively

Maurizio has put together a thorough presentation for the Alfresco Day which describes all the features and current state of the SDK.  You can find it below and we welcome your feedback.

NOTE: While it can work with Alfresco Enterprise, the Alfresco SPK is currently in Community Preview, which means it is NOT supported by Alfresco.

So currently we are looking for feedback and validation, rather recommending than leveraging this tool in production (use at your own risk!).

We are working on a more solid timeline for an EA (Early Adopter) and the GA (General Availability), so stay tuned!

Take your Alfresco productivity to the next level with the Alfresco SDK 2.1.0

As a first tangible result (first relase out!) in my new role in Alfresco Product Management and of the renewed investment Alfresco is putting in the developer platform, developer services and communications, no more than 5 months later than the 2.0.0 release, I am pleased to announce that the Alfresco SDK 2.1.0 is now released and available in Maven Central.

Many thanks go to Martin, for bringing new life to the Alfresco Developer Platform team and to Ole for stepping above and beyond his Developer Evangelist role and help tie up the release. Kudos also to Community members like Bindu who helped with testing and feedback.

This release works for Community and Enterprise (fully supported by Alfresco) and it shows strong signs of our cross-department (Dev, Product, Docs, Support) effort on a much more seamless and productive developer experience for our beloved ecosystem out there.

Here are the full release notes, but let me give you a couple of highlights:

  1. First off, building on the SDK 2.0.0 Spring Loaded approach, we have completed the effort for a full hot reloading experience, both for Alfresco and Share. Until more fixes go in product, we have introduced for this purpose a couple of plugin goals in the alfresco-maven-plugin, which are automatically invoked (or you can manually do that) to refresh webscripts on Alfresco and Share. If you are using your Eclipse, this should happen automatically on save, for IDEA you might night a bit of manual configuration. See RAD (Rapid Application Development) documentation for details.
  2. The SDK now supports Solr4 in the All-in-One archetype: this was the top-most requested issue in 2.0.0, so I am glad we got that out!
  3. In an effort to deliver higher and higher quality extension, we have introduced support for functional and regression testing, leveraging the Selenium based share-po library, which we use internally at Alfresco to perform black box testing. For details, here’s the issue and command details on how to regression test your customizations.
  4. Thanks to Martin and the Docs team  have fully re-written and improved documentation for the SDK 2.1.0  and we also started properly versioning docs for all SDK supported versions (e.g. see 2.0.0 docs)
  5. The SDK is an officially supported Alfresco product as of 1.1.1, but the SDK 2.1.0 marks an important step towards a much more predictable, supported, sustainable development support on the SDK. From now on, you can check the Product Support Status to verify the support status of the SDK and also, if you are an Enterprise customer, engage with Developer Support to get Dev savvy engineering help you on Development matters

I do hope you guys enjoy the new SDK and I am eager to collect feedback, via comments to this post, forums or you can reach out to me or Ole, as well as raise issues in the SDK project.

Handing my baby, the SDK, off 🙂

One more service announcement: as followers of this blog, you know I have always used this channel to announce SDK news and releases. While I’ll keep posting on SDK usage tips and more in general on my visions on the Alfresco Developer Platform directions, I think it’s time to hand off the baby and in line with the full Alfresco (“the company”) support for the SDK, move all communications to the Alfresco Developer Blog.

In particular, our Developer Evangelist is going to play a key role in keeping you updated and collecting feedback on our development experience.

So stay tuned on our Dev Blog, as Ole will soon post a more comprehensive update on this release.

[Maven is not so evil] Plexus-archives 2.3 is a life saver

Maven is very good with J2EE WARs. Resources, packaging, overlays.

I admit that if it wasn’t for the great features and clean development model imposed by the maven-war-plugin, I would not even probably looked at Maven in the first place when I developed the first version of the Maven Alfresco SDK.

Lately though, especially working with large WARs overlays like Alfresco (120MB) and Share (40MB), I have seen all my efforts for standardized development falling badly short on the rapid development side. Suddenly (on my Mac, or I assume on any Unix/Linux environment) my WAR overlays started to take minutes to complete, with a big so long to the quick dev-fix-test cycle. As a Java performance expert, I decided to debug it and understand where it was actually wasting time.

Turns out that for each and every unpacked resource from WAR overlays the Maven unarchiver would fork a chmod process, to modify some file permissions!!! (Windows users, you can stop reading here 🙂

Leaving out that I’m not sure of the original reason for this, I later found out (also thanks to Samuel, kudos!) that this was due to a misconfiguration of the plexus-archiver package, which instead of using native JVM chmod capabilities was forking the process. All details unfolded in a related maven-dependency-plugin issue, MDEP-368.

Basically turns out this properly fixed in plexus-archiver-2.3 (the Maven shared component used by plugins to manage ZIP & co. archives), where no process is forked and the overall WAR packaging / overlay time is reduced drastically. And now the fix, and later the wow effect (I hope) :)))

Luckily enough the fix is pretty simple and only requires to have your plugins (unless they do it already) depend explicitly on plexus-archiver version 2.3. In other words just add the following snippet (the example is for the maven-war-plugin, but you can use the trick for the maven-dependency-plugin and any other plugin you see poorly performing on resource unpacking) to your pom.xml:

 <plugin>
     <artifactId>maven-war-plugin</artifactId>
     <dependencies>
         <dependency>
             <groupId>org.codehaus.plexus</groupId>
             <artifactId>plexus-archiver</artifactId>
             <version>2.3</version>
         </dependency>
     </dependencies>
 </plugin>

EDIT: This is fixed in the maven-war-plugin version 2.4 (and you don’t need the explicit plexus-archiver dependency anymore), see related issue MWAR-280).

Just to give you an idea how the performance improvements with this little fix, I will use as an example the Maven Alfresco All-In-One Archetype, which allows to develop and run an Alfresco Repository, Share, Solr and an AMP in the same Maven multi-module project. The SDK currently does NOT use plexus-archiver 2.3 (there is an issue for it), so it’s extremely (and unnecessary) slow in the Alfresco and Share overlays.

After I created the project from the archetype as per docs, here’s the output of a simple

mindthemac:alfresco-parent mindthegab$ mvn clean install -DskipTests
...
[INFO] Reactor Summary:
[INFO]
[INFO] Quickstart of Alfresco and Share with DB and runner embedded  SUCCESS [0.978s]
[INFO] Alfresco AMP Module ............................... SUCCESS [11.271s]
[INFO] Alfresco Repository and Explorer Client ........... SUCCESS [4:10.599s]
[INFO] Alfresco Apache Solr customization ................ SUCCESS [6.504s]
[INFO] Alfresco Share Client ............................. SUCCESS [4:08.038s]
[INFO] Alfresco and Share Runner ......................... SUCCESS [0.045s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8:41.062s
[INFO] Finished at: Thu Jun 20 14:54:35 CEST 2013
[INFO] Final Memory: 38M/782M
[INFO] ------------------------------------------------------------------------

What? 8m41s total build time? 4 minutes for each WAR? On my very decent Mac 8GB RAM+SSD?

Such a slow cycle is just going to kill the Maven Alfresco Community, to be a little dramatic 🙂

And now, get ready for the wow…after applying the little plexus-archiver fix above (in the parent POM <plugins> or <pluginManagement> sections) this is the amazingly improved performance of the vary same build:

mindthemac:alfresco-parent mindthegab$ mvn clean install -DskipTests
...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Quickstart of Alfresco and Share with DB and runner embedded  SUCCESS [0.848s]
[INFO] Alfresco AMP Module ............................... SUCCESS [5.630s]
[INFO] Alfresco Repository and Explorer Client ........... SUCCESS [35.372s]
[INFO] Alfresco Apache Solr customization ................ SUCCESS [5.054s]
[INFO] Alfresco Share Client ............................. SUCCESS [23.945s]
[INFO] Alfresco and Share Runner ......................... SUCCESS [0.019s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:13.143s
[INFO] Finished at: Thu Jun 20 15:56:43 CEST 2013
[INFO] Final Memory: 44M/496M
[INFO] ------------------------------------------------------------------------

WOW!!!  Only 1m13 total build time…a neat 85.96% build speed up 🙂

I think it’s really worth making sure this trick is known as it’s going to be a life saver to the Maven Alfresco SDK and I’m also using it to speed up our internal Alfresco Maven builds.

Hope you find this useful, to the next tip of this newly born Maven post series!

Embrace change. Out of the comfort zone. Panta rhei. Agile life.

In life like in software, changes should not be feared but just embraced, as a potential opportunity for improvement and a way to augment your personal cultural level, learn new things and generally train your brain to be flexible and responsive to new conditions.

This might sound obvious to all of you, but there are many more aspects of our lives (or at least mine) which, instead of driving me to change and constant improvement of my and other selfs, would be much better off if we were “constantly” stable and predictable.

Think about it, our governments would love to have all of us adhere to perfect statistics, being as controllable and predictable as a mass.  Same goes for a discreetly successful love relationship. Our minds, or at least them, once in a comfort zone, tend to naturally relax and search for common solutions to problems already solved. Even our parents, in many cases, would love for us to do the most conservative choice possible for our lie, i.e. stick with what they know and with what they thinks it’s safe for you, in many cases ignoring the world is “actually” changing.

Read more Embrace change. Out of the comfort zone. Panta rhei. Agile life.