Skip navigation
All Places > Alfresco Content Services (ECM) > Blog > 2016 > December
2016

With the release of Alfresco 5.2, some exciting new changes are coming to Alfresco’s search capabilities. This post looks at some of the highlights of the new Alfresco Search Services 1.0, with a brief description of the major enhancements and most relevant features.

 

Alfresco Search Services Introduction

 

Welcome to Alfresco Search Services 1.0, based on Solr 6.3.

 

New Packaging / Execution

 

Apache Solr is an integral part of the Alfresco ECM solution, that’s why a Solr.war has always been bundled with the Alfresco repository.  The Solr.war is a repurposed version of Solr with lots of Alfresco-specific goodies added in.  For Solr 5.0, the Apache Solr project decided to take control over how Solr runs; instead of relying on an application server - Solr became an independently runnable executable.

 

With Alfresco Search Services now based on Solr 6.3 we have picked up this change - you no longer deploy a Solr.war to your application server.  The Alfresco team also took the opportunity to repackage and refactor some of the search code, as well as improve test coverage (up 30%) and performance. For example, SSL support has been enhanced and improved.

 

Independent Releases

 

Independent packaging allows customers and the community to be freed from a monolithic release lifecycle.  New features and bug fixes can be released quicker and more easily.  When new features are added to the Apache Solr project we are now able to benefit from them quicker and with less disruption.  To mark this change we decided to give the solr artifact a new name “Alfresco Search Services”.

 

New features and Enhancements

 

With the introduction of the new Alfresco Search Services, lets look at some of the major new features and enhancements.

 

Change Tracking

 

The Alfresco Trackers are the heart of the Alfresco Search Solution,  the tracking logic has changed significantly in this release.  Some of the changes came out of the work on the 1 Billion Benchmark, other changes have come as a result of analysis and performance enhancements.  Indexing and committing new information to the index no longer waits for warming, we have eliminated some of the pausing during indexing and querying at the same time, as well as enhanced the commit tracking.  In particular, large indexes doing bulk loading should see significant improvements.

 

Sharding

 

Some larger repositories benefit from sharding the data across multiple cores. For an introduction to sharding, see the documentation.  You now have greater control over how your data is sharded.  

  • The default sharding method is to use the Alfresco node reference - this is the most common use case.  
  • If your repository makes extensive use of Access controls then an ACL_ID approach may be most suitable.
  • If it makes sense to group your data by DATE then date-based sharding is now available.
  • If your use case doesn’t fit into any of the previous categories then you can choose any text-based property to shard on.  This is known as PROPERTY based sharding.

 

Fingerprints

 

It is now possible to generate a "fingerprint" of a document's content. This can be used to find similar documents. This feature makes use of Minhash work contributed by Alfresco to the Apache Lucene project.

 

Hit Highlighting

 

Search result term highlighting is now available in Alfresco Share, this was made possible by changes to Alfresco Search Services.  If you are not a Share user then an API is available for your own custom solution.

 

Tagging and Excluding Facet Filters

 

You can now present a user with a list of facets that can be independently selected, this is known as “multi-select faceting”.  Normally, facets only apply to the data that is being filtered, with multi-select faceting it is now possible to show facets for all documents not just those that are seen after filtering.  This feature hasn’t made it into Share yet but look out for it in the advanced search screen - “filter by” categories.

 

Indexing Multiple Document Versions

 

A standard search usually looks at the most recent version of a document. It is possible to tell Alfresco to index every version of a document, then you can search for content in old versions.  This would clearly increase the size of your index so should be used with caution but some use cases require searching the entire history of a document - this is now possible.  This new feature is not exposed in Share, it is only available via the REST API.  

 

New JSON API

 

With recent Alfresco releases a great deal of effort has gone into enhancing the Alfresco REST API to give a comprehensive and consistent developer experience; Alfresco Search is no exception.  There are 3 new targeted search apis for specific use cases:

GET public/alfresco/versions/1/queries/nodes
GET public/alfresco/versions/1/queries/people
GET public/alfresco/versions/1/queries/sites

 

There is a fourth, powerful, api for searches that fall outside the basic requirements.

POST /public/search/versions/1/search

 

The Apis are JSON based so can be called directly from clients - no Java classes/config are required.  Of course, the new features such as search highlighting, multi-select faceting are already available for you to use.

 

What to do next

 

In this post we introduced the new Alfresco Search Services 1.0, released with Alfresco 5.2. By working with, and committing to, the Solr project, Alfresco’s open source culture brings benefits to the Alfresco community and the wider Solr community.  Relevant features and enhancements has been introduced thanks to the use of Solr 6.3. In the coming weeks, more technical documentation will be written to describe how to use it in practice. Until then, we encourage you to try it, join the community and contribute to the success of Alfresco Search.

 

Download Alfresco Search Services 1.0 EA (md5: e72e43be38292564765264b40f1d7ad3)

In the last post we took a look at associations, this time we're going to cover some of the collaboration APIs that have been present for a few releases, namely comments, ratings and tags.

 

As usual this post is accompanied by a Postman collection, click the "Run in Postman" button below to import it into your client.

 

 

We'll need some content to use throughout this post, either refer back to part 3, execute the first request in the Postman collection or POST the following body to http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/-my-/children to create a node:

{
  "name": "collaboration.txt",
  "nodeType": "cm:content"
}

 

Let's start by creating a comment on our content (2nd request in the Postman collection), POST the body below to http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/comments:

{
  "content": "This is my comment"
}

 

This will result in a response similar to the one below:

{
  "entry": {
    "createdAt": "2016-12-13T18:15:30.450+0000",
    "createdBy": {
      "firstName": "Test",
      "lastName": "Test",
      "emailNotificationsEnabled": true,
      "company": {
       
      },
      "id": "test",
      "enabled": true,
      "email": "test@alfresco.com"
    },
    "edited": false,
    "modifiedAt": "2016-12-13T18:15:30.450+0000",
    "canEdit": true,
    "modifiedBy": {
      "firstName": "Test",
      "lastName": "Test",
      "emailNotificationsEnabled": true,
      "company": {
       
      },
      "id": "test",
      "enabled": true,
      "email": "test@alfresco.com"
    },
    "canDelete": true,
    "id": "42fbc51b-3e9a-4237-ad23-bfd8c6bcbc6a",
    "content": "This is my comment"
  }
}

 

Take a copy of the newly created id as we'll need it shortly, hereinafter referred to as "fileId".

 

To retrieve all the comments on a node we use http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/comments (3rd request in the Postman collection), at the moment our node only has the one comment:

{
  "list": {
    "pagination": {
      "count": 1,
      "hasMoreItems": false,
      "totalItems": 1,
      "skipCount": 0,
      "maxItems": 100
    },
    "entries": [
      {
        "entry": {
          "createdAt": "2016-12-13T18:15:30.450+0000",
          "createdBy": {
            "firstName": "Test",
            "lastName": "Test",
            "emailNotificationsEnabled": true,
            "company": {
             
            },
            "id": "test",
            "enabled": true,
            "email": "test@alfresco.com"
          },
          "edited": false,
          "modifiedAt": "2016-12-13T18:15:30.450+0000",
          "canEdit": true,
          "modifiedBy": {
            "firstName": "Test",
            "lastName": "Test",
            "emailNotificationsEnabled": true,
            "company": {
             
            },
            "id": "test",
            "enabled": true,
            "email": "test@alfresco.com"
          },
          "canDelete": true,
          "id": "42fbc51b-3e9a-4237-ad23-bfd8c6bcbc6a",
          "content": "This is my comment"
        }
      }
    ]
  }
}

 

Each comment has a canEdit (line 27) and canDelete (line 39) property, this can be used to determine whether the current user has the permission to edit or delete that comment, respectively. 

 

What if we made a typo in our comment and we need to update it, we can achieve that by using PUT with a Content-Type of application/json to http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/comments/{{commentId}} (4th request in the Postman collection). Send the following body to update the content text to "Updated comment".

{
  "content": "Updated comment"
}

 

Presuming we have permission to delete a comment we can do so by sending a DELETE request to http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/comments/{{commentId}} (5th request in the Postman collection).

 

Now let's take a look at "liking" a node. To do this we use the ratings API, likes is one of the rating schemes available out-of-the-box, fiveStar is the other, although this is not currently used by any Alfresco client. An explanation of rating schemes is beyond the scope of this article so I'll that as an exercise for the reader.

 

To "like" a node we can POST the body below to: http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/ratings (6th request in the Postman collection). The value required for the myRating property depends on the rating scheme, for likes it's a boolean flag.

{
  "id": "likes",
  "myRating": true
}

 

This will result in the following response. It shows our rating (line 3), the rating type we applied (line 5) and the number of times this node has been rated (line 7) with that particular scheme.

{
  "entry": {
    "myRating": true,
    "ratedAt": "2016-12-13T19:28:23.792+0000",
    "id": "likes",
    "aggregate": {
      "numberOfRatings": 1
    }
  }
}

 

If we retrieve the ratings for the node using http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/ratings (7th request in the Postman collection) we'll see an entry for each rating scheme installed in the repository. If you've rated the node with one of the schemes the myRating property will be present (line 21). Each scheme also shows the total number of ratings applied (lines 15 and 25).

{
  "list": {
    "pagination": {
      "count": 2,
      "hasMoreItems": false,
      "totalItems": 2,
      "skipCount": 0,
      "maxItems": 100
    },
    "entries": [
      {
        "entry": {
          "id": "fiveStar",
          "aggregate": {
            "numberOfRatings": 0
          }
        }
      },
      {
        "entry": {
          "myRating": true,
          "ratedAt": "2016-12-13T19:28:23.792+0000",
          "id": "likes",
          "aggregate": {
            "numberOfRatings": 1
          }
        }
      }
    ]
  }
}

 

If you rate the node again using another user and retrieve the ratings again you'll notice the numberOfRatings property on line 25 change to 2.

 

So what if we change our mind and decide that we no longer "like" the node, how do we "unlike" it? To do this we DELETE the rating scheme from the node using http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/ratings/likes (8th request in the Postman collection). This doesn't remove the whole rating scheme, it just removes our rating from the node.

 

If we retrieve just the likes ratings for the node using http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/ratings/likes (9th request in the Postman collection) you'll notice the myRating property has been removed and the numberOfRatings count has decreased as shown below:

{
  "entry": {
    "id": "likes",
    "aggregate": {
      "numberOfRatings": 0
    }
  }
}

 

Let's now turn our attention to tags. To add a tag to a node we simply POST the body shown below to http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/tags (10th request in the Postman collection):

{
  "tag": "blog"
}

 

This results in the following response:

{
  "entry": {
    "tag": "blog",
    "id": "ecbe5049-a879-4b96-8c25-c8a776e43464"
  }
}

 

Add another tag called "post" to the same node (11th request in the Postman collection). We can then retrieve the tags applied to the node by using http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/tags (12th request in the Postman collection), you should see a response similar to the one below:

{
  "list": {
    "pagination": {
      "count": 2,
      "hasMoreItems": false,
      "totalItems": 2,
      "skipCount": 0,
      "maxItems": 100
    },
    "entries": [
      {
        "entry": {
          "tag": "blog",
          "id": "2c1be151-bbae-418a-9ddc-541a6a867c94"
        }
      },
      {
        "entry": {
          "tag": "post",
          "id": "9e41ee8b-fdac-4cbd-bde1-082b486ccc7c"
        }
      }
    ]
  }
}

 

When we add a tag to a node we're actually creating a global tag in the repository which we can see by retrieving all tags using http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/tags (13th request in the Postman collection). If you look closely at the response you'll notice the id of the tags are the same as the ones returned for the node above.

{
  "list": {
    "pagination": {
      "count": 2,
      "hasMoreItems": false,
      "skipCount": 0,
      "maxItems": 100
    },
    "entries": [
      {
        "entry": {
          "tag": "blog",
          "id": "2c1be151-bbae-418a-9ddc-541a6a867c94"
        }
      },
      {
        "entry": {
          "tag": "post",
          "id": "9e41ee8b-fdac-4cbd-bde1-082b486ccc7c"
        }
      }
    ]
  }
}

 

As tags are global to the repository removing them from a node does not delete the tag itself, let's see this in action by removing the "blog" tag from our node. To do this we need to DELETE using http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1/nodes/{{fileId}}/tags/{{tagId}} (14th request in the Postman collection) where tagId is "2c1be151-bbae-418a-9ddc-541a6a867c94".

 

Following a successful 204 response retrieve the tags for the node and the repository (12th and 13th requests in the Postman collection) and you'll see the "blog" tag has gone from the node but is still in the repository and available for use on other nodes.

 

Those of you paying close attention would have noticed that we only supplied the text of the tag when we added it to our node earlier. Before creating a new tag the API will first look for a matching existing tag and use that if it's present, meaning clients do not have to keep track of the underlying id of the tag.

 

That concludes our overview of the collaboration APIs for nodes, in the next post we'll look at the collaboration APIs for sites.

Improving Aikau Performance

Posted by ddraper Dec 12, 2016

When I presented a Tech Talk Live session on Aikau a few weeks ago I mentioned that I would be looking at performance issues. We'd already had some excellent analysis done by Axel Faust and I had done some initial prototyping but I'd not had the opportunity to properly focus on it. The discussion also came up in the IRC channel so I figured that it was about time that I looked into it properly.

 

I set up a performance test page that consisted of a list with a single view containing a table of 100 rows, each with 50 cells containing a single Property (so over 10,000 widget instances) where each Property was assigned 50 custom properties. This is not typical for an Aikau page but was the scenario that Axel Faust had previously encountered and was designed to help flush out where performance issues were originating.

 

This test page took 55 seconds to load in debug mode (PLEASE NOTE: This is not representative of normal page rendering - a typical existing Aikau page in Share on the same system takes under 2 seconds to render in debug mode).

 

Chart showing worst performance timeline

 

I spent a few days working through the troublesome areas and figuring out ways to drastically improve performance and have reached the point where once all the changes are integrated the performance test page load was reduced to under 15 seconds (a 366% improvement!) and under 10 seconds in production mode.

 

Improved Performance

In a more realistic test (with 25 rows and 5 columns) the rendering went from around 2 seconds to under 1 (but still a 100% improvement):

 

More reprentative Test before performance enhancmentsMore reprentative Test after performance enhancments

 

The biggest challenge was is to be able to fold all these improvements back into the main code base in such a way that:

 

  • All widgets work exactly as they did before
  • Function signatures in modules don't change
  • Execution code path doesn't change

 

It is the last 2 issues that are the most technically challenging but are important because we don't want to break any 3rd party extensions to existing widgets - and ensuring backwards compatibility of Aikau releases is our top priority!

 

The main changes that we've made can be summarized as follows:

 

  1. Building widget elements outside of the main document body (to avoid unnecessary browser rendering cycles)
  2. Building each widget DOM natively (rather than using the capabilities provided by dijit/_TemplatedMixin
  3. Bypassing unnecessary property handling in dijit/_WidgetBase
  4. Switching from using dojo/Deferred to native Promises

 

There is a new BaseWidget module that new widgets should use (and existing widgets will be converted to use) and a new ChildProcessing mix-in module that provides the function "createChildren" to replace "processWidgets". 

 

We made some initial improvements in the 1.0.100 release but the main improvements will be in the 1.0.101 release. At the moment that changes are confined to the list rendering and I will be working through the remaining renderer widgets to update them to build their DOM natively. The next stage will be to convert the remaining widgets to take advantage of the new code.

 

There's still a way to go before all the whole Aikau codebase is updated to take advantage of these improvements but I thought it would be useful to let people know that it was something we were focusing on. It also further shows how we're able to make these sorts of improvements in Aikau without impacting any existing pages.

Since yesterday, 5th of December 2016, you can find the new Alfresco Mobile for iOS 2.4 release available into the Apple App Store. The release comes with new improvements, mainly about new Sync content action, customised profiles, enhanced privacy and configuration with MDM providers.

 

 

Beyond the enhancements, the iOS and Android clients remain the unique products in the ECM market, provided as Open Source, allowing the users to work with content and Alfresco’s built in workflows.

 

 

Thanks to the Open Source model, the Partners and Customers are allowed to customise the App and even build their own one, using the SDK that you can find available here.

 

This version of Alfresco Mobile for iOS is certified for MobileIron AppConfig deployment.

A huge welcome to Desktop Sync 1.0.0 GA for Windows, available for Alfresco One. With Alfresco Desktop Sync, Windows users will be able to connect to Alfresco One on premise to select folders and sites to view and work with on their desktop, even when working offline. Files can be viewed and edited using standard desktop applications such as Microsoft Office and changes will be uploaded to the server. Every 5 minutes server changes are synced to the desktop, the latest content is always available.

 

Alfresco Desktop Sync is only available to paid subscribers.

 

 

For further informations you can refer to the documentation site.

Filter Blog

By date: By tag: