A helper Node module for the Encryptamajig helper library

I recently had to work with a value encrypted using the Encryptamajig helper library. The value needed to be decrypted in NodeJS using CryptoJS. I ran into several gotchas while replicating the decryption logic in [1] and created a simple node module to help others. The post here [2] helped immensely in determining how to generate the IV and Key.


You can find the encryptamajig-helper module here [3]. I will update this post once I finish the encrypt method.

References

[1] https://github.com/jbubriski/Encryptamajig
[2] https://stackoverflow.com/questions/29807108/derive-key-and-iv-from-string-for-aes-encryption-in-cryptojs-and-php
[3] https://github.com/splinter/encryptamajig-helper

Setting up a private VSTS NuGet feed on VS2017 for Mac

This is a personal memo on setting up a private VSTS feed  on a Mac with Visual Studio 2017 for Mac OS.

1.  Create a new Personal Access Token (PAT). You can follow the documentation here [1] to set this up
• Important: Make sure to set Accounts field to "All accessible (<your account>) accounts
2.  Go to the feed details page and  click the "Connect to Feed" button. It will give you a NuGet add source command.
3.  Load a solution with a private feed
• Visual Studio should automatically prompt you for credentials
• Provide the username and the PAT as the password
• If Visual Studio does not prompt you for credentials navigate to Tools > Add Custom Tool > NuGet > Sources and enter your username and PAT (as the password)

Notes

• The credentials are not stored under the apiKeys configuration element but rather it is stored under packageSourceCredentials as an encrypted password

References

[1] Using Personal Access Tokens with VSTS, URL: https://docs.microsoft.com/en-us/vsts/accounts/use-personal-access-tokens-to-authenticate?view=vsts
[2] NuGet restore from a secured feed, URL: https://www.xavierdecoster.com/post/2012/12/12/nuget-package-restore-from-a-secured-feed.html 
[3] NuGet API Keys, URL: https://docs.microsoft.com/en-us/nuget/reference/nuget-config-file#apikeys
[4] NuGet config file, URL: https://docs.microsoft.com/en-us/nuget/reference/nuget-config-file
[5] Add a private feed to your NuGet configuration (Windows) , URL: https://docs.microsoft.com/en-us/vsts/package/nuget/consume?view=vsts#windows-add-the-feed-to-your-nuget-configuration

Hosting an Angular 2 app with deep linking support in OpsWork (Chef 11 Stack)

The Problem

We recently developed an Angular 2 web application and after deploying it to a staging we encountered an issue with link sharing not working. The application was hosted on an AWS OpsWorks stack using the static web server layer.The static web server layer uses NGINX to serve the web application. The link sharing did not work as the links did not correspond to actual resources available on the server.

How do we support deep links ?

The knowledge of the link resource is available to the application router and thus NGINX needs to be instructed to route all traffic to the application root when a resource is not found. This can be achieved using the try_files directive [6] as follows:

try_files $uri$args $uri$args/ /index.html;

To test this we SSHed into the instance and accessed enabled sites configuration for the application. This is usually accessible at /etc/nginx/sites-enabled. The try_files  directive was added to the following block:

After making the change we restarted the NGINX service using the following command:

sudo service nginx restart

Working with OpsWorks

It worked! we were able to share links. However, we were now faced with the problem of adding this configuration to the OpsWork Stack configuration. The above configuration cannot be added using an attribute and instead must override a configuration template. The steps for doing this are listed here [3]:

1. Identify the cookbook that contain the configuration template. In this case it was the nginx cookbook which is located here [7].
2. Replicate the cookbook structure making sure only to keep the template that needs to be modified. 
3. Create a zip archive and upload it to a location which OpsWorks can access. In our case we used S3.

A cookbook with this updated template is available here [1].

Deploying to OpsWorks

1. Enable custom cookbooks for your OpsWork Stack
2. Provide the location of zip archive as the location of the custom cookbook
3. Execute a Run Command for the Update Cookbook action
4. Deploy the application

References
[1] https://github.com/splinter/angular2-deeplinks-opsworks-cookbook
[2] http://stackoverflow.com/questions/38991541/nginx-and-angular-2
[3] http://docs.aws.amazon.com/opsworks/latest/userguide/workingcookbook-template-override.html
[4] http://docs.aws.amazon.com/opsworks/latest/userguide/cookbooks-101-opsworks-templates.html
[5] https://serverfault.com/questions/378581/nginx-config-reload-without-downtime
[6] http://nginx.org/en/docs/http/ngx_http_core_module.html#try_files
[7] https://github.com/aws/opsworks-cookbooks/tree/release-chef-11.10/nginx
[8] https://github.com/aws/opsworks-cookbooks/blob/release-chef-11.10/nginx/templates/default/site.erb

CloudWatch DataAlreadyAcceptedException issue

The Problem:

We recently started using AWS Cloud Watch Agent for uploading logs for a few of our NodeJS processes. Soon after our initial move we began to notice that logs were not been uploaded for the majority of the processes.

Debugging Process:
In order to debug the issue further we began by looking at the CloudWatch agent logs located in the var/log directory.

We noticed the following logs in the /var/log/awslog.log:

2017-03-24 05:32:48,758 - cwlogs.push.publisher - WARNING - 14818 - Thread-5 - Caught exception: An error occurred (DataAlreadyAcceptedException) when calling the PutLogEvents operation: The given batch of log events has already been accepted. The next batch can be sent with sequenceToken: 49567766878003081255417777553615842777473923345314516562

The cause of DataAlreadyAcceptedException is a bit vague:

The event was already logged.

However, reading about the component which throws this exception gives an inkling of the cause [2]. The exception is thrown by the PutLogEvents API. The official documentation [1] lists the following conditions under which a log may fail to upload:

 - The maximum batch size is 1,048,576 bytes, and this size is calculated as the sum of all event messages in UTF-8, plus 26 bytes for each log event.
 - None of the log events in the batch can be more than 2 hours in the future.
 - None of the log events in the batch can be older than 14 days or the retention period of the log group.
 - The log events in the batch must be in chronological ordered by their timestamp (the time the event occurred, expressed as the number of milliseconds since Jan 1, 1970 00:00:00 UTC).
 - The maximum number of log events in a batch is 10,000.
 - A batch of log events in a single request cannot span more than 24 hours. Otherwise, the operation fails.

In our case logs which failed to upload met all criteria except the following:

 A batch of log events in a single request cannot span more than 24 hours. Otherwise, the operation fails.

Oddly enough this was preventing even current logs from been uploaded to CloudWatch.

Solution:

The state of the AWS CloudWatch agent is maintained in a file defined in the CloudWatch configuration file.

In our case the state file was defined in our cwlogs.cfg file was:

/var/awslogs/state/agent-state


In order to force the CloudWatch Agent to force upload the logs we deleted the state file.



References

[1] http://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html
[2] http://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html#API_PutLogEvents_Errors
[2] CloudWatch Agent Reference, http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html
[3] CloudWatch Agent start command, http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/StartTheCWLAgent.html
[4] CloudWatch Agent stop command, http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/StopTheCWLAgent.html
[5] CloudWatch Agent status command, http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/ReportCWLAgentStatus.html

Adding a custom list order to asset types

Introduction

This brief tutorial will provide a set of instructions to reorganize the order in which the icons appear in the WSO2 Enterprise Store . The tutorial will illustrate the steps for achieving this goal in the Store application. Our goal is to add a new asset type and get it to appear as the first entry in the asset type navigation menu as shown below:

A deploy-able sample with all the necessary resources is available here [1] . The content of this tutorial is valid for WSO2 Enterprise Store 2.1.0.

How are the icons rendered?

The list of available assets is populated using two page decorators (both called navigationBar) :

1. In the app extension, https://github.com/wso2/carbon-store/blob/master/apps/store/extensions/app/store-common/app.js#L122 
2. In the default asset extension, https://github.com/wso2/carbon-store/blob/master/apps/store/extensions/assets/default/asset.js#L223

In both cases a single bit of logic is invoked:

The implementation of  the navigationBar page decorator can be viewed here.

The above snippet of code iterates through all of the available asset types and builds an array which is used to render the asset icons. In order to control the order we will be writing a new page decorator which will be invoked by both application and asset pages.

Changing the order of the asset types

We will need to introduce a new page decorator to alter the order of the assets.The new decorator defines a list of predefined asset types as shown below:

Important: When you define new asset types you will need to modify the hard coded list of asset types

The above snippet of code is defined in the decorators.js located here. The key change is located at line 16 where the UI activated asset type list is replaced with a hard coded list of values to reflect the order depicted in the original screenshot.

Once the new decorator is written we will need to wire it up by adding a new default asset extension and by changing the decorator used to populate asset types for global pages.

Affecting changes to the asset pages

In order to affect changes to the icon list that appears in the asset pages we can deploy an asset extension which will modify the default asset behaviour (the order of icons in this case).

This is packaged as an app extension and is available here. The instructions for deploying the app extension is described in the README.md of the repo.To summarize, you will need to:

1. Stop the running WSO2 Enterprise Store Server
2. Copy the mycustom-store folder to the {CARBON_HOME}/repository/deployment/server/jaggeryapps/store/extensions/app directory
3. Restart the WSO2 Enterprise Store Server

Affecting changes to the Top Assets page

In order to affect changes to the icon list in global pages such as the top-assets page we will need to directly modify the navigationBar decorator defined in the store-common app extension  here [2].

Done!

To view these changes navigate to the WSO2 ES Store application: https://localhost:9443/store

References

[1] https://github.com/splinter/es-menu-sample
[2] https://github.com/wso2/carbon-store/blob/master/apps/store/extensions/app/store-common/app.js#L122

How to extend an asset's store view in WSO2 GREG 5.3.0

This blog post is an amendment to Niranjan's blog post [1] on the same topic. The blog post while correct needs to be updated for G-REG 5.3.0 as there have been several minor changes to the templates. A set of ready to use templates with updated changes can be found here [2]

I will first summarize the steps performed by Niranjan:

1. The default implementation of the details page in the Store application does not display all of the attributes visible in the Publisher. One important factor behind the decision to reduced information is that some of the data such as endpoints are not in a format which is easily digestible by templates
2. In order to rectify this Niranjan has introduced the jsonFormatter method which converts these hard to digest attributes into a format easily consumed by templates.
3. To display the information he has introduced several new templates which mirror the information in the Publisher [3]

The release of G-REG 5.3.0 has necessitated several changes to the above steps:

1. Alterations to the overview.hbs (Notice that . has been replaced with assets)
2. Omitting the  asset-attributes.hbs
3. Adding rows to the doclinks template

EDIT: A colleague (Chandana Napagoda) has pointed out an issue in the doclinks template.This has now been corrected in the git repo.

Reference
[1] http://niranjankaru.blogspot.com/2016/02/how-to-extend-assets-store-view-in-wso2.html
[2] https://github.com/splinter/greg-tutorial-1/tree/master

Destructuring in ES2015

This is a first in a series of blog posts that will chronicle my exploration of the ECMA Script 2015 language update. The ES2015 is a specification which updates JavaScript with several additions to the language syntax. These improvements fall under the broad umbrella of ECMA Script 6 (ES6).

Why now?

I recently started working on a hobby project involving React and React Native which contain a lot of examples written in ES6.Hence, I thaught I would kill two birds with one stone and learn a bit about ES6 language features.

Important: I am in no ways a JS language expert and these blogs are a self learning tool (Caveat Emptor!)

What is Destructuring?

The process of extracting values from an object or an array [1]. It allows you to write concise code by:

1. More cleanly handling functions which return multiple values
2. More concisely access nested properties in objects

The syntax for destructuring in ES6 changes based on whether your working with an object or an array(Refer to Case#1 and Case#2 in the example below).

JS Sample on JSBIN 

When destructuring an Object JS will call toObject method on the left side of the expression.Hence, attempting to destructure undefined or null will cause an exception [1] [2].This means that primitives can be destructured too! (Refer to Case#5)

Loads of examples can be found here [4] and here [5].

I will be updating this post with more info once I get a bit of practice with this new syntax.

References

[1] http://exploringjs.com/es6/ch_destructuring.html
[2] http://www.ecma-international.org/ecma-262/6.0/#sec-toobject
[3] https://babeljs.io/docs/learn-es2015/
[4] https://ponyfoo.com/articles/es6-destructuring-in-depth
[5] https://gist.github.com/mikaelbr/9900818