Technology - KidsIL

1. Home »
2. Technology

Adding Serverless to your Web Application

Posted on Oct 31, 2017

As you've probably heard, Serverless is the next big thing in the cloud industry.
Here's I'll go over some of the ways you can integrate Serverless technology in your existing web apps!

Serverless is essentially a system of Function-as-a-Service, where you have certain functions in the cloud, and you get billed by their usage & required computation.
This is a further abstraction from the actual Virtual Machines in common use today, and goes hand-in-hand with a Microservices approach to web development.
But enough about dictionary definitions, let's go to the good stuff:

Examples of integrating Serverless with your existing Web App

Reports

Database reports are a pretty common request, often by marketing/business departments to get a better understanding of your users.
This is more common in early-stage applications, before you connect them to proper analytics tools.
Using Serverless, it's quite simple to have a function that runs a report on the database.
We'll use AWS Lambda in this example:

Go to Lambda on your AWS Dashboard:
Choose "Author from scratch":

In your Lambda function, add the following code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
'use strict';
module.exports.runReport = (event, context, callback) => {
  const response = {
    statusCode: 200,
    body: {
      message: 'Result: ',
      input: event,
    },
  };
  const mysql = require('mysql');
  const connection = mysql.createConnection({
    host: 'localhost',
    user: 'user',
    password: 'password',
    database: 'database',
    port: 'port' //if you remove this line port defaults to 3306
  });
  connection.connect();
  connection.query('SELECT * FROM users', function(err, results) {
    if (err) throw err;
    response.body.message += JSON.stringify(results);
    response.body = JSON.stringify(response.body);
    callback(null, response);
  })

  connection.end();

};

As an easier alternative, get the entire Serverless Boilerplate here

Admin Panel

In most web applications there's some sort of an Admin Panel.
Ideally, the admin panel should be a separate application, to limit potential systemic weaknesses.
Using Serverless, we're able to take advantage of some of the most secure systems available - IAM permissions.
Examples of cool things you could do to further secure your Admin Panel:

Restrict Admin functions permissions - Since Lambda functions have their own IAM roles, you can have very fine-grained permission settings. A great example of this is with DynamoDB.
You can literally give your function access to specific tables with a custom IAM policy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1509470869566",
      "Action": [
        "dynamodb:BatchGetItem",
        "dynamodb:DescribeTable",
        "dynamodb:GetItem",
        "dynamodb:GetRecords",
        "dynamodb:ListTables",
        "dynamodb:Query",
        "dynamodb:Scan"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:dynamodb:YOUR-REGION:YOUR-AWS-ID:table/TABLE-NAME"
    }
  ]
}

Or another great example where a function can only write items to the table without reading any:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1509470869566",
      "Action": [
        "dynamodb:PutItem"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:dynamodb:YOUR-REGION:YOUR-AWS-ID:table/TABLE-NAME"
    }
  ]
}

Combine API Gateway with Cognito User Pools for authentication - This is quite complex, but this guide has a pretty good walkthrough. In the following days (probably weeks) I'll try to add this to the boilerplates repository.
Write the entire Admin Panel on Serverless - If your app doesn't have a fully fledged Admin Panel yet (and you understood the last 2 items on the list), This could offer both a high level of security as well as a great way to integrate Serverless into an important but not-yet-customer-ready parts of your system.

Cron Jobs

Most applications have some form of Cron Jobs, but more often than not,
there's a good chance that you don't have a lot of logging/analytics for your Cron Jobs.
With CloudWatch you can schedule functions to run every minute/hour/day/week, just like a normal Cron Job.
Combine that with CloudWatch Alarms, and you have both a logging system and an alarm system in case something goes wrong.

These were just a few examples of how you could integrate Serverless into your web app right away.
In the future, I'll try to work on the boilerplate repository, to add more cases.

View-specific JS Files in Sails.js

Posted on Jun 15, 2017

One thing I really loved about Meteor, is that you can fine-tune which files are included (and the order of inclusion).
This is especially useful when structuring everything in small Meteor packages.
So for example, if you simply want to add relevant client JS files (on package.js):

1
2
3
4
api.addFiles([
    'client/page-login.html',
    'client/page-login.js',
], 'client');

page-login.html:

1
2
3
4
5
6
7
<template name="PageLogin">
  <form id="login-form">
    <input name="username" type="text" />
    <input name="password" type="password" />
    <input id="submit-login" type="submit" />
  </form>
</template>

page-login.js:

1
2
3
4
5
6
Template.PageNewDomain.events({
  'submit #login-form': function (event, template) {
    event.preventDefault();
    //handle form submission..
  }
});

Together with specific Template namespaces, it allows you to have template-specific JS code that doesn't interfere with other templates. I looked for a way to have that feature in Sails.js.

Instead of trying to implement the entire complicated Template rendering engine, I figured it should be enough if every view gets a corresponding JavaScript file that is optionally loaded with said view.

How it should work

Each View in the views/ folder has a name, let's create a corresponding JS file with the same name
On Sails.js, JavaScript files are stored separately in the js/ folder, let's store all view-specific JS files under js/views/
JS filenames should be the same as view filenames. views/dashboard/login.ejs will have a JS file named js/views/dashboard/login.js

How to implement

add this to config/views.js:

1
2
3
4
locals: {
    existsSync: require('fs').existsSync,
    jsPath: __dirname + '/../assets/js/views/'
}

This is so that we're able to check if a JS file exists.

on the bottom of views/layout.ejs, under blocks.localScripts

1
2
3
<% if (existsSync(jsPath + req.options.controller + '/' + req.options.action + '.js')) { %>
  <script src="/js/views/<%= req.options.controller + '/' + req.options.action %>.js"></script>
<% } %>

In tasks/pipeline.js in the jsFilesToInject array, we need to remove
```
js/**/*.js
```
This is Critical. Otherwise, all JS files under js/views will be automatically included in ALL views.
Instead, to automatically inject other views we can have the following line:
```
js/dependencies/**/*.js
```

That's it! Now you'll be able to write view-specific JavaScript files!

Meteor Alternative to Kadira

Posted on Feb 3, 2017

or: How to integrate SPM with Meteor

It's been far too long since I wrote a post, life gets super busy sometimes.
At any rate, today I'm going to be writing about something that is searched by quite a lot of people:
An Alternative to Kadira.

What is Kadira?

Kadira is the only Cloud-based Meteor Debugging tool out there.
If you're familiar with tools like New Relic, Kadira is basically a lean version of that.
It allows you to see live bugs/errors coming from your application, with a full backtrace so you know which part of the application is causing the problem (to some extent). It also has limited tools for reporting/statistics, so you can see if there's a great uptick of bugs lately or a certain type of bug after introducing a new feature.

Why look for a Kadira Alternative?

It was recently published that the founder and maintainer of Kadira has moved away from Meteor.
Leaving the fate of Kadira very uncertain. In fact, a full year and a half before the announcement,
there weren't any changes to Kadira's feature set.
What disappointed me the most was that Kadira's source wasn't released, so on the one hand it won't be maintained any longer, but on the other hand, there's no option for the community to step in.

On top of that issue, Kadira is quite limited, searching/navigating through issues is very difficult and at times - impossible. there's a lot of repeated bugs that are not registered correctly (without any backtrace or info), making it very hard to find which is a Kadira bug, and which is an actual bug in my application.

That's why I thought that it was time to seek an alternative.

Who plays nice with Meteor?

Most of the major services (New Relic, AppDynamics), didn't support Meteor, or if they did - it was a very limited support, and considering the price point (in the hundreds, sometimes thousands monthly), I expected to have a complete turnkey solution with minimal integration issues.

After looking at a few posibilities, I decided to go with SPM from Sematext.
This tool is not the most popular debugging tool in the world, but their API is clear, easy to use, and their interface is quite nice as well.

Implementing SPM Debugger in Meteor

This can be done with 2 easy steps

Add the following to your package.js or .meteor/packages for chaotic packageless projects (SHAME, SHAME):

1
2
3
4
5
Npm.depends({
	'spm-agent-nodejs': '1.30.2',
	'winston': '2.3.0',
	'winston-logsene': '1.2.1'
})

Add the following code to your Server-side code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
/* Loading SPM Monitoring Agent */

var spmStart = function() {
  process.env.NODE_OPTIONS = '-r spm-agent-nodejs';
  var logsene_token = LOGSENE_TOKEN; //Insert Logsene token here
  var spmAgent = Npm.require('spm-agent-nodejs');

  //Logsene
  var winston = Npm.require('winston');
  var logsene = Npm.require('winston-logsene');
  var logger = new winston.Logger();
  logger.add(logsene, {token: logsene_token, type: 'logs'});

  // Overriding Meteor Error function, simply adding the
  // logger before the original function runs
  var util = Npm.require('utils');
  var errorFunc = Meteor.Error;
  Meteor.Error = function(error, reason, details) {
    logger.error('Error: ' + error, {
      error: error,
      reason: reason,
      details: details,
      stack: Error().stack,
      tags: ['error', 'winston']
    });
    errorFunc.call(this, error, reason, details);
  };
  util.inherits(Meteor.Error, errorFunc);
};

spmStart();

Special thanks to Wojtek Krysiak for adding inheritance here

I suggest creating a separate file in your Meteor Project (or better yet, a separate package!)

That's it! Meteor errors AND Performance Information should be visible on your SPM dashboard.

I might organize the code snippet and add it to GitHub at a certain point (fingers crossed).

Until next time!

Asaf Zamir – Chief Technology Officer and Cloud Consultant