Serverless and the Expo

24 December 2018

The festive period is upon us but it’s hard not to keep pressing on exploring the wealth of content revealed and renewed at re:Invent 2018.  I am still buzzing with ideas about new ways of architecting tech solutions. I’ve already started having a play with the new Lambda and Aurora Serverless features and they are awesome.  Like most engineers out there I am looking for ways to tactically incorporate these features into my designs. To qualify for a tech shift they must test positive against less complexity, less cost and awesome tech.  Most do! I am working to quantify those tests and make a hard shift over the next few weeks/months as support grows in the community (Serverless Aurora for MySQL 5.7 please!).

I know I’m not the first to fly the Serverless flag but at the end of 2018 it really is the only game in town for new development.  At re:Invent we heard so much commitment to this journey from AWS (again). We can now write Lambda functions in Bash, target them with an ALB and put WebSockets through API Gateway, use transactions on DynamoDB…  what else do we really need?


Alongside all the Serverless buzz another huge highlight for me was the Expo.  I got 3 big steers from visiting vendors that are set to change my architecture in 2019.

1 – ServiceNow

I have worked on integrations to ServiceNow and thought I had what I needed for a proper integration flow.  To get data into ServiceNow (raise a change, create an incident etc…) call the API; to get data out (e.g. change approval status) call the API or setup a business rule.  Little did I know there was a full workflow engine sitting on top of it called Flow Designer. Flow Designer looks like the next game changer for my integrations and I found out all about it by talking to an engineer at the Expo.

2 – Nessus

If you haven’t come across this product before Nessus performs vulnerability scanning of your server fleet (ec2, on-prem etc…).  To scan a server in the fleet Nessus requires credentials to login with sufficient permissions to look at software inventory checking versions and signatures.  The default setup is to store credentials in a centrally managed Nessus service platform which may or may not pose a risk depending on how well you manage the storage of those credentials.   While at the expo I sought out the Tennable Nessus booth and quizzed the engineers on something a friend had mentioned – credential-less scans using Nessus Agents. Seems like a no brainer that this should exist but I had missed it.  Anyway gap filled, no need to store credentials centrally in the platform, backlog item added! If you haven’t seen this before either there is a great summary here.

3 – Elastic

The AWS ElasticSearch service has been vital for us to minimise operational overhead and provide a robust system analytics platform but it’s missing a few features on the periphery.  After talking to the Elastic team we found that the roadmap for new features such as indices lifecycle management, RBAC and Canvas (canvas helps you move from CIO PowerPoint to beautiful live data boards) are all out of scope for the opensource license used by AWS.  So, if you are all-in on Elastic and want more features you need to take a look at the SaaS offering by Elastic. According to the engineer they are soon to offer a AWS Private Link endpoint for their SaaS service. At which point we will look to shift our workload from AWS native to Elastic SaaS.

So here we are at the end of 2018 with a massive tech boost for 2019 thanks to AWS and it’s partners.  If you are planning to go to next year’s re:Invent or any conference before it then make sure you go to the expo, speak to the vendors and use their expertise to litmus test your architecture.

If you missed anything at re:Invent 2018, catch up here (thanks @jasonpoley).

Steve Fletcher