Introduction

This is a new feature of Blake’s Log where Kristi and Matt can add their thoughts on specific topics that may in some way relate to the world of AWS and share them with everyone. Matt was asked to write this article as part of a campaign with Freeman and Forrest, so no hidden intentions here. It just seemed like a fun thing to do.

What Is n8n?

Website: n8n.io
Description: “Secure, AI-native workflow automation. The world’s most popular workflow automation platform for technical teams.”

After some time spent using the product I would describe it as a way of building patterns for repeated processes using a visual builder that allows deep integrations with external services like gmail, OpenAI, slack etc or you could hook it into your internal services as well.

What Does It Look Like?

It is a very aesthetically pleasing tool, it is based around a visual workflow editor that allows you to schedule, execute or debug those flows. You can even add documentation at each step to help anyone else understand the workflow in the future. The template below is called Host your own Uptime Monitoring with Scheduled Triggers.

You don’t always have to start from scratch with the workflow editor above because the platform is built around the idea of reusable templates to solve a problem which can be private to your team or shared with everyone. As you can see in the image below, it self describes as having over 900 templates submitted by the global creator community.

The image below shows some of the templates available under the “IT Ops” tag:

Who Is It Aimed At?

Visual editors sometimes fool people into thinking they are a tool to replace engineers but that is not what we have here. This product is aimed squarely at engineers/technical folks because you will need to know how to setup the authentication for all of the integrations used in a workflow. Want to send an email? Well, you need to setup the GCP Email API with OAuth client credentials for instance. The other feature that leans into the technical audience is that it still allows you to write JavaScript or Python code for the various steps through their code node feature.

So basically if you are writing a bash script for repeating processes today, you may fall in love with it because it is lightyears ahead developer experience wise. They do make a point of directly comparing themselves to Zapier and Make, so users of those tools should take a look.

After doing some research, another area where n8n shines that I hadn’t initially considered is in the introduction of AI into existing processes. Nobody wants AI to be integrated unchecked, you always need “a human in the loop” so n8n allows you to add that human friendly step and take advantage of AI safely.

Another great feature of the platform is that they offer Self Hosted and Hosted Cloud versions. I spent most of my time on the Hosted Cloud version but just know for places like large enterprises with heavy compliance needs, you can self host your workflows. I have been told it can be up and running through docker in less than 15 minutes.

How Much Does It Cost?

The prices below are shown in Pound Sterling (GBP), please note that prices could be significantly different in other countries.

The Hosted Cloud Version

I used the 14 day free trial to experiment with it, so if you are interested but are worried about the costs just know there is a grace period.

After the trial period you have options based on the amount of active workflows you want to run and how frequently they will run:

The Self Hosted Version

An important thing to note is that there are two versions of the self hosted product. There is a community version and an enterprise one.

The community version is free but does not include some enterprise grade features, the enterprise exclusive features are listed out here if you are worried. I am not sure about the cost of the self hosted enterprise version, I didn’t see a price listed on the website so I assume it is priced per customer based on the above pricing image.

Comparing It To AWS’s Products

I am an AWS Hero and this site is called Teach Me AWS, so this seemed like a great opportunity to cross the streams and pick something from my regular toolset to compare n8n to. When I reached into my tool box, the tool that came closest to hand was an AWS Step Function.

Step functions are a natural comparison as they give you a studio based on visual flows with error handling/debugging built in similar to what n8n offers. If Step Functions are new to you, the image below is a saga pattern that I created using step functions and AWS CDK:

AWS takes a different approach to n8n in terms of being an integrated platform with pre-baked templates available inside the platform itself, they lean on community contributions from people like the Heroes or content from their Dev Advocate team. With that said, the equivalent of the templates offered in n8n for customers would be the AWS serverlessland website, The AWS Serverless Application Repository or the AWS CDK Construct Hub.

Then in terms of native integrations with external services like OpenAI, you can absolutely do this easily with a Step Function but you will be configuring it yourself and juggling two sets of documentation whereas n8n has over 1000 pre-configured integrations available today which is a time saver, plus the documentation around those integrations is fantastic from my time spent here.

Having said that, the unique benefit of a Step Function is that they can natively integrate with a lot of AWS Services with 0 config. This is powerful because you can do cool things like send events that anyone can subscribe to into an EventBridge event bus, fire off an SNS Message to create a fan out pattern or use Amazon Bedrock to call third party AI models all from within AWS.

In terms of cost, AWS Step Functions charge per state transition rather than per workflow execution but you get 4000 free state transitions per month forever in the free tier. This is not an expensive service unless you build some seriously big workflows but neither is n8n.

The huge upsell of n8n over a step function is that it can run anywhere since there is a self hosted dockerised version whereas the AWS product only runs on AWS. This is not a feature to overlook because there is a huge market for people in very regulated industries. In the interest of full disclosure, technically you can drop huge money to get an AWS Outpost installed inside your data center to access some AWS products on prem but that’s not a fair comparison to this containerised feature.

Outpost brings me to my last point of comparison in that using a Step Function means if you weren’t using AWS before that you are now using AWS and with it brings the opportunity/risk to start using other things on AWS. Every new product you use unless you have a robust cloud monitoring process will bring billing and security implications. AWS has no “I want to spend £20 this month then stop” switch. So, if you have no exposure to AWS today then there is a smaller potential cliff to fall off using a product like n8n with no real downside because an n8n workflow could still trigger a Step Function later if you need, this is not a one way door.

My Conclusion

n8n is a visual tool to enable engineers to build repeatable processes quickly. It allows them to run those flows locally, on prem, on the cloud of their choice or use n8n’s hosted cloud option. It has hundreds of integrations into external services with amazing documentation to help you configure access from your workflows. It’s not expensive unless you start creating large numbers of active workflows at which point I would assume it’s just a quick phone call to discuss your needs and if it gets too expensive, the community version is free at the cost of some enterprise features.

If I was in a position where my organisation was adopting AI to speed up business processes and I needed to build a robust, repeatable “person in the loop” flow quickly with no deep knowledge of AWS or with a need to run on prem then this is a fantastic choice.

Personally, I will probably still reach for the Step Function from my regular tool set but that is due to my deep knowledge of AWS and I don’t have any real need to run my workflows anywhere else. It’s always a good idea to broaden my exposure to other tools though and this one was a fun one to use.

Showing My Experience

Everything said above was the output of my experience below and it could have been enough by itself to publish as an article but to be totally fair and transparent to anyone reading I wanted to show my flow because I did genuinely use the product.

Choosing a Template

This was harder than you’d think, not because of the developer experience of n8n but because for me to test it out I needed something better than super basic but not so complex that I spend a week configuring connections to third party APIs that all have their own costs.

I did like the idea of the template that connects to the OpenAI API to generate a website based on user provided text but OpenAI wasn’t offering me any trial credits or free tier and I didn’t fully know how much it would cost. I decided to pick the template called Host your own Uptime Monitoring with Scheduled Triggers that generates a scheduled task, the task pulls down a spreadsheet from Google Docs, for each record found there (which is a URL), it will do an HTTP status check and store the status in another Google sheet. If the status changes between tests then it will send an email alert to me through the GMail API. It can also post a slack message after the email but I didn’t feel like that step was necessary to test the tool.

This is a fairly complex pattern involving scheduling, external APIs, logical choices and persistence plus as a bonus I already had accounts for all of the pieces involved.

Setting Up Credentials

After clicking the “Use Workflow” button above, I was taken to a new screen where n8n knew the key external services that the pattern would use. It asked me to setup and enter all of the needed credentials before allowing me to proceed. After entering values it even did a call to the API to test they worked. The documentation around this was top notch, gold star for all involved. To setup the GMail integration for instance you need a GCP account and it talks you through setting up GCP and enabling the API before telling you how to setup scoped credentials. I liked this screen a lot.

Running The Workflow

Now that I had working integrations with the necessary external services, I was presented with the visual workflow showing all of it’s steps in an editable format. You can just click into any of the icons to reconfigure what they do. There is also an orange button at the bottom of the editor which lets you run the workflow… so I pressed it… and I hit an immediate error.

This was when I started reading the text on the screen and noticed that each section has notes:

The get sites step stated that I need to manually create a google sheet with two columns “Property” and “Status”. It didn’t actually tell me how to connect the sheet to this step but I would circle back to that after uploading my spreadsheet:

I uploaded the file and clicked on the google docs icon in the workflow to be presented with a configuration screen where the fields would pull their values automatically from Google Drive. Again, gold star. Super easy to configure.

Problem In Chair, Not In Computer

I was excited at getting the drive configuration done so I clicked the test workflow button again… and error. The way it represents itself is a red border on the step that failed and a small toast notification on the bottom right corner with the error message. To be honest at this point I wasn’t sure how to debug the error because the last time I opened one of these boxes it was to change configuration settings.

It said invalid URL in the toast notification so I decided it had to be in the Drive step. It was at this moment I noticed that the notes said the columns should be called “Property” and “Status” whilst I had called mine “property” and “status”. PICNIC (Problem In Chair, Not In Computer). After changing those column names I clicked test workflow again and it worked!

Or so I thought. I noticed that the workflow didn’t go the whole way to the end, it wasn’t sending status alert emails or logging the current status to Drive. The Slack error below is because if you remember from the start I decided it was unnecessary in this test so didn’t configure it and therefor is nothing to do with the current execution termination problem.

Debugging The Error

This was was very puzzling to me because the “Status Router” step was supposed to take a different action for every possible permutation of status condition. It has different flow lines for “stayed up” or “changed from up to down” etc but none of them were triggered. So, I opened the configuration to see what was happening.

The JSON object that comes into this step has 4 boolean variables:

• UP_FROM_UP
• UP_FROM_DOWN
• DOWN_FROM_UP
• DOWN_FROM_DOWN

all of these had the value of false but to execute a next step logically one of them needed to be true.

So, I moved further up the flow and checked the calculate status step.

It was here I spotted the first anomaly, the logic here had expected a variable called $json.Status to be present but when I looked at the input object on the left it was not there. So I went even further up stream to see what was happening and opened the site test step:

I can see the value is on the left in the input but does not translate over to be mapped to the output. I looked at the configuration options and couldn’t see any lever I could pull to map that across so for the purposes of this experiment I decided to just move on and change the logic in the status router to send the email if the value is false instead of true to see if the gmail integration worked.

One Last Run

I clicked the test workflow button again and voila! We got through to the gmail task. I checked my email and there it was, a status update email from my workflow.

Wrap Up

I genuinely don’t know if I made a mistake in misconfiguring something or if this template is broken, for the purposes of this it wasn’t worth diving too deeply but in retrospect, with no knowledge of how this template was built I was easily able to follow the error upstream to see where the disconnect is. I can even explain it to anyone reading this visually using a few screenshots. That’s pretty cool on it’s own. n8n was at no point difficult to use and I would expect with more time spent there are a tonne of tips and tricks that would accelerate me on future workflows.