Search for a command to run...
Observe your event bridge events in real-time using AppSync
👨💻 Sofware Engineer · 💬 Consultant · 🚀 Indie Maker · ⚡️ Serverless
🔧 serverless-appsync-plugin · serverless-appsync-simulator · middy-appsync
🐈 and 🍵
No comments yet. Be the first to comment.
Originally published on Medium Updated on 2020–12–26: After posting this, I realized that it could be improved this, even more, by using DynamoDB TTL. I updated this article accordingly. While I was working on a GraphQL API, I needed a couple of res...
If you're on Spain's regulated electricity tariff (PVPC), timing is everything. Prices change every hour based on supply, demand, and energy source — meaning your dishwasher costs three times more to
Invoking Private API Gateway endpoints from EventBridge
How to Invoke Private API Gateway From Step Functions
On-Demand Self-Destructed Sandbox AWS Accounts
Using JSONata capabilities to unmarshall DynamoDB data natively in Step Functions
Benoit Boure
16 posts
I am a software engineer with a focus on serverless technologies. I blog about Serverless Follow me on Twitter - Need help?
I recently came across David Boyne's blog post: How to Observe EventBridge Events with Postman and WebSockets. What a great idea! But, then I thought:
I can do the same with AppSync Subscriptions!
I had to try! Here is what I achieved:
The idea was simple. I needed the following components:
An AppSync API
A Mutation that receives events from EventBridge
A Subscription that is attached to the aforementioned Mutation
An EventBridge rule that sends events to the AppSync Mutation (target)
I also wanted to be able to filter events I was interested in. Here, I thought about two options:
Filter the events in the EventBridge rule.
Send all events to AppSync and use AppSync to filter them, thanks to subscription arguments
I went with the second approach. It would give me more flexibility to filter the events at query time instead of having to re-deploy each time I wanted a new filter.
Here is the GraphQL Schema I created:
type Mutation {
sendEvent(event: EventBridgeMessageInput!): EventBridgeMessage
}
type Subscription {
subscribe(
source: String
detailType: String
account: String
resources: [String!]
): EventBridgeMessage
@aws_subscribe(mutations: ["sendEvent"])
}
type EventBridgeMessage {
id: ID!
version: String!
detailType: String!
source: String!
account: String!
time: AWSDateTime!
region: String!
resources: [String!]
detail: AWSJSON!
}
input EventBridgeMessageInput {
id: ID!
version: String!
detailType: String!
source: String!
account: String!
time: AWSDateTime!
region: String!
resources: [String!]
detail: AWSJSON!
}
I also needed to setup the Mutation. I used a NONE data source for that and a simple mapping template that just returns the received payload.
All done! Now, by executing the sendEvent Mutation, it gets delivered to the subscription! 🙌
All that was left to do was to configure EventBrige and set the Mutation as a target.
My first attempt was to use API Destinations. I followed this awesome tutorial and defined my Input Path and Input Transformer rules which looked like this:
InputPathsMap:
version: $.version
id: $.id
detailType: $.detail-type
source: $.source
account: $.account
time: $.time
region: $.region
resources: $.resources
detail: $.detail
InputTemplate: |
{
"query": "mutation SendEvent($event: EventInput!) { sendEvent(event: $event) { version id detailType source account time region resources detail } }",
"operationName": "SendEvent",
"variables": {
"event": {
"version": "<version>",
"id": "<id>",
"detailType": "<detailType>",
"source": "<source>",
"account": "<account>",
"time": "<time>",
"region": "<region>",
"resources": "<resources>",
"detail": <detail>
}
}
Unfortunately, that didn't work! 😞
The problem is that in EventBridge, the detail attribute is an arbitrary JSON object which could have any shape. This is the reason I used an AWSJSON type in my GraphQL schema (I wanted to receive any event). The problem is that AppSync expects the JSON to be stringified!
After some investigation, I could not find any way for EventBridge to stringify JSONs. So, that was a dead end.
If EventBridge cannot do it, Lambda surely can! So, I wrote a simple lambda that receives the event, reformats it and calls the AppSync endpoint. I then just configured the Lambda as an EventBridge target. (See the code here).
✍️ Note: I also added an IAM authentication method to the AppSync API that Lambda can use to call the Mutation (in addition to the API key used by the subscription).
All set! Now, running the following subscription:
subscription MySubscription {
subscribe {
resources
region
source
version
detailType
detail
}
}
And sending an event into Event Bridge
aws events put-events --entries '[{"DetailType": "my.detail.type", "Source": "my.source", "Detail": "{\"foo\": \"bar\"}"}]'
Response:
{
"data": {
"subscribe": {
"resources": [],
"region": "us-east-1",
"source": "my.source",
"version": "0",
"detailType": "my.detail.type",
"detail": "{\"foo\":\"bar\"}"
}
}
}
It works! 🎉
One of the great features of AppSync subscriptions is that you can specify which changes you are interested in at query time. You can do that by adding arguments to the subscription endpoint. Whatever value you pass in the input, you will only receive changes that match the Mutation's response fields values.
So, I can now do queries such as
## Will match events with detail-type = "my.detail" only
subscription {
subscribe(detailType: "my.detail") {
id
detailType
detail
}
}
## Will match events with source = "my.source" only
subscription {
subscribe(source: "my.source") {
id
detailType
detail
}
}
## Will match events with detail-type = "my.detail" AND source = "my.source"
subscription {
subscribe(detailType: "my.detail", source: "my.source") {
id
detailType
detail
}
}
Isn't that great? I can now listen to exactly the events I am interested in 🔥
Unfortunately, this technique has some limitations. It cannot filter events based on the content of the detail field. This is because the data comes stringified.
Also, filters only work when the values exactly match. You cannot use advanced filters such as prefix, anything-but, etc. These are filters supported by eventBridge, not by AppSync.
Note that any advanced filter can still be achieved through filters at the EventBridge rule level, of course!
In this post, I showed you how we can observe EventBridge events through AppSync subscriptions and how we can even filter them at query time. Although its usage is somewhat limited, it can probably still be very helpful when you only need to filter on the detailType or source values, for example. You can easily use it to debug/test your application.
Find the full code of this implementation on Github
A big thanks to David Boyne for the inspiration!