How to Observe EventBridge Events with AppSync Subscriptions
Observe your event bridge events in real-time using AppSync
I recently came across David Boyne's blog post: How to Observe EventBridge Events with Postman and WebSockets. What a great idea! But, then I thought:
I can do the same with AppSync Subscriptions!
I had to try! Here is what I achieved:
Building the basic AppSync API
The idea was simple. I needed the following components:
An AppSync API
A
Mutation
that receives events from EventBridgeA
Subscription
that is attached to the aforementioned MutationAn EventBridge rule that sends events to the AppSync Mutation (target)
I also wanted to be able to filter events I was interested in. Here, I thought about two options:
Filter the events in the EventBridge rule.
Send all events to AppSync and use AppSync to filter them, thanks to subscription arguments
I went with the second approach. It would give me more flexibility to filter the events at query time instead of having to re-deploy each time I wanted a new filter.
Here is the GraphQL Schema I created:
type Mutation {
sendEvent(event: EventBridgeMessageInput!): EventBridgeMessage
}
type Subscription {
subscribe(
source: String
detailType: String
account: String
resources: [String!]
): EventBridgeMessage
@aws_subscribe(mutations: ["sendEvent"])
}
type EventBridgeMessage {
id: ID!
version: String!
detailType: String!
source: String!
account: String!
time: AWSDateTime!
region: String!
resources: [String!]
detail: AWSJSON!
}
input EventBridgeMessageInput {
id: ID!
version: String!
detailType: String!
source: String!
account: String!
time: AWSDateTime!
region: String!
resources: [String!]
detail: AWSJSON!
}
I also needed to setup the Mutation. I used a NONE
data source for that and a simple mapping template that just returns the received payload.
All done! Now, by executing the sendEvent
Mutation, it gets delivered to the subscription! 🙌
All that was left to do was to configure EventBrige and set the Mutation as a target.
First attempt: API Destinations
My first attempt was to use API Destinations. I followed this awesome tutorial and defined my Input Path and Input Transformer rules which looked like this:
InputPathsMap:
version: $.version
id: $.id
detailType: $.detail-type
source: $.source
account: $.account
time: $.time
region: $.region
resources: $.resources
detail: $.detail
InputTemplate: |
{
"query": "mutation SendEvent($event: EventInput!) { sendEvent(event: $event) { version id detailType source account time region resources detail } }",
"operationName": "SendEvent",
"variables": {
"event": {
"version": "<version>",
"id": "<id>",
"detailType": "<detailType>",
"source": "<source>",
"account": "<account>",
"time": "<time>",
"region": "<region>",
"resources": "<resources>",
"detail": <detail>
}
}
Unfortunately, that didn't work! 😞
The problem is that in EventBridge, the detail
attribute is an arbitrary JSON object which could have any shape. This is the reason I used an AWSJSON
type in my GraphQL schema (I wanted to receive any event). The problem is that AppSync expects the JSON to be stringified!
After some investigation, I could not find any way for EventBridge to stringify JSONs. So, that was a dead end.
AWS Lambda to the rescue!
If EventBridge cannot do it, Lambda surely can! So, I wrote a simple lambda that receives the event, reformats it and calls the AppSync endpoint. I then just configured the Lambda as an EventBridge target. (See the code here).
✍️ Note: I also added an IAM authentication method to the AppSync API that Lambda can use to call the Mutation (in addition to the API key used by the subscription).
All set! Now, running the following subscription:
subscription MySubscription {
subscribe {
resources
region
source
version
detailType
detail
}
}
And sending an event into Event Bridge
aws events put-events --entries '[{"DetailType": "my.detail.type", "Source": "my.source", "Detail": "{\"foo\": \"bar\"}"}]'
Response:
{
"data": {
"subscribe": {
"resources": [],
"region": "us-east-1",
"source": "my.source",
"version": "0",
"detailType": "my.detail.type",
"detail": "{\"foo\":\"bar\"}"
}
}
}
It works! 🎉
The power of AppSync subscriptions
One of the great features of AppSync subscriptions is that you can specify which changes you are interested in at query time. You can do that by adding arguments to the subscription endpoint. Whatever value you pass in the input, you will only receive changes that match the Mutation's response fields values.
So, I can now do queries such as
## Will match events with detail-type = "my.detail" only
subscription {
subscribe(detailType: "my.detail") {
id
detailType
detail
}
}
## Will match events with source = "my.source" only
subscription {
subscribe(source: "my.source") {
id
detailType
detail
}
}
## Will match events with detail-type = "my.detail" AND source = "my.source"
subscription {
subscribe(detailType: "my.detail", source: "my.source") {
id
detailType
detail
}
}
Isn't that great? I can now listen to exactly the events I am interested in 🔥
Limitations & gotchas
Unfortunately, this technique has some limitations. It cannot filter events based on the content of the detail
field. This is because the data comes stringified.
Also, filters only work when the values exactly match. You cannot use advanced filters such as prefix
, anything-but
, etc. These are filters supported by eventBridge, not by AppSync.
Note that any advanced filter can still be achieved through filters at the EventBridge rule level, of course!
Conclusion
In this post, I showed you how we can observe EventBridge events through AppSync subscriptions and how we can even filter them at query time. Although its usage is somewhat limited, it can probably still be very helpful when you only need to filter on the detailType
or source
values, for example. You can easily use it to debug/test your application.
Find the full code of this implementation on Github
A big thanks to David Boyne for the inspiration!