At its heart, event-driven integration means enhanced real-time customer engagement and better-informed, predictive decision-making. The Solace PubSub+ Connector for Boomi enables organizations to reap those benefits faster by creating a combined Boomi and Solace experience that’s greater than the sum of its parts.
Boomi developers get to stick with the UI they know and love, while getting added benefits through the event-driven Solace platform. For Boomi architects, the connector fits hand-in-glove with a time-tested methodology and comprehensive platform powering your enterprise’s event-driven evolution.
In this post, I'll introduce the concept of API-first development and walk through the first three steps of a 6-step methodology that will help you get the most out of your event-driven integration investment.
Architects using synchronous connections like REST and SOAP have long acknowledged the importance of APIs. There’s a reason that tooling like API Management is such a crucial part of the Boomi AtomSphere Platform. API Management enables modeling synchronous APIs, establishing service level agreements, regulating security, enforcing SLAs, registering data formats, and ensuring that API users are notified of changes.
Despite their importance, traditional development methodologies (and the realities of rushed modern development) typically leave API definitions until late in the development process: after coding is complete, the developer exports the data structure schema and uploads it to API management.
Lately, thought leaders have turned this development process on its head. Instead of a late–in–the–game export, these thought leaders urge enterprises to define APIs at the start of the development cycle, then implement them using coding tools such as Boomi Integration. This philosophy is called API-first. Here is a summary of Boomi's approach to API-first development:
Until recently, asynchronous implementations haven’t had the tooling or methodologies necessary for an API-first development process. That is no longer the case when pairing Boomi and Solace. Much like synchronous API management, architects and developers can now develop event-driven, asynchronous APIs and model them using graphical tools provided in Solace PubSub+ Event Portal. The newly released Solace PubSub+ Connector for Boomi can then seamlessly integrate your freshly created APIs into Boomi and use them as the basis for creating processes that follow best-practices for event-driven integration.
The enhanced tooling aligns with a standardized, repeatable methodology tailored for use in event-driven integration, both with organizations new to the concepts of event-driven architecture (EDA), and those well along the journey to a mature EDA implementation.
Before you begin the journey, it’s helpful to review and solidify why you are moving towards event-driven integration with Boomi and Solace. These goals can serve as key performance indicators in your new architecture and provide a helpful list of reminders of why you pursued this initiative in the first place.
Adding Solace to an existing Boomi infrastructure brings:
Adding Boomi to an existing Solace infrastructure brings the ability to:
While event-driven integration provides numerous benefits, enterprises need to exercise care in their evolution towards event-driven maturity. Making sure that your organization is on board, from executive stakeholders down to junior developers, requires thought leadership and persistence. The integration team needs to start thinking about choreography, rather than orchestration. The API teams need to start thinking about event-driven APIs, rather than just request/reply. Support personnel need to understand how events move through the enterprise. Leadership needs to grasp the benefits of real-time information flow across multiple clouds.
Fortunately, Boomi and Solace have collaborated on a wide range of materials to help you evangelize an organization. Here are a few recent posts:
It’s important to note that evangelism is not a one-time activity. Keeping an EDA evolution on track requires a continuing process of education and realignment to goals as the journey continues. As your company onboards more events and starts to see the fruits of your labor, the evangelism shifts from “why are we doing EDA?” over to “check out these events available within our enterprise, how can your group use them?”. An event portal plays a crucial role in that.
The remaining steps in the methodology ensure that your enterprise captures the benefits of event-based integration, while minimizing the uncertainly, resistance and pain associated with any large architectural shift.
While nothing is more exciting than the possibilities of event-driven integration, bringing the concepts of EDA into reality take a firm foundation. While there are multiple options for EDA, this post assumes that you’ve decided to use the Boomi and Solace product stack. It’s a potent combination. But getting the most of your foundation requires the strengths of each of the components.
Boomi Components:
Integration: Drag and drop UI, data mapping tools, and a comprehensive library of connectors. Coupling this with support for various integration patterns enables you to build any integrations with exceptional speed.
API Management: Supporting the full lifecycle of synchronous APIs in a hybrid environment. First, configure synchronous APIs and expose their functionality to both internal and external clients, then test and deploy synchronous APIs and enforce contracts and policies with an API gateway.
Flow: A low-code platform for apps that allows everyone to build workflow applications, without needing to be a software developer. Simply drag and drop your workflow as a flow diagram, and the Boomi Flow platform converts this into a fully functioning workflow application.
Master Data Hub: A cloud-native master data management (MDM) solution that sits at the center of the various data silos within your business and provides easy to implement, scalable, flexible, and secure master data management hub as a service.
Solace Components
Solace PubSub+ Event Broker: Modern messaging layer that can be deployed across every environment (appliance, on-premises software, cloud-based software, messaging-as-a-service) and component of the distributed enterprise.
Event Mesh: A collection of event brokers in different physical locations, woven together that dynamically routes events from one application or connected device to any other— no matter where they are deployed. Whether it’s in legacy on-premises systems or in the public cloud, private cloud, or somewhere in between, an event mesh takes the guesswork out of getting events where they need to be.
Event Portal: Event management toolset that enables you to create, design, share, and manage various aspects of your event-driven architecture. Event Portal brings to the async world the same functionality that lets enterprises create API-first architectures.
Cloud management: PubSub+ Event Broker: Cloud is a software as a service (SaaS) platform that gives you access to advanced features like the publish-subscribe messaging pattern, queuing, streaming, replay, and request-reply, while doing all the heavy lifting to dynamically deploy, upgrade, patch, and operate your event broker services. It is engineered to be secure, resilient, and cost-optimized so you can focus on building applications.
These components interact to create a complete ecosystem for event-driven interaction:
At the heart of the Boomi-Solace relationship is the PubSub+ Connector for Boomi. This connector lives inside the Boomi Integration GUI, both establishing runtime connectivity and letting you carry out common Solace tasks like discovering event definitions without ever leaving the friendly confines of Boomi. This integration creates a better experience for developers and keeps your EDA design vision on track. As part of the PubSub+ Connector for Boomi, there are multiple sample processes available in the Integration Process Library, which in turn link to extensive walk-through documentation. We’ll talk much more about the connector and process library later in Step 5 where we make our designs a reality.
While we are getting acquainted with the Solace and Boomi components, it might be helpful to review the terminology of event-driven integration. We’ll illustrate using a scenario where a fictional taxicab company hires a new driver through the SaaS provider Bamboo. In turn there are three downstream applications that are all interested in a new hire: Slack, Salesforce, and ServiceNow.
Note: there are many ways you can implement an event-driven architecture with Solace. However, the typical scenario and best practice is: publish to a topic, read from a queue. Some of us have tattoos that say exactly that. So, to simplify things, that’s what we’ll describe here. There’s much more in the documentation about other use cases.
Event: an event is a business moment where something changes somewhere in the enterprise. Here, the event is the taxi company hiring the driver. The key information about the hiring event is put into the payload (or content) of the event. For instance, the driver’s name, social security number, and address would be in the payload in a JSON or XML format.
Topic: a topic is a hierarchal string that briefly describes the content of the event to make routing more efficient. The topic is meta-information, separate from payload. The format of the string depends on the organization. The topic for this message is: /taxico/hr/bamboo/Driver/hired/v1/349034
Which can be broken down like this:
Subscription: downstream applications indicate what events they are interested in using subscriptions. These subscriptions follow the same format at the topic string, but the true power comes in being able to use wildcards. So if Slack wants to know about all drivers that are hired, it can subscribe to the topic string: /taxico/hr/bamboo/Driver/hired/v1/* where * is the wildcard for “all”
And if Salesforce needs to know about drivers that are both hired, fired and promoted, it can use the subscription: /taxico/hr/bamboo/Driver/*/v1/*
Queue: When the event broker finds a subscription that matches the topic string of the incoming event, it creates a copy and places it on the queue. The queue holds messages until the downstream application is ready to process them. Some remarkable things about queues: they guarantee order, they preserve messages if there is an issue with the application and they serve as a buffer if there is heavy volume for a period of time and the application can’t handle it.
There might be many matching subscriptions, in which case all associated queues get a copy of the message, which makes publishing to a topic and reading from a queue a terrific way to spread information from one source to multiple interested consumers.
No matter how great your foundation, biting off more than you can chew is sure way to sink a promising event-driven integration project. The key is a juicy pilot project that shows off event-driven integration without a big commitment. But with most enterprises featuring hundreds if not thousands of integrations, how do you pick a winner?
One excellent feature of Boomi is that you can view deployed processes at any stage of the lifecycle through the “Deployments” view of the UI. This can give you a sense of how your organization is using Boomi, and which processes would be good candidates for Solace. These are signs that your Boomi process would be a strong candidate for introducing Solace for event-driven integration:
If you’re using Solace PubSub+ Event Portal, then you have an excellent catalog of applications and events moving through your enterprise. Use this as a starting point to locate good candidates event flows. These are signs that Boomi could enhance your existing Solace event integration:
Applications featuring these characteristics (the more the better) should form a short list of potential candidates from which to choose. From there, winnow down further. The best event-driven integration pilot projects deliver a potent combination of:
A quick win that delivers an improved customer experience can breathe optimism into both technical and business teams, relieve anxious stakeholders and provide momentum for additional projects. If you can simultaneously make developers and support personnel’s lives better by alleviating obvious technical pain (brittleness, performance, resiliency) so much the better.
In my next post, I'll cover the final three steps to take to get the most out of your event-driven integration investment.
Boomi has partnered with Solace to extend the value of the industry leading Boomi Platform with enterprise grade event streaming and event management capabilities. Learn more here.