From Noise to Trackable Work: Routing Slack Alerts into Jira Automatically

1 day ago
9 min read

Hi, I'm CY Lee (이창윤), DevOps/SRE engineer on the dev team.

I stopped by once before to talk about the ClawHub plugin — and now I'm back, this time to share a problem we ran into with service alerting and how we worked through it.

How many notifications does your team receive in a day?

User feedback, payment events, signup events, 500 errors from SigNoz. Each of these was arriving in a separate Slack channel. Keeping channels separated by type made it easy to find a specific category of notification, but it made it hard to see the full picture of what was happening across the service at any given moment. So we consolidated everything into a single Slack channel. Seeing customer events and system alerts side by side in one stream made it much easier to follow what was going on. But once they were all together, a different question surfaced.

"Are the urgent alerts actually getting handled? And by whom?" Aggregating notifications in one place was not enough to answer that. We needed to know who owned a given alert, and how far along the response was. That kind of tracking belongs in a task management tool like Jira, not a chat channel. Bringing alerts together was one problem. Knowing whether they were resolved was a separate one entirely. That realization led us to split Slack notifications into two categories: feeds and alerts. Feeds flow through Slack as reference. Alerts get converted into Jira issues automatically via Activepieces. This post documents how we built that system.

1. The Problem With Consolidating Alerts Into One Channel

💁 We started with one channel per event type

Originally, each type of notification had its own Slack channel.

Slack channel	Notification type
#general-alerts	User Feedback
#aircloud-alerts	Signups, payments
#alert-prod-critical	System monitoring (urgent)
#alert-prod-warning	System monitoring

Separate channels made it straightforward to filter by type. But they also meant you had to switch between channels to see whether, for example, a spike in payment failures was happening at the same time as a signup error wave. The full picture required hopping.

🤔 After consolidation, we lost visibility into response status

Merging everything into one channel fixed the context problem. But it introduced a new one: actionable alerts and informational notifications now lived in the same stream, and it became hard to tell:

Who had acknowledged a particular alert
Which notifications needed a Jira issue
What the status of each alert was
Whether an urgent message had been buried under reference noise

Consolidating notifications and tracking their resolution turned out to be two different problems.

😞 Emoji reactions helped a little, but not enough

One option was using emoji reactions to signal status within Slack. For example: 👀 means "I'm looking at it," ✅ means "resolved." That works fine for lightweight acknowledgment. It does not hold up for actual work tracking.

Emoji can say "seen." They cannot say "here is where we are in handling this." That distinction pushed us toward separating the notification space from the task tracking space. Slack handles the former; Jira handles the latter.

2. Defining the Line Between Feed and Alert

1️⃣ Not every notification needs a response

If we converted every Slack message into a Jira issue, we would simply recreate the noise problem inside Jira. Signup confirmations and status log messages do not need a ticket.

The first thing to settle before building any automation was the classification criteria, not the tooling.

2️⃣ Feed is reference; Alert is actionable

We divided all Slack notifications into two buckets.

Category	Meaning	Jira issue?	Examples
feed	Reference only	No	Successful, signup, status change, information log
alert	Requires response	Yes	Payment failure, signup error, user complaint, 500 error from SigNoz

Feeds can scroll by without any follow-up. Alerts cannot. Importantly, this split was already being made upstream: the services sending the webhooks were already routing alerts to a dedicated alert channel. That meant our automation only needed to watch the alert channel.

3️⃣ Alerts need owner assignment and status tracking

Once we classified an alert as actionable, it needed more than an emoji reaction. It needed an owner, a status, and a record of how it was resolved. That is exactly the territory emoji could not cover, as we had already found in Section 1. Slack is well-suited for quickly reviewing notifications. Jira is better suited for tracking ownership and response status. So we decided to convert alerts into Jira issues and treat them as trackable work items from that point on.

4️⃣ The Jira routing criteria we settled on

Notification	Route to Jira?	Examples
User complaint (usability)	Yes	"Payment is not working," "Can't log in"
Payment failure	Yes	Card authorization failure, PG error
Signup error	Yes	Auth failure, registration error
SigNoz 500 error	Yes	API 500, timeout, exception
Simple signup confirmation	No	"User signed up" event
Informational log	No	Routine status notification

Writing this out in a table had a practical benefit: later, when configuring the automation, we never had to re-debate "should this become an issue?" The answer was already in the table.

3. Building the Activepieces Flow

With the criteria defined, we used Activepieces to automate the Jira conversion.

We chose Activepieces because its Slack and Jira connectors made it fast to set up, and unlike Zapier, it does not charge per task execution. It bills based on the number of active flows, which suited our situation since alert volume was unpredictable.

The basic structure

The simplest version of the flow is: new Slack message in the alert channel triggers a Jira issue creation. Connecting Slack and Jira is straightforward. The limitation is that raw Slack messages land in Jira exactly as they arrive: bot tags, emoji, line breaks and all.

A Code step to extract the message URL and date

We added a Code step between the Slack trigger and the Jira action to clean up the data before it hit the issue. The step generates a Slack permalink, formats the creation date for the Asia/Seoul timezone, and builds an ADF (Atlassian Document Format) description that includes the original Slack message text along with a clickable "Open message" link.


export const code = async (inputs) => {
  const ts = String(inputs.ts ?? '');
  const channel = String(inputs.channel ?? '');
  const raw = inputs.text ?? '';

  // Slack permalink (used for both the dedup key and the "Open message" link)
  const link = `https://<workspace>.slack.com/archives/${channel}/p${ts.replace('.', '')}`;

  // start date in Asia/Seoul (YYYY-MM-DD)
  const start_date = new Date(parseFloat(ts) * 1000)
    .toLocaleDateString('en-CA', { timeZone: 'Asia/Seoul' });

  // Build ADF: newlines -> hardBreak, text properly escaped by JSON serialization
  const nodes = [{ type: 'text', text: 'Original Slack message: ' }];
  raw.split('\n').forEach((line, i) => {
    if (i > 0) nodes.push({ type: 'hardBreak' });
    if (line.length) nodes.push({ type: 'text', text: line });
  });
  nodes.push({ type: 'hardBreak' });
  nodes.push({
    type: 'text',
    text: 'Open message',
    marks: [{ type: 'link', attrs: { href: link } }],
  });

  const description = {
    type: 'doc',
    version: 1,
    content: [{ type: 'paragraph', content: nodes }],
  };

  return { link, start_date, description };
};

The returned link , start_date and description map directly into the next Jira step. Including the permalink in the issue means whoever picks it up can jump back to the original Slack message without searching.

4. Using an LLM to Generate a Clean Jira Title and Description

Raw Slack messages make bad Jira titles

The Code step handled the message body cleanly. The remaining problem was the issue title. Slack messages often contain bot mentions, emoji, and line breaks, so pasting them directly into a Summary field produces something long and messy. Having a human clean up each title manually would defeat the purpose of automating anything.

The solution: use an LLM to rewrite the raw Slack text into a concise title and a structured description. The model's role here is not to classify or decide anything. It is purely to reformat content that already has a clear meaning.


Slack message
→ LLM (OpenRouter Chat)
→ title / description (JSON)
→ Jira Issue creation

Calling OpenRouter

We used OpenRouter's chat completions endpoint with a JSON output constraint. The Slack message text from the previous step goes into the user message; the model returns a JSON object with exactly two keys.

URL: POST <https://openrouter.ai/api/v1/chat/completions>


{
  "model": "qwen/qwen3.6-flash",
  "messages": [
    {
      "role": "system",
      "content": "You create a Jira issue from a Slack message. Respond with a single JSON object with exactly two keys: \"title\" (short summary) and \"description\" (details with context). Output JSON only, no extra text."
    },
    {
      "role": "user",
      "content": "Slack message:\n\n{{trigger['output']['text']}}"
    }
  ],
  "reasoning": { "enabled": false },
  "response_format": { "type": "json_object" }
}

One implementation note: this model does not support a strict json_schema constraint, so we used json_object mode instead. That mode requires the word "json" to appear somewhere in the prompt, which is why the system message explicitly says "Output JSON only."

Parsing the JSON response

The model returns its output as a JSON string inside choices[0].message.content . We added a second Code step to parse it, including a guard for cases where the model wraps the response in a ````json` block or returns a parsed object directly.


export const code = async (inputs) => {
  const value = inputs.ai_response;

  // 이미 객체로 들어오면 그대로 사용
  let parsed = value;

  if (typeof value === 'string') {
    let raw = value.trim();

    // 모델이 ```json ... ``` 으로 감싼 경우 제거
    if (raw.startsWith('```')) {
      raw = raw.replace(/^```(?:json)?\s*/i, '').replace(/\s*```$/, '').trim();
    }

    parsed = JSON.parse(raw);
  }

  return {
    title: parsed.title,
    description: parsed.description,
  };
};

The title maps to the Jira Summary field; description maps to the Description field. Combined with the original message text and permalink from Section 3, each Jira issue ends up with a clean, human-readable summary at the top and the full original context preserved below.

5. What Changed After Automation

(Before) alerts and reference noise in the same stream

Before the automation, both actionable and informational notifications arrived in the same Slack channel. Payment failures and 500 errors sat next to signup confirmations and status logs. As volume increased, the actionable items became harder to spot. Someone had to read every message and decide whether it warranted a response. Jira issues, when created, were done by hand.

(After) alerts flow directly into trackable Jira issues

After the automation, feeds and alerts are separated at the source. The Activepieces flow watches only the alert channel. When an alert arrives, the first Code step extracts the permalink and formats the date. The LLM step produces a clean title and description. A Jira issue is created automatically. From there, an engineer assigns ownership and manages status inside Jira.

Roles clarified between Slack and Jira

Dimension	Before	After
Notification stream	Feeds and alerts mixed together	✅ Feeds and alerts in separate channels
Deciding what needs action	Manual review per message	✅ Alert channel = actionable by definition
Jira issue creation	Manual	✅ Automated via Activepieces
Title and description	Written by hand	✅ Generated by LLM; original message preserved
Owner and status tracking	Emoji reactions (limited)	✅ Tracked in Jira

Slack became the place for fast notification review. Jira became the place for ownership and resolution tracking. Activepieces bridges the two.

6. What We Learned

Collecting alerts and resolving them are different problems

Centralizing alerts creates the feeling that you are on top of everything. It does not mean alerts are actually getting resolved. As volume grows, the absence of a routing and tracking layer makes the situation harder to manage, not easier. We tried to solve both problems with a single tool, and that approach produced friction. Separating the two concerns into dedicated tools clarified everything.

Define the criteria before building the automation

The most time-consuming part of this project was not configuring Activepieces. It was agreeing on where to draw the line between feeds and alerts. "What counts as an actionable alert?" is a question different team members answer differently, and the answers do not converge until someone writes them down. If we had skipped that step and gone straight to automation, we would have ended up with a Jira board full of noise, which is the same problem we started with, just in a different tool.

We scoped the LLM to formatting only

For this flow, the LLM's job is narrow: take a Slack message and produce a readable title and description. We considered having it classify alerts or suggest owners, but we left those out. The channels sending alerts had already done the classification upstream (critical vs. warning). There was no need to re-derive what was already known.

Extending automation to cover judgment calls is something we can expand gradually, after enough validation. For now, we started with formatting — where a wrong result carries low cost — and left classification and owner assignment as the next problem to tackle.

Automation should hand off cleanly to a human

The goal here was never to automate the entire response lifecycle. It was to ensure that actionable alerts were not missed, and that the person picking up a Jira issue had everything they needed to start immediately. That is why each issue includes the original Slack message alongside the LLM-generated summary. If someone had to go back to Slack and search for context, the automation would not have saved much time.

The boundary question is not "how much can we automate?" but "where does the automation end and the human begin, and is that handoff smooth?"

Automation's job here was not to close the loop. It was to open the right door for the right door for the person who would.

7. Closing

Back to the original question: were urgent alerts actually getting handled? Aggregating them into a shared channel did not answer that. Knowing who owned an alert, and how far along the response was, required something Slack is not designed for.

So we split alerts at the source, routed them through Activepieces, preserved the original context via a Code step, and used an LLM to generate a readable Jira summary. Slack handles notification review. Jira handles ownership and resolution tracking.

This project was not simply automation that moves Slack messages to Jira. It was a process of transforming notifications that used to flow by untracked into trackable units of work. Alerts that previously could disappear into the stream now have an owner, a status, and a history.

What is not yet automated is assigning an owner within Jira. That remains an open problem, and we are leaving it for a human to handle from that point on. Going forward, rather than expanding scope indiscriminately, we plan to extend the automation one step at a time — as much as the evidence supports.

| Dev Team

| Author: CY Lee

| Site: Linkedin