AI in your SOC without breaking the bank: testing Jev on blue-team workflows

Introduction

Security tools make a lot of decisions before an analyst sees an alert. TypeSafe’s new model, Jev, gives us another way to handle some of them. You give it data and questions, and it returns structured answers with probabilities rather than paragraphs of text. With its promise of speed and low cost, we wanted to see how it would handle security telemetry.

We tried two blue-team use cases using logs from one of our Dynamic Threat Ranges: spotting suspicious activity and grouping related evidence into investigation threads. Jev picked out useful leads, including evidence a baseline set of rules didn’t match. Grouping proved trickier: related events sometimes stayed unlinked, while separate activities ended up together. Here’s what worked, what didn’t, and where we think Jev could fit alongside existing detections.

What is Jev, and why try it here?

TypeSafe released Jev in September 2026 as its first “System One” model, built around fast, structured decisions rather than open-ended text generation. You give it shared context and questions, then choose the answer type: noul for yes/no probabilities, choice for selecting between options, or score for rating against defined criteria. TypeSafe’s training approach, Reinforcement Learning for Calibrated Decisions, aims to make those probabilities reflect how often the model is right. For blue teams, that could mean flagging suspicious activity or prioritizing events for review.

The example below asks three questions about the same event in one request.

Example request

{
  "model": "jev-latest",
  "state": {
    "process": "powershell.exe",
    "parent_process": "WINWORD.EXE",
    "arguments": "-EncodedCommand ..."
  },
  "questions": {
    "needs_review": {
      "type": "noul",
      "instructions": "Does this activity warrant investigation?"
    },
    "activity": {
      "type": "choice",
      "instructions": "Which behavior is visible in this event?",
      "criteria": {
        "script_execution": "Running a script or interpreter.",
        "credential_access": "Accessing authentication secrets.",
        "other": "Neither behavior is supported."
      }
    },
    "priority": {
      "type": "score",
      "instructions": "How urgently should this activity be reviewed?",
      "criteria": [
        "Routine review",
        "Investigate soon",
        "Urgent investigation"
      ]
    }
  }
}

‍

Example response

{
  "answers": {
    "needs_review": {
      "type": "noul",
      "noul": 0.91
    },
    "activity": {
      "type": "choice",
      "choice": "script_execution",
      "probabilities": {
        "script_execution": 0.95,
        "credential_access": 0.02,
        "other": 0.03
      }
    },
    "priority": {
      "type": "score",
      "score": 1.2,
      "legend": {
        "0": "Routine review",
        "1": "Investigate soon",
        "2": "Urgent investigation"
      },
      "probabilities": {
        "0": 0.05,
        "1": 0.70,
        "2": 0.25
      }
    }
  }
}

‍

In this example, Jev gives a 91% probability that the activity warrants investigation. It picks script_execution as the best-fitting category, with a 95% probability.

For priority, it assigns 5% to “Routine review”, 70% to “Investigate soon”, and 25% to “Urgent investigation”. The score of 1.2 is the probability-weighted average of the level numbers we provided: zero, one, and two. Different distributions can produce the same score, so we read probabilities to understand the assessment and use score when we want a single number for sorting or applying a threshold. Each answer comes back under the question name we supplied, making the values straightforward to use in code.

Testing environment and scope

We used one of our Dynamic Threat Ranges as our testing ground, with an attack scenario based on the Mustang Panda threat actor. We exported 4,134 unique Windows events covering process execution, file activity, registry changes, authentication, and network connections. This gave us realistic telemetry and a known attack sequence to check Jev’s findings against.

We evaluated flagging and grouping as separate tasks. For flagging, we gave Jev the exported logs and checked its findings against the attack script and surrounding telemetry. We also compared its coverage with our existing detection rules. For grouping, we supplied a curated set of attack-related events and checked whether it connected related records while keeping distinct investigation threads apart.

Use case 1: Flagging suspicious activity

We grouped events by host in 10-second windows and included related earlier activity as context. Jev answered two questions about each event: whether it warranted investigation, and which type of activity it showed. We flagged events with a review probability of 0.7 or higher.

Jev flagged 220 events, or 5.3% of the capture. Reviewing those flags against the attack script and telemetry identified 66 records containing direct attack evidence and 133 supporting records linked to attack activity. The remaining 21 appeared routine or unrelated. That means around 90% of the events Jev flagged could be linked to the attack.

We also checked coverage across 94 attack-related events. Jev flagged evidence from all eight target-side attack steps, though some were covered more thoroughly than others. It picked up all 12 log-clearing records and 10 of 13 discovery-and-collection records, while initial access and credential access were patchier, at five of 17 and one of four.

We tested the same 94 events against 18 generalized Sigma rules. The rules matched 34 events, while Jev flagged 57, with 21 picked up by both. Together, they covered 70 of the 94 events.

Jev flagged 36 valid events that didn’t match the rules, but didn’t flag 13 that the rules picked up. That suggests a useful role alongside existing detections, giving analysts another way to find activity worth investigating.

The flagging calls recorded around 12.7 million input tokens. At TypeSafe’s published input rate of $0.042 per million tokens, that would cost roughly $0.54, with no charge for output tokens. Jev’s standard input rate is about a fifth of GPT-5.6 Luna’s published rate. In our run, classification calls had a median response time of about half a second, with 95% completing within roughly two seconds. Each request handled between one and seven events, with two questions per event.

Use case 2: Grouping related evidence

For grouping, we supplied 96 attack-related events and asked Jev to organize them into investigation threads. A thread might cover a discovery burst, a persistence change, or credential-access activity. We checked its decisions against 38 pairs of events that should belong together and eight pairs that should remain separate.

We compared two ways of using Jev’s answers. The first required a probability of at least 0.6 before accepting a case assignment (lower-confidence choices were ignored). The second accepted the highest-probability choice without a minimum score (we went with its best guess).

With the 0.6 cutoff, the workflow joined six of the 38 related pairs and left 32 unresolved. It also merged two of the eight pairs that should have stayed separate, leaving the others unresolved. Without a minimum score, it joined 36 of the 38 related pairs, but merged five of the eight pairs that should have stayed apart.

The two runs show different problems. Requiring higher confidence left much of the evidence ungrouped, so legitimate connections slipped through the cracks. Accepting the top proposal recovered more connections, but also combined distinct activities. Finding links wasn’t enough on its own, because the workflow also needed to respect the boundaries between investigation threads.

Earlier decisions mattered as well; once a case existed, it became an option for later events to join, so the case history changed the choices Jev received.

That makes grouping more demanding than recognizing suspicious behavior in an individual event. It needs context about relationships between records, while unrelated detail can make those relationships harder to judge. TypeSafe’s documented limitations call out both indirection and distracting context as weak spots. Those are relevant considerations when deciding how much of this work to automate.

What this means for blue teams

Jev did better at flagging events than we expected, especially given its speed and low cost. It surfaced attack-linked evidence that our generalized rules didn’t match, while the rules picked up events Jev missed. That makes it worth exploring as an additional layer alongside existing detections.

Grouping was less consistent in our setup. The prompts, context selection, and confidence policy all shape that workflow, and further tuning could improve the results. For this prototype, we’d keep a human in the loop to check the proposed connections and decide how the evidence fits together.

A chat-style LLM could support that review by turning Jev’s flags and their context into a short, evidence-linked summary. That would be another workflow to test: Jev handling the structured decisions, a generative model helping present the evidence, and an analyst checking the interpretation against the original events.

If you’re exploring similar workflows, start with a controlled dataset and a decision you can check against known activity. Jev’s low inference cost makes it practical to try different approaches and see whether the results are useful for your team.