Adding WCET and Timing Checks to Your CI Pipeline with RocqStat + VectorCAST

Stop timing regressions before they reach production: adding WCET and timing checks to CI with RocqStat + VectorCAST

Hook: If your embedded builds occasionally miss deadlines in the field, or timing regressions show up after weeks of integration work, your CI pipeline needs timing checks — not just unit tests. With Vector's 2026 acquisition of RocqStat, teams can now automate Worst-Case Execution Time (WCET) estimation and timing verification inside CI/CD to catch regressions early and keep safety goals on track.

Why this matters now (2026 context)

Timing safety is no longer a niche concern. In 2025–2026 we're seeing stronger regulatory pressure (ISO 26262, DO-178C guidance for real-time timing assurance), broader adoption of software-defined vehicle architectures, and an industry-wide push to shift safety verification left. Vector Informatik's acquisition of StatInf's RocqStat (announced January 2026) signals consolidation: timing analysis and testing are converging inside toolchains such as VectorCAST. That makes it practical to integrate WCET checks directly into CI/CD pipelines rather than doing expensive, late-stage timing audits.

What this recipe delivers

• Automated WCET and timing regression checks as CI gates
• Repeatable measurements and hybrid static + measurement-based analysis
• Practical config snippets for popular CI systems (GitHub Actions / Jenkins)
• Fail/pass criteria, baseline management, and artifact strategies

High-level workflow (inverted pyramid first)

At a glance, your CI pipeline should:

1. Build a reproducible image of the firmware (ELF, binary) (same compiler flags, map files) and produce a testable artifact (ELF, binary)
2. Run VectorCAST unit/integration tests (on host simulator or HIL) and export execution traces
3. Run RocqStat timing analysis (static/hybrid) using the traces and map/symbol information
4. Compare WCET estimates against stored baselines and configured thresholds
5. Fail the pipeline on regression beyond thresholds; publish reports/artifacts for debug

Prerequisites and recommended environment (practical checklist)

• Tooling: VectorCAST with the RocqStat integration or compatible CLI; RocqStat license; cross-compiler toolchain.
• CI runner: Containerized agents (Docker) or dedicated HIL agents for hardware tests.
• Deterministic build: Fixed compiler version, reproducible flags, and toolchain container images.
• Trace capture: VectorCAST tests must export execution traces consumable by RocqStat (configure trace export in VectorCAST project).
• Baseline store: An artifact repository (S3, Artifactory, Git LFS) or CI artifact storage to persist baseline WCETs.

Step-by-step recipe

1) Make builds deterministic

Timing analysis depends on reproducible binaries and symbol information. Lock these down:

• Pin compiler version and toolchain (use toolchain container images).
• Set stable compiler flags: disable debug-only optimizations that change layout between builds. Use -fno-omit-frame-pointer only if necessary for tracing consistency.
• Always generate a linker / map file and keep it as a CI artifact: the map file maps code addresses to function names and is required for WCET attribution.

2) Run VectorCAST tests and export traces

VectorCAST orchestrates unit and integration tests and can capture execution traces on simulator or target. For CI you can use either a software simulator (QEMU, vendor simulator) or a hardware test runner. The essential data to extract is an execution trace and an associated map/ELF file.

Example (pseudo-CLI) VectorCAST run inside CI:

# Example commands - adapt to your VectorCAST CLI
vectorcast_setup --project my_ecu_project --compiler /opt/gcc-arm-none-eabi
vectorcast_build --project my_ecu_project --config Release --map-output build/my_ecu.map
vectorcast_run --project my_ecu_project --runner simulator --trace-output traces/run1.trace

Note: the exact CLI flags will depend on your VectorCAST version. The goal is: build -> run tests -> produce a trace file plus the map/ELF.

3) Run RocqStat timing analysis (static or hybrid)

RocqStat supports static WCET estimation and hybrid approaches that combine measurements with static analysis to improve tightness. The integrated flow typically needs:

• Compiled binary/ELF
• Linker map and symbol information
• Execution traces (for measurement or hybrid analysis)
• Target platform description (CPU model, cache, pipeline parameters) — either built-in or a config file

Example (illustrative) RocqStat CLI:

# Pseudo CLI - adapt to your RocqStat / VectorCAST integration
rocqstat analyze \
  --binary build/my_ecu.elf \
  --map build/my_ecu.map \
  --traces traces/run1.trace \
  --platform configs/cortex-m7.json \
  --output artifacts/rocq_report.json

Output: a machine-readable WCET report (JSON/XML) with per-function and per-test-case WCET estimates and confidence metadata. Store that artifact for baseline comparison and audit trails.

4) Baseline and regression detection

Design baseline strategy before gating pipelines. A good baseline includes:

• Per-test-case WCET estimates and timestamps
• Measurement variance, number of runs, and toolchain metadata
• Threshold rules: absolute value (e.g., WCET <= 2.5ms) and relative regression limit (e.g., < 5% increase)

Store baseline JSON in your artifact repo. CI step compares new RocqStat report to baseline and decides pass/fail.

Example baseline JSON snippet:

{
  "tests": {
    "brake_ctrl_cycle": { "wcet_ms": 2.3, "variance": 0.05 },
    "sensor_fusion": { "wcet_ms": 4.1, "variance": 0.08 }
  },
  "toolchain": { "gcc": "11.3.0", "linker": "ld-2.35" }
}

5) CI gating script (example Python)

Below is a practical script to compare rocq_report.json to baseline.json and fail when thresholds are exceeded. Adapt thresholds to your project rules.

#!/usr/bin/env python3
import json
import sys

BASELINE = 'artifacts/baseline.json'
REPORT = 'artifacts/rocq_report.json'

# policy: fail if >10% regression or absolute exceedance
MAX_REL_DELTA = 0.10

with open(BASELINE) as f:
    base = json.load(f)
with open(REPORT) as f:
    report = json.load(f)

failures = []
for test, b in base['tests'].items():
    new = report['tests'].get(test)
    if not new:
        failures.append(f"Missing test in report: {test}")
        continue
    b_wcet = b['wcet_ms']
    n_wcet = new['wcet_ms']
    rel = (n_wcet - b_wcet) / b_wcet
    if n_wcet > b_wcet and rel > MAX_REL_DELTA:
        failures.append(f"{test} WCET regressed: baseline {b_wcet} ms -> {n_wcet} ms ({rel*100:.1f}% )")

if failures:
    print("Timing regression detected:\n" + "\n".join(failures))
    sys.exit(1)
print("Timing checks passed")

Hook this script as a CI step. On non-zero exit, the pipeline fails and artifacts are preserved for triage.

6) CI pipeline examples

GitHub Actions (simplified)

name: timing-ci

on: [push, pull_request]

jobs:
  wcet-check:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - name: Setup toolchain container
        run: docker run --rm -v ${{ github.workspace }}:/work myorg/vector-tools:2026 /bin/bash -c "./ci/run_vectorcast_and_rocq.sh"
      - name: Compare WCET
        run: ./ci/compare_timing.py

Jenkins Pipeline (simplified)

pipeline {
  agent { label 'linux-embedded' }
  stages {
    stage('Build') { steps { sh './ci/build.sh' } }
    stage('Run Tests & Trace') { steps { sh './ci/run_vectorcast.sh' } }
    stage('RocqStat') { steps { sh './ci/run_rocqstat.sh' } }
    stage('Compare') { steps { sh './ci/compare_timing.py' } }
  }
}

Practical hardening tips to reduce noise and false positives

• CPU isolation: Reserve a CI runner core and disable hyper-threading / frequency scaling during test runs.
• Power & thermal considerations: Run timing tests on stable-power test benches or HIL racks to avoid thermal throttling differences.
• Repeat runs: Execute multiple runs and use trimmed means or worst-N percentiles for measurement-based inputs to RocqStat.
• Deterministic I/O: Mock non-deterministic peripherals or use recorded traces to avoid variability from sensors or networks.
• Compiler flags: Avoid flags causing non-deterministic code layout between minor build changes. Use -fdata-sections and -ffunction-sections with deterministic linkers to produce consistent layouts.
• Map verification: Validate that symbol addresses in the map file align to the binary used for trace collection.

When to use static vs measurement vs hybrid WCET

• Static WCET: Best for safety certification and when hardware characteristics are well known; provides conservative upper bounds but may be pessimistic.
• Measurement-based: Useful when you can exercise realistic worst-case paths on target; cheaper but requires exhaustive input scenarios and careful setup.
• Hybrid (recommended in CI): Combine traces from tests with static analysis to get tighter, certified bounds — RocqStat specializes in hybrid approaches and will be directly accessible via the VectorCAST integration in 2026.

Artifact management and traceability

Keep the following as CI artifacts for auditability and post-failure triage:

• Binary/ELF and map file
• Execution traces used by RocqStat
• RocqStat output (JSON/XML and human-readable PDF)
• Baseline JSON used for comparison
• CI logs and toolchain version metadata

Scaling to many targets and bringing hardware into CI

As teams scale, orchestration becomes important. Two common patterns:

• Sim-first: Run timing checks in simulator for fast feedback, then schedule nightly hardware tests for critical targets.
• Hardware-as-a-Service: Use a hardware farm or cloud-accessible HIL labs and tag CI jobs to run on physical boards when a PR reaches a maturity gate.

VectorCAST + RocqStat integration (post-2026) will simplify trace capture across simulators and hardware, and will provide standardized artifacts to help scale multi-target pipelines.

Real-world example: ECU control loop

Scenario: you have an ECU control loop with a 5 ms deadline. Baseline RocqStat WCET for the main control handler is 3.8 ms. Team rule: WCET must stay < 4.5 ms and regression must be < 10%.

1. Every PR triggers a build in CI and VectorCAST unit tests on the simulator.
2. Traces are exported and RocqStat runs in hybrid mode.
3. Compare script flags any test where new WCET > 4.5 ms or > 4.18 ms (3.8 * 1.10).
4. On failure, the PR is blocked and the ROCQ report plus trace artifacts are uploaded to an artifact server for developer triage.

Outcome: a compiler change that increases inlining in a lower-level module triggered a 12% WCET regression on the control handler. CI failed early, developers adjusted optimization flags and added a tighter unit test to prevent future regressions.

Governance: configuring thresholds and approval flows

Define who can override timing gates and under what evidence. Common governance patterns:

• Non-blocking warnings for small regressions < 5% with a 48-hour mitigation requirement
• Blocking failures for critical paths or absolute thresholds
• Policy-driven exemptions for experimental branches with mandatory review

2026 trends: what to expect next

• Tighter toolchain integration: Vector's acquisition of RocqStat (Jan 2026) accelerates unified workflows—expect first-class APIs and out-of-the-box CI integrations in 2026–2027. Many teams will standardize on devcontainer-like toolchains for reproducibility.
• Continuous timing observability: Vendors will offer timing observability and timing dashboards in CI similar to performance dashboards for web apps, with historical baselines and anomaly alerts.
• Cloud-HIL orchestration: The industry is moving toward cloud-accessible HIL labs and scheduler integration for running hardware timing checks on-demand from CI.
• AI-assisted triage: Machine learning will recommend likely root causes for regressions (e.g., inlining, cache changes) based on diffs and tool outputs.

Common pitfalls and how to avoid them

• Pitfall: Relying only on a single run. Fix: use multiple runs and statistical summaries.
• Pitfall: Not storing map files with artifacts. Fix: archive map/ELF per-run for traceability.
• Pitfall: Letting CI runners vary in CPU scaling. Fix: use dedicated runners or disable frequency scaling during test runs.
• Pitfall: No governance—overrides become a habit. Fix: require documented justification and temporal limits for overrides.

Actionable takeaways (quick checklist)

• Instrument your VectorCAST test projects to export execution traces and map files.
• Run RocqStat (or the VectorCAST-integrated timing module) in CI to produce machine-readable WCET reports.
• Persist baselines and use scripted comparisons to gate PRs on timing regressions.
• Use hybrid analysis (measurement + static) for tighter, defensible bounds.
• Automate artifact retention so triage teams have trace + binary + map for every failed CI run.

"Timing safety is becoming a critical ..." — Vector statement on the RocqStat acquisition (Jan 2026).

Final notes: certification and audit

For safety certification, maintain full traceability between requirements, tests, binary artifacts, and WCET reports. Automated CI timing checks do not replace formal timing analysis artifacts required by cert bodies, but they dramatically reduce the risk of regressions and speed up verification cycles — making certification efforts more predictable and less costly.

Call to action

Start small: add a single critical task’s timing check to your CI this sprint. Export its VectorCAST trace, run RocqStat, automate a compare step, and gate PRs. If you want a ready-to-run template for GitHub Actions or Jenkins with example VectorCAST and RocqStat scripts tailored to your tool versions and target CPU, contact our team at newservice.cloud. We’ll help you create a reproducible CI timing pipeline and a baseline strategy aligned with your safety goals.

Related Reading

• Localhost Tool Showdown: Devcontainers vs Nix vs Distrobox (2026)
• Storage Workflows & Artifact Management (Creators, 2026)
• MLOps in 2026: Feature Stores, Responsible Models, and Cost Controls
• Security Audit: Firmware Supply‑Chain Risks for Power Accessories (2026)
• Weighted Bats, Overload/Underload and Overspeed: What Actually Helps Bat Speed?
• Stream It Live: Planning a Twitch‑Ready Cocktail or Cooking Broadcast
• Ant & Dec’s 'Hanging Out': Is Celebrity Podcasting Still a Growth Play or Saturation Signal?
• Retailing New Beauty Launches in Spas: Inventory, Training and Promotion Checklist
• Artisan Leatherwork for the Faithful: Ethical Tanning, Craft Stories and Gift Ideas